RFC: logging: experimental batching writer by pongad · Pull Request #2910 · googleapis/google-cloud-java (original) (raw)
Manually written batching implementation. Unlike GAX batching, it implements flushing and does not deal with partition keys.
If we're OK with where this is heading, we should make this work with batching settings (shouldn't be hard) and load-test before migrating things to it.
If not, this can serve as a starting point for a better GAX batching implementation.
In an experiment, I publish 1M messages of 300 bytes each. Using LoggingHandler gives me ~14,000 msg/s. BatchingWriter gives ~67,000, using similar BatchingSettings. Letting users configure these settings might be important though; I have observed ~300K msg/s given enough CPU and memory.
This human has signed the Contributor License Agreement.
label
chingor13 pushed a commit that referenced this pull request
chore: Update generation configuration at Sat Mar 15 02:25:50 UTC 2025
chore: Update generation configuration at Mon Mar 17 02:28:47 UTC 2025
suztomo pushed a commit that referenced this pull request
lqiu96 pushed a commit that referenced this pull request
chore: Update generation configuration at Sat Mar 15 02:25:50 UTC 2025
chore: Update generation configuration at Mon Mar 17 02:28:47 UTC 2025
chingor13 pushed a commit that referenced this pull request
meltsufin pushed a commit that referenced this pull request
Co-authored-by: Mattie Fu mattiefu@google.com
This was referenced
May 28, 2026
blakeli0 added a commit that referenced this pull request
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})