How to configure out_stackdriver batch size #9283
Unanswered
rafal-dudek
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
I was checking stackdriver output configuration: https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver
And didn't find an option to configure batch size.
Configuring that might be important for performance and quota limitations for high logs traffic:
https://cloud.google.com/logging/docs/reference/v2/rest/v2/entries/write
Here are the limits for entries.write: https://cloud.google.com/logging/quotas#api-limits
Size of an entries.write request | 10 MB
Number of entries.write requests | 120,000 per minute, per Google Cloud project
I've checked it briefly and I think the batch size depends on the chunk size in fluent-bit.
https://docs.fluentbit.io/manual/administration/buffering-and-storage#chunks
Is there any way we can influence this size? I only found an option to change the buffer size for all chunks, not the single chunk size.
Beta Was this translation helpful? Give feedback.
All reactions