Breaking configuration changes for Kafka clients
KAFKA_CONFIG_FILEtakes a file path of config file for kafka client parameters (See provided example file / discussion in readme)- The following previously optional config for setting specific kafka parameters are no longer supported, and must be set using
KAFKA_CONFIGorKAFKA_CONFIG_FILE.
| Prior Config | Kafka Parameter to Set | Source/Sink | Default if unset |
|---|---|---|---|
| KAFKA_MAX_BUFFER_KB | queued.max.messages.kbytes | Both | 16384 (16MB) |
| KAFKA_CONSUME_LATEST_FIRST | auto.offset.reset | Source | earliest |
| KAFKA_COMPRESSION | compression.type | Sink | none |
Required configuration (KAFKA_BROKERS, KAFKA_TOPICS, KAFKA_CONSUMER_GROUP) should still be set via environment variable.