configuration.md

Configuration Reference

Configuration	Description
database.hostname	Source Database HostName
database.port	Source Database Port number
database.user	Source Database Username(user needs to have replication permission, Refer https://debezium.io/documentation/reference/stable/connectors/mysql.html) GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON . TO 'user' IDENTIFIED BY 'password';
database.password	Source Database Password
database.include.list	List of databases to be included in replication.
table.include.list	List of tables to be included in replication. MySQL(db_name.table_name), PostgreSQL(schema_name.table_name)
clickhouse.server.url	ClickHouse URL, For TLS(use https and set port to 8443)
clickhouse.server.user	ClickHouse username
clickhouse.server.password	ClickHouse password
clickhouse.server.port	ClickHouse port, For TLS(use the correct port 8443 or 443
clickhouse.server.database	ClickHouse destination database
snapshot.mode	"initial" -> Data that already exists in source database will be replicated. "schema_only" -> Replicate data that is added/modified after the connector is started.<br/> MySQL: https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-snapshot-mode \ PostgreSQL: https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-property-snapshot-mode MongoDB: initial, never. https://debezium.io/documentation/reference/stable/connectors/mongodb.html
connector.class	MySQL -> "io.debezium.connector.mysql.MySqlConnector" PostgreSQL -> Mongo ->
offset.storage.file.filename	Offset storage file(This stores the offsets of the source database) MySQL: mysql binlog file and position, gtid set. Make sure this file is durable and its not persisted in temp directories.
database.history.file.filename	Database History: Make sure this file is durable and its not persisted in temp directories.
schema.history.internal.file.filename	Schema History: Make sure this file is durable and its not persisted in temp directories.
disable.ddl	Optional, Default: false, if DDL execution needs to be disabled
enable.ddl.snapshot	Optional, Default: false, If set to true, the DDL that is passed as part of snapshot process will be executed. Default behavior is DROP/TRUNCATE as part of snapshot is disabled.
database.allowPublicKeyRetrieval	Optional, MySQL specific: true/false
auto.create.tables	When True, connector will create tables(transformed DDL from source)
persist.raw.bytes	Debezium.BYTES data(usually UUID) is persisted as raw bytes(CH String) if set to true.
database.connectionTimeZone	Example: "US/Samoa, Specify MySQL timezone for DATETIME conversions.https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-temporal-types
enable.snapshot.ddl	When true, pre-existing DDL statements from source(MySQL) will be executed. Warning: This might run DROP TABLE commands.
clickhouse.datetime.timezone	Override timezone for DateTime columns in ClickHouse server.
skip_replica_start	If set to true, replication is not started, the user is expected to start replication with the sink-connector-client program.
restart.event.loop	If set to true, replication will be re-started based on the restart.event.loop.timeout.period parameter(which is defined in seconds)
restart.event.loop.timeout.period.secs	If the last change record(CDC) received from source database exceeds this threshold period defined in seconds, then replication is restarted.
batch.max.records	Size of the batch that is persisted to ClickHouse.(Default 100000)
sink.connector.max.queue.size	Size of the Queue(in Memory) that holds the CDC records(Use a lower number for Out of Memory exceptions)
schema.history.internal.store.only.captured.tables.ddl	Works with table.include.list/table.exclude.list , set this to true to avoid debezium capturing schemas of all tables.
schema.history.internal.store.only.captured.databases.ddl	Works with table.include.list/table.exclude.list , set this to true to avoid debezium capturing schemas of all databases.