Missing ServiceLoader configurations can cause plugins to take much longer to load.

[chimeyrock999]

Description:

We are encountering a performance issue with Kafka Connect plugin loading due to missing ServiceLoader manifests. The logs indicate that the following plugins are missing the necessary ServiceLoader manifests:

Plugin Path: /opt/kafka/plugins/iceberg-kafka-connect-runtime-hive-0.6.19/
Connector: io.tabular.iceberg.connect.IcebergSinkConnector (version 1.5.2-kc-0.6.19)
Transformations:
io.tabular.iceberg.connect.transforms.CopyValue (undefined version)
io.tabular.iceberg.connect.transforms.DebeziumTransform (undefined version)
io.tabular.iceberg.connect.transforms.DmsTransform (undefined version)
io.tabular.iceberg.connect.transforms.JsonToMapTransform (undefined version)
io.tabular.iceberg.connect.transforms.KafkaMetadataTransform (undefined version)
io.tabular.iceberg.connect.transforms.MongoDebeziumTransform (undefined version)

Impact:

The absence of ServiceLoader manifests can significantly impact plugin loading times. Kafka Connect may spend extra time trying to resolve or handle these plugins, which can result in slower startup times and potentially affect overall performance.

Logs:

2024-09-17 06:04:02,497 WARN One or more plugins are missing ServiceLoader manifests may not be usable with plugin.discovery=service_load: [
file:/opt/kafka/plugins/iceberg-kafka-connect-runtime-hive-0.6.19/	io.tabular.iceberg.connect.IcebergSinkConnector	sink	1.5.2-kc-0.6.19
file:/opt/kafka/plugins/iceberg-kafka-connect-runtime-hive-0.6.19/	io.tabular.iceberg.connect.transforms.CopyValue	transformation	undefined
file:/opt/kafka/plugins/iceberg-kafka-connect-runtime-hive-0.6.19/	io.tabular.iceberg.connect.transforms.DebeziumTransform	transformation	undefined
file:/opt/kafka/plugins/iceberg-kafka-connect-runtime-hive-0.6.19/	io.tabular.iceberg.connect.transforms.DmsTransform	transformation	undefined
file:/opt/kafka/plugins/iceberg-kafka-connect-runtime-hive-0.6.19/	io.tabular.iceberg.connect.transforms.JsonToMapTransform	transformation	undefined
file:/opt/kafka/plugins/iceberg-kafka-connect-runtime-hive-0.6.19/	io.tabular.iceberg.connect.transforms.KafkaMetadataTransform	transformation	undefined
file:/opt/kafka/plugins/iceberg-kafka-connect-runtime-hive-0.6.19/	io.tabular.iceberg.connect.transforms.MongoDebeziumTransform	transformation	undefined
]
Read the documentation at https://kafka.apache.org/documentation.html#connect_plugindiscovery for instructions on migrating your plugins to take advantage of the performance improvements of service_load mode. To silence this warning, set plugin.discovery=only_scan in the worker config. (org.apache.kafka.connect.runtime.isolation.Plugins) [main]