Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Access kafka cluster topic in confluent cloud #281

Open
arshahmad1 opened this issue Aug 2, 2024 · 1 comment
Open

Access kafka cluster topic in confluent cloud #281

arshahmad1 opened this issue Aug 2, 2024 · 1 comment

Comments

@arshahmad1
Copy link

arshahmad1 commented Aug 2, 2024

I'm trying to connect this connector to the kafka topic present in my confluent cloud. I was able to access the confluent kafka topic from an ec2 instance. Now I'm creating this connector in same VPC, subnet and attaching same security group as of ec2 instance but I'm unable to get the data.
These are the configurations that I'm using but I'm getting timeout

iceberg.kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<CONFLUENT_API_KEY>" password="<CONFLUENT_API_SECRET>";
iceberg.kafka.security.protocol=SASL_SSL
iceberg.kafka.sasl.mechanism=PLAIN

connector.class=io.tabular.iceberg.connect.IcebergSinkConnector
table.write-format=parquet
iceberg.tables.evolve-schema-enabled=true
iceberg.fs.s3a.path.style.access=true
table.namespace=pocckafkaflink2
iceberg.catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog
tasks.max=1
topics=confluent_test_topic2
iceberg.catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO
iceberg.catalog.client.region=us-east-1
iceberg.fs.s3a.aws.credentials.provider=com.amazonaws.auth.DefaultAWSCredentialsProviderChain
iceberg.fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem
iceberg.tables=pocckafkaflink2.mskmytopicmaib
value.converter.schemas.enable=false
iceberg.catalog.warehouse=s3://bucket/data4
value.converter=org.apache.kafka.connect.json.JsonConverter
table.auto-create=true 


@arshahmad1
Copy link
Author

is this the correct format to provide sasl.jaas.config, security.protocol and sasl.mechanism

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant