Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating Iceberg Table in snowflake using Glue Catalog #266

Open
sharpsoul opened this issue Jun 19, 2024 · 0 comments
Open

Creating Iceberg Table in snowflake using Glue Catalog #266

sharpsoul opened this issue Jun 19, 2024 · 0 comments

Comments

@sharpsoul
Copy link

sharpsoul commented Jun 19, 2024

Hi All,

I'm using Kafka connector to Read CDC data sent by Postgres Source connector. I'm able to create Iceberg table using Glue catalog and I'm able to query it using Athena.

Now when I try to create Table in snowflake using same table from AWS Glue it gives me error "Iceberg tables using row-level deletes are not supported."

Snowflake Says they don't support external catalog with row-level-delete. In the following documentation they also talk about Copy-on-write (default) and Merge-on-read.
Snowflake Document

I tried setting up tables write properties to copy-on-write, and it is working. I used following config

iceberg.tables.auto-create-props.write.merge.mode=copy-on-write
iceberg.tables.auto-create-props.write.update.mode=copy-on-write
iceberg.tables.auto-create-props.write.delete.mode=copy-on-write

But I still get and error about "Iceberg tables using row-level deletes are not supported" in Snowflake

Is there any way to create compatible iceberg table which we can attach to snowflake.

Thanks in advance.

Update

I tried using version 1 of iceberg but in AWS Glue I can see table created but no data even though there are data files in S3 bucket under data folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant