Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

show table extended in {{relation.shema}} like * fails #1099

Open
2 tasks done
rahulgoyal2987 opened this issue Sep 6, 2024 · 1 comment
Open
2 tasks done

show table extended in {{relation.shema}} like * fails #1099

rahulgoyal2987 opened this issue Sep 6, 2024 · 1 comment
Labels
awaiting_response bug Something isn't working

Comments

@rahulgoyal2987
Copy link
Contributor

Is this a new bug in dbt-spark?

  • I believe this is a new bug in dbt-spark
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

show table extended in {{relation.shema}} like * fails if any other dbt model is running on {{relation.shema}} same time.

Expected Behavior

Please change show table extended in {{relation.schema}} like '*' with below implementation to avoid parallel running model issue in same database.
for table in spark.catalog.listTables({{relation.schema}}):
spark.sql("show table extended in {{relation.schema}} like '{table.name}'")

Steps To Reproduce

Run two model parallel in same database

Relevant log output

No response

Environment

- OS: AWS EMR image
- Python: 3.11
- dbt-core: 1.0.0
- dbt-spark: 1.0.0

Additional Context

No response

@rahulgoyal2987 rahulgoyal2987 added bug Something isn't working triage labels Sep 6, 2024
@amychen1776
Copy link
Contributor

Hello @rahulgoyal2987 , you're currently on a EOL supported dbt core version. Could you update to 1.8 for both core and adapters and let me know if you're still experiencing this issue? We have made some significant changes with our metadata queries since 1.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting_response bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants