Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically create database if it doesn't exist #24

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rabidaudio
Copy link

With Meltano, I like to namespace each extractor to a new database to avoid conflicts (like tap-github and tap-shopify both having users tables) by setting the database name dynamically:

config:
  athena_database: $MELTANO_EXTRACTOR_NAMESPACE

However this means the database needs to exist before the tap can be run.

This change runs CREATE DATABASE IF NOT EXISTS before uploading.

Because meltano/sdk#246 hasn't been implemented, there's not a clean place to put a setup step to run once before the whole target run. Instead this will run for each sink in the constructor, which is not ideal but it should be fine.

@ndrluis
Copy link
Member

ndrluis commented Jan 6, 2023

Hello @rabidaudio Thank you for your contribution.

In our use case at Gupy we have the expectation to create the database manually, because of that I think that would be nice to have a setting to enable this feature.

What do you think about it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants