Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discuss] How to support dimension lookup query for transform #7937

Open
2 of 3 tasks
hailin0 opened this issue Oct 29, 2024 · 1 comment
Open
2 of 3 tasks

[Discuss] How to support dimension lookup query for transform #7937

hailin0 opened this issue Oct 29, 2024 · 1 comment
Labels

Comments

@hailin0
Copy link
Member

hailin0 commented Oct 29, 2024

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

How to support dimension lookup query for transform

And as EtLT data synchronization, I think we need to support lookup query for transform, and I want to hear from you all how you think it can be implemented in SeaTunnel.

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@hailin0 hailin0 changed the title [Discuss]How to support dimension lookup query for transform [Discuss] How to support dimension lookup query for transform Oct 29, 2024
@zhdech
Copy link

zhdech commented Oct 29, 2024

Implementation method:
Option 1 for batch strategy: When starting the strategy, connect to the database, query the database, and save the data results to the cache; When each line of data arrives, search for cached data.
Option 2 for streaming media strategy: On the basis of 1, regularly refresh cached data to ensure that cached data is updated regularly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants