Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Choosing Dask Clusters for specific Flows #120

Open
ctuguinay opened this issue Jul 25, 2024 · 0 comments
Open

Choosing Dask Clusters for specific Flows #120

ctuguinay opened this issue Jul 25, 2024 · 0 comments

Comments

@ctuguinay
Copy link

After talking with Wu-Jung and Valentina, we think it would be good to explore using a specific Dask Cluster / no Dask Cluster for specific Flows.

One case where this would be useful is if you need to restart a Dask Cluster for a specific Flow (e.g. accumulation of RAM in the ep.open_raw subflow) and not disturb other Flows. Another case where this would be useful is if you have a Flow that needs a Dask Cluster and another Flow that doesn't need one, then you would assign a Dask Cluster to the Flow that needs a Dask Cluster, and not assign a Dask Cluster to the Flow that doesn't need one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant