Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate/add support for Kubeflow component registry hosts #2733

Open
ptitzler opened this issue May 18, 2022 · 4 comments
Open

Evaluate/add support for Kubeflow component registry hosts #2733

ptitzler opened this issue May 18, 2022 · 4 comments
Labels
component:catalog connectors Access to component catalogs external:Upstream Depended on deliverables in other repos outside the elyra org kind:enhancement New feature or request

Comments

@ptitzler
Copy link
Member

ptitzler commented May 18, 2022

Is your feature request related to a problem? Please describe.
An upcoming release (Q2 2022) of Kubeflow Pipelines will introduce support for component registries. We need to evaluate how Elyra can support those registries via its catalog connector infrastructure. Details will be added to this issue as they emerge.

Related: elyra-ai/examples#107 (MLX will implement the registry host API).

@ptitzler ptitzler added kind:enhancement New feature or request component:catalog connectors Access to component catalogs labels May 18, 2022
@ptitzler ptitzler added external:Upstream Depended on deliverables in other repos outside the elyra org and removed status:Needs Triage labels May 18, 2022
@ptitzler ptitzler added this to the 3.10 milestone May 19, 2022
@thesuperzapper
Copy link
Member

@ptitzler can you send a link to where you saw something about Kubeflow Pipelines component registries?

@ptitzler
Copy link
Member Author

I learned about registries in a recent conference talk from the MLX team. They are planning to implement the registry host API in a future release.

@thesuperzapper
Copy link
Member

@ptitzler I think that is pretty specific to MLX, and while cool, not everyone will be using MLX, so we will need to keep pushing multiple solutions for component registries.


Also, I think if we can find a way to expose a "public" interface for "generic" Elyra components (possibly like proposed in #2699), then we would need an Elyra-specific component catalog API for these generic components, as they would be higher-level than Kubeflow or Airflow.

We might even be able to make something like DockerHub for generic Elyra components, but who knows.

@ptitzler
Copy link
Member Author

ptitzler commented Jun 2, 2022

@ptitzler I think that is pretty specific to MLX, and while cool, not everyone will be using MLX, so we will need to keep pushing multiple solutions for component registries.

Right! We are continuing to support (and improve) the existing catalog connector API based on feedback we receive from contributors who've to to implement one. Since connector development requires relatively few insights into the innner workings of Elyra, we (the maintainers) are relying on the extended community to contribute connector implementations.

@akchinSTC akchinSTC modified the milestones: 3.10.0, Backlog Medium Term Jun 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:catalog connectors Access to component catalogs external:Upstream Depended on deliverables in other repos outside the elyra org kind:enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants