Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] support of customized components in monai_bundle_inference_operator #367

Open
mingxin-zheng opened this issue Oct 12, 2022 · 7 comments
Assignees
Labels
enhancement New feature or request

Comments

@mingxin-zheng
Copy link

Is your feature request related to a problem? Please describe.

Currently MONAI bundle allows user to define their components (class) and supports the initiation of the class from the bundle template during inference. We can possibly extend monai_bundle_inference_operator.py to better support these customized components

@mingxin-zheng mingxin-zheng added the enhancement New feature or request label Oct 12, 2022
@MMelQin
Copy link
Collaborator

MMelQin commented Oct 16, 2022

Thanks @mingxin-zheng.

As the MONAI Bundle expands its capability, the App SDK MONAI Bundle (automated) Inference operator will need to follow suite in order to support more use cases (as of now, only pre-processing, post-processing, and inferer attributes, besides metadata.json, are supported).

I see this enhancement of low to medium difficulty, especially after the load and save image logic is separated out from pre and post processing respectively.

@mingxin-zheng mingxin-zheng self-assigned this Oct 17, 2022
@mingxin-zheng
Copy link
Author

@MMelQin Thanks for the suggestion and evaluation. I will submit a PR for your review.

By looking at the code, I am confused by the way to get MONAI bundle config and files.
MonaiBundleInferenceOperator expects the metadata.json in the extra folder in the zipped bundle package (TorchScript style?)

metadata_relative_path = "extra/metadata.json"

But the MONAI bundles are in a different format (example).

Is there an example how MONAI deploy can deploy the current MONAI bundle in the model zoo?

@MMelQin
Copy link
Collaborator

MMelQin commented Oct 17, 2022

@mingxin-zheng MONAI Bundle IS a TorchScript with extra config data in JSON.

The example application using MONAI Bundle is the Spleen Seg, and the model is from Model Zoo Spleen Bundle. I have notice the bundle itself has changed over time, but the format remains the same.

@mingxin-zheng
Copy link
Author

@MMelQin Thanks for the confirmation. I started with the assumption that bundle isn't always TorchScript. By looking at the storage, it seems most has .ts in the model folder but there are a few that did not follow. For example:
https://github.com/Project-MONAI/model-zoo/releases/download/hosting_storage_v1/renalStructures_UNEST_segmentation_v0.2.0.zip

Maybe we make deploy to adapt both (torchscript and non-torchscript)? @MMelQin @Nic-Ma

@MMelQin
Copy link
Collaborator

MMelQin commented Oct 17, 2022

@MMelQin Thanks for the confirmation. I started with the assumption that bundle isn't always TorchScript. By looking at the storage, it seems most has .ts in the model folder but there are a few that did not follow. For example: https://github.com/Project-MONAI/model-zoo/releases/download/hosting_storage_v1/renalStructures_UNEST_segmentation_v0.2.0.zip

Maybe we make deploy to adapt both (torchscript and non-torchscript)? @MMelQin @Nic-Ma

That would be too arbitrary and was the exact reason why we picked the common and standard model archive format.

@mingxin-zheng
Copy link
Author

Okay I see. I downloaded all the bundles and check which model actually HAS model.ts file. Here is a snapshot @MMelQin @Nic-Ma

  • brats_mri_segmentation_v0.3.0.zip
  • endoscopic_inbody_classification_v0.2.0.zip
  • endoscopic_tool_segmentation_v0.2.0.zip
  • lung_nodule_ct_detection_v0.3.0.zip
  • mednist_gan_v0.4.0.zip
  • pancreas_ct_dints_segmentation_v0.3.0.zip
  • pathology_tumor_detection_v0.3.0.zip
  • prostate_mri_anatomy_v0.3.0.zip
  • renalStructures_UNEST_segmentation_v0.2.0.zip
  • spleen_ct_segmentation_v0.3.2.zip
  • spleen_deepedit_annotation_v0.3.0.zip
  • swin_unetr_btcv_segmentation_v0.3.0.zip
  • valve_landmarks_v0.4.0.zip
  • ventricular_short_axis_3label_v0.3.0.zip
  • wholeBrainSeg_Large_UNEST_segmentation_v0.2.0.zip

I may close this ticket as the issue sounds different that how it was first framed...

@dbericat
Copy link
Member

dbericat commented Nov 9, 2022

Expand support for more Bundles with additional attributes or components.

Some of them are custom and already supported for Core and Label.

Today we only support pre-transform, inference and post-transform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants