-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pickling a SageMaker step corrupts part of the object. #3272
Comments
There is a bug in this metaclass for the StepTypeEnum: https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/workflow/entities.py#L41 Basically, |
Could you elaborate on your use case? Why would you like to pickle/unpickle the pipeline objects? If you like to share the pipeline definition with someone else, you either share the source code or do Even though we fix the StepTypeEnum bug, I still see a couple of problems with pickling the pipeline objects. For example, dill couldn't handle Enum object: uqfoundation/dill#250 |
Closing this issue as the underlying StepTypeEnum bug has been fixed, and other questions have been abandoned. Feel free to re-open. |
Describe the bug
This issue is related to boto/boto3#3365. I have been trying to pickle and unpickle a SageMaker step and the relevant part for this repo is the fact that after unpickling a step the step.step_type becomes CONDITION from what it was originally:
Original:
ProcessingStep(name='calls', display_name=None, description='Processing missing calldata.', step_type=<StepTypeEnum.PROCESSING: 'Processing'>, depends_on=None)
Becomes:
ProcessingStep(name='calls', display_name=None, description='Processing missing calldata.', step_type=<StepTypeEnum.CONDITION: 'Condition'>, depends_on=None)
To reproduce
I cannot give the complete code for the ProcessingStep but I think any initialization would produce the bug.
Expected behavior
I expect once unpickled the object to not have corrupted fields. the step.step_type should not change.
System information
A description of your system. Please provide:
Additional context
To pickle a SageMaker step sometimes you have to add the following code as well. This is due to botocore and related to the boto/boto3#3365 issue.
The text was updated successfully, but these errors were encountered: