Opening model directory from s3 for sagemaker scheduled jobs #1546
ColinFerguson
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi all, thanks in advance for any help. I've used simpletransformers for several different projects but now I've come up against a new (to me) issue.
I am using AWS Sagemaker Studio, and have no problems training models and then using them for inference, e.g.
'''model = ClassificationModel('bert', 'path_to_model_directory', num_labels=2, use_cuda=True, args=model_args)'''
However, now I need to schedule a recurring job via sagemaker to run inference once per day. The sagemaker scheduler does not mount the local file system, so 'path_to_model_directory' effectively does not exist to a scheduled job.
The solution that makes the most sense to me would be to create a tar.gz file from the model directory, save that file to s3, and then somehow open it in my script (putting a tar.gz file in the call to ClassificationModel doesn't work). Does any one have any suggestions for how to do that, or another idea for how to access the model?
Thanks.
Beta Was this translation helpful? Give feedback.
All reactions