Loading weights into timm models #353
-
I have trained a bunch of Resnet50 and ViT-B-32, and I would like now to use these weights to create a new model using Thanks. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 2 replies
-
@AlaaKhaddaj if you used a
The open-clip ViT model and ResNet model need remapping though, I don't have a remap for open_clip ResNet -> timm, but do have code for ViT. NOTE, the ViT model is different than the default timm vit, so needs to be one of the |
Beta Was this translation helpful? Give feedback.
-
Hey @rwightman, is there any way to convert the CoCa models to timm? |
Beta Was this translation helpful? Give feedback.
@AlaaKhaddaj if you used a
timm-
model, it's very easy, you just need to modify the state dict, and then re-save...The open-clip ViT model and ResNet model need remapping though, I don't have a remap for open_clip ResNet -> timm, but do have code for ViT. NOTE, the ViT model is different than the default timm vit, so needs to be one of the
vit_xxx_patchxx_clip_xx
models not the ones w/o the 'clip' in them, there is an extra norm near the beginning of the network.https://github.com/rwightman/pytorch-image-models/blob/94a9159…