What is the result of sample running code?

[zw615]

Hi, there!

I am new to this codebase. I have found several sample running code on cc12m or laion-400m, like this one

cd open_clip/src
torchrun --nproc_per_node 4 -m training.main \
    --train-data '/data/cc12m/cc12m-train-{0000..2175}.tar' \
    --train-num-samples 10968539 \
    --dataset-type webdataset \
    --batch-size 320 \
    --precision amp \
    --workers 4 \
    --imagenet-val /data/imagenet/validation/

But I cannot find much description about what performance this code may achieve, like the zero-shot classification accuracy on ImageNet. Only a very short description in the released pretrained weights here, which shows a zero-shot acc of 36.45 of ResNet50. I wonder is that achieved by the sample running code? Or how I can reproduce performace of that level?

Similarly, what is the running code to achieve the released checkpoints' accuracy like vit_b_16-laion400m_e32 or vit_l_14-laion400m_e32?

[mitchellnw]

Hello!

If you look for "slurm script" here you'll be able to find the command corresponding to the model: https://huggingface.co/laion/CLIP-ViT-L-14-laion2B-s32B-b82K.

Sample accuracy curves for conceptual captions are here https://github.com/mlfoundations/open_clip/blob/main/docs/clip_conceptual_captions.md