Attempt to speed up deploy

[yosifkit]

Now that we have rate limits under control, we should always try a HEAD request on the indexes & manifests before the PUT request (since the PUT is much slower).

Some local testing for riscv64, arm32v6, and s390x using docker-library/meta@6c3188a shows some significant speed increases. I chose these arches since they were fully pushed and would only do head requests (and my Docker Hub user doesn't have access to push to the arch namespaces anyway).

$ jq -L ../meta-scripts/ 'include "deploy"; arch_tagged_manifests("riscv64") | deploy_objects[]' builds.json > deploy.json
$ time ../meta-scripts/bin/deploy < deploy.json
...
real    0m6.736s
user    0m0.065s
sys     0m0.032s
$ # compared to Jenkins: ~ 2min 49s - 4min 49s

$ jq -L ../meta-scripts/ 'include "deploy"; arch_tagged_manifests("arm32v6") | deploy_objects[]' builds.json > deploy.json
$ time ../meta-scripts/bin/deploy < deploy.json
...
real    0m32.444s
user    0m0.053s
sys     0m0.035s
$ # compared to Jenkins: ~ 5min 16s - 8min 58s

$ jq -L ../meta-scripts/ 'include "deploy"; arch_tagged_manifests("s390x") | deploy_objects[]' builds.json > deploy.json
$ time ../meta-scripts/bin/deploy < deploy.json
...
real    3m12.348s
user    0m0.278s
sys     0m0.252s
$ # compared to Jenkins: ~ 19min 36s - 24min 22s

[whalelines]

So this only removes the size check for manifests? Are all our requests still properly accounted for when considering pull and abuse limits?

[yosifkit]

So this only removes the size check for manifests? Are all our requests still properly accounted for when considering pull and abuse limits?

Yup, the rate limiting is active on every request to Docker Hub using this registry client:

meta-scripts/registry/client.go

Line 35 in 9c6f9d4

clientOptions.Transport = &rateLimitedRetryingRoundTripper{

[tianon]

Because of the way that's implemented, it technically applies to requests to auth.docker.io too (any HTTP request that's in the path to our actual lookups gets accounted individually, even including redirects), so it's technically a little tiny bit overaggressive, but I think it's probably better to limit too much than too little. 😅

[tianon]

Confirmed, deploy-riscv64 went from ~3m15s down to ~13s 😂

[tianon]

For unrelated reasons, deploy-arm32v7 has been failing for a little bit, so it had a definitely load-bearing run at ~15m9s with a prior successful run at ~27m, so definitely very net positive. 👍