ci: support git-cache-rs [backport 2024.01] #20321
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #20311
Contribution description
This PR adds support for the Rust version of git-cache.
The shell version that we've integrated in
dist/tools/git
is quite a mess, barely maintainable. But most importantly, it is slow, and gets slower over time, due to the way the cache stores tags. Also, it doesn't support sparse checkouts. Crude measurements show that for our CI builds, for cortex-m builds 20-50% of time spent on each built are spent on checking out "pkg/cmsis".git-cache-rs' syntax is compatible with regular
git clone
to the point where anygit clone ...
command that doesn't work with just changing togit cache clone
is considered a bug.Unfortunately that requires some changes. This PR introduces git-cache-rs support as fully optional, it'll only be used if
GIT_CACHE_RS
is set to the binary name. The legacy built-in shell git-cache should continue working like before.git-cache-rs works a bit differently than the shell git-cache: instead of having one large bare cache repository that adds repos as remote, git-cache-rs just stores a "--mirror" repository for each cached repo. This is much simpler, although it might use a bit more space.
git-cache-rs supports sparse checkouts, which has been implemented here for
pkg/cmsis
. I've updated dwq and the murdock container, and am currently testing those changes on ci-staging.riot-os.org.Testing procedure
I've been testing this PR together with the CI support in RIOT-OS/riotdocker#240 for a day or so on https://ci-staging.riot-os.org/.
Issues/PRs references