Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOPS AZKV getDefaultAzureCredential triggers API rate limit #840

Open
apeschel opened this issue Apr 21, 2023 · 3 comments
Open

SOPS AZKV getDefaultAzureCredential triggers API rate limit #840

apeschel opened this issue Apr 21, 2023 · 3 comments

Comments

@apeschel
Copy link
Contributor

apeschel commented Apr 21, 2023

When the Kustomize controller comes up, the getDefaultAzureCredential function will hammer the Azure API with login requests and get rate limited for a few minutes, resulting in failed Kustomize runs.

It would be preferable to cache the auth token and reuse it, rather than retrieve a new one on each SOPS decryption.

https://github.com/fluxcd/kustomize-controller/blob/main/internal/sops/azkv/keysource.go#L216

@kingdonb
Copy link
Member

kingdonb commented May 3, 2023

I don't know if we can cache the auth token and reuse it across Kustomizations without violating multi-tenancy. You can set retryInterval to a shorter value than interval if you don't want to wait for the whole interval (and since you've probably already set it longer than the default interval to avoid the rate limiting as much as possible.)

There may be some improvement that's possible to handle the rate limiting behavior a bit more cleanly.

@Maxcodesthings
Copy link

So for every kustomization that uses SOPS it will decrypt the secret each interval? That does sound overkill. I am also nearing a limit for the number of calls SOPS is making to AKV.

Is the feature gate CacheSecretsAndConfigMaps supposed to help with this?

@souleb
Copy link
Member

souleb commented May 21, 2024

This is being worked on in fluxcd/pkg#766

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants