Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically Pause Zombie Clients #7738

Open
Tracked by #7406
beautifulentropy opened this issue Oct 2, 2024 · 0 comments
Open
Tracked by #7406

Automatically Pause Zombie Clients #7738

beautifulentropy opened this issue Oct 2, 2024 · 0 comments
Assignees

Comments

@beautifulentropy
Copy link
Member

beautifulentropy commented Oct 2, 2024

Manual Pausing Background

In #7406, we deployed all the necessary code and infrastructure to manually pause specific account-identifier pairs. Two batches of manual pauses were conducted based on 90 days of authorization logs:

  • Batch 1: Averaged 50 authorization failures per day, with no successful attempts, over 90 days.
  • Batch 2: Averaged 40 authorization failures per day, with no successful attempts, over 90 days.

After a few weeks with no complaints and very few unpauses, it seems reasonable to move forward with automated detection and pausing for accounts that meet the criteria established in our second batch.

Automatic Pausing Requirements

To efficiently identify pairs for pausing, we'll implement a new rate limit within our existing key-value rate limit system. This limit will be similar to our current FailedAuthorizationsPerDomainPerAccount limit and will use the same bucket key format of enum:regId:domain.

However, there are some differences:

  1. The configured period will match our longest issuance time, 90 days.
  2. The configured count will be our period (90) * acceptable failures per day (40), or 3600.
  3. The bucket will always be reset to 0 if the subscriber successfully validates an authorization for that identifier.
  4. When the limit is reached, the account and identifier will be added to our paused table by calling SA.PauseIdentifiers().

Any subsequent new-order requests from this account for certificates containing this identifier will then be rate limited. The rate limit notice will include a URL they can use to automatically unpause all paused identifiers associated with their account.

@beautifulentropy beautifulentropy changed the title Pause misconfigured clients automatically Automatically Pause Mis-configured Clients Oct 2, 2024
@beautifulentropy beautifulentropy changed the title Automatically Pause Mis-configured Clients Automatically Pause Zombie Clients Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants