Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(system-backup): check outdated backup in if-not-present volume backup policy #3246

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

c3y1huang
Copy link
Contributor

Which issue(s) this PR fixes:

Issue longhorn/longhorn#6027

What this PR does / why we need it:

When the system backup volume backup policy is set to if-not-present, check the last backup and create a backup when it's outdated.

longhorn/longhorn#6027 (comment)

Special notes for your reviewer:

None

Additional documentation or context

None

@c3y1huang c3y1huang self-assigned this Oct 30, 2024
Copy link

coderabbitai bot commented Oct 30, 2024

Walkthrough

The changes in the pull request focus on the SystemBackupController within controller/system_backup_controller.go. Key modifications include the introduction of new methods for creating backups from snapshots and handling snapshot creation, along with enhanced error handling for timeout scenarios. The previous method for directly creating volume backups has been removed, and the logic for checking if backups are up-to-date has been encapsulated in a new method. Overall, the control flow for backup operations has been significantly adjusted to incorporate these changes.

Changes

File Change Summary
controller/system_backup_controller.go - Removed createVolumeBackup method.
- Added createVolumeBackupFromSnapshot method.
- Added createVolumeSnapshot method.
- Added isVolumeBackupUpToDate method.
- Updated error handling for snapshot creation timeouts.
controller/system_backup_controller_test.go - Modified TestReconcileSystemBackup to set datastore.SkipListerCheck to true.

Assessment against linked issues

Objective Addressed Explanation
Check if the latest backup data is up-to-date for if-not-present policy (#6027)

Possibly related PRs

Suggested reviewers

  • PhanLe1010
  • derekbit

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@c3y1huang c3y1huang force-pushed the 6027-system-backup-skip-when-backup-up-to-date branch from 514365a to c664be8 Compare October 30, 2024 07:48
@c3y1huang
Copy link
Contributor Author

@coderabbitai review

Copy link

coderabbitai bot commented Oct 30, 2024

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Outside diff range and nitpick comments (2)
controller/system_backup_controller.go (2)

49-63: Ensure Consistent Naming for Error Constants

The error constant names could be standardized for better consistency. Some constants use the action last (e.g., SystemBackupErrArchive), while others include additional context (e.g., SystemBackupErrTimeoutSnapshot). Consider adopting a uniform naming convention, such as <Entity><Action><Outcome>, to enhance readability and maintainability.


959-961: Add Logging for Snapshot Readiness

You might consider adding a log statement when the snapshot becomes ready to improve observability.

 if snapshot.Status.ReadyToUse {
+    log.Infof("Snapshot %v is ready to use", snapshot.Name)
     return snapshot, nil
 }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 235b1ae and c664be8.

📒 Files selected for processing (1)
  • controller/system_backup_controller.go (4 hunks)
🔇 Additional comments (2)
controller/system_backup_controller.go (2)

774-786: Verify Backup Freshness Check Logic

The backupVolumesIfNotPresent method checks if the volume backup is up-to-date using the isVolumeBackupUpToDate function. Ensure that this function accurately determines the need for a new backup by correctly comparing snapshot and backup timestamps.


928-964: Validate Timeout Handling in Snapshot Creation

The createVolumeSnapshot function correctly implements a timeout mechanism when waiting for the snapshot to become ready. Ensure that datastore.SystemBackupTimeout is set appropriately to prevent premature timeouts or excessively long waits.

controller/system_backup_controller.go Outdated Show resolved Hide resolved
controller/system_backup_controller.go Show resolved Hide resolved
controller/system_backup_controller.go Outdated Show resolved Hide resolved
controller/system_backup_controller.go Outdated Show resolved Hide resolved
controller/system_backup_controller.go Outdated Show resolved Hide resolved
@c3y1huang c3y1huang force-pushed the 6027-system-backup-skip-when-backup-up-to-date branch from c664be8 to 3865408 Compare October 30, 2024 08:32
@c3y1huang
Copy link
Contributor Author

@coderabbitai review

Copy link

coderabbitai bot commented Oct 30, 2024

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

1 similar comment
Copy link

coderabbitai bot commented Oct 30, 2024

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@c3y1huang c3y1huang force-pushed the 6027-system-backup-skip-when-backup-up-to-date branch from 3865408 to 70ad3d0 Compare October 30, 2024 08:54
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments.

Actionable comments posted: 1

🛑 Comments failed to post (1)
controller/system_backup_controller.go (1)

867-916: 🛠️ Refactor suggestion

Consider adding metrics for backup freshness tracking.

The isVolumeBackupUpToDate method effectively checks backup freshness. Consider adding metrics to track:

  • Time since last backup
  • Number of outdated backups
  • Average backup freshness

This would help in monitoring backup health and setting up alerts.

…ackup policy

longhorn/longhorn-6027

Signed-off-by: Chin-Ya Huang <[email protected]>
@c3y1huang c3y1huang force-pushed the 6027-system-backup-skip-when-backup-up-to-date branch from 70ad3d0 to d85549b Compare October 30, 2024 10:10
@c3y1huang
Copy link
Contributor Author

@c3y1huang c3y1huang marked this pull request as ready for review November 1, 2024 00:41
@c3y1huang c3y1huang requested review from derekbit and a team November 1, 2024 00:41
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (2)
controller/system_backup_controller_test.go (1)

Line range hint 89-114: Add test cases for outdated backup scenarios.

The current test cases verify if a backup exists but don't cover the new functionality of checking if the backup is outdated. Consider adding test cases for:

  1. Backup exists but is outdated (should create new backup)
  2. Backup exists and is recent (should not create new backup)
  3. Edge cases like backup timestamp exactly at the threshold

Would you like me to help draft these additional test cases?

controller/system_backup_controller.go (1)

867-916: LGTM: Well-implemented backup freshness check.

The implementation correctly checks if a backup is up-to-date by comparing snapshot timestamps, with proper error handling and logging.

Consider extracting time format constant.

The RFC3339 time format is used multiple times. Consider extracting it to a constant for better maintainability.

+const (
+    snapshotTimeFormat = time.RFC3339
+)

 func (c *SystemBackupController) isVolumeBackupUpToDate(volume *longhorn.Volume, systemBackup *longhorn.SystemBackup) (bool, error) {
     // ... existing code ...
-    lastBackupTime, err := time.Parse(time.RFC3339, lastBackupSnapshot.Status.CreationTime)
+    lastBackupTime, err := time.Parse(snapshotTimeFormat, lastBackupSnapshot.Status.CreationTime)
     if err != nil {
         return false, err
     }
     // ... more code ...
-    snapshotTime, err := time.Parse(time.RFC3339, snapshot.Status.CreationTime)
+    snapshotTime, err := time.Parse(snapshotTimeFormat, snapshot.Status.CreationTime)
     if err != nil {
         return false, err
     }
     // ... rest of the code ...
 }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between c664be8 and d85549b.

📒 Files selected for processing (2)
  • controller/system_backup_controller.go (6 hunks)
  • controller/system_backup_controller_test.go (1 hunks)
🔇 Additional comments (5)
controller/system_backup_controller_test.go (2)

Line range hint 147-167: LGTM! Comprehensive error handling test coverage.

The test cases effectively cover various error scenarios including:

  • Upload timeouts
  • Upload failures
  • Configuration retrieval failures
  • State transitions during errors

58-58: Document the purpose of SkipListerCheck and consider test isolation.

Setting global state in tests can lead to test pollution. Consider:

  1. Documenting why SkipListerCheck is necessary
  2. Moving this setting to individual test cases where needed
  3. Resetting the value after each test to prevent side effects

Let's check if this setting is used elsewhere:

controller/system_backup_controller.go (3)

50-64: LGTM: Error constants are well-defined.

The new error constants are clear, descriptive, and follow the existing naming pattern.


740-742: LGTM: Proper context management.

Good addition of context with cancel for managing operation lifecycle in backup functions.

Also applies to: 776-778


934-976: LGTM: Well-structured snapshot creation with proper timeout handling.

The implementation includes proper context cancellation, timeout handling, and status checks.

controller/system_backup_controller_test.go Show resolved Hide resolved
controller/system_backup_controller.go Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant