Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase log retention by expanding log storage #865

Open
gsanchietti opened this issue Oct 18, 2024 · 3 comments
Open

Increase log retention by expanding log storage #865

gsanchietti opened this issue Oct 18, 2024 · 3 comments
Assignees
Labels
feature New feature or request verified All test cases were verified successfully

Comments

@gsanchietti
Copy link
Member

The current log retention period is approximately one day in a small to medium-sized installation, primarily due to the limited log storage space. Currently, only 50MB is allocated for non-rotated logs, significantly restricting the ability to review logs from past days, which makes it difficult to analyze system behavior or investigate security events.

Proposed solution

Increase the memory allocated for log storage by setting the default size of tmpfs to 10% of the available system memory, with a minimum size of 50MB. This adjustment would utilize the underutilized system memory, as modern installations typically have at least 4 GB of RAM leaving ample room to allocate more space to logs, allowing for longer retention and reducing the risk of losing critical information.

Additionally, users should have the option to configure a custom log storage size to fit their specific needs, while ensuring that the minimum size of 50MB is respected.

Alternative solutions

One alternative would be to maintain the current log size allocation and rely on external solutions for log storage or archiving. However, this would require additional configuration and might not address the immediate need for extended on-device log retention.

@gsanchietti gsanchietti added this to the NethSecurity 8.4 milestone Oct 18, 2024
@gsanchietti gsanchietti added the feature New feature or request label Oct 18, 2024
gsanchietti pushed a commit that referenced this issue Oct 21, 2024
Set log rotation size dynamically of 10% tmpfs.
Add ns-log-size script to customize the value.

#865
Copy link
Contributor

Testing image version: 8-23.05.5-ns.1.3.0-20-g57d36c404b3

@github-actions github-actions bot added the testing Packages are available from testing repositories label Oct 21, 2024
@gsanchietti
Copy link
Member Author

Test case 1

  • Install a machine with 1GB of RAM with the testing image
  • Verify the rotation size is the minimum (50MB)

Test case 2

  • Install a machine with 2GB of RAM with the testing image
  • Verify the rotation size is 10% of tmpfs

Test case 3

  • Update an existing installation with the testing image
  • Verify the rotation size is not changed
  • Follow the documentation and change the rotation size
  • Verify the rotation size is reflected inside rsyslog configuration file

@gsanchietti
Copy link
Member Author

Test case 1: config, $outchannel log_rotation,/var/log/messages, 52428800, /usr/sbin/rotate-messages
Test case 2: config $outchannel log_rotation,/var/log/messages, 104414412, /usr/sbin/rotate-messages
Test case 3: get and set work, rsyslog is correctly restarted

@gsanchietti gsanchietti added verified All test cases were verified successfully and removed testing Packages are available from testing repositories labels Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request verified All test cases were verified successfully
Projects
Status: In progress 🛠
Development

No branches or pull requests

2 participants