Configure Logrotate | Linux Log Management Tool

In this comprehensive guide, we will delve deep into understanding and configure Logrotate; Linux log management tool. Mastering its usage to setup circular logging on Linux systems. #centlinux #linux #logrotate

What is Circular Logging?

Circular logging is a technique used in computer systems, particularly in logging mechanisms, for efficient log management. In circular logging, instead of continuously creating new log files as old ones fill up, the system reuses a fixed amount of space by overwriting the oldest log entries with new ones when the allocated space is exhausted.

This approach ensures that the log files never grow too large and consume excessive storage space. However, it also means that older log entries are lost once they are overwritten by newer ones. Circular logging is often used in scenarios where preserving all log entries indefinitely is not necessary, such as in real-time monitoring systems or environments where storage space is limited.

What is Logrotate?

Logrotate is a utility designed to simplify the log management in Linux systems. You can configure Logrotate to automates the rotation, compression, removal, and mailing of log files as per specified criteria, ensuring that log files do not consume excessive disk space and are organized in a manageable manner.

Features of Logrotate?

Logrotate in Linux is equipped with following log management features:

1. Log Rotation: Automatically rotates log files at scheduled intervals or when they reach a certain size, ensuring that log files don’t grow indefinitely.
2. Compression: Supports compressing rotated log files to conserve disk space. It can compress logs using gzip, bzip2, or other compression tools.
3. Retention Policies: Allows specifying retention policies for rotated log files, including how many rotated log files to keep and for how long.
4. Post-Rotation Actions: Supports executing custom scripts or commands after rotating log files, enabling actions such as restarting services or notifying administrators.

Installation of Logrotate in Linux

Installing Logrotate in Linux system is a straightforward process. Simply use your distribution’s package manager to install the Logrotate package. For example, on Debian-based systems, you can install it using the following command:

sudo apt-get install logrotate

To install logrotate on Red Hat Linux based distros, you can use the package manager yum or dnf (depending on the version of Red Hat Linux, you are using). Here’s how you can do it:

sudo yum install logrotate   # For Red Hat versions 6 and older

or

sudo dnf install logrotate   # For Red Hat versions 7 and newer

Once the installation of logrotate in Linux is completed, you can verify that logrotate is installed by running:

logrotate --version

This command should display the version of logrotate command, that is installed on your Linux system.

Configure Logrotate in Linux

Basic Configuration

The configuration of Logrotate in Linux is primarily managed through the /etc/logrotate.conf file, along with additional configuration files located in the /etc/logrotate.d/ directory.

Let’s explore some key directives used to configure Logrotate:

1. rotate

This directive specifies the number of rotated log files to keep. For example, rotate 7 would retain the last seven rotated log files.

2. weekly

The weekly directive determines the frequency of log rotation. Logrotate will rotate logs on a weekly basis.

3. compress

By using the compress directive, Logrotate compresses rotated log files with gzip by default, conserving disk space.

4. create

The create directive specifies the permissions and ownership of newly created log files.

5. postrotate

This directive allows you to execute custom commands or scripts after log rotation completes. It’s useful for tasks like restarting services to ensure they begin writing to the new log file.

Custom Log Rotation Configuration

In addition to the global configuration file, you can create separate configuration files for individual log files or directories within the /etc/logrotate.d/ directory. This allows for granular control over log rotation settings.

Configure Logrotate Log Management

Example 1: Rotating a Specific Log File

/path/to/logfile.log {
    rotate 5
    weekly
    compress
    create 0644 root root
}

Example 2: Rotating All Logs in a Directory

/path/to/logs/*.log {
    rotate 10
    weekly
    compress
    create
    postrotate
        /usr/bin/systemctl restart myservice
    endscript
}

Example 3: Rotating Apache access logs weekly and keeping the last 4 weeks’ worth of logs:

/var/log/apache/access.log {
    weekly
    rotate 4
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        systemctl reload apache2
    endscript
}

Example 4: Rotating syslog logs daily and compressing logs older than 7 days:

/var/log/syslog {

    daily

    rotate 7

    compress

    missingok

    notifempty

    delaycompress

    sharedscripts

    postrotate

        systemctl reload rsyslog

    endscript

}

Example 5: Rotating Nginx error logs monthly and deleting logs older than 3 months:

/var/log/nginx/error.log {
    monthly
    rotate 3
    missingok
    notifempty
    create 0644 www-data adm
    sharedscripts
    postrotate
        systemctl reload nginx
    endscript
}

Example 6: Rotating MySQL slow query logs hourly and retaining logs for 24 hours:

/var/log/mysql/mysql-slow.log {
    hourly
    rotate 24
    create 0640 mysql mysql
    missingok
    notifempty
    sharedscripts
    postrotate
        mysqladmin -uroot -p<password> flush-logs
    endscript
}

Example 7: Rotating application-specific logs based on size, with log files limited to 100 MB each and keeping the last 10 rotated logs:

/path/to/application.log {
    size 100M
    rotate 10
    missingok
    notifempty
    copytruncate
    compress
    sharedscripts
}

Replace /path/to/application.log with the actual path to the application log file.

Example 8: Rotate Log and Email Previous Logfile

Here is another configuration example of logrotate in Linux that rotates a log file and sends an email with the previous log file:

/path/to/your/logfile.log {
    rotate 5              # Keep 5 rotated copies
    weekly                # Rotate logs weekly
    missingok             # If the log file is missing, don't show an error
    notifempty            # Don't rotate the log file if it's empty
    compress              # Compress rotated log files
    delaycompress         # Postpone compression until the next rotation
    create 0644 username username   # Create new log files with these permissions and ownership
    sharedscripts         # Run postrotate script only once even if multiple log files match the wildcard pattern
    postrotate
        /usr/bin/mail -s "Log Rotation Report" alaric@centlinux.com < /path/to/your/logfile.log.1
    endscript
}

Above example is more complex than prior. Therefore, you need to understand it’s each directive

• /path/to/your/logfile.log is the path to the log file you want to rotate.
• rotate 5 specifies that only 5 rotated copies of the log file will be kept.
• weekly specifies that logs will be rotated weekly.
• missingok tells logrotate to not show an error if the log file is missing.
• notifempty instructs logrotate to not rotate the log file if it’s empty.
• compress enables compression of rotated log files.
• delaycompress postpones compression until the next rotation.
• create 0644 username username specifies the permissions and ownership for newly created log files. Replace username with the appropriate username.
• sharedscripts ensures that the postrotate script is only run once, even if multiple log files match the wildcard pattern.
• postrotate and endscript define a script that sends an email with the previous log file using the mail command. Replace alaric@centlinux.com with your email address.

Make sure to replace placeholders like /path/to/your/logfile.log and alaric@centlinux.com with your actual log file path and email address.

Managing Logrotate with Cron

Logrotate is typically scheduled to run via a Linux cron command. By default, Logrotate in Linux configuration includes a cron job that executes daily. However, you can customize the schedule by modifying the cron configuration file (/etc/cron.daily/logrotate).

Verifying Logrotate Linux Configuration

After configuring Logrotate in Linux, it is essential to verify that it’s functioning as expected. You can manually trigger log rotation using the following command:

sudo logrotate -vf /etc/logrotate.conf

This command will perform a verbose rotation based on the specified configuration file, allowing you to observe any errors or unexpected behavior.

Disable Logrotate Configurations

To disable all logrotate configurations, you can simply move or rename the /etc/logrotate.conf file and the entire /etc/logrotate.d/ directory. Here’s how you can do it:

To disable the main logrotate configuration file, you can either move it or rename it:

sudo mv /etc/logrotate.conf /etc/logrotate.conf.disabled

To disable all logrotate configurations stored in the /etc/logrotate.d/ directory, you can move the entire directory:

sudo mv /etc/logrotate.d/ /etc/logrotate.d.disabled/

After performing these steps, logrotate will no longer rotate any log files or directories on your system.

Remember that disabling logrotate configurations may result in log files growing indefinitely and consuming disk space, so it’s important to monitor log file sizes and manage them appropriately.

If needed, you can re-enable logrotate configurations by moving the files and directories back to their original locations.

Bonus Stuff: Logrotate Cheatsheet

Recommended Training: Linux Command Line

Conclusion

Configure Logrotate in Linux is crucial for efficient log management. By understanding its configuration options and leveraging its capabilities, system administrators can ensure log files are appropriately managed, preventing disk space issues and facilitating easier log analysis.