Log Rotation

"Log rotation" refers to the practice of archiving an application's current log, starting a fresh log, and deleting older logs. As we know, a "log" is a file where an application stores information that might be useful to an administrator or developer - what it's been doing, what errors it's run into, that sort of thing. So logs are good, we just usually don't want to keep a ton of them around. That's where logrotate comes in.

Logrotate is a utility/tool that manages activities like automatic rotation, removal and compression of log files in a system. This is a tool to manage our logs conserve disk space. By having a simple yet powerful configuration file, different parameters of logrotation can be controlled. This gives complete control over the way logs can be automatically managed and need not necessitate manual intervention.

Each log file may be handled daily, weekly, monthly, or when it grows too large. Normally, logrotate is run as a daily cron job. It will not modify a log multiple times in one day unless the criterium for that log is based on the log's size.

logrotate is a program used to rotate logs. The system usually runs logrotate once a day, and when it runs it checks rules that can be customized on a per-directory or per-log basis.

$ logrotate
logrotate 3.8.7 - Copyright (C) 1995-2001 Red Hat, Inc.
This may be freely redistributed under the terms of the GNU Public License

Usage: logrotate [-dfv?] [-d|--debug] [-f|--force] [-m|--mail=command] [-s|--state=statefile]
        [-v|--verbose] [--version] [-?|--help] [--usage] [OPTION...] <configfile>

Since the logrotate is based on configuration files, the above command will not rotate any files and will show us a brief overview of the usage and the switch options available.

Our system runs logrotate on a schedule, usually daily. In fact, we'll find the script that runs logrotate daily at:

/etc/cron.daily/logrotate

If we want logrotate to run more often (for hourly log rotation, for example) we'll need to look into using cron to run logrotate through a script in /etc/cron.hourly.

When logrotate runs it reads its configuration files to determine where to find the log files it needs to rotate, and to check on details like how often the files should be rotated and how many archived logs to keep.

The default options for the logrotate utility are present in /etc/logrotate.conf. Some of the important configuration settings are:

1. rotation-interval
2. log-file-size
3. rotation-count
4. compression

Application-specific log file information (to override the defaults) are kept at /etc/logrotate.d/.

BTW, where are the files to manage our server's and application's log for logrotate?

# On Ubuntu, most log files can be found here:
/var/log

# Some may appear in subdirectories
/var/log/nginx

# logrotate configurations are stored here:
/etc/logrotate.d

As log files grow, it becomes necessary to manage the logging mechanisms to avoid filling up disk space. Log rotation is the process of switching out log files and possibly archiving old files for a set amount of time.

The logrotate application is a simple program to rotate logs. It is installed on Ubuntu by default, and Nginx on Ubuntu comes with a custom logrotate script (/etc/logrotate.d/nginx):

/var/log/nginx/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 0640 www-data adm
        sharedscripts
        prerotate
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                        run-parts /etc/logrotate.d/httpd-prerotate; \
                fi \
        endscript
        postrotate
                [ -s /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
        endscript
}

The first line of the file specifies the location that the subsequent lines will apply to.

The rest of the file specifies that the logs will be rotate weekly and that 52 older copies will be preserved.

Here are the description of the sample configuration:

1. the logrotation for nginx monitors the /var/log/nginx/*.log file and does this on a weekly basis - this is the rotation interval.
2. rotate 52 signifies that 52 days worth of logs would be kept.
3. logfiles can be compressed using the gzip format by specifying compress and delaycompress delays the compression process till the next log rotation. delaycompress will work only if compress option is specified.
4. missingok avoids halting on any error and carries on with the next log file.
5. notifempty avoid log rotation if the logfile is empty.
6. create <mode> <owner> <group> creates a new empty file with the specified properties after log-rotation.

We can also set the logrotation as a cron so that the manual process can be avoided and this is taken care of automatically. By specifying an entry in /etc/cron.daily/logrotate, the rotation is triggered daily.

To verify if a particular log is indeed rotating or not and to check the last date and time of its rotation, check the /var/lib/logrotate/status file. This is a neatly formatted file that contains the log file name and the date on which it was last rotated.

Here are a few entries from the file:

k@laptop:~$ cat /var/lib/logrotate/status
logrotate state -- version 2
...
"/var/log/dpkg.log" 2017-2-1-2:38:16
"/var/log/nginx/error.log" 2017-2-7-11:32:48
"/var/log/nginx/access.log" 2017-2-10-10:9:24
"/var/log/upstart/*.log" 2017-2-10-10:0:0
"/var/log/apache2/access.log" 2017-1-18-13:33:47
...

The default options for rotation interval include:

daily
weekly
monthly
yearly

So, there is no hourly rotation interval, but we want to use an hour interval. Here, we'll have to get clever with cron and a separate config file.

1. Copy the /etc/cron.daily/logrotate script to /etc/cron.hourly directory:

   $ sudo cp /etc/cron.daily/logrotate /etc/cron.hourly
   

2. Put our app configuration file (myappname) into /etc/logrotate.d, for example:

   /var/log/myappname/app.log {
          hourly
          rotate 24
          delaycompress
          compress
          notifempty
          missingok
   }
   

   Note the first line should point to the location of our app.log.

DevOps

DevOps / Sys Admin Q & A