CLI Magic: Logrotate

This week’s CLI Magic comes from Mayank Sharma. While some might think that Logrotate is strictly a tool for system administrators, Mayank disagrees. He argues that even those as far down on the food chain as ordinary Linux desktop users — not just system admins — can benefit from the tool.You probably don’t visit /var/log every day, or take care to ensure its contents — program log files — are safe. Most people see log files as a record of events: something that has to do with the past rather than something with an impact on the present or the future.

In reality, effective log file management is critical for high system availability. And you don’t need to host a few websites or monitor a space craft to care about it. If you’re running a Linux distribution on your desktop, or managing a network of such desktops, you need an effective log management policy for everything from troubleshooting server errors to monitoring how devices interact with the kernel.

While they are simple text files, logs have a tendency to grow and grow and eat up valuable disk resources. Most distributions include a nifty tool, Logrotate, that rotates log files: copying them to a backup file and creating a new log. Backup logs are usually removed when they are out of date.

Understanding Logrotate

Many tools use the services of logrotate if it’s included with your distribution. If you have some logs under /var/log that are suffixed with a dot and a number (/var/log/daemon.log.1) or even compressed (/var/log/mysql.log.5.gz), it means they are being regulated by logrotate.

Logrotate needs a simple configuration file to help it manage the logs. You can have multiple configuration files, one for every log, or maintain multiple logs through one configuration file. The default configuration file is /etc/logrotate.conf. If you have services that keep a log under /var/log like Apache and MySQL, you’ll find their configuration files under /etc/logrotate.d.

Here is a sample logrotate.conf:


weekly
rotate 4
create
include /etc/logrotate.d

/var/log/wtmp {
missingok
monthly
create 0664 root utmp
rotate 1
}

The first four keywords, weekly, rotate 4, create and include are global directives, whereas the directives missingok, monthly, create and rotate are specific to the log file /var/log/wtmp. A local directive takes precedence over a global directive if it appears in both, like rotate in the listing above.

Working with Logrotate

If you have a program that keeps a log, you can save some serious effort by using logrotate to keep them in order for you. Logrotate can also work on logs of services that are run by a normal user. It also comes in handy if you have a program that keeps multiple logs and writes a lot to them.

Let’s assume we have a new service “foobar” which is configured to log under /home/bodhi/logs/foobar and runs as a normal user’s process. We keep its logrotate configurations under /home/bodhi/config in a file named foobar. Like this one:


/var/log/foobar/*.log {
daily
missingok
rotate 7
compress
delaycompress
create 640 bodhi bodhi
mail bodhi@localhost
sharedscripts
postrotate
/etc/init.d/foobar restart
endscript
}

We use daily to ensure that the logs are rotated everyday and rotate 7 to make sure no more than 7 days of logs are kept. When a log expires, it’s emailed to bodhi@localhost before being deleted as specified by mail. compress shrinks rotated logs except yesterday’s because of delaycompress. Logrotate doesn’t complain if the log files are missing thanks to missingok. After the logs have been rotated, a new one is created by create as being owned by the user and group bodhi and the service is restarted.

Putting it to work

Now that we have our configuration file in place, it’s time to create a cron job for it to actually make logrotate work. You do this by running crontab -e and adding a new cron job to the file.


00 00 * * * /usr/sbin/logrotate -s /home/bodhi/config/logrotate.status /home/bodhi/config/foobar


Cron ensures that the command runs at midnight everyday. The command has three parts. /usr/sbin/logrotate is the path to logrotate. The -s /home/bodhi/config/logrotate.status option specifies where logrotate keeps its status information. This file has to be writeable by the user running the cron. The last argument /home/bodhi/config/foobar tells logrotate to use our custom configuration file.

To test the configuration file, issue the command specified in the cron job with an additional -f switch. This will force the logs to be rotated. It helps to run logrotate with this switch every time the configuration files are changed.

Logrotate is a very simple but extremely useful tool. Many people fail to realize that logrotate can work for them as well. In this article, the configuration file instructs logrotate to take charge of a process running under a user.

We used several configuration options, but a visit to man logrotate for more options is a good idea before writing your own. If you have logrotate managing logs on your machine, reviewing its configuration files of the services its maintaining is a good way to learn logrotate’s common usage.