Improve system performance by moving your log files to RAM

The Ramlog project lets you keep your system logs in RAM while your machine is running and copies them to disk when you shut down. If you are running a laptop or mobile device with syslog enabled, Ramlog might help you increase your battery life or the life of the flash drive on your mobile device. As a side effect of using Ramlog, you will be less likely to be caught out by a daemon that suddenly starts sending a message to syslog every 30 seconds and saps your battery keeping the hard disk spinning.

The life of the flash drive is an important consideration if you are using a solid state drive. It is likely that the /usr partition does not get written to very much, with /var and /tmp taking the lion’s share of disk writes compared to the handful of times you might save source code in your home directory per day. Moving your system logs to RAM means that the solid state drive will see a lot less write activity. That means it should take longer to start producing unwritable sectors. The tradeoff is that if the laptop crashes instead of shutting down gracefully, you will lose the logs that were in RAM, but most of the time on a laptop the logs are not mission-critical.

The repositories for openSUSE, Ubuntu, and Fedora do not contain packages for Ramlog. The project’s home page includes .rpm files for Fedora, .deb files for Ubuntu, and a source tarball. I’ll build from source on a 64-bit Fedora 8 machine using Ramlog version 1.1.0.

The first steps of installation are shown below. Note that the 1.1.0 tarball of Ramlog expands directly into the current directory instead of creating a new subdirectory. These steps put the files in place ready for the final stage of installation. If you are running Ubuntu, see the Ramlog INSTALL file for the commands that you should execute instead of the chkconfig ones shown here for Fedora.

$ mkdir ramlog-1.1.0
$ cd ramlog-1.1.0
$ tar xzvf /.../ramlog-1.1.0-1.tar.gz 
$ su
...
# install -m 750 ramlog /etc/init.d/ramlog
# install -m 444 ramlog.8.gz /usr/share/man/man8/ramlog.8.gz
# install -m 755 ramlog.cron /etc/cron.daily/ramlog.cron
# install -m 644 ramlog.logrotate /etc/logrotate.d/ramlog

# chkconfig --add ramlog
# chkconfig ramlog on

The getlogsize option to the ramlog command causes it to print how large your current /var/log directory is. You then select how large you want your RAM disk to be so that your log files will fit onto it. If you are using logrotate to keep your log files from growing infinitely, adding 30-50% to the current size of your log files should provide sufficient space in the RAM disk for your logging. If you do not set the size of your RAM disk explicitly then you are relying on the value that your distribution has selected for you. To explicitly set the value, edit your grub.conf file and append ramdisk_size=n, where n selects how many kilobytes to make the RAM disk. Because my existing /var/log was a little over 50MB, I selected a RAM disk size of 80MB.

# ./ramlog getlogsize
Ramlog: Size of /var/log is: 52640 kbytes
# vi /boot/grub/grub.conf
...
title Fedora (2.6.24.4-64.fc8 ramlog)
        root (hd0,0)
        kernel /vmlinuz-2.6.24.4-64.fc8 ro root=/dev/VolGroup00/LogVol00 rhgb quiet ramdisk_size=80000
        initrd /initrd-2.6.24.4-64.fc8.img

To start using Ramlog you can either reboot, or stop every program that is using /var/log, start ramlog, and then restart the programs you stopped. One upshot of rebooting is that your RAM disk size parameter will be in effect; if you try to force ramlog to start on a running system, you are at the mercy of whatever RAM disk size is currently available (64MB according to the Ramlog documentation). The below command shows you what is currently using /var/log; in my case, quite a few daemons and the X Window System are active, making a reboot the more attractive option for starting to use Ramlog. If you accidentally boot a kernel that uses a RAM disk that is too small, then Ramlog will fail to start during boot, your /var/log will remain on hard disk, and things will function as they did before you installed Ramlog.

# service ramlog teststartstop
The list of open files: (You need to close below daemons if you want to start/stop ramlog manually)

COMMAND     PID   USER   FD   TYPE DEVICE    SIZE    NODE NAME
auditd     1680   root    5w   REG  253,0 4590592 3936085 /var/log/audit/audit.log
rsyslogd   1934   root    2w   REG  253,0   23860 3933163 /var/log/messages
...
setrouble  1966   root    4w   REG  253,0     131 3935366 /var/log/setroubleshoot/setroubleshootd.log-20080705
httpd      2053   root    2w   REG  253,0    8160 3935447 /var/log/httpd/error_log
...
cupsd      2178   root   11u   REG  253,0   14729 3933123 /var/log/cups/access_log
X          2430   root    0r   REG  253,0   30710 3933132 /var/log/Xorg.0.log
...
mysqld    25212  mysql    1w   REG  253,0    1468 3935384 /var/log/mysqld.log
...
Test result: ramlog cannot be started or stopped at the moment.

When you boot from a kernel that has a sufficient RAM disk size and the ramlog service is enabled, during system boot your /var/log directory is copied to a RAM disk and the RAM disk is mounted at /var/log. The contents of the log directory on hard disk are still available through the /var/log.hdd path. This allows you to see both filesystems if you want to perform any manual operations. For example, if you want to delete a file, you should delete it from /var/log.hdd to avoid it being copied back to the RAM disk the next time you reboot. As shown below, the log.hdd directory is located on the LogVol LVM, and /var/log itself is on the ram9 disk.

# df -h  /var/log /var/log.hdd
Filesystem            Size  Used Avail Use% Mounted on
/dev/ram9              76M   52M   21M  72% /var/log
/dev/mapper/VolGroup00-LogVol00
                       16G   11G  3.8G  75% /

If you want to save the log files from your RAM disk to your hard disk, execute service ramlog restart as root.

It would be nice if ramlog were aware of logrotate and kept the logrotate files themselves on your hard disk, storing only softlinks in the RAM disk. This would allow a smaller RAM disk to be used for storing only the log files that are still active. Perhaps the developers will consider this enhancement for future versions.

Wrap up

Knowing that you have a fixed chunk of RAM that contains your log files means that your laptop’s hard disk doesn’t have to spin up regularly if something needs to be logged by an overly verbose daemon. The fixed-size RAM disk Ramlog uses will keep an overly verbose daemon from exhausting all your system RAM. If you are using a solid state disk on a laptop with a few gigabytes of RAM, Ramlog can save you many write cycles on your solid state disk by sacrificing 50-80MB of RAM.