Author: Brian Warshawsky
There are several filesystem integrity checker applications, both commercial and open source. I chose to deploy afick, because it is written in Perl, which makes it lightweight and easily portable between different operating systems. Though by nature designed for the command line, afick also has an optional Webmin module and a graphical interface written in perl-Tk.
For this article we will focus on the command-line implementation on a SUSE Enterprise 8.0 server, but what you see here should be applicable to just about any *nix distribution.
Installation
You can either download and build afick from the source code, or, if you’re using a package-based distribution, you can install from an RPM or Debian binary packages. Let’s walk through building it from source. To begin, download and unpack the latest version, navigate to the afick directory, and run the following command:
# perl Makefile.pl
Since you aren’t installing the GUI, you can safely ignore any errors about missing perl-Tk modules.
Next, type make install to install the console tool. When your machine finishes copying all the necessary files, afick will be installed on your system. Installation is only half the battle, though. The real fun lies in the configuration and testing phase.
Configuration
Afick begins by creating a database of values representing the current state of your filesystem. When you run it later in update mode, afick compares the current filesystem to the original and notes changes.
The exact attributes of your filesystem that are checked are controlled by a configuration file. Afick provides a default configuration file for both Linux and Windows in the directory where it was originally unpacked. In our case, we are interested in the file linux.conf. You can modify a wealth of options in this file, but we will focus only on the essentials.
Afick provides multiple ways to check every file and directory on your system, so no one configuration file is going to work for everyone. For instance, I am running PHPNuke on a Web server that includes forums, which are going to change constantly as users post items and change their preferences. I don’t want those changes dumped into my mailbox every day, possibly burying something important. Someone else with a static Web site, however, might want to see that that content is never changed unexpectedly, and would therefore closely monitor that directory. It takes a bit of trial and error to fine-tune afick (or any other integrity checker for that matter) for your specific needs.
Let’s begin by initializing the database with the command:
# afick -c /[path_to_linux.conf]/linux.conf -i
The -c tells afick where to find the configuration file it should use, while the -i instructs it to make an initial copy of the filesystem database. This process will take a few minutes. Once it has finished, let your server run for a while, then compare the databases with this slightly different command:
# afick -c /[path_to_linux.conf]/linux.conf -k
The -k argument tells afick to compare the current filesystem against the database specified in the first line of the linux.conf file, which is the initial database. Any changes will be noted on stdout.
Repeat this process a few times until you’re getting a feel for what is changing and how so. For instance, if you’ve got busy log files somewhere outside of /var, they might produce a bundle of changes every time you run afick, which will create white noise around potentially useful data. After you’ve got a list of repeat offenders, you can tune the linux.conf file.
The linux.conf file actually has a decent description of all the file attributes you can monitor, including device, inode, permissions, owner, last time modified, and several others. You can even create your own rule sets for certain types of files and directories. For instance, you don’t want afick reporting warnings about the individual files in /var/log being modified, as these files are going to be modified almost constantly on some systems. To create a rule set that would check the user and group ownership, the device they reside on, and the permissions, you would first add:
#alias
specialrule = u+g+d+p
Then, to apply this custom rule to the /var/log directory, you would add the following line to the =/Dir section of the conf file:
/var/log specialrule
If you want to define a rule set that ignores the /tmp directory, checks only the files ending in .backup in /root, and ignores all files in /home/user that end in .old, you would add the following lines to the alias section of the config file.
!/tmp
/root/*.backup special_rule
exclude_re := /home/user/.*.old
Afick recognizes the standard Unix wild cards and regular expressions in rule sets. With a little bit of tweaking, you can tune afick to completely monitor your filesystem in all the necessary places, while ignoring the spots that would generate useless noise.
After you’ve spent some time tweaking your configuration file you need to ensure that afick itself cannot be modified. The most secure way to accomplish this is to put the database, found by default at /var/lib/afick/afick.pag, somewhere that is write-only. Unfortunately a diskette isn’t an option because of the size of the afick database; my database is roughly 15MB for a 4GB server. I recommend using an Iomega Zip disk for a couple reasons. Primarily, you can switch a Zip disk from read-only to read-write with the flip of a switch. This is convenient because every time you make a change to your filesystem you’ll need to update the database to clear the warnings that afick will produce every time it runs thereafter, which could lead to wasting a lot of CDs if you tried to deploy the database on a CD-R.
No matter where you store your database, you still need to tell the configuration file where to find it. Mount your Zip drive (or your CD) and copy /var/lib/afick/afick.pag to your mount point. Then change the entry for the database location in the first section of the config file to represent the path to your removable media.
database=/mnt/zip/afick
In addition to storing the database on read-only media, I also choose to err on the side of paranoia, and I keep my config file on read-only media as well. In this case, it’s a diskette that I’ve write-protected. This doesn’t prevent afick from being run with a separate config file created locally on the server, but it does allow me to be sure that no small detail within the file can be changed without someone physically touching my servers.
Automating it
The final step in configuring afick is automating it. (Note that this is not strictly necessary if you wish to run afick only after certain tasks.) The easiest way to automate afick is with the afick.cron script included in the original directory where you unpacked the source code. If you installed via an RPM, then afick.cron was implemented upon your installation, and should be emailing root as changes occur. If you followed the instructions here and installed from source, you have to add it manually. At your command prompt, just type:
crontab -e
0*/2 * * * root /[path_to_afick.cron]/afick.cron
Then save and exit the editor. This tells your operating system to run afick once every two hours and mail the results, if any, to root.
Updating it
Every time you make a change to a filesystem, you’ll want to update the afick database. Unfortunately, this is a point where security can begin to become an inconvenience. You must physically be at the server to change the Zip disk to read-write mode so the database can be updated.
To initiate update mode, simply replace the -k in the afick command line with -u. This will update the database to include any changes that have happened since the last time afick was run. Since afick will continue using the new database to ensure file integrity, you should always run afick once in compare mode before updating it, to be sure that the database you are about to create won’t report recently compromised system commands or files as legitimate. As a further level of protection, afick has built-in integrity checks it performs on its own executables to ensure that afick itself hasn’t been modified.
On a short side note, you can also use the update feature of afick to monitor exactly what changes a program’s installation procedure makes to a filesystem by using the update feature immediately before and immediately after the installation. This is extremely useful in situations where you cannot verify the authenticity of an application before you install it. Just make sure when you test it with afick that you don’t do so on a mission-critical machine. You can also use afick for retroactive testing to ensure uninstalling the software actually returns your filesystem to its previous state.
The future of afick
Afick is a work in progress. In recent conversations, developer Eric Gerbier said he intends to include in future releases a daemon-enabled version that doesn’t rely on cron to run afick, thus delivering real-time filesystem monitoring. An option to export afick’s results in HTML/XML is in the works for version 3.0, due out sometime in the next few months.
No system will ever be completely safe from malicious users and unauthorized access. If a machine under your control becomes compromised, you must have the proper precautions in place to quickly mitigate the damage and restore services. With a file integrity checker such as afick in place, when that dreaded day comes, you will be prepared to determine exactly what has happened (or is happening) and react accordingly.
In order for afick, or any other file integrity checker, to work as needed, you’ll need to take special care in observing the general actions and changes in your filesystem in order to correctly and efficiently craft your configuration rules. Once you’ve done that, it’s just a matter of staying on top of the ever-changing filesystem. Update regularly and update often, so as to catch problems as they begin, and not when it’s too late. If you make these practices a regular part of your daily administration routines, you’ll be prepared to react efficiently to a breach in security should the need arise.
Brian Warshawsky has built, supported, and administered mission-critical IT infastructure for the United States Naval Research Laboratory, Virginia Commonwealth University, RichmondAir Wireless, and is currently employed by Sungard Collegis at Virginia State University.
