Author: Mikhail Zotov
Fwlogwatch
Fwlogwatch is a packet filter and firewall log analyzer originally written by Boris Wesslowski for the Computer Emergency Response Team at the University of Stuttgart, RUS-CERT.
Fwlogwatch can prepare reports like this one, obtained from a real log written by a router of a small LAN:
67 packets from 84.104.251.207 (84-104-251-207.cable.quicknet.nl) 62 packets from 84.16.224.42 (altan.waptr.com [forward lookup failed]) 28 packets from 80.11.33.2 (AClermont-Ferrand-108-1-17-2.w80-11.abo.wanadoo.fr) 26 packets from 213.136.16.150 (213-136-16-150.adsl.bit.nl)
Here, fwlogwatch was told to find the top four offenders, to perform DNS lookups, and not to differentiate destination addresses. The command I used for this is fwlogwatch -M4 -n -D /path/to/the/logfile.
A more detailed report can differentiate destination ports using the -d option, and perform service lookups using the -N option:
62 tcp packets from 84.16.224.42 (altan.waptr.com [forward lookup failed]) port 25 (smtp) 57 tcp packets from 84.104.251.207 (84-104-251-207.cable.quicknet.nl) port 25611 (-) 26 tcp packets from 213.136.16.150 (213-136-16-150.adsl.bit.nl) port 25 (smtp) 12 tcp packets from 66.214.5.192 (66-214-5-192.dhcp.lnbh.ca.charter.com) port 3440 (-)
The second report differs from the first one because some of the hosts stopped by the firewall used different destination ports to perform probes.
Fwlogwatch supports many other options. It can be controlled by options given on the command line, and by settings in a configuration file. It can run as a cron job to produce summary reports, and as a daemon doing realtime log monitoring and anomaly reporting. The fwlogwatch tarball also includes two user-contributed bash scripts that can generate abuse reports and modify ipchains or iptables rules in response to an attack.
Besides iptables’ logs, fwlogwatch can analyze logs produced by many other firewall programs and by Snort. The man page, the README file, and the configuration file provide all the information you need to use fwlogwatch effectively. I’ve covered a number of ways one can use fwlogwatch with iptables’ logs in this guide.
Junkview
Fwlogwatch is great, but it does have some limitations. It doesn’t provide graphical or pseudo-graphical output, and it can’t group IP addresses that originate from the same subnet. This gap is filled by junkview, an awk script written by Grant Coady.
Junkview performs an analysis of an iptables firewall log, and results are presented in a kind of a histogram using ASCII art. Applied to the same firewall log as the example shown above, with essentially the same options, junkview produced the following report:
junkview 2006-02-28b
21/tcp 14 |((( . . . . 1.5
22/tcp 52 |((((((((((( . . . 5.6
23/tcp 41 |(((((((((. . . . 4.4
25/tcp 167 |(((((((((((((((((((((((((((((((((((( . 17.9
80/tcp 32 |((((((( . . . . 3.4
106/tcp 15 |((( . . . . 1.6
443/tcp 17 |(((( . . . . 1.8
445/tcp 31 |((((((( . . . . 3.3
1080/tcp 12 |((( . . . . 1.3
3440/tcp 12 |((( . . . . 1.3
4745/tcp 11 |(( . . . . 1.2
10000/tcp 17 |(((( . . . . 1.8
25611/tcp 57 |(((((((((((( . . . 6.1
25611/udp 10 |(( . . . . 1.1
33437/udp 27 |(((((( . . . . 2.9
33438/udp 15 |((( . . . . 1.6
others 404 |((((((((((((((((((((((((((((((((((((((((! 43.3
total 934 + - - - - + - - - - + - - - - + - - - - + - - -
0 5.0% 10.0% 15.0% 20.0%
Scanned 934 records, examined 934 to find 934 events from router.
Options hours:0 prefix:Dropped bars:16 hits:3 showhits:4 noclient:1
nohost:0 stophost:0 nohostdport:0 dports:5 nonetadd:0 no_ip2c:0.
Reading /usr/local/share/junkview/ip2country, 71868 records.
Top 4 offenders by host, dports:
hits code host name/dports
67 NL 84.104.251.207 84-104-251-207.cable.quicknet.nl
42 25611/tcp-s, 15 25611/tcp-f, 10 25611/udp
62 DE 84.16.224.42 altan.waptr.com
62 25/tcp-s
28 FR 80.11.33.2 ~Clermont-Ferrand-108-1-17-2.w80-11.abo.wanadoo.fr
9 4206/tcp-s, 9 4203/tcp-s, 9 4198/tcp-s
1 4209/tcp-s
26 NL 213.136.16.150 213-136-16-150.adsl.bit.nl
26 25/tcp-s
Top 4 offenders by net addr, host, dports:
hits code net/host lookup/dports
67 NL 84.104.251.207/32 84.104.0.0/14
42 25611/tcp-s, 15 25611/tcp-f, 10 25611/udp
62 DE 84.16.224.42/32 84.16.224.0/19
62 25/tcp-s
52 FR 80.11.33.0/24 80.8.0.0/13
28 80.11.33.2 9 4206/tcp-s, 9 4203/tcp-s, 9 4198/tcp-s
1 4209/tcp-s
9 80.11.33.29 9 1182/tcp-s
9 80.11.33.156 9 3632/tcp-s
6 80.11.33.245 6 1801/tcp-s
26 NL 213.136.16.150/32 213.136.0.0/19
26 25/tcp-s
key: s syn, a ack, f fin, r rst, p psh
Notice that junkview found not only the top four offending hosts, but also the top four offending networks. This comes in handy when your machine or LAN is probed from a number of hosts in the same network and each of the hosts performs just a few or even a single probe.
Junkview finds net addresses and the countries of origin of the offenders without performing whois queries. Rather than using whois, junkview uses the IpToCountry database.
The project includes a number of other tools that can be useful for a network or system administrator. Check the junkshow page to find out more about it.
sXid
sXid is a suid/sgid monitoring program written by Ben Collins and released under the GPL. Its primary purpose is to track any changes in suid/sgid files and directories. If it detects new suid/sgid files, files that aren’t set sgid/suid anymore, or files and directories that have changed bits or other modes, then sXid reports the changes via email or on the command line, depending on how it was run. sXid can inform you about the results of its operation even if it hasn’t found any changes.
sXid also can be used as an integrity checker for any file in the system. To use the program this way, put a list of files to be monitored in a plain text file defined as EXTRA_LIST in sxid.conf. An administrator will be notified if any of the listed files is changed. For example, the following report reveals that the MD5 sum (m in the first position), the inode (i), and access options of lilo.conf have changed:
Checking for changed attributes or sums/inodes: mi /etc/lilo.conf root:root 644->640
The sXid database, actually a log file, is created automatically the first time you run the program and is updated every time sXid finds a change in the attributes of files listed in the database.
The project has a clearly written manual page with a number of examples of its usage and configuration.
Recovery is possible
Finally, a few words on recovery tools are in order. Anyone administering a Linux machine has probably faced a situation when it was necessary to boot from media other than the hard drive. Perhaps you installed Linux on your colleague’s machine but forgot the root password after a few days, or you installed a new kernel but didn’t run lilo before rebooting the machine, or severe problems with the root partition were detected during boot.
In these situations a live CD or a bootable USB flash distro can help. The first installation CD of Slackware Linux is also a busybox-based live CD, and I use it in situations like those mentioned above.
But sometimes, one needs more. For example, you may want to resize partitions on a hard drive, or perform a network test while the hard drive can’t be used. I have found a Slackware-based project called Recovery Is Possible (RIP) by Kent Robotti to be the perfect tool in those situations. In addition to “standard” programs for maintaining different types of file systems, it includes parted and partimage and a number of programs to deal with NTFS: mkntfs, ntfsresize, ntfsclone, and ntfsmount.
Besides these, RIP includes htop, which is a nice replacement for the traditional top utility, and a number of programs to use in networking, such as Lynx, Mutt, Fetchmail, NcFTP, and wget, and even a program to view information and change user passwords in a Windows NT/XP SAM user database file.
RIP is only around 30MB in size, and thus can be downloaded even over a slow connection. In spite of its small size, RIP includes some useful apps, such as ccal, a “colorized” version of cal that can be configured to work as an organizer. A complete list of programs included in the latest version of RIP is in the contents.txt file.
You can find a number of other interesting RIP-based projects, including a FreeBSD version of RIP, at the project Web site. Overall, I have found RIP to be perfectly polished. I think it’s a tool Linux and perhaps experienced Windows users should always have on hand.
Let us know about your most valuable utilities and how you use them. There need not be 10 of them, nor do they need to be in order. If we publish your work, we’ll pay you $100.
