The Tenth Commandment of system administration

If you’re a system administrator, eventually you’re going to need to write a shell script. If you’re like me and you enjoy scripting, you’ll find reasons to write shell scripts for just about everything.

X. Thou shalt not waste time doing repetitive and mundane tasks

hell scripts are nothing more than a semi-sequential list of commands for your shell to run to automate a process that might take you days or weeks to do. This can include sorting syslog entries on a remote logging server, polling your servers for newly available network services, automating your backup policies, creating user accounts, and any other number of tasks. These tasks can be accomplished with any number of different shells, but by far the most common is bash, so that’s what we’ll focus on here.

I have always believed the best way to learn is by example, so we’ll look at an actual shell script to explain some of the fundamentals. In the following example, we’re going to take a look at a script I wrote to facilitate packet captures within a a network for visual playback in a program called EtherApe. Don’t worry if you don’t understand everything in it right away. I’ll explain it all in due time.

Except for the first line, a pound sign within a script denotes a comment, and anything
following on that line is ignored when the script is parsed.

#!/bin/bash

# This is called the 'shebang' and all your scripts should start with #it.
# It tells the operating system where to find the shell to interpret
# your commands.

# The following line is a denotes a function, which is a set of code
# that you can write once, but call as many times as you'd like during
# the execution of your script.
kapture ()
{
name=`date +%T`
tethereal -i eth0 -a duration:900 -n -w/kapture/current/$name.kap
}

cleanup ()
{
find/kapture/archive -mtime +7 -exec rm {};
}

archive ()
{
archive=`date +%d.%m.%y`
cd/kapture
tar --create --file=./archive/$archive.tar./current/*
rm/kapture/current/*.kap
}
# The Kapture function uses a CLI based version of ethereal to capture
# packets and stores them into a time stamped file.  The cleanup
# function is used to go through once a day and remove any file modified
# more than 7 days ago.  The archive function is used to collect 24 hours
# worth of data into a single tar file for easier storage.

kapturecount=0
a=1
b=2
# Above I've declared variables for use in my upcoming loop.  Note that
# a and b are used only as a cheap method of making sure that the loop
# never ends.

while [ "$b" != "$a" ]    # 1 will never equal 2, so the loop continues
do
kapture  # here we're calling the function we declared at the beginning.
((kapturecount++))  # This is called an incremental, or a counter.
#It's what keeps track of how many times the loop has run.
# Below are if loops to check the value of the counter, and act
# accordingly.  Because each kapture runs for 15 minutes, this works
# out to about every 24 hours.
if [ "$kapturecount" == 95 ]
then
	archive
fi
if [ "$kapturecount" == 96]
then
	cleanup
	$kapturecount=0   # reset the loop count and start again.
fi
done

There is more to this script, in the form of playing with the.kap files that it creates, but we’ll leave that for now. Notice that functions are always created and declared before they are run within the script. Basically, we’ve just told the computer to remember each of those sets of commands for later use. You cannot call a function before you’ve declared it, or it will not work.

The loops are important as well. These can prove useful for determining the direction the script takes depending on the outcome of defined tests, as illustrated above. They can also perform a variety of actions on lists of files, files in a directory, or any number of other options. A basic understanding to loops is essential for effective shell scripting.

This shell script isn’t perfect, and it isn’t the only way to accomplish the task at hand. Indeed, there are probably more powerful commercial tools available that would do a much better job of traffic analysis. However, I developed this in a hurry to troubleshoot a problem, and as a result, it meets my specific needs better than any commercial application could.

If you’re interested in learning more about shell scripting, I recommend you pick up Ken Burtch’s book “Linux Shell Scripting with Bash.” It is an excellent guide for explaining the intricacies of bash shell scripting, and provides many fine examples of solutions to various problems.

Conclusion

We’ve made it to the end. I hope you’ve been able to pick up a few helpful tidbits of information to facilitate your role as a system or network administrator. I know over the years these 10 basic rules have served me well, and I sincerely hope they can do the same for you. It is important to never forget the fundamentals of your profession, as it is often easy to get carried away and overwhelmed by the details that make up our careers. When something like that happens, it becomes easy to forget the reasons why we all started this work in the first place — likely, because of an underlying love for technology and the things we can make it do. Whether you’re designing a backup system, tracking log files, monitoring your network, or any of the other tasks, routine or not, that you tackle every day, you should always enjoy what you do.

The 10 commandments:
I. Thou shalt make regular and complete backups
II. Thou shalt establish absolute trust in thy servers
III. Thou shalt be the first to know when something goes down
IV. Thou shalt keep server logs on everything
V. Thou shalt document complete and effective policies and procedures
VI. Thou shalt know what cable goes where
VII. Thou shalt use encryption for insecure services
VIII. Thou shalt not lose system logs when a server dies
IX. Thou shalt know the openings into your servers
X. Thou shalt not waste time doing repetitive and mundane tasks