Author: Mark McGrew
Other system administrators have fantastic toolboxes for their work. My tools consist of two everyday programs: OpenSSH and the GNU Bourne-Again Shell (bash). No other tool, whether console-based or GUI, has been so consistently useful to me as these two programs.
As a system administrator, I have used OpenSSH’s piping abilities more times than I can remember. The typical ssh call gets me access to systems for administration with a proven identity, but ssh is capable of so much more. In combination with bash’s subshell invocation, OpenSSH can distribute the heavy work, reduce trace interference on a system under test, and make other “impossible” tasks possible. I’ve even used it to make Microsoft Windows remote administration easier.
In the examples below, I have tried to avoid GNU-specific idioms for tools which have non-GNU counterparts. This practice improves portability of shell scripts in heterogeneous environments.
OpenSSH as a pipe
A pipe normally feeds the output of one program to the input of another. OpenSSH allows these two programs to run on different computers. For example:
$ ls -lR | ssh remotehost grep lost-file.txt
This (contrived) example shows the simplicity of connecting programs on a network, whether trusted or untrusted. OpenSSH even passes the exit code of the remote command back to the local shell.
Note two things here. First, the local shell interprets the command line before calling ssh. This means the usual quoting rules apply. In particular, any environment variables to be evaluated remotely must have their dollar signs escaped. If you forget to do this, the local shell will evaluate the variables, before invoking OpenSSH. This example shows the difference:
$ ssh localhost echo $PATH #escaped--evaluated remotely
Password:
/usr/bin:/bin:/usr/sbin:/sbin
$ ssh localhost echo $PATH #unescaped--evaluated locally
Password:
/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/3.4.4:/opt/
blackdown-jdk-1.4.2.02/bin:/opt/blackdown-jdk-1.4.2.02/jre/bin:/usr/qt/3/bin:/us
r/kde/3.4/bin:/usr/kde/3.3/bin:/usr/games/bin:/opt/insight/bin
This shows the other important point to remember. OpenSSH may not apply your login shell profile (such as ~/.bash_profile) or even the non-interactive shell profile (~/.bashrc), so your paths, aliases, and functions might not be available. The sshd manual page has more information under the LOGIN PROCESS heading.
Backup to remote storage
Here’s a problem my office faces regularly. Part of our quality testing involves backups and restores of large files, but the nature of the files hinders good compression, and many of our systems do not have sufficient space for uncompressed local archive storage. OpenSSH lets us use another system for temporary storage:
TMPFILE=${HOSTNAME}-${PID}
tar cf - dirname | ssh remotehost "cat > ${TMPFILE}.tar" # quoted for remote redirection
This creates the tar locally, then feeds the stream through OpenSSH into $TMPFILE on the remote host. The restoration is simply the reverse sequence:
ssh remotehost "cat ${TMPFILE}.tar" | tar xv
Backup with remote compression and storage
When compression is necessary (and feasible), workload distribution becomes more effective with OpenSSH. Just as distcc allows multiple machines to compile simultaneously, OpenSSH lets one system create the archive, while another system compresses it:
tar cf - dirname | ssh remotehost "gzip -c > ${TMPFILE}.tar.gz"
In this case, OpenSSH carries the uncompressed archive over the network. If you’re on a LAN slower than 100Mbps, you probably will not see a benefit from this. The workload on the local system does involve the OpenSSH encryption, but the compression workload is moved to the remote system.
OpenSSH may have compression enabled by the remote server, causing redundant compression workloads. To expedite the data flow through OpenSSH, try setting Compression to no in the remote system’s sshd_config.
To restore the archive, again, simply reverse the data flow:
ssh remotehost "zcat < ${TMPFILE}.tar.gz" | tar xv
Backup with local compression and remote storage
One of our systems has four CPUs, making it more efficient when creating compressed archives:
tar cf - dirname | gzip -c | ssh remotehost "cat > ${TMPFILE}.tar.gz"
This speeds up the network transfer even more, by compressing the data first. Restoration is equally simple; just run:
ssh remotehost cat ${TMPFILE}.tar.gz | gunzip -c | tar xv
Parenthesized subshells, too
A pipe sequence using OpenSSH can also use parenthesized subshells, which is to say, commands enclosed in parentheses are run in a subshell. Used judiciously, they can reduce complexity:
ssh remotehost zcat ${TMPFILE}.tar.gz | ( cd /other/path ; tar xv ) | wc -l
This restores the archive to /other/path, and the wc -l command shows us the number of files restored. While tar -C /other/path would work for Linux, other Unix systems may not use GNU tar. Using bash’s parentheses makes this command much more portable.
Reducing filesystem interference
One of our project leaders asked me to run a kernel call trace on a system with nearly full disks. I decided to record the trace on my desktop system instead of on the test system. According to the strace man page, the -o option allows a pipe to process the trace output, but ssh in this case required stdin/stdout for password entry. Using a named pipe on the filesystem left stdin, stdout, and stderr unchanged for the test program. I ran the following commands on the test system:
mkfifo /home/me/pipe
strace -f -o /home/me/pipe test-program & cat /home/me/pipe | ssh desktop "cat > trace-record"
The strace command went into the background, but waited until another program was ready to read from the FIFO. The tested program ran more slowly, but exceeding local disk space was not a concern.
Remote Windows administration
Finally, a trick that doesn’t rely on bash, except as my login shell. Using multiple SSH connections, with X forwarding, I can use Xvnc to translate the Windows desktop to something that works in my Linux-only home environment. This sequence connects to my desktop system from outside, then forwards the Windows desktop to an X window at home:
$ ssh -X -C termsrv #from home
Password:
$ ssh -X my-desktop #from termsrv
Password:
$ vncviewer windows #from office desktop
The -C compresses the forwarded X protocol, making Xvnc much more bearable through a DSL connection. The VNC server on the Windows system is configured to run as a service, so that I can connect first, then log in as the Administrator for whatever purpose I need at the moment.
I have used many tools in both CLI and GUI to install, use, and administer both operating systems and programs. Nothing has matched the usefulness and flexibility of OpenSSH and bash together. With them, I can do much more than log in to a remote system securely. The examples shown here are a small sample of the possibilities.
Mark McGrew is a small-office system administrator in Silicon Valley.
