Author: Vassil Dichev
OpenSSH
OpenSSH is my all-time favorite set of tools. I often use it for port forwarding to networks I don’t otherwise have direct access to. You can also use SSH as a SOCKS proxy using dynamic port forwarding, like this:
ssh -D 8080 remotehost
I can then set a SOCKS proxy in my browser to localhost, port 8080, and voilà — my browser can access everything that remotehost can.
Socat
Socat has been described as “netcat on steroids.” It can read from and write to files, pipes, and all kinds of sockets. I use it mostly for port forwarding (TCP relay). For example, to forward all incoming connections on port 1234 of my host to port 1234 of destination.host, I run:
socat TCP4-LISTEN:1234 TCP4:destination.host:1234
I also use it as a replacement for Stunnel to tunnel connections through SSL. This is, for example, how I sometimes make an HTTP-only server support HTTPS in a pinch:
socat OPENSSL-LISTEN:443 TCP4:localhost:80
I even use Socat to establish a TCP connection through an HTTP proxy in a restrictive firewalled environment, by setting ProxyCommand in .ssh/config:
ProxyCommand "socat - PROXY:proxy:%h:%p"
In order for this to work, SSH servers outside the firewall must listen on port 443, which is often the only port allowed for the HTTP CONNECT method.
Bash
Most Linux users use GNU Bash, but few know all of its shortcuts. The most time-saving ones for me are Ctrl-R for incrementally searching through the command history, and Meta-. to iterate over the last argument of commands in the history. I also like cd - for returning to the previous directory, since I always forget to use pushd.
Bash can do many different types of expansion. For instance, to generate a list of IP addresses on a subnet, you can type:
echo 192.168.{0..4}.{0..255} > ips.txt
Midnight Commander
Midnight Commander is a real time-saver. Its most useful feature is support for virtual filesystems, which means you can transparently operate with files on servers through SSH or FTP, or in tar archives.
Filesystems can also be nested. I take advantage of this feature when I connect to my server via SSH and copy a file to an archive on a server via FTP. I sometimes use Midnight Commander to view all packages on the system I’m on by doing a quick cd to #apt, #dpkg, or #rpms. The former two show package descriptions on Debian systems, while the latter shows installed RPMs on RPM-based systems, all grouped in directories by section.
Midnight Commander’s built-in editor is easy to use, but I prefer Vim’s power, so I associate editing files with an external editor. I usually use Midnight Commander’s subshell and occasionally switch to and from it using Ctrl-O.
Aptitude
The Aptitude package management tool can automatically resolve dependencies from several Debian repository sources, so downloading and installation of dependencies is nearly seamless. Aptitude remembers which packages have been selected explicitly for installation and which have been installed as dependencies, so that when dependencies are orphaned, Aptitude suggests removing them.
I often use regular expression searches, for instance to show all packages containing the word “relay” after the string “tcp” in package descriptions:
aptitude search ~dtcp.*relay
You can also use regular expressions to print all packages in section net that do not start with “lib”:
aptitude search !^lib~snet
Another useful search is to display all installed packages that depend on iptables:
aptitude search ~i~Diptables
Knoppix
Carrying an operating system with me at all times is bliss. I rarely leave home without a Knoppix CD. Knoppix has always been great, but it got even better with the inclusion of UnionFS, which allows me to install packages as if the CD were writable.
I have a notebook from work with a single NTFS partition, and I’m not allowed to repartition it. When I need to use Linux in my work, I load Knoppix from an ISO stored on the C: drive by using the following boot-time parameter:
knoppix bootfrom=/dev/hda1/Install/iso/KN*.iso
I need to retain my settings, so I use an encrypted C:KNOPPIX.IMG image file. Knoppix automatically detects it and mounts it as a UnionFS partition, which means it can be written to. Currently, only loopback devices are safe for writing in Linux, so I use the Windows utility MkImage-ct.exe, included with Knoppix, to create this image on NTFS.
I also find that Auditor, a Knoppix-derived live CD, is invaluable for security auditing. I often use it for recovering forgotten Windows passwords by extracting the SAM database and then using John the Ripper, a password cracker. I also love to demonstrate how weak WEP keys are using aircrack.
awk
Awk is a favorite text-processing tool of mine. I usually use the GNU project awk clone, gawk. It’s always useful for use with text files or streams that have a predictable structure or fields. For instance, I use gawk to compare Debian’s popularity-contest list to my local package list to find out how others rate the packages I have installed:
zcat by_vote.gz | awk 'BEGIN { while ( getline pkg < "my_packages") pkgs[pkg] }; !/^#/ && $2 in pkgs {print $1, $2}'
Expect
Having this automation tool in my arsenal may not be what most people, ahem, expect. With Expect, I can automate interactive programs without having to know dozens of Perl modules in detail. With Expect, I can multiply the power of automation in distributed shells such as Distributed Shell (DSH) and Fanout.
I found recently that I only had Telnet access to certain hosts, but I needed to copy binary files to those hosts. I used the script ftp-inband, which is bundled with Expect. When I press ~~g I am prompted for the file name to transfer to my local machine, and then the files are promptly uuencoded and transferred through Telnet’s text-only connection.
Scite
Both Vim and Emacs are capable editors, and I use jEdit too, but if you need a light but powerful graphical editor for programming, try Scite.
I really like Scite’s features for folding text and word completion, among others, and the editor supports regular expressions. You must configure Scite using “property” files, but when you think about it, it makes sense — what’s more natural for a text editor than editing text?
Ipcalc
Ipcalc is a small but useful utility for calculating the broadcast and IP range, given an IP address and netmask. You can do it yourself, but why not save time and effort? Ipcalc generates colored output at the console or in HTML. Here’s a small sample of Ipcalc’s output, after running ipcalc 192.168.1.112/26:
| Address: | 192.168.1.112 | 11000000.10101000.00000001.01 110000 |
| Netmask: | 255.255.255.192 = 26 | 11111111.11111111.11111111.11 000000 |
| Wildcard: | 0.0.0.63 | 00000000.00000000.00000000.00 111111 |
| => | ||
| Network: | 192.168.1.64/26 | 11000000.10101000.00000001.01 000000 |
| HostMin: | 192.168.1.65 | 11000000.10101000.00000001.01 000001 |
| HostMax: | 192.168.1.126 | 11000000.10101000.00000001.01 111110 |
| Broadcast: | 192.168.1.127 | 11000000.10101000.00000001.01 111111 |
| Hosts/Net: | 62 | Class C, Private Internet |
Let us know about your most valuable utilities and how you use them. There need not be 10 of them, nor do they need to be in order, and if we publish your work, we’ll pay you $100.
