I have this book — a Spanish edition of Stephen Coffin’s seminal manual, Unix System V Release 4: The Complete Reference. You can open it on any of its 700+ pages and bet your bottom dollar that the commands on the page will work in a modern-day Linux. Well, except where teletypes and tape storage are involved.
Said like that, you may think the *NIX command line hasn’t changed a lot since the early 1990s. This is not entirely true.
Take for instance the moreutils collection. You can install it on most distros with your regular package manager. On Ubuntu and Ubuntu-based distros, do
sudo apt install moreutils
On Debian, the following will do the trick:
su
apt-get install moreutils
On Fedora, you can do:
yum install moreutils
OpenSUSE requires one more step of adding a specific repository, or you could simply visit openSUSE’s online package search and use the 1 Click Install service.
Shiny new tools
Moreutils provides you with a set of new tools that are not part of the standard Linux set but probably should be. For example, vidir provides an editor interface for modifying files and directories. Despite its name, vidir uses the default command-line editor, so if you have vi(m), sure, it will show a vim interface (Figure 1); but if you have nano or emacs configured, it’ll show nano or emacs interface (Figure 2).
You can edit whole directories with:
vidir directoryname
Or just a subset of files with, for example:
find Pictures -iname "*.png" | vidir -
Notice the “-“. This tells vidir to take its input from a pipe.
You use your regular key combinations to modify your directories and files. If you’re using a vi-like interface, press i to modify directory and file names; press d[number]d to delete files or directories — note that vidir has a security feature built in, which won’t let you erase non-empty directories; press u to undo changes, and so on.
Soaking it all up: Sponge
According to its man page, sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before writing the output file.
Now, that’s useful. To appreciate how much, try this: Create a plain-text file containing the names of the knights of the round table, or the days of the week, or whatever. Basically any file with a list of items *not* in alphabetical order:
Arthur
Lancelot
Gawain
Geraint
Percival
Bors the Younger
Lamorak
Kay
Gareth
Bedivere
Gaheris
Galahad
Tristan
Save it as knights.txt.
If you wanted to sort the names alphabetically, what would you do? You’d probably try something like this:
sort knights.txt > knights_sorted.txt; rm knights.txt; mv knights_sorted.txt knights.txt
Because, if you did this:
sort knights.txt > knights.txt
you would start overwriting the file before you had finished reading from it, ruining it in the process. So, you need to mess around with that intermediate file, knights_sorted.txt. And you still have that original, unsorted file hanging around, too, which you now have to erase before renaming the sorted file, hence the long unwieldy chain of instructions. With sponge, however, you can do this:
sort knights.txt | sponge knight.txt
Check it out: no intermediate file!
cat knights
Arthur
Bedivere
Bors the Younger
Gaheris
Galahad
Gareth
Gawain
Geraint
Kay
Lamorak
Lancelot
Percival
Tristan
Thanks to sponge, you can grab the content from a text file, do all the chained processes on it using things like sort, unique, sed, grep, and tr. Sponge will soak it all up, wait until all the lines have been processed, and then wring it all out to the same file, all in one blast.
Talking of soaking, let’s discuss pee. Despite its name, pee has *nothing* to do with bodily fluids. In fact, the name is a combination of pipe (as in the way you pass the output from one command onto another in *NIX systems) and tee (as in the tee *NIX command line instruction).
While tee re-routes the output from a command to files (cat knights.txt | tee k1 k2 k3 creates files k1, k2, and k3 containing the content cat‘ed from knights.txt), pee pipes the output into a list of commands:
cat knights.txt | pee sort "wc -l" "grep - -e ^G.*"
Arthur
Bedivere
Bors the Younger
Gaheris
Galahad
Gareth
Gawain
Geraint
Kay
Lamorak
Lancelot
Percival
Tristan
13
Gawain
Geraint
Gareth
Gaheris
Galahad
In the example above, using the output from the original and unordered knights.txt, you pipe it first to sort to get an ordered list; then to wc (word count), which, with the -l option, returns the number of lines (13); and finally to grep, which then uses a simple regular expression pattern to print out only the lines that start with a capital “G“.
Pipe dream
Getting back to editors but staying with pipes that push stuff hither and thither, you must try vipe. Again, this is a twist on your default shell editor. Plunk it into your chain of piped commands and it will open up with the output from the prior instructions. For example:
cat knights.txt | pee sort "wc -l" "grep - -e ^G.*" | vipe | unique
will show all the outputs we saw in the previous example in a (in my case) vi-like editor. You can now edit the output to your heart’s content, removing, adding, and modifying lines. When you’re done, save and quit, and your edited output will be passed on to the next command in the chain.
Pretty cool, no?
Moreutils has more…
Moreutils comes with many more goodies. The combine utility merges the lines from two files using Boolean operations; ts adds a handy, human readable timestamp to each line taken from an input; ifdata makes extracting data from a network interface supereasy — very useful for scripting, and so on. Check out the project’s web page and the man pages for each command to see how they all work.
Although it is true you could emulate many of the behaviors of these new commands with a bit of command-line fu, tools like sponge, pee, and vipe just make working on the shell easier and more of a pleasure.
The Moreutils package is always evolving, and new stuff gets added from time to time. I, for one, am excited to see what gets included next.
Advance your career in Linux System Administration. Check out the Essentials of System Administration course from The Linux Foundation.
