Command line automation with Expect-lite

Expect is a venerable tool for scripting interactive command-line tools. One normally sees expect coupled with the TCL programming language — for example, in the DejaGNU test environment. Expect-lite is a wrapper for expect designed to allow you to capture an interactive session more directly mapped into a script. The expect-lite language also includes simple conditionals and other programming language elements, and can be readily mixed with bash programming.

Expect-lite is itself an expect script, so to use it you will need to install expect from your distribution’s package repository. Once you have it installed, expand the expect-lite tarball and copy the expect-lite.proj/expect-lite file to somewhere in your $PATH and give it appropriate permissions.

# tar xzvf /.../expect-lite_3.0.5.tar.gz 
# cp expect-lite.proj/expect-lite /usr/local/bin
# chown root.root /usr/local/bin/expect-lite
# chmod 555 /usr/local/bin/expect-lite

The main goal of expect and expect-lite is to automate an interactive session with a command. I’ll refer to the command that expect(-lite) is talking to as the spawned command. A basic expect(-lite) script might spawn a command, wait for the command to ask for some input, and send some data in response to that request.

In expect, the send command sends information to the spawned command, and the expect command is used to wait for the spawned command to send a particular piece of information. Normally there is a timeout set to handle the case where the spawned command does not provide the expected output within a given time; in other words, when the spawned program does not behave as planned by the expect(-lite) script.

I’ll use gdb as an example to demonstrate the difference between using expect and expect-lite to automate a simple interaction. Shown below is the code for a trivial C++ application and the start of a normal gdb debug session for this application.

$ cat main.cpp 
#include 

using namespace std;

int main( int, char** )
{
        int x = 2;
        x *= 7;
        cout 
The expect program below will automate the execution of the above program with gdb printing out the value of x at a point in the program execution.

$ cat ./gdb-main-expect
#!/usr/bin/expect

spawn gdb ./main;
send "br mainn";
expect "Breakpoint 1 at";
send "rn";
expect "Breakpoint 1, main";
expect "(gdb)";
send "stepn";
expect "(gdb)";
send "stepn";
expect "(gdb)";
send "print xn";
send "quitn";
expect "Exit anyway";
send "yn";

$ ./gdb-main-expect
spawn gdb ./main
br main
GNU gdb Red Hat Linux (6.6-35.fc8rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...
Using host libthread_db library "/lib64/libthread_db.so.1".
(gdb) br main
Breakpoint 1 at 0x400843: file main.cpp, line 7.
(gdb) r
...
Breakpoint 1, main () at main.cpp:7
7               int x = 2;
(gdb) step
8               x *= 7;
(gdb) step
9               cout 
The same automated interaction is expressed in expect-lite below. Notice that the quoting and other syntax is now missing and the expect-lite program is much closer to what a user would actually see and type. Lines starting with >> are things that expect-lite will send to the spawned command. Lines starting with < are things that expect-lite will wait to see in the output of the spawned command.

$ cat gdb-main-expect-lite.elt
> gdb /home/ben/expect-lite-examples/main
>>br main
>r
>step
>step
>print x
>>quit
>y

Expect-lite was created primarily for running automated software tests. A side effect of this heritage is that any script execution requires the specification of the host where the test is to be executed. In the example below I have set up SSH to allow me to log in to the same user on localhost without a password, as detailed after the example. There should be no security implications with this. At the end of the expect-lite interaction you can see an overall result as passed, which is also due to expect-lite's primary purpose being software testing.

$ expect-lite remote_host=localhost cmd_file=gdb-main-expect-lite.elt
spawn ssh localhost

Last login: ... 2008 from localhost.localdomain
$ bash
->Check Colour Prompt Timed Out!
$  gdb /home/ben/expect-lite-examples/main
br main
...
(gdb) br main
Breakpoint 1 at 0x400843: file main.cpp, line 7.
(gdb) r
Breakpoint 1, main () at main.cpp:7
7               int x = 2;
(gdb) step
8               x *= 7;
(gdb) step
9               cout 
The below commands set up SSH to allow you to connect to the localhost without a passphrase.  In this case the "elo" hostname is also available as a shortcut to connect using the correct SSH identity file. In the above session I could have used expect-lite remote_host=elo cmd_file=... to execute the command on localhost.

$ mkdir -p  ~/.ssh
$ chmod 700 ~/.ssh
$ cd ~/.ssh
$ ssh-keygen -f expect-lite-key
$ cat expect-lite-key.pub >> authorized_keys2
$ vi ~/.ssh/config
Host localhost
  IdentityFile ~/.ssh/expect-lite-key

Host elo
  HostName localhost
  IdentityFile ~/.ssh/expect-lite-key
$ chmod 600 config
$ chmod 600 authorized_keys2
$ ssh elo

For the gdb example I could not use the more normal > to describe what expect-lite should send, because the > expect-lite command has some built-in logic to first detect a prompt from the spawned command. >> on the other hand just sends what you have specified right now. The prompt detection logic in expect-lite did not detect the gdb prompt in the gdb-main-expect-lite.elt example above and so would time out waiting for a prompt from the spawned command.
Expect-lite did not detect the gdb prompt because the regular expressions that it uses in wait_for_prompt did not handle that prompt format. One way to change that situation would be to customize those regular expressions in the expect-lite file if you are using expect-lite against gdb often. Because I was forced to use >> to send data to gdb, I had to also include lines at some places which explicitly waited for gdb to send a prompt. If the prompt detection regular expressions can detect your prompts, then using the > command to send data will make a less cluttered script, as you will not need to explicitly wait for prompts from the spawned program.
You can assign values to variables by capturing part of the output of the spawned command. When your expect-lite script starts executing, the spawned command will be bash, so you can directly run programs by sending the command line to bash with the > expect-lite command. As seen above, if you execute an interactive program such as gdb, then the > expect-lite command talks with gdb.
The below script captures the output of the id command and throws away everything except the user name. The conditional expression in expect-lite has a similar syntax to that of a similar expression in C; the difference is that in expect-lite the IF portion is prefixed and terminated with a question mark and the separator between the THEN and ELSE statement is a double colon instead of a single colon. The reason that I wait for the final "..." at the last line of the script is so expect-lite will not close the session after issuing the echo command and before the results of echo are shown.

$ id
uid=777(ben) gid=777(ben) 

$ cat branches.elt
>id
+$user=[a-z]+=[0-9]+(([a-z]+)
? $user == ben ? >echo "howdee ben..." :: >echo "A stranger eh?..."
<...>
Expect-lite supports loops by using labels and jump to label together with conditionals. Because expect-lite starts a bash shell on the remote host, you can also use the shell's conditionals and looping support. Shown below is an expect-lite script that uses bash to perform a loop:

$ cat shell-conditionals.elt
>for if in `seq 1 10`; do
> echo $if
>done
>echo ...
<...>
A caveat is that to get at shell variables from expect-lite you have to first >>echo $bashvar and then read it into an expect-lite variable with something like +$expectvar=.*n. For instance:

$ cat pwd-test.elt
>echo $PWD
+$expectpwd=n(.*)n
>echo $expectpwd...
<...>
You can also directly embed expect code into an expect-lite script. This might come in handy if you already have some familiarity with expect and need to perform a more advanced interaction at some stage in the script. Expect-lite lines starting with ! are embedded expect.
Needing to specify the host name to expect-lite all the time requires a level of preparation work before one can start using expect-lite.  The usage message for expect-lite informs you that you can set EL_REMOTE_HOST to define a default value for the host, but if you try this then you will discover that expect-lite/read_args{} only works when passed two command-line arguments (version 3.0.5), so if you define a default host with the environment variable, you will need to pass a dummy argument to expect-lite in order for its argument parsing to allow script execution.