Author: Ben Martin
shd-tcp-tools provides a collection of tools for port forwarding, load balancing, and rate-limiting TCP connections. They can be useful if you want to offer SSH services but also limit how much of your bandwidth each user can consume, so that a single long-running SCP operation cannot starve the link from your server to the Internet.
Although SSH provides port forwarding, using shd-tcp-tools in combination with SSH port forwarding allows you to introduce transfer rate limiting to your forwarded ports. The port forwarding in shd-tcp-tools by itself is not a replacement for the port forwarding in SSH though, as shd-tcp-tools operates at the TCP level without strong authentication or encryption. So SSH and shd-tcp-tools complement each other, the former providing security while the latter builds on SSH to protect your network capacity with rate limiting and possibly load balancing.
shd-tcp-tools is not in the Ubuntu, Fedora, or openSUSE repositories. I’ll build the most recent shd-tcp-tools, version 0.05, from source on a 64-bit Fedora 9 machine. Although shd-tcp-tools does not use autotools, it has a configure script, and its Makefile targets all and install, so it can be built as though it were an autotools project. Installation is shown below. The final step is hardwired in the Makefile to install the binaries into /usr/local/bin.
$ tar xjvf /.../shd-tcp-tools-0.05.tar.bz2 $ cd shd-tcp-tools-* $ ./configure $ make $ sudo make install
The main tools that shd-tcp-tools includes are tcppipe, to create a unidirectional TCP pipe; tcp-pf, which lets you create two-directional port forwarding over TCP; and the listentwo and connecttwo combination, which allow you to set up port forwarding to a machine via an intermediate host.
The tcppipe tool lets you send or receive data in one direction over TCP with transfer rate limiting. The command takes parameters followed by the specification of the source and destination of the data to transfer. You can optionally specify a hyphen for the source and destination to use stdin and stdout respectively.
The next two code blocks are run in parallel from different terminals, because at times the server process waits for the client to open the other end of the TCP connection and send data. The first block contains the server-side commands and the second block the client ones. I left a gap in the displayed code between each command that transfers with tcppipe, with the client “sending” commands shown in the latter command block.
In the first command block below, the server block, the first command starts tcppipe listening for connections on port 5001 on localhost. The - makes any data sent to tcppipe be written to stdout. The second tcppipe command in the server block uses the -p option to show the progress. As there is only 1MB of data written to the connection there is no progress but only final statistics printed by this second server command. Although you can see that there are fewer bytes in total transferred for the third server command, the client is using rate limiting, so you can see progress information printed by the third server command during the transfer.
$ tcppipe :5001 - client connected from 0.0.0.0.0.0.0.0.0.0.0.0.0.0.255.255:57897 hi $ tcppipe -p :5001 - client connected from 0.0.0.0.0.0.0.0.0.0.0.0.0.0.255.255:52458 15729 ->: average bandwidth: 10.20 MiB/s 15729 ->: total bytes: 1048576 $ tcppipe -p :5001 - client connected from 0.0.0.0.0.0.0.0.0.0.0.0.0.0.255.255:52460 ->: bandwidth: 14.99 KiB/s bytes: 30720 ->: bandwidth: 10.01 KiB/s bytes: 61440 ->: bandwidth: 10.00 KiB/s bytes: 81920 ->: bandwidth: 9.99 KiB/s bytes: 102400 15737 ->: average bandwidth: 11.11 KiB/s 15737 ->: total bytes: 102400
The client commands that correspond with the server comments above are shown below. The first command has no corresponding server command as it just copies stdin to stdout using - as both the source and destination. The second command copies stdin to localhost port 5001. The third command copies 1MB from /dev/zero to port 5001 on localhost. The fourth command uses the --limit parameter to send at 10KBps. tcppipe performs a little bit of network buffering, which explains why dd finishes early and thinks it has written at 51KBps.
$ echo hi | tcppipe - - hi $ echo hi | tcppipe - localhost 5001 $ dd if=/dev/zero bs=1024 count=1024 | tcppipe - localhost 5001 1024+0 records in 1024+0 records out 1048576 bytes (1.0 MB) copied, 0.0806214 s, 13.0 MB/s $ dd if=/dev/zero bs=1024 count=100 | tcppipe --limit 10240 - localhost 5001 100+0 records in 100+0 records out 102400 bytes (102 kB) copied, 2.00108 s, 51.2 kB/s
While tcppipe is handy for one-off, single-direction transfers, the tcp-pf command allows you to set up two-way port forwarding. One major advantage of using tcp-pf over SSH port forwarding is the availability of the -r option to limit the transfer rate of the port forwarding.
Below is an example of using tcp-pf to create a port forwarding connection to the SSH daemon. The --rate option limits network transfers to 1,024 bytes per second, which makes the SSH shell reasonably slow. The -s option makes port 10022 on localhost the source of the port forwarding, and the -d nominates port 22 on localhost as the destination of the port forwarding. As long as you can connect to SSH on port 22 of localhost, with this tcp-pf command running you should be able to connect to SSH on port 10022 on localhost and have a rate-limited session. The -w option tells tcp-pf to wait for a given number of seconds before contacting the server and setting up the connection. I found that SSH would not connect properly without a slight delay in tcp-pf.
$ tcp-pf --rate 1024 -s 10022 -d 127.0.0.1:22 -w 1 client connected from 0.0.0.0.0.0.0.0.0.0.0.0.0.0.255.255:50715 => accept 16408 ->: bandwidth: 624.63 B/s bytes: 1268 16408 <-: bandwidth: 888.23 B/s bytes: 1804 16408 ->: bandwidth: 536.28 B/s bytes: 2724 ... 16408 ->: average bandwidth: 168.63 B/s 16408 ->: total bytes: 3524 16408 <-: average bandwidth: 860.33 B/s 16408 <-: total bytes: 17980 client handled
You can explicitly set a list of hosts that are allowed to connect to a tcp-pf source port using the -a or --allow option. This option can be used many times to specify a list of accepted hosts. If no -a option is present, any host may connect.
If you have a collection of server machines which can provide the same service, such as a group of Apache Web servers, you can have tcp-pf choose which server to connect to automatically for you to provide load balancing. The -p or --dport option lets you set the port on the destination machine that should be connected to. You specify the name of a file that contains the server IP addresses with the -l or --list option. You must create the file containing the list of IP addresses, and the command will update the number of bytes for each address as it goes along. This file contains a server IP address followed by a number that records the number of bytes that have been sent to that particular server by tcp-pf. When a client connects to the source port, tcp-pf will select the server that has had the least number of bytes sent to it and set up the current port forwarding to that server.
The listentwo and connecttwo tools allow you to set up port forwarding to a host behind a firewall. The primary difference between connecttwo and tcp-pf is that you can specify that the source port is on a different machine than your localhost. I found that these tools did not print any information about how to use them by default or when using the -h command-line option. They also do not support rate limiting or load balancing. As there are many ways to set up plain TCP port forwarding without frills, I’ll leave discussion of these tools to the readme file in the shd-tcp-tools distribution.
The port forwarding commands also work over SSH. As mentioned above, using the shd-tcp-tools port forwarding over SSH port forwarding might seem a little strange at first, but it gives you the ability to use the shd-tcp-tools rate-limiting feature and also set up new port forwardings over an existing single set of SSH forwarded ports. As an example, first use tcppipe to stream anything from port 2002 to stdout on the server with the command tcppipe :2002 -, then run the two commands shown below on the client machine. The SSH command sets up SSH port forwarding of port 2001 on the local machine to port 2002 on the server machine. The tcppipe command is similar to the one shown above, only this time traffic on port 2001 is sent over the SSH port forwarding connection to port 2002 on the server.
$ ssh -N -L 2001:localhost:2002 ben@server.example.com $ echo hi | tcppipe - localhost 2001
So far it might seem that all the heavy lifting is done by SSH and you have not gained anything using shd-tcp-tools in the above example, but you can use the --limit option to create many different port forwardings over the same single SSH port forwarding connection, where each port forwarding can have different transfer rate limits.
Normally the shd-tcp-tools will bind to all network interfaces. You can tell them to use only a specific network interface by exporting the INADDR_DEFAULT environment variable; for example, export INADDR_DEFAULT="pyx.example.com".
Final words
Some of the command-line options are a little inconsistent across these tools. The readme.txt mentions that the --rate option is used by tcp-pf while the --limit option is used by tcppipe to specify transfer rate limiting. In fact the tcppipe command accepts both --rate and --limit, making the rate option the common way to specify transfer limits between the two tools. Another example is that tcppipe takes the source and destination information as the final arguments on the command line, while tcp-pf uses the -s and -d parameters for this purpose.
That’s a minor quibble in a tool that’s still at an early beta stage, at least if you go by version numbers. These utilities work well and serve a worthwhile purpose. Being able to set up rate limiting on a per-port forwarding basis is useful when you wish to allow SSH access but also want to limit how much of your network bandwidth each user can possibly use.
Categories:
- Networking
- System Administration
- Tools & Utilities
