Inducing “RST” TCP Resets in a Test Environment

When performing client testing, we try to cover every base: Malformed packets, various server response codes, and angry TCP packets, such as TCP resets (RST).

There are various reasons a TCP reset may be thrown. For example, if a mobile proxy is angry you’ve been connected too long, they may issue RST thinking you are idle and just wasting their bandwidth, (even when you’re not).

User with machine gun sends tons of messages, Rambo induces TCP reset, and the line crumbles.

Also, if something bad happens within the system, a TCP reset may also be sent from the server. For example, if the web server process just ran out of memory mid-way through serving you a page, its swan song may be a TCP reset, letting the client know its game over. (I’m sure you’ve all seen the “The Connection has been Reset” browser error page.)

Of course, its best to test for all this stuff before it happens in production. Testing for headers and malformed JSON packets is relatively easy thing to do, since most of these responses can be emulated by high-level languages and frameworks, such as Ruby on Rails, Python, etc.

You can say something like:

Serve this to your client, and watch the reaction. If there is a reaction, fix it, and if not, you done good.

Why Induce a TCP Reset?

But how to induce a TCP RST? Its not as easy as setting headers or output on a response body. I won’t go into it much here, but part of the reason its not so easy is because:

1. Its actually relatively complex — TCP/IP is not trivial, let alone the details of setting flags, ACK/SEQ numbers, etc.
2. If it was so easy to send a TCP RST, the Internet would be a much crashier place — improvements have been made on how to properly send (and receive) TCP RSTs to ensure that they are not used maliciously (they are a tool in the DoS toolkit.)

But what if I’m an innocent engineer, just trying to make ends meet, taking it day by day, trying to test how my software reacts to TCP resets ? I scoured the web looking for an easy way to do it, but couldn’t find anything too easy to implement. After discussing with Jochen Roth, our Senior Architect here at PubNub, I got an idea for a relatively simple way to do it.

How to Induce a TCP Reset

We’ll go through three steps:

1. Setup a port forward from the remote server to your local machine
2. Configure your clients to connect to your local machine
3. Use IPFW to reject connections and issue a RST

So let’s get right to it. You’ll need SSH (Secure Shell) and IPFW (IP Firewall) installed on your system. If you are running a Mac, these are installed by default. If not, click the above links to read more about how to install them on your system.

Also, be sure your sshd_config has the setting: GatewayPorts client-specified

This will enable your port forwards to be accessible by remote hosts. Then port forward to your remote server infrastructure:

sudo ssh gcohen@localhost -L 0.0.0.0:80:remoteServer.example.com:80

Replace as appropriate the login, and remote host info. The above command line says:

Logging in as myself to my local machine, I am forwarding the remote port 80 of remoteServer.example.com to my local machine’s port 80. The 0.0.0.0 dictates for SSH to bind on all local IPs (my physical, and local interfaces). sudo is needed because port 80 (locally) has special requirements, which require admin access. If you were already running say, a web server on port 80 locally, then you would get an error like “Port already in use” or “bind: Address already in use”.

You can test to see if its working now, by telnetting to port 80 on your local host. If it works, then the tunnel is working. If not, make it work before proceeding.

Next comes the magic… we’ll use IP Firewall to send a reset to anyone currently connecting to example.com via your machine.

Run the following command:

Caution: Doing this on a remote system where you have no physical access will most likely deny you [pseudo-]permanent access to that machine. Use at your own risk.

sudo ipfw add 130 reset tcp from any to any 80; sleep 3; ipfw -q flush

This command says:

• Refuse and send a TCP RST to anyone trying to connect from anywhere to anywhere on port 80
• Wait 3 seconds
• Throw out that TCP RST rule we just created, and restore the system to the default (allow anyone to connect)

You can make this command more precise, by giving the exact IPs of the source and destination traffic:

sudo ipfw add 130 reset tcp from s.s.s.s/32 to d.d.d.d/32 80; sleep 3; ipfw -q flush

You could also replace 80 with 443 (to test SSL for example).

Just be warned, using this the wrong way can lock the Internet out from your machine, so be sure you have the ability to run the flush command:

sudo ipfw -q flush

If anything goes wrong, it should restore your config to where it was before. I took note of my default config:

root@:/Users/gcohen$ ipfw list
65535 allow ip from any to any

Just in case I did something stupid. Never hurts to have a backup plan.