We are upgrading this site on Friday 5 March commencing at 17:00 UTC and running until 20:00 UTC. During the maintenance window there will be several reboots and service interruptions so we strongly recommend that you don’t attempt to use the site during the maintenance window.
Page tree
Skip to end of metadata
Go to start of metadata

NERSC Case Study: Stateful Firewall


This case study was presented at Joint Techs 2006 (Albuquerque, NM) by Brian Draney of NERSC. NERSC is the US DoE's scientific computer centre, which has ~20 TFlops of processing power and 8.8PB of storage. It uses a 10GE LAN backbone and connects to EDnet at 10 Gbps.


Traffic between a pair TCP-tuned hosts was being lost. When un-tuned TCP was used there were no such drops.


The point at which the transfer hung can be seen here in Xplot (the end of the white line)

A close up shows where there are 3 DUPACKs (marked with a green 3 in the plot below).

Using tcpdump it could be seen that the sender was sending re-transmits, but these were never getting through to he receiver. The re-transmits would time out and the next re-transmit would be sent (after an increased interval), and this would contine until the sender gave up.


It was determined that there was a stateful firewall in the path that did not believe the re-transmits were legitimate (because the TCP SEQ numbers were so different from what preceded them).

– Main.TobyRodwell - 16 Feb 2006

  • No labels