We are upgrading this site on Friday 5 March commencing at 17:00 UTC and running until 20:00 UTC. During the maintenance window there will be several reboots and service interruptions so we strongly recommend that you don’t attempt to use the site during the maintenance window.
Page tree
Skip to end of metadata
Go to start of metadata

NERSC Case Study: Stateful Firewall

Background

This case study was presented at Joint Techs 2006 (Albuquerque, NM) by Brian Draney of NERSC. NERSC is the US DoE's scientific computer centre, which has ~20 TFlops of processing power and 8.8PB of storage. It uses a 10GE LAN backbone and connects to EDnet at 10 Gbps.

Symptom

A sluggish transfer of data between two end hosts.

Troubleshooting

For a specific IP flow, original packets did not seem to be getting through, but all the re-transmits were. In the Xplot below all the red points represent re-transmitted packets.

The sender's route table showed that the correct PMTU (Path Maximum Transmission Unit) was being used for the destination, but tcpdump showed 64kB packets leaving a 9kB capable interface.

Outcome

It was determined that a Large Send Offload NIC was being used, and this was not honouring the path MTU (becasue it did not appear to have access to the host's routing table). Over-sized packets were being sent and these were being dropped. However, the re-trnasmitted packets were managed by the host's kernel and not the LSO engine, and these did honour the PMTU, so did get through.

– Main.TobyRodwell - 16 Feb 2006

  • No labels