Linux Traffic Shaping Example
The following is a sample configuration from a real network. The host in question is connected via 100 Mb/s Fast Ethernet to a "branch office" LAN (university institute). The "branch office" is connected to the main campus LAN via a 40 Mb/s CBR (Constant Bit Rate) ATM (Asynchronous Transfer Mode) PVC (Permanent Virtual Channel/Connection/Circuit). The host acts as an NFS-over-TCP client, both with servers on the local (branch office) LAN segment and with servers on the main campus.
It was observed that write performance from the host to NFS servers on the main campus was limited to about 3 Mb/s, far below the bottleneck capacity of 40 Mb/s.
The issue was diagnosed as being related to bursty loss. Apparently when the client started sending larger bursts of traffic over the ATM circuit to the main campus, the ATM network dropped excess traffic in a way that caused bursty loss, which in turn caused TCP to severely back off its sending rate.
After some experimentation, the following QoS policy was configured on the host. On-net traffic (matching the
U32 filter) is limited to 100 Mb/s, which means that it is only limited by the interface speed. Other traffic - which is sent over the 40 Mb/s ATM circuit - is limited to 43 Mb/s. Note that TSO is turned off using
ethtool , because traffic shaping doesn't coexist well with it. Apparently, traffic shaping won't prevent the bursts resulting from TSO's bundling of packets.
With this configuration, NFS write rate comes close to 40 Mb/s, provided that the
nosync option is used for the NFS mounts.
– Main.SimonLeinen - 21 Jul 2006 (based on input from Tobi Oetiker)