Web100 Linux Kernel Extensions
The Web100 project was run by PSC (Pittsburgh Supercomputing Center), the NCAA and NCSA. It was funded by the US National Science Foundation (NSF) between 2000 and 2003, although development and maintenance work extended well beyond the period of NSF funding. Its thrust was to close the "wizard gap" between what performance should be possible on modern research networks and what most users of these networks actually experience. The project focused on instrumentation for TCP to measure performance and find possible bottlenecks that limit TCP throughput. In addition, it included some work on "auto-tuning" of TCP buffer settings.
Most implementation work was done for Linux, and most of the auto-tuning code is now actually included in the mainline Linux kernel code (as of 2.6.17). The TCP kernel instrumentation is available as a patch from http://www.web100.org/, and usually tracks the latest "official" Linux kernel release pretty closely.
An interesting application of Web100 is NDT, which can be used from any Java-enabled browser to detect bottlenecks in TCP configurations and network paths, as well as duplex mismatches using active TCP tests against a Web100-enabled server.
In September 2010, the NSF agreed to fund a follow-on project called Web10G.
TCP Kernel Information Set (KIS)
A central component of Web100 is a set of "instruments" that permits the monitoring of many statistics about TCP connections (sockets) in the kernel. In the Linux implementation, these instruments are accessible through the
TCP Extended Statistics MIB (TCP-ESTATS-MIB, RFC 4898)
The TCP-ESTATS-MIB (RFC 4898) includes a similar set of instruments, for access through SNMP. It has been implemented by Microsoft for the Vista operating system and later versions of Windows. In the Windows Server 2008 SDK, a tool called TcpAnalyzer.exe can be used to look at statistics of open TCP connections. IBM is also said to have an implementation of this MIB.
Besides the kernel extension, Web100 comprises a small set of user-level tools which provide access to the TCP KIS. These tools include
libweb100library written in C
- the command-line tools
- a set of GTK+-based GUI (graphical user interface) tools under the
gutil shows a small main panel with an entry field for specifying a TCP connection, and several graphical buttons for starting different tools on a connection once one has been selected.
The TCP connection can be chosen either by explicitly specifying its endpoints, or by selecting from a list of connections (using double-click):
Once a connection of interest has been selected, a number of actions are possible. The "List" action provides a list of all kernel instruments for the connection. The list is updated every second, and "delta" values are displayed for those variables that have changed.
Another action is "Display", which provides a graphical display of a KIS variable. The following screenshot shows the display of the DataBytesIn variable of an SSH connection.
Microsoft's Windows Software Development Kit for Server 2008 and .NET Version 3.5 contains a tool called
TcpAnalyzer.exe , which is similar to
gutil and uses Microsoft's RFC 4898 implementation.
The SIFTR module for FreeBSD can be used for similar applications, namely to understand what is happening inside TCP at fine granularity. Sun's DTrace would be an alternative on systems that support it, provided they offer suitable probes for the relevant actions and events within TCP. Both SIFTR and DTrace have very different user interfaces to Web100.
- http://www.web100.org/ - Web100 site
- http://www.psc.edu/~mathis/TcpExtendedMib/status.html - TCP Extended Statistics MIB status page
- RFC 4898, TCP Extended Statistics MIB, M. Mathis, J. Heffner, R. Raghunarayan. May 2007.
– Main.SimonLeinen - 27 Feb 2006 - 25 Mar 2011
– Main.ChrisWelti - 12 Jan 2010