A class of switches that is typically used in data center networks to connect servers within a rack. ToR switches are typically based on "merchant silicon" forwarding engines. They have a set of fixed ports to connect servers, and "uplink" ports to connect to the rest of the data center network, for example to "spine" switches in a leaf-spine topology. These uplink ports are sometimes modular. The standard form factor has a height of a single rack unit (1RU).
State-of-the-art ToR switch configurations in 2014 are based on a chipset such as the Broadcom "Trident II" and support 48 10GE server connections (either SFP+ modules or 10GBASE-T) and six QSFP+ 40GE connections. Future chipsets such as the Broadcom "Tomahawk" should allow higher port counts and support 100GE interfaces. There are variants of these switches with only 40GE ports (typically 24-32), mainly for use as spine switches.
Limitations
ToR switches are designed for space- and cost-efficiency. Most of the forwarding logic is implemented on a single ASIC, a highly integrated application-specific integrated circuit. Memory space for buffers as well as for forwarding tables is generally quite limited.
Small Buffers
For the impact of limited buffer sizes, see NetworkBufferSizing. In general, small buffers can work fine if there is no overload, or if end-to-end delays remain small - e.g. for traffic staying in the confines of the data center. However, even in the data center there can be situations where small buffers harm performance, such as Incast.
Small Forwarding (and other) Tables
The chipset has limited space for forwarding tables. These limitations are expressed in the maximum numbers of MAC addresses or IPv4 and IPv6 prefixes. These limitations have to be taken into account when designing networks. For example, in a large data center one should avoid distributing routes for individual servers or, even worse, individual virtual machines.
Limited Feature Set
The set of data-plane functions are limited by the intersection of what the forwarding chipset can do and what is exposed by the software. High-speed forwarding of Ethernet (including VLANs), IPv4 and IPv6 are standard features as of 2014. Other functions such as MPLS are supported by state-of-the-art chipsets, but not necessarily by switch operating systems.
Some switches support OpenFlow to allow external control to some of the low-level forwarding logic. This can be used for SDN approaches with external/centralized controllers.
Limited Control-Plane Processing Power
Space, cost and thermal constraints mean that there is usually no place for a high-end CPU alongside the forwarding ASIC. This limits what can be done outside the ASIC-based fast path. In particular, it is problematic to "punt" packets to the CPU that the ASIC cannot process, as the limits of how many packets can be sent to and processed by the CPU will be orders of magnitude lower than what the many high-speed ports can deliver.
-- SimonLeinen - 2014-12-30 - 2014-12-31