Skip to end of metadata
Go to start of metadata

InfiniBand

InfiniBand is a high-performance network architecture developed by the IBTA (InfiniBand Trade Association). IBTA is a group of 220 or more companies founded in 1999 with the purpose of developing a technology

which would substantially outstrip the capabilities of industry-standard I/O systems.

The InfiniBand architecture (IBA) is mainly based on switching fabric and is designed for use in I/O networks, such as storage area networks (SAN) or cluster networks, where low latency plays an extremely important role.

A large IBA network is composed by different subnets joined together by routers.

IBA Subnet

The elements of a subnet are endnodes, switches, links, and a subnet manager.

InfiniBandSubnet.png

Endnodes exchange messages routed by switches over links. No routing exists in IBA. A subnet discovery is performed in the beginning by the Subnet Manager.

IBA offers point-to-point bidirectional links with five different signaling data rates:

  • SDR - Single Data Rate with signalling rate equal to 2.5 Gbit/s in each direction per connection;
  • DDR - Double Data Rate with 5 Gbit/s per lane;
  • QDR - Quad Data Rate with 10 Gbit/s per lane;
  • FDR - Fourteen Data Rate with 14.0625 Gbit/s per lane;
  • EDR - Enhanced Data Rated with 25.78125 Gbit/s per lane.
    The physical links can be aggregated in parallel (4X, 8X or 12X) to achieve greater bandwidth.
  • The IBTA‘s InfiniBand™ roadmap:

IBTAsBandwidth.png

In IBA, QoS is supported through Virtual Lanes (VL) which also provide a mechanism to avoid Head of Line (HoL) blocking . VLs are separate logical communication links which share a single

physical link. As a packet traverses the subnet, a Service Level (SL) is defined to ensure its QoS level. Each link along a path can have a different VL, and the SL provides each link a desired priority of communication.

Each switch/router has a SL to VL mapping table that is set by the subnet
manager to keep the proper priority with the number of VLs supported on each link.
Therefore, the InfiniBand Architecture can ensure end-to-end QoS through switches, routers and over the long haul.

References

– Main.AlessandraScicchitano - 03 May 2012

  • No labels