This Wiki is available to view but is still under maintenance. PLEASE DO NOT EDIT THE WIKI UNTIL FURTHER NOTICE. We are attempting to restore missing edits which took place between Monday 8 and Thursday 11 April 2019, therefore the site is likely to be taken off line at any time. Updated 20:43 CEST 16 April 2019.
Page tree
Skip to end of metadata
Go to start of metadata

InfiniBand

InfiniBand is a high-performance network architecture developed by the IBTA (InfiniBand Trade Association). IBTA is a group of 220 or more companies founded in 1999 with the purpose of developing a technology

which would substantially outstrip the capabilities of industry-standard I/O systems.

The InfiniBand architecture (IBA) is mainly based on switching fabric and is designed for use in I/O networks, such as storage area networks (SAN) or cluster networks, where low latency plays an extremely important role.

A large IBA network is composed by different subnets joined together by routers.

IBA Subnet

The elements of a subnet are endnodes, switches, links, and a subnet manager.

InfiniBandSubnet.png

Endnodes exchange messages routed by switches over links. No routing exists in IBA. A subnet discovery is performed in the beginning by the Subnet Manager.

IBA offers point-to-point bidirectional links with five different signaling data rates:

  • SDR - Single Data Rate with signalling rate equal to 2.5 Gbit/s in each direction per connection;
  • DDR - Double Data Rate with 5 Gbit/s per lane;
  • QDR - Quad Data Rate with 10 Gbit/s per lane;
  • FDR - Fourteen Data Rate with 14.0625 Gbit/s per lane;
  • EDR - Enhanced Data Rated with 25.78125 Gbit/s per lane.
    The physical links can be aggregated in parallel (4X, 8X or 12X) to achieve greater bandwidth.
  • The IBTA‘s InfiniBand™ roadmap:

IBTAsBandwidth.png

In IBA, QoS is supported through Virtual Lanes (VL) which also provide a mechanism to avoid Head of Line (HoL) blocking . VLs are separate logical communication links which share a single

physical link. As a packet traverses the subnet, a Service Level (SL) is defined to ensure its QoS level. Each link along a path can have a different VL, and the SL provides each link a desired priority of communication.

Each switch/router has a SL to VL mapping table that is set by the subnet
manager to keep the proper priority with the number of VLs supported on each link.
Therefore, the InfiniBand Architecture can ensure end-to-end QoS through switches, routers and over the long haul.

References

– Main.AlessandraScicchitano - 03 May 2012

  • No labels