Some Knowledge about InfiniBand

What is InfiniBand ？

InfiniBand technical specification?is also called Wireless bandwidth technology,IB for short,standardized by?IBTA.InfiniBand Trade Association(IBTA) is composed by Future I/O Developers Forum and NGI/O Forum Industrial Organization in 1999,work under the operation committee?as HP、IBM、Intel、Mellanox、Oracle、QLogic、Dell、Bull and other related members.IBTA is focus on the Product compliance and interoperability testing,and always strive to development and updated the infiniBand standard.

InfiniBand is a network communication protocol,It provides a switch-based, point-to-point bidirectional serial link structure (Fabric) between processor nodes and between processor nodes and input/output nodes (such as disk or storage).Each link has a device connected to each end of the link, so that well controled and defined the transmission (send and receive) at each end.The movement of data and messages is facilitated through the creation of a dedicated protected channel directly between nodes via switches and remote direct memory access (RDMA) and send/receive offloading managed and performed by InfiniBand adapters.One of the end of Adapter through（Peripheral Component Interface, PCI）PCI Express（PCIe）interface connect to CPU,the other end through InfiniBand network port connect to InfiniBand subnet.Compare with the other network communication protocol,there have more advantage in Infiniband,higher throughput、lower latency and flexible scalability. InfiniBand is used for interconnect between sever and storage system,and the interconnect between storage system.

InfiniBand network is the natural SDN network,users can choose different network topologies according to the properties of the application to achieve the best performance, as Fat-Tree、DragonFly+、and different kinkd of Torus. For example, the DragonFly+ network topology enables one million nodes to communicate simultaneously within four switch hops, which is far beyond the scale required for an E-class device, or even a 10E-class device or a hundred E-class device;Its natural SDN properties also make dynamic routing and network congestion control easier to implement.

What’s the difference between InfiniBand&Ethernet?

Infiniband is different from Ethernet,the latter is network-centric and the operating system handles various network-layer protocols;whereas infiniband is application-centric, bypass the operating system and CPU for network communication and directly offloads the CPU.

Ethernet technical specification is standardized by Xerox,and the baseband LAN specification developed by Xerox, Intel, and DEC,and released the general Ethernet technical specification in 30th Sep 1980.It is most widely used in network communication nowdays. At present, IEEE802.3 standard organization has developed its technical standards,and 100GE, 200GE and 400GE have been released, which is the highest speed data transmission technology till now.

There have each fetures and differences between InfiniBand and Ethernet,they have play the critical roles in the the network communication world.

As we know,Ethernet and IP technologies have builted the footstone of the entire Internet?world today,all humans and the smart device connect by Ethernet that is accord with the original intention of design, compatible with the different systems easy to connect with each other,whereas the original intention of design for InfiniBand interconnect technical specification is to break the bottleneck of data transmission in high performance computing scenario,high performance communication is the point.Therefore,compare with Ethernet and InfiniBand based on the different technical orientation?caused many difference with each other, mainly reflects in bandwidth、latency、scalability、network fabric.

Features of Infiniband

Higher throughput----InfiniBand support for the highest end-to-end throughput constantly between server and storage and other HPC scenario interconnection.InfiniBand support 40Gb/s (QDR) to market in 2007, while Ethernet supported 10Gb merely in 2011;InfiniBand support 56Gb/s（FDR）to market in 2011,while Ethernet supported 40Gb merely in the market in the same year;more details please reference below table listed.

Low latency----RDMA?Zero-copy networks reduce the operating system overhead and enable data to move quickly across the network.InfiniBand?latency as low as 0.7microsecond that is much less than the minimum latency for data transfer?as required.

Enhanced scalability----InfiniBand?can accommodate about 40,000 nodes?flat network?in a single subnet,can accommodate about 2^128 nodes (no limited for the nodes numbers in fact when adding the switches with unlimited numbers ).

Higher CPU efficiency----Unload by data movement,CPU can executing more computing cycles in the application,so that can reduce the running time and increase the tasks very day.In a word,it can dealing the data more faster then others in the same time.

Reduce the management cost----InfiniBand?can running under the software defined networks (SDN ) mode,and allow them running?as a part of fabric?without CPU management.

Easy to configure—When building a simple fat tree cluster,InfiniBand?is very easy to configure,whereas Ethernet?requires a variety of high-level protocols knowledge to build IT clusters.

The most important is InfiniBand can offers the cost-effective solution in high performance computing scenario,more fast, more effective.

Welcome to inquiry our related products by email [email protected]