Interconnect technologies: InfiniBand, 1G Ethernet, 10G Ethernet

guodong.zhang@hp.com — Thu, 02 Jul 2009 16:40:00 GMT

Interconnect is an integral part of any scale-out solutions. The question often asked is something like "which interconnect should I use for my next scale-out project?" Well, a simple answer is "it depends". For many applications in the high performance computing (HPC) area, their performance highly depends on the latency and bandwidth of the interconnect fabric -- majority of MPI-based parallel applications are in this category. InfiniBand has been proven to be the best choice for these applications. In the latest released Top500 (June 2009), I counted 151 entries with InfiniBand as the interconnect, increased from 124 6 months ago. Another interesting observation is about what server platforms were used to build these clusters -- I found that the HP BladeSystem c-Class is the most popular server platform with 37 entries, followed by IBM's pSeries with 17 entries. In the last couple of years, we also see more and more cases in commercial applications where InfiniBand is being used as a transport interconnect to meet the low latency and scalability requirements, examples including HP ExaData storage server, Market data systems deployed in Financial Services Industry, and applications in rendering applications.

Not every application needs InfiniBand. There are workloads, include some HPC workloads, that their performance is not sensitive to the latency of the underline interconnect. For these application, Gigabit Ethernet provides a very cost effective solution. However, we see increased bandwidth requirement in many of these cases, partially driven by the adoption of server virtualization, or simply to match higher I/O requirement driven by significant performance increase of the multi-core servers. 10Gigabit Ethernet is becoming a natural choice to meet this higher bandwidth requirement, yet, allow people to stay within the technology they are familiar with.

So, to summarize, the decision on the choice of interconnect should be based on the performance requirements of applications: for majority of MPI parallel applications as well as those that require low latency and high bandwidth, InfiniBand is becoming the interconnect of choice. And InfiniBand is also a technology that is very easy to use and manage.

Search results matching tag 'interconnect'

Interconnect technologies: InfiniBand, 1G Ethernet, 10G Ethernet