OceanStor 9000 InfiniBand Technical White Paper. Issue V1.01 Date HUAWEI TECHNOLOGIES CO., LTD.
|
|
- Anissa Wilkinson
- 6 years ago
- Views:
Transcription
1 OceanStor 9000 Issue V1.01 Date HUAWEI TECHNOLOGIES CO., LTD.
2 Copyright Huawei Technologies Co., Ltd All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademarks and Permissions and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders. Notice The purchased products, services and features are stipulated by the contract made between Huawei and the customer. All or part of the products, services and features described in this document may not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, either express or implied. The information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute a warranty of any kind, express or implied. Huawei Technologies Co., Ltd. Address: Website: Huawei Industrial Base Bantian, Longgang Shenzhen People's Republic of China support@huawei.com Huawei Confidential Page i
3 Contents 1 InfiniBand Overview Technical Background Technical Features of IB Introduction to IB Working Principles IB Architecture IB Protocol Layering Technical Features Customer Benefits Acronyms and Abbreviations Huawei Confidential Page ii
4 1 InfiniBand Overview 1.1 Technical Background The layered structure of the traditional TCP/IP protocol brings high network latency to cache management and extra overhead to the operating system. As network technologies develop, the network requires an open architecture that features high bandwidth, low latency, solid reliability, and flexible scalability and regards switching as the core. InfiniBand (IB) comes into being to meet such requirements. With its excellent features, IB is applicable to the storage network and computing networks. By using the Remote Direct Memory Access (DRMA) technology and a virtual addressing solution, IB enables servers to identify and utilize some memory of other servers without occupying the kernel resources of the operating system. IB inherits the bus's benefits such as high bandwidth and low latency while easing the pressure for processing CPU loads. Therefore, IB perfectly suits the storage cluster applications Huawei Confidential Page 1
5 2 Technical Features of IB 2.1 Introduction to IB Working Principles Compared with other network protocols (such as TCP/IP), IB has a higher transfer efficiency. Some protocols provide the ability to resend lost data packets. However, repetitive acknowledgement and resending during packet forwarding causes communication based on such protocols to slow down, negatively affecting system performance. It should be noted that the TCP protocol is a transfer protocol widely used in a variety of devices from fridges to super computers. However, it has disadvantages, for example, the TPC protocol is extremely complicated, the number of codes is huge, different exceptions need to be handled, and the TCP protocol is hard to uninstall. Unlike the TCP protocol, IB uses a trustworthy flow control mechanism to ensure intact connections and near-zero packet loss. When IB is used, data is sent only when the receiving cache has sufficient space. After receiving the data, the recipient returns a signal to specify the available cache space. In this way, IB eliminates the resending delay caused by packet loss, improving transfer efficiency and overall performance. IB supports channel-based bi-directional, serial-connection transfer. Switched Fabric is applied in the IB topology, where IB switches are used. If the channel is not long enough, IBA repeaters can be used to extend. Each IBA network is called a subnet, which has a maximum of 65,536 nodes. IBA switches and IBA repeaters apply only to the subnet. If communication takes place across multiple IBA subnets, IBA routers or IBA gateways are required. Nodes are connected to IBA subnets through adapters. CPU and memory are connected to IBA subnets through host channel adapters (HCAs). Disks and I/Os are connected to IBA subnets through target channel adapters (TCAs). The connection between two components is named one link. All these components form a complete IBA. Different transfer materials are available in IB. Inside a device, the copper foils of the printed circuit boards (PCBs) are used for transfer (especially used in the backplanes of the industrial control and telecom equipment). Outside a device, copper writes are used for transfer. For a long distance, optical fibers can be used for transfer. When copper foils or copper wires are used, the longest transfer distance is 17 m. However, if optical fibers are used, the transfer distance can be as far as 10 km. In addition, the IBA is hot pluggable and supports the auto-detection and auto-adaption Active Cable intelligent connection mechanism Huawei Confidential Page 2
6 2.1.2 IB Architecture Figure 2-1 IB architecture The IB standard defines a set of devices that are used for system communication, including channel adapters, switches, and routers. 1. A channel adapter is used to connect devices, and can be classified into HCAs for main control nodes and TCAs for peripheral nodes. In this way, I/O devices are independent of hosts and can directly sit on the network. A channel adapter incorporates the functions of the physical layer, link layer, network layer, and transport layer. A channel adapter is an important part of the IB network interface. It is a programmable DMA component with specific protection features, allowing local and remote DMA operations. 2. A switch is a basic component of the IB architecture and is responsible for forwarding packets within an IB subnet. 3. A router is also a basic component of the IB architecture and is responsible for forwarding packets among different IB subnets Huawei Confidential Page 3
7 2.1.3 IB Protocol Layering Figure 2-2 IB protocol laying Similar to the traditional TCP/IP protocol, IB is also a layered protocol. Each layer provides different functions. The lower layers provide services for the upper layers. Different layers are independent of each other. 1. The upper-level protocol layer provides a Verbs interface between the applications and hardware drivers, allowing the upper-layer applications to enable RDMA programming based on the Verbs interface. 2. The transport layer is responsible for distributing packets to the target and segmenting and reconstructing packets that exceed the maximum transfer unit (MTU). Specifically, the transport layer is responsible for packet distribution, multiplexing, basic transmission service, and sending, receiving, and restructuring of packet segments. The transport layer sends a data packet to a specific QP and instructs the QP to process the data packet. If the data path load of the message exceeds the path MTU, the transport layer divides the message into multiple data packets. The QP of the receiving end is responsible for reconstructing data to a specific data buffer area. 3. The network layer is responsible for routing packets between different subnets. The network layer describes the protocols for forwarding data packets between subnets. It is similar to the network layer of the IP network. The network layer is not required when data is transferred within the subnet. 4. The link layer is responsible for resolving packet formats and packet operations, such as flow control and packet switching on the subnet. The link layer describes the protocols for data packet formats and operations, such as flow control and data packet routing on subnets. The link layer supports two types of packets: link management data packets and data packets Huawei Confidential Page 4
8 Link management data packages are used to perform and maintain the link operations. Link management data packets are generated and consumed at the link layer and do not conflict with flow control. Data packets contain different optional head information. These packets can determine bit rates and link bandwidth among ports at the end of the links and to transport flow control credits and maintain link completeness. 5. The physical layer is responsible for transferring data frames through cables by bit. IB provides lossless network transfer. The physical layer meets the requirement for 10e -12 bit error rate. The physical layer defines the electrical and mechanical features, including the features of the cables and sockets that use optical fibers and copper as media, the base connectors, and the heat exchangers. In addition, the physical layer defines three types of physical ports: backplane ports, electrical ports, and optical ports. The electric cables are made of copper wires and support 100 m of transfer distance. Optical fiber ports support as far as 10 km of transfer distance. 6. The physical layer also specifies how bits are changed into symbols in channels and defines signals for creating frames (packet start and end), data symbols, and idles. In addition, the physical layer describes signaling protocols for constructing effective packets, such as symbol encoding, frame flag arrangement, invalid or non-data symbols between start and end delimiters, non-parity errors, and synchronization methods. 2.2 Technical Features 1. High bandwidth In IB, 1, 4, 8, and 12 cables are parallel to increase the channel bandwidth, and SDR, DDR, QDR, and FDR technologies are used to further increase bandwidth. Bandwidth varies depending on the number of parallel channels and technologies, as shown in the following table: Number of Channels SDR DDR QDR FDR 1X 2.5 GB/s 5 GB/s 10 GB/s 14 GB/s 4X 10 GB/s 20 GB/s 40 GB/s 56 GB/s 8X 20 GB/s 40 GB/s 80 GB/s 112 GB/s 12X 30 GB/s 60 GB/s 120 GB/s 168 GB/s 2. Low latency Latency is an important index to determine the interconnection of high-performance computers. The latency of switches in the IB topology is less than 100s and the application delay less than 1 to 3 us. 3. Flexible scalability Millions of terminal devices are interconnected in the P2P switched fabric to provide an IB network without congestion. 4. QoS Huawei Confidential Page 5
9 16-level virtual channels that can be mapped to 16 service layers are available. By setting priority for each virtual channel, you can manage service and quality of different SLs of different service levels, credit-based flow control mechanism, and injection rate control mechanism. The congestion can be controlled as a result. 5. Support for RDMA The RDMA technology enables IB servers and the services in the storage network to exchange data with the memory of other servers in a high speed. 6. Dedicated protocol offload engine IB implements efficient and reliable P2P transmission through hardware and supports channel-based message transmission and memory image technologies. It also provides the ability to bypass the core of the operating, sharing the CPU load and improving the overall performance. 7. Separation of the I/O subsystem from the host Channel adapters provide links to the I/O controllers and allow the I/O devices to be detached from hosts. The subnet management pattern (one primary SM + multiple SMAs) is a secure and efficient management pattern. It helps conserve the cabinet space, provide excellent scalability, and shatter the distance bottleneck between the host and I/O system. If the transfer materials are made of copper, the transfer distance is 17 m. If the transfer materials are optical fibers, the transfer distance 10 km. 8. Support for partitioning IB subnets are divided into multiple partitions, providing improved performance and security. 9. Error tolerance Multiple physical channels (separate from each other) are built between the host system and the I/O devices to achieve fault tolerance. 10. IPV6 addressing IB uses the IPv6 header format, in which a data packet contains the source address and destination address of the data. These addresses enable the IB switches and routers to directly forward data to specific devices according to their forwarding tables (configured by the SM-SMP-SMA). 2.3 Customer Benefits Typical Application Scenarios Customer Benefits The IB network applies to the high-concurrency and high-performance computing application scenarios. In these scenarios, customers have demanding bandwidth and latency requirements. General networking is that both front-end and back-end networks use IB, or the front-end network uses 10GE and the back-end network uses IB. The OceanStor 9000 supports both the prior networking modes. In the typical scenario, 4-channel QDR IB adapters and switches are used. With its high bandwidth, low latency, high reliability, and massive cluster scalability, and the adaption of the RDMA technology and dedicated protocol offload engine, IB provides storage customers with sufficient bandwidth and low response latency Huawei Confidential Page 6
10 3 Acronyms and Abbreviations DDR FDR IB QDR SDR Double Data Rate Fourteen Data Rate InfiniBand Quad Data Rate Single Data Rate Huawei Confidential Page 7
Introduction to High-Speed InfiniBand Interconnect
Introduction to High-Speed InfiniBand Interconnect 2 What is InfiniBand? Industry standard defined by the InfiniBand Trade Association Originated in 1999 InfiniBand specification defines an input/output
More informationIntroduction to Infiniband
Introduction to Infiniband FRNOG 22, April 4 th 2014 Yael Shenhav, Sr. Director of EMEA, APAC FAE, Application Engineering The InfiniBand Architecture Industry standard defined by the InfiniBand Trade
More informationHUAWEI AR Series SEP Technical White Paper HUAWEI TECHNOLOGIES CO., LTD. Issue 1.0. Date
HUAWEI AR Series SEP Technical White Paper Issue 1.0 Date 2015-01-19 HUAWEI TECHNOLOGIES CO., LTD. 2015. All rights reserved. No part of this document may be reproduced or transmitted in any form or by
More informationHUAWEI OceanStor Enterprise Unified Storage System. HyperReplication Technical White Paper. Issue 01. Date HUAWEI TECHNOLOGIES CO., LTD.
HUAWEI OceanStor Enterprise Unified Storage System HyperReplication Technical White Paper Issue 01 Date 2014-03-20 HUAWEI TECHNOLOGIES CO., LTD. 2014. All rights reserved. No part of this document may
More informationHUAWEI USG6000 Series Next-Generation Firewall Technical White Paper VPN HUAWEI TECHNOLOGIES CO., LTD. Issue 1.1. Date
HUAWEI USG6000 Series Next-Generation Firewall Technical White Paper VPN Issue 1.1 Date 2014-03-14 HUAWEI TECHNOLOGIES CO., LTD. 2014. All rights reserved. No part of this document may be reproduced or
More informationAnti-DDoS. User Guide (Paris) Issue 01 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 01 Date 2018-08-15 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2018. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationOperation Guide for Security NEs Management
imanager U2000 Unified Network Management System V100R002C01 Operation Guide for Security NEs Management Issue 03 Date 2010-11-19 HUAWEI TECHNOLOGIES CO., LTD. 2010. All rights reserved. No part of this
More informationHuawei OceanStor ReplicationDirector Software Technical White Paper HUAWEI TECHNOLOGIES CO., LTD. Issue 01. Date
Huawei OceanStor Software Issue 01 Date 2015-01-17 HUAWEI TECHNOLOGIES CO., LTD. 2015. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without
More informationTechnical White Paper for NAT Traversal
V300R002 Technical White Paper for NAT Traversal Issue 01 Date 2016-01-15 HUAWEI TECHNOLOGIES CO., LTD. 2016. All rights reserved. No part of this document may be reproduced or transmitted in any form
More informationVirtual Private Cloud. User Guide. Issue 21 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 21 Date 2018-09-30 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2018. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationesdk Storage Plugins 1.0.RC4 Compilation Guide 01(vRO) Issue 01 Date HUAWEI TECHNOLOGIES CO., LTD.
1.0.RC4 Issue 01 Date 2017-04-06 HUAWEI TECHNOLOGIES CO., LTD. 2017. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent
More informationAnti-DDoS. FAQs. Issue 11 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 11 Date 2018-05-28 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2019. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationHuawei esight LogCenter Technical White Paper HUAWEI TECHNOLOGIES CO., LTD. Issue 1.0. Date PUBLIC
Huawei esight LogCenter Technical White Paper Issue 1.0 Date 2013-12-03 PUBLIC HUAWEI TECHNOLOGIES CO., LTD. 2013. All rights reserved. No part of this document may be reproduced or transmitted in any
More informationAD SSO Technical White Paper
Issue V1.0 Date 2017-02-28 Huawei Technologies Co., Ltd. 2017. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of
More informationespace UMS V100R001C01SPC100 Product Description Issue 03 Date HUAWEI TECHNOLOGIES CO., LTD.
V100R001C01SPC100 Issue 03 Date 2012-07-10 HUAWEI TECHNOLOGIES CO., LTD. . 2012. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior
More informationHuawei FusionCloud Desktop Solution 5.1 Resource Reuse Technical White Paper HUAWEI TECHNOLOGIES CO., LTD. Issue 01.
Huawei FusionCloud Desktop Solution 5.1 Resource Reuse Technical White Paper Issue 01 Date 2014-03-26 HUAWEI TECHNOLOGIES CO., LTD. 2014. All rights reserved. No part of this document may be reproduced
More informationInfiniBand SDR, DDR, and QDR Technology Guide
White Paper InfiniBand SDR, DDR, and QDR Technology Guide The InfiniBand standard supports single, double, and quadruple data rate that enables an InfiniBand link to transmit more data. This paper discusses
More informationAspects of the InfiniBand Architecture 10/11/2001
Aspects of the InfiniBand Architecture Gregory Pfister IBM Server Technology & Architecture, Austin, TX 1 Legalities InfiniBand is a trademark and service mark of the InfiniBand Trade Association. All
More informationMPLS OAM Technology White Paper
MPLS OAM Technology White Paper Issue 01 Date 2012-10-30 HUAWEI TECHNOLOGIES CO., LTD. 2012. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without
More informationDomain Name Service. Product Description. Issue 03 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 03 Date 2018-08-15 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2018. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationMark Falco Oracle Coherence Development
Achieving the performance benefits of Infiniband in Java Mark Falco Oracle Coherence Development 1 Copyright 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy
More informationHigh Performance Computing: Concepts, Methods & Means Enabling Technologies 2 : Cluster Networks
High Performance Computing: Concepts, Methods & Means Enabling Technologies 2 : Cluster Networks Prof. Amy Apon Department of Computer Science and Computer Engineering University of Arkansas March 15 th,
More informationInfiniband Fast Interconnect
Infiniband Fast Interconnect Yuan Liu Institute of Information and Mathematical Sciences Massey University May 2009 Abstract Infiniband is the new generation fast interconnect provides bandwidths both
More informationPerformance monitoring in InfiniBand networks
Performance monitoring in InfiniBand networks Sjur T. Fredriksen Department of Informatics University of Oslo sjurtf@ifi.uio.no May 2016 Abstract InfiniBand has quickly emerged to be the most popular interconnect
More informationEnergy Saving Technology White Paper HUAWEI TECHNOLOGIES CO., LTD. Issue 01. Date
Energy Saving Technology White Paper Issue 01 Date 2012-08-13 HUAWEI TECHNOLOGIES CO., LTD. 2012. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means
More informationHuawei MZ110 NIC V100R001. White Paper. Issue 07 Date HUAWEI TECHNOLOGIES CO., LTD.
V100R001 Issue 07 Date 2016-11-21 HUAWEI TECHNOLOGIES CO., LTD. 2016. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent
More informationRoCE vs. iwarp Competitive Analysis
WHITE PAPER February 217 RoCE vs. iwarp Competitive Analysis Executive Summary...1 RoCE s Advantages over iwarp...1 Performance and Benchmark Examples...3 Best Performance for Virtualization...5 Summary...6
More informationInformatix Solutions INFINIBAND OVERVIEW. - Informatix Solutions, Page 1 Version 1.0
INFINIBAND OVERVIEW -, 2010 Page 1 Version 1.0 Why InfiniBand? Open and comprehensive standard with broad vendor support Standard defined by the InfiniBand Trade Association (Sun was a founder member,
More informationUltraPath Technical White Paper
HUAWEI OceanStor Enterprise Unified Storage System Issue 01 Date 2014-04-02 HUAWEI TECHNOLOGIES CO, LTD Copyright Huawei Technologies Co, Ltd 2014 All rights reserved No part of this document may be reproduced
More informationBGP/MPLS VPN Technical White Paper
V300R001C10 BGP/MPLS VPN Technical White Paper Issue 01 Date 2013-12-10 HUAWEI TECHNOLOGIES CO., LTD. 2013. All rights reserved. No part of this document may be reproduced or transmitted in any form or
More informationS Series Switches. MACsec Technology White Paper. Issue 1.0. Date HUAWEI TECHNOLOGIES CO., LTD.
S Series Switches MACsec Technology White Paper Issue 1.0 Date 2016-03-25 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2016. All rights reserved. No part of this document may be
More informationInfiniBand Networked Flash Storage
InfiniBand Networked Flash Storage Superior Performance, Efficiency and Scalability Motti Beck Director Enterprise Market Development, Mellanox Technologies Flash Memory Summit 2016 Santa Clara, CA 1 17PB
More informationWorkshop on High Performance Computing (HPC) Architecture and Applications in the ICTP October High Speed Network for HPC
2494-6 Workshop on High Performance Computing (HPC) Architecture and Applications in the ICTP 14-25 October 2013 High Speed Network for HPC Moreno Baricevic & Stefano Cozzini CNR-IOM DEMOCRITOS Trieste
More informationesight V300R001C10 SLA Technical White Paper Issue 01 Date HUAWEI TECHNOLOGIES CO., LTD.
V300R001C10 Issue 01 Date 2013-12-10 HUAWEI TECHNOLOGIES CO., LTD. 2013. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written
More informationHG531 V1 300Mbps Wireless ADSL2+ Router Product Description. Issue _01 HUAWEI TECHNOLOGIES CO., LTD.
HG531 V1 300Mbps Wireless ADSL2+ Router Issue 203275_01 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2013. All rights reserved. No part of this document may be reproduced or transmitted
More informationespace SoftConsole V200R001C02 Product Description HUAWEI TECHNOLOGIES CO., LTD. Issue 01 Date
espace SoftConsole V200R001C02 Issue 01 Date 2012-07-10 HUAWEI TECHNOLOGIES CO., LTD. 2012. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without
More informationHuawei FusionSphere 6.0 Technical White Paper on OpenStack Integrating FusionCompute HUAWEI TECHNOLOGIES CO., LTD. Issue 01.
Technical White Paper on OpenStack Integrating Issue 01 Date 2016-04-30 HUAWEI TECHNOLOGIES CO., LTD. 2016. All rights reserved. No part of this document may be reproduced or transmitted in any form or
More informationQuidway S5700 Series Ethernet Switches V100R006C01. Configuration Guide - Ethernet. Issue 02 Date HUAWEI TECHNOLOGIES CO., LTD.
V100R006C01 Issue 02 Date 2011-11-21 HUAWEI TECHNOLOGIES CO., LTD. 2011. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written
More informationElastic Load Balance. User Guide. Issue 01 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 01 Date 2018-04-30 HUAWEI TECHNOLOGIES CO., LTD. 2018. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of
More informationAgile Controller-Campus V100R002C10. Permission Control Technical White Paper. Issue 01. Date HUAWEI TECHNOLOGIES CO., LTD.
V100R002C10 Permission Control Technical White Paper Issue 01 Date 2016-04-15 HUAWEI TECHNOLOGIES CO., LTD. 2016. All rights reserved. No part of this document may be reproduced or transmitted in any form
More informationCDN. Product Description. Issue 03 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 03 Date 2018-08-30 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2018. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationLive Streaming Accelerator. Quick Start. Issue 03 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 03 Date 2018-08-30 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2018. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationAdvanced Anti-DDoS. User Guide. Issue 17 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 17 Date 2018-08-13 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2019. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationMellanox Infiniband Foundations
Mellanox Infiniband Foundations InfiniBand Trade Association (IBTA) Founded in 1999 Actively markets and promotes InfiniBand from an industry perspective through public relations engagements, developer
More informationStudy. Dhabaleswar. K. Panda. The Ohio State University HPIDC '09
RDMA over Ethernet - A Preliminary Study Hari Subramoni, Miao Luo, Ping Lai and Dhabaleswar. K. Panda Computer Science & Engineering Department The Ohio State University Introduction Problem Statement
More informationNew Storage Architectures
New Storage Architectures OpenFabrics Software User Group Workshop Replacing LNET routers with IB routers #OFSUserGroup Lustre Basics Lustre is a clustered file-system for supercomputing Architecture consists
More informationHi3520D V300 H.264 CODEC Processor. Brief Data Sheet. Issue 04. Date
H.264 CODEC Processor Brief Data Sheet Issue 04 Date 2016-04-18 . 2015-2016. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written
More informationThe Road to ExaScale. Advances in High-Performance Interconnect Infrastructure. September 2011
The Road to ExaScale Advances in High-Performance Interconnect Infrastructure September 2011 diego@mellanox.com ExaScale Computing Ambitious Challenges Foster Progress Demand Research Institutes, Universities
More informationImproving Application Performance and Predictability using Multiple Virtual Lanes in Modern Multi-Core InfiniBand Clusters
Improving Application Performance and Predictability using Multiple Virtual Lanes in Modern Multi-Core InfiniBand Clusters Hari Subramoni, Ping Lai, Sayantan Sur and Dhabhaleswar. K. Panda Department of
More informationHuawei Sx7 Series Switches. SVF Technology White Paper. Issue 01. Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 01 Date 2014-11-20 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei 2015. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written
More informationDragon Slayer Consulting
Dragon Slayer Consulting Introduction to the Value Proposition of InfiniBand Marc Staimer marcstaimer@earthlink.net (503) 579-3763 5/27/2002 Dragon Slayer Consulting 1 Introduction to InfiniBand (IB) Agenda
More informationAdvanced Computer Networks. Flow Control
Advanced Computer Networks 263 3501 00 Flow Control Patrick Stuedi Spring Semester 2017 1 Oriana Riva, Department of Computer Science ETH Zürich Last week TCP in Datacenters Avoid incast problem - Reduce
More informationCERN openlab Summer 2006: Networking Overview
CERN openlab Summer 2006: Networking Overview Martin Swany, Ph.D. Assistant Professor, Computer and Information Sciences, U. Delaware, USA Visiting Helsinki Institute of Physics (HIP) at CERN swany@cis.udel.edu,
More informationStorage System. David Southwell, Ph.D President & CEO Obsidian Strategics Inc. BB:(+1)
Storage InfiniBand Area Networks System David Southwell, Ph.D President & CEO Obsidian Strategics Inc. BB:(+1) 780.964.3283 dsouthwell@obsidianstrategics.com Agenda System Area Networks and Storage Pertinent
More informationScaling to Petaflop. Ola Torudbakken Distinguished Engineer. Sun Microsystems, Inc
Scaling to Petaflop Ola Torudbakken Distinguished Engineer Sun Microsystems, Inc HPC Market growth is strong CAGR increased from 9.2% (2006) to 15.5% (2007) Market in 2007 doubled from 2003 (Source: IDC
More informationHUAWEI Secospace USG Series User Management and Control White Paper
Doc. code HUAWEI Secospace USG Series User Management and Control White Paper Issue 1.0 Date 2014-03-27 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2012. All rights reserved.
More informationApplication Acceleration Beyond Flash Storage
Application Acceleration Beyond Flash Storage Session 303C Mellanox Technologies Flash Memory Summit July 2014 Accelerating Applications, Step-by-Step First Steps Make compute fast Moore s Law Make storage
More informationThe Exascale Architecture
The Exascale Architecture Richard Graham HPC Advisory Council China 2013 Overview Programming-model challenges for Exascale Challenges for scaling MPI to Exascale InfiniBand enhancements Dynamically Connected
More informationTABLE I IBA LINKS [2]
InfiniBand Survey Jeremy Langston School of Electrical and Computer Engineering Tennessee Technological University Cookeville, Tennessee 38505 Email: jwlangston21@tntech.edu Abstract InfiniBand is a high-speed
More informationBest Practices for Deployments using DCB and RoCE
Best Practices for Deployments using DCB and RoCE Contents Introduction... Converged Networks... RoCE... RoCE and iwarp Comparison... RoCE Benefits for the Data Center... RoCE Evaluation Design... RoCE
More informationNetworking for Data Acquisition Systems. Fabrice Le Goff - 14/02/ ISOTDAQ
Networking for Data Acquisition Systems Fabrice Le Goff - 14/02/2018 - ISOTDAQ Outline Generalities The OSI Model Ethernet and Local Area Networks IP and Routing TCP, UDP and Transport Efficiency Networking
More information2008 International ANSYS Conference
2008 International ANSYS Conference Maximizing Productivity With InfiniBand-Based Clusters Gilad Shainer Director of Technical Marketing Mellanox Technologies 2008 ANSYS, Inc. All rights reserved. 1 ANSYS,
More informationModule 2 Storage Network Architecture
Module 2 Storage Network Architecture 1. SCSI 2. FC Protocol Stack 3. SAN:FC SAN 4. IP Storage 5. Infiniband and Virtual Interfaces FIBRE CHANNEL SAN 1. First consider the three FC topologies pointto-point,
More informationHUAWEI TE Mobile&TE Desktop V100R001C10. Product Overview. Issue 01. Date HUAWEI TECHNOLOGIES CO., LTD.
V100R001C10 Issue 01 Date 2014-10-10 HUAWEI TECHNOLOGIES CO., LTD. 2014. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written
More informationNetwork bandwidth is a performance bottleneck for cluster computing. Especially for clusters built with SMP machines.
Mingzhe Li Motivation Network bandwidth is a performance bottleneck for cluster computing. Especially for clusters built with SMP machines. Multirail network is an efficient way to alleviate this problem
More informationFibre Channel Gateway Overview
CHAPTER 5 This chapter describes the Fibre Channel gateways and includes the following sections: About the Fibre Channel Gateway, page 5-1 Terms and Concepts, page 5-2 Cisco SFS 3500 Fibre Channel Gateway
More informationComparing Server I/O Consolidation Solutions: iscsi, InfiniBand and FCoE. Gilles Chekroun Errol Roberts
Comparing Server I/O Consolidation Solutions: iscsi, InfiniBand and FCoE Gilles Chekroun Errol Roberts SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies
More informationQuickSpecs. HP Z 10GbE Dual Port Module. Models
Overview Models Part Number: 1Ql49AA Introduction The is a 10GBASE-T adapter utilizing the Intel X722 MAC and X557-AT2 PHY pairing to deliver full line-rate performance, utilizing CAT 6A UTP cabling (or
More informationS Series Switch. Cisco HSRP Replacement. Issue 01. Date HUAWEI TECHNOLOGIES CO., LTD.
Cisco HSRP Replacement Issue 01 Date 2013-08-05 HUAWEI TECHNOLOGIES CO., LTD. 2013. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior
More informationRouting Verification Tools
Routing Verification Tools ibutils e.g. ibdmchk infiniband-diags e.g. ibsim, etc. Dave McMillen What do you verify? Did it work? Is it deadlock free? Does it distribute routes as expected? What happens
More informationThe following InfiniBand products based on Mellanox technology are available for the HP BladeSystem c-class from HP:
Overview HP supports 56 Gbps Fourteen Data Rate (FDR) and 40Gbps 4X Quad Data Rate (QDR) InfiniBand (IB) products that include mezzanine Host Channel Adapters (HCA) for server blades, dual mode InfiniBand
More informationAdvanced Computer Networks. Flow Control
Advanced Computer Networks 263 3501 00 Flow Control Patrick Stuedi, Qin Yin, Timothy Roscoe Spring Semester 2015 Oriana Riva, Department of Computer Science ETH Zürich 1 Today Flow Control Store-and-forward,
More informationExtending InfiniBand Globally
Extending InfiniBand Globally Eric Dube (eric@baymicrosystems.com) com) Senior Product Manager of Systems November 2010 Bay Microsystems Overview About Bay Founded in 2000 to provide high performance networking
More informationChelsio Communications. Meeting Today s Datacenter Challenges. Produced by Tabor Custom Publishing in conjunction with: CUSTOM PUBLISHING
Meeting Today s Datacenter Challenges Produced by Tabor Custom Publishing in conjunction with: 1 Introduction In this era of Big Data, today s HPC systems are faced with unprecedented growth in the complexity
More informationNFS/RDMA over 40Gbps iwarp Wael Noureddine Chelsio Communications
NFS/RDMA over 40Gbps iwarp Wael Noureddine Chelsio Communications Outline RDMA Motivating trends iwarp NFS over RDMA Overview Chelsio T5 support Performance results 2 Adoption Rate of 40GbE Source: Crehan
More informationDRH Hardware Maintenance Guide
Issue 1.0 Date 2013-04-10 HUAWEI TECHNOLOGIES CO., LTD. 2013. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of
More informationInfiniBand and Mellanox UFM Fundamentals
InfiniBand and Mellanox UFM Fundamentals Part Number: MTR-IB-UFM-OST-A Duration: 3 Days What's in it for me? Where do I start learning about InfiniBand? How can I gain the tools to manage this fabric?
More informationMultifunction Networking Adapters
Ethernet s Extreme Makeover: Multifunction Networking Adapters Chuck Hudson Manager, ProLiant Networking Technology Hewlett-Packard 2004 Hewlett-Packard Development Company, L.P. The information contained
More informationCray XD1 Supercomputer Release 1.3 CRAY XD1 DATASHEET
CRAY XD1 DATASHEET Cray XD1 Supercomputer Release 1.3 Purpose-built for HPC delivers exceptional application performance Affordable power designed for a broad range of HPC workloads and budgets Linux,
More informationDomain Name Service. FAQs. Issue 07 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 07 Date 2019-03-05 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2019. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More informationThe Tofu Interconnect 2
The Tofu Interconnect 2 Yuichiro Ajima, Tomohiro Inoue, Shinya Hiramoto, Shun Ando, Masahiro Maeda, Takahide Yoshikawa, Koji Hosoe, and Toshiyuki Shimizu Fujitsu Limited Introduction Tofu interconnect
More informationSmartAX MA5821/MA5822 V800R016C10
SmartAX / V800R016C10 Product Overview The SmartAX / is a multi-service access module launched by Huawei Technologies Co, Ltd. The / applies to fiber to the building (FTTB) or fiber to the curb (FTTC)
More informationHuawei FusionCloud Desktop Solution 5.3. Branch Technical White Paper. Issue 01. Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 01 Date 2015-06-30 HUAWEI TECHNOLOGIES CO., LTD. 2015. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of
More informationIntroduction Electrical Considerations Data Transfer Synchronization Bus Arbitration VME Bus Local Buses PCI Bus PCI Bus Variants Serial Buses
Introduction Electrical Considerations Data Transfer Synchronization Bus Arbitration VME Bus Local Buses PCI Bus PCI Bus Variants Serial Buses 1 Most of the integrated I/O subsystems are connected to the
More informationMulticomputer distributed system LECTURE 8
Multicomputer distributed system LECTURE 8 DR. SAMMAN H. AMEEN 1 Wide area network (WAN); A WAN connects a large number of computers that are spread over large geographic distances. It can span sites in
More informationVPI / InfiniBand. Performance Accelerated Mellanox InfiniBand Adapters Provide Advanced Data Center Performance, Efficiency and Scalability
VPI / InfiniBand Performance Accelerated Mellanox InfiniBand Adapters Provide Advanced Data Center Performance, Efficiency and Scalability Mellanox enables the highest data center performance with its
More informationThe NE010 iwarp Adapter
The NE010 iwarp Adapter Gary Montry Senior Scientist +1-512-493-3241 GMontry@NetEffect.com Today s Data Center Users Applications networking adapter LAN Ethernet NAS block storage clustering adapter adapter
More informationEECS 122: Introduction to Computer Networks Switch and Router Architectures. Today s Lecture
EECS : Introduction to Computer Networks Switch and Router Architectures Computer Science Division Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley,
More informationObject Storage Service. Product Introduction. Issue 04 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 04 Date 2017-12-20 HUAWEI TECHNOLOGIES CO., LTD. 2017. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of
More informationLightning Fast Rock Solid
Lightning Fast Rock Solid OceanStor Dorado V3 all-flash storage is the ideal storage solution to handle mission-critical businesses for TM enterprises and organizations. Based on Huawei's unique FlashLink
More informationBirds of a Feather Presentation
Mellanox InfiniBand QDR 4Gb/s The Fabric of Choice for High Performance Computing Gilad Shainer, shainer@mellanox.com June 28 Birds of a Feather Presentation InfiniBand Technology Leadership Industry Standard
More informationCMSC 611: Advanced. Interconnection Networks
CMSC 611: Advanced Computer Architecture Interconnection Networks Interconnection Networks Massively parallel processor networks (MPP) Thousands of nodes Short distance (
More informationImage Recognition. SDK Reference. Issue 09 Date HUAWEI TECHNOLOGIES CO., LTD.
Issue 09 Date 2019-01-31 HUAWEI TECHNOLOGIES CO., LTD. Copyright Huawei Technologies Co., Ltd. 2019. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any
More information10-Gigabit iwarp Ethernet: Comparative Performance Analysis with InfiniBand and Myrinet-10G
10-Gigabit iwarp Ethernet: Comparative Performance Analysis with InfiniBand and Myrinet-10G Mohammad J. Rashti and Ahmad Afsahi Queen s University Kingston, ON, Canada 2007 Workshop on Communication Architectures
More informationVPI / InfiniBand. Performance Accelerated Mellanox InfiniBand Adapters Provide Advanced Data Center Performance, Efficiency and Scalability
VPI / InfiniBand Performance Accelerated Mellanox InfiniBand Adapters Provide Advanced Data Center Performance, Efficiency and Scalability Mellanox enables the highest data center performance with its
More informationNOTE: A minimum of 1 gigabyte (1 GB) of server memory is required per each NC510F adapter. HP NC510F PCIe 10 Gigabit Server Adapter
Overview The NC510F is an eight lane (x8) PCI Express (PCIe) 10 Gigabit Ethernet SR (10GBASE-SR fiber optic) network solution offering the highest bandwidth available in a ProLiant Ethernet adapter. The
More informationFuture Routing Schemes in Petascale clusters
Future Routing Schemes in Petascale clusters Gilad Shainer, Mellanox, USA Ola Torudbakken, Sun Microsystems, Norway Richard Graham, Oak Ridge National Laboratory, USA Birds of a Feather Presentation Abstract
More informationQuickSpecs. HP InfiniBand Options for HP BladeSystems c-class. Overview
Overview HP supports 40Gbps (QDR) and 20Gbps (DDR) InfiniBand products that include mezzanine Host Channel Adapters (HCA) for server blades, switch blades for c-class enclosures, and rack switches and
More informationThe following InfiniBand products based on Mellanox technology are available for the HP BladeSystem c-class from HP:
Overview HP supports 56 Gbps Fourteen Data Rate (FDR) and 40Gbps 4X Quad Data Rate (QDR) InfiniBand (IB) products that include mezzanine Host Channel Adapters (HCA) for server blades, dual mode InfiniBand
More informationPerformance Optimizations via Connect-IB and Dynamically Connected Transport Service for Maximum Performance on LS-DYNA
Performance Optimizations via Connect-IB and Dynamically Connected Transport Service for Maximum Performance on LS-DYNA Pak Lui, Gilad Shainer, Brian Klaff Mellanox Technologies Abstract From concept to
More information