Welcome to the IBTA Fall Webinar Series A four-part webinar series devoted to making I/O work for you Presented by the InfiniBand Trade Association The webinar will begin shortly. 1
September 23 October 21 November 11 December 9 Why I/O is Worth a Fresh Look The Practical Approach to Applying InfiniBand in Your Data Center InfiniBand Technology: No Magic, Just Good Engineering An Expanding Role for in Future Data Centers 2
Webinar Logistics All attendees are muted Listen via your computer speakers or telephone Audio broadcast through your computer speakers is the default To listen by telephone, dial the phone number in your invitation or the number displayed in your control panel Submit questions via the Questions pane in your control panel Questions will be addressed at the end of the webinar A recording of the webinar will be available at www.infinibandta.org 3
Paul Grun Chief Scientist, System Fabric Works pgrun@systemfabricworks.com Jim Ryan Intel jim.ryan@intel.com 4
We need better results, more quickly and it can t cost more 5
Application performance Scalability Low Latency CPU Utilization Bandwidth Flexible resource allocation Reduced power Reduced cooling Reduced floor space 6
How To Innovate in One Easy Step Assumptions about shared I/O devices Assumptions about how applications access I/O Assumptions about the role of the OS Assumptions about the underlying wire Assumptions about buffer copying 7
An End-to-End Problem demands an End-to-End Solution network 8
Hypothesizing an End-to-End Solution I/O Service I/O Service network 9
Hypothesizing an End-to-End Solution Message Service Message Service network 10
An Message Service Message Service Transport Message Service Transport network 11
Accessing the Message Service Message Service Transport Software I/F A virtualized I/O interface -mapped into application virtual space -message based -asynchronous network 12
A Queue-based Interface Work requests are put on a QP Software I/F Message Service Transport QP CQs responses are put on a CQ network 13
QP QP QP QP QP message message message QP An asynchronous, queue-based virtual interface for message passing message message message 14
The InfiniBand Transport Software I/F Message Service InfiniBand Transport Switched Fabric Transport operations -SEND/RECEIVE - READ - WRITE -Atomics 15
The InfiniBand Transport Software I/F Message Service InfiniBand Transport Transport Services -RC: Reliable Connected -UD: Unreliable Datagram -UC: Unreliable Connected -RD: Reliable Datagram Transport operations -SEND/RECEIVE - READ - WRITE -Atomics Switched Fabric 16
An I/O Channel QP QP QP Think of the queue pairs as being the endpoints of a channel between two applications. 17
Figures of Merit 1. Original motivation: Solve significant problems in clustering (IPC) Figure of merit: latency 2. Faster networks = faster packet rate = less protocol processing time Trouble ahead for software-based network protocols Figure of merit: CPU utilization 3. Storage: We need to move data fast, and lots of it Figure of merit: bandwidth 18
The Verbs API API Software I/F InfiniBand Transport Verbs API -optimized for message passing -designed for an asynchronous interface API -based on data structures Software -queue pairs, completion queues I/F -supports direct access from the application -APIs for memory registration Transport Switched Fabric 19
Open Fabrics Alliance Software API Software I/F OFED distribution from OFA includes: API -Verbs APIs for Linux and Windows -Upper Layer Protocols (ULPs) -Mid-Layer tools: management, Software connection establishment -Hardware-specific drivers I/F InfiniBand Transport Transport Switched Fabric 20
Physical Interconnects API API Software I/F Transport The Wires Software I/F Transport HCA Switched Fabric HCA 21
The InfiniBand Switched Fabric API API Software I/F IB Transport IB switched fabric design makes the IB transport fast and efficient. Software I/F IB Transport HCA InfiniBand Switched Fabric HCA 22
The OSI Reference Model OSI Reference Model Application Session Transport Network Link Phy the wire, whatever it happens to be 23
InfiniBand Architecture OSI Reference Model InfiniBand Application Session Transport Network Link Phy Application S/W Interface IB Transport IB Network IB Link IB Phy The part IB wire: HCAs, switches, cables 24
RoCE over Converged Ethernet OSI Reference Model InfiniBand RoCE Application Session Transport Network Link Phy Application S/W Interface IB Transport IB Network IB Link IB Phy Application S/W Interface IB Transport IB Network Enet Link Enet Phy 25
Choosing a Switched Fabric API Message Service HCA OR API Message Service NIC IB switched fabric Ethernet switched fabric 26
A Full Range of Options Enet RoCE Native IB native verbs apps n/a investment protection: preserves familiar Ethernet Full features, fastest wire speeds sockets apps ride the Ethernet evolution curve n/a investment protection: supports legacy apps 1GbE 10GbE DCB IB wire type 27
Layer 2 Fabric Management fabric manager FM A fabric can be configured autonomously, or it can be actively managed 28
Bandwidth Roadmap 29
Architecture API Message Service HCA The architecture consists of An message service, A set of APIs used by the application to access the message service, and A physical interconnect for moving messages between applications. 30
Architecture Solves the problem end-to-end drives latency down, app-to-app APP Verbs API APP Verbs API reduces demand on the CPU and on system memory bandwidth interface transport interface transport delivers high bandwidths HCA HCA 31
Next Webinar: December 9 th, 2011 An Expanding Role for in Future Datacenters - December 9, 2011-11 am ET/ 8am PT Any good technology has to have legs beneath it, and is no exception. In the last webinar in the series, we ll look at some surprising areas where the use of might be expanding in the near future. To register for the next webinar, visit www.infinibandta.org. 32
Questions? If we do not answer your question today, please email pgrun@systemfabricworks.com. 33