Networks Distributed Systems Philipp Kupferschmied Universität Karlsruhe, System Architecture Group May 6th, 2009 Philipp Kupferschmied Networks 1/ 41
1 Communication Basics Introduction Layered Communication Layers in the ISO/OSI Model 2 Static Interconnection Networks Dynamic Interconnection Networks 3 Routing in Switched Networks 4 Connection Handling Reliable Communication Philipp Kupferschmied Networks 2/ 41
Why Communication? Introduction Layered Communication Layers in the ISO/OSI Model Threads of a distributed application collaborate Lack of shared memory messages Preconditions: Physical interconnection network Electric signals on wires, etc. Rules obeyed by each communication partner Communication protocols Common language and common semantics Otherwise no mutual understanding Philipp Kupferschmied Networks 3/ 41
Information, Data, Signals Introduction Layered Communication Layers in the ISO/OSI Model information: abstract idea of what I want data: formal representation of information using a known vocabulary signal: physical representation of data in time/space on a medium Philipp Kupferschmied Networks 4/ 41
Information, Data, Signals Introduction Layered Communication Layers in the ISO/OSI Model information: abstract idea of what I want data: formal representation of information using a known vocabulary signal: physical representation of data in time/space on a medium Philipp Kupferschmied Networks 4/ 41
Information, Data, Signals Introduction Layered Communication Layers in the ISO/OSI Model information: abstract idea of what I want data: formal representation of information using a known vocabulary tree signal: physical representation of data in time/space on a medium Philipp Kupferschmied Networks 4/ 41
Information, Data, Signals Introduction Layered Communication Layers in the ISO/OSI Model information: abstract idea of what I want data: formal representation of information using a known vocabulary tree signal: physical representation of data in time/space on a medium Philipp Kupferschmied Networks 4/ 41
Base Model of Communication Introduction Layered Communication Layers in the ISO/OSI Model sender service access point/ service entry point receiver service interface medium message Participants act as sender (client) or receiver (server) Using a service by a client takes place via a service-interface on a specific service entry point Via the medium the spatial distance is met Philipp Kupferschmied Networks 5/ 41
Introduction Layered Communication Layers in the ISO/OSI Model Medium The Physical Essence in Networks Signals need a medium Air for sound waves Paper for written letters Copper wire for electrical signals Fibre for optical signals The ether for wireless transmission Some media relay the signals to many receivers, others don t Broadcast, shared medium Point-to-point communication Philipp Kupferschmied Networks 6/ 41
Introduction Layered Communication Layers in the ISO/OSI Model What May Happen When Communicating Identify the communication partner Need to find the communication partner Get a communication line (with certain properties) Make sure we talk the same language Come to a common understanding what we agreed upon Deal with partial or complete network breakdown (more) Philipp Kupferschmied Networks 7/ 41
Introduction Layered Communication Layers in the ISO/OSI Model Reliable vs. Unreliable Communication Reliable Communication All messages are delivered either exactly once... or at least once... or at most once Unreliable Communication Each message may be delivered once... or multiple times... or not at all Philipp Kupferschmied Networks 8/ 41
Introduction Layered Communication Layers in the ISO/OSI Model Uni- vs. Bidirectional Communication Notification One way, unidirectional Sender continues immediately Usually unreliable (may get lost) E.g., remote log messages Request Bidirectional Sender waits for response ACK or Some result Usually reliable required E.g., RPC Philipp Kupferschmied Networks 9/ 41
Point-to-Point vs. End-to-End Introduction Layered Communication Layers in the ISO/OSI Model Point-to-point Data exchange between directly physically connected hosts + No routing required Completely meshed network required End-to-end Data exchange between between potentially non-neighbored hosts Routing required (depending on topology) + Less expensive network topology possible Philipp Kupferschmied Networks 10/ 41
Layered Communication Introduction Layered Communication Layers in the ISO/OSI Model Layer C data Layer C Layer B data Layer B Layer A data Layer A protocols transmission medium Seemingly: horizontal communication within layers Actually: vertical communication between layers per system Philipp Kupferschmied Networks 11/ 41
Protocols and Interfaces Introduction Layered Communication Layers in the ISO/OSI Model Layer n+1 (client) Two adjacent layers in a protocol stack work together well-defined interfaces Layer n Layer n-1 (server) Server offers service Client wants service Interface must be specified Philipp Kupferschmied Networks 12/ 41
Headers and Encapsulation Introduction Layered Communication Layers in the ISO/OSI Model Layer C data Layer C Layer B data Layer B Layer A header data trailer Layer A protocols transmission medium Each layer wraps data from previous layer Type of contents Explicitly given in outer protocol header Implicitly inferred from received data Philipp Kupferschmied Networks 13/ 41
The ISO/OSI Reference Model Introduction Layered Communication Layers in the ISO/OSI Model 7. Application application protocol Application 6. Presentation 5. Session 4. Transport 3. Network 2. Data link 1. Physical presentation protocol (hide different data representations) session protocol (identify users, manage sessions) transport protocol (packet-error handling, connections) network protocol (transmit packets between systems, routing) data link protocol (bit-error detection, medium access) physical protocol (transmit bits between neighbours) Presentation Session Transport Network Data link Physical Philipp Kupferschmied Networks 14/ 41
The ISO/OSI Reference Model Introduction Layered Communication Layers in the ISO/OSI Model 7. Application application protocol Application 6. Presentation presentation protocol Presentation 5. Session session protocol Session 4. Transport transport protocol Transport 3. Network Network Network 2. Data link Data link Data link Data link 1. Physical Physical Physical Physical Philipp Kupferschmied Networks 14/ 41
Role of ISO/OSI Layers 1 4 Introduction Layered Communication Layers in the ISO/OSI Model 1. Physical layer (e.g., RS-232, Manchestercode) Unreliable point-to-point communication Carry signals between neighbors 2. Data link layer (e.g., Ethernet) Reliable point-to-point communication Medium access control (MAC), bit error detection 3. Network layer (e.g., IP) Unreliable end-to-end communication Routing 4. Transport layer (e.g., TCP) Reliable end-to-end communication Detect packet loss, out-of-order reception Philipp Kupferschmied Networks 15/ 41
Role of ISO/OSI Layers 5 7 Introduction Layered Communication Layers in the ISO/OSI Model 5. Session layer Manage sessions Used for example by RPC 6. Presentation layer Convert data formats Little-/big-endian Relative/absolute data Textual vs. numeric representation Mostly unused/integrated in application layer 7. Application layer (e.g., HTTP, FTP) Whatever you want Philipp Kupferschmied Networks 16/ 41
Overview Communication Basics Static Interconnection Networks Dynamic Interconnection Networks switch/hub Communication networks: usually star, bus, ring,... Multicomputer/DS: usually ring, lattice, hypercube,... Philipp Kupferschmied Networks 17/ 41
Token Ring Communication Basics Static Interconnection Networks Dynamic Interconnection Networks Constant, low node degree (always 2) (Practically) limited extensibility Sender puts message on line If he received the token message All nodes inspect message and pass it on Original sender does not pass the message on Converts message into token message Medium access controlled by token Philipp Kupferschmied Networks 18/ 41
Lattice/Torus Communication Basics Static Interconnection Networks Dynamic Interconnection Networks Torus generalized ring Constant node degree (2 #dimensions) Incrementally extensible in (small) steps Good for distributed apps. with local communication pattern (e.g., meterologic simulations) Philipp Kupferschmied Networks 19/ 41
Hypercube Communication Basics Static Interconnection Networks Dynamic Interconnection Networks Logarithmic diameter (max. distance between nodes) Extensible by doubling number of nodes only Take two d-dimensional hypercubes Combine corresponding nodes with edges Also increases node degree (#dimensions) Philipp Kupferschmied Networks 20/ 41
Analysis Hypercube Communication Basics Static Interconnection Networks Dynamic Interconnection Networks N = 2 d nodes d neighbours per node (degree d) d 2 d edges Diameter: log 2 N = log 2 2 d = d Simple routing Many paths (robust, parallel communication) Expensive Philipp Kupferschmied Networks 21/ 41
Cube Connected Cycles Static Interconnection Networks Dynamic Interconnection Networks d 2 d nodes (d: dimension) Logarithmic diameter Constant, low node degree (always 3) Extensible by (more than) doubling number of nodes only Philipp Kupferschmied Networks 22/ 41
Completely Meshed Communication Basics Static Interconnection Networks Dynamic Interconnection Networks + Low diameter (1) High cost Limited extensiblility by max. node degree Philipp Kupferschmied Networks 23/ 41
Summary Communication Basics Static Interconnection Networks Dynamic Interconnection Networks Lattice L(a 1 a 2... a d ) Torus T (a 1 a 2... a d ) Hypercube H(d) Cube Connected Cycles CCC(d) Lattice Torus nodes edges degree diameter d d a i (a k 1) 2d d (a k 1) k=1 d a k k=1 i k k=1 a k d k=1 k=1 k=1 H(d) 2 d d 2 d 1 = d d CCC(d) d 2 d 3d 2 d 1 = 3 2d + d/2 a k = 2d d a k /2 Philipp Kupferschmied Networks 24/ 41
Bus-Connected Nodes Static Interconnection Networks Dynamic Interconnection Networks Cheap Blocking Poor scalability E.g., PCI, SCSI, Ethernet Philipp Kupferschmied Networks 25/ 41
Bus-Connected Nodes Static Interconnection Networks Dynamic Interconnection Networks Cheap Blocking Poor scalability E.g., PCI, SCSI, Ethernet Philipp Kupferschmied Networks 25/ 41
Bus-Connected Nodes Static Interconnection Networks Dynamic Interconnection Networks Cheap Blocking Poor scalability E.g., PCI, SCSI, Ethernet Philipp Kupferschmied Networks 25/ 41
Example: Ethernet Communication Basics Static Interconnection Networks Dynamic Interconnection Networks Developed 1973 1976 at Xerox Physical bus, hubs and switches Medium access: CSMA/CD 2.94 MBit/s (today: 10 GBit/s) Philipp Kupferschmied Networks 26/ 41
Crossbar Communication Basics Static Interconnection Networks Dynamic Interconnection Networks + Non-blocking + Fast Expensive Not scalable Philipp Kupferschmied Networks 27/ 41
Omega Nets Communication Basics Static Interconnection Networks Dynamic Interconnection Networks 000 001 010 011 100 101 110 111 bit 2 bit 1 bit 0 either or 000 001 010 011 100 101 110 111 Hierarchical crossbar + Less expensive Higher latency Limited connectivity 000 000 precludes 100 001 Simple routing src dst interpret dst bitwise, MSB first 0: upper port, 1: lower port Philipp Kupferschmied Networks 28/ 41
Fat Tree Nets Communication Basics Static Interconnection Networks Dynamic Interconnection Networks root switch inner nodes are switches leaves are processors or memory Upper levels with higher bandwidths or multiple redundant links Philipp Kupferschmied Networks 29/ 41
Circuit vs. Packet Switching Static Interconnection Networks Dynamic Interconnection Networks Circuit switching Guarantees bandwidth Connection established if all systems can fulfill requirements State in intermediate systems represents the connection E.g., telephone, desired for streaming Packet switched Gains bandwidth by multiplexing Packets are treated independently E.g., Internet, suited for message-oriented communication Philipp Kupferschmied Networks 30/ 41
Connecting LANs Communication Basics Purpose Routing in Switched Networks switch/hub Philipp Kupferschmied Networks 31/ 41
Schematic Outline of a Router Purpose Routing in Switched Networks Incoming ports and queues Coupling network, e.g. crossbar Outgoing ports and queues Routing table Packets are queued in each router Increased lantency Packets are dropped if incoming queue is full Philipp Kupferschmied Networks 32/ 41
Link-State Routing (Dijkstra) Purpose Routing in Switched Networks 5 C 5 2 B 3 1 F A 2 3 2 A 1 D 1 E 1 2 D B Convert graph to tree Root = node whose routing table is desired Successively add node with smallest accumulated distance 1 E 1 2 C F Requires whole system image not scalable Philipp Kupferschmied Networks 33/ 41
Distance Vector Routing Purpose Routing in Switched Networks Simple adaptive routing protocol Originally used as RIP for the Internet (now: BGP) Every router maintains a table (node, out-port, distance) per node Tables are regularly exchanged with neighbours If (distance+1) < my distance, update out-port and distance Problems Slow convergence to consistent state Count-to-infinity for broken nodes High overhead due to exchange of complete tables Philipp Kupferschmied Networks 34/ 41
Purpose Communication Basics Purpose Connection Handling Reliable Communication Use unreliable, packet switched network Provide reliable transmission of data streams Introduce ACKs and sequence numbers Retransmit unacked packets after timeout Multiplex multiple connections on a link Flow control Congestion control E.g., TCP UDP (more or less) offers IP at transport layer Philipp Kupferschmied Networks 35/ 41
Notions Communication Basics Purpose Connection Handling Reliable Communication Acknowledgements Positive: OK, packet received (ACK) Negative: Packet missing, resend! (NACK) Sequence numbers Allow correct reassembly of data Packets may come in out-of-order Packets may be lost Timeouts Determine whether a packet should be resend E.g., if no ACK was received after 3 s Philipp Kupferschmied Networks 36/ 41
Establishing a Connection Purpose Connection Handling Reliable Communication Connection Request Connection Indication Connection Response Connection Confirm Are connections bidirectional? Philipp Kupferschmied Networks 37/ 41
Establishing a Connection Purpose Connection Handling Reliable Communication Connection Request Connection Indication Connection Request Connection Indication Connection Confirm Connection Response Connection Confirm Connection Response Are connections bidirectional? Does this open one or two connections? Philipp Kupferschmied Networks 37/ 41
Stop and Wait Communication Basics Purpose Connection Handling Reliable Communication Sender sends one packet and waits for ACK Resend after timeout If ACK was lost, receiver gets duplicates! Use sequence numbers to identify duplicates Slow due to waiting for ACKs Philipp Kupferschmied Networks 38/ 41
Sliding Window Communication Basics Purpose Connection Handling Reliable Communication 1 2 3 4 5 ACK(2) ACK(5) Send consecutive packets i to i + W 1 Send further packets only after i has been ACKed + Increased throughput due to Multiple outstanding packets Aggregated ACKs Possibly overlapping data and ACK packet transmission Philipp Kupferschmied Networks 39/ 41
Retransmission Communication Basics Purpose Connection Handling Reliable Communication 1 2 3 4 5 3 4 5 1 2 withheld from application ACK(2) 3 4 5 A sends 5 packets Packet 3 gets lost Packets 4 and 5 are withheld from the application at B B ACKs reception of all packets 2 (or NACKs packet 3) A resends 3 5 (or just 3) + Complete data is passed in-order to the application Sender must keep sent packets Philipp Kupferschmied Networks 40/ 41
The End Communication Basics Purpose Connection Handling Reliable Communication Feel free to ask questions! Philipp Kupferschmied Networks 41/ 41