On Packet Switched Networks for On-Chip Communication

Transcription

1 On Packet Switched Networks for On-Chip Communication Embedded Systems Group Department of Electronics and Computer Engineering School of Engineering, Jönköping University Jönköping 1 Outline : Part 1 Introduction Evolution of Network on Chip Direct Connections Buses Platforms Networks Interconnection Network Concepts and Issues Data Integrity and Data Loss Ordering Dead Locks Destination Naming Latency and Bandwidth Bisection-Cut Network Load Routing Modes 2

2 Outline : Part 2 Network on Chip Architectures and Design KTH NoC Architecture Switch/Router Designs Resource Network Interface The Philips Router 2-D fabric Crossbow Other proposals 3 Outline: Part 3 Layered Communication in NoC NoC Protocols NoC Architecture : Open issues On-Chip Memory Evaluation of NoC Architectures Architectural Parameters Application Parameters Performance Parameters Conclusions Some references 4

3 An Interesting Cross-road FPGA Chip Design SoC Chip Multiprocessors Computer Architecture 5 An Interesting Cross-road FPGA Chip Design SoC Chip Multiprocessors Computer Architecture NoC 6

4 Merging Roads NoC Computer Networks 7 Driving Forces for Billion Transistors Architectures Scalability Reusuability Cores Software O.S. Higher Abstraction Levels Small Engineering Cost Small Time to market 8

5 Chip Multi-processor Road Question: New computer architecture which can use such a high available capacity? Options: Super-computer on a chip: Powerful instruction set; Higher accuracy, On-chip main memory, On-chip interfaces; Super-scalar Architecture Multi-threaded super-scalar architecture Multi-processor on a chip 9 Evolution: ASIC Vs. PLDs Building Blocks/ Architecture ASIC Cores PBD RTL Library Cores FPGAs Cell Lib. CLBs Custom Gates Time

6 Large FPGAs: Cluster- Based Logic Blocks Logic Cluster Pins: Larger capacity of FPGA requires more pins ( Rent s Rule). Capacity : No. of gates / No. of CLBs/ No. of clusters of CLBs Long-wires:?? CLB CLB 11 SoCs Today 12

7 General Architecture of Core Based SoCs P1 P2 P3 P4 M1 Glue Logic M2 CODEC FFT DSP FPGA Serial Ports 13 Options for connecting cores Hardwire core to core connections Programmable connections Buses Circuit Switched interconnection network Packet switched communication 14

8 Bus based connections Bus Controller RISC Mem Graphic Controller DSP Simple to wire Not scalable Low bandwidth Useful for small chips 15 Hierarchical bus based connections Bus Level 1 Bus Level 2 Bus Controller 16

9 Platform Based Design Fixed interconnection infrastructure Time-share the resources Bus based platform is not scalable P DSP Core2 FFT Memory 17 Limitation of buses Scalability: Shared buses donot scale beyond 8 resources Latency The delay is decided by longest physical distance between two resources Contention for the bus and arbitration slows down data movement Bus based systems are suitable when the communication requirement is: Low on average communication Only a few resources are sources- majority are destinations. 18

10 Circuit Switched Communications An electrical path is established from source till destination for the duration of communication. At every cross point there is a programmable switch. 1 2 Switch Matrix 3 4 A B C D 19 Advantages/Limitation of Circuit Switching Circuit Switching similar to previous slide has been proposed in PROPHID SoC Architecture[5] + Higher and guaranteed point to point bandwidth Useful for real time video-processing applications - Lack of reactivity to traffic and random traffic - Not very scalable and adaptable 20

11 Packet Switched Network A packet switch network also consists of a network of switches (many times called routers). The data moves from source to destination in small formatted blocks called packets. Packet consists of Header, Payload, Terminator Header: Destination/source address, error detection/correction bits, priority, etc. Payload: Actual data Trailor: To identify the packet if the packet size is variable A packet may go through many switches before reaching the destination 21 Advantages/Limitations of Packet Switching + Routing is reactive to traffic + The network is very scalable and easily adaptable to new applications - Overheads - Contentions in Switches and higher latency - Difficult to gaurantee performance 22

12 Network Issues: Integrity and Data Loss Data Reliability and Integrity The received data is identical to the sent data Data Loss Correction/retransmission if error is detected Due to error introduced by the network Due to contention for network link or lack of buffer space Mechanisms: Acknowledgement Retransmission 23 Data Ordering and Deadlocks Data Ordering Data to destination may reach out of order Different paths in the network Data loss Mechanism to ensure ordering at destination Deadlocks A situation in which the flow of data is blocked Circular paths in the network Limited Buffer capacity and circular dependency Solution Avoid circular paths Dynamic routing 24

13 Destination Naming/Addressing Destination Naming/Addressing Bus: data available to all destinations Destination do the address decoding Multi-casting is very easy Network Routers/Switches must decide the direction Multi-casting is difficult Can cause flooding of network 25 Latency Definition: Network latency is the time to transfer n bytes of information from its source to its destination. Time(n) S-D = Routing Delay + Contention Delay + Channel Occupancy + Overhead Routing Delay: Depends on the distance between source and destination Channel Occupancy: Depends on extra bits for header, error detection and correction, control signals etc. Contention Delay: Overheads: 26

14 Bandwidth In general, bandwidth refers to the rate at which data can me moved. Two views of Bandwidth: Global aggregate bandwidth available Estimate of time to move M bytes of data in a network with bandwidth B is = M/B Assumes uniformly distributed traffic in the network Local individual bandwidth available for a node Depends on raw bandwidths of links connected to the node Also depends on delay due routing decisions and contentions 27 Bisection Bandwidth Bisection Cut removes minimum number of channels from the network such that the network is partitioned into two equal networks. The sum of the bandwidth of the removed channels is the Bisection Bandwidth Bisection Cut Bisection bandwidth gives the upper limit on the communication between two halves of network. 28

15 Network Load Network Load is the measure of the actual communication traffic in the network with respect to maximum possible traffic. Maximum traffic rate = Number of links * link bandwidth Network Load Actual traffic rate = ( Total data transfered in the network)/ Time Average load = Actual traffic rate/ Maximum traffic rate 29 Routing Mode Store and Forward A complete packet moves from one router to the next Storage required for the full packet Packet can only be forwarded further after complete reception Higher communication delay Virtual-Cut Routing A packet can be forwarded if the next router is ready to receive the full packet Every router in the chain require full packet buffer Parts of packets in a chain of routers: low latency till destination Warm-hole Routing Packets partitioned into flits The first flit of a packet finds and reserve a path for the complete packet. Low buffer requirement and low latency 30

16 Interconnection Network Parameters Topology Types of nodes: Resources and Switches Regular/irregular Direct/indirect Routing Algorithm Static vs. Dynamic Decides delay, congestion and deadlocks Switching Strategy Circuit switched vs. Packet switched Network utilization is higher in packet switched networks Flow control Buffering, packetization, dropping packets, 31 KTH-NoC Architecture Overview Switch Resource Slot Scalable packet switched communication infrastructure Physical-Architectural Level design integration: o A Resource must fit in the slot o Layout same as topology -Predictable electrical properties 32

17 Architectural-Physical Level Design Integration The layout of the chip matches with the architecture Placement and route tool will not change the macrolevel floor plan Electrical properties like delay, cross talk, signal quality is predictable and controllable Some masks of NoC chip can be shared among applications- leads to sharing of cost of the masks. 33 Resource-Network Interface Resource RNI Resource Types: Processor of any type with/without local memory Memory IP Functional Cores FPGAs Dedicated Hardware block 34

18 NoC Router/Switch with Input Buffer 35 Buffer-less Switch Motivation: a small switch Basic Idea: A packet which enters the switch is sent out through the best possible direction Contention resolved through priority Priority Number of hops already made: hop counter with every packet Distance to the destination 36

19 Buffer-less Switch: Example N E S E 37 Buffer Less Switch: Internals Controller E E W N S Controller generates select signals for MUXs Priority based on network traffic and packet header N S W 38

20 Routing Algorithms Static Algorithms: The path(s) from source to destination is(are) fixed. Shortest paths Contention for links resolved using priority Low priority message may have very high latency Dynamic Algorithms: The path is decided dynamically looking at the other traffic in the network Contention for links: resolved using priority Better average performance than static routing Better link utilization Lower average delay 39 NoC layout: Square Switch -Technology: 60 nm -22 mm X 22 mm -Switch: 0.2mm X 0.2mm Resource Slot: 2mm X 2mm -100 resources -256 wires in each direction

21 NoC Layout: Thin Switch Resource Switch Compact Layout as compared to square switch Wiring over the resources 41 Comparison with communication in core based SoC. Send( B, Data) A 64 B Send(C, Data) Two Send commands will be implemented differently by A Interfaces A-B and A-C may use different protocols C 42

22 Application Layer in NoC Architecture. Send( B, Data) A C Virtual Channel Send(C, Data) Network grantees that a message can be reliably sent from any source to any destination. B 43 Network Layer in NoC Architecture C Packet size vs. word size Packets routed independently Routing algorithm Static vs. dynamic Priority classes Buffer in switch A B 44

23 Data-link layer C Moving a word from one switch to a neighboring switch using interconnection resources A Error detection and correction Encoding for efficiency B 45 Data Format at various layers Message Transport Layer Addr PSN Part1 Network Layer Pri Header Payload Data Link Layer EC Physical Layer Word 46

24 Protocol Space for NoC Performance Direct Connections NoC protocols Internet Protocols Flexibility/Generality/Overheads 47 Protocol Decisions Driving forces Simplicity Efficiency Hardware cost Decisions Fixed packet width One packet per word No acknowledgement at Data Link Level Unequal Error protection for various fields of word at data link level Only header field in the word may be provided with error detection and correction capability 48

25 Concept of Region Resources larger than a slot FPGA Shared Memory blocks Special parallel processor Wrapper will make the region transparent to outside traffic Communication within a region could happen differently than outside Wrapper 49 Typical Regions in NoC Audio Processing Region External Interfaces Region Video Processing Region 50

26 Other NoC Architectures Many other researchers have proposed packet switched networks for System on Chips[1,4,6] The basic ideas are similar, the details are quite different 51 Scalable, Programmable Integrated Network (SPIN) Provides Packet switched system on chip interconnections The topology of the network is based on fat trees Nodes are connected by two one-way 32-bit links. Provide layered communication protocols 52

27 A fat-tree network Fat tree has been shown to be suitable for VLSI implementation. Two stages of routers nodes at the leaf levels 53 Philips Router Architecture Handles two traffic classes Guaranteed Throughput (GT) Routing Provides reliable connection oriented service In-order packet transmission Guaranteed Bandwidth Best Effort (BE) Routing Loss-less connection ( No dropped packets) Uses input queuing Also delivers packets in order 54

28 Router Architecture The router has a N X N cross-bar switch inside it Routing Table Routing Table N rows with S slots Each row has connection information for each output A column represents a time slot For GT connections slots are reserved 55 2_D Interconnection Fabric A standardized wrapper around cores to get a two dimensional logical topology. Specially suitable for making mesh connected multi-processor systems Can be used for connecting FPGAs, Processors and other cores if a proper wrapper is added. More Information at: 56

29 2_D Fabric Architecture Wrapper One Layer Signal Routing, No Glue-Logic Shashi Required Kumar 57 Packet Switched Networks: Handling Memories Most of the real chips will have over 60% area used for memory Memory access through packet switching mechanism is very inefficient. Direct connections to nearby memories will be very useful. 58

30 Hybrid Interconnections: best of both worlds Circuit switched: resource to resource connections Packet switched communication through switch matrix Desirable: Main application is going to be system emulation 59 Options for a Resource RSI Processor Processor Mem Memory 60

31 Hybrid NoC with Memory 61 NoC for a Data Intensive applications Shared Memory Blocks 62

32 Tools for NoC Architecture Design NoC Specialization Tools Resource Selection and Specialization Tool Protocol selection and specialization tool Code and Configuration Generation Tools NoC Evaluation Tool NoC Architecture Simulator NoC Emulators 63 NoC Simulators Objectives: To evaluate various design options in NoC architecture Switch Design Buffer Size vs. performance( delay, drop probability) Routing Algorithm Protocols Performance Parameters Delay in packets Drop probability Congestion and hot spots 64

33 NoC Simulation using ns-2 (by Yi-Ran Sun) ns-2 is a public domain network simulator from Univ. of Berkeley Has been extensively been used for research and teaching of computer networks Facilities Definition of network topology Selection of communication protocols: TCP, UDP Selection of routing algorithms Provision to add user defined protocols, routing algorithms, traffic generation Provision for visualization, traces, statistics collection,.. 65 Simulation Experiments Architectural Parameters Topology: 5 X 5 Protocol: UDP( no acknowledgement) Link Bandwidth: 200 Mbits/sec Routing algorithm: Static-shortest distance to destination Application Parameters Communication Traffic: bursty traffic with strong bias for locality Performance Experiments Packet delay vs. buffer size Drop probability vs. buffer size Packet delay vs. network load Drop probability vs. network load 66

34 Drop probability vs. Buffer size Network-on-Chip(NOC) Simulation 24 Yiran Sun, ESDLab, 17, December, Drop probability vs. Communication load Network-on-Chip(NOC) Simulator Simulation 26 Communication load Yiran Sun, ESDLab, 17 December,

35 Modeling NoC in SDL ( Jönköping) Objectives Explore Router Design Both input and output buffers Effect of router speed vs. resource speed Bursty Traffic Model Generic model of NoC Any size of NoC can be modeled Delay in various components in NoC can be specified Simulation can be done at any layer of protocol Data link or Network layer 69 Simulation Experiments Architectural Parameters Topology: 4 X 4 Both input and output buffers Protocol: Simple data format with 1 byte pay-load No error detection and correction at data link level Transfer of whole packet in one word Routing algorithm: A simple adaptive algorithm Application Parameters Communication Traffic: Heavy bursty traffic Performance Experiments Throughput with buffer sizes and router clock Dropped Packets Average packet delay Spread : Percentage of packets taking longer than shortest route 70

36 Simulation Results (1) 0,4 0,5 0,6 Number of transferred packets Number of transferred packets (14 Bursty Resources, 8 pkts per Burst) :01 01:02 01:03 02:01 02:02 03:01 Bufferconfig (In:Out) 71 Simulation Results (2) Transfer time (14 Bursty Resources, 8 pkts per Burs t) 0,4 0,5 0,6 Network Speed, Period Time 01:01 01:02 01:03 02:01 02:02 03:01 72

37 Simulation Results (3) Number or dropped packets 0,4 0,5 Number of dropped packets 0,6 (14 Bursty Resources, 8 pkts per Burst) :01 01:02 01:03 02:01 02:02 03:01 Bufferconfiguration (In:Out) 73 Simulation Results (5) Spreading 60% 40% 20% 0% 01:01 01:02 01:03 02:01 02:02 03:01 Clk=0,8 48% 8% 5% 38% 10% 41% Clk=0,7 34% 5% 3% 33% 3% 31% Clk=0,5 11% 2% 1% 9% 1% 9% 74

38 Conclusions Network on Chip Architecture is an importnat choice for future SoCs Provides reusability at various levels Scalable General purpose Various types of tools will be required before NoC Architecture can be used for SoC design NoC Architecture based SoC design can learn a lot from computer network design Handling memory within NoC is an important open problem. 75 References 1. On Packet Switched Networks for On_chip communication, Chapter 5 in the book entitled Network on Chip, edited by Axel Jantsch and Hannu Tenhunen, Kluwer Publishers, L. Benini and G. DeMicheli, Network on Chip: a new SoC paradigm, IEEE Computer Jan Ahmed Hemani et. al., Network on Chip: An Architecture for billion transistor era, Proc. Of the IEEE NorChip Conference, Nov et. al. A Network on Chip Architecture and Design Methodology, Proceedings of IEEE International Symposium on VLSI 2002( ISVLSI 2002), April, 2002, Pittsburgh, Pennsylvania, USA. 5. Pierre Guerrier and Alain Greiner, A Generic Architecture for On Chip Packet-Switched Interconnections, Proceedings of Design, Automation and test in Europe, pp ,

39 References 6. Michael Bedford Taylor et. al., The RAW Microprocessor: A computational fabric for software circuits and general purpose programs, IEEE Micro, pp 25-35, W. Dally and B. Towles, Route Packets, Not wires: on chip Interconnection Networks, Proceedings of the Design Automation Conference, June Rickard Holsmark and Magnus Högberg, Modeling and Prototyping of a Network on Chip, Master thesis, Department of Electronics and Computer Engineering, School of Engineering, Jönköping Univ