Zooming in on the QNoC Architecture

Transcription

1 Technion s NoC esearch: Israel Cidon Department of Electrical Engineering Technion Israel Institute of Technology Zooming in on the QNoC Architecture PIs Israel Cidon (Networking) an Ginosar (VLSI) Idit Keidar (Dist. Systems) Isaac Kesslassy (Networking) Avinoam Kolodny (VLSI) Uri Weiser (Architecture) Students: Evgeny Bolotin, oman Gindin euven Dobkin, Zvika Guz an Manevich Arkadiy Morgenshtein, Zigi Walter May One NoC does not fits all! NoC for ASICs Traffic uncertainty un time FPGA CMP Design envelop / constraints Well define inter-modules traffic Automatic synthesis Variable QoS requirement Configuration ASSP Main cost Power and area Chip design SOC single application General purpose computer Flexibility I. Cidon and K. Goossens, in Networks on Chips, G. De Micheli and L. Benini, Morgan Kaufmann,

2 QNoC Quality-of-service NoC architecture for ASICs Message routing path Traffic requirements are known a-priori Overall approach Wormhole switching QoS based on priority classes Small buffer/vc budget In-order SP XY routing Irregular topology Optimized link capacities (0,0) (1,0) (2,0) (2,1) (0,2) (0,3) (1,4) (0,4) (5,0) (2,2) (2,3) (2,4) (4,3) (3,4) (4,4) Fixed shortest-path routing (X-Y) Simple outer No deadlock scenario No reordering of messages SP is power-efficient * E. Bolotin, I. Cidon,. Ginosar and A. Kolodny., QNoC: QoS architecture and design process for Network on Chip, JSA special issue on NoC, Wormhole Switching Quality-of-Service in QNoC Small number of buffers Low latency IP2 Multiple priority classes Define latency Preemptive Possible ASIC classes Signaling eal Time Stream N ead-write DMA Block Transfer Statistical guarantees E.g. <0.01% arrive later then required T IP1 * E. Bolotin, I. Cidon,. Ginosar and A. Kolodny., QNoC: QoS architecture and design process for Network on Chip, JSA special issue on NOC,

3 outer structure Virtual Channels or another router outer Flits stored in input ports Output port schedules transmission of flits 9 10 QNoC router with Virtual Channels Simulation Model Flit-accurate simulations VC 5 SL COSS-BA 5 Delay of high-priority service levels is not affected by load 11 12

4 QNoC Design Flow QNoC Design Flow Extract intermodule traffic Extract intermodule traffic Place modules Place modules Allocate link capacities Allocate link capacities Verify QoS and cost Verify QoS and cost QNoC Design Flow Extract intermodule traffic Place modules Allocate link capacities Verify QoS and cost NoC for FPGA Design envelope / constraints Many ASIC like applications per a given FPGA Hard NoC infrastructure efficient but inflexible Soft logic is reusable but has inferior performance Architecture of choice: egular and uniform grid In-order/load balanced routing Hard logic for links, routers Soft logic for routing algorithms, headers, CNIs Soft NoC tuning (routing, CNI) for a given implementation Optimize capacity for performance/power tradeoff Capacity allocation is a traditional WAN problem Never solved for wormhole routing and NoC *. Gindin, I. Keidar and, I.Cidon, NoC Architecture for Future FPGAs., NoCs

5 NoC Based FPGA Architecture FPGA outing Optimization Problem Functional unit NoC for interrouting Common efficient NoC outers Configurable region User logic Set of Applications Different Architectures Different Traffic Patterns Configurable network interface Implemented on the same chip Ordered outing Algorithms Source Toggle XY One route per source-destination (S-D) pair No traffic splitting Unlike TXY, traffic to same destination is not split Maximum capacity similar to TXY The route is a bitwise XO of source and destination ID Can be extended to weighted source toggle (WOT) XY YX XY YX XY YX YX XY YX XY YX XY YX XY YX XY YX XY YX XY YX XY YX XY Unordered outing Ordered outing 19 20

6 Two Hotspots Maximum Capacity C a p a c i t y 20 XY TXY STXY WTXY WOT Design Envelope for various distances between the hotspots for WOT NoC and CMP Uniprocessors cannot provide Power-efficient performance growth Interconnect dominates dynamic power Global wire delay doesn t scale Interconnect Instruction-level parallelism is limited (Polack) Progress to parallel computations Chip Multi Processors (CMP) Thread-Level Parallelism (TLP) Built around NoC Uniprocessor dynamic power (Magen et al., SLIP 2004) Gate Diff Minimum Distance between the hotspots NoC for Shared Memory CMP NoC can facilitate system services Constraints Multiple access to shared cache equires cache coherency Heterogeneous traffic pattern QoS requirements (fetch, pre-fetch) Architecture of choice: Wormhole switching In-order routing Simple QoS mechanisms Embedded functions (multicast, search ) P6 P7 P0 P1 0 7 Distributed L P5 P4 P3 P2 * E.Bolotin, Z. Guz, I.Cidon,. Ginosar and A. Kolodny, The Power of Priority: NoC based Distributed Cache Coherency, NoCs

7 Delay eduction [%] Priority NoC: esults L2 Access Delay eduction by Priority-based NoC ead ead Exclusive apache zeus fft ocean radix Speedup [%] P6 P7 P0 P1 0 7 Distributed L P5 P4 Total Program Speedup by Priority-based NoC apache zeus fft ocean radix P3 P2 Generic NoC Issues Many shared problems across design spectrum, examples: Providing an extremely low latency class of service Providing broadcast/multicast functions Analysis and solutions for hot modules outing in irregular networks eliability and fault tolerance Network verification Power control of NoCs NoC physical layout Circuits for NoC lines Bus-Enhanced NoC (BENoC) Dynamic Non-Uniform Cache Access L2$ Low latency bus and high-throughput NoC Multicast/broadcast services Low latency module to module signaling NoC management & control CMP Cache example *. Manevich, I, Walter, I.Cidon and A. Kolodny, Best of Both Worlds: Bus Enhanced NoC (BENoC), NoCs Non uniform cache access (NUCA) - split into independent banks Dynamic NUCA - cache lines are moved to shorten access time Before fetching into, needs to find the L2$ storing the line When a modifies the data, all L1 copies should be invalidated 27 28

8 Bus-Enhanced NoC (BENoC) Bus-Enhanced NoC ecent Work Main messages: BENoC facilitates advanced services BENoC s broadcast is energy efficient Model NoC and bus broadcast latency and energy analytically BENoC is good for performance Use simulator to run benchmarks Short latency signaling NoC management Multicast services CMP Cache (focus of recent research) Analyzing Latency and Energy Bus-Enhanced NoC ecent Work NoC links Input C in C load Model NoC and bus broadcast latency ( P C0 Cin ) Tnet n Tlink n τ τ ( Clink Cin ) τ n + = = + + = + τ Cld Cld (( ) 0 ) τ τ T = C + τ = n+ n PC + nc + τ bus bus in Cbd Cbd Bus wire Approximate broadcast power Latency: Energy: τ T = Cload +τ C 2 in E= V ( C + C ) load in Enet = V K( Cld + Clink + Cin ) τ( npc0+ npc0+ nc in) Ebus = V( ( n+ n) PC0+ ncin) + V n τ ( P C0+ Cin ) + τ τ β C ld 31 32

9 BENoC saves energy BENoC is Good for Performance Network broadcast Bus transaction (beta=1) Bus transaction (beta=10) Network unicast ead transaction time performance gain Energy per bit [pj/bit] Split cache traffic Broadcast over Bus Unicast over NoC n Generic NoC Issues NoC Clogging Hot modules cannot absorb traffic from NoC Off chip systems Shared memory subsystems Expensive functional units IP1 IP2 IP3 HM is not a local problem Transparent to NoC performance Walter, Cidon, Ginosar and Kolodny, Access egulation to Hot-s in Wormhole NoCs, NOCS

10 Source Fairness No fairness is guarantied since routers arbitration is based on local state The further is the source from the destination, its worm has to win more arbitrations The HM module bandwidth isn t fairly shared IP (HM) Hot Distributed Arbitration Control is part of interface or a core serving multiple HM eceives all requests and employs a scheduling scheme equests and grants sent in a high service level or bus equests and grants includes additional data: requested quota, source queue size, priority, deadline, etc. Granted quota, scheduling of transmission's, etc. Initial credits hides light load request-grant latency Hot vs. non-hot Traffic Optimizing routing on Irregular Mesh HM Traffic With Control Around the Block Dead End HM Traffic Without Control Other Traffic Without Control Goal: Minimize the total size of routing tables Other Traffic With Control 39 E. Bolotin, I. Cidon,. Ginosar and A. Kolodny, "outing Table Minimization for Irregular Mesh NoCs", DATE

11 Saving Table Hardware Traditional solutions - full routing tables Destination Based outing - at router Source outing at sources Solution idea: Use educed Tables Store only relevant destinations (PLA) Default function ( Go XY or Don t turn ) + Table for deviations outing Cost [gates] outing Heuristics for Irregular Mesh 90,000 80,000 70,000 60,000 50,000 40,000 30,000 20,000 10,000 0 outing Cost in 12x12 NoC (many andom holes, high problem hotspost instances probability) Hotspot Number D XYDT S SDP Distributed outing (full tables) X-Y outing with Deviation Tables Source outing Source outing for Deviation Points Log ( outing Cost ) outing Systems Cost with eduction real applications in eal Aplications D TT XYDT S SDP 1 MPEG4 VOPD Efficient outing esults Scaling of Savings Savings Summary NoC became a major research field Key to SoC, FPGA and CMP future Different NoC architectures required NoC is not a miniature off-chip net Power major optimization factor Latency critical and solvable May integrate fine grain operations and control May integrate multiple best-of-breed architectures Network Size 43 44