Architecture and Design of Efficient 3D Network-on-Chip (3D NoC) for Custom Multicore SoC

Transcription

1 Architecture and Design of Efficient 3D Network-on-Chip (3D NoC) for Custom Multicore SoC Akram Ben Ahmed, Abderazek Ben Abdallah, Kenichi Kuroda The University of Aizu Graduate School of Computers Science and Engineering Aizu-Wakamatsu , Japan Abstract During this last decade, Network-on-Chips (NoC) have been proposed as a promising solution for future systems on chip design. It offers more scalability than the sharedbus based interconnection, allows more processors to operate concurrently. Because NoC has dedicated wires, performance can be predicted. In this context, we proposed a 2D-NoC named OASIS, which is a 4x4 mesh topology design using Wormhole switching and Stall-and-Go flow control scheme. Although OASIS-NoC has its advantages over the shared-bus based systems, it has also some limitations such as high power consumption, high cost communication, and low throughput. To overcome those limitations we propose a 3D-NoC (3D OASIS-NoC) which is an extension to our 2D OASIS-NoC. In this paper we describe the 3D OASIS-NoC architecture in a fair amount of detail and present preliminary evaluation results. Keywords-3D NoC; OASIS; Design; Concurrent; I. INTRODUCTION Global interconnects are becoming the principal performance bottleneck for high performance Systems-on-Chip (SoCs) [1], [2]. Since the main purpose for this system is to shrink the size of the chip as smaller as possible while seeking at the same time for more scalability, higher bandwidth and lower latency. Conventional bus-based-systems are no longer reliable architecture for SoC due to a lack of scalability and parallelism integration, high latency and power dissipation, and low throughput. Network-on-Chip [1], [3] is presented as a revolutionary method that can overcome these problems by presenting a simple and scalable architecture platform, inspired by the Internet. NoC connects processors, memories and other custom designs together using switching packets instead of switching messages or words, allowing NoC to provide a higher bandwidth and higher performance. On the other hand, as long as future applications get more complicated, the need for a high performance system that handles increased complexity becomes really important. For these ultra high performance applications, such us real-time radio telescope signal processing, image/video processing, and computer architecture emulation, a good architecture has to be provided to ensure a sufficient bandwidth for any transaction between memories and cores as well as communication between different cores on the same chip. Not forget to mention the importance of taking a good mapping approach into consideration. Mapping can be a crucial factor on defining the system performance. All these issues make NoC a limited solution at certain level, when we talk about tens and hundreds of cores mapped in one single chip dedicated for future complex applications because of its limited scalability and floor-planning choices. This clear weakness is not desirable for high demands of future technology. The seek for optimizing this architecture becomes more and more necessary. In the past few years, three dimensional integrated circuits (3D-ICs) [4] have attracted a lot of attention. Results obtained so far show that 3D-ICs can achieve better performances, more flexibility, and higher throughput compared to traditional ICs. This may offer an opportunity to continue performance improvements using CMOS technology, with smaller form factors, higher integration densities, and supporting the realization of mixed-technology chips [5]. As Topol et al in [6] stated, 3D-IC can improve the performance even in absence of scalability. Aside from this clear benefit, package density is increased significantly, power is reduced from use of shorter wires, and circuitry is more immune to noise [4]. Combining the NoC structure with the benefits of the 3D integration lead us to present 3D-NoC as a new architecture. This architecture responds to the scaling demands for future SoC, exploiting the short vertical links between the adjacent layers that can clearly enhance the system performance. According to Feero et al [7], 3D-NoC has the ability to decrease the number of hops, a basic and important factor to evaluate the system performance. For example, converting a 8x8 2D mesh to 4x4x4 3D mesh NoC shows that 3D Mesh NoC has 40% less hops than the 2D MESH [7]. This reduction can effectively increase throughput, and consequently latency decreases because flits traverse less hops while traveling from source to destination. Another crucial factor used to evaluate the performance of a system is power dissipation. Power dissipation becomes more and more important in designing an efficient SoC, because as the geometrical area of the chip shrinks in deep sub-micron designs, static power will consume a larger portion of the

2 sw_grant (5) port_req (25) xaddr (3) Local yaddr (3) switch_req (5) data_sent (5) data_in_l (76) South Switch_allocator Second pipeline stage: Switch allocation tail_sent (5) data_in_s (76) Tail Sent North control (25) data_in_n (76) data_in_w (76) data_in_e (76) stop_in (5) West East First pipeline stage: Routing calculation data_in (380) Crossbar Third pipeline stage: Crossbar traversal data_out_l (76) data_out_s (76) data_out_n (76) data_out_w (76) data_out_e (76) stop_out (5) Figure 1. 2D OASIS-NoC pipeline stages. power budget, which is not desirable. According to Feero et al, results show that 3D-NoC uses less energy per packet than 2D-NoC implementations [7]. This can be logically explained by the fact that reducing power dissipation and latency by reducing the number of hopes, energy, which is related to these two factors, will naturally decrease. 3D- NoC architectures outperform the 2D-NoC implementations regarding these two factors. More concrete evaluation results about 2D and 3D-NoC performance are explained in details in [7] and [8]. In this paper, we present a 3D-NoC design based on our previously designed 2D OASIS-NoC [3], [9]. We show that this proposed architecture improves the system performance in term of latency. The rest of the paper is organized as follow: In Section 2, we present an overview of 2D OASIS-NoC and then propose the architecture of the 3D layout in Section 3. In Section 4, we evaluate our proposed architecture and compare it with the 2D layout. We end the paper with concluding remarks in section 5. II. OASIS-NOC OVERVIEW Figure.2 represents OASIS-NoC [9] which is a 4x4 mesh topology network using wormhole switching. Each switch can have a maximum number of five ports, one for each direction (local, north, south, east and west). The number of ports can be reduced depending on the position of the switch in the network. We identify each switch by its X- Y coordinates (x-addr and y-addr) in the network. These addresses (3 bits each) are embedded in each input 76 bits flit which also contains information about the direction of the next port (Next-Port 5 bits), the tail indicating the end Figure D OASIS-NoC 4x4 mesh topology. of a packet (1 bit), and the payload (64 bits). See Fig.3 for more details. The routing process consists of three pipeline stages as indicated in Fig.1. First, in the routing calculation stage, destination addresses are fetched from the flit then decoded. The direction of the next port is obtained by comparing the decoded address with the processing switch address. This information is sent to the switch allocator (switch allocation stage) which decides when and to whom to allocate the output direction of flits for each input. In case of conflict between two or more input ports requesting the same output, the arbiter try to serve each request fairly using the round-robin scheme [10]. Finally, in the crossbar traversal stage, the crossbar manages the traversal of the flits to the calculated output using the switching control signals

3 provided by the switch allocator. More details about OASIS- NoC architecture is mentioned in [9] Tail Next_Port X-dest Y-dest Payload 1 Bit 5 Bit 3 Bit 3 Bit 64 Bit Figure 3. 2D OASIS-NoC flit format. We test the OASIS-NoC design using the JG codec application [11]. We set the network size to 3x3 and we assume the task implementation for the JG2000 codec [12], [13] as shown in Fig.4. We randomly mapped the tasks into OASIS-NoC as illustrated in Table.1. The OASIS- NoC design simulation results are shown in Fig.5. The graph presents the total number of clock cycles depending on the buffer size. Figure 5. 2D OASIS-NoC simulation result showing transmission time depending on the buffer depth tributed to each router and define its X,Y and Z coordinates respectively. The mesh topology is chosen for our design thanks to its several properties like regularity, concurrent data transmission, and controlled electrical parameters [14], [15]. Each switch can have a maximum of seven input ports, where five are dedicated to the intra-layer connection (local, north, south, east and west) and the two other ports (Up and Down) ensure inter-layer communication. The number of ports depends on the position of the switch in the design, since we have to eliminate any unused links that have no connections with other switches in order to reduce power consumption. Figure 4. JG codec task implementation. Table I RANDOM MAPPING FOR JG CODEC FOR 2D OASIS-NOC:PROCESS ARE MAPD INTO ADDRESSED CORES. Random core mapping Address (Y-X) Arithmetic coding 00 Bit modeling 10 Quantization 20 Formatting 01 FF, CS, RIO 11 Wavelet transform 21 Memory 01 Preprocessing 12 Color transform 10 III. OASIS 3D-NOC ARCHITECTURE A. Topology 3D Oasis-NoC, as illustrated in Fig.6, is a simple 2x2x4 mesh topology where x-addr, y-addr and z-addr are at- Figure 6. 3D OASIS-NoC 2x2x4 mesh topology.

4 B. Switching policy 3D OASIS-NoC adopts wormhole switching. Each flit is s where the first bit indicates the tail and the next seven bits are dedicated to indicate the Next-Port direction. Then three bits are used to store destination information about each (xdest, ydest and zdest). Data can be found in the remaining 64 bits. Figure.7 shows more details about the 3D OASIS-NoC flit format. Each switch has an input Tail Next_Port X-dest Y-dest Z-dest Payload node to prevent from sending more flits to the destination node. Data_in_L nearly_full_out Data_in_D nearly_full_out In_port_L In_port_D Switch_all Crossbar nearly_full_in nearly_full_in Data_out_L Data_out_D 1 Bit 7 Bit 3 Bit 3 Bit 3 Bit 64 Bit Figure 7. 3D OASIS-NOC flit format. Figure 9. Stall-Go flow control mechanism. buffer for each input-port where the incoming flits from different ports are stored while waiting to be processed. Arbitration between different flits is managed using FIFO queue technique. Controlled by two signals enqueue and dequeue as described in Fig.8, the incoming flits enter the input buffer and then transmitted to the route module for routing computation. Each input buffer has four as depth (it can host up to four s flits). Buffers occupy a significant portion of router area but can imply also increase in overall performance. zaddr (3) yaddr (3) xaddr (3) data_in (81) sw_grant (1) enque (1) F I F O fifo_out (81) mux fifo_out[16:14] = zdest fifo_out[13:11] = ydest fifo_out[10:8] = xdest fifo_out[1:7] = next_port data_out[1:7] = next_port route D. Routing 3D OASIS-NoC roting is based upon X-Y-Z static routing algorithm, where the X,Y and Z coordinates are satisfied in order. X-Y-Z routing is presented as the vertically balanced routing algorithm which has the best performance, since it s simple to implement, it is free of deadlock and lifelock, and also because packet ordering is not required [17], [18], [19]. Figure.10 demonstrates that the routing process at each switch can be defined by three main pipeline stages: 1) Routing calculation: At each input-port for each switch, an s flit input signal is fetched and decoded in order to extract the information about the destination address and the Next-Port direction. The router then compares the actual addresses of the router and the destination addresses to define the New-Next-Port: mux 1 b0 0 b0 Figure 8. mux Input-port scheme. C. Stall-Go Flow Control. In the case where FIFO buffer exceeds its limitation on hosting flits (if the number of flits waiting for process are greater than the depth of the buffer), a flow control has to be considered to prevent from buffer overflow. 3D OASIS-NOC uses Stall-Go flow control as represented in Fig.9, which proves to be a low-overhead efficient design choice showing remarkable performance comparing to the other flow control schemes [16]. It communicates with the switch allocator module to give grant or send a stop signal to the source data_out (81) stop_out (1) sw_req (1) port_req (7) if xdest is larger than xaddr then New-Next-Port will be EAST. In the opposite case New-Next-Port will be WEST. if ydest is larger than yaddr then New-Next-Port will be NORTH, else New-Next-Port will be SOUTH. if zdest is larger than zaddr then New-Next-Port will be UP, and if this condition is not satisfied New-Next-Port will be DOWN. if xdest is equal to xaddr, ydest is equal to yaddr and zdest is equal to zaddr then New-Next-Port will be SELF. 2) Switch allocation: After the routing calculation stage, the information about the New-Next-Port is transmitted to the switch allocator as shown in Fig.11. The main function of the switch allocator is to decide which output port should be granted to which input port, and when this grant should be allocated. When several flits compete for the same output port, 3D OASIS-NoC arbiter in the switch allocator

5 sw_grant (7) xaddr (3) yaddr (3) zaddr (3) Local switch_req (7) port_req (35) Second pipeline stage: Switch allocation data_sent (7) data_in_l (81) Switch_allocator South tail_sent (7) data_in_s (81) North control (49) Tail Sent data_in_n (81) data_in_w (81) data_in_e (81) West East Crossbar Third pipeline stage: Crossbar traversal data_out_l (81) data_out_s (81) data_out_n (81) data_out_w (81) data_out_e (81) data_out_u (81) data_out_d (81) West data_in (567) data_in_u (81) East stop_out (7) data_in_d (81) stop_in (7) First pipeline stage: Routing calculation Figure 10. 3D OASIS-NoC pipeline stages: (1)Routing calculation, (2) Switch allocation and (3) Crossbar traversal. Sw_cntrl_reg 7 / mux-out-l data_out_l (81) control (49) mux-out-n data_out_n (81) mux-out-e data_out_e (81) mux-out-s data_out_s (81) mux-out-w data_out_w (81) mux-out-u data_out_u (81) data_in (567) mux-out-d data_out_d (81) Figure 12. Crossbar circuit. Figure 11. Switch allocator circuit. can free the used channel so it can be exploited by another flit. schedules a round-robin scheme. This scheme allows for each request to be served in a fair way, without taking into consideration priority. 3) Crossbar traversal: The switch allocator sends the sw-cntrl to the crossbar circuit where information about the selected input port and the Next-Port are embedded. Depending on the information, the crossbar sends flits to the appropriate ports as illustrated in Fig.12. When all the flits are transmitted, the tail bit informs the switch allocator via a tail-sent signal that the flit transmission is completed and IV. EVALUATION A. Hardware complexity 3D OASIS-NoC is designed in Verilog HDL. We compared the complexity in terms of area, power and speed between the two systems architectures of OASIS-NoC (2D and 3D architectures). Table.2 presents the parameters used for the synthesis for both 2D and 3D OASIS-NoC designs, and Table.3 illustrates the hardware evaluation results obtained comparing the two designs. The results show that the logic utilization is increased by 52%, compared to the 2D-NoC design. The increased number of ALUTs can be explained

6 by the additional number of ports and buffers at each switch. In term of clock speed 3D OASIS-NoC under-performs the 2D-NoC architecture by 8.5% on average with a small 1.74% power overhead. Table II SIMULATION PARAMETERS. OASIS-NoC Parameters 2D 3D Network Size 4x4-mesh 2x2x4-mesh Buffer Depth 4 4 Flit Size 28bit 33bit Header Size 12bit 17bit Payload Size 16bit 16bit Switching Wormhole Wormhole Flow control Stall-Go Stall-Go Scheduling Round-robin Round-robin Routing Static X-Y Static X-Y-Z Target Device Altera Stratix III Altera Stratix III Table III COMPARISON BETWEEN 2D AND 3D OASIS-NOC HARDWARE COMPLEXITY. Architecture Area (ALUTs) Power(mW) Speed(MHz) Balance Speed Area 2D D Table IV DELAY COMPARISON BETWEEN 2D AND 3D OASIS-NOC. 2D 3D Improvement Node (Y-X) Delay (ns) Node (Z-Y-X) Delay (ns) % % % % B. Performance evaluation To evaluate our 3D-NoC architecture, first we randomly generate flits and we send them from different source nodes to one destination node in order to estimate the delay caused by this traffic and then compare it with the 2D-NoC design. Table.4 illustrates the evaluation results. The nodes 00 (2D- NoC) and 000 (3D-NoC) are selected as the destination node and four source nodes are selected to represent the possible longest path in the network. The table illustrates the delay between the time of sending all the flits and the time when the destination node receives them all. The last column represents the improvement obtained with our proposed architecture when compared with the 2D-NoC design. It s shown that the 3D-NoC design reduces the delay with about 22% in overall when compared with the 2D-NoC architecture. V. CONCLUSION AND FUTURE WORK 3D OASIS-NoC is a natural extension of the 2D-NoC design previously developed by our group. In this paper we present a hardware design for 3D OASIS Network-on-Chip, and also present a preliminary evaluation results. Evaluation results show that in term of speed 3D OASIS-NoC underperforms the 2D OASIS-NoC architecture observing a small increase in power overhead and the area utilization penalty. Despite the increasing hardware complexity, 3D OASIS- NoC shows an improvement in term of latency by reducing the delay to 22% in overall compared to the 2D OASIS- NoC. In future we will try to optimize the routing algorithm in order to enhance the performance of our design. We will also simulate 3D OASIS-NoC with real workloads like the JG application previously tested with 2D OASIS-NoC in section II to evaluate the real performance of our system. REFERENCES [1] L. Benini and G. De Micheli, Networks on Chips: A New SoC Paradigm, IEEE Computer, Jan. 2002, pp [2] P. Magarshack and P.G. Paulin, System-on-Chip beyond the Nanometer Wall,h Proceedings of 40th Design Automation Conf. (DAC 03), ACM Press, 2003, pp [3] A. Ben Abdallah, M. Sowa, Basic, Network-on-Chip Interconnection for Future Gigascale MCSoCs Applications: Communication and Computation Orthogonalization, Proc. of The TJASSST2006 Symposium on Science, DEC [4] G. Philip, B. Christopher, and P. Ramm, Handbook of 3D Integration: Technology and Applications of 3D Integrated Circuits, Wiley-VCH, [5] L. P. Carloni, P. Pande, and Y. Xie, Networks-on-chip in emerging interconnect paradigms: Advantages and challenges, In Proceedings of the 3rd ACM/IEEE International Symposium on Networks-on-Chip (NOCS09), San Diego, CA, May 2009, pp [6] A. W. Topol, J. D. C. La Tulipe, L. Shi, D. J. Frank, K. Bernstein, S. E. Steen, A. Kumar, G. U. Singco, A. M. Young, K. W. Guarini, and M. Ieong, Three-dimensional integrated circuits, IBM Journal of Research and Development, vol. 50, no. 4/5, pp , July [7] B. Feero, P. Pratim Pande, Performance Evaluation for Three- Dimensional Networks-on-Chip, Proceedings of IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 9th- 11th May 2007, pp [8] V. F. Pavlidis, E.G. Friedman, 3-D Topologies for Networkson-chip, IEEE Transactions on VLSI Systems, Oct. 2007, pp [9] K. Mori, A. Ben Abdallah, K. Kuroda, Design and Evaluation of a Complexity Effective Network-on-Chip Architecture on FPGA, Proc. of The 19th Intelligent System Symposium (FAN 2009), pp , Sep [10] N. Kavaldjiev, G.J.M. Schmit and P.G. Jansen, A virtual Channel router for On-chip Networks, IEEE International SoC conference, September 2004, pp [11] J. Rosethal, JG Image Compression Using an FPGA, Master of Science in Electrical and Computer Engineering, University of California Santa Barbara DEC

7 [12] M. J. Gormish, D. Lee, M. W. Marcellin, JG-2000: Overview, Architecture and Applications, Proceedings of ICIP-2000, September 2000, vol.2. [13] Zhang, H. and Fritts, J.,EBCOT coprocessing architecture for JG2000, SPIE Electronic Imaging - Video Communications and Image Processing 2004, San Jose, CA, pp [14] C. J. Glass and L. M. Ni, The Turn Model for Adaptive Routing, in Proc.19th Ann. Int l Symp. Computer Architecture, May 1992, pp [15] J. Hu and R. Marculescu, Exploiting the Routing Flexibility for Energy/Performance Aware Mapping of Regular NoC Architectures, in Proc. DATE 03, 2003, pp [16] A. Pullini, F. Angiolini, D. Bertozzi and L. Benini, Fault tolerance overhead in network-on-chip flow control schemes, Proceedings of the 18th annual symposium on Integrated circuits and system design, Florianolpolis, Brazil, September 04-07, 2005, pp [17] K. Lahiri, A. Raghunathan, and S. Dey, Efficient Exploration of the SoC Communication Architecture Design Space, in Proc. IEEE/ACM ICCAD 00, 2000,, pp [18] K. Dev, Multi-Objective Optimization using evolutionary Algorithms, John Wiley and Sons Ltd, 2002, pp [19] C. H. Chao, K. Y. Jheng, H. Y. Wang, J. C. Wu, and An-Yeu Wu, Traffic- and thermal-aware run-time thermal management scheme for 3D NoC systems, in Proc. ACM/IEEE Int. Symp. Networks-on-Chip (NoCS), Grenoble, France, May 2010, pp