The (Low) Power of Less Wiring: Enabling Energy Efficiency in Many-Core Platforms Through Wireless NoC (Invited Paper)

Transcription

1 The (Low) Power of Less Wiring: Enabling Energy Efficiency in Many-Core Platforms Through Wireless NoC (Invited Paper) Partha Pratim Pande, Ryan Gary Kim, Wonje Choi School of EECS, Washington State University, Pullman, WA, USA {pande, rkim, Abstract During the last decade, we have witnessed a major transition from computation- to communication-centric design of integrated circuits and systems. In particular, the network-on-chip (NoC) approach has emerged as the major design paradigm for multicore systems-on-chip (SoC). The major challenges in traditional wire-based NoCs are the high latency and power consumption of the multi-hop links. By inserting single-hop long-range wireless links in place of multi-hop wired links, the overall system performance can be significantly improved. We should adopt novel architectures inspired by the on-chip wireless links to design highperformance multi-core chips. In this regard, the small-world network-inspired wireless NoC (WiNoC) has emerged as an enabling interconnection infrastructure to design highbandwidth and energy-efficient multicore chips. In this paper we present the various challenges and possible solutions for designing energy-efficient massive multicore chips enabled by the WiNoC paradigm. Keywords NoC, VFI, Dynamic V/F, Wireless, Multicore, Low-power. I. INTRODUCTION The design of high-performance massive multicore chips is dominated by power and thermal constraints. Indeed, increased power consumption not only raises the chip temperature and cooling cost, but also decreases chip reliability and performance. Consequently, it is imperative to come up with new interconnection architecture and power management strategies. The small-world network-enabled wireless NoC (WiNoC) has emerged as an enabling interconnection infrastructure to design high-bandwidth and energy-efficient multicore chips [1]. In this WiNoC architecture, long-range communication predominately takes place through the wireless shortcuts, whereas the short-range data exchange occurs through conventional metal wires. This results in performance advantages (lower latency and energy dissipation) mainly stemming from using the wireless links as long-range shortcuts between far apart cores. Incorporating suitable power management strategies will further enhance the improvement in energy efficiency of WiNoC-enabled multicore chips. Multiple Voltage Frequency Island (VFI) partitioning presents one such power management strategy for multicore chips [2]-[6]. Indeed, by tailoring the voltages and frequencies of each VFI domain, we can achieve significant Zhuo Chen, Diana Marculescu, Radu Marculescu Department of ECE, Carnegie Mellon University Pittsburgh, PA, USA {tonychen, dianam, radum}@cmu.edu energy savings subject to specific performance constraints. Most of the existing VFI-partitioned designs use the conventional multi-hop mesh-based NoC architecture. However, for large-scale systems, the inter-vfi data exchanges through traditional mesh NoCs introduce unnecessary latency and energy overheads. The multi-vfi platform represents a natural fit for the WiNoC architecture where the synchronous buffers in some of the NoC routers can simply be replaced with mixed-clock/mixed-voltage FIFOs as needed. In this paper, we discuss how by integrating the WiNoC and VFI paradigms in a synergistic manner, we can design energy-efficient multicore platforms without introducing noticeable performance penalty. II. VFI DESIGN METHODOLOGY Generally speaking, VFI in multicore systems is desirable from two perspectives: 1) Reducing the overall communication cost; and 2) Improving the energy efficiency by producing better opportunities for voltage and frequency (V/F) scaling. From the perspective of the overall network cost, the communication between two cores within the same VFI is significantly cheaper than communication between two cores residing in two different VFIs. In the first case, the exchange of data requires fewer hops and no mixed clock and voltage interfacing. Thus, it is desirable that the subsets of cores that heavily communicate with each other get clustered into the same VFI. From the perspective of energy efficiency, it is also desirable to cluster together those cores that have similar utilizations. This way, cores running similarly behaved workloads can share the same V/F pair; this can help reduce the number of such distinct pairs without violating the performance constraints. A. VFI Clustering There are three primary ways of implementing VFIbased clustering, namely: 1) clustering based on communication, 2) clustering based on core utilization and 3) hybrid clustering that takes both utilization and communication into account. The utilization-based VFI clustering minimizes the intra-vfi core utilization variation. Communication-based clustering encapsulates the inter-core communication within clustering with best effort. Hybrid clustering uses both communication and utilization to achieve the best of both worlds.

2 B. V/F Tuning After VFI partitioning, we can determine the static V/F level of each VFI such that it minimizes the power consumption under a certain performance constraint. Power and performance models, such as those proposed in [7], can be used to accomplish this. However, the application characteristics, core utilization and traffic information, tend to vary throughout the runtime of every application. Therefore, static V/F tuning, although simple, tends to be suboptimal. Hence, we should take advantage of the temporal variations in the application by dynamically tuning the V/F of each VFI. C. VFI Interfaces In this VFI-enabled system, each island can work with its own voltage and frequency. As such, communication across different VFIs is achieved through mixedclock/mixed-voltage (MCMV) first-input first-output (FIFO) interfaces. This provides the flexibility to scale the frequency and voltage of various VFIs in order to minimize the overall energy consumption [8], [9]. Fig. 1: WiNoC: Small-world network architecture with short- and long-range links. III. WINOC ARCHITECTURE SUPPORTING VFI The goal of on-chip communication network design is to transmit data with low latencies and high throughput using the least possible power and resources [10]. Modern complex network theory [11] provides a powerful method to analyze network topologies. Between a regular, locally interconnected mesh network and a completely random Erdös-Rényi topology, there are other classes of graphs, such as small-world and scale-free graphs. Small-world graphs have very short average path length, defined as the number of hops between any pair of nodes. The average shortest path length of small-world graphs is bounded by a polynomial in log(n), where N is the number of nodes, making them particularly interesting for efficient communication with minimal resources [12]. NoCs exhibiting small-world characteristics can perform significantly better than locally interconnected mesh-like networks [13], yet require far fewer resources than a fully connected system. These properties of small-world networks make them desirable for VFI-based systems. Moreover, WiNoCs are particularly attractive for creating these smallworld networks since the long-distance shortcuts typically found in these types of networks can be realized using highbandwidth, low-energy, wireless interconnects while the local links can be designed with traditional metal wires. A. Small-World Connectivity The topology of the WiNoC is a small-world network where the links between routers are established following a power law distribution. More precisely, the probability P(i,j) of establishing a link between two routers i and j, separated by a Euclidean distance l ij, is proportional to the distance l ij raised to a finite power as in: l Pi, ( α j ij f ij (1) )= i jl ij α f ij The frequency of traffic interactions between cores i and j, f ij, is also factored in, so that frequently communicating cores have a higher probability of having a direct link inserted between them. This frequency is the percentage of traffic generated by core i that is destined for core j. This approach implicitly optimizes the network architecture for a non-uniform traffic scenario. Getting into details, the parameter governs the nature of connectivity, e.g., a larger increases the probability of a locally connected network with a few, or even no longrange links. By the same token, a zero value of generates an ideal small-world network following the Watts-Strogatz model one with long-range shortcuts that are independent of the distance between the cores. It has been shown that a value of less than D + 1, D being the dimension of the network, ensures the small-world property; with 1.8, the average hop count is minimized with a fixed wiring cost [14]. As long metal wires are costly both in terms of power and latency, we propose to use wireless links to connect the routers that are far apart. In practice, depending upon the available wireless resources, we can only allow a limited number of long links in the WiNoC to be wireless, while the others would remain wireline. This way, we can make the distant cores socialize with each other, and hence reduce the communication costs when running real applications. To get some intuition, Fig. 1 represents such a WiNoCbased VFI system with 16 cores where each core is associated with a router (not shown for clarity). This architecture has many short-range (local) links, as well as a few long-range links schematically represented by the arching interconnects. As mentioned above, depending on available resources, a limited amount of these shortcuts will be implemented by using millimeter (mm)-wave wireless links operating in the GHz range [1]. We note that the long-range wireless links can play important roles in achieving various goals, e.g., optimizing for performance, exchanging control signals between multiple VFIs for efficient power management, etc. Moreover, unlike other interconnects such as RF-I or normal metal wires [15], the mm-wave wireless links can establish communication channels between any pair of nodes by virtue of their broadcasting capability.

3 The power law connectivity-based WiNoC is basically an irregular network topology. We assume an average number of connections,, from each NoC router to the other routers. We propose that the value of be chosen to be four so that the WiNoC does not introduce any additional router overhead with respect to a conventional mesh. Also, an upper bound, k max, need to be imposed on the number of ports attached to a particular router so that no router becomes unrealistically large in the WiNoC [16]. Due to the nature of the VFI clustering, additional constraints need to be applied to the connectivity of the WiNoC routers. The distribution of links should be divided into the VFI intracluster connections needed to ensure each cluster s connectivity and communication, and the VFI inter-cluster connections to enable communication between different clusters [17]. B. Wireless Link and Core Placement To help facilitate predominantly long-distance communication, we use mm-wave wireless links to communicate among distant cores. It is possible to create three non-overlapping channels with on-chip mm-wave wireless links. By extending the wireless frequency range to sub-thz, the number of non-overlapping channels can be extended even further. Using these wireless channels, we overlay the wireline small-world connectivity with the wireless links such that a few routers get an additional wireless port. Each of these wireless ports will have a wireless interface (WI) tuned to one of the three wireless channels. One WI is replaced by a gateway WI that has all three channels assigned to it; this facilitates data exchange between the non-overlapping wireless channels. The WI placement is most energy-efficient when the distance between them is at least 7.5 mm for the 65 nm technology node [1]. The optimum number of WIs is twelve for a 64- core system size [16]. The overall performance of the WiNoC also depends on the placement of the wireless links and physical locations of the cores. As examples, we describe two possible strategies. One such strategy is to minimize the trafficweighted hop count, while another is to maximize the wireless utilization. The first methodology physically arranges the cores in their specific VFI configurations in order to minimize the distance of highly communicating cores (minimize hop-count). Effectively, the objective is to minimize the average traffic weighted hop count. In the second methodology, we aim at increasing the amount of traffic going through the wireless links (maximize wireless usage). The wireless nodes are placed near the center of each VFI cluster. This allows most of the cores to have wireless access opportunities while maintaining the minimum distance requirement for energy efficient wireless transmissions compared to corresponding wireline links. Then the physical core placement uses this knowledge and the idea of logically near, physically far (by using the wireless links) to place highly communicating cores closer to the WIs. It is shown in [18] that, for a 64-core system divided in to four equal VFI clusters, the maximized wireless usage methodology improves the overall Energy- Delay-Product (EDP) of the system compared to the minimized hop-count strategy. C. Routing and Flow Control As the WiNoC has an overall irregular topology, it is essential to design and optimize suitable deadlock-free routing mechanisms for it. Routing in irregular networks is more complex, because routing methods are typically topology agnostic. Hence, it is necessary to investigate suitable routing mechanisms for small-world networks. Routing in irregular networks can be classified into two broad categories, viz., rule- and path-driven strategies [19]. Rule-driven routing is typically done by employing a spanning tree for the network. Messages are routed along this spanning tree with specific restrictions to achieve deadlock freedom. Because deadlock freedom is taken into account first for these routing strategies, minimal paths through the network for every source-destination pair cannot be guaranteed [19]. Conversely, for path-driven routing, minimal paths between all source-destination pairs are first guaranteed and then deadlock freedom is achieved by restricting portions of traffic from using specific resources such as the virtual channels [19]. The first routing strategy for WiNoC that we consider is an up/down tree-based routing algorithm, belonging to the rule-based classification. This routing strategy utilizes a multiple tree roots (MROOTS)-based mechanism [20]. MROOTS allows multiple routing trees to exist, where each tree routes on a dedicated virtual channel. Hence, traffic bottlenecks can be reduced in the upper tree levels that are inherent in this type of routing. An allowed route never uses an up direction along the tree after it has been in the down path once. In addition, a packet traveling in the downward direction is not allowed to take a shortcut, even if that minimizes the distance to the destination. Hence, channel dependency cycles are prohibited, and deadlock freedom is achieved [20]. The second routing strategy is an adaptive layered shortest-path routing (ALASH) algorithm [21], which belongs to the path-based classification. ALASH is built upon the layered shortest path (LASH) algorithm, but has more flexibility by allowing each message to adaptively router paths, letting the message choose its own route at every intermediate router. The LASH algorithm takes advantage of the multiple virtual channels in each router port of the NoC routers in order to route messages along the shortest physical paths. In order to achieve deadlock freedom, the network is divided into a set of virtual layers, which are created by dedicating the virtual channels from each router port into these layers. The shortest physical path between each source-destination pair is then assigned to a layer such that the layer s channel dependency graph remains free from cycles. We have demonstrated that ALASH provides similar or better performance compared to MROOTS, while offering an improved temperature profile for WiNoC [16]. In the WiNoC, data is transferred via a flit-based, wormhole routing. Between a source-destination pair, the

4 Fig. 2. Inter-paradigm benefits between WiNoC, VFI and V/F Tuning paradigms wireless links, through the WIs, are only chosen if the wireless path reduces the total path length compared to the wireline path. This can potentially give rise to hotspot situations in the WIs. Many messages will try to access the wireless shortcuts simultaneously, thus overloading the WIs, which would result in high latency and energy dissipation. Token flow control [22] is used to alleviate overloading at the WIs. Tokens are used to communicate the status of the input buffers of a particular WI to the wireline routers, giving messages a chance to reroute if the WI is congested. An arbitration mechanism is designed to grant access to the wireless medium to a particular WI, including the gateway WI, at a given instant to avoid interference and contention between the WIs that have the same frequency. To avoid the need for centralized control and synchronization, the arbitration policy adopted is a wireless token passing protocol. It should be noted that the use of the word token in this case differs from the usage in the above mentioned token flow control. The wireless token passing protocol here is a simple media access control (MAC) mechanism to access the wireless channels. According to this scheme, the particular WI possessing the token can broadcast flits into the wireless medium in its respective frequency. A single flit circulates as a token in each frequency channel. All other WIs of the same channel will receive the flits, but only the WI whose address matches the destination address will accept the flit for further processing. The wireless token is released and forwarded to the next WI operating in the same frequency channel after all flits belonging to a message at a particular WI are transmitted. Packets are rerouted, through an alternate wireline path, if the WI buffers are full or if it does not have the token. As rerouting packets can potentially lead to deadlock, a rerouting strategy similar to Dynamic Quick Reconfiguration (DQR), as presented in [23], is used to ensure deadlock freedom. In this situation, the current WI becomes the new source for the packet, which is then forced to take a wireline only path to the final destination, still following the original routing strategy restrictions. IV. OVERALL INTEGRATION So far we have described how the VFI, WiNoC and V/F tuning can be co-designed to create an optimized fullsystem. Fig. 2 summarizes the key ideas that are traded among these three paradigms. When implementing VFIs, we can cluster highly communicating cores to aid the WiNoC and cluster cores with similar application characteristic variations to aid V/F tuning. WiNoCs can be designed to take into account the shape and characteristics of the VFI in order to reduce the overall system performance degradation inherent in VFIs. Lastly, V/F tuning can take advantage of the application slack present in both VFIs and WiNoCs to optimize the energy with low performance penalties. V. CONCLUSION In this paper we have discussed that by incorporating WiNoCs, VFIs and VF tuning in a synergistic manner, it is possible to design energy efficient multicore chips without introducing significant performance overhead. Through the use of VFI-WiNoCs, it is possible to save significant fullsystem energy-delay product (EDP) over traditional non- VFI Mesh architectures. As such, we have highlighted the importance of an integrated design approach involving VFI, V/F tuning and wireless NoC to achieve energy efficiency for multicore chips. ACKONWLEDGEMENT This work was supported in part by the US National Science Foundation (NSF) grants CCF , CNS , CNS and CCF as well as Army Research Office grant W911NF REFERENCES [1] S. Deb et al., "Design of an Energy-Efficient CMOS-Compatible NoC Architecture with Millimeter-Wave Wireless Interconnects," Computers, IEEE Transactions on, vol.62, no.12, pp.2382,2396, Dec [2] U. Y. Ogras, R. Marculescu, D. Marculescu, Eun Gu Jung, "Design and Management of Voltage-Frequency Island Partitioned Networks-on-Chip," Very Large Scale Integration (VLSI) Systems, IEEE Transactions on, vol.17, no.3, pp.330,341, March [3] S. Rusu, et al., A 45nm 8-core enterprise xeon processor, in Proc. of A-SSCC, 2009, pp [4] B. Stackhouse, et al., A 65 nm 2-billion transistor quad-core itanium processor, IEEE J. Solid-States Circuits, vol. 44, no. 1, 2009, pp [5] F. Friedrich, et al., Design of the power6 microprocessor, in Proc. of ISSCC, 2007, pp [6] H. Mair et al., A 65-nm mobile multimedia applications processor with an adaptive power management scheme to compensate for variations, in Proc. of VLSIC, 2007, pp [7] D.-C. Juan, et al. "Learning the optimal operating point for manycore systems with extended range voltage/frequency scaling." Hardware/Software Codesign and System Synthesis (CODES+ ISSS), 2013 International Conference on. IEEE, [8] T. Chelcea, and S. M. Nowick, A low latency FIFO for mixedclock systems, Proceedings of IEEE Computer Society Workshop on VLSI, April [9] D. M. Chapiro, Globally asynchronous locally synchronous systems, PhD thesis, Stanford University, 1984.

5 [10] R. Marculescu et al. Outstanding research problems in NoC design: system, microarchitecture, and circuit perspectives, IEEE Trans. on CAD, vol. 28, Jan. 2009, pp [11] D. J. Watts and S. H. Strogatz, Collective Dynamics of Small- World Networks. Nature. 393, [12] T. Petermann and P. De Los Rios. Physical realizability of smallworld networks. Physical Review E, 73:026114, [13] U. Y. Ogras and R. Marculescu, It s a small world after all: NoC performance optimization via long-range link insertion, IEEE Trans. Very Large Scale Integr. Syst., vol. 14, no. 7, 2006, pp [14] T. Petermann and P. De Los Rios, Spatial small-world networks: a wiring cost perspective, arxiv:cond-mat/ v2. [15] S. Deb, A. Ganguly, P. P. Pande, B. Belzer, and D. Heo, Wireless NoC as Interconnection Backbone for Multicore Chip: Promises and Challenges, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 2, No. 2, June 2012, pp [16] P. Wettin et al., "Design Space Exploration for Wireless NoCs Incorporating Irregular Network Routing," Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions on, vol.33, no.11, pp.1732,1745, Nov [17] R. Kim et al. "Energy-efficient VFI-partitioned multicore design using wireless NoC architectures," Compilers, Architecture and Synthesis for Embedded Systems (CASES), 2014 International Conference on, vol., no., pp.1,9. [18] R. G. Kim et al., "Wireless NoC for VFI-Enabled Multicore Chip Design: Performance Evaluation and Design Trade-offs," Computers, IEEE Transactions on, (in press) [19] J. Flich et al., A survey and evaluation of topology-agnostic deterministic routing algorithms, IEEE Trans. On Parallel and Distributed Systems, vol. 23, no. 3, 2012, pp [20] H. Chi and C. Tang, A deadlock-free routing scheme for interconnection networks with irregular topology, in Proc. of ICPADS, 1997, pp [21] O. Lysne, et al., Layered routing in irregular networks, IEEE Trans. On Parallel and Distributed Systems, vol. 17, no. 1, 2006, pp [22] A. Kumar, L.-S. Peh, and N.K. Jha, Token flow control, in Proc. of MICRO, 2008, pp [23] F.O. Sem-Jacobsen and O. Lysne, Topology agnostic dynamic quick reconfiguration for large-scale interconnection networks, in Proc. of CCGrid, 2012, pp