Labeled RISC-V: A New Perspective on Software-Defined Architecture

Size: px

Start display at page:

Download "Labeled RISC-V: A New Perspective on Software-Defined Architecture"

Jessie Kelley
5 years ago
Views:

1 Labeled RISC-V: A New Perspective on Software-Defined Architecture Zihao Yu, Jiuyue Ma, Bowen Huang, Xin Jin, Huizhe Wang, Yaoyang Zhou, Zihao Chang, Yan Cao, Sa Wang, Yungang Bao May 9 th, ShangHai Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS) 1

2 Software-Defined Architecture Why? Agenda How? What Effect?

3 Tradeoff in Data Center: Util. vs. QoS LOW utilization[1,2]: 6%~12% Sharing causes interference Sacrifice utilization to guarantee QoS Lo et al. Heracles: Improving Resource Efficiency at Scale, ISCA, [1] [2] J. M. Kaplan, W. Forrest, and N. Kindler. Revolutionizing data center energy efficiency. McKinsey & Company,

4 More Hardware Support Needed 4

5 QoS-unawared Hardware Shared resources Unable to distinguish Unmanaged Sharing No architectural support for QoS Ambulance? 5

6 Software-Defined Architecture Why? Agenda How? What Effect?

7 Labeled Networking Fine-grain: every packet has a label Semantic association: correlate labels with users demand Propagation: propagate labels in a whole network DiffServ: process packets differentiately based on labels MPLS is widely used for VPN and QoS

8 The Computer as a Network Hardware components communicate via internal packets, e.g., PCIe packets, NoC packets, QPI packets Yes!

9 Labeled von Neumann Architecture (LvNA) 4. Software-defined control logic 2. Sematic association Input Core Core P3 P1 P0 Processor Output 1. Fine-grained object Memory 3. Propagation Bao and Wang, Labeled von Neumann Architecture for Software-Defined Cloud, Journal of Computer Science and Technology, 2017 Vol. 32 (2):

10 Add label registers to req. sources VM0 VM1 VMn Core Core Core Add label registers Shared Last Level Cache I/O Chipset Memory Controller Disk Disk Disk NIC 10

11 Sematic association VM0 VM1 VMn Core Core Core Shared Last Level Cache I/O Chipset Memory Controller Disk Disk Disk NIC 11

12 Propagation VM0 VM1 VMn Core Core Core Shared Last Level Cache CPU request I/O Chipset DMA Memory Controller Disk Disk Disk NIC 12

13 Software-defined label-based control logic VM0 VM1 VMn Core Core Core Shared Last Level Cache CL I/O Chipset CL Memory CL Controller Disk Disk Disk NIC 13

14 Programmable Architecture for Resourcing-on-Demand PARD Leverage LvNA to perform fine-grained control Ma et. al, Supporting Differentiated Services in Computers via Programmable Architecture for Resourcing-on-Demand (PARD), ASPLOS,

15 Tables as CL Cache Controller Memory Controller 15

16 Closed-loop Control stat op > < = threshold Trigger Table 1 Cond-1 Action-1 1 Cond-2 Action-2 2 Cond-3 Action-3 e.g. miss_rate > 30% Statistics Table 1 Stat1 Stat2 2 Stat1 Stat2 Parameter Table 3 1 Param1 Param2 2 Param1 Param2 3 Trigger => Action action signal + e.g. adjust way mask firmware action script 16

Controller CL CL /sys/cpa cpa0 ident type ldoms ldom0 parameter param1 param2 statistics trigger

17 Platform Resource Manager (PRM) Augmented IPMI Connect all control logics Run linux-based firmware Abstract CLs as files Core CL VM0 I/O Chipset VM1 Core VMn Core Shared Last Level Cache Memory Controller CL CL /sys/cpa cpa0 ident type ldoms ldom0 parameter param1 param2 statistics trigger ldom1 ldom2 cpa1 cpa2 Disk Disk Disk CL CL CL CL NIC Monitoring & Interrupts Programming Centralized PRM 17

18 Access Control Logics Query Control Logic Info cat /sys/cpa/cpa0/ident cat /sys/cpa/cpa0/type Query Parameters cat /sys/cpa/cpa0/ /parameter/param1 Setting Parameters echo 10 > /sys/cpa/cpa0/ /parameter/param2 /sys/cpa cpa0 ident type ldoms ldom0 parameter param1 param2 statistics trigger ldom1 ldom2 cpa1 cpa2 18

19 Software-Defined Architecture Why? Agenda How? What Effect?

Implementation Full-system cycle-accurate

20 Implementation Full-system cycle-accurate simulator Open Sourced * FPGA prototype on Xilinx VC709 evaluation board MicroBlaze version RISC-V version Deprecated Open Sourced + *

AXI4 Memory Bus AXI4 I/O Bus Labeled Intr.

2.168.1.1 (dhcp, tftp, httpd) LDom #1 LDom

Logic Memory Controller (MIG7) I/O CL Eth0

21 AXI4 Memory Bus AXI4 I/O Bus Labeled Intr. CN Bus (I 2 C-based) PC IP: (dhcp, tftp, httpd) LDom #1 LDom #2 LDom #3 LDom #4 SFP+ UART Core Control Logic Core0 Core1 Core2 Core3 Cache Control Logic Cache / XBar Memory Control Logic Memory Controller (MIG7) I/O CL Eth0 Eth1 Eth2 UART*4 CN Switch PRM Xilinx VC709 Evaluation Board 21

22 Evaluation - Performance Isolation 4 Ldoms: 1 X 429.mcf + 3 X Attacker Allocate different LLC capacities Perf. degradation: 7%(w/ PARD) vs. 48%(w/o PARD) solo attacker Attacker + T->A 22

23 LvNA + RISC-V = Labeled RISC-V Switching to RISC-V Add labels in the core easier riscv-go to run container Goal - establish Labeled RISC-V branch 23

24 LvNA + RISC-V = Labeled RISC-V Switching to RISC-V Add labels in the core easier riscv-go to run container Goal - establish Labeled RISC-V branch 24

25 LvNA + RISC-V = Labeled RISC-V Switching to RISC-V Add labels in the core easier riscv-go to run container Goal - establish Labeled RISC-V branch CL 25

26 LvNA + RISC-V = Labeled RISC-V Switching to RISC-V Add labels in the core easier riscv-go to run container Goal - establish Labeled RISC-V branch CL TL2.dsid <-> axi.user 26

27 LvNA + RISC-V = Labeled RISC-V Switching to RISC-V Add labels in the core easier riscv-go to run container Goal - establish Labeled RISC-V branch CL Address mapping Base Len 1 0x0000 0x x8000 0x8000 Labeled token bucket CL TL2.dsid <-> axi.user 27

28 LvNA + RISC-V = Labeled RISC-V Switching to RISC-V Add labels in the core easier riscv-go to run container Goal - establish Labeled RISC-V branch PRM CL Address mapping Base Len 1 0x0000 0x x8000 0x8000 Labeled token bucket CL TL2.dsid <-> axi.user 28

29 Overheads 16 lines of chisel code to add labels into RocketChip Control Logics Add dsid member in the Bundle of TileLink2 Attach labels at the TileLink2 masters of core tiles < 5% resource overheads for CLs Much more less with complex cores, e.g. BOOM 29

30 Demo 1 - NoHype Push the software hypervisor down to LvNA Isolate the resources (address space, device) by CLs Partitioned into several sub-machines Label-based CL Label-based CL Label-based CL 30

31 Demo 1 - NoHype LDom0 LDom1 31

32 Demo 2 Memory Bandwidth Control Use labeled token buckets to isolate the bandwidth attacker Solo interfered isolated 32

33 Hardware - LvNA Labels everywhere in the hardware Plan to tape out with 40nm TSMC! PRM (rocket) ROB boom + SMT CL CL CL PCI-E boom + SMT CL L2 boom + SMT CL CL boom + SMT CL MEM Application Schedule Framework Compiler Runtime Library Operating System Hypervisor Hardware... 33

34 Hypervisor - NoHype Finished Push the software hypervisor down to LvNA Remove run-time overhead VMentry, VMexit Isolated by NoHype VM 4 VM n VM 1 VM 2 VM 3 software VMM Application Schedule Framework Compiler Runtime Library Operating System Hypervisor Hardware LvNA 34

35 Operating System - Fine-grained labeling Add fine-grained label as context resource Process (integrated into Cgroup) Process/container-level Thread-level Address space Function-level Object-level Provide libraries pthread_create_with_dsid() malloc_with_dsid() Process relative labeling Finished P0 VM0 P1 Core dsid start end 1 0x8000 0xffff 3 0x2000 0x27ff Application Schedule Framework Compiler Runtime Library Operating System Hypervisor Hardware Address space labeling 35

36 Compiler - collect QoS info. from prog Express QoS info. from source files Additional compilation results Address space labeling info Extra ELF sections for loader Resource requirement QoS desc. file for schedule framework Binary QoS Desc. Application Schedule Framework Compiler Runtime Library Operating System Hypervisor dsid start end 1 0x8000 0xffff call sort SLA = 10s working set = 64KB Hardware 3 0x2000 0x27ff #progma qos(10s) sort(); 36

37 Sche. Framework - QoS resource schedule Expose QoS resources to schedule frameworks Integrate QoS resources into OpenStack Finished Application Schedule Framework Compiler Runtime Library Operating System Hypervisor Hardware 37

38 A lot to explore! Theory: How does LvNA impact on RAM, PRAM, LogP models? Finished On-going Have ideas Feature work Hardware/Arch: How to implement LvNA at CPU pipeline/smt, memory, storage, networking? How to correlate LvNA and SDN by labels? OS/Hypervisor: How to correlate labels with VMs, containers, processes, threads? How to abstract programming interfaces for labels? Programing Model and Compilers: How to express users requirements and propagate to the hardware via labels? How to make compilers support labels? Distributed systems: How to correlate labels with distributed resources? How to manage distributed systems with label mechanisms? Measurement/Audit: How to leverage labels to gauge and audit resource usages? 38

39 Summary LvNA: a model of software-defined architecture PARD: a proof of concept of LvNA Labeled RISC-V: an implementation of LvNA 39

40 Thanks Scan the QR code to join the discussion group with WeChat Labeled RISC-V: A New Perspective on Software-Defined Architecture 40

Labeled RISC-V: A New Perspective on Software-Defined Architecture

Labeled RISC-V: A New Perspective on Software-Defined Architecture Zihao Yu, Bowen Huang, Jiuyue Ma, Ninghui Sun, Yungang Bao Oct 14 th, 2017 @ Boston Institute of Computing Technology (ICT), Chinese Academy