Department of Computer Science, Institute for System Architecture, Operating Systems Group. Real-Time Systems '08 / '09. Hardware.

Size: px

Start display at page:

Download "Department of Computer Science, Institute for System Architecture, Operating Systems Group. Real-Time Systems '08 / '09. Hardware."

Alyson Collins
5 years ago
Views:

1 Department of Computer Science, Institute for System Architecture, Operating Systems Group Real-Time Systems '08 / '09 Hardware Marcus Völp

2 Outlook Hardware is Source of Unpredictability Caches Pipeline Interrupt Latency Special Purpose Hardware for Embedded Real-Time Systems Low Latency Interrupt Mode Peripheral Event Controller Capture Compare Units Scratchpad Memories Real-Time Clocks Use unpredictable Hardware more predictable Cache Partitioning Real-Time Communication / Buses in separate Lecture December 4, 2008 Marcus Völp 2

3 Sources of Unpredictability Memory Subsystem TLB Caches Store Buffers store R1, 0x x ^R1 va: Cache: TLB: hit PT Walker miss pa: Level 2 - Cache Memory Controller Main Memory December 4, 2008 Marcus Völp 3

4 Sources of Unpredictability Processor Pipeline IF IF DE ISS ALU MAC MAC MAC ADD DC DC WB WB WB (ARM 11 Microarchitecture) December 4, 2008 Marcus Völp 4

5 Sources of Unpredictability Interrupt Latency Device incoming signal Interrupt Controller CPU interrupts enabled? await instruction boundary kernel entry select handler code (index into interrupt vector table) handle_irq:... return from interrupt December 4, 2008 Marcus Völp 5

6 Interrupt Response Time RTLinux Frequency log.fl.josephina.rtlinux: a Interrupt response time: Time from occurrence of interrupt to first instruction of RT Task Frequency IRQ occurence rate (µs) log.t3.josephina.rtlinux: lat_proc_sh_dynamic No parallel load (idle): 13 µs High parallel load: 68 µs (Benchmark, Cache-Flodder) Measured on Intel P4 1.6 GHz IRQ occurence rate (µs) December 4, 2008 Marcus Völp 6

7 Interrupt Response Time L4RTL (+ AS switch) Frequency log.fl.josephina.fiasco: a Interrupt response time: Time from occurrence of interrupt to first instruction of RT Task No parallel load (idle): 43 µs IRQ occurence rate (µs) log.t3.josephina.fiasco: lat_proc_sh_dynamic High parallel load: 85 µs (Benchmark, Cache-Flodder) Frequency Measured on Intel P4 1.6 GHz IRQ occurence rate (µs) December 4, 2008 Marcus Völp 7

8 Outlook Hardware is Source of Unpredictability Caches Pipeline Interrupt Latency Special Purpose Hardware for Embedded Real-Time Systems Low Latency Interrupt Mode Peripheral Event Controller Capture Compare Units Scratchpad Memories Real-Time Clocks Use unpredictable Hardware more predictable Cache Partitioning Real-Time Communication / Buses in separate Lecture December 4, 2008 Marcus Völp 8

9 Low Latency Interrupts Sampling of Data in a High Rate (Sensors, Video,...) Fast Response Times (Break Control,...) ARM IRQ + FIQ FIQ interrupts IRQ handler 5 immediately available registers for FIQ handler no need to save register content ARM Low Interrupt Latency Configuration Minimize worst case interrupt latency disable Hit-under-Miss in Cache abandon pending restartable memory OPs restart memory OP on return from interrupt software must not use multi-word load / store instructions software should avoid accesses to slow memory (device memory, strong ordering of memory accesses) Worst Case FIQ Latency (PL190 VIC) : Interrupt synchronization 3 cycles Worst case execution time of current instruction 7 cycles Entry to first instruction 2 cycles / 12 cycles December 4, 2008 Marcus Völp 9

10 Low Latency Interrupts Obtaining precise timestamps with interrupt-based sampling is difficult system load increases jitter in interrupt response time atomic sections in OS delay interrupts jitter in signal delivery (from signal source to CPU interrupt pin) buses, interrupt controller Peripheral Event Controller (PEC) executes one memory-to-memory move triggered by arriving signal Programmers Interface: counter decrement on signal, trigger CPU interrupt on 0 byte / word select select width of transfer (byte or word) source / dest increase update source / destination address source / dest address signal () { if (counter--) trigger_interrupt(); else *dest++ = *src; } => minimal response time (~3 cycles) => PEC can execute at much lower clock rates December 4, 2008 Marcus Völp 10

11 Capture + Compare Unit Problem precise timestamp of event (engine sensor triggered at cylinder position = t) generate precise events (fire the engine at t + x) interrupt handlers have too high a jitter to meet the points in time Capture Compare event interrupt handler Compare Register Capture Register = event Clock Clock December 4, 2008 Marcus Völp 11

12 System on a Chip / Chip Multiprocessor Processor Chip contains entire system CPU Cores / Peripheral Components / Memory available as masks for weaver Configure system as required Examples: Mobile Phones today: General Purpose Core (e.g., ARM) + Baseband DSP Processor Game Console (Cell): General Purpose (PPC) + special purpose Graphics Sensor + 8-bit Controller + CAN bus December 4, 2008 Marcus Völp 12

13 Scratch-pad Memory vs. Caches Synonyms: Scratchpad / on Chip / Tightly Coupled Memory Cache: transparent addressing scheme cache controller fills / replaces cachelines in parallel to CPU activity difficult to predict worst-case execution time Main Memory L2 L1 December 4, 2008 Marcus Völp 13

14 Scratch-pad Memory vs. Caches Synonyms: Scratchpad / on Chip / Tightly Coupled Memory Cache: transparent addressing scheme cache controller fills / replaces cachelines in parallel to CPU activity difficult to predict worst-case execution time Main Memory L2 L1 December 4, 2008 Marcus Völp 14

15 Scratch-pad Memory vs. Caches Synonyms: Scratchpad / on Chip / Tightly Coupled Memory Cache: transparent addressing scheme cache controller fills / replaces cachelines in parallel to CPU activity difficult to predict worst-case execution time Main Memory L2 L1 December 4, 2008 Marcus Völp 15

16 Scratch-pad Memory vs. Caches Synonyms: Scratchpad / on Chip / Tightly Coupled Memory Cache: transparent addressing scheme cache controller fills / replaces cachelines in parallel to CPU activity difficult to predict worst-case time Main Memory L2 L1 CPU 1 cycle ~10 cycles ~50 cycles December 4, 2008 Marcus Völp 16

17 Scratch-pad Memory vs. Caches Way 1 Way 2 Set = = Cache lockdown: Tag RAM Data Lock cacheway Allocate only to unlocked ways December 4, 2008 Marcus Völp 17

18 Scratch-pad Memory vs. Caches Scratchpad Memory: memory can be addressed directly (device mapped to physical memory space) 2 cycles latency / currently ~ 64 KB (more ~4 MB to come) must relocate data explicitly Compiler-based approaches: static: determine code + data hot spots + allocate scratchpad memory for hot spots dynamic: copy data to / from scratchpad memory int a[20]; int a[20]; int b[5] = {a[5], a[6], a[7], a[8], a[9]}; => for (int j = 0; j < 40; j++) for (int j = 0; j < 40; j++) for (int i = 5; i < 10; i++) for (int i = 0; i < 10; i++) a[i] *= 42; b[i] *= 42; December 4, 2008 Marcus Völp 18

19 Scratch-pad Memory vs. Caches ARM Tightly Coupled Memory: RAM TCM RAM TCM overlay normal RAM with TCM simplified cache logic for TCM memory keeps track whether RAM or TCM holds current value on miss: copies data from underlying RAM to TCM December 4, 2008 Marcus Völp 19

20 Real Time Clocks Multiple Clocks in SoC Core Cycle Count 200 MHz fine grained + fast to access accuracy ; clock skew ; subject to power management Audio Codec PCI 33 MHz/ 66 MHz Timer RTC khz ; high precision ; small clock skew need to synchronize clocks within core / system wide ~> separate lecture December 4, 2008 Marcus Völp 20

21 Outlook Hardware is Source of Unpredictability Caches Pipeline Interrupt Latency Special Purpose Hardware for Embedded Real-Time Systems Low Latency Interrupt Mode Peripheral Event Controller Capture Compare Units Scratchpad Memories Real-Time Clocks Use Unpredictable Hardware more predictable Cache Partitioning Real-Time Communication / Buses in separate Lecture December 4, 2008 Marcus Völp 21

22 OS Controlled Cache Predictability goals partition cache to minimize interference between RT Applications OS controlled transparent to application no need to rely on cooperating applications survives malicious, erroneous applications Cache Partition Address regions that map to a given set of cachelines (a partition) tool support: Compiler / Linker December 4, 2008 Marcus Völp 22

23 OS Controlled Cache Predictability Cache Coloring 16 bit address (binary representation): tag idx offset into 32 byte cacheline 0x93 Cache Size December 4, 2008 Marcus Völp 23

24 OS Controlled Cache Predictability Cache Coloring Address translation and caches virtual page number virtual address: 0x MMU physical frame number physical address: 0x30a 864 offset tag idx ofs L2 Cache: 4MB, 4 Way, 32byte CL December 4, 2008 Marcus Völp 24

25 OS Controlled Cache Predictability Cache Coloring Address translation and caches virtual page number virtual address: 0x MMU physical frame number physical address: 0x30a 864 offset tag idx ofs L2 Cache: 4MB, 4 Way, 32byte CL 2 bits: subject to address translation evaluated to determine cache set => assign different colors to different RT + non RT Apps. => allocate in the OS memory frames of respective color December 4, 2008 Marcus Völp 25

26 OS Controlled Cache Predictability Scenario high priority real-time task (color = 00) runs in frequent short intervals other tasks in the backgound (e.g., cache flodder) Example: Filter Worst case: wo. cache partitioning: background tasks may evict all cachelines background tasks may load conflicting cachelines, which need writeback Flush Flush Flush December 4, 2008 Marcus Völp 26

27 OS Controlled Cache Predictability 400% 350% Filter PentiumPro, 133 MHz 300% 250% 200% 150% unpartitioned sngl wr part partintioned 100% (sngl wr = single writer only RT app may write to pages of color x ; others may read) December 4, 2008 Marcus Völp 27

28 OS Controlled Cache Predictability Example 2: low priority task; frequently interrupted e.g., matrix multiplication Flush Flush Flush Flush timeslice 64 x 64 Matrix Multiplication 64 3 x 5.51 cycles: 10.9 ms December 4, 2008 Marcus Völp 28

29 OS Controlled Cache Predictability 1700% 1600% 1500% 1400% 1300% 1200% 1100% 1000% 900% 260% 240% 220% 200% 180% 160% 140% 120% 64 x 64 Matrix Multiplication Pentium, 133 MHz 800% 700% 600% 500% 400% 300% 100% [µs] unpartitioned sngl wr part partitioned 200% 100% December 4, 2008 timeslice [µs] Marcus Völp 29

30 OS Controlled Cache Predictability 500% 450% 400% 135% 130% 125% 120% 115% 64 x 64 Matrix Multiplication Pentium, 133 MHz write through 350% 110% 300% 105% 250% 100% [µs] 200% unpartitioned partitioned 150% 100% timeslice [µs] [µs] December 4, 2008 Marcus Völp 30

31 OS Controlled Cache Predictability Experiment 3: Combination: Matrix Multiplication + Filter 64 x 64 Matrix Multi- plication [ms] unpartitioned partiotioned MM + Filter 280% 260% 240% 220% 200% 180% 160% 140% % 100% 20 khz 10 khz 5 khz 3.3 khz 2 khz khz 10 khz 5 khz 3.3 khz 2 khz December 4, 2008 Marcus Völp 31

32 OS Controlled Cache Predictability Caveat: application transparency some applications require knowledge about physical addresses e.g., drivers need physical addresses for direct memory address transfers (DMA) modify driver use recent hardware (e.g., Intel VT-d) with address translation for DMA December 4, 2008 Marcus Völp 32

Conclusions High performance CPUs are rarely used in embedded systems HW means to increase predictability SW tweaks to use unpredictable HW in a more predictable way References: Liedtke,

33 Conclusions High performance CPUs are rarely used in embedded systems HW means to increase predictability SW tweaks to use unpredictable HW in a more predictable way References: Liedtke, Härtig, Hohmuth (RTAS '97): Operating system control cache predictability for real-time applications ARM Real View Emulation Board User Guide ARM 1176jzfs Manual December 4, 2008 Marcus Völp 33

HARDWARE IN REAL-TIME SYSTEMS HERMANN HÄRTIG, WITH MARCUS VÖLP

Faculty of Computer Science Institute of Systems Architecture, Operating Systems Group HARDWARE IN REAL-TIME SYSTEMS HERMANN HÄRTIG, WITH MARCUS VÖLP A SIMPLE PROBLEM event PC how to create a precise timestamp