How to get realistic C-states latency and residency? Vincent Guittot

Size: px

Start display at page:

Download "How to get realistic C-states latency and residency? Vincent Guittot"

Osborn Lewis
5 years ago
Views:

1 How to get realistic C-states latency and residency? Vincent Guittot

2 Agenda Overview Exit latency Enter latency Residency Conclusion

3 Overview

4 Overview PMWG uses hikey960 for testing our dev on b/l system Cluster off and residency values in DT binding were looking really high: CPU off (big and LITTLE) LITTLE cluster off Big cluster off Entry latency (us) Exit Latency (us) Residency time (us) Decided to find a way to check the correctness of the figures How to easily get realistic figures for the C-states table of my platform? Without expensive materials Without deep knowledges in power management and idle states Define values for a platform or check current values

5 Prepare : Exec Exit Exec Entry Idle Entry : Cache maintenance Abortable Prepare C-state latency HW & SW sequence to enter idle step Not abortable Exit : HW & SW sequences needed to bring back CPU to running state * Read Documentation/devicetree/bindings/arm/idle-states.txt for details

6 How to measure latency? Trigger contentions Compete for accessing critical resources Look for worst values Trigger slowest path Cache flush for entering latency

7 Test environment CPU isolation Boot CPU in little cluster Interruptions pinned to CPU0 Lot of spurious activity pinned on little cluster Sync wake up of CPUs Range of wake up periods Log events and phases duration Hikey960 Use rt-app Isolate CPUs from external noise and background activity Works great for big cluster Not enough for little cluster Modified for accessing VDD_4V2 voltage domain Arm Energy Probe USB dongle

8 Exit latency

9 1st test: exit latency Enable only 1 state to force cpuidle Not fully robust Wake up CPUs simultaneously Timer IRQ CPU0 CPU1 CPU2 CPU3 rt-app logs wake up latency Get min, max, average and std-dev Read clock

10 1st test: Max 95% Min

11 1st test:

12 1st test: exit latency One CPU wakes up faster than others Frequency of other cluster impacts exit latency Around 200us on the 2900us budget Sync wake up with other cluster has a limited impacts latency Flatten the difference between min and max OPP +400us for max OPP when other cluster runs at lowest OPP Local frequency has a limited impact at the end Most probably the one that gets one lock first Few dozen of us Firmware mode has an impact Release vs debug mode

13 All latencies Big cluster off slower than LITTLE cluster off Most probably more thing are shut down compared to little Like powering down power domain Measured latency includes full wake up path timer interrupt fires (at almost the programmed timestamp as the granularity of the timer is 52ns) PM coprocessor HW wakes up sequence (when involved) ATF firmware resume sequence (when involved) cpuidle driver cpuidle framework Idle thread including starting/stopping tick nohz idle Switching to rt-app thread Read time clock big cluster CLUSTER exit 2900 CPU 550 little cluster WFI CLUSTER CPU 650 WFI 100

14 Entry latency

15 2nd test: entry latency Timer IRQ Enable only 1 state to force cpuidle Not fully robust CPU0 Timer IRQ rt-app logs phases duration Get min, max, average and std-dev CPU0 Timer IRQ Increase the sleep duration step by stepcpu0 Phase duration entry latency Timer IRQ CPU0 Timer IRQ CPU0

16 2nd test: entry latency (single cpu) sleep duration becomes longer than entry latency Spurious wake up that can be discarded

17 2nd test: entry latency (multi cpu) sleep duration becomes longer than wake up latency 1st abort point

18 2nd test: entry latency Wake up duration includes rt-app task events Entry latency Extra sleep time Exit latency Steps in charts Show the different abortable points big cluster CLUSTER entry 900 CPU 400 little cluster WFI CLUSTER ~0 500 CPU 400 WFI ~0

19 All latencies big cluster CLUSTER entry CPU little cluster WFI CLUSTER CPU WFI ~ exit wake up

20 Residency time

21 Residency time Minimum idle time above which it s worth selecting the C-state Exec Prepare C-state residency Exit Exec Entry Idle Estimated idle duration Select longest residency time Exec Wakeup latency Prepare Exec Idle Skip some C-states Exec Exec Idle

22 How to estimate residency time? Measure precisely each step independently Imply Having access to all power domains Having very precise power meters (some steps are short, transient and difficult to measure) Don t really care of absolute value Energy consumed during each step of each state Isolate CPUs power domain from others Just want to compare idle states to each others Don t really care about power impact of each step Only interested by end results

23 How to estimate residency time? Wake up periodically the CPU and measures power consumption Task don t do anything else than wake up and sleep Power impact is mainly entry/exit sequence With decreasing periods, entry and exit steps take more and more importance Run the same number of wakeup/sleep sequence Thousands of times Relax power meters precision constraint Don t need to have access to dedicated power domain Only interested in difference Side and noise power consumption will be removed as long as stable across tests

24 How to estimate residency time? Use rt-app to generate periodic wake up Task don t do anything else than wake up and sleep Run thread with a decreasing period Minimize impact of background activity of other cluster(s) 10ms down to 1ms with a step of 0.5ms has been used for hikey960 Enable only WFI use lowest OPP Run long enough (20 seconds) and several times (x8) Filter background activity of the system Keep iteration with min value Test is really long : more than 3 days of continuous tests for hikey960

25 3rd test: residency time Wake up latency for cluster off Break even point between cpu off and WFI Break even point between cluster and cpu off

26 3rd test: residency time Break even point between cpu off and WFI Wake up latency for cluster off

27 Residency Big cluster CLUSTER Little cluster CPU WFI CLUSTER CPU WFI Lowest OPP N/A N/A Highest OPP N/A N/A

28 3rd test: residency time Residency time differs widely with OPP Understandable when we looks the static power consumption big lowest OPP: cluster off is 8% < WFI (absolute value) big highest OPP: cluster off is 25% < WFI (absolute value) Need to weight residency time value of each OPP with % saved New residency value means increase the usage on cluster off state Can see some responsiveness increases 20ms residency time for cluster off versus 16ms for display sync event Use CPU latency constraint instead: per CPU or system wide

29 Conclusion

30 Conclusion More rt-app test cases can be used: With memory event as an example Not real difference has been shown OPP has a significant impact on residency time Scripts will be publicly available soon Run tests and gather results Next step Automate charts creation Automate entry, exit, and residency values extraction

31 Thank You #HKG18 HKG18 keynotes and videos on: connect.linaro.org For further information:

Realtek Ameba-1 Power Modes

Realtek Ameba-1 Power Modes Table of Contents 1 Power State... 3 1.1 Deep Sleep Mode... 3 1.2 Deep Standby Mode... 4 1.3 Sleep Mode... 4 1.3.1 Wakeup from sleep mode by UART... 4 1.3.1.1 Solution A, select