Display Management: Outline

Transcription

1 Display Management: Sensing User Intention and Context (HOTOS 2003) Outline Motivation and Research Objective FaceOff Architecture and Prototype Evaluation Best Case Feasibility Study Responsiveness Study Future Work Related Work ACPI Dark Windows 2 1

2 Motivation Current energy management techniques tied to process execution Can we use low power sensors to match I/O behavior more directly to user behavior and reduce system energy consumption? Sensing User Intention and Context for Energy Management 3 Case Study: FaceOff Displays: Typically responsible for large power drain Power State can be controlled by software State transition strategies naïve A display is only necessary if someone is looking at it. 4 2

3 Image Capture Face Detector No Face=off Main Control Loop Face=on 5 Prototype IBM ThinkPad T21 running RedHat Linux Base Power Consumption = 9.6 Watts Max CPU = 8.5 Watts over Base Display = 7.6 Watts Logitech QuickCam Web Cam Power Consumption = 1.5 Watts Software components: Image capture, face detection, display power state control 6 3

4 Face Detection Skin detection used for prototype Real time proprietary methods exist 7 Outline Motivation and Research Objective FaceOff Architecture and Prototype Evaluation Best Case Feasibility Study Responsiveness Study Future Work Related Work ACPI Dark Windows 8 4

5 Best Case Feasibility Study What is the potential for energy savings? Assume perfect accuracy Best case user behavior start it and leave. Tradeoff of energy costs: CPU/Camera vs. Display Effect on System Performance Network file transfer (113 MB) CPU intensive process (Linux kernel compile) MP3 Song (no display necessary) 9 File Transfer Traces 10 5

6 Kernel Compile Traces 11 Energy and Time Comparisons Energy (J) Default With FaceOff % Savings Compile Transfer Time (s) Default With FaceOff % Overhead Compile Transfer

7 MP3 Application Playing an MP3 Display not necessary Song completes before default timeout turns off display Energy comparison 3,403 J with FaceOff vs. 4,714 J with Default 28% energy savings No noticeable effect on playback 13 Responsiveness Study Use full prototype including skin detection Establish baseline timing Examine Responsiveness varying system load varying polling rate 14 7

8 Responsiveness Timing polling latency detection latency Face arrives (or departs) Image acquired Total responsiveness latency detection complete display signaled 15 Baseline Detection Latency Measured over a period of one hour with no programs other than background processes running Latency increased over time Started at ~110ms Increased to ~160ms Why? Appears to be an effect of Linux scheduler reducing priority of long running jobs 16 8

9 Detection Latency over Time 17 Detection Latency Under Load Workload Average Maximum Minimum (99% Confidence) Network Transfer 175±7ms 305ms 116ms Kernel Compile 230±5ms 669ms 51ms MP3 Song 154±3ms 229ms 84ms 18 9

10 Outline Motivation and Research Objective FaceOff Architecture and Prototype Evaluation Best Case Feasibility Study Responsiveness Study Future Work Related Work ACPI Dark Windows 19 Varying Polling Rate Reduce overhead by reducing polling rate Increases responsiveness latency Adaptive polling rate Eliminate polling in presence of UI events Begin polling as duration without UI events increases and face is detected Reduce polling when no face present Similar problem with latency increase upon return 20 10

11 Optimization with Motion Sensor Combine adaptive polling & motion sensing Meet responsiveness requirements with minimal FaceOff system overhead Eliminate image polling when no motion Switch display state on immediately when motion detected and restart image polling 21 Implementation Prototype using X10 ActiveHome Wireless Motion Sensor and Receiver Receiver connects to serial port Reading port blocks until sensor triggers Takes up to 10 seconds to recharge Promising addition to FaceOff system 22 11

12 More Roles for Sensors Touch Sensor Detect picking up of a PDA Light, Sound sensors Adjust display brightness (Compaq ipaq) Adjust speaker volume Signal Strength sensor Determine possibility of offloading computation 23 Enhanced Sensors Active Camera Perform some or all of the face detection Color filtering Preprocessing skin color segmentation Low Power processor for external sensor control, computation 24 12

13 Discussion: Other ideas for using sensors to save energy? Future Work Continue work on optimizing responsiveness Comprehensive user study Survey of usability Characterization of usage patterns End-to-end experiment Implementation with available very low power camera/motion sensor and prototype for small device (handheld) 26 13

14 Conclusions Context information offers promising method of energy management FaceOff illustrates feasibility of approach Available very low power sensors as well as optimization techniques would improve upon the FaceOff energy savings 27 Outline Motivation and Research Objective FaceOff Architecture and Prototype Evaluation Best Case Feasibility Study Responsiveness Study Future Work Related Work ACPI Dark Windows 28 14

15 Related Work Display Power Management Industry Specifications APM, ACPI, DPMS Zoned Backlighting Energy-Adaptive Display System Design Attentive/Perceptual UIs Smart Kiosk System: Gesture analysis CAMSHIFT: Game control IBM PupilCam: Head gesture recognition 29 ACPI Advanced Configuration and Power Initiative Brought to you by Intel, Microsoft, and Toshiba and designed to enable OS Directed Power Management (OSPM). Goal is to be able to move power management into software for more sophisticated policies Abstract OS-HW interface Replaces APM interface 30 15

16 What ACPI Offers Standardization industry-wide (Vendors to support ACPI in products instead of building their own power mgt) System and device power states Thermal model Thermal zones, indicators, cooling methods BIOS interfaces Motherboard configuration tables Interpreted control methods Plug-and-play Complexity moved into OS 31 What ACPI Offers System Mechanisms for putting computer as a whole in sleep/wake states Devices ACPI tables describe motherboard devices Power states Controls for managing states Processor Detecting idle state and swapping to low power Batteries Querying and controlling battery behavior 32 16

17 Power States G: global states apply to entire system and are visible to user D: states of individual devices S: sleeping states within the G1 state Legacy C: CPU states G2-S5 Soft off G3 mech off wakeup G0-S0 working S2 S3S4 S1 G1 Sleep Dx modem x Dx HDD x Dx cdrom x Cx cpu 33 ACPI Internal Structure Applications Kernel OS Dependent Application APIs OSPM System Code Device Driver ACPI Driver/ AML Interpreter OS Specific technologies, interfaces, and code. Existing industry standard register interfaces to: CMOS, PIC, PITs,... ACPI Register Interface ACPI Registers ACPI BIOS Interface ACPI BIOS ACPI Table Interface ACPI Tables OS Independent technologies, interfaces, code, and hardware. Platform Hardware BIOS - ACPI Spec Covers this area. - OS specific technology - Hardware/Platform - Provided by ACPI CA 34 17

18 Transmeta Crusoe ACPI Power States 35 SetSystemPowerState initiate sleep state, query apps(?) SetThreadExecutionState OSDM: OnNow specifies level of support needed Applications (e.g. display required) WM_POWERBROADCAST a message notifying of power state changes to which applications can OS respond SetWaitableTimer ACPI Spec ensure PC is awake at scheduled time HW RequestDeviceWakeup RequestWakeupLatency - to specify latency requirements GetSystemPowerStatus and GetDevicePowerState OnNow WIN32 ext 37 18

19 Outline Motivation and Research Objective FaceOff Architecture and Prototype Evaluation Best Case Feasibility Study Responsiveness Study Future Work Related Work ACPI Dark Windows 38 Energy-Adaptive Display System Designs for Future Mobile Environments S. Iyer, L. Luo, R. Mayo, P. Ranganathan, MOBISYS

20 Opportunity New technology: OLEDs Organic Light Emitting Diodes Energy consumption on a per-pixel basis by determining each pixel s brightness and color Energy consumption of different regions of screen to be changed independently No separate backlight In development by Kodak, Sanyo, Sony, 40 Dark Windows Software support modifications to windowing system to ensure energy is spent mostly on window-of-focus (capturing user s area of visual interest) Non-active screen area is changed (dimmed or re-colored for energy optimization) 41 20

21 Justification Usage study what are the user s needs and how well do they match display characteristics? Characterization of display usage in Microsoft Windows by 17 typical users Application logger recorded, for up to 14 days, when user was active. Window of focus (the one accepting keyboard input) its size, location, title Size of total screen area used (all non-minimized windows 42 Screen Usage Results On average only 59% of area used by window-of-focus, additional 17% by background windows High variability among users Large fraction of smaller windows have very low content (notifications, alerts) don t need full color characteristics of display to convey it

22 Dark Windows Design Prototyped on X Windows under Linux Used VNC Virtual Network Computing Server provides a virtual representation of display virtual framebuffer where pixels can be manipulated between application and display Track window-of-focus, apply modifications to pixels outside of it 44 Modifications Half Dimmed areas outside window-offocus are dimmed to 50% brightness Fully Dimmed areas outside are turned off Gray Scale areas outside are changed to gray by setting rgb to average value. Green Scale areas outside are changed to green which is lowest power color for OLEDs. Dims to 67% 45 22

23 46 Evaluation Energy benefits measured by generating synthetic trace from usage study and playing trace on prototype. 15 inch OLED displays were not available so they used a software power model to calculate power consumption Controller power set to 0.5W Driver power 1W 1024x768 pixels individually consume -- red power 4.3µW, green power 2.2µW, blue power 4.3µW. User experience users want to see background but willing to use dark windows 47 23

24 Results based on default Teal Background* * Benefits depend on original Choice of background and window colors. 48 Conclusions While benefits depend on usage scenario, significant energy savings can be achieved with these optimizations Further opportunities for application specific adaptivity More meaningful notions of area of focus can be defined (e.g. most recent message, most recently changed region of screen) Better match to (low) content e.g., notifications could be audible signal instead of popup window 49 24

25 Discussion: Wish List what can hardware engineers do to enable/assist software energy/power management? Information exchanged between HW/SW Hooks available to SW Granularity of independently managed devices For more information: Milly Watt Project People: Angela Dalton, Xiaobo Fan, Alvy Lebeck, Amin Vahdat, Heng Zeng s: cs.duke.edu Info:

26 Thank You! 26