Starfish Efficient Concurrency Support

Transcription

1 Efficient Concurrency Support for Computer Applications Robert LiKamWa Lin Zhong Rice University 1

2 In the year

3 Continuous mobile vision Oh, I haven't talked to Fred recently... Where did I place my keys? Remind me to tell Lin to let me graduate!?????????? 3

4 Continuous mobile vision Energy consumption overwhelms wearable battery! Insufficient concurrency support for vision! 4

5 Application Capture Service Headers 5

6 Application Capture Service Headers Scale Face 6

7 200+ ms 200+ ms 200+ ms Camera is overworked Computation is overworked We need efficient concurrency support! 7

8 Observations: apps utilize common vision library 8

9 Scale Face common primitives Scale Face Scale SURF Observations: apps utilize common vision library Capture/Computation is redundant across apps 9

10 Scale Face common primitives Scale Face Scale SURF Observations: apps utilize common vision library Capture/Computation is redundant across apps 10

11 Proposal: Split- process design for centralized control Scale SURF Face Key Idea: Share computations, Reduce redundancy 11

12 Core Service Uses vision headers to intercept library calls Core Service Executes, tracks, and shares library call computations 12

13 Core Service Split- process library solution for efficient, transparent multi- app service 13

14 facedetect(img) Core facedetect(img) Function Cache Arg Search Exec. Function Cache Storage/Retrieval First Call Execute call ( ms) Store results Subsequent Call(s) Skip execution (0 ms) Retrieve results 14

15 Core Cache Search Function Cache Share computations, Reduce redundancy 15

16 Core Cache Search Function Cache Timing optimizations: + Reuse "Fresh Frames to promote cache hits + Delay call return to encourage device sleep 16

17 Core Mitigating Cache Search Split- Process Overhead Function Cache + Promote sharing by reusing "Fresh Frames + Maintain code privacy by delaying call return + Manage concurrent requests through fine- grained locks 17

18 Core Cache Search Function Cache Reduce expense of argument passing Avoid library object modification 18

19 Split- Process in the Literature Object Redefinition Lightweight RPC SUN RPC Zero- copy Require library redesign Cloud Transfer COMET MAUI CloneCloud Object tracking Optimized for code offload 19

20 Split- Process Argument Passing Shared Memory Core facedetect( ) 3.5 ms Execution Goal: Minimize deep copy overhead is expensive 20

21 1) Protected shallow copy Shared Memory Core facedetect( ) CoW Execution CoW Issue copy- on- write (mprotect) on received objects 21

22 2) Direct output marshalling Shared Memory Core facedetect( ) CoW CoW Execution malloc() Write new data directly into shared memory 22

23 3) Reuse arguments from prior calls Shared Memory Core facedetect( ) CoW Argument Table (img, shm_ptr) CoW Execution malloc() Track, reuse previous inputs/outputs 23

24 (Usually) Zero- Copy Argument Passing Shared Memory Core facedetect( ) CoW Argument Table (img, shm_ptr) CoW Execution malloc() + Reduce deep copies through argument reuse + Reduce allocation through buffer reuse 24

25 Optimizations Share Computations Maintain expectations of developers & users Increase sharing efficiency by relaxing timing Decrease redundancy Reduce argument copy Reuse memory buffers 25

26 Experimental Platform OpenCV + Android + Google Glass Monsoon Power Monitor OMAP4430 dual- core Cortex- A9 pinned to 600 MHz Benchmarks: 1) Per- call micro- benchmarks 2) Multi- app benchmarks 26

27 Memory optimizations cut overhead in half Prepare&Inputs& Send&Inputs& Search&Cache& 'call'execution:'resize() Native& &Unopt.& & First Call Native : 21 ms Unopt. : 42 ms : 24ms Exec.&Function& Allocate&Outputs& Prepare&Outputs& Receive&Outputs& 0" 5" 10" 15" 20" 25" Execution&time&(ms)& 27

28 Memory optimizations cut overhead in half Prepare&Inputs& Send&Inputs& Search&Cache& Exec.&Function& Allocate&Outputs& Prepare&Outputs& 'call'execution:'resize() Native& &Unopt.& & First Call Native : 21 ms Unopt. : 42 ms : 24ms Second Call Native : 21 ms Unopt. : 10 ms : 6 ms Receive&Outputs& 0" 5" 10" 15" 20" 25" Execution&time&(ms)& works well when native function execution >> 5 ms 28

29 Places where fails 5-6 ms performance overhead per- call: Bad for fast computations Bad with large, deep arguments Non- cacheable functions Random, temporal, external dependencies Functions with specific parameters But great for many high- level functions: Face detect, Corner detect, Image resize 29

30 vs. Multi- App Workload If Lin then That Social Logger Facebook Google+ Twitter MySpace Whatsapp... Scale Face Detect Face Recog 0.3 FPS 30

31 When running multiple apps, achieves higher performance 5" 4.5" 4" 3.5" 3" Frame&Rate& 2.5" (FPS)& 2" 1.5" 1" 0.5" 31 0" Na/ve" " 1" 2" 3" 4" 5" 6" 7" 8" 9" 10" Number&of&App&Instances& Higher is better

32 When running multiple apps, draws single- app power Na.ve" " Power& Consump-on& (mw)& 2500" 2000" 1500" 1000" 500" 0" 1" 2" 3" 4" 5" 6" 7" 8" 9" 10" Number&of&App&Instances& Lower is better 32

33 When running multiple apps, draws single- app power Na.ve" " Power& Consump-on& (mw)& 2500" 2000" 1500" 1000" 500" 0" 1" 2" 3" 4" 5" 6" 7" 8" 9" 10" Number&of&App&Instances& achieves efficient concurrency support! 33

34 Core Cache Search Function Cache Share computations, Reduce redundancy 34

35 Continuing mobile vision Mitigating Analog Signal Chain Bandwidth Preserving User/Subject Privacy Designing Efficient Systems 35

36 : Share computations, Reduce redundancy Transparent, efficient library call caching Timing optimizations for frame freshness and performance preservation Memory optimizations for minimal deep copy and small cache footprint Google Glass Experiments: Low overhead Memory optimizations slash overhead in half Reduced power draw Multi- app workloads draw Single app power 36