Embedded Software Streaming via Block Stream

Transcription

1 Embedded Software Streaming via Block Stream A Dissertation by Pramote Kucharoen Dissertation Advisor Professor Vincent J. Mooney III 7 April 2004

2 Outline Introduction Related Work Block Streaming Stream-Enabled Program Files Stream-Enabled File I/O Performance Enhancement Experiments and Results Conclusion 2

3 Introduction Package Download & Remote execution Software Streaming 3

4 Running Applications On clients (download and install) Advantages Less servers computing requirements Faster user interaction after installation complete Disadvantages Long download time Clients limited resources Unused features downloaded On servers (remote execution) Advantages Applications not downloaded Less clients computing requirements Disadvantages Overloading of servers Slower user interactions Connection loss 4

5 Software Streaming

6 Software streaming Definitions A method for allowing the execution of streamenabled software even while transmission/streaming of the software may still be in progress Application load time the amount of time from when the application is selected for download to when the application can be executed Application suspension time the amount of time from when the application is suspended due to missing code to when the application can be resumed 6

7 Outline Introduction Related Work Block Streaming Stream-Enabled Program Files Stream-Enabled File I/O Performance Enhancement Experiments and Results Conclusion 7

8 Java Related Work (1) Allows execution without obtaining all classes Sends class files when requested Suspends the application for every class not in memory Requires JVM Assumes clients has enough memory to store the entire application 8

9 Related Work (2) Software caching [CH02] Has high occurrence of application suspension Liquid software [JH96] Requires a fast ( gigabit ) compiler 9

10 Related Work (3) Function/module streaming [CK98][UR01] Allows execution without obtaining all functions Transfers functions speculatively to reduce the occurrence of application suspension Assumes clients has enough memory to store the entire application More difficult to manage memory 10

11 Block streaming Related Work (4) Divides files into blocks Streams at block level No known prior work 11

12 Related Work (4) Block Streaming Java Function/Module Streaming application load time application suspension Direct Download Software Caching resources Remote Execution 12

13 Outline Introduction Related Work Block streaming Stream-Enabled Program Files Stream-Enabled File I/O Performance Enhancement Experiments and Results Conclusion 13

14 Block Streaming Does not require virtual machines, virtual file system, compiler at client, special support from OS Uses a binary rewriting technique Supports embedded applications and small memory footprint devices Implements at user level (not OS dependant) Provides stream-enabled file I/O support 14

15 Objectives To reduce application load time To reduce application suspension time and occurrence of application suspension due to missing code/data To efficiently utilize resources such as bandwidth and memory To support small memory footprint embedded devices To optimize the above four objectives simultaneously (as opposed to tradeoffs) 15

16 High Objective tradeoffs occurrence of application suspension application load time, resources Low Send when requested Send the entire application Data transfer 16

17 Block Streaming: Stream Units Stream Units 0 Stream-Enabling Stream-Enabling Info Info Block Block Program File/Data File File 1 Stream-Enabling Stream-Enabling Info Info Block Block 2 Stream-Enabling Stream-Enabling Info Info Block Block 17

18 Softstream Client/Server Model 18

19 Softstream Protocol Layers OSI Layers: Stream-enabled Application Softstream Assembly Softstream Protocol TCP TCP IP IP Subnetwork Application Presentation Session Transport Network Link Physical 19

20 Transmission Profile Is created using a profiling method Is used to send stream units j=0 j= i=0 i=

21 Continuous stream Flow Control (1) Sends stream units according to the transmission profile Restarts a new sequence when requesting a missing stream unit 21

22 Flow Control (2) On-demand stream Sends stream units according to the transmission profile based on resource constraints 22

23 Overhead Bandwidth Memory Processing time Performance Metrics Application load time Application suspension time Occurrence of application suspensions due to missing code/data 23

24 Outline Introduction Related Work Block Streaming Stream-Enabled Program Files Stream-Enabled File I/O Performance Enhancement Experiments and Results Conclusion 24

25 Stream-Enabled Program Files Code generation Code modification Code profiling 25

26 System Overview Divide Divide Binary Binary Image Image into into Blocks Blocks Request Request Stream Unit Stream Unit Generate Generate Stream Stream Units Units Not in Memory Receive Block Create Create a a Transmission Transmission Profile Profile Link Link Stream Stream Block Block Off-block Load Load Stream Stream Block Block Accept Accept Request Request Encounter Off- Block Branch In Memory Program Entry Point Send Send Stream Stream Unit Unit Read Read Transmission Transmission Profile Profile Run Run the the Application Application Server Client 26

27 Generating Blocks Binary Image Stream-Enabled Application Source Code #include #include <stdio.h> <stdio.h> int int main() main() int int i; i; GCC Stream-Enabling Stream-Enabling Info Info Stream Unit Stream-Enabling Stream-Enabling Info Info Stream Unit Stream-Enabling Stream-Enabling Info Info Stream Unit 27

28 Exiting and Entering a Block bl bl comp comp comp: comp: stwu stwu r1,-31(r1) lwz lwz r0,8(r1) cmpwi cmpwi r0,1 r0,1 bne bne.l3.l3 li li r0,0 r0,0 stw stw r0,8(r31) b.l4.l4.l3:.l3: sc sc li li r0,1 r0,1 stw stw r0,8(r31) li li r3,0 r3,0.l4:.l4: blr blr branch exception after executing the last instruction return Off-block branch: Branch instruction that may cause the CPU to execution an instruction in a different block 28

29 Code Generation Preventing the execution of non-existing code Static branches Dynamic branches (return, function pointer) Exception instructions Last instruction of the block Coping with non-interruptible sections Stream execution code prior to the current block Generating stream-enabling information Location of the off-block branches Branch number assigned sequentially 29

30 Code Modification Load time code modification Modifies off-block branches to jump to the branch table Stores in off-block branch information in Branch Info Table Run time code modification Modifies the off-block branches to jump to the intended code 30

31 Code Modification Example (1) if (i==1) i=0; else i=1; cmpwi cmpwi r0,1 r0,1 bne bne.l3.l3 li li r0,0 r0,0 stw stw r0,8(r31) b.l4.l4.l3:.l3:.l4:.l4: li li r0,1 r0,1 stw stw r0,8(r31) 31

32 Code Modification Example (2).L3:.L3:.L4:.L4: cmpwi cmpwi r0,1 r0,1 bne bne load2_1.l3 li li r0,0 r0,0 stw stw r0,8(r31) b load2_2 li li r0,1 r0,1 stw stw r0,8(r31) b load3_0 load2_1: load2_2: load3_0: 32

33 Code Profiling Program Entry Point Program Exit Point

34 Recap: Block Streaming for Program File Divide Divide Binary Binary Image Image into into Blocks Blocks Request Request Stream Unit Stream Unit Generate Generate Stream Stream Units Units Not in Memory Receive Block Create Create a a Transmission Transmission Profile Profile Link Link Stream Stream Block Block Off-block Load Load Stream Stream Block Block Accept Accept Request Request Encounter Off- Block Branch In Memory Program Entry Point Send Send Stream Stream Unit Unit Read Read Transmission Transmission Profile Profile Run Run the the Application Application Server Client 34

35 Outline Introduction Related Work Block Streaming Stream-Enabled Program Files Stream-enabled File I/O Performance Enhancement Experiments and Results Conclusion 35

36 Stream-Enabled File I/O To reduce file I/O operation latency Motivation A game application renders a 1MB scene 36

37 Stream Units for Data Block Generate stream units by dividing file into fixed size blocks Create transmission profile by profiling data Provide SIO function calls sio_open() sio_read() sio_write() sio_lseek() sio_close() 37

38 Block Table ID N-1 Address 0x x xFFFFFFFF 0x xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0x x xFFFFFFFF 0xFFFFFFFF 0x : 0x : 0x : 0x : 0x : 0x : 0x : 0x : 0x : 0x : 0x : 0x : 38

39 Data Profiling Example (1) 1 MB data file, data sorted in ascending order Divide the file into 10 equal-size blocks A database application searches for a record using a binary search algorithm Create a transmission profile according to the binary search algorithm Assume that the record is in block 5 39

40 Data Profiling Example (2) 1 MB

41 Recap: Block Streaming for Data File Divide Divide File File into into Blocks Blocks sio_open() sio_open() Generate Generate Stream Stream Units Units Create Create a a Transmission Transmission Profile Profile Not in Memory sio_lseek() sio_lseek() Not in Memory Request Request Stream Stream Unit Unit sio_write() sio_write() Send Stream Send Stream Unit Unit Accept Accept Request Request Read Read Transmission Transmission Profile Profile sio_read() sio_read() Receive Block Load Load Stream Stream Block Block sio_close() sio_close() Server Client 41

42 Outline Introduction Related Work Block Streaming Stream-Enabled Program Files Stream-Enabled File I/O Performance Enhancement Experiments and Results Conclusion 42

43 Objectives Performance Enhancement To reduce occurrence of application suspensions To support small memory footprint embedded devices Code transformation Enforcing block boundaries Remapping functions Steam unit removal Unlinking mechanism Stream unit replacement Requirement Fixed sized stream blocks 43

44 Enforcing Block Boundaries Example fn1: fn1: stwu stwu 1,-31(1) stw stw 3,8(1) lwz lwz 0,8(1) blr blr fn2: stwu 1,-31(1) stw 3,8(1) fn3: li 0,1 blr fn1: fn1: stwu stwu 1,-31(1) stw stw 3,8(1) lwz lwz 0,8(1) blr blr fn2: stwu 1,-31(1) stw 3,8(1) fn3: li 0,1 blr 44

45 Observations Remapping Functions Programmer places functions in the file randomly Compiler places the functions the same order as written Program jumps from block to block Higher occurrence of software suspensions 45

46 Remapping Function Example int int fn1( ) fn1( ) { x = fn5(a,b); fn5(a,b); } int int fn2( ) fn2( ) { } int int fn3( ) fn3( ) { } int int fn4( ) fn4( ) { } int int fn5( ) fn5( ) { y = fn7(z); fn7(z); } int int fn6( ) fn6( ) { } int int fn7( ) fn7( ) { } int int fn8( ) fn8( ) { } int int fn9( ) fn9( ) { } 46

47 Linking Unlinking Mechanism Run efficiently, no code checking Unlinking Remove blocks Need to know location of incoming branches to the block to be removed 47

48 Unlinking Mechanism Example cmpwi cmpwi r0,1 r0,1 bne bne.l3 load2_1.l3 li li r0,0 r0,0 stw stw r0,8(r31) b load2_2 load2_1: load2_2:.l3:.l3:.l4:.l4: li li r0,1 r0,1 stw stw r0,8(r31) 48

49 Objective Stream Unit Replacement Reduce number of retransmissions Example Game application (e.g., Doom) 6 MB 6 blocks, 1MB each PDA with 3 MB memory available 3 Blocks, 1 MB each 49

50 Stream Unit Replacement Example Execution profile: Transmission profile: Client memory: occurrences of application suspension for demand loading potentially 6 occurrences with block streaming 50

51 Outline Introduction Related Work Block Streaming Stream-Enabled Program Files Stream-Enabled File I/O Performance Enhancement Experiments and Results Conclusion 51

52 Experiments and Results Hardware setup Stream-enabled program file (SPF) Stream-enabled file I/O (SIO) Stream-enable program with stream-enabled file I/O 52

53 Simulation Environment VCS VCS Seamless CVE CVE XRAY XRAY Main processor I/O processor MPC750 MPC750 MPC750 MPC750 Address/Data Bus Memory Memory MPC750: 400 MHz Bus: 83 MHZ Memory: 16 MB 53

54 MBX860 Board Environment Network Cloud 10Mbps PC: Linux Traffic Shaper MBX860: PowerPC 860, 40MHz 10BaseT Ethernet 4 MB DRAM, 2 MB Flash Linux

55 Code Size Implementation softstream server softstream client softstream loader/linker stream-enabled file I/O softstream generator C lines Server: softstream server softstream generator Client: softstream client softstream loader/linker stream-enabled file I/O 55

56 SPF Overhead (1) Overhead per off-block branch Bandwidth 4 bytes (location of the branch) Memory 20 bytes (12 bytes for block table, 4 bytes for instruction, 4 bytes for location) Type of overhead Bandwidth Memory Overhead per off-block branch 4 bytes 20 bytes 56

57 Overhead per block SPF Overhead (2) Bandwidth: 12 bytes for headers + 4*n Memory: 20*n n = number of off-block branches 57

58 SPF Simulation Scenario Adaptive autonomous robot exploration Impossible to write and load software for all possible environments The mission control needs to update the robot software over a 128Kbps link The new code is 10MB The robot does not need all 10MB initially The robot must run the software to react to the new environment within 120 s 58

59 SPF Simulation Results Block size (bytes) Total # of blocks Added code/block Load time (s) 10M % M % M % M % M % K % K % K % %

60 SPF MBX860 Board Results Application Load Time (s) K 10K 100K 0.5M 1M 2M 5M 10M Block Size (bytes) 60

61 Stream-Enabled Program File Results Sending the whole software takes over 10 minutes: the deadline is missed Using software streaming with the first blocks of size of 1MB, the new software can be executed within 66 seconds: the deadline is met The application load time improves by a factor of 10X Function streaming can potentially achieve the same result But function streaming lacks file I/O support 61

62 Stream-Enabled File I/O (SIO) Overhead per block Bandwidth 8 bytes (Stream Unit ID, Data Size) Memory 4 bytes (Entry in Block Table) 62

63 File size 1 MB Benchmarks SIO Board Experiment (1) Seq: read data sequentially Rand 1K: read 1KB randomly Stat: calculate various statistical values of distinct pieces the data BSearch: find a specific value in the file using a binary search algorithm Implementations DD using Linux TCP 1.0 for NET4.0, NFS version 3, SIO 63

64 SIO Board Results (1) Time (s) Seq Rand 1K Stat BSearch SIO NFS DD Up to 55X faster 64

65 SIO Board Experiment (2) Data acquisition Read a certain amount of data from a 1 MB file Link speed 128 Kbps 65

66 SIO Board Results (2) Time to acquire a certain amount of data Time (s) Data (Kbytes) SIO NFS DD 66

67 SIO Board Experiment (3) Data utilization (Kbytes per second) How fast data being consumed Process a 1 MB file using various data utilization rates Link speed 128 Kbps 67

68 SIO Board Results (3) The amount of time it takes to process a 1 MB file Time (s) Data Utilization Rate (KB/s) SIO NFS DD 68

69 Stream-Enabled Application Experiment Game application Program size 512 KB Data size 1 MB 128 KB code and 256 KB data needed for the first scene Stream-enabled program file (SPF) embeds data inside program Implementation: SIO+SPF, SPF, NFS, DD 69

70 Stream-Enabled Application Results User Perceived Application Load Time (s) SIO+SPF SPF NFS DD 70

71 Conclusion Reduce application load time by enabling execution while transferring (10X) Lower application suspension time by profiling Reduce the occurrence of application suspensions Support small memory footprint embedded devices Provide stream-enabled file I/O (55X) 71

72 Publications Kuacharoen, P. and Mooney, V., Memory management for embedded devices using software streaming, to be published in Proceedings of the Mobility Conference & Exhibition, Aug Akgul, B., Mooney, V., Thane, H., and Kuacharoen, P., Hardware Support for Priority Inheritance, in Proceedings of the IEEE Real-Time Systems Symposium, pp , Dec Kuacharoen, P., Mooney, V., and Madisetti, V., Software streaming via block streaming, in the book Embedded Software for SoC, edited by Jerraya, A., Yoo, S., Verkest, D. and Wehn, N., Boston, MA: Kluwer Academic Publishers, pp , Sep Kuacharoen, P., Mooney, V., and Madisetti, V., Software streaming via block streaming, in Proceedings of the Design Automation and Test in Europe, pp , Mar Kuacharoen, P., Shalan, M., and Mooney, V., A congurable hardware scheduler for real-time systems, in Proceedings of the International Conference on Engineering of Recongurable Systems and Algorithms, pp , June Kuacharoen, P., Akgul, T., Mooney, V., and Madisetti, V., Adaptability, extensibility, and exibility in real-time operating systems, in Proceedings of the EUROMICRO Symposium on Digital Systems Design, pp , Sep Akgul, T., Kuacharoen, P., Mooney, V., and Madisetti, V., A debugger RTOS for embedded systems, in Proceedings of the 27th EUROMICRO Conference, pp , Sep

73 Patents Kuacharoen, P., Mooney, V., and Madisetti, V., Methods and systems for transmitting application software, U.S. Patent Application , Jan Kuacharoen, P., Akgul, T., Mooney, V., and Madisetti, V., Dynamic operating system, U.S. Patent Application , Apr Akgul, T., Kuacharoen, P., Mooney, V., and Madisetti, V., Debugger operating system for embedded systems, U.S. Patent Application , Apr

74 Security: Issues not Addressed Network security Stream applications from trusted site SSL Certificate Memory protection 74

75 Questions? 75