Dynamic Backlight Adaptation for Low Power Handheld Devices 1

Transcription

1 Dynami Baklight Adaptation for ow Power Handheld Devies 1 Sudeep Pasriha, Manev uthra, Shivajit Mohapatra, Nikil Dutt and Nalini Venkatasubramanian 444, Computer Siene Building, Shool of Information & Computer Siene University of California, Irvine, CA {sudeep, mluthra, mopy, dutt, nalini}@is.ui.edu Abstrat The limited lifetime of batteries imposes stringent onstraints on the power onsumption of mobile handheld devies. Power onservation to prolong battery life is of primary importane. Typially, power onsumption depends on several fators like baklight intensity, the network interfae, the CPU and the nature of eah appliation. While signifiant researh efforts have been made to optimize power onsumption at the appliation, network and proessor levels, omparatively little work has been done to redue the power onsumed by the baklight. Our work proposes an adaptive middleware based approah to optimize baklight power onsumption for mobile handheld devies when playing streaming video. While reduing baklight power onsumption, our approah simultaneously minimizes any negative impat on the pereived video quality. Performane results for MPEG-1 video indiate that up to 60% of the power onsumed by the baklight an be saved by using the proposed approah. 1 This work was supported by funding from the NSF NGS award ACI

2 1. Introdution Modern handheld mobile devies with modest sizes and weights are inreasingly being used to provide users with streaming multimedia ontent. These devies have limited omputing, storage and battery resoures while the multimedia appliations they run tend to be extremely resoure hungry. Typially, the display, the CPU and the network interfae onstitute the three primary soures of power onsumption in suh low power devies. In this work we fous on ahieving energy savings from the baklight display of the devie, without signifiantly ompromising on the quality of the streamed video. This work was performed in the larger framework of the FORGE projet [11] that attempts to perform power and QOS optimizations aross multiple abstration layers, inluding the appliation, middleware, OS and hardware. Several shemes to optimize power onsumption of low-power devies in mobile environments have been proposed by researhers in reent years. Tehniques suh as ompiler/os/middleware based adaptation [4, 5, 11], dynami power management of network interfaes [8], disks [10] and dynami voltage saling (DVS) [2, 3] attempt to redue power onsumption at various omputational levels. However, efforts to redue the power onsumption of the baklight have reeived relatively little researh interest. Some interesting work in this area has been done by Choi et. al. [1] in an attempt to address this problem. They propose ompensating the brightness of still images while simultaneously reduing the baklight level. However, their proposed ontrast ompensation does not preserve the original olor of the image whih limits the pratial appliation of their sheme. Moreover, they implement their approah in the limited ontext of still images. In this work, we explore more aggressive approahes to brightness ompensation and devie baklight ontrol for streaming video. Furthermore, the adaptation is shifted away from the handheld devie and performed at a network proxy server, obviating the need for the deoder on the devie to be modified. We find that aggressive brightness ompensation without signifiant impat on the visual quality is possible for streaming video when ompared to still 2

3 images. This is beause artifats (introdued due to aggressive ompensation) notieable in still images are less disernable in video where several frames are displayed on the sreen every seond. We also propose an effetive brightness ompensation algorithm and integrate it with our middleware based adaptation shemes to enable us to ahieve low power baklight operation while streaming video ontent to mobile handheld devies. 2. System Arhiteture We implemented the system model depited in Figure 1. The system entities inlude a multimedia server, a proxy server that has aess to a database of (i) profiled luminosity values for every video stream at the server and (ii) handheld devie speifi parameters (e.g. number of baklight levels, average luminosity at eah level et.), a rule base to determine ompensation values and a video transoder (Figure 1); and low-power wireless devies apable of displaying streaming MPEG video ontent. Moreover, all ommuniation between the handhelds and the multimedia server are routed through the proxy server that an hange the video stream in realtime. Eah lient has an appliation layer where the video stream is deoded and a middleware layer whih routes the information flowing from and to the video deoder appliation. The video requests from the appliations are sent to the proxy by the ommuniation manager module, whih also sends devie speifi features (suh as the number of baklight levels, the urrent baklight level and information identifying the type and make of the handheld) gathered by the system monitor module. The system monitor an also diret the handheld to hange baklight levels using the appropriate API alls to the underlying OS. The ommuniation manager also reeives video streams from the proxy server and direts them to the appliation layer for display on the handheld sreen. The middleware on the proxy performs the dynami adaptation of the streaming video ontent 3

4 (brightness and ontrast ompensation) and ommuniates ontrol information to the lient middleware (operating baklight levels) through the ommuniation manager module whih interfaes with the low bandwidth ontrol stream. The ontrol stream is used (i) by the lient to send the video stream request along with urrent baklight level and other handheld speifi information to the proxy and (ii) by the proxy to diret the lient handheld devie to set a new baklight level while playing streaming video. The proxy maintains a database of information about the videos available at the server and information speifi to different handheld types suh as the number, luminous intensity and average power onsumption of the baklight levels. Additionally, the proxy also employs a stati rule base whih speifies onditions whih determine values for baklight and video ompensation. The database and ertain parameters of the rule base are populated by extensive profiling and subjetive assessment of videos on different handhelds [11]. Database Rule Base Appliation Middleware API Interfae Middleware API Interfae Middleware runtime System Monitor OS API Interfae Communiation manager middleware Video stream Control stream Middleware runtime Communiation manager Middleware runtime Video Server Hardware Driver Middleware API Interfae Client Handheld Proxy Transoder 3. Baklight Compensation Figure 1: System Arhiteture 4

5 We measured power dissipated (averaged for several different MPEG-1 video streams) at various baklight levels for the Compaq ipaq handheld devie running Windows CE. For the five baklight levels of the ipaq (Super Bright, High Bright, Medium Bright, ow Bright and Power Save), the average power dissipated was 2.80W, 2.51W, 2.32W, 2.16W and 1.72W respetively. The results indiate that signifiant energy savings are possible by operating the devie at a lower baklight intensity level. The inherent problem with proposing shemes to redue power is that the baklight diretly affets the display quality and user experiene. For example, even a slight redution in baklight intensity during multimedia playbak on the handheld (with the intention of saving power) an degrade the human pereption of quality. Indeed, simply reduing the baklight is not a viable solution. We explore the use of a video ompensation algorithm that indues power savings without notieably affeting video quality. Our prior work [11] has foused on several aspets of video quality on handheld devies. We have determined that user pereption of video playbak quality is signifiantly influened by the environment and the type of handheld used to view the video. Consequently, objetive assessment [7] of video quality is extremely diffiult and subjetive assessment [6] is still the primary method for assessing video quality. In our profiling and surveys, we attempted to follow the reommendations given in [6] and hose a diverse olletion of video streams (movie lips, animations, sports, doumentaries et) to use in assessing the suitability and effetiveness of our proposed shemes. To validate our assessments, we seleted 30 individuals to be part of an extensive survey to subjetively assess the human pereption of video quality when viewing streaming video on a handheld devie. The subjets were first shown a full sreen version of an original unaltered video stream. Next, they were shown the ompensated stream and asked to reord their observations pertaining to differenes pereived in the video quality. We then attempted several different ombinations of parameters used in our ompensation algorithm to determine the values 5

6 whih would give the most power saving without pereptibly degrading video quality. This phase was repeated for several different video streams and feedbak from the subjets was reorded. Based on our extensive analysis of the feedbak reeived, we arrived at a set of values for the rule base, luminane thresholds and parameters in our ompensation algorithm whih gave us substantial power savings with aeptable degradation in video quality. It is important to emphasize that in this work we do not intend to sarifie video quality in order to gain savings in handheld power onsumption. The following setions desribe our ompensation algorithm and the middleware adaptations used in our system. 4. Dual Compensation Algorithm In this setion, we haraterize the problem and propose our ompensation algorithm. et n be the number of baklight levels supported and P(n) be the power at eah level. Then P save = P(n) - P(n-k) denotes the power savings when dimming the baklight from level n to n-k. The pereived intensity of an image is denoted by I = Y fr where is the transmittane of the CD panel, is the baklight luminane and Y fr is the average luminane value of the frame [1]. The luminane value for a pixel (Y pix ) an be obtained from its RGB values, after applying standard onversion funtions to onvert it from RGB to the YC r C b oordinate spae [9]. et the luminane of the baklight at level n be given by and the luminane at level n-k be given by. From [1], if we derease the baklight level from n to n-k, then in order to preserve the pereived intensity, the new luminane value Y pix` for eah pixel in the frame is given by (Y pix`) = min(1, (Y pix ) + ) where (Y pix`) gives the normalized value of Y pix`, and = ( 1 `/ ) 6

7 However, pixels already having a high luminosity value annot be ompensated adequately resulting in a loss of ontrast due to saturation and we observe degradation in video quality. This limits the amount of ompensation that an be applied to an image without degrading its quality intolerably. However, the loss in ontrast an also be addressed and ompensated for, as will be explained later, whih then allows even further savings in power resulting from a larger redution in baklight intensity. Next we introdue the onept of a group of senes (GOS) whih defines the granularity at whih baklight ompensation is performed. We define a group of senes as a group of ontiguous frames in a video stream suh that the variane of the average luminosity values of eah frame belonging to the group is less than a threshold value. The average luminosity value Y fr of a frame an be alulated as Y fr = ( w 1)( h 1) Yw, h i= 0 wxh where w and h are the width and height of the image in terms of number of pixels and Y w,h is the luminosity of the pixel at (w,h). The onept of GOS is used to split a video stream into several groups of frames. These groups form the basi entities on whih ompensation is performed. Video streams in general have the property that a lot of frames having similar average luminosity values are lustered together and this provides ample sope for optimization for low power by uniformly ompensating entire GOS entities and reduing the handheld baklight level. There must be a minimum number of frames in a GOS for it to be eligible for ompensation. The reason for introduing this parameter is that there needs to be a minimum duration between hanges in baklight levels. We introdue a funtion that provides the baklight level ompensation fator, (k i,,,ygos) = k` The input parameters to are the urrent baklight level (k i ), the type of the handheld () and the average luminosity of the GOS being onsidered (Y gos ) for whih the number of frames > and 7

8 the funtion returns the optimal baklight level to be set for the GOS (k`). The average GOS luminosity value is given by n Y gos = 1 i= 0 Yavg n where n is the number of frames in the GOS. This funtion uses the extensively profiled video information for a partiular type of handheld stored in the proxy database to selet and return a suitable value for baklight level based on the value of Y gos and k i. Next we introdue another funtion, that provides the video luminosity ompensation fator (k i,, (k i,, Ygos)) = ` The input parameters for are the urrent baklight level (k i ), the type of the handheld () and the value for the next baklight level (returned by the funtion ) and it returns the brightness ompensation value for the GOS being onsidered. The return value is the differene in luminosity for the two baklight levels, for the partiular handheld devie being used as a lient. The average luminosity values for different baklight levels supported by the handheld are obtained from the database at the proxy, whih stores these default values for several handheld devies. If k g is the urrent baklight level on a handheld of type and Y GOS is the average luminosity of an eligible GOS streaming to the proxy from the video server, then a ontrol message is sent to the lient asking it to set its baklight level to (k g,,, Y gos ) = n g while at the same time the group of pitures is ompensated with a brightness of (k i, T, (k i,,,y gos )) = ` before it is sent to the lient. This results in power savings of P save = P(k g ) - P(n g ) over the time interval when the GOS is played bak on the lient. Balaning ontrast and luminosity: Inreasing the luminosity of a frame an ause a loss in ontrast between different regions in the frame, whih makes it diffiult to identify edges of objets in the frame and degrades piture quality. To overome this, we propose performing an additional ompensation step. In this step, the luminosity ompensated frame is passed through a high pass filter whih performs a spatial onvolution on the luminosity values of the frame. This 8

9 onvolution step sharpens the edges and makes objets in the frame more reognizable. A onvolution kernel is of the form k = pix where 1, 2, 8 are values arefully seleted to inrease the amplitude of the high-frequeny ontent in the frame. The onvolution kernel is not limited to a 3x3 matrix and an be larger (e.g. 5x5). Now let the luminosity value of the pixel under onsideration be pix and that of its 8 neighboring pixels be as shown: 3x3 = pix Then the modified pixel value after onvolution is given by: 8 1 pix `= * + 8 pix pix k = 1 k = 1 k k * k if + 8 pix k k = 1 > 0 and 8 pix `= pix * pix + k = 1 k * k if + 8 pix k k = 1 = 0 An alternative to the high pass filter is a median filter whih is a non-linear filter that also sharpens a frame and also possesses the additional property of noise tolerane. This is generally harder and more time onsuming to implement than the high pass filter desribed above. Our 9

10 experiments have shown that this partiular filter produes images that are of the same quality as those produed by the high pass filter for the video streams onsidered. 5. Proxy based middleware adaptation shemes We now present three middleware adaptation poliies that utilize the ompensation algorithm. The first two poliies an be implemented with limited operating system interfae ontrol and the third is our proposed dual ompensation algorithm that requires both proxy and OS interfaes for optimal operation. Simple Baklight Compensation (SBC): Using this poliy, power an be saved by identifying GOS entities for whih Y GOS is high (above a threshold level ) and reduing the baklight level at the lient for these GOS entities. Note that the proxy simply alulates new baklight levels without ompensating the brightness of the video stream. The proxy has a reord of the GOS entities in the video stream and whenever it expets to be sending a GOS with Y GOS >, it sends ontrol information to the lient to redue its baklight level, suh that the pereived differene in quality is minimal. This sheme has the disadvantage that it degrades video quality in every ase and is attrative only beause of its simpliity (no video ompensation performed). Constant Baklight with Video uminosity Compensation (CBVC): A more interesting and pratial approah is to set a onstant baklight level at the start of the video stream and then dynamially ompensate different GOS entities based on their Y GOS values. However, sine the baklight value is fixed for the entire duration of the video, the level has to be hosen onservatively so that video quality is not affeted adversely if there are dramati variations in luminous intensity of onseutive GOS entities in the video stream. This is a stati approah and would provide a onstant power saving (depending on the initial baklight level hosen), with no dynami adaptation of baklight intensity levels. The limitations of the previous approahes an 10

11 be overome with a hybrid dual ompensation approah desribed below. Dual Compensation Approah (DCA): In this tehnique, we simultaneously ompensate the video stream and the baklight levels for different GOS entities. The proxy dynamially ompensates the GOS entities in the video stream and begins streaming the video to the lient, simultaneously direting the lient to hange its baklight level through the ontrol stream. The lient middleware sets the appropriate baklight intensity levels for the video playbak. This approah provides more flexibility for aggressive optimizations with muh greater power savings. 6. Performane Evaluation Parameter Desription Value Minimum number of 60 frames in GOS Variane threshold for 40 k Y GOS Threshold level for SBC sheme Convolution kernel used in high pass filter Table 1: Parameter values for ompensation algorithm used in experiments To evaluate these approahes, we onduted experiments with several different MPEG-1 video streams and a Compaq ipaq 3600 series mobile handheld devie, whih omes with a olor refletive thin film transistor CD sreen with a pixel pith of.24mm and a display resolution of 240 x 320 pixels. 11

12 MPEG Video Name Resolution FPS Duration (se) uminosity Variation Video Type bipolar.mpg 320 x ittle Dark, 3D animation ieegg.mpg 240 x Moderate Bright, 3D animation intro.mpg 160 x Very High Flashy, TV show lip simpsons.mpg 192 x High Colorful, 2D animation Table 2: Charateristis of video streams used in experiment The parameters we used for our ompensation algorithm are given in Table 1. In addition, the rule base for determining (k x ) and (k x ) for the five baklight levels (k 0, k 1, k 2, k 3 and k 4 ) for the Compaq ipaq is given below (k 0 ) = k 0 ; (k 0 ) = 0 ; (k 1 ) = k 1 ; (k 1 ) = 0 ; for all Y GOS for all Y GOS (k 2 ) = k 1 ; (k 2 ) = 30; for Y GOS < 140 (k 3 ) = k 2 ; (k 3 ) = 30; for 80 < Y GOS < 190 = k 1 ; (k 3 ) = 55; for 190 < Y GOS < 220 (k 4 ) = k 3 ; (k 4 ) = 30; for 190 < Y GOS <

13 = k 2 ; (k 4 ) = 55; for 60 < Y GOS < 190 = k 1 ; (k 4 ) = 65; for Y GOS < 60 The above rule base, the luminane threshold values and the ompensation algorithm parameters in Table 1 were determined after extensive profiling and subjetive assessment of several video streams [11] beause no analytial method exists for deriving them. The values were hosen to ahieve substantial power savings without signifiant degradation in video quality and stored in the database at the proxy. Our ompensation algorithm then queries these parameters for different GOS entities to determine the amount of ompensation to perform. It should be noted that our approah is not limited to the Compaq ipaq and is valid for other handhelds with fewer or greater number of baklight levels than the ipaq (suh as the Palm Tungsten and HP Jornada handheld series). Eah handheld type would possess its own unique rule base, luminane threshold values and ompensation algorithm parameter alulated using extensive profiling and assessment, just like we did for the ipaq. We analyzed the effetiveness of the three shemes desribed in the previous setion for several different video streams of different durations (number of frames), resolutions (frame width, height) and types (ontaining dynamially/statially hanging senes). Due to lak of spae we present the results for a limited set of streams. Table 2 shows the video streams used and also desribes them in detail. For all our experiments, we assume that the initial level of the handheld is set to a brightness level that takes into aount the brightness of the lient s environment, suh that the quality of the video playbak is deemed aeptable by the user at the set baklight level. Our shemes are not appliable if the initial baklight setting is low or power save and this information is aptured in the rule base desribed earlier. We used a National Instruments PCI DAQ board to sample voltage drops aross a resistor and the ipaq, and sampled the voltages at 200K samples/se to determine variations in power onsumption of the ipaq. The instantaneous and average power is alulated using the following formulae 13

14 P = V ipaq * V R / R Power saving (mwatts) SBC CBVC DCA Power saving (mwatts) SBC CBVC DCA 0 ieegg simpsons intro bipolar 0 ieegg simpsons intro bipolar (a) (b) 180 Power saving (in mwatts) ieegg simpsons intro bipolar SBC CBVC DCA () Figure 2: Power savings for initial referene point of (a) super bright, (b) high bright and () medium bright The first experiment assumes that the initial baklight level of the handheld is set to super bright. Figure 2 (a) shows the results for this ase. It is easy to see that the DCA sheme outperforms the other two shemes in all the ases. The CBVC sheme outperforms the simplisti SBC sheme in all ases exept for intro.mpg. This video has a very high variation of average luminous intensity from one frame to the next and very high intensity values for some frames. Consequently there are very few GOS entities eligible for ompensation in this ase. Sine the CBVC sheme sets the lient baklight level just one (in the beginning), lowering the baklight level an result in signifiant degradation of quality for many of the frames in this ase, whih the ompensation annot retify. As a result, the CBVC sheme takes the onservative approah and does not request the lient to lower its baklight level from its preset value. It is 14

15 interesting to note that for the ase of bipolar.mpg, the SBC sheme does not provide any power saving. This is beause the video is very dark (low luminane) on an average and there is no GOS entity with a value of Y GOS >. Consequently, for this video, the baklight level is never redued in this sheme. Figures 2 (b) and () show the results when we assume that the initial baklight level of the handheld is set to high bright and medium bright respetively. As we lower the initial baklight levels, we expet that the sope for reduing power onsumption in the handheld dereases, whih an be seen from the results. For the experiment with the initial level set to medium bright, it is interesting to note from the figure that both the CBVC and sheme DCA shemes perform more or less the same. An exeption to this observation is the ase of intro.mpg (for reasons mentioned earlier). The reason for the similar performane of the two shemes is that the DCA sheme an only redue the baklight by 1 level at most further redution makes it diffiult to maintain quality even with video ompensation. This fat is refleted in our rule base. The CBVC sheme also manages to lower the baklight level by 1 level for all the ases exept intro.mpg and hene performs similar to the DCA sheme. Overall, it an be seen from these experiments that the DCA sheme performs just as well as the other two shemes in a few ases, but performs muh better ompared to them in a majority of the ases. The power savings for the DCA sheme ranges from 100 mw to 625 mw depending on the type of video being played and the initial baklight setting, whih is a range that orresponds to roughly 9% to 60% redution in power onsumed by the baklight on the mobile handheld lient. 7. Conlusion For the Compaq ipaq, the baklight an onsume as muh as 40% of the total power when playing streaming MPEG video. To redue the ontribution of the baklight to overall power 15

16 onsumption, we proposed reduing the handheld baklight level while simultaneously ompensating the video stream by inreasing frame luminosity. We onvolved video frames with a high pass filter to minimize impat on piture detail after aggressive luminosity ompensation. We presented three middleware adaptation tehniques that utilize a proxy server to implement our approah. Our experiments with several different MPEG-1 streaming videos on the Compaq ipaq handheld show that the dual ompensation approah (DCA) whih is a hybrid of the other two proposed middleware adaptation approahes is effetive in reduing as muh as 60% of the power onsumed by the baklight on the handheld for ertain video streams. 8. Referenes [1] Inseok Choi, Hojun Shim and Naehyuk Chang, ow-power Color TFT CD Display for Hand-Held Embedded Systems, In ISPED, [2] C. J. Hughes et al. Saving Energy with Arhitetural and Frequeny Adaptations for Multimedia Appliations. In MICRO-34, De [3] Pillai, P., and Shin, K. G Real Time Dynami Voltage Saling for ow Power Embedded Operating Systems. In SOSP (2001) [4] Mohapatra, S., and Venkatasubramanian, N. PARM: Power-Aware Reonfigurable Middleware. In ICDCS-23 (2003). [5] Noble, B. D., Satyanarayanan, M., D.Narayanan, J.E.Tilton, and Flinn, J. Agile Appliation- Aware Adaptation for Mobility. In SOSP (Otober 1997). [6] ITU-R Reommendation BT-500.7, Methodology for the subjetive assessment of the quality of television pitures. In ITU Geneva Switzerland (1995) [7] Sarnoff Corporation white paper, JND: a human vision system model for objetive piture quality measurements. In (2001). [8] S. Chandra. Wireless network interfae energy onsumption impliations of popular streaming formats. In Multimedia Computing and Networking MMCN'02, 2002 [9] Ralf Steinmetz, Klara Nahrstedt. MUTIMEDIA: Computing, Communiations and Appliations. Prentie Hall, July, 1995 [10] Fred Douglis and P. Krishnan and B. Marsh, Thwarting the power hungry disk, WINTER USENIX onferene, Jan

17 [11] Radu Cornea et al Integrated Power Management for Video Streaming to Mobile Handheld Devies, CECS Tehnial Report #03-19, University of California, Irvine, 2003 Sudeep Pasriha is a PhD andidate in Information and Computer Siene at the University of California, Irvine. His researh interests inlude system modeling methodologies, ommuniation arhiteture exploration and low power distributed embedded system design. He has an MS in Computer Systems Design from the University of California, Irvine and is a student member of the IEEE. Manev uthra is a PhD andidate in Information and Computer Siene at the University of California, Irvine. His researh interests are in embedded systems design, low power systems design and distributed systems. He has a B.Teh. degree in Eletrial Engineering from the Indian Institute of Tehnology, Delhi and is a student member of the IEEE. Shivajit Mohapatra is a PhD andidate in Information and Computer Siene at University of California, Irvine. His researh interests inlude Power-Aware Middleware frameworks, owpower systems and Distributed Systems. He has an MS in Computer Systems and Networking from the University of California, Irvine. Nikil D. Dutt is a professor at the Center for Embedded Computer Systems at the University of California, Irvine. He is a senior member of the IEEE, serves on the advisory boards of ACM SIGBED and ACM SIGDA, and is Vie-Chair of IFIP WG His researh interests are in embedded systems design automation, omputer arhiteture, optimizing ompilers, low power design and system speifiation tehniques. He has a Ph.D. in Computer Siene from the University of Illinois at Urbana-Champaign. Nalini Venkatasubramanian is an assistant professor at the Department of Information and Computer Siene, University of California, Irvine. Her researh interests inlude distributed and parallel systems, middleware, real-time multimedia systems, mobile environments and formal reasoning of distributed systems. She has a Ph.D. in Computer Siene from the University of Illinois, Urbana-Champaign and is a member of the IEEE and ACM. 17