Android Graphic System Acceleration Based on DirectFB

Size: px

Start display at page:

Download "Android Graphic System Acceleration Based on DirectFB"

Theodore O’Connor’
6 years ago
Views:

1 Rsarch Journal of Applid Scincs, Enginring and Tchnology 5(18): , 2013 ISSN: ; -ISSN: Maxwll Scintific Organization, 2013 Submittd: July 27, 2012 Accptd: Sptmbr 17, 2012 Publishd: May 05, 2013 Android Graphic Systm Acclration Basd on DirctFB 1 Jiang Zhong-Qiu, 2 Zhang Min and 1 Liu Chang-Rong 1 Huaian Collg of Information Tchnology, Huai an Jiangsu, , China 2 National ASIC Systm Enginring Rsarch Cntr, Southast Univrsity, Nanjing , China Abstract: In this study, basd on analyzing th hardwar abstraction layr and nativ graphics libraris of Android graphics systm, th drawback of Skia library which could only support softwar rndring is pointd out. And thn th third-party opn graphics library DirctFB which supports 2D hardwar acclration is introducd, th architctur and intrfac of DirctFB and Skia ar analyzd and compard with ach othr in dtail. Aftr DirctFB bing portd into Android systm, a novl hardwar acclration layr with doubl-buffr tchnology is dsignd and implmntd, which will mak Skia and DirctFB coxist and complmnt with ach othr and ultimatly implmnt th 2D hardwar acclration in Android systm. A JNI intrfac is dsignd for Java programs. Th optimization schm is vrifid by th spcializd tst bnchmarks df-dok, th xprimntal rsults indicatd that th prformanc of Android graphics systm in layr blnding oprations is acclratd by an avrag of 5.58x as wll as 2.18x spdup on avrag in bitblit oprations whn procssing complx graphics oprations such as layr blnding and bitblit tc. Kywords: Adaptation layr, android graphics systm, DirctFB, hardwar acclration, Skia INTRODUCTION With th rapid dvlopmnt of Mobil Intrnt, a varity of mobil dvics, such as nt-book, intllignt cll phon and tablts tc., mrg to our lif and thy ar bcoming indispnsabl partitions of our daily work and lif. Th hardwar configuration is continual nhancd and updatd, whil usr fling is not only dcidd by th hardwar configuration, but also sriously lays on th softwar systm coupld with th hardwar platforms. Up to Now, th stat-of-art public intllignt OS (Oprating Systm) is mainly composd of Windows mobil sris of Microsoft Corp android of Googl and ios of Appl Corp tc. Googl s Android systm is firstly rlasd in Nov. 5 th, 2007 (Android (Oprating Systm), 2010) formally. It is mainly constructd by opration systm, middl firmwar, usr intrfac and applications. It is th nwst OS, but th most public on to nginrs. Compard to othr closd OS, it adopts th Apach licns, its sourc cod is opn and fr, which maks it to b th most popular on for many corporations hom and abroad, such as Motorola, Samsung, Sony Ericsson, China mobil and Huawi tc., most of thm hav slctd it as thir basd dvloping platforms. Android has bcom th hot-spot in industry and acadmia scops. Th graphics systm and UI (Usr Intrfac) ar th ky-partition of th Android systm, th prformanc and vn th succss or failur of on mobil dvic mostly rlis on thm. As th cor ngin of 2D graphics sub-systm, Skia only supports softwar rndring acclration, which has mad th graphic systm to b bottl-nck of systm prformanc, spcially in scnris involving numrous imag drawing and complx UI displaying. Thr ar fw litraturs in this scop, th major optimization approachs is to adopt th hardwar acclrating port of Copybit modul to improv th 2D graphic procssing, but it could only issu a fw simpl oprations and couldn t covr th whol 2D oprations. From this viwpoint, th Android graphic systm is paid mphasis on in this study to find othr hardwar schm to acclrat 2D graphic rndring. It is difficult to modify th Skia dirctly to mak it support hardwar acclration taking larg risk. DirctFB with th charactristic of opn sourc is introducd and portd to Android systm, which will mak it coopration with th nativ Skia and vntually acclrating 2D graphics rndring by hardwar. Th rlation Rsarch works on Android could b classifid into thr lvls. Th first on is Android applications. Ping-Xin (2009) implmntd input mthod into th input fram of Android. Ughtti t al. (2008) dsignd a P2P mobil application program basd on th agnt mchanism and this communication modl of JADE fram will mak dvloprs to dsign P2P softwar with high fficincy. Th scond on is focusd on th Android fram, which lis in th crossplatform porting and optimization of Dalvik VM Corrsponding Author: Jiang Zhong-Qiu, Huaian Collg of Information Tchnology, Huai an Jiangsu, , China 4438

2 (Virtual Machin), including dsign and optimization of th application fram and local fram. Tapia (2008) improvd th prformanc of opn cor in Android for Audio codc and th rlatd oprations. Chung (2009) adoptd a novl Markov dcision procss to optimiz softwar in Android. Th third class is about th drivr lvl and systm porting. Th drivrs in low lvl ar important partitions of Android, which includs Linux krnl, drivrs and Android HAL (Hardwar Abstract Lvl) tc. Wn-Chang Chung (Xuguang, 2009) discussd th Android systm porting for PX270 platform of Marvll. Thr ar som rsarch works at this point. Yonggang t al. (2010) studid th application of 2D hardwar graphics acclration in th mbddd multimdia systm basd on Linux, a slfadaption and samlss architctur of softwar and hardwar graphics acclration has bn dsignd and a schm for buffr submitting basd on computing workloads has bn introducd. Compard to th prvious rsarchs, th hardwar adaption lvl of Android is focusd on and optimizd in ours, DirctFB is discussd in dtail and 2D hardwar acclration units is utilizd to improv th prformanc of graphics systm as wll as th whol systm. In this study, th drawback of Skia library which could only support softwar rndring is pointd out. And thn th third-party opn graphics library DirctFB which supports 2D hardwar acclration is introducd, th architctur and intrfac of DirctFB and Skia ar analyzd and compard with ach othr in dtail. Aftr DirctFB bing portd into Android systm, a novl hardwar acclration layr with doubl-buffr tchnology is dsignd and implmntd, which will mak Skia and DirctFB coxist and complmnt with ach othr and ultimatly implmnt th 2D hardwar acclration in Android systm. A JNI intrfac is dsignd for Java programs. Th optimization schm is vrifid by th spcializd tst bnchmarks df-dok, th xprimntal rsults indicatd that th prformanc of Android graphics systm in layr blnding oprations is acclratd by an avrag of 5.58x as wll as 2.18x spdup on avrag in bit lit oprations whn procssing complx graphics oprations such as layr blnding and bitblit tc. ANDROID GRAPHIC SYSTEM Android is a larg and complx systm, it includs Linux krnl, hardwar adaption lvl, systm fram lvl and Java application lvl. Linux krnl is th cor and fundamntal partition, th schdul algorithm is optimizd for Android basd on th standard Linux krnl and som othr drivrs ar addd into it. Drivrs oprat th hardwar in low lvl dirctly, thrfor dsigning fficint drivrs for on fixd hardwar platform is vry important. Th hardwar adaption lvl is th intrfac packag btwn krnl and application lvl. Th nxt lvl is th Android systm fram layr, Rs. J. Appl. Sci. Eng. Tchnol., 5(18): , graphic oprations window managr input systm Imag loading Font rndring vido playback rsourc Managmnt Fig. 1: Graphic sub-systm of DirctFB which includs th application fram layr to provid API for Java applications. And som othr opn sourc libs such as Bionic, C lib and Skia tc., ar addd into it. Dalvik VM (Yi-Min and Rong, 2010) compils th Java cod to class fils and transforms it into.dx fils, th compild Java program will ignor th platforms diffrncs and run cross-platform. Th graphics systm adaption layr is analyzd and improvd in our work, which includs graphics hardwar adaption layr android graphics lib and application fram layr rlatd to graphics. Th graphics hardwar adaption layr is constructd by PMEM, Frambuffr drivr modul and Gralloc graphics mmory allocation modul; thy ar th basmnt for systm running. Graphics fram includs Libui lib, SurfacFlingr lib, OpnGL ES lib and Skia lib. Skia is th nativ 2D graphics rndring ngin lib of Android which is our focus point. ANALYSIS OF DIRECTFB GRAPHICS LIB FRAME As th graphics lib of th mbddd systm, DirctFB is a complt systm, as showing in Fig. 1, it composd thr layrs. Th first on is Window Managr (2011), which is utilizd to control th layout and display of windows. Th scond on is th implmntation of DirctFB, it could b classifid into graphic opration, input systm, imag loading, Font rndring and vido playback according to functions. Th window managr is not isolatd, but tightly coupld to th middl layr. Th rason lis in that th oprations such as window rndring, click on, imag display and vido playback in thos windows ar basd on th corrsponding sub-systm in this layr. Th third on is th rsourc managmnt layr, which is basmnt for th middl layr. Its functions includs two sids, on

3 Rs. J. Appl. Sci. Eng. Tchnol., 5(18): , 2013 v l L r s U rs riv D v ic D DirctFB Applications DirctFB Cor API Modul Gnric GFX Drivr DirctFB GFX drivrs Frambuffr Front Buffr Buffr addr rg fb_pan_display LCD Controllr DirctFB Back Buffr Drawing a r w a rd H Frambuffr Drivr Display Unit Fig. 2: Architctur of DirctFB Java Fram Local Fram Linux 2D Graphics Hardwar Android.viw Android. graphics Android. opngl Libui DirctFB Frambuffr FBDv SurfacFlingr OpnGL ES Libagl Fig. 3: Intgration of Skia and DirctFB in android Copybit sid is in charg of mmory allocation and rlas for th sub-systms in middl layr and th othr sid is for accss to th lock and signal transmitting tc. Thy ar th fundamntal for th intra-systm of sub-systms and intr-systm to shar rsourc of th whol platform. This study focuss on prformanc of th graphics rndring and th dsign architctur of DirctFB graphics oprations is paid mphasis on. As showd in Fig. 2, th architctur of DirctFB (Rnsas Solution Corp, 2005) composs usr application layr, krnl drivr layr and hardwar layr. Th usr application layr could b dividd into DirctFB application and lib layr. Th DirctFB applications utiliz th standard API (Application Intrfac) to finish th rlatd oprations such as rctangl filling/drawing and block transmitting tc. And thos API will call down for th gnral GFX drivrs which lis in usr stat, it will chck whthr th systm supports hardwar acclration or not. If th hardwar layr supports graphics acclration moduls with th corrspondnt krnl drivrs, GFX will utiliz th hardwar acclration modul with krnl drivr for graphics rndring, th nativ softwar API of DirctFB will b usd for rndring on th othr sid. Whthr hardwar acclration or softwar rndring, th final graphics data will b transmittd to Frambuffr for display. Skia Gralloc PMEM GPU Drivr 4440 Fig. 4: Switch procss of doubl buffr Dsign and implmntation of DirctFB: As showd in Fig. 3, th whit rctangls construct th Skia graphics systm. Skia has a hardwar acclration modul and th corrspondnt drivr, but all of th 2D graphics oprations could only b issud by softwar du to lack of hardwar calling intrfac in its ngin. This mchanism will dgrad th prformanc of Android graphics systm significantly. Thos gray rctangls outlin th DirctFB s layout in Android graphics systm. DirctFB and Skia ar in th sam layr of Android and provid 2D graphics drawing API for th uppr applications, whil DirctFB can call for th 2D hardwar acclration modul in GFX drivr fram for hardwar acclration. Th uppr applications will own two schms for graphics rndring, Skia will b slctd in scnris of low prformanc or without hardwar acclration and DirctFB should b chos for th scnris nding high prformanc and with hardwar acclration intrfac. Hardwar acclration for DirctFB: Fram buffr drivr is utilizd to control th hardwar display unit which is th final stp in graphics systm. Usr applications will call for th opration intrfac of graphics lib firstly, slct hardwar or softwar rndring intrfac according to th hardwar functions and systm configurations and vntually transmit th graphics data into Frambuffr to display on scrn. Eithr for th Gralloc modul of Skia or th FBDv modul of DirctFB, thy ar all basd on standard Frambuffr drivr. Thus optimization for th Frambuffr drivr is vry important in intgrating DirctFB into th Android systm. Th Frambuffr drivr could b optimizd from two sids. Th first on is to allocat mor than on cach buffr and control thm to output graphics data to display unit altrnativly. Whil th scond on lis in that dsigning multi-layr Frambuffr and composd thm by hardwar to output graphics data. Ths two sids will b discussd in dtail as following. Adopting doubl buffr: As showd in Fig. 4, doubl buffr transmitting schm xcuts in th xchang mchanism. Th standard Frambuffr drivr uss fb-

4 Rs. J. Appl. Sci. Eng. Tchnol., 5(18): , 2013 Layr 1 Layr 2 Layr N Frambuffr Softwar Composition Fig. 5: Graphic layrs combind by softwar Layr 1 Layr 2 Layr 3 Layr 4 Frambuffr1 Frambuffr2 Frambuffr3 Frambuffr 4 Hardwar Composition LCD Controllr Fig. 6: Graphic layrs combind by hardwar st-par( ) to complt displaying altrnativly btwn thos two buffrs, which nds to prob and configur som paramtrs such as LCD horizontal and vrtical timing, pixl format and alpha composition tc. And ths oprations will caus significant systm ovrhad, vn lad fuzzy and flickr in larg-rsolution display and high rfrsh rat scnris. A ddicatd intrfac calld fb-pan-display( ) is dsignd for th Frambuffr drivr, it is utilizd to issu th xchang opration which will rduc th calling and configuration tim significantly and improv th display quality. Adopting multi-physical layr for display output: Display multi-layr nds th support of th hardwar display modul. Th krnl allocats multi-layr Frambuffr, ach layr provids consistnt uppr API, whil th composition of thm is compltd by hardwar. In normal applications, th multi-layr composition is issud by softwar. As showd in Fig. 5, th krnl provids Frambuffr drivr for th LCD controllr. Although doubl buffrs hav bn adoptd, th composition of uppr multi-layr is issud by softwar and thn writtn to th Frambuffr. Whil th multilayr composition will involv alpha hybrid computation (Pulli t al., 2005) and this will occupy a significant of CPU rsourc. If th composition could b satisfid by hardwar, th graphics prformanc will b improvd dramatically. As showd in Fig. 6, th procssor in our schm support four hardwar display layrs, ach physical layr adopts doubl buffr, th uppr graphics layr will 4441 output data to th Fram buffr and thos four Frambuffr will b composd by hardwar to display vntually. Adaptation layr of hardwar acclration modul: Th adaptation layr (Takanari t al., 2007) hids th hardwar dtail downsid and provid unifid intrfac upsid. Th rlatd sourc cod could b found in gfxdrivrs/sp0611, which will b compild and xist as dynamic lib in systm. Th hardwar acclration adaptation layr of DirctFB nds to implmnt two major data structurs of GraphicsDrivrFuncs and GraphicsDvicFuncs, which dfinitions locat in src/cor/gfxcard.h. Th adaptation layr (Xiao-Xu t al., 2010) includs six intrfac functions for GraphicsDrivrFuncs, such as Prob( ), GtDrivrInfo( ), InitDrivr( ), InitDvic( ), ClosDrivr( ), sp0611-stup-drivr( ). Prob( ) is usd to prob whthr th systm support spcial graphics acclration modul or not. GtDrivrInfo( ) is utilizd to gt th drivr information of hardwar acclration modul. InitDrivr( ) and ClosDrivr( ) will initializ and clos th drivr. Sp0611-stup-drivr( ) opns th dvic nod of hardwar acclration modul, gt th display mmory and IO mmory and thn rmap thm into usr stat for th uppr applications to accss display mmory and hardwar acclration unit dirctly. Thraftr, th structur of GraphicsDvicFuncs, which includs th control port of hardwar acclration modul and th ntir 2D graphics hardwar

5 Rs. J. Appl. Sci. Eng. Tchnol., 5(18): , 2013 acclration intrfac, will b instantiatd (Enhua and Youquan, 2004). Aftr GraphicsDrivrFuncs and Graphics DvicFuncs bing implmntd, th hardwar acclration systm xists as dynamic lib in systm and it will b loadd in systm initialization, GraphicsDrivrFuncs will b rgistrd to systm. Aftr this stp, DirctFB could call th hardwar intrfac of GraphicsDvicFuncs for graphics rndring. JNI uppr intrfac dsign: Th uppr applications of Android systm ar dvlopd by Java languag and th application fram of Android provids Java API for dvloprs. All of th local libs ar implmntd in C or C++ languag and this nd to utiliz JNI to provid intrfac, which will mak Java to call th local lib. Thrfor, aftr porting DirctFB to Android and dsigning th hardwar acclration adaptation layr, w implmnt a JNI intrfac in DirctFB for Java applications. EXPERIMENTS AND ANALYSIS Exprimntal platform: W choos SEP6200 (ASIC Cntr, 2010) procssor as our xprimntal platform. It adopts TSMC65LP CMOS procss and cntrd in handhld mobil communication markts with high prformanc. It is composd of fiv major function parts, which ar systm clk control, intrfac and port, multi-mdia systm, GPS navigation systm and mmory systm. Th platform configurd with 4.3 Inch TFT LCD scrn with 800*480 display rsolution, 24 bit pixl data format and rsistor touch panl. In softwar sid, w slct Uboot as th systm boot tools android 2.3 basd on Linux krnl as OS. LCD Frambuffr drivr has bn dsignd and optimizd. 2D hardwar acclration drivr and touch panl drivr hav bn writtn and addd into th bas systm and systm imag fils ar compild on 64 bit ubuntu srvr. Dirct FB is downloadd from its official wbsit of with th vrsion to b DirctFB tst rsults and analysis: DirctFB provids hardwar acclration intrfac, whil softwar rndring is th dfault way. Aftr porting DirctFB to Android systm, th df-dok tst bnch is usd to vrify th prformanc of graphics rndring in DirctFB and Skia ngin and th rsults ar compard with ach othr quantitativly. Th tst rsults ar showd in Tabl 1 and Fig. 7. It could b concludd from Tabl 1 and Fig. 7 that th prformanc of softwar rndring for Skia and DirctFB ar almost at th sam lvl, whil compard to th prformanc of 2D hardwar acclration schm in DirctFB, larg diffrncs will xist and th dtail analysis as following: Th prformanc of Skia is prfct whn its basic drawing intrfac is calld and utilizd for simpl oprations, such as draw straight lin and rctangl tc. Whil if ths oprations involv composition of multipl graphics layrs, th rndring prformanc will dgradd dramatically. Th softwar rndring prformanc of Skia and DirctFB ar basically th sam. Th prformanc improvmnt by DirctFB softwar rndring lis in 1-2% and almost has no prformanc gains, thus hardwar acclration rndring schm by DirctFB is th only way to optimiz th Android graphics systm. Aftr implmnting hardwar rndring of DirctFB, for simpl drawing oprations such as drawing straight lin and rctangl tc., compard to Skia, it might dcras som prcntag of rndring prformanc. Th main rason lis in that it will lad to numrous systm stats switching whn adopting th hardwar acclration schm. This additional ovrhad will incras th systm burdn and it could nutraliz or vn ovrpass th prformanc gains du to th hardwar acclration modul. Whn it is utilizd to tackl with complx graphics oprations which involv numrous computations, such as composition of multipl graphics layrs and Bitmap blitting, it could bring about significant prformanc gains for graphics systm. Compard to Skia lib, th prformanc nhancmnt for th composition of multipl graphics layr is 558% on avrag and Bitmap Tabl 1: Tst rsults of Skia and DirctFB Bnch no Bnch nam Statics unit Skia softwar rndring DirctFB softwar rndring DirctFB hardwar rndring 1 Draw txt KChars/sc Draw txt2 KChars/sc Draw rct MPixl/sc Draw rct2 MPixl/sc Draw polygon MPixl/sc Draw polygon2 MPixl/sc Draw lins KLins/sc Draw lins2 KLins/sc Draw imag MPixl/sc Draw imag2 MPixl/sc Draw imag3 MPixl/sc Draw imag4 MPixl/sc

6 Rs. J. Appl. Sci. Eng. Tchnol., 5(18): , 2013 REFERENCES Fig. 7: Tst rsults comparison btwn Skia and DirctFB blitting could obtain 218% prformanc gains on avrag. In summary, it nds to oprat multipl graphics layrs, which involvs graphics layr composition and Bitmap blitting frquntly in actual applications. Ths oprations ar complx and comput-intnsiv and could b acclratd by hardwar rndring of DirctFB for prformanc improvmnt. Thr is som prformanc loss in simpl graphics oprations, but this kind of oprations will not b utilizd frquntly. Thus our hardwar acclration schm by DirctFB will bring about significant application prformanc gains. CONCLUSION Th Android graphics systm is studid in dtail in this study. From th adaptation layr, aftr th bas moduls of Android graphics systm such as PMEM tc., bing analyzd, th architctur and intrfac of Skia and DirctFB ar analyzd dply and compard to ach othr in dtail. DirctFB is portd to th Android systm and mad to b coxistd and complmntd with th nativ Skia lib. Doubl buffr tchniqu is adoptd into graphics systm. An adaptation layr for hardwar acclration modul is dsignd and th hardwar acclration for 2D graphics rndring is implmntd vntually. A local JNI intrfac is dsignd for Java applications in DirctFB. Th acclration schm is vrifid by df-dok tst bnch. Th tsting rsults indicatd that, aftr adopting our hardwar acclration schm by DirctFB, xcpt for a fw simpl graphics oprations, whthr vrtical compard to DirctFB softwar schm, or longitudinal compard to Skia softwar schm, our schm could improv th prformanc of Android graphics systm significantly as wll as th ovrall systm prformanc. ACKNOWLEDGMENT This study is supportd by Innovation Fund projct in Scinc and tchnology ntrpriss of Jiangsu Provinc (No. BC ). Android (Oprating Systm), Rtrivd from: n. wikipdia. org/wiki/ Android_ (oprating_ systm). ASIC Cntr, SEUIC SEP6200 Dvlopmnt Documnt. Southast Univrsity, Nanjing, pp: Chung, L.T., Markov Dcision Procss (MDP) framwork for optimizing softwar on mobil phons. Intrnational Confrnc on Compilrs, Architctur and Synthsis for Embddd Systms of th 7th ACM Intrnational Confrnc, ACM, Nw York, pp: Enhua, W. and L. Youquan, Gnral purpos computation on GPU. J. Comp. Aid. Dsign Comp. Graph., 16(5): Ping-Xin, L., Android plug-in input mthod programs dsign. J. Comp. Knowl. Tchnol., 5(35): Pulli, K., T. Aarnio, K. Roimla and J. Vaarala, Dsigning graphics programming intrfacs for mobil dvics. J. IEEE Comp. Graph. Appl., 25(6): Rnsas Solution Corp, Writing Custom GFX Drivrs for Dirct FB. Takanari, H., M. Hisao and O.K. Dnis, How to writ Dirct FB Gfxdrivrs for your Embddd Platform. Tchnology Consulting Company IGEL Corp. Tapia, J., Introduction to th opncor audio componnts usd in th android platform. Nw Trnds in Audio for Mobil and Handhld Dvics of th 34th AES Intrnational Confrnc, pp: Ughtti, M., T. Trucco and D. Gotta, Dvlopmnt of agnt-basd, pr-to-pr mobil applications on ANDROID with JADE. Procdings of UBICOMM, Valncia, pp: Window Managr, Rtrivd from: n. wikipdia. org/wiki/window_managr. Xiao-Xu, Y., W. Li-Hu, Y. Jia-Ning and N. Li-ping, Dirct FB applid in mbddd rmot dsk control systm. J. Comp. Eng. Dsign, 31(9): Xuguang, H., An Introduction to Android [DB/OL].. Inha Univristy, Dababas Lab. Yi-Min, Z. and CH. Rong, Analysis about procss in dalvik virtual machin. J. Comp. Tchnol. Dv., 20(2): Yonggang, J., Q. Zhngwi, P. Juanchun and Z. Quan, Prformanc tst and analysis for 2d graphic acclrator on pxa300 platform. J. Comp. Appl. Softwar, 27(5):

Reimbursement Requests in WORKS

Reimbursement Requests in WORKS Rimbursmnt Rqusts in WORKS Important points about Rimbursmnts in Works Rimbursmnt Rqust is th procss by which UD mploys will b rimbursd for businss xpnss paid using prsonal funds. Rimbursmnt Rqust can