A Method of Malicious Application Detection

Transcription

1 5th Iteratioal Coferece o Educatio, Maagemet, Iformatio ad Medicie (EMIM 2015) A Method of Malicious Applicatio Detectio Xiao Cheg 1,a, Ya Hui Guo 2,b, Qi Li 3,c 1 Xiao Cheg, Beijig Uiv Posts & Telecommu, Sch Comp Sci, Beijig , Peoples R Chia 2 Ya Hui Guo, Beijig Uiv Posts & Telecommu, Sch Comp Sci, Beijig , Peoples R Chia 3 Qi Li, Beijig Uiv Posts & Telecommu, Sch Comp Sci, Beijig , Peoples R Chia a chegxiaomlxq@163.com, b yhguo@bupt.edu.c, c liqi2001@bupt.edu.c Keywords: Malware detectio; Data aalysis optimizatio; Path optimizatio; Symbolic computatio; Stream tracig Abstract. With the rapid developmet of broadbad wireless access techology ad mobile termial, the mobile iteret developed quickly i recet years. However, malicious applicatios have become oe of the key factors threateig the developmet of mobile iteret. I order to protect vital iterest of mobile termial users, mobile malicious applicatios should be effectively preveted ad cotrolled. This paper aalyzes the existece limitatio of curret mobile applicatio detectio techology ad uses symbolic executio o the base of stream tracig. Malicious code writers usually hide malicious code executio path, ad i some special circumstaces to trigger some malicious behaviors. We costrait solve executio routes with sesitive calls, ad ultimately solve the specific backstage behaviors ad trigger coditios. We did some experimets to evaluate the performace of the proposed method. The experimetal results show that our method ca work well. Itroductio Mobile applicatios static aalysis is a method for feature extractio ad behavior aalysis through the biary samples. By aalyzig ad detectig malicious features ad sesitive behaviors, security aalysts ca grasp the malicious samples of harmful way ad make the correspodig meas to deal with these security risks. For malicious code detectio i feature, a accurate descriptio of the characteristics decides the detectio capabilities ad efficiecy of the methods. I the descriptio of the istructio features, most of the work is carried out for assembly istructios o the x86 platform. For example, Sog et al. raised BitBlaze [1] tool o the x86 platform. They proposed a x86 assembly formal descriptio laguage, Vie Itermediate Laguage (VIL), i the literature [2], which has played a good role i the program defect aalysis. Literature [3] proposed a improved itermediate laguage accordig to the characteristics of malicious code, focusig for biary malware behavior aalysis o the x86 platform. Shabtai et al. [4] also try to extract ad aalyze applicatio characteristics o the x86 eviromet usig -gram model i recet years. But there is rarely targeted characterizatio work for Dalvik or Java istructio. Felt et al. [5] aalyze privileges i Adroid system. They put privilege as a feature ad determie whether malware by Adroid applicatio privileges. Adroguard [6] characterize the fuctio ad itroduce the etropy ad the maximum compressio distace NCD [7] to compute the similarity of differet fuctios. But the extracted feature does ot delve ito the similarity betwee the istructios The authors - Published by Atlatis Press 579

2 I the formal descriptio of the code, the existig aalysis tools are mostly use Java bytecode or sequece of fuctio calls to describe the behavior of the sample. Literature [8] first proposed a formal descriptio of Java bytecode i the Java Card Virtual Machie, for solvig the problem of applicatio defect aalysis; O this basis, literature [9, 10] proposed static aalysis methods for Dalvik istructios to trackig data streams durig program executio. But descriptio method aims at defect detectio still ca ot effectively express the behavior of malicious code. I malicious code program behavior aalysis i the mobile platform, existig tools do ot use a valid abstractio itermediate laguage to describe the behavior of the applicatio. Due to the lack of such a descriptio, makig malicious code aalysis requires a lot of aalysis of persoel ivolved. The accuracy of maual aalysis to a great extet depeds o the ability ad experiece of the aalyst. I the eviromet of ueve capacity ad experiece of aalysis, some samples of malicious behavior ted to be ucosciously igored. System desig challege aalysis ad framework overview. The mai challege of the mobile applicatio static aalysis is two poits. The mai problem is the executable file static aalysis ad source code static aalysis vary greatly. Aalytical work requires complex machie istructios. Ad program aalysis requires accurate ad effective formal represetatio of complex istructio set. Typically, a commo istructio set cotais hudreds of istructios, such as Java byte code cotais 200 istructios, ad Dalvik bytecode cotais 218 istructios. I each istructio there are may complex sytax, which is a tremedous impact o the aalytical work. Secodly, the presece of mobile applicatios easily decompiled ad packagig. Geeral ati-virus software caot detect some malicious applicatios after a heavy pack because of the feature library. This differece eeds to be cosidered durig feature aalysis of the program. Mobile applicatios feature extractio i the istructio level ca avoid malicious applicatio usig a simple meas to bypass. Feature compariso algorithm based o istructio sequece ca efficietly fid the kow malicious applicatios. However, the pace of developmet of mobile malicious applicatios far exceeds the sigatures base update rate. I the routie aalysis, we use istructio characteristics aalysis to filter the applicatios simply ad quickly. Ad o this basis, this paper further studied the sematic aalysis of related istructios, which studies the behavior of the sample, thus to discover ukow malicious applicatios. Based o the above aalysis, this paper studies behavioral aalysis methods of mobile malicious applicatios. Ad we develop a aalytical tool for detectig mobile platforms malicious applicatios. I this paper, we use symbolic computatio. The mai purpose is to calculate the trigger coditios of characteristics sesitive behavior i mobile applicatios. Thus we do ot eed to traverse the etire cotrol flow. So before the symbolic computatio, cotrol flow graph iside fuctio ca be preprocessed, greatly reducig the umber of braches i symbolic computatio ad avoidig the path explosio problem i symbolic calculatio process. 580

3 Figure. 1 System Framework Path optimizatio based o data flow aalysis. Path extractig is uable to clear the sesitive call parameters ad path trigger coditio. To do this, o the basis, we eed to trace the data flow ad cotrol flow of the path, thikig symbolic computatio, to costrait solve path trigger coditio. I this paper, the mai idea of the optimizatio for the cotrol flow iside a fuctio is a combiatio of data flow aalysis ad calculatio of urelated braches of sesitive data to achieve optimizes computatio refereces to Reachig Defiitios. Reachig Defiitios is a method of data flow aalysis, which requires slovig equatios composed by i[b], out[b], ge[b], ad kill[b]. The purpose is to fid the relatioship betwee variables ad statemets i a cotrol flow graph. Table 1 Defiitio i Reachig Defiitios Symbol Meaig B i[b] out[b] ge[b] kill[b] a give basic block i the cotrol flow graph; set of defiitios that come before B; set of defiitios comig out of B; set of ew defiitios geerated withi B; set of defiitios whose variable is killed by B. I Reachig Defiitios, ge[b] ad kill[b] ca be obtaied directly by aalyzig the cotrol flow graph of basic blocks. i[b] ad out[b] ca be calculated by the followig formulas: out[ B ] ge[ B ] ( i[ B ] kill[ B ]) (1) 581

4 i[ B ] out[ B ] p pred ( ) p (2) Bp is the set of B's predecessors. Cosiderig there may be circulatig i the cotrol flow graph, i[b] ad out[b] may eed for iterative calculatios util it reaches a fixed poit, ie util steady state. O the basis of Reachig Defiitios, accordig to the data costrait required traced, we ca distiguish the urelated braches i cotrol flow graph Gc N, E, start, ed, so as to achieve the purpose of optimizatio. Data depedecies ca be divided ito the followig: Table 2 Data depedecies i Reachig Defiitios Symbol Detail f S S flow depedece S S ati depedece a S S output depedece o S S cotrol depedece c Algorithm2-2:Path Preprocessig Algorithm Iput: cotrol flow graph Gc,,, parameters passed i by curret fuctio costrait- cotaiig ode B Output: reachable path sequece 1. for each block B i G c do 2. i[ B] out[ B] 3. i[ B start ] = S i 4. do 5. for each block B i 6. i[ B ] out[ B ] p pred ( ) N E start ed, ge[ B ], kill[ B ] G c do 7. out[ B ] ge[ B ] ( i[ B ] kill[ B ]) 8. while o more chages to ay of out[ B ] do 11. for each paret block B of B do 12. f if ( B! c ) && ( B! ) 13. remove B from c 14. else 15. for each stmt i f 16. if stmt p G B do 17. update 18. B B B B, set of S i, data costrait, 19. while start Data flow optimizatio algorithm is cosist of seve steps: step 1: Iitialize i[b], out[b], ge[b], ad kill[b] i Reachig Defiitio. The iitializatio of i[b] eeds to cosider two cases: 582

5 { p1,..., p} start i[ B ] otherwise (3) p,..., 1 p are passed i parameters of the fuctio, out[ B ]. step 2: Iterative calculatio to get all the basic blocks i[b] ad out[b]. step 3: Determie the iitial basic block B ad data costrait. step 4: Traverse the cotrol flow graph i iverted order startig from B. Calculate all of the data depedecies betwee B ad its paret ode B. Remove the basic block if B ad B are irrelevat. step 5: If B ad B are relevat, aalyze data depedecies ci=exp. step 6: If B Bstart, B B ad retur to step 4. step 7: Geerate ew cotrol flow graph G. Save the reachable path sequece. c Experimet As show i Fig.2, fig(a) shows the cotrol flow graph before usig the Data flow optimizatio algorithm, while fig(b) shows the optimizated cotrol flow graph after usig the proposed algorithm. Figure. 2 Compariso of cotrol flow before ad after the proposed algorithm Fig.3 shows the fuctio relatioships of a sample mobile applicatio. As it shows, by usig the proposed algorithm, mostly urelated paths ca be discarded, which radically reduce the amout of calculatio. 583

6 Coclusio Figure. 3 fuctio relatioships of a sample mobile applicatio This paper proposed a aalytical tool for mobile platforms malicious applicatios. We proposed a malicious code detectio method. The method extract applicatio feature or applicatio behavior descriptio from a kow malicious applicatios, calculate the path costrait coditio o the basis of Reachig Defiitios ad extract sesitive behavior path i applicatios. Thus we ca aalyze the descriptio characteristics ad sesitive behavior of mobile platforms to detect malicious applicatios by characteristics ad behavior. Fially, experimets show our proposed method ca effectively aalyze malicious applicatio characteristics, ad ca effectively detect malicious behavior executio paths i ukow malicious applicatio. Ackowledgemets This work is supported by Natioal Natural Sciece Foudatio of ChiaProject( , ). Refereces [1] D. Sog, D. Brumley, H. Yi ad et al. BitBlaze: A New Approach to Computer Security via Biary Aalysis//Sekar R, Pujari A. Spriger Berli Heidelberg, 2008:1-25. [2] D. Brumley. Aalysis ad defese of vulerabilities i biary code. ProQuest, [3] J.X. Zhog. Key techologies of malware behavior biary aalysis. Beijig Uiversity of Posts ad Telecommuicatios, 2012.(I Chiese) [4] A. Shabtai, R. Moskovitch, C. Feher ad et al. Detectig ukow malicious code by applyig 584

7 classificatio techiques o OpCode patters. Security Iformatics, 2012,1(1):1-22. [5] A.P. Felt, E. Chi, S. Haa ad et al. Adroid permissios demystified: Proc of the 18th ACM Cof o Computer ad Commuicatios Security, New York, 2011[C]. ACM. [6] Desos A. Adroguard: Reverse egieerig, malware ad goodware aalysis of Adroid applicatios... ad more (ija!)[cp/ol]. [7] R. Cilibrasi ad P.M.B. Vitayi. Clusterig by compressio. Iformatio Theory, IEEE Tras o, 2005,51(4): [8] S.I.A. Operatioal sematics of the java card virtual machie. The Joural of Logic ad Algebraic Programmig, 2004,58(1):3-25. [9] E.R. Wogse ad H.S. Karlse. Study, Formalizatio, ad Aalysis of Dalvik Bytecode[G]. [10] E.R. Wogse ad H.S.N.Karlse. Static Aalysis of Dalvik Bytecode ad Reectio i Adroid[G]. 585