An Empirical Study on Detecting and Fixing Buffer Overflow Bugs

Transcription

1 A Empirical Study o Detectig ad Fixig Buffer Overflow Bugs Lizhag Wag Joit work with Tao Ye, Xuadog Li, Najig Uiversity, Chia Ligmig Zhag,Uiversity of Texas at Dallas, USA Jue 6, 2016

2 Outlie Backgroud ad motivatio Empirical study Experimetal results Take home messages 2

3 Backgroud ad motivatio Careless programmig with umaaged C/C++ laguages may result i security vulerabilities Iappropriate memory maipulatio Mistake assumptios about size Makeup of a piece of data Misuse of API Buffer overflow is oe of the best kow security vulerabilities. Missig Iput validatio or boud checkig before memory maipulatio or API callig may overwrite the allocated bouds of buffers. 3

4 Backgroud ad motivatio Statistics of buffer overflows i CVE 14.6% of all, 3 rd most popular Prevalet attacks agaist legacy or ewly deployed systems Vulerabilities by type 13.1% 19.4% 14.6% 21.6% 31.2% Deial of Service Execute Code Overflow XSS Others 4

5 Backgroud ad motivatio Buffer overflow void fuc (char *str) { char buf[4]; strcpy(buf, str); } a bufferb c d sfp eoverflow f Stack grows ret addr str... Stack top str= abc Local variables str= abcdef... Poiter to previous frame Retur address Argumets 5

6 Backgroud ad motivatio Buffer overflow causes severe damage system crash edless loop executig arbitrary code What ca we do to deal with buffer overflow? Detectio Repair Prevetio/mitigatio 6

7 Backgroud ad motivatio Dyamic testig versus static aalysis Dyamic approach (Stackguard[1], CCured[2]): Isertig special code ito software to moitor buffer status Advatage: few false-positives Disadvatage: performace overhead, false-egatives Static approach (Fortify, Checkmarx, Split): scaig source code Advatage: discoverig buffer overflow before software deploymet, highly automated ad scalable Disadvatage: may false-positives [1] C. Cowa, C. Pu, D. Maier, J. Walpole, P. Bakke, S. Beattie, A. Grier, P. Wagle, Q. Zhag, ad H. Hito, Stackguard: Automatic adaptive detectio ad prevetio of buffer-overflow attacks. i USENIX Security, vol. 98, 1998, pp [2] G. C. Necula, S. McPeak, ad W. Weimer, Ccured: Type-safe retrofittig of legacy code, i ACM 7 SIGPLAN Notices, vol. 37, o. 1, 2002, pp

8 Backgroud ad motivatio Static techiques are widely used Few studies o effectiveess ad efficiecy of static techiques (Kratkiewicz[1]) Coductig a study o static techiques Effectiveess ad efficiecy of detectig buffer overflow [1] K. J. Kratkiewicz, Evaluatig static aalysis tools for detectig buffer overflows i c code, Master s thesis, Harvard Uiversity,

9 Outlie Backgroud ad motivatio Empirical study Experimetal results Take home messages 9

10 Empirical Study Research questios Subject system Selected techiques Experimetal setup Experimetal steps 10

11 Research questios RQ1. Effectiveess? False positive ad false egative RQ2. Efficiecy? Resource cosumptio RQ3. API? Root cause of buffer overflow vulerabilites RQ4. Maual fix patters? Official repair 11

12 Subject systems Start Selectio criterio Ope-source Radomly select bo bugs from CVE website No Buffer overflow If it is from ope source 100 buffer overflow bugs from 63 real-world projects, totalig 28MLoc, ragig from CVE-1999 to CVE No Yes Obtai buggy ad fixed versios If we have 100 bugs Yes Ed 12

13 Subject systems Subject Descriptio Size(LoC) # BO bugs (#Versios) mapserver libzip ma libthai Platform for publishig data to web Library for hadlig zip archives Commad used to display user maual Thai laguage support routies 276K 4(2) 10K 1(1) 10K 3(2) 6K 1(1) Total 28M 100(81) 13

14 Static techiques Represetative static techiques: Fortify, Checkmarx, ad Split Accordig to Garter Group report, Fortify ad Checkmarx are leadig commercial products i applicatio security market. Split is oe of the first ope-source tools that cocer safety issues, ad it is widely used. 14

15 Static techiques Fortify (versio ) Code compiled Data flow aalysis source ad sik Cotrol flow aalysis -- a set of operatios Sematic aalysis dagerous use of fuctios ad APIs 15

16 Static techiques Checkmarx (versio 7.1.6) U-compiled, icomplete code Meticulous model betwee users ad other data Trackig data ad logic flows Idetifyig vulerabilities accordig to static aalysis rules 16

17 Static techiques Split (versio 3.1.1) Modelig buffer ad aotatig buffer size Precoditio ad postcoditio of buffer access Costrait solver 17

18 Experimetal setup Fortify ad Split: o a server with Itel Xeo CPU E (1.80GHz) ad 128GB RAM o Ubutu Liux Checkmarx o a server with Itel Xeo CPU E (2.30GHz) ad 384GB RAM o Widows Server

19 Experimetal steps Apply all techiques to the subjects buggy versio-> fid the bugs that caot be detected ->falseegatives. fixed versio->fid the fixed bugs that are still idetified as bugs- >false-positives. Categorize the root cause of BO->API Start Apply techiques to buggy versios Apply techiques to fixed versios Collect APIs ad fix patters Falseegatives ad time Falsepositives ad time APIs ad fix patters Categorize the maual fix patter. Ed 19

20 A example Subject Descriptio Size(LoC) # BO bugs (#Versios) libzip ma libthai Library for hadlig zip archives Commad used to display user maual Thai laguage support routies 10K 1(1) 10K 3(2) 6K 1(1) Total 28M 100(81) A commad used to display user maual 20

21 CVE iformatio ID: CVE Descriptio: Buffer overflow i ultimate_source fuctio of ma 1.5 ad earlier allows local users to gai privileges. 21

22 Source code Buggy code (ma-1.5i2): static cost char* ultimate_source(cost char* ame0) {... static char ultame[bufsize]; } strcpy(ultame, ame0);... Fixed code (ma-1.5p): static cost char* ultimate_source(cost char* ame0) {... static char ultame[bufsize]; if (strle(ame0) >= sizeof(ultame)) retur ame0; strcpy(ultame, ame0);... } 22

23 Results O the buggy versio Forfity: detected, 3m50s Checkmarx: detected, 2m25s Split: preprocess error O the fixed versio Fortify: udetected (fixed), 3m58s Checkmarx: detected (ot fixed), 2m25s Split: preprocess error API: strcpy Fix patter: add boudary check 23

24 Outlie Backgroud ad motivatio Empirical study Experimetal results Take home messages 24

25 RQ1: Effectiveess Techs # Idetified Bugs FN Rate # Idetified Fixes By combiig these techiques, we ca get a lower false-egative rate. FP Rate Fortify %(41/60) %(6/19) Checkmarx %(68/100) 8 75%(24/32) Split %(13/23) %(10/1 0) The cost is a relatively higher false-positive rate. Ft+Cm %(58/100) %(28/42) Checkmarx ca detect most buffer overflow bugs. Cm+Sp 39 61%(61/100) %(32/39) Ft+Sp %(38/64) %(13/26) Split performs best i terms of false-egative rate. All %(53/100) %(34/47) Fortify performs best i terms of false-positive rate. 25

26 RQ2: Efficiecy (s) Checkmarx teds to be the most costly techique to apply, followed by Fortify Fortify Checkmarx Split

27 RQ3: API distributio Top three APIs related to buffer overflow: array, memcpy, spritf

28 APIs ad techiques API # Istaces Fortify Checkmarx Split array 31 4/21 2/31 8/12 memcpy 15 0/5 9/15 0/1 spritf 13 Split 6/7 works well 7/13 o array. 1/2 poiter 7 Split 0/4 reports a 0/7 warig whe 0/1 the costrais are usolvable. strcpy 6 4/4 3/6 0/1 Evidece: Split idetifies 0/8 strcpy 5 fixes. 1/4 1/5 0/0... Fortify... ad Checkmarx... ca fid... most... bugs o APIs like spritf ad strcpy. They ted to report a warig o usafe API. Evidece: strcpy 1/4 1/5 28

29 RQ4: Fix strategy distributio Addig boudary check ca fix early half of the studied bugs. 29

30 Fix strategy ad API Most APIs prefer add boudary check For some of them, there may be a more suitable way. Strategies array memcpy spritf... Total Add boudary check Use larger buffer API substitutio Total

31 Outlie Backgroud ad motivatio Empirical study Experimetal results Take home messages 31

32 Take home messages Effectiveess: Use Fortify aloe to achieve low false-positive rate. Use techiques together to achieve low falseegative rate. Efficiecy: Checkmarx is the most costly techique amog studied techiques. 32

33 Take home messages API: Array, memcpy ad spritf are top three APIs related to buffer overflow. Split has the lowest false-egative rate o array Fortify ca fid most bugs o APIs like spritf ad strcpy, followed by Checkmarx. 33

34 Take home messages Fix strategy: Most buffer overflow bugs ca be fixed usig strategy addig boudary check. If addig boudary check fails, oe ca choose other strategy accordig to the API ivolved. 34

35 Coclusios A quatitative study of the state-of-art static techiques for buffer overflow detectio o 100 bugs from 63 real-world projects totalig 28 MLoC A qualitative aalysis of the false-positives ad falseegatives of studied static detectio techiques, which ca guide the desig ad implemetatio of more advaced buffer overflow detectio techiques. A categorizatio o the fix patters of buffer overflow bugs to guide both maual ad automated buffer overflow repair techiques. 35

36 Ogoig ad Future Work Automatic static Buffer Overflow Warig Ispectio Maual ispectio of static report is time cosumig ad lab itesive Static warig + dyamic symbolic executio Automatic Buffer Overflow Bug Repair Maual repair eeds programmig expertise ad may itroduce ew bugs. For validated true buffer overflow vulerabilities, we automatically geerate fix suggestios accordig to the predefied templates which are created based o huma repair patters. 36

37 More ifo Tao Ye, Ligmig Zhag, Lizhag Wag ad Xuadog Li. A Empirical Study o Detectig ad Fixig Buffer Overflow Bugs, i proceedigs of ICST2016, Aril 9-16, 2016, Chicago, US. Cotact lzwag@ju.edu.c 37

38 Thaks Questios? 38