On Error-Class Distribution in Automotive Model-Based Software Harald Altinger, Yanja Dajsuren, Sebastian Siegl, Jurgen J. Vinju, and Franz Wotawa

Transcription

1 On Error-Class Distribution in Automotive Model-Based Software Harald Altinger, Yanja Dajsuren, Sebastian Siegl, Jurgen J. Vinju, and Franz Wotawa

2 Topics Todays Talk 1 Automotive Basics 3 2 Setup 9 3 Automotive Data Set 13 4 Error Class Analysis 16 5 Results 23 6 Application 25 2 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

3 Automotive Basics 1 3 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

4 Audi Known for Source: AUDI AG 4 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

5 Modern day Car Premium Up to 90 ECU Up to 11 Networks Up to LOC Up to two SIM Cards Up to 40% of total development Budget Embedded Software Time critical Mission critical Connectivity Source: AUDI AG 5 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

6 Traditional Testing years ago Source: AUDI AG 7 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

7 Testing Stages During Software Development In The Loop Tests MiL Model in the Loop SiL Software in the Loop PiL Processor in the Loop HiL Hardware in the Loop Whole Car Lab Car Test Car Protoype Pre Production car Source: AUDI AG 8 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

8 Setup 2 9 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

9 Automotive Domain Development MISRA Guidelines [3] No pointer Integer, Float, Structs No dynamic String types Static memory Functionality Safety critical functionality None Safety critical Hard- & Firm Realtime V-Process based development Strict time shedules during development SOP 10 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

10 Software Lifecycle Development phase LOC Requirements Freez 100% Software SOP Feature Freez time 11 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

11 Automotive Domain MSR Dataset [4] 12 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

12 Dataset Automotive 3 13 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

13 Dataset on Automotive Projects Simple Indicator See [5] 14 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

14 Project # Requirements # Sub-Projects LOC # Testcases # Authors # src. Files # mdl files # commited files # error prone files AUTOSAR Safety function Dataset on Automotive Projects Key Figures software type A L K logic, timing dependent behaviour logic, timing dependent behaviour mainly logic operations, branching yes no yes yes yes yes Get the Audi Dataset: See [4] 15 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

15 Dataset on Automotive Projects Marking a Bug SZZ Algorithm See [4] 16 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

16 Error Class Analysis 4 17 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

17 Research Questions To be answered RQ1 Is the software metric affected by any of the bugs reported? RQ 2 Which type of error classes might be already prevented by existing techniques? RQ 3 Do complexity / length of boolean conditions and number of bugs correlate? Source: AUDI AG 18 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

18 Results Operators in use A K L % Operator % Operator % Operator Logic and comperator 76,38% = 46,17% = 43,88% = 6,60% && 15,14% && 19,78% == 3,59%! 12,39%! 10,33% && 2,90% 9,02% & 6,49%!= See Table II 19 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

19 Results Operators in use A K L % Operator % Operator % Operator Flow controll 35,31% if 55,49% if 47,46% if 24,91% else 34,42% else 24,73% else 17,55% break 3,40% break 12,27% break 14,12% case 2,62% case 9,04% case See Table II 20 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

20 Results Datatypes in use A K L % Operator % Operator % Operator Datatypes 95,38% static int 95,90% static int 99,06% static int 4,62% struct 1,91% int 0,94% struct 1,91% unsigned 0,29% struct See Table II 21 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

21 rank type Project LOC McCab N1 N2 n1 n2 Results Most common Bugs Discovered explanation 1 2 boolean logic error missing condition 3 casting error 4 mistmatch boolean, bitwise logic miscalculations with boolean K, conditions L missing variable within condition K, variable type cast error, including wronge variable declarations e.g. < instead of << L K, L n n n n n n n w y y w w n n n n n n A, K n n w w w w See Table III 22 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

22 Results Boolean length and error distribution #operands within error prone statements 1 34,92 % 2 06,35 % 3 15,87 % 4 17,46 % 5 01,59 % 6 03,17 % 7 03,17 % 8 07,94 % 9 04,76 % 10 01,59 % 11 03,17 % See Table I 25 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

23 Summary 5 26 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

24 Research Questions Review RQ1 Is the software metric affected by any of the bugs reported? RQ 2 Which type of error classes might be already prevented by existing techniques? RQ 3 Do complexity / length of boolean conditions and number of bugs correlate? See Table III See Table I Source: AUDI AG 27 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

25 Application 6 28 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

26 Audi self driving cars Piloted Racing Maximum Software, no space for SW error Source: AUDI AG 29 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

27 Audi self driving cars Piloted Driving near future Source: AUDI AG 30 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

28 Thank You! Any Questions? 31 On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016

29 Literature In addition to Paper [1] H. H. Braess and U. Seiffert, Vieweg Handbuch Kraftfahrzeugtechnik. Springer Fachmedien Wiesbaden, [2] H. Altinger, F. Wotawa, and M. Schurius, Testing Methods Used in the Automotive I ndustry: Results from a Survey, in Proc.\ JAMAICA, San Jose, CA, 2014, pp [3] The Motor Industry Software Reliability Asso- and ciation, MISRA-C: Guidelines for the use of the C language in critical systems, 2nd ed. Warwickshire: MISRA, [4] H. Altinger, S. Siegl, Y. Dajsuren, and F. Wotawa, A Novel Industry Grade Dataset for Fault Prediction based on Model-Driven Developed Automotive Embedded Software, in Proceedings of the 12th Working Conference on Mining Software Repositories, Florence, Italy, [5] J. Peleska, F. Lapschies, H. Lding, P. Smuda, H. Schmid, E. Vorobev, and C. Zahlten., Turn indicator model overview, uni-bremen.de/agbs/testingbenchmarks/turn indicator/, April On Error-Class Distribution in Automotive Model-Based Software, Altinger, et.al. 16/03/2016