Automated Software Defect Prediction Using Machine Learning

Transcription

1 Automated Software Defect Prediction Using Machine Learning Leandro L. Minku School of Computer Science University of Birmingham

2 Software Defect Prediction Software code is composed of several components.

3 Software Defect Prediction Testing all these components can be very expensive.

4 Software Defect Prediction If we know which components are likely to be defective, we can increase testing cost-effectiveness.

5 Software Defect Prediction Predictive models can be created to identify components likely to be defective by using past software releases and bug fixes as training data for learning machines. New release Learning Machine Components likely to be defective in the new release

6 Vectorising Past Projects Past projects need to be represented in a format suitable for learning machines. Example: vectorising components from past projects based on static code features. Quickly and automatically collected from the source code. Branch count Code + comment LOC Input features Halstead difficulty Cyclomatic complexity Target class... Defective? No No No Yes Yes

7 Naive Bayes: an example of learning machine Bayes theorem: Assuming independence: Naive bayes classifier:

8 How to Use Naive Bayes: an illustrative example Branch count Code + comment LOC Defective? 5 15 No 3 5 No 9 20 No Yes Yes Example: classify (bc = 16, loc = 39) P(C = No) = 3/5 = 0.6 P(bc = 16 No) = Gauss(x = 16, mean = 5.67 stdev = 3.06) = P(loc = 39 No) = Gauss(x = 39, mean = stdev = 10.41) = C = No --> 0.6 * * =

9 How to Use Naive Bayes: an illustrative example Branch count Code + comment LOC Defective? 5 15 No 3 5 No 9 20 No Yes Yes Example: classify (bc = 16, loc = 39) P(C = Yes) = 2/5 = 0.4 P(bc = 16 Yes) = Gauss(x = 16, mean = 15.5, stdev = 0.71) = P(loc = 39 Yes) = Gauss(x = 39, mean = 37.5, stdev = 3.53) = C = Yes --> 0.4 * * =

10 How to Use Naive Bayes: an illustrative example Branch count Code + comment LOC Defective? 5 15 No 3 5 No 9 20 No Yes Yes Example: classify (bc = 16, loc = 39) C = No --> 0.6 * * = C = Yes --> 0.4 * * = Class = Yes

11 WEKA Open source software that contains implementations of several learning machines.

12 Issues to Consider: class imbalance Image from the-impact-of-neurotransmitterimbalance/ The number of examples of faulty modules is usually much smaller than the number of non-faulty modules. Machine learners may tend to classify everything as negative! Possible fix: Undersample examples from non-faulty class. Other more advanced techniches. WANG, S.; MINKU, L. L.; YAO, X.. "Online Class Imbalance Learning and Its Applications in Fault Detection", International Journal of Computational Intelligence and Applications, 12(4): :1-19, 2013.

13 Issues to Consider: data availability Image from computer_fundamentals/computer_data.htm There is no data from a project before its first version is rolled out. How to predict defects for a project in its first version? Possible fix: Use data on other projects. MINKU, L. L.; YAO, X.; "How to Make Best Use of Cross-company Data in Software Effort Estimation?", Proceedings of the 36th International Conference on Software Engineering (ICSE'2014). NAM, J.; PAN, S.J.; KIM, S.."Transfer Defect Learning", Proceedings of the 35th International Conference on Software Engineering (ICSE'2013).

14 Issues to Consider: temporal behaviour Image from cio-priority-providing-data-time-people/ Typically, all available examples from all previous versions of a software are used to build fault prediction models. However, changes may happen from one version to the other: modules that are likely to be faulty in one version may not be faulty in another. Possible fix: Try and identify which previous versions are more useful. HARMAN, M.; ISLAM, S.; JIA, Y.; MINKU, L.; SARRO, F.; SRIVISUT, K.; "Less is More: Temporal fault predictive performance over multiple Hadoop releases", Symposium on Search-Based Software Engineering (SSBSE'2014).

15 Thank you! To appear in Dec 2014.