Model-based Mining of Software Repositories. Markus Scheidgen

Transcription

1 Model-based Mining of Software Repositories Markus Scheidgen

2 Agenda Mining Software Repositories (MSR) and current approaches srcrepo a model-based MSR system srcrepo components and analysis process a meta-model for source code repositories gathering software metrics with an OCL-like internal Scala DSL work in progress - discussion of remaining problems and limitations 2

3 Relevant Research Fields Mining Software Repositories (MSR) The term mining software repositories (MSR) has been coined to describe a broad class of investigations into the examination of software repositories. The premise of MSR is that empirical and systematic investigations of repositories will shed new light on the process of software evolution. [] Software Metrics A software metric is a mathematical definition mapping the entities of a software system to numeric metrics values. [...] to express features of software with numbers in order to facilitate software quality assessment. [2] Reverse Engineering Reverse engineering is the process of analyzing a subject system to () identify the system s components and their interrelationships and (2) create representations of the system in another form or at a higher level of abstraction [3] Software Evolution Research (SER) (dis-)proving Lehmann s Laws of software evolution empirical investigations of software repositories through statistical analysis of software and software change metrics over the evolutionary cause of many software systems. Model-based Mining Software Repositories (with srcrepo) Overcoming heterogeneity and accessibility by raising the level of abstraction, while ensuring scalability and retaining meaningful information depth.. H. Kagdi, M.L. Collard, J.I. Maletic: A survey and taxonomy of approaches for mining software repositories in the context of software evolution; Journal of Software Maintenance and Evolution: Research and Practice; Vol.9/Nr.2/ R. Lincke, J. Lundberg, W. Löwe: Comparing Software Metrics Tools; 8th International Symposium on Software Testing and Analysis; E.J. Chikofsky, J.H. Cross: Reverse engineering and design recovery: A taxonomy; IEEE Software; Vol.7/Nr./990 3

4 Contemporary Approaches to Large Scale MSR for SER FLOSS Metrics [] database for over 3000 open source software projects contains data about all revisions Alitheia, multiple version control systems (VCS), but only text-based metrics not only source code repositories (SCR) via VCS, also issue-tracking systems, mailinglists, etc. Sourcerer [2] database and searchable index of declarations from over 4000 Java software projects tracks only release revisions metrics based on declarations (classes, methods, fields, etc., e.g. CK-metrics), but not based on actual implementations (e.g. McCabe, Halstead) Boa [3] domain specific language (DSL) for mining meta-data in ultralarge software repositories only tracks VCS meta-data, e.g. How many revisions are there in all Java projects using SVN? Scalability Heterogeneity Accessibility Information Depth a project large scale: multiple related projects, e.g. Apache, Eclipse ultra-large scale: 00k+ unrelated projects with varying quality [,2,3] abstraction from VCS [,2,3] abstraction from programming language: only metadata [3] or text [] programming database with index [,2] DSL [3] all revisions [,3], sample revisions [2] meta-data [3] text [] declarations [2]. G. Gousios, D. Spinellis: Alitheia core: An extensible software quality monitoring platform; Proceedings of the 3st International Conference on Software Engineering; E. Linstead, S. Bajracharya, T. Ngo, P. Rigor, C. Lopes, P. Baldi.: Sourcerer: mining and searching internet-scale software repositories; Data Mining and Knowledge Discovery; Vok.8/Nr.2/ R. Dyer, H.A. Nguyen, H. Rajan, T.N. Nguyen: Boa: A Language and Infrastructure for Analyzing Ultra-Large-Scale Software Repositories; Proceedings of the 203 International Conference on Software Engineering; 203 4

5 Goals and Hypothesis Scalability Heterogeneity Accessibility Information Depth a project large scale: multiple related projects, e.g. Apache, Eclipse ultra-large scale: 00k+ unrelated projects with varying quality [,2,3] abstraction from VCS [,2,3] abstraction from programming language: only metadata [3] or text [] programming database with index [,2] DSL [3] all revisions [,3], sample revisions [2] meta-data [3] text [] declarations [2] 4

6 Goals and Hypothesis Scalability Heterogeneity Accessibility Information Depth hypothesis goals approaches a project large scale: multiple related projects, e.g. Apache, Eclipse ultra-large scale: 00k+ unrelated projects with varying quality [,2,3] cluster- (batching) and cloud- (Map/Reduce)- computing support distributable databases distributable model persistence distributed processing of models abstraction from VCS [,2,3] abstraction from programming language: only metadata [3] or text [] common meta-model for VCSs meta-models for programming languages common meta-model for metrics abstraction for different VCSs exists abstraction regarding metrics for diff. progr. languages exists abstraction for diff. languages exists programming database with index [,2] DSL [3] internal DSL: DSL + programming with models common modeling framework existing tools/ frameworks is there a reasonable programming abstraction for gathering metrics/ change metrics all revisions [,3], sample revisions [2] meta-data [3] text [] declarations [2] all revisions abstract syntax trees (AST) differences between revisions (e.g. metrics on adaptations and refactorings) 5

7 srcrepo Components and Process 6

8 srcrepo Components and Process large scale software repositories (e.g. github, sourceforge) software projects issue tracker, mailing lists, wiki source code repository (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) 6

9 srcrepo Components and Process large scale software repositories (e.g. github, sourceforge) software projects srcrepo storage (EMF-models via EMF-Fragments, e.g. on mongodb) revision tree issue tracker, mailing lists, wiki source code repository (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) import A 2 3 A3 C3 AST-models of new and changed CUs 6

10 srcrepo Components and Process large scale software repositories (e.g. github, sourceforge) software projects srcrepo storage (EMF-models via EMF-Fragments, e.g. on mongodb) revision tree srcrepo runtime (headless eclipse RCP) revision tree issue tracker, mailing lists, wiki source code repository (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) import A 2 3 A3 C3 AST-models of new and changed CUs analysis 2 3 S S2 S2 A A A3 C3 fully resolved snapshot models 6

11 srcrepo Components and Process large scale software repositories (e.g. github, sourceforge) software projects srcrepo storage (EMF-models via EMF-Fragments, e.g. on mongodb) revision tree srcrepo runtime (headless eclipse RCP) revision tree issue tracker, mailing lists, wiki source code repository (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) import A 2 3 A3 C3 AST-models of new and changed CUs analysis 2 3 S S2 S2 A A A3 C3 fully resolved snapshot models revisions sources 6

12 issue tracker, mailing lists, wi (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) import A 2 3 A3 C3 AST-models of new and changed CUs analysis 2 3 S S2 S2 A A A3 C3 fully resolved snapshot models isions sources 7

13 issue tracker, mailing lists, wi (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) import A 2 3 A3 C3 AST-models of new and changed CUs analysis 2 3 S S2 S2 A OCL A M M2 M3 A3 C3 fully resolved snapshot models isions sources metrics 7

14 issue tracker, mailing lists, wi (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) import A 2 3 A3 C3 AST-models of new and changed CUs analysis 2 3 S S2 S2 A OCL EMF-Compare A M M2 M3 D-2 OCL D2-3 A3 C3 fully resolved snapshot models isions sources metrics M-2 M2-3 7

15 issue tracker, mailing lists, wi (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) import A 2 3 A3 C3 AST-models of new and changed CUs M M2 M2 analysis store 2 3 S S2 S2 A OCL EMF-Compare A M M2 M3 D-2 OCL D2-3 A3 C3 fully resolved snapshot models isions sources metrics M-2 M2-3 M-2 M2-3 timelines of metrics 7

16 issue tracker, mailing lists, wi (e.g. controlled by Git, SVN, CVS) source code (e.g. java, C++, eclipse*) import export A 2 3 A3 C3 AST-models of new and changed CUs M M2 M2 analysis store 2 3 S S2 S2 A OCL EMF-Compare A M M2 M3 D-2 OCL D2-3 A3 C3 fully resolved snapshot models isions sources metrics M-2 M2-3 M-2 M2-3 statistics software (e.g. R, Matlab) timelines of metrics 7

17 A Meta-Model for Source Code Repositories «fragments» revision tree 8 e

18 MoDisco source code «fragments» revision tree 9

19 Demo 0

20 A OCL-like internal Scala DSL for Computing Metrics OCL-like internal Scala DSL analog to our internal Scala model transformation language [] OCL collection operations mapped to Scala s higher-order fuctions [2]:. L. George, A. Wider, M. Scheidgen: Type-Safe Model Transformation Languages as Internal DSLs in Scala; Theory and Practice of Model Transformations - 5th International Conference, ICMT; Filip Krikava: Enrichting EMF Models with Scala; Slideshare

21 A OCL-like internal Scala DSL for Computing Metrics ownedelements OCL-like internal Scala DSL analog to our internal * Scala Model Package model transformation language [] Abstract Type Declaration OCL collection operations mapped to Scala s higher-order fuctions [2]: * ownedpackages * ownedelements pure OCL context Model: self.ownedelements->collect(p p.ownedelements)->size. L. George, A. Wider, M. Scheidgen: Type-Safe Model Transformation Languages as Internal DSLs in Scala; Theory and Practice of Model Transformations - 5th International Conference, ICMT; Filip Krikava: Enrichting EMF Models with Scala; Slideshare

22 A OCL-like internal Scala DSL for Computing Metrics ownedelements OCL-like internal Scala DSL analog to our internal * Scala Model Package model transformation language [] Abstract Type Declaration OCL collection operations mapped to Scala s higher-order fuctions [2]: * ownedpackages * ownedelements pure OCL context Model: self.ownedelements->collect(p p.ownedelements)->size OCL-like expression in Scala def numberoffirstpackageleveltypes(self: Model): Int = self.getownedelements().collect(p=>p.getownedelements()).size(). L. George, A. Wider, M. Scheidgen: Type-Safe Model Transformation Languages as Internal DSLs in Scala; Theory and Practice of Model Transformations - 5th International Conference, ICMT; Filip Krikava: Enrichting EMF Models with Scala; Slideshare

23 A OCL-like internal Scala DSL for Computing Metrics Extending OCL s collection operations: convenience operations closure aggregation execution trait OclCollection[E] extends java.lang.iterable[e] { def size(): Int def first(): E def exists(predicate: (E) => Boolean): Boolean def forall(predicate: (E) => Boolean): Boolean def select(predicate: (E) => Boolean): OclCollection[E] def reject(predicate: (E) => Boolean): OclCollection[E] def collect[r](expr: (E) => R): OclCollection[R] def selectoftype[t]:oclcollection[t] def collectnotnull[r](expr: (E) => R): OclCollection[R] def collectall[r](expr: (E) => OclCollection[R]): OclCollection[R] 2 def closure(expr: (E) => OclCollection[E]): OclCollection[E]

24 trait OclCollection[E] extends java.lang.iterable[e] { def size(): Int def first(): E def exists(predicate: (E) => Boolean): Boolean def forall(predicate: (E) => Boolean): Boolean def select(predicate: (E) => Boolean): OclCollection[E] def reject(predicate: (E) => Boolean): OclCollection[E] def collect[r](expr: (E) => R): OclCollection[R] } def selectoftype[t]:oclcollection[t] def collectnotnull[r](expr: (E) => R): OclCollection[R] def collectall[r](expr: (E) => OclCollection[R]): OclCollection[R] def closure(expr: (E) => OclCollection[E]): OclCollection[E] def aggregate[r,i](expr: (E) => I, start: () => R, aggr: (R, I) => R): R def sum(expr: (E) => Double): Double def product(expr: (E) => Double): Double def max(expr: (E) => Double): Double def min(expr: (E) => Double): Double def stats(expr: (E) => Double): Stats def run(runnable: (E) => Unit): Unit 3

25 Complex Example: Average Weighted Methods per Class (WMC) WMC is the first CK-metric []. There different commonly used weights; here we use cyclomatic complexity def classes(model:model):oclcollection[classdeclaration] = model.getownedelements().collectclosure(pkg=>pkg.getownedpackages()).collectall(pkg=>pkg.getownedelements()).collectclosure(typedcl=> typedcl.getbodydeclarations().selectoftype[classdeclaration]) def WMC(model:Model):Double = classes(model).stats(clazz=> clazz.getbodydeclarations().selectoftype[methoddeclaration]().sum(method=>cyclmaticcomplexity(method))).average def cyclomaticcomplexity(method:methoddeclaration):int =.... S.R. Chidamber, C.F. Kemerer: A Metrics Suite for Object Oriented Design; IEEE Transactions on Software Eng.; Vol.20/Nr.6/994 4

26 Complex Example: Average Weighted Methods per Class (WMC) WMC is the first CK-metric []. There different commonly used weights; here we use cyclomatic complexity def classes(model:model):oclcollection[classdeclaration] = model.getownedelements().collectclosure(pkg=>pkg.getownedpackages()).collectall(pkg=>pkg.getownedelements()).collectclosure(typedcl=> typedcl.getbodydeclarations().selectoftype[classdeclaration]) def WMC(model:Model):Double = Model classes(model).stats(clazz=> clazz.getbodydeclarations().selectoftype[methoddeclaration]().sum(method=>cyclmaticcomplexity(method))).average def cyclomaticcomplexity(method:methoddeclaration):int type = bodydeclarations *... Abstract Body Declaration. S.R. Chidamber, C.F. Kemerer: A Metrics Suite for Object Oriented Design; IEEE Transactions on Software Eng.; Vol.20/Nr.6/994 4 ownedelements Package * * ownedpackages usagesintypeaccess * Abstract Type Declaration * TypeAccess ownedelements * superinterfaces returntype superclass Class Declaration

27 Complex Example: Average Weighted Methods * per Class * (WMC) ownedpackages WMC is the first CK-metric []. There different Abstract commonly used weights; here we use cyclomatic complexity Model bodydeclarations Abstract ownedelements Package Type Declaration def classes(model:model):oclcollection[classdeclaration] = Body usagesintypeaccess * * model.getownedelements() Declaration.collectClosure(pkg=>pkg.getOwnedPackages()) TypeAccess.collectAll(pkg=>pkg.getOwnedElements()).collectClosure(typeDcl=> typedcl.getbodydeclarations().selectoftype[classdeclaration]) Method Declaration def WMC(model:Model):Double = classes(model).stats(clazz=> clazz.getbodydeclarations().selectoftype[methoddeclaration]().sum(method=>cyclmaticcomplexity(method))).average def cyclomaticcomplexity(method:methoddeclaration):int =... * type * ownedelements superinterfaces returntype superclass method Class Declaration Abstract Method Invocation. S.R. Chidamber, C.F. Kemerer: A Metrics Suite for Object Oriented Design; IEEE Transactions on Software Eng.; Vol.20/Nr.6/994 4

28 Implementation of the OCL-Collection Operations Just in time iterator-based implementation rather than straight forward aggregation of result collections. :RepositoryModel 2 :Rev :Rev :Rev 3 :Par... :Par... :Par... :Par... :Par... :Par... 4 :Diff :Diff :Diff :Diff :Diff :Diff :Diff :Diff :Diff :Diff 5

29 Future Work, Remaining Problems, and Limitations Scalability Heterogeneity Accessibility Information Depth very large compilation units incremental snapshot creation batching OCL execution experiments with large scale repository (e.g. git.eclipse.org) MoDisco for different programming languages common metrics meta-model (e.g. OMG, KDM) VCS abstraction and support for different VCS relating results to software repository entities persisting and exporting results diff-models from comparison of compilation units Very large compilation units (CU): e.g. a 3 MB, 600 kloc CU in org.eclipse.emf tends to have lots of dependencies changes often makes problem even bigger CUs are smallest common denominator between text-based VCS view and syntaxbased AST view smaller units require model-comparison or text-to-ast mappings Support for different programming languages: either abstraction, parallel meta-models, or mixed approach MoDisco is extendable, but only Java support exists; other languages need to be implemented parallel meta-models A reasonable abstraction for multiple (or all) programming language probably does not exist. A shared abstract meta-model that all language meta-models extends could be an sensible compromise. 6

30 Summary Overall model-based MSR with srcrepo works, but it still needs work. 80/20: Uncommonly large CUs are problematic and require complex additions to srcrepo. Ignored for now. Main goal heterogeneity is theoretically plausible, but requires lots of efforts to show practically. Not a matter of if, but of how much. Large experiments are still unfeasible due to lots of small issues rooted in the engineering complexity of the subject matter. 7