Challenges in Mining Whole Software Universe

Transcription

1 Challenges in Mining Whole Software Universe Katsuro Inoue Osaka University

2 Cover Ratio Analyzing Evolution of kern_malloc Lites 1.0 Kernel Source 2 Archive - CMU Mach Lites 1.1.u Kame SimOS Kame Lites The Rio (RAM I/O) Project ftp in The University of Edinburgh 7 Mip-summer98 8 freebsd/sparc 9-12 ftp in Stockholm University 13 freebsd-cam2.1.5r SonicOSX 16 Labyrinth BSD(labyrinthos) 47 Netnice 48 Kame Psumip 51 Netnice 52 Reflexprotocol 53 Netnice 54 NetBSD v OpenBSD PV Xen 56 OpenBSD v Oskit 18 Psumip 57 Pmon Proyecto A.T.L.D GNU/hurd(extremeli nux) 19 Mach 63 OpenBSD v Savannah Unofficial OSKit 23 source Unofficial OSKit source(oskit) ftp in Stockholm 27 University 64 Pmon Chord-ns3 67 Openbsd-loongsonvc 0.4 Results by G(Google Code Search) 1993/01/ /10/ /07/ /04/ /01/ /10/ /07/06 and 2012/04/01 K(Koders) Last modified time : File in New BSD License : File in Original BSD License 2

3 v v1.2.1 v1.2.7 v1.2.8 v1.2.5 v v v v v v v v v v v v v v v v v1.4.1 v v1.4.2 v1.4.4 v v1.4.6beta06 v1.5.1 v1.5.4 v1.5.7 v v1.4.8 v v1.5.9 v v v Analyzing Reuse of Outdated Libraries Vulnerability of 50 OSS Projects Using libpng 0 Vulnerabilities reported No defects reported Result from Google Code Search and Koders 3

4 Experience and Concern Mining source code repositories, e.g., SourceForge, Github, Open Hub, Google Code, Marven,... (BlackDuck) Outcomes heavily depend on repository contents Aren't we mining a small world? There may be many other source code contents in the universe

5

6

7

8

9 Whole Software Universe U Whole Software Universe U Collection of All Software Developed by Human in the Past Open source software Personally-developed software Proprietary software... any others P : Set of all meaningful software U P (a countable infinite set)

10 Questions for U A) How do we get U?? B) What do we mine from U? C) How do we mine U? D) Why do we mine U?

11 A) How Do We Get U? No one knows actual U So we would collect many repositories, and construct a subset U U U should be as large as possible, of course U should reflect characteristics of U Challenges Collecting and unifying different repositories into U Duplication, coherence,... Performance and capacity for U Updating and maintaining U

12 B) What Do We Mine from U? Examples Simple metrics of U over history Size U t1, U t2, Language usage Density of U with respect to P History and evolution of code c in U Origin version of c Closely related code c (clone, variation, family,...) Future prediction for c

13 C) How Do We Mine U (U )? 1. Direct mining Good model Powerful machine 2. Indirect mining Use external services Reconstruct mining result from those external services

14 Direct Mining U Copy of U U

15 Indirect Mining Want to know about U Mashup Engine U Query Decomposition and Result Composition U

16 D) Why Do We Mine U? Objectives of mining U Reuse and knowledge transfer We do not want to reinvent the wheel Historical Archive... Frontier's wisdom

17 Discussion! Is it interesting research topics? Can we get useful research results? Is it feasible research target?

18 Thank you

19