Reuse and Beyond: Innovative Software Retrieval Approaches

Size: px

Start display at page:

Download "Reuse and Beyond: Innovative Software Retrieval Approaches"

Brice Washington
6 years ago
Views:

1 Palladio Days 2012 Reuse and Beyond: Innovative Software Retrieval Approaches Oliver Hummel SOFTWARE DESIGN AND QUALITY GROUP INSTITUTE FOR PROGRAM STRUCTURES AND DATA ORGANIZATION, FACULTY OF INFORMATICS sdq.ipd.kit.edu KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association

2 Three Problems I ssdfsd I was having three problems: No job, no money, and no clue how to continue. Otto The Movie (1986) Photo: [Andreas Reiner, Wikipedia] Software Retrieval Beyond Reuse

3 Three Problems II Software reuse is the process of creating software systems from existing software rather than building software systems from scratch. [Krueger 92] A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third parties. [Szyperski 02] [ ] And there they were again, my three problems No repository structure, no reusable software, and no clue how to retrieve something. The Reuse Community (1999) Software Retrieval Beyond Reuse

4 Overview I m going to share some insights on software search & retrieval with you 1. Using the web as a reuse repository 2. How to build a software search engine with Lucene and how to use it 3. How to add functional semantics to software searches 4. A Word on proactive reuse recommendations in Eclipse 5. Can we utilize the wisdom of the crowd in software development? 6. Outlook, summary and conclusion Software Retrieval Beyond Reuse

5 Solution Idea I Using the Web (and Google) as a Reuse Repository (2006) with some additional keywords filetype:java class stack void push Object pop Stack +push(o:object):void +pop():object + simple, easy to use + no overhead + precise for simple requests - still too imprecise - dark web not covered anymore - no reliable API for IDE integration Software Retrieval Beyond Reuse

Solution Idea II Build a specialized software search engine (2008) with google-like interface and IDE integration and suport for programming languages, signatures, interfaces etc.

6 Solution Idea II Build a specialized software search engine (2008) with google-like interface and IDE integration and suport for programming languages, signatures, interfaces etc. Use Database or Lucene or both? relational model vs. field-based approach De-Normalizing allows effective software searches in Lucene Stack +push(o:object):void +pop():object name method method msign msign minter minter stack push pop 1_pt:object_rt:void 1_pt:void_rt:object 1_mn:push_pt:object_rt:void 1_mn:pop_pt:void_rt:object 10 million files (~40 GB) in < 3 seconds with.com Software Retrieval Beyond Reuse

7 Simple Applications What can we do with such a software search engine? Keyword Searches Calculator lang:java Name Matching method:push Signature Matching $(float,float):float; Interface Matching isleapyear(int):boolean; Open Source Lookup java.util.stack Library Identification findjar:org.apache.lucene.search.query Software Retrieval Beyond Reuse

8 Simple Applications II Searches for full object interfaces are also possible even in Java/C# syntax public class Matrix { public Matrix add(matrix) {} public Matrix multiply(matrix) {} } Only test cases refuse to deliver test cases..? import junit.framework.testcase; public class CalculatorTest extends TestCase { public void testadd() { Calculator calc = new Calculator(); assertequals(calc.add(2, 1), 3); } } Software Retrieval Beyond Reuse

9 Precision from a Reuse Perspective Preliminary comparison of retrieval approaches has shown limited top 25 precision results for 13 method queries even a full syntax matching is not precise enough how can we support semantics in searches? Software Retrieval Beyond Reuse

Semantic Software Retrieval And what about contractually specified

[Hummel, 2008] and a seamless integration in modern development

Test-Driven Reuse test-driven development candidate selection based on

10 Semantic Software Retrieval And what about contractually specified interfaces? [Hummel, 2008] and a seamless integration in modern development processes and tools? Test-Driven Reuse test-driven development candidate selection based on interface of C.U.T. assessment of semantics based on test cases very precise more candidates by ignoring names in interface expensive, but still precise Software Retrieval Beyond Reuse

11 Success Examples & Discussion [Hummel, 2008] Software Retrieval Beyond Reuse

12 Eclipse Integration Precise reuse recommendations on the fly [CodeConjurer.org] Software Retrieval Beyond Reuse

13 The Wisdom of the Crowd Book Amazon API Recommendations with ParseWeb (Thummalapenta & Xie, 2007) Software Retrieval Beyond Reuse

14 The Wisdom of the Crowd Developers who have coded a Stack have implemented a push, pop and isempty method Towards crowd-based design recommendations (2010) via intersecting interfaces of search result or attributes or dependencies etc Software Retrieval Beyond Reuse

Ongoing & Future Work Component recognition tracing dependencies -> e.g. delivered a full-grown FTP Client goal is to get this into the Merobase index [Kakarontzas &

15 Ongoing & Future Work Component recognition tracing dependencies -> e.g. delivered a full-grown FTP Client goal is to get this into the Merobase index [Kakarontzas & Stamelos] have been working on Facade detection in Open SME EU project [Janjic & Atkinson subm.] Test case recommendation Software Retrieval Beyond Reuse

16 Conclusion Numerous applications for software retrieval beyond reuse Code Recommendations Design Recommendations explorative searches Snippet Reuse Component Reuse Library Reuse Back-to-Back Testing Test Recommendation Libray Lookup Open Source Lookup concrete searches Defect information Impact Analysis Program Understanding Software Retrieval Beyond Reuse [Janjic et al., 2010]

17 The End. Thank you for your attention! Time for some questions? or maybe you want to contact me later Software Retrieval Beyond Reuse

18 References C. Atkinson, O. Hummel, W. Janjic, "Search-enhanced testing: NIER track", in Software Engineering (ICSE), rd International Conference on, p O. Hummel, C. Atkinson, "Extreme harvesting: Test driven discovery and reuse of software components", in Information Reuse and Integration, IRI Proceedings of the 2004 IEEE International Conference on, p O. Hummel, C. Atkinson, "Using the web as a reuse repository", Reuse of Off-the-Shelf Components, p O. Hummel, W. Janjic, C. Atkinson, Evaluating the Efficiency of Retrieval Methods for Component Repositories, Proceedings of the Int. Conf on Software Engineering & Knowledge Engineering, O. Hummel, W. Janjic, C. Atkinson, "Code conjurer: Pulling reusable software out of thin air", IEEE Software, vol. 25, no. 5 p O.Hummel, Semantic Component Retrieval in Software Engineering, PhD thesis, University of Mannheim, O. Hummel, W. Janjic, C. Atkinson, "Proposing software design recommendations based on component interface intersecting", in Proceedings of the 2nd International Workshop on Recommendation Systems for Software Engineering, p W. Janjic, O. Hummel, C. Atkinson, "More archetypal usage scenarios for software search engines", in Proceedings of 2010 ICSE Workshop on Search-driven Development: Users, Infrastructure, Tools and Evaluation, p G. Kakarontzas, I. Stamelos, S. Skalistis, A. Naskos, "Extracting Components from Open Source: The Component Adaptation Environment (COPE) Approach", in Software Engineering and Advanced Applications (SEAA), th EUROMICRO Conference on, p S. Thummalapenta, T. Xie, "Parseweb: a programmer assistant for reusing open source code on the web", in Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, p Software Retrieval Beyond Reuse

19 Three Problems Again Otto: There they were again, my three problems. Sylvia: Are you going to make me happy tonight?" Otto: Yeah." Sylvia: Are you going to make me happy twice? Otto: Yeah. Sylvia: Even three times? Otto: There they were again, my three problems Software Retrieval Beyond Reuse

Continuous the research Software Engineering. Cycle through Test-Driven Search the teaching

Continuous the research Software Engineering. Cycle through Test-Driven Search the teaching Overview 1 1. Find out why software engineering is important see some software engineering failures 2. Integrating Get acquainted with Reuse into the Rapid, the Chair of Software Engineering Continuous