A Method of Detecting Code Cloning in Open Source Software

Size: px

Start display at page:

Download "A Method of Detecting Code Cloning in Open Source Software"

Elijah Bernard Lamb
5 years ago
Views:

1 A Method of Detecting Code Cloning in Open Source Software Z. Wei 1 and D. Deugo 2 1 The School of Computer Science, Carleton University, Ottawa, Ontario, Canada 2 The School of Computer Science, Carleton University, Ottawa, Ontario, Canada Abstract - In this paper, we discuss an approach to detect code copying in open source software. This approach generates a small fingerprint that includes the main features of the class files contained in a Java Archive (Jar) file. By comparing the intrinsic and extrinsic features of the class files, the approach can find the cloned code in anonymous Jar files. This approach provides a tool to detect cloned code correctly and effectively Keywords: Open Source Software, Code Cloning, Intrinsic Features, Extrinsic Features 1 INTRODUCTION Open source software (OSS) has become critical for most organizations, as there are many advantages of using it [1]: such as high-quality software, lower software costs, numerous technical support and accountability, active communities, and less dependence on vendors. OSS has impact on more than developers and IT-managers but also potentially for all the people throughout the value chain of an organization including suppliers, customers, and partners. Today, more and more OSS is developed and adopted in commercial products. Although organizations have already obtained benefits from OSS, they have to take a critical consideration of OSS and face some questions raised by using it. For example, OSS uses licenses to protect itself and its developers. However, many of these licenses are different and mutually incompatible. Before using OSS, organizations must survey the licenses of the software and obey them to develop their commercial products. However, code of unknown origin is encountered occasionally in development and its pedigree must be made clear. This action is related to the legality and success of software products. 1.1 Problem In this paper, we look at detecting software clones that are written in the Java programming language. We use the known software code to compare with the software that is copied from an anonymous source. After comparing, we can determine whether an organization is copying another s code illegally. There are many existing clone detection methods, such as string matching. But sometimes those methods don t work when the original source code is not available. To avoid this disadvantage, our method focuses on accessible attributes of class files and calculates the similarity of their attributes. 1.2 Motivation The Open Source community attracts very bright, very motivated developers, who although are frequently unpaid and lack disciplined [2]. For many developers, peer review and acclaim is important. They prefer to build software with clean design, reliability and maintainability that is admired by their peers. They develop software to contribute the open source community and get benefits from others contribution. But there are some phenomena, to which we have to pay attention, destroying the balance of the open source community. Some organizations or individuals copy intellectual property, and just make it their own by modifying the code with out any legal right to do so. This behavior shows a complete lack of respect for other developers work and creates an attack on the ecosystem of the community. We must take action to prevent these unethical behaviors and protect authors rights and the community. This is our main motivation for detecting cloned code in OSS. The problems of exploiting OSS to improve one s own product and avoiding infringing others intellectual property are concerned by commercial software developers and project managers. Some issues [3] are related to this consideration: Some commercial software suppliers have been sued by open source advocates for downloading the open source while ignoring the license obligations. Customers want to know the details of the software that are provided by IT companies. Software engineers use some open source in their programs but they do not declare them. Some OSS licenses may influence the intellectual property of software products which include the corresponding open source code. Based on those issues, organizations must take measures to protect against these legal risks of open source. They should periodically conduct a complete audit of their source code, making sure developers and managers know exactly what open source is inside and what the license obligations of that open source are. But the price of the service supported by a professional audit firm is high, and sometime there is no access to the original source code [8]. We devote our focus to audit open source software with low cost. 1.3 Goals There are many available applications detecting code cloning with simple string-matching [10]. We pay more attention to the specific attributes of a class, such as parameters of methods and return value. We take those features from a Jar file to create a unique footprint and then calculate the similarity between the known Jar s fingerprint and an unknown Jar file. The application outputs the result to display a match percentage between the two Jars.

2 1.4 Objectives Several clone detection techniques have been described and implemented [8] [9] [10]. But most of those techniques require the original source code and require much memory. Our objective is to make a small fingerprint using the particular features of the compiled classes in a Jar file and not all the source code. Our approach does not rely upon the original source code because it does not tend to be available in the Jar files. We mainly pick up basic features from the Jar files. Although we utilize less information that describes the features of the code, it does not mean low accuracy. Following Walenstein [4], clone detection adequacy depends on applications and purposes. Intellectual property thieves may modify the name and order of methods but rarely features of the class. Class files contain distinguished features that are the key points to comparing two files. Comparing these features is the core idea of our approach. 1.5 Outline In section 2, we will discuss relevant background work. In section 3 we concentrate on our design, algorithm and what decisions we made. In the section 4 and section 5, we compare several groups of files, present results, draw our conclusions and look at future work. 2 BACKGROUND Much research in the field of clone code detection has been done. Most of it has a high degree of accuracy. However some disadvantages exist in that software, e.g. time-consuming, memory-waste, and especially requiring the source files. Cate Huston [10] uses winnowing to fingerprint Jar files and some potentially interesting information (e.g. filenames, size of the Jar file, the number of entities, and the Jar name) of the Jars to detect significant similarities of those Jars. From the conclusion of that paper we can learn that those potentially interesting information are less consistent. This demonstrates what can be changed in the software clone and the key point of comparison is the features or contents of methods. Some researchers have paid attention to the situation that there is no source code available but compiled code. Carson Browns and David Barrera [8] use the modification of n-gram method to generate the fingerprint of the compiled Java program. They make much improvement including detection speed, small fingerprint and good accuracy. Patrice Arruda et al [9] use graph to describe the dependency of classes and calculate the similarity with matrix. Their approach focuses on the relationship between classes. We follow this research. We also take Jar files as input, and generate the fingerprint of the Class files that are Java s compiled files. The difference between our approaches and [8] is that we do not consider all the byte code of methods in Class files but just the features of Class files, e.g. the number and type of the input parameters of methods, the type of the return value of methods and the like. These features are termed as intrinsic features in our approach. Although software clones can include many modifications, such as renaming classes, variables and methods, adding some inessential code lines, and changing the path of files, the main function of the method will not be changed. In our approach, the relationship between the classes is considered as the extrinsic features. The idea of extracting extrinsic features of a class in our approach is enlightened by [9]. 3 APPROACH 3.1 Design Considering that Java Jar files mainly comprise class files, our approach analyses the class files and uses features of class files to describe fingerprints of Jar files. We divide the features of class files into two groups: intrinsic features and extrinsic features. Intrinsic features of a class file include the description of its methods. The number and types of input parameters and the type that the method returns compose the basic features of a method in a class. The extrinsic features are composed of the relationships of classes, for example super classes, interfaces, and inner classes of a class. These features can indicate the purpose of the class file so that the approach can distinguish the behavior of cloning. Employing these intrinsic and extrinsic features we discover the cloning code. 3.2 Rules Generally, Jar files are kinds of Zip files. Thus before extracting features from class files, we need to decompress the Jar file using the Java Class Foundation Library API [5], java.util.jar, to finish this job. We use the JarResource class which is from a Java World article [6] to obtain the byte codes of all classes in a Jar file. The org.netbeans.modules.classfile API [7] can transform the byte codes of a class to an object in memory. Therefore our algorithm can manipulate class files to get all the features of methods in a class file. To clarify our rules of calculation, we assume that is an original Jar file which is used to generate a fingerprint and is an anonymous Jar file which is possible the cloning Jar file. denotes a Class file that belongs to. denotes a Class file that belongs to. denotes a method that belongs to. denotes a method that belongs to. In our algorithm, we use eight basic rules to demonstrate how to calculate the similarity of two Jar files. Based on the similarity, the cloning Jar file can be detected correctly. The similarity of two methods is calculated by Rule 1: Rule 1: If two methods have same number of parameters, same type of parameters and the same type of return value, then the similarity of two methods similarity is 0. is 1; otherwise, the If the similarity of two methods is obtained, the similarity of intrinsic features of two classes can be analyzed by Rule 2: Rule 2: the similarity of classes S in class = n k =1 S method n

3 is the number of methods in a class. The similarity of extrinsic features of two classes can be calculated by Rule 3 to Rule 6: Rule 3: the similarity Rule 4: the similarity classes are same; otherwise is 0. Rule 5: the similarity is 1 if the types of super is 1 if the number of interfaces is same and the types of interface are same; otherwise is 0. Rule 6: the similarity is 1 if the number of inner classes is same and the types of inner classes are same; otherwise is 0. The similarity of two classes is calculated by combining and as Rule 7: Rule 7: S class = 0.6S in class + 0.4S ex class The weight of is set to sixty percentage points because our approach mainly focuses on methods in a class. The similarity of two Jar files is calculated by the Rule 8: Rule 8: is the number of classes in the original Jar file, indicates the maximum of similarity between and, is the number of classes in the anonymous Jar file. 3.3 Algorithm Our approach compares the original Jar file and the anonymous Jar file with two components. The first component is Fingerprint Generator that manages to generate a fingerprint from the original Jar file. Figure 1 describes the main steps of generating fingerprint. The Fingerprint Generator visits every class in the original Jar file, obtains the intrinsic features and extrinsic features of a class and finally produces the fingerprint of the Jar file. The second component is Fingerprint Detector that computes the similarity of two Jar files through eight rules mentioned. Figure 2 describes the main steps of generating the similarity. 3.4 Decisions Made Our approach only focuses on class files in a Jar file because class files are the core of a Jar file. The similarity between two methods is simply calculated by Rule 1 and expressed by 1 or 0. There is no value of similarity between 0 and 1. As a prototype of first implementation, we intuitively keep the algorithms simple. Although this calculation losses some accuracy, the intention can be described clearly and results are reasonably. We also only consider the basic types of Java such as int, short, long, byte, float, double, string, char and boolean, and the original objects of Java such as Integer. Except these primary types, other self-defined types are uniformly defined as Object type. Through these simple categories of types of parameter, our approach can cover most cases when comparing two methods. In the course of implementing the algorithm, some classes in the different Jar files are very similar but the similarity calculated by our algorithm is not high. However, some classes are different but the similarity is high. Through analyzing the source code artificially, we find that these classes have several methods that have no parameters and no return value. This circumstance will influence the result of comparison. Assuming that there are three classes A, B and C, Class A has 6 methods (X1, X2, X3, X4, X5, X6) among which there are two methods (X5 and X6) have no parameters and no return value. Class B has 4 methods (Y1, Y2, Y3, Y4). Class C has 4 methods (Z1, Z2, Z3, Z4) among which there are two methods (Z3 and Z4) have no parameters and no return value. Assuming X1, X2, X3 are equal to Y1, Y2, Y3, respectively, so the similarity of Class A and B is 0.5. Assuming X1, X2 are equal to Z1, Z2, respectively, the similarity of Class A and C is 0.67 because Class C has two methods Z3 and Z4 which have no parameters and no return values, and Class A also has X5 and X6 which have no parameters and no return values. Even if Z3, Z4 and X5, X6 are entirely different, the similarity is high based on rules. Actually, Class A is more similar to Class B than Class C. Thus methods of no parameters and return value have negative effect in our algorithm. In most classes, methods of no parameter and return value tend to be less important. Considering this situation, we decided to remove methods that have no parameters and return value when calculating the similarity of methods. 4 RESULTS To examine the correctness and effectiveness of our approach, eight selected Jar files are used as the testing set. The environment of testing and the performance are described first. Then the testing results and analysis are discussed. 4.1 Performance The implementation runs on a laptop. Its technical parameters are shown as below: Processor: Pentium(R) Dual-Core CPU 2.30GHz Memory: 4.00 GB System Type: 64-bit Operating System The size of Jar files ranges from 1Kb to 2.7Mb. There are four Jar files that come from Eclipse IDE [11] plugins, two files from our own implementations, and two files from Spring Framework [12]. The worst case is the largest pair of Jar file. Our approach spends seconds for comparison. Although the algorithm runs slowly, the results were good.

4 Figure 1. Fingerprint Generator Figure 2. Fingerprint Detector

5 4.2 Testing Results In the set of testing Jars, we first choose the simplest Jar file, test.jar, which contains one class including three simple methods, to compare with this Jar itself. The result is very good as is the execution time. Next, when comparing two entirely different Jar files (test.jar and com.zxwei.comp5900.jar), the result computed by our approach is correct. The first two Jar files are devised to test our basic idea. The Jar files, org.eclipse.help.ui_3.2.0.v jar and org.eclipse.help.ui_3.5.0.v , were obtained from Eclipse plugins folder under Eclipse IDE. They have similar functions, but they are different versions. The similarity of org.eclipse.help.ui_3.2.0.v jar to itself is first calculated and then the similarity to org.eclipse.help.ui_3.5.0.v is done. The result shows that the former s similarity is higher than the latter s. This data is reasonable because the difference between two versions is larger than one version itself. Working with the different versions of Jar files of JUnit [13], we discover that their similarity is low because JUnit version 4 has many modifications of structures and functions compared with JUnit version 3.8. JUnit version 4 introduces the annotation function to facilitate programming test units. Our approach exploits the number of classes in the first Jar file. Although the similarity is different, the time consumed by calculation is almost equal. Finally, we compare two larger Jar files, one from Spring Framework which is 2645KB and another from Spring Framework 2.5 which is 2773KB. The result reveals that the similarity of these two Jar files is 89.07% and the calculation execution time is s. This indicates the higher version of the Spring Frame has some modification but the main part is similar to the old one. Table 1 lists the all our results of our testing set. Through the testing, our approach shows the good performance on both correctness and efficiency. 5 CONCLUSION Our approach of detecting code cloning in open source software focuses on class files in Jar files. We believe that utilizing the intrinsic and extrinsic features of classes can effectively seek out code cloning and our approach is immune to simple refactoring. Although our approach has slow execution speed, the result were correct and accurate. 5.1 Review goal and contributions Our algorithm and implementation fulfill the objective of detecting code cloning and provide a tool to help companies or institutions, who want to use open source software, to detect the existence of copied code in their software or software they use, thus avoiding lawsuits from distributed or using code with license incompatibilities. 5.2 Future work Our approach has three main aspects for improvement. In the calculation of the similarity of two methods, we can set different weight to the number of parameters, types of parameters and types of return value instead of a Boolean value (0 or 1). We believe this improvement can further increase the accuracy of our algorithms. In the aspect of extrinsic features, attributes of a class and references of other objects in a class are both important properties of a class. Adding these features into the calculation of extrinsic features can promote the accuracy of calculating the similarity of two classes. In the aspect of intrinsic features, considering structures of if-else and while clauses can strengthen detailed description of a method, thereby increasing the accuracy of the algorithm. 6 REFERENCES [1] Jason Williams, Peter Clegg, Emmett Dulaney. Expanding Choice: Moving to Linux and Open Source with Novell Open Enterprise Server. Sams, Mar 7, [2] Nic Peeling, Julian Satchell. Analysis of the Impact of Open Source Software. QinetiQ, [3] Manage The License Obligations and Risks That Come With Developing Commercial Software While Embedding Open Source Inside. Accessed December 4, [4] A. Walenstein, Nitin Jyoti, Junwei Li, Yun Yang, and Arun Lakhotia. Problems creating task-relevant clone detection reference data. In Proceedings of the Working Conference on Reverse Engineering (WCRE 03). IEEE Computer Society Press (2003): [5] Java Platform API Specification Accessed December 4, [6] Arthur Choi, Java Tip 49: How to extract Java resources from JAR and zip archives, JavaWorld.com, January 3rd, 1998, Accessed December 4, [7] org.netbeans.modules.classfile, Accessed December 4, [8] Carson Brown, David Barrera, Dwight Deugo: FiGD: An Open Source Intellectual Property Violation Detector. SEKE (2009): , [9] Patrice Arruda, Pierre Chamoun, Dwight Deugo: A Framework for Detecting Code Piracy Using Class Structure, The 22nd International Conference on Software Engineering and Knowledge Engineering (SEKE 2010), Knowledge Systems Institute Graduate School, , [10] Cate Huston, Fingerprinting Jar Files Using Winnowing, %20using%20winnowing.pdf. Accessed December 4, [11] Eclipse IDE, December 4, [12] Spring Framework, Accessed December 4, 2010.

6 TABLE I. [13] JUnit, Accessed December 4, TESTING RESULTS Fingerprint Generated From Anonymous Java Jar File Certainty Time (s) test.jar test.jar 100% test.jar com.zxwei.comp5900.jar 0% org.eclipse.help.ui_3.2.0.v20060 org.eclipse.help.ui_3.2.0.v % Jar Jar org.eclipse.help.ui_3.2.0.v20060 org.eclipse.help.ui_3.5.0.v % Jar Jar junit-3.8.jar junit-3.8.jar 98.0% 1.01 junit-3.8.jar junit-4.1.jar % junit-4.1.jar junit-4.1.jar % spring jar spring-2.5.jar 89.07%

Creating a Lattix Dependency Model The Process

Creating a Lattix Dependency Model The Process Whitepaper January 2005 Copyright 2005-7 Lattix, Inc. All rights reserved The Lattix Dependency Model The Lattix LDM solution employs a unique and powerful