Error Propagation in Large Software Projects

Transcription

1 Error Propagation in Large Software Projects M. Faisal Shehzad, M. IkramUllah Lali, M. Idrees and M. Saqib Nawaz Department of Computer Science & IT, University of Sargodha, Sargodha, Pakistan Abstract. In software development, various debugging techniques are used to remove errors that are found in the program. However, in some situations, errors still remain in the code after debugging process. Also, debugging provides no mechanism to find out pattern of bugs in different versions of the project. In this article, our focus is to find the reasons why these errors propagate in different releases of the software and how the developers can focus on it. We focused on extending debugging process by including the functionality of mining software repositories in order to find out those patterns of errors. Source code as per revision of the projects can be found on the central servers. Our job is to investigate how these errors propagate between different revisions of project and to apply the software mining techniques on the repositories to extract the useful information to detect real defects in projects and finding the pattern of errors that propagate in different revisions of projects. Keywords: Software Development Software Repository Data Mining Error Propagation Eclipse CVS Source Code Revisions. 1 Introduction Software development refers to activities performed to develop software. There are different stages in software development that can be organized in the form of System Development Life Cycle (SDLC). SDLC consist of different phases that include planning, analysis, design, implementation, testing, software deployment and maintenance. Different models are introduced to represent the activities of software development such as Water Fall model, Spiral model, Rapid Application Development (RAD) model and Agile SDLC model [1]. Software engineering is a combination of different techniques and tools and a systematic, disciplined and quantifiable approach that helps in developing cost effective, reliable and quality software in time according to the users requirements [2]. Different process models in software engineering are used for requirement gathering, analysis of requirement and architecture, testing and maintenance [1]. A major area of software engineering that is now in focus is repository mining [3]. Many researchers and developers are using software repositories to extract and analyze source code. Software repositories are the storage locations for software products. A large amount of data is stored in repository. As the software development is performed, data about the software and its releases is stored in central repositories. Software repositories are valuable source of information for tracking the elements of defects in software products. Concurrent Versions System (CVS) [5] is a free version control in the software development field. Eclipse [6] is a popular and widely used open source framework tool and application. Knowledge discovery or data mining is a technique of analyzing data from different perspective and generating a summarized and useful form of information. Data mining provide different tools for analyzing data from multiple dimensions, categories or from different angles to construct and identify the relationships between data elements [7]. Usually data mining is applied to find out patterns and trends about data and helps in decision making about business. In broader sense, we can say that data mining provides the patterns and associations between different data components in large data storage such as a large database [8]. Data

2 2 M. Faisal Shehzad et al. mining is also used for knowledge discovery from large databases using historical patterns and identifying future trends [9]. Debugging process came soon after the development of programming languages [26]. From the beginning of debugging to the present day, researchers and developers have proposed a surplus of tools to find errors in the programs. Although different debugging tools are commonly used and are very effective but they are really missing one specific area that we are talking about, is the repository mining. In this technique, we analyze different releases of the same project to see the errors that were not caught by debugging tools and we will use this idea along with existing debugging rules to further investigate these type of errors and the pattern of their propagation. 1.1 Related Work Software maintenance is the process of modifying a software system or component after delivery in order to correct reported bugs and errors, to improve performance of the software or to adapt the changing environment as well as due to other attributes [2]. Different static analysis tools are available that verify the properties of produced code and are available as part of development tools. Moreover, some tool like Lint or Metal searches for patterns to flag common errors [3], but they can produce false positive which frustrate customers. Benton and Fischer [10] introduced a framework named as DIMPLE for Java, that provides not only representation of byte code in database of relations but also a declarative domain specific language of analysis as queries over the database. Error propagation is properly understandable only when all the code in all the releases of the softwares are available for inspection. In program analysis, the main task is to analyze the errors. Once an error is understood we may rewrite, reset or can find its propagation [22]. Change propagation is the reason of error propagation. Change propagation means changes required to other components of the software system to ensure the consistency of assumptions in a software system after a particular change in the software [11]. In 1972, Parnas [23] introduced a technique of information hiding in designing systems that was used for the change propagation. Technique based on Aspect Oriented Programming (AOP) encapsulates concepts which cross-cut the structures [24]. In [25], they designed a tool named C-REX for the extraction of data from the repositories. Gall et al. [12] in their article proposed a technique named as CAESAR for the pattern detection. Co-change is one of the most frequently used techniques for mining version archives. Zimmermann et al. [9] used annotation graph for mapping co-changed lines at line level. The article introduced an algorithm named as GNU s diff for finding difference between different version of the same software [9]. A tool named as CVSSearch was proposed and developed by Chen et al. [13]. This tool annotates the source code with the help of log messages. Data mining techniques are also applied by Zimmermann et al. [14] and Ying et al. [15] in their papers, to map co-changes in the related methods and files of the revision. Mapping the lines of different revisions is the main focus now a days. In [9], they introduced an annotation graph for mapping lines. This graph considers each part as one element of a single revision and the nodes are created as single line and edges as origin of line from one revision to other revision due to addition or modification or due to deletion. Our focus is also on mapping lines in different revision but instead of using annotation graph, we have developed our own technique with the help of annotation graph for mapping the lines. 1.2 Overview Much has been written about mining the software repositories to extract useful information for various purposes. But the area that has not been focused too much is the analysis of software repositories for tracking bugs that propagate from version to version and cause the failure of projects. In section 2, CVS and eclipse repositories are analzed for research work.

3 Error Propagation in Large Software Projects 3 In order to analyze software repositories, it is necessary to first extract raw data from the version control and problem tracking systems. But these systems are designed for keeping the specific purpose in mind and are not useful to analyze the exact patterns of defects. In section 3, we present our methodology to analyze software repositories in order to find out not only the error propagation but also their patterns. In Section 4, error pattern recognition and propagation of errors is discussed. In section 5, conclusions are drawn. Main activities in this work involve extraction of code from the eclipse project. Extracted code is then explored at line level and its mapping to previous revisions to find errors that not only propagate in different files of a project but also in different releases of the files in the project. Finally the pattern of errors that propagate are categorized in this work. 2 Concurrent Versions System and Eclipse Repository CVS (Concurrent Versions System) [5] is a tracking system for all the modification and work done in source code. It is used to keep track of all changes in different versions of the software release and synchronize changes by multiple developers in source code. Its structure is combination of directories in a hierarchy which contain source code for revision files. It is based on two main components: repository and working directory. The repository contains historical data about different versions of the same project. Developers can take source code of a revision for editing and developing new version of the file in working directory and then can upload again that code in repository [14]. As the developers modify the code of a revision, add some code, delete some code or may found some errors in the previous version and save changes in the source code, it will be called a new revision and a number like will be assigned to that file. This file is also saved in the repository and it is a different version of a file. The most latest and current revision is called as head revision. We can use log command on root directory of the project in the repository and can extract all information that are available in repository. Eclipse [6] is a popular and widely used open source framework tool and application. It provides Java integrated development environment as well as advance design techniques and implementation. It is an open source but publically licensed and have a lot of extension that can be added by different developers [19]. Eclipse provides intellectual property, development process, IT infrastructure and eco-systems. Our focus is on IT infrastructure as it maintains open source projects and code repositories. Eclipse is continuously evolving to include more sophisticated applications and tools [20]. To develop Java applications, JDT provides plug-in tools, eclipse plug-in s and different editors, refactoring tools, builders, wizards and advance techniques for software developments. The JDT project consist of 5 components APT (Java 5.0 Annotation processing infrastructure), Core (Java IDE headless infrastructure), Debug (Debug support for Java), Text (Java editing support) and UI (Java IDE user interface). Debug component is selected for this study. Debug component of JDT support in code running and debugging. It is compatible with Java virtual machine and can work independent of language by the platform debugger [21]. 3 Methodolgy Historical repositories contains large amount of source code. Projects can be selected form these repositories. Selection of the projects depends upon some factors such as maturity, acceptability, well documentation and supporting community of the projects [20]. Eclipse repositories contain more code, patches, traces and a large numbers of projects as well as it is open source and easily available, so eclipse project named as org.eclipse.jdt.core is selected for data extraction.

4 4 M. Faisal Shehzad et al. 3.1 Data Extraction and Preparation Data extraction is performed on two most important software repositories: eclipse and CVS repositories. Revisions data is taken from CVS [19]. MYSQL database is used to store revisions data. A Java source code named as log parser program is used to store revisions data into MySQL database. This program inserts data into two tables. One of the tables store data of RCS files and its related data. Second table is used to store revision logs of these RCS files. MySQL command CONSOL is used for querying, importing and exporting data. Org.eclipse.jd.core repository contains various directories and we have selected the 5 directories for extraction of difference files. These directories are ant, antadapter, batch, codeassist and compiler. Data extracted form eclipse repository is stored in MySQL database. We have different files in database that contain the different information about the revision files. The first file extracted from eclipse repository is rcsfilesjdtcore log. This file has the log information about the the revision files. The details of the dataset are given in the table 1. The attributes of this file are also given in table 1. These attributes provide us the detailed information of the revision file. But we have to find the difference between different revisions of the same file. Table 1: Data Set with Label and Revision Files Label Decsription Rcscode Revision file id Rf, Wf Revision file repository file and Work file Wfd Work file detailed path Tr Total number of revisions of each file Rev Revision number Date, Time Date and time of revision performed Author, State Which author performed the revision and state of revision Lines0, Lines1 Starting and ending line number for that revision Branches Origin of that revision Des Description of that revision Bug, Bug id Bug reported and id of that bug Change id Change hunk representing the difference area in two files Change code Actual code changed in that revision Loc Total lines of code in each revision To find the difference between revision files, we have created a file named as files log. This file will help us in finding the difference between revision from the CVS repositoties. We have used the files log for finding the difference between revision of the same file. 3.2 Total Lines of Code in each Revision and Source Code Files CVS commands only provide the difference code in only the last revision of the file. This information is not sufficent for our analysis. So we have find the total number of lines of code in each file so that we can extract all code in each revision of the file. We found total lines of code in each revision from eclipse repository by searching through each file. Now we have information about the total lines of code in each revision. We can extract the source code files of each revision. Source code in all the files have been extracted revision wise. Now this source code is available for mapping the lines in different releases. These files are transformed into a database for performing different operation on the source code.

5 Error Propagation in Large Software Projects Reading the diff Hunks In order to map the lines in different revision of the file, we used an algorithm named as GNU s diff. This program find the textual difference between two revisions of the file, the last revision and its previous revision. The program provides the regions of difference where two files differ. This point is named as hunk. The pattern of this command is (e.g 2d5,6). Here integer before d character represents the latest revisions line number and after d represents the previous revisions line numbers. Character c represents the type of change made in this region. Figure 1 shows hunk of two revisions. Fig. 1: Hunk information for two revisions The hunk command provides three type of information. Addition in the revision means some line have added in the next current revision of the file. For example 5a11 means that line 11 was inserted in revision 1.12 of the file. Deletion in revisions such as 5d11 shows that line 5 was deleted in the revision Modification of lines such as 5c6 represents that line 6 has been modified in the current revision. Hunk such as 5,6 c 7,8 represents that line 5 to 6 in revision 1.11 are modified and now these are 7 to 8 lines in 1.12 revision. At this stage we are able to find the difference between all the revisions of the same file. On the bases of these results we can find the propagation of errors and patterns of errors. 4 Error Propagation and Error Patterns Error propagation in large software projects can be found only by thorough study of source code in all revisions of all the files in a project. SCM and other software repositories contains source code of projects file wise as well as revision wise. But these repositories have limited functionality of data comparison at revision level. They only provide the difference of last two revisions only. This information in not sufficient for finding the propagation of errors in all revesion. We have downloaded the source code of all files revision wise form the eclipse repository org.eclipse.jdt.core and store that code in database for further process. Errors may be propagated in different revisions of the file. Eclipse log file provides only the change hunks, having information about the changed area in the two revisions of the file. The data set used for finding the propagation of errors is given in table 2. In table 2, we can see that we have some information of author, error and error id. This information is helpful for finding the propagation of errors. We can identify the error in a file

6 6 M. Faisal Shehzad et al. Table 2: Data Set with Label and Revision Files Label Decsription Rcscode Revision file id File name, Revision no Name of file and revision number of the file Date, Time Date and time at which line or file is modified Author, Status Author of file changed and status of the change Line0 Line change range in previous revision Line1 Line change range in next revision Description Brief description of change Error, Error id Actual Error and error id with reference to file Change id Hunk change between revisions Change code Code in which changes has done by error information provided in log file. How to find error propagation is the question to be answered. To find the propagation of error, we collected the source code. Figure 2 shows the sample code in a file. Fig. 2: Source code sample File in the figure 2 contains only the information about the source code lines of the single file. To identify the propagation, we have to map all the files. For example, we have a line number 2 in revision 1.12 and we want to map that file in all revisions, then we have to map it with line number 2 of 1.11 and up to version 1.1 of the same file. We map each line of the revision with each line of the previous to find the propagation of errors. Now we can perform different analysis on the data set. After manually analyzing the source code at all revisions, we have found 8 similarities of errors in the different releases of the file. These errors represents syntax error, semantic error, logical error or complie type error that occur during the software development process. After analyzing these errors, we have found that these error have been propagated in different revisions of the files as well as in different files of the project and these errors are categorized as follows:

7 Error Propagation in Large Software Projects 7 Input Output Exception Handling Error: Input Output (I/O) exceptions generally occur while reading or writing to a file. This must be handled carefully. During string matching and lines mapping, we have found the missing I/O exception throws. This errors is found in 4 files of different revision. This error indicate missing exception handling or throwing of exception in these files. As this error found in the different revisions and files of the the same project, we can say that errors propagate in different revisions of the same project. Figure 3 shows this error with their file name and related revisions. In ClassPathDirectory.java file at line 14 in revision 1.32, exception handling is added in source code which was missing in version Similary this error found in FileSystem.java at line 53 and other 2 files in figure also indicate this error is found in their revisions. So we can say that generally this type of error propagate in different revision. Fig. 3: Missing exception handling String Literal and NON-NLS Comments: In eclipse framework, NON-NLS are used to represent string literals. The main idea behind NON-NLS is that User Interface (UI) messages should not be treated as string literal. Similarly some strings like regular expressions should not be treated as UI messages. So NON-NLS comments are added with string literal. Any NON-NLS comments without string literal is considered as warning and may produce errors. During string matching, the pattern of this type of error is found in 52 different source files and in different revisions of these file. This also proves the propagation of same error in different releases as well as in different files of the same project. Unnecessary Duplicated Entries: In eclipse framework and in Java, we have two path for any entry to be specified. One is class path and other is source path. Source path is searched for source and class path is searched for binaries. Both of these must be preserved, but generally programmers keep one of them. This may leads to errors. The same situation has been found in 5 different files. The authors are unnecessary duplicating entries in a class. Same type of error found in all the files shows that error propagate in different files. Reading Directory Path Error: In eclipse, it is found that source code may be reading files from different directories at different locations. Directory path must be normalized for

8 8 M. Faisal Shehzad et al. reading and writing to files. If the source code is not supporting the character operations, errors are produced. The same error has been found in 4 different files and in different revisions of theses files. This error represents compile type nature of error. Absolute Targeted Path not Followed: Absolute path and normalized path are two different approaches used in Java. Absolute path may be different in different platform. This may lead not to follow the targeted path in source code. This type of error is found in 4 files of the same directory but in different regions of theses file. This error represents that the path that was indicated in source code in not being honored. Missing Encoding Scheme: As different encoding scheme are available and Java source code must support these encoding scheme. If the source code is reading a file that is not supported by Java then this may produce errors. This type of error is found in two files but in different revisions of the files. It is an error of missing encoding scheme for these files. Figure 4 shows that error found in JDTCompilerAdapter.java at revision 1.69 is repeated in revision Similarly error found in Main.java in revision is repeated again in revision This indicates that same error was repeated in revisions of same file. Fig. 4: Missing encoding schemas Empty String Constants: Constants in Java by nature are single static reference to instances in a thread. String are generally enclosed in ( ). When the compiler parse the code and ( ) appeared in code then the compiler will consider it as a string. Strings are also defined in this manner. How the compiler will not read and treat a string constant just like a string? There is no proper way for this. When this confusion arises, the parser will not parse properly and lead to errors. This type of error is found in 52 files of the same project. As in these files, string and string constants are declared in the same way. The parser may generate the unexpected result. This error exist in different revision of files. Local Variable Scope: Local variable have scope limited to function in which they are declared. In class inheritance, a child class can access the variables that are declared in

9 Error Propagation in Large Software Projects 9 the parent class. If variables in the parent class are declared as private or protected, then child class cannot access these variables or can have limited access. During lines mapping in different revisions, we have observe a pattern where local variable are accessed multiple times in protected mode. It may produce error. This error was found in 8 files. In some files, this error is repeated multiple times. This indicate that errors propagate in different revisions as well as in different files. Figure 5 shows some files having same type of error. Fig. 5: Local variables scope 5 Conclusion In this work, a new approach for finding the evolution of lines as per revision is proposed. This work also addresses the different categories of errors on the basis of dependencies and the error propagation. An eclipse repository org.eclipse.jdt.core is selected for source code extraction. Source code from all revisions of all files is extracted for miming. The work done includes exploring the file at line level and its mapping to previous revisions in order to find errors propagation in different files in a project as well as in different releases of the files in the project. Finally the pattern of errors that propagate is discussed in this work. Data mining is very effective tool for analyzing the source code as per revision but it is an expensive and time consuming process. In future, we will address the factors given below. Enhancing Mining Performance: Finding the error pattern and their propagation requires more time and is extensive. Finding other optimization techniques for pattern recognition will be focused as compared to existing history information techniques. Plug-in Support: The future need is to add this technique in the form of plug in. We will focus on designing plug in to support development tools for the exploration of evolution of line and annotation of lines. Automation of Error Pattern Detection: During the study, it was difficult to detect patterns of errors on the basis of similarity. As tools did not support much in this respect. Our focus in future will be to automate this process. Actual Origin of Lines: In this study we have focused on line number in different revision. Future work will focus on how to apply origin analysis algorithm to find the actual origin of line. This will provide the more close result for line mapping.

10 10 M. Faisal Shehzad et al. References 1. Roger S. Pressman. Software Engineering: A Practitioners Approach, 7 th edition. McGraw-Hill Publishers, IEEE Standard Glossary of Software Engineering Terminology. IEEE Standard, September Jim Whitehead and Thomas Zimmermann. Introduction to the Special Issue on Mining Software Repositories (MSR). Empirical Software Engineering, 17[4-5]: , August Keith H. Bennett and Rajlich. Software Maintenance and Evolution: A Roadmap. In Proceedings of the Conference on The Future of Software Engineering, pp , NY, USA, ACM, Harald Gall, Mehdi Jazayeri and Jacek Krajewski. CVS Release History Data for Detecting Logical Couplings. In Proceedings of the 6 th International Workshop on Principles of Software Evolution, pp , IEEE Computer Society, Eclipse Foundation. About the Eclipse Foundation, 19th July Jiawei Han, Michelline Kamber and Jian Pei. Data Mining: Concepts and Techniquese, 3 rd edition. Morgan Kaufmann Publishers, Bill Palace. Data Mining: What is Data Mining? 15th September Thomas Zimmermann, Sunghun Kim, Andreas Zeller, and E. James Whitehead Jr. Mining Version Archives for Co-changed Lines, William C. Benton and Charles N. Fischer. Interactive, Scalable, Declarative Program Analysis: From Prototype to Implementation. In proceedings of the 9 th ACM SIGPLAN International Conference on Principles and Practice of Declerative Programming, pp , NY, USA, Ahmed E. Hassan and Richard C. Holt. Predicting Change Propagation in Software Systems. In Proceedings of the 20 th IEEE International Conference on Software Maintenance, pp , IEEE Computer Society, Washington, DC, USA, Harald Gall, Karin Hajek and Mehdi Jazayeri. Detection of Logical Coupling Based on Product Release History. Technical Report, University of Zurich, Annie Chen, Eric Chou, Joshua Wong, Andrew Y. Yao, Qing Zhang, Shao Zhang, and Amir Michail. CVSSearch: Searching Through Source Code using CVS Comments. In IEEE International Conference on Software Maintenance, pp , Thomas Zimmermann, Peter Weissgerber, Stephan Diehl and Andreas Zeller. Mining Version Histories to Guide Software Changes. IEEE Trans. on Software Engineering, 31[6]: , Annie T. T. Ying, Gail C. Murphy, Raymond Ng, and Mark C. Chu-Carroll. Predicting Source Code Changes by Mining Change History. IEEE Transactions on Software Engineering, 30[9]: , Sunghun Kim, Kai Pan and E. James Whitehead. When Functions Change their Names: Automatic Detection of Origin Relationships. In Proceedings of 12 th Working Conference on Reverse Engineering, IEEE Computer Society, pp , Michael W. Godfrey and Lijie Zou. Using Origin Analysis to Detect Merging and Splitting of Source code Entities. IEEE Transactions on Software Engineering, 31[2]: , Ranjith Purushothaman and Dewayne E. Perry. Toward Understanding the Rhetoric of small Source Sode Changes. IEEE Transactions on Software Engineering, 31[6]: , Eclipse Foundation. Eclipse Projects, 14th June J. Gonnet. Data Mining within Eclipse: Building a Data Mining Framework with Weka and Eclipse: Diploma Thesis. University of Zurich, Department of Informatics, Eclipse Foundation. Eclipse Debug Project, 14th June Douglas Thain and Miron Livny. Error Scope on a Computational Grid: Theory and Practice. In Proceedings of the 11 th IEEE Symposium on High Performance Distributed Computing, IEEE Computer Society, pp , David Lorge Parnas. A Technique For Software Module Specification with Examples. j-cacm, 15: , Tzilla Elrad, Robert E. Filman and Atef Bader. Aspect-Oriented Programming: Introduction. Commun. ACM, 44[10]:29-32, Ahmed E. Hassan and Cahmed E. Hassan. Mining Software Repositories to Assist Developers and Support Managers. Technical Report, University of Waterloo, Chris Parnin and Alessandro Osro. Are Automated Debugging Techniques Actually Helping Programmers? In proceedings of the 2011 International Symposium on Software Testing and Analysis, pp , 2011.