An Eclipse Plug-In for Generating Database Access Documentation in Java Code

Transcription

1 An Eclipse Plug-In for Generating Database Access Documentation in Java Code Paul L. Bergstein and Aditya Gade Dept. of Computer and Information Science, University of Massachusetts Dartmouth, Dartmouth, MA, USA Abstract Enterprise applications typically consist of a web layer, the business logic layer, and a relational database. However, the interaction between these various layers is not sufficiently captured by the current generation of IDE (Integrated Development Environment). For example, current Java IDE's do not evaluate the relationship of classes with the database, or how a particular java method interacts with database tables and columns. We report here our progress in developing an Eclipse plug-in that helps the programmer document the interactions between Java code and relational databases. A primary motivation is to facilitate code maintenance in the face of database modifications. Keywords: Software maintenance, software documentation, CASE tools. 1 Introduction Modern tools have simplified the development of enterprise applications by bridging gaps across various technologies like file systems, relational databases, messaging, and web services. However, this has also led to challenges in maintenance and enhancement of enterprise applications. An enterprise application usually consists of a web layer, the business logic and relational database, often enhanced with frameworks like Struts and Hibernate for web and persistence. However, the interaction between these various layers is not sufficiently captured by the current generation of IDE (Integrated Development Environment). For example, the Eclipse IDE provides support for syntax and debugging of java classes, but it does not evaluate the relationship with the database, or how a particular java method interacts with database tables and columns. For example, it does not flag a warning where an SQL query might be formed incorrectly. Similarly, the Visual Studio.NET would not flag a warning if an XPath applied on an XML document does not correspond to a valid value according to the schema. This makes it very difficult to maintain and enhance applications written by a third party, since a change in code may break some other layer, and the problem will become known only after extensive testing. Our goal is to develop a framework that will help programmers in bridging the gap between different technologies used in an enterprise application. However, this is very substantial initiative and we have so far focused our efforts on developing a suite of tools to support development and maintenance of database applications using Java in the Eclipse environment. We report here our recent progress in developing a plug-in tool for generating documentation of the interactions between Java code and relational databases. The obvious benefit of the documentation is to facilitate code maintenance in the face of requirement changes or database modifications. Certain database accesses may be fairly well self-documenting, e.g. by a simple statically defined SQL statement. However, there are many other situations where additional documentation can greatly enhance understanding the code dependencies on a database. Consider for example a query string that is built dynamically from SQL fragments which may be taken from user input, stored in variables, or passed as parameters from other methods. Furthermore, the point in the code where the query is executed may be far removed from the place where the query string is built. Even in a situation where a query is statically defined and used immediately, it may take some effort to determine which database elements are being read or written if the query is complex. With our tool, the developer may use our find and document interface to locate each database access of interest in their Java source code and selectively generate documentation where desired, in a manner similar to the familiar find and replace function of a word processor. Alternatively, they may choose to automatically generate documentation of all database

2 access with a single click on the Document All button. Our tool also provides considerable flexibility in specifying search criteria and scope as well as documentation style. 2 Background We have previously reported [1-3] our development of tools to provide a visual mapping of java code-to-database and code-to-code (via database) couplings. We have implemented our tool as a fully integrated plug-in to the popular Eclipse development environment. Developers can view the database and the project at various levels of granularity and easily find the types of coupling they are most interested in. For example, users can choose to view couplings of code to anywhere in the database, to a particular table in the database, or to a specific column in a table. Similarly, they can adjust the granularity of their project view between the project, class, and method levels. Search facilities enable users to quickly identify important dependencies. For example, when a method that stores information in the database is modified, it is easy to find all of the methods (or classes or projects) that retrieve the same information and might be affected. The database access of an application is discovered by our visualization tool through a combination of static and dynamic code analysis, or by processing user supplied java annotations. The static and dynamic code analysis approaches each have their relative advantages and disadvantages, but share some common characteristics. Both approaches have the advantage of automatic discovery that makes them suitable for maintaining legacy applications where the database access is not well documented. However, neither is 100% effective in finding all possible interactions between the application and the database. The static code analyzer uses the Sun java compiler API [4] and the Compiler Tree API [5] to parse the java source and walk the abstract syntax tree. It looks for string literals that are included either directly or after assignment to String variables in calls to the execute, executequery, and executeupdate methods of the JDBC Statement class. The analyzer attempts to identify column and table names occurring in select, from and where clauses and record these dependencies in the coupling repository. The code analyzer considers certain string concatenations including some concatenations that are built from a combination of string literals and variables to detect simple cases of dynamic SQL generation in the code. However, static analysis in general is a hard problem and it will never be possible to detect all couplings to the database that may occur at runtime, possibly dependent on user input. The main component of the dynamic analyzer is a JDBC bridge driver that logs the database accesses to the coupling data repository. Our driver acts as a bridge between the application and the "real" driver that communicates with the user's database. The implementation is conceptually simple. Most of the methods in our driver classes simply pass requests on to the underlying "real" driver and return whatever data is returned from the real driver. The main exception is in the Statement class methods (e.g. execute, executequery, executeupdate) that take SQL statements as arguments. These methods receive only complete, valid, fully formed SQL statements as arguments (unless there are errors in the application) even if they have been built dynamically. The SQL statements processed in the JDBC driver are parsed to determine the database elements that are being accessed and the coupling information is recorded in the repository. The main drawback to the dynamic analysis approach is that it will only find database accesses that occur during testing with the bridge driver in place. Therefore the success of this technique is highly dependent on the developer s ability to generate adequate test cases. In [3], we describe an extension of our previous work to allow the developer to explicitly supply the database access information in the form of java annotations to replace or supplement the code analysis tools. We chose annotations rather than comments since they are easier to process by machine. For legacy projects, or when the developer hasn't completely documented the database access, code analysis is still extensively used to obtain the necessary information. Our plan was to further enhance the system by adding a tool to inject the access information obtained through static and dynamic code analysis into the source code in the form of java annotations in order to automatically document the code. We realized, however, that annotations are more verbose and harder to read than ordinary comments, and decided to use comments for generated documentation. In the future,

3 we plan to add an option to generate documentation in the form of annotations. 3 Results Figure 1 shows the overall architecture of the system. The system uses annotation processing as well as both static and dynamic analysis of the java code to find database couplings. The results of all analysis methods are combined in the coupling data repository. For every code-to-database coupling that is detected, there is an entry in the repository. Each coupling entry in the repository includes the code location (class, method, file, and line number), the database element (database, table, and column), the SQL statement type (select, insert, update, etc.) and the type of access (read, write, or read/write). The statement type does not necessarily determine the access type. For example, a field occurring in the set clause of an update statement indicates a write access, but a field occurring in the where clause of the same statement indicates a read access. In order to detect changes over time, the repository also records the first time and last time that a coupling is detected. Also, each time the tool is run, the structure of the database is checked using the JDBC metadata API, and any structural changes are recorded in the repository. The user interface, implemented as an Eclipse plug-in, displays the results to the user, allows easy navigation to code based on its database coupling, and allows the user to generate documentation using the Find and Document feature. Figure 2 shows the details of the Find and Document interface, with all the option nodes expanded. The simplest form of the Find and Document interface is shown in the screenshot of Figure 3, where all option nodes are collapsed. The user can select the scope of the search from the drop down menu under the search node. Available options are to search the current file, the current project, or the entire workspace. In the Data Elements section, the user can choose to search only for code that accesses a particular database, column, or table, or they can choose all databases, all tables, or all columns. The dropdown menus are populated with the names of databases, tables, and columns that appear in the repository. Coupling Visualizer and Code Navigator Documentation Generator Coupling Data Repository Annotation Processor Static Code Analyzer Dynamic Code Analyzer Figure 1

4 The user may also choose to find and document database access only of a certain type (read, write, or read/write) or by a particular type of SQL statement (select, insert, update, or delete). Note that the tool will find the place where an SQL statement is executed, which may be far removed from the place where the SQL string is declared or built. Several options are available to customize the generated documentation, including comment style, a prefix to be included in comments, and if the dynamic code analyzer was employed, the option to include recently executed SQL statements. Listing 1 contains a fragment of code from a web application for managing stock portfolios that has been documented by the documentation generator. The generated documentation is highlighted in bold font. In this case, the documented code that executes the query is only removed from the code where the query string is built by a few lines. Even so, the documentation makes it easier to see which database elements are accessed, and the included sample query is much easier to read than the code used to build the query string. Figure 2 Figure 3

5 String datecondition; if (sale.getterm() == Sale.SHORT_TERM) datecondition = " and purchase_date > '" + cutoffdate + "' "; else datecondition = " and purchase_date <= '" + cutoffdate + "' "; String query = "select purchase_date, shares, round(adjusted_basis/shares, 2), + "lot from purchases p, lots l where p.pnum = l.pnum" + " and ticker = '" + ticker + "' and snum = 0" + datecondition + "order by purchase_date"; Connection dbconnection = null; try { PrintWriter out = res.getwriter(); dbconnection = Database.getConnection(); Statement statement = dbconnection.createstatement(); /* Auto-generated Documentation * * Read access to funds.purchases: purchase_date, pnum, ticker * Read access to funds.lots: shares, adjusted_basis, lot, pnum, snum * * Examples: * select purchase_date, shares, round(adjusted_basis/shares, 2), lot * from purchases p, lots l * where p.pnum = l.pnum and ticker = 'FB' and snum = 0 * and purchase_date <= * order by purchase_date */ ResultSet rs = statement.executequery(query); out.println("<html>"); out.println("<body>"); Listing 1 4 Related Work There is a large body of work on software visualization [6-10] and also on database visualization. There is also a good deal of work on reverse engineering of databases and CASE tools that support reverse engineering with visualization techniques. However, we are not aware of any other system designed to support the development and maintenance of software through the visualization or automated documentation of program code dependencies on the database. 5 Conclusions Many researchers have investigated to resolve the dependencies between different technologies involved in an enterprise application. Our tool significantly enhances understanding of dependencies between java code and relational databases. The principal benefit is the ability to more easily maintain application code in the face of structural changes to the database, changes in the format of data stored in the database, or changes to application requirements. We have tested our documentation generator by generating documentation for each of the five tools in our system (see Figure 1), including the documentation generator itself, and our team has found the results to be very useful. Static and dynamic analysis of java code to discover database couplings each have their advantages and disadvantages. Dynamic analysis is easier to implement and will find all couplings that occur during testing. Static analysis is harder to implement and cannot identify couplings that only occur dynamically

6 (e.g. based on user input). However, static analysis may identify couplings that are missed during the testing phase. By combining the results of static and dynamic analysis in a coupling data repository, we get the combined benefits of each. The repository also allows for tracking of changes over time so that areas of code that may be affected by a change could be flagged for the developer. 6 Future Work In the near term, we intend to continue testing our system on a wider range of database applications, particularly other projects under development by our team, and student projects developed in our database courses. We will continue to refine the functionality and user interface based on feedback from our users. In the long term, we plan to extend this tool to handle additional languages and technologies. For example, we plan to extend our java code analyzers to support JSP by analyzing the java snippets embedded in JSP pages, so that we can document couplings of JSP code to the database. This would also allow the visualization tool to display couplings between the presentation layer (JSP) and business logic code that occur through the database in a typical J2EE environment. If all these dependencies between the various layers of a J2EE application can be documented and visualized, the task of maintaining and enhancing such applications would be greatly facilitated. Eventually, we would also like to support additional programming languages such as C# and C++ and add support for ODBC applications. 7 References [1] Paul L. Bergstein, Priyanka Gariba, Vaibhavi Pisolkar, and Sheetal Subbanwad. An Eclipse Plug-In for Visualizing Java Code Dependencies on Relational Databases. In Proceedings of the 2009 International Conference on Software Engineering Research and Practice (SERP 09), Pages 64-69, July 13-16, 2009, Las Vegas, Nevada. CSREA Press ISBN [2] Paul L. Bergstein and Ashwin Buchipudi. Coupling Detection to Facilitate Maintenance of Database Applications. In Proceedings of the 2011 International Conference on Software Engineering Research and Practice (SERP 11), Pages , July 18-21, 2011, Las Vegas, Nevada. CSREA Press ISBN [3] Paul L. Bergstein. Documenting Java Database Access with Type Annotations. In Proceedings of the 2012 International Conference on software Engineering Research and Practice (SERP 12), Pages , July 16-19, 2012, Las Vegas, Nevada. CSREA Press ISBN [4] Java 6 Compiler API rce-code-analysis-using-java-6-compilerapis.html [5] Compiler Tree API ee/index.html [6] G. C. Roman and K. C. Cox. A taxonomy of program visualization systems. IEEE Computer, Vol. 26(12), Pages 11-24, [7] Blaine A. Price, Ian S. Small, and Ronald M. Baecker. A Principled Taxonomy of Software Visualization. Journal of Visual Languages and Computing, Vol. 4, Pages , [8] Jonathan I. Maletic, Andrian Marcus, and Michael L. Collard. A task oriented view of software visualization. In Proceedings of the First International Workshop on Visualizing Software for Understanding and Analysis (VISSOFT), Pages 32-40, [9] Christian Collberg, Stephen Kobourov, Jasvir Nagra, Jacob Pitts, and Kevin Wampler. A system for graph-based visualization of the evolution of software. In Proceedings of the 2003 ACM symposium on Software visualization, Pages 77-86, ACM Press. [10] M. D. Storey, K. Wong, F. D. Fracchia, and H. A. Müller. On Integrating Visualization Techniques for Effective Software Exploration. In Proceedings of IEEE Symposium on Information Visualization, Pages 38-45, 1997.