Elucidative Programming in Open Integrated Development Environments for Java

Elucidative Programming in Open Integrated Development Environments for Java Thomas Vestdam Department of Computer Science, Aalborg University Fredrik Bajers Vej 7E 9220 Aalborg, Denmark odin@cs.auc.dk ABSTRACT In this paper we present an integration of tool support for internal documentation in the modern IDE TogetherJ. The specific kind of internal documentation stems from the Elucidative Programming tradition where documentation and source code are separate entities tied together using hyperlinks. The tool supports the programmer in creating and managing internal documentation of Java source code. The hypertext-based documentation is presented in the IDE where a range of navigational features helps the programmer with finding relevant documentation for a given part of the source code. It is concluded that integration of such a tool is indeed possible as well as easy to do in an open IDE like TogetherJ. We expect that we can obtain similar results using other open IDEs. Categories and Subject Descriptors D.2.7 [Software Engineering]: Distribution, Maintenance, and Enhancement Documentation; H.5.4 [Information Interfaces and Presentation]: Hypertext/Hypermedia Navigation General Terms Documentation Keywords Documentation, Elucidative Programming, Java, Literate Programming 1. INTRODUCTION Modern integrated development environments (IDEs) for Java provide programmers with several features that make code production easier and faster. Such features include code completion, dynamic syntax highlighting, convenient Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Copyright 200X ACM X-XXXXX-XX-X/XX/XX...$5.00. code navigation, code templates and help in performing various refactoring tasks. In addition, IDEs help programmers with managing and configuring code projects. For example, automatically setting up web-servers and deploying complex web-service projects as well as automatically configuring ordinary applications. With the advent of JavaDoc [8] interface documentation now plays an important role in software development. Most IDEs for Java therefore provide support such as JavaDoc popups during code completion, JavaDoc indices, and wizards. Wizards help the programmer with producing actual JavaDocs for a given code project, detecting undocumented interfaces, and inserting code comment templates. In addition, some IDEs [19] support UML modeling which, at least at an overall design level, can be considered as a way to document software as well. Although interface documentation is popular and does preserve an understanding of the different interfaces between the components of a program it does not preserve an actual understanding of a program. Internal documentation is a more thorough kind documentation that captures an understanding of the internal properties of a program. Internal documentation is intended for current and future developers and explains how and why the program works in order to preserve the program understanding over time. As time passes the understanding of how and why a program works disappears. Later on, when the program is to be maintained or further developed even the original programmers will have a hard time re-gaining an understanding of the program. A well known form of internal documentation is Literate Programming [10]. A literate program is represented as text interleaved with fragments of a program. Each program fragment is explained in the surrounding text, and as such a literate program can be seen as an essay explaining the program. Due to the practical problems that arise when program and documentation are represented as a single entity an alternative approach called Elucidative Programming has been coined [12, 11, 13]. In Elucidative Programming the program and the documentation are separate entities. Explanations in the documentation are related to relevant program fragments through bi-directional links. The links allow navigation from an explanation to the specific place in the program that is being explained as well as navigation from program fragments to explanations in the documentation. Writing internal documentation is often thought of as a time consuming task, although internal documentation is often considered a desirable supplement to source code. How-

ever, modern IDEs lack support for writing internal documentation. Such tools are often standalone tools such as the tools known from the world Literate Programming [9, 14, 15]. Some of these tools include functionality known from modern IDEs, but they never become nearly as powerful. It is simply too big a task or not feasible for a small team of researches to implement an entire IDE. Other standalone tools are non-integrated tools. For example, the current tool support for Elucidative Programming is a nonintegrated tool using Emacs for editing documentation and source code, whereas a web-browser must be used in order to utilize the bi-directional links when reading documentation. This setup disturbs the work of the programmer as it is necessary to switch between the two tools and manually bring what is in focus in one tool into focus in the other tool [22]. Hence, the aim of the research presented in this paper is to investigate how internal documentation can be created, managed, and presented in a modern IDE. Making use of an already existing IDE relieves us from developing a programming environment from scratch. In addition, many real programs are written using modern IDEs. It is therefore desirable to have support for internal documentation as an integrated tool in an IDE. The main contribution of this paper is to demonstrate that it is possible to integrate tool support that provides tractable navigation between internal documentation and Java source code in a modern IDE. As a minor contribution we demonstrate that it is possible to integrate our own programming tools in a modern commercial IDE. The rest of this paper is organized as follows. Section 2 provides a short background on Elucidative Programming. Section 3 presents the considerations done prior to integrating support in an IDE along with some related work. Section 4 describes the integrated elucidative support in TogetherJ. Finally, in section 5 we conclude on the work done so far and outline some future work. 2. TOOL BACKGROUND Elucidative Programming [12, 11, 13] is a variation of Literate Programming and was coined because Literate Programming was found to be unfit for modern software development [13]. When using literate programming, the programmer must split the program into pieces and physically embed these in the documentation. This is often a nuisance for programmers and programming tools. It is more attractive to keep program and documentation separated, which also allows an independent evolution of both program and documentation. Elucidative Programming is general useful in any situation where there is a need for writing about a program, such as when writing internal documentation, during code reviews, and when writing program tutorials [22]. In internal documentation we wish to represent understandings of selected aspects of the program, much like in program tutorials where we wish to inform the user about the internal properties of a program especially with the goal of informing the user on how to use the program for construction or composition of another program. Elucidative Programming, unlike Literate Programming, allows us to write about a program without altering the program. Furthermore, Elucidative Programming allows us to explain source code without drowning the source code in code comments. In order to tie program and documentation together the programmer can create links from documentation to abstractions in the program. This is for example useful when explaining how aspects that involve several parts of a program are implemented because the programmer can point out the actual source code that is involved. Such aspects are for example instances of design patterns in a program [22]. As Elucidative Programming supports addressing several program fragments in a single paragraph of documentation we can document an instance of a design pattern using a few paragraphs of text that relate all the involved program fragments. We use the term transverse issue for an aspect of a program whose description involves details from separate program units [22]. We also find a use for writing about the program in review situations where we wish to annotate parts of the program. Using Elucidative Programming reviewers can create documents that annotate the source code, and following programmers can easily find the actual parts of the source code that the reviewers has commented on. Similar, Elucidative Programming also makes it feasible to produce traces from requirement specifications to actual source code [22]. Currently, there exist tools for Elucidative Programming in Scheme [11] and in Java [13]. Both these tools are supported by Emacs [16, 17] at the editor level and by an ordinary web-browser at presentation level. The tool used in the work presented in this paper is The Java Elucidator, which has been evaluated through two experiments [20, 5]. An elucidative program is a collection of nodes (documentation and source code files) and links that tie the nodes together. As with any hypertext system, without feasible navigation between nodes programmers risk getting lost in the information. Mutual navigation between program and documentation is therefore provided in a two-framed layout where links from documentation to source code are rendered as hyperlinks anchored at both destination and source. This makes it easy to spot the parts of a program that are documented and bring up the corresponding documentation. The parts of a program that can be addressed from documentation are named syntactical abstractions such as classes, methods, attributes, and variables. In addition, by placing special comments, called source markers, in the code it is also possible to address arbitrary points in the source code. The Java Elucidator utilizes language knowledge and a database for storing information on program abstractions as well as abstractions in the documentation (links, sections and documents). This enables the Java Elucidator to provide link creation support and check the validity of links in the documentation. The Java Elucidator represents documentation in XML format using a collection of elements for specifying document structure, such as sections, sub-sections, document abstract, author, and last revision date. Special elements are used for specifying links from documentation to documentation and from documentation to source code. The Java Elucidator also supports having source code fragments inlined in the documentation [21]. These fragments are specified using a special language and are extracted when the documentation is presented (two source code extracts can be seen in figure 1, window 3). 3. INTEGRATED TOOL SUPPORT FOR IN- TERNAL DOCUMENTATION

An overall goal of integrating tool support for internal documentation in an IDE is bringing documentation in close proximity of the program. Internal documentation should not be considered as an external or secondary artifact, but rather as equally important as the source code [6]. Programmers rarely update documentation [7], but if the documentation is in close proximity of the program, while the programmer is working on the program, it becomes harder to ignore and the programmer is likely to be more compelled to maintain the documentation. A recent study among software professionals shows that a variety of tools are used when writing documentation, but a majority of the developers found word processors and JavaDoc-like tools to be the most useful [7]. JavaDoc-comments are hard to ignore as they are embedded in source code but documents written in a word processor external of the programming environment are more likely to be ignored. Seen from a tool perspective it is desirable to experiment with tools that can alleviate this problem and bring internal documentation into programming environments in such a way that the documentation is easy to manage and create as well as hard to ignore. The DOgMA tool combines hypertext and literate programming in order to support documentation and maintenance of C++ programs [15]. As in Elucidative Programming, documentation and source code is stored separately and linked together. From the documentation programmers can either address identifiers or any number of lines of code in succession. The first are rendered as hyperlinks and the latter are extracted from the source code and merged into the presentation of the documentation. Source code and documentation is presented in the DOgMA tool and highlighting schemes are used to improve readability of source code and documentation (e.g. identifies are rendered in boldface). The DOgMA tool provides two indices. The first provide information on all available nodes in the current project (source code files and classes, and documentation files and chapters). The second index shows nodes that have a relation to the currently displayed node (include files, subclasses, sub chapters). When studying source code the second index also shows documentation addressing any part of the currently displayed node. These indices provide the programmer with a useful context sensitive overview of both documentation and source code. The browsing facilities, except the context sensitive overview, provided by the DOgMA tool are similar to the facilities supplied by web-browser based solution used in the Java Elucidator (see section 2). Østerbye has experimented with a system for literate Smalltalk programming using hypertext where documentation and source code also can be written separately and interlinked freely [14]. As a contrast to the DOgMA tool, source code and documentation is not merged together when presented in Østerbyes tool. Instead they are displayed side-by-side, which is necessary because there can be several code fragments per explanation and vice versa. As in Elucidative Programming, Østerbyes approach introduces a tight coupling between documentation and program because the explanation of a single global effect can be linked to all pieces of code that are affected [14]. However, using the tool presented in [14] programmers can still loose the feeling for where the information at the screen belongs in the overall document. It is suggested to improve the system by having the destination of links in-lined in both documentation and source code. Modern IDEs provide source code navigation that is superior to the navigation supplied in both the tools described above and the current Java Elucidator. In order to provide support for internal documentation inside a modern IDE it is necessary to introduce a documentation perspective that can be plugged into the IDE. The documentation perspective must support creation and management of internal documentation as well as produce presentations of internal documentation inside the IDE. The main purpose of this documentation perspective is to bring documentation into close proximity of the program while it is being developed and maintained. By presenting source code and documentation side-byside, in two windows, we can ensure a close proximity between explanation and the actual source code that is being explained [14]. When reading documentation in one window, links to source code can be activated and the associated source code element is scrolled into view in the other window. This makes it possible to read source code and documentation sequentially as advocated for in [18], which addresses a hierarchy of cognitive issues to consider during design of software exploration tools. When studying a source file all links from documentation to any part of that specific file are potentially relevant. It is desirable to be able to mark-up source code elements in order to inform the programmer of which of these are addressed in the documentation. These marked source code elements can be used as anchor points for links to the documentation and provide the programmer with a context sensitive overview of relevant documentation. We use the term navigational proximity [13] for this kind of tight coupling between source code and documentation. Although links and indices are provided, navigation in an elucidative program can become confusing. When using the Java Elucidator the documentation nodes (XML documents) are placed in catalogues starting from a specific root catalogue. This helps the programmer to organize the documentation. For example, different catalogues can be created to hold documentation of design, implementation and test. Instead of visualizing the documentation nodes and their associations as a web it is often enough to visualize the catalogue structure. This structure is convenient when managing the documentation because documents can be categorized and grouped according to the aspects of the program that are being explained. Besides navigational facilities there are some more basic facilities connected with creating, managing, and browsing documentation that must be supported by the documentation perspective. These activities can roughly be divided into editing and reading activities. When editing it is crucial to be able to easily create links from documentation to source code (and from documentation to documentation). Making these by hand is simply not feasible. This is also a concern in [15] and [14] where both tools support link creation in a desirable point and click fashion. When reading an XML document the volumes footprint of the elements often becomes too disturbing. It is desirable to have editor support such as syntax highlighting, structure outlining, element completion or a WYSIWYG editor. The

latter is especially preferable when reading documentation. 4. INTEGRATING TOOL SUPPORT IN AN OPEN IDE An open API allows programmers to integrate tools in a given IDE. First of all, an open API must provide means for reacting on events within the IDE, such as when code projects are opened and closed, and editor events such as cursor movement. In addition, the open API must provide an interface for loading source code into code editors and bringing specific parts of the source code into focus. Finally, the API must provide means for integrating special editors and viewers. More and more IDEs has an open API. Not just open source Java environments such as NetBeans [4] and Eclipse [2], but also commercial environments such as Borlands family of IDEs [1] and Microsoft Visual Studio.NET [3]. Through comparison of a number of APIs [19, 2, 3, 1] we have found that although the APIs are quite different all the IDEs are open enough to allow integration of the Java Elucidator. However, appearance and ease of implementation will differ from IDE to IDE because of their individual limitations. The integrated tool support presented in this paper has been performed in TogetherJ [19]. TogetherJ is written in Java and supplies an adequate open API for our purposes. TogetherJ was chosen because it is becoming an increasingly popular commercial IDE and it is written in Java making it easy to reuse parts of the Java Elucidator. In addition, TogetherJ supports UML modeling which might be interesting to combine with internal documentation in future experiments. The screenshot in figure 1 shows the current elucidative documentation perspective for TogetherJ (windows 2 and 3). Documentation is written and read in window 3, which consists of four tabs: Editor tab is a simple text editor with syntax highlighting for XML. Links are created in two steps. First the destination of the link is marked in the source code editor (window 1). Following, the source of the link is marked in the documentation in the editor tab, and the tool then inserts a link element. Preview tab displays XML documents in a pretty-print version in a web-browser. Various documentation indices can be presented in the previewer. This includes a documentation index with keyword search options. Message tab mainly displays information from the background abstraction process that synchronizes the database with documentation and program files (see section 2). When an invalid link in the documentation is detected this is reported as a clickable error message that brings the invalid link into focus in the editor tab. Catalogue tab displays a file explorer which provides an overview of the physical catalogue organization of the documentation, see figure 2. From this tab documents can be opened in either the preview or editor tab, and new documents can be created by selecting from a range of template documents. Navigation is provided from links in both the preview and editor tab to their actual destination in the source code (window 1). In addition, if holding the control key while switching from one tab to the other, the document being displayed in the first tab is automatically loaded into the second tab. It has proved to be a bit more difficult to provide a good way to display relevant documentation for source code. The open API for TogetherJ does not allow us to manipulate the visual appearance of the code editor, such as highlighting specific syntactical elements. It would be desirable if the API included an interface for visually marking elements in the code editor, which also could prove useful when integrating other kinds of tools such as code visualization tools. The open API for TogetherJ is however not the only one that has this limitation. It is often not possible to manipulate the visual appearance, such as syntax highlighting, of source code editors in an IDE. As a remedy relevant documentation for the current cursor position in the program source is listed in a tree index in window 2. Elements directly under the root in the tree represent links to the source code element at the current cursor position. When the cursor is inside a method body any links to the method or any parts of its body are displayed under a branch in the tree. Similar, links to the surrounding class are displayed under a separate branch in the tree. In this way the programmer is informed of any link in the documentation addressing the specific named abstraction at the current cursor position as well as documentation addressing the context of the named abstraction. By right clicking in the tree index the programmer can choose to open the respective document in either the preview or editor tab. In addition, the programmer can choose to have the actual link in question scrolled into view. If the programmer double clicks on an element in the tree the corresponding source code element is scrolled into focus in window 1. 5. CONCLUSIONS We have presented a prototype tool that demonstrates how Elucidative Programming can be integrated in a modern IDE. The tool introduces a documentation perspective in TogetherJ that provides navigational proximity between Java source code and its internal documentation. The tool suggests how hypertext based documentation can be integrated in an IDE. The focus so far has been to produce a prototype that demonstrates that it is indeed possible to integrate Elucidative Programming in a modern IDE. The integrated tool support has been used to document the tool itself and study existing documentation of the Java Elucidator. However, real users must be exposed to the tool in order to fully evaluate the tool and direct further development. Based on previous experiments [20, 5] we expect that the tool to be an attractive alternative to existing documentation tools that allow the programmer to write documentation ranging from persistent documents (e.g. requirements) to more transient documents (e.g. notes and code commentaries). We have found that integration of programming tools in an IDE like TogetherJ is quite easy when one first become familiar with the open API. In the future we expect to see more tools integrated into IDEs, especially the Eclipse IDE is becoming increasingly popular. Eclipse is open source and it is therefore possible to extend the source code editor. This makes it attractive to experiment with a documentation perspective for Eclipse because it becomes possible to

1 2 3 Figure 1: Screenshot of the integrated support for Elucidative Programming in TogetherJ. Figure 2: Screenshot of the integrated support for Elucidative Programming in TogetherJ showing the catalogue tab.

markup source code that is documented. 6. REFERENCES [1] Borland software corporation. Available via http://www.borland.com/. [2] Eclipse platform, open source platform. Available via http://www.eclipse.org/. [3] Microsoft Visual Studio.NET. Available via http://msdn.microsoft.com/vstudio/. [4] Netbeans, open source platform. Available via http://www.netbeans.org/. [5] M. R. Andersen and C. N. Christensen. Evaluating elucidative programming in an industrial setting. Unpublished paper available via http://dopu.cs.auc.dk, December 2000. [6] D. Deveaux, R. Fleurquin, and P. Frison. Software engineering teaching: a docware approach. In Proceedings of the 4th annual SIGCSE/SIGCUE ITiCSE conference on Innovation and technology in computer science education, pages 163 166. ACM Press, 1999. [7] A. Forward and T. C. Lethbridge. The relevance of software documentation, tools and technologies: a survey. In Proceedings of the 2002 ACM symposium on Document engineering, pages 26 33. ACM Press, 2002. [8] L. Friendly. The design of distributed hyperlinked programming documentation. In S. Frass, F. Garzotto, T. Isakowitz, J. Nanard, and M. Nanard, editors, Proceedings of the International Workshop on Hypermedia Design (IWHD 95), Montpellier, France, 1995. [9] D. E. Knuth. The WEB system of structured documentation. Technical Report STAN-CS-83-980, Department of Computer Science, Stanford University, September 1983. [10] D. E. Knuth. Literate programming. The Computer Journal, 27(2):97 111, May 1984. [11] K. Nørmark. Elucidative programming. Nordic Journal of Computing, 7(2):87 105, 2000. [12] K. Nørmark. Requirements for an elucidative programming environment. In Eight International Workshop on Program Comprehension. IEEE, June 2000. [13] K. Nørmark, M. Andersen, C. Christensen, V. Kumar, S. Staun-Pedersen, and K. Sørensen. Elucidative programming in java. In Proceedings on the eighteenth annual international conference on Computer documentation (SIGDOC), pages 483 495. IEEE Educational Activities Department, September 2000. [14] K. Østerbye. Literate Smalltalk programming using hypertext. IEEE Transactions on Software Engineering, 21(2):138 145, February 1995. [15] J. Sametinger and G. Pomberger. A hypertext system for literate c++ programming. Journal of Object-Oriented Programming, 4(8):24 29, January 1992. [16] R. Stallman. Emacs: The extensible, customizable, self-documenting display editor. In D. Barstow, H. Shrobe, and E. Sandewall, editors, Interactive Programming Environments, pages 300 325. McGraw-Hill, 1984. [17] R. Stallman. GNU Emacs manual. The Free Software Foundation Inc, June 1985. [18] M.-A. Storey, F. Fracchia, and H. Muller. Cognitive design elements to support the construction of a mental model during software visualization. In Proceedings of the Fifth International Workshop on Program Comprehension, pages 17 28, March 1997. [19] TogetherSoft. TogetherJ. Available via http://www.togethersoft.com/. [20] T. Vestdam. Introducing elucidative programming in student projects. Unpublished paper available via http://dopu.cs.auc.dk, January 2001. [21] T. Vestdam. Elucidative program tutorials. Nordic Journal of Computing, 9(3):209 230, 2002. [22] T. Vestdam and K. Nørmark. Aspects of internal program documentation - an elucidative perspective. In International Workshop on Program Comprehension. IEEE, July 2002.