From Patterns to Frameworks to Programs

Transcription

1 From Patterns to Frameworks to Programs S. MacDonald, S. Bromling, D. Szafron, J. Schaeffer, J. Anvik and K. Tan Department of Computing Science, University of Alberta, Edmonton, Alberta CANADA T6G 2H1 Abstract A shortcoming of design patterns is that they are only applicable at the design phase, offering no support during the implementation phase of program development. Part of the problem is that a pattern must be adapted to the problem, a characteristic that cannot be captured by a single reusable implementation. To solve this problem, this paper develops a generative design pattern form called a design pattern template. A pattern template is a pattern form that can be customized using a set of template parameters, to adapt it to the problem. The customized template is used to generate object oriented framework code, providing implementation support. These pattern templates lead to a process for creating pattern based programming systems that can be the target of tool support. As an example of such a tool, we present CO 2 P 3 S, one implementation of the process. We further argue the need for meta templates to create extensible tools that allow users to add new pattern templates. To illustrate tool support for meta templates, we also present MetaCO 2 P 3 S. All of the pattern templates in the latest versions of CO 2 P 3 S have been created using MetaCO 2 P 3 S. 1 Introduction Design patterns capture the knowledge of experienced object oriented software designers in a form that can be distributed to others. They recognize that designers do not solve all problems from first principles, but rather that experience plays a key role. Patterns have proven popular as design aids among the object oriented community, as is evidenced by the large (and growing) amount of patterns material that is available. However, while patterns are useful as design constructs, they provide little support once the implementation process begins. Each pattern must be implemented each time it is used. The primary reason for this is that a pattern is not a single solution to a given design problem, but rather is a family of solutions. Each member of this family has the same basic structure but must be adapted to its specific context. Adapting a pattern to a problem may require that a variation of the basic structure be selected or that application specific behaviour be added to the classes making up the pattern. In addition, some patterns are complicated to implement correctly or may require more code than a simpler solution. This paper demonstrates a method for expanding the role of design patterns beyond the design phase and into the implementation phase of program development. To do so, we must create a more concrete design pattern form that includes source code. 1

2 The first concrete pattern form we consider is a framework, which provides a partial implementation of the structure of its associated design pattern. An application is created by specializing some of the framework classes with application specific implementations of hook methods that are called by the supplied structure. While frameworks seem like a good solution, we show that they are limited in their ability to adapt to a problem, which is a crucial element of successfully applying design patterns. To resolve this problem, we consider a generative pattern form called a design pattern template. A pattern template can generate a family of parametrically related frameworks, preserving the idea of a pattern as a family of design solutions. The user guides the framework generation process by supplying values for pattern template parameters, creating a framework tailored for the application. Thus, a pattern template is more flexible and adaptable than a static framework. In addition, the generated code can include extra classes that make the framework easy to instantiate and use. Using design pattern templates, we have constructed a process for writing pattern based programs that can be supported by programming systems (tools). Much like the patterns they are based on, pattern templates and their generated frameworks have their own forces that must be balanced. They must balance usability against flexibility when considering the parameters that the template supports. These forces must be resolved before our pattern based programming model will be seriously considered by the software development community. We provide one possible resolution. To demonstrate the validity of the design pattern templates and the emergent programming model, we also present CO 2 P 3 S, a pattern based programming system for writing parallel object oriented programs. Although the tool is targeted at the parallel program domain, the lessons derived from it apply equally well to general pattern based systems. CO 2 P 3 S uses our design pattern templates to generate customized framework code implementing the complete parallel structure for a set of supported patterns. However, CO 2 P 3 S is not limited to just the parallel programming domain. Templates for sequential design patterns can also be included. Any programming system based on pattern templates will eventually encounter some serious limitations. Inevitably, there will problems that cannot be implemented by the set of pattern templates supported by a tool. Further, new patterns continue to emerge. Pattern based programming systems must provide facilities that allow end users to customize their tools. Users should be able to create new pattern templates for new patterns they wish to use in their programs. Users should also be able to modify existing pattern templates to make them more specific to their needs. To accommodate these needs, CO 2 P 3 S supports meta templates. Meta templates are a description of design pattern templates that can be created and modified by pattern designers. The MetaCO 2 P 3 S tool guides pattern designers through the process of pattern template creation and manipulation. To validate the effectiveness of MetaCO 2 P 3 S, we have used it to regenerate all of the design pattern templates in CO 2 P 3 S, and added several more patterns as well. Our meta templates also provide a distribution format for exchanging pattern templates among a user community. With some forethought, it is possible to create a system independent meta template format. Any compatible tool can use meta templates to create an appropriate pattern template that can be used in programs. A repository of these meta templates could provide a large supply of templates that all tools can draw upon. The research contributions of this paper are: An exploration of the relationship between design patterns and frameworks. Ultimately, 2

3 we express this relationship as a design pattern template, which captures the flexibility of a pattern in an implementation construct based on frameworks. A process for creating pattern based programs based on user guided framework generation. The user describes the framework structure by supplying pattern template parameter values. This process can serve as the basis for tool support for future programming systems. An exploration of the forces present in a programming system based on design pattern templates, and one possible resolution of these forces. A meta system for pattern based programming systems that describes pattern templates. The meta system allows a tool to grow and adapt to the needs of its users. A validation of the meta system. This validation shows that it is possible to construct pattern templates that can, in turn, be used to write real programs. Section 2 develops and describes pattern templates and a model for pattern based programming based on them. Section 3 describes the different parts of a meta template description of pattern templates. Section 4 considers the possibility of a repository of system independent meta templates. In Section 5, we describe CO 2 P 3 S and MetaCO 2 P 3 S, our tools supporting the pattern based programming model and the meta system. We discuss related work in this field in Section 6, and conclude with Section 7. 2 A Development Process for Pattern based Programming This section develops a process for creating pattern based programs based on a generative design pattern form. We first explore the problems with design patterns once the implementation phase of application development begins. To address these problems of design patterns, we try to use frameworks to create reusable pattern implementations. We will show why static frameworks are insufficient for this purpose. Instead, we provide reusable pattern implementations using design pattern templates, a generative pattern form that creates custom frameworks from a design pattern description. The user specializes the template to describe the desired pattern structure using template parameters, adapting the pattern to its context. This description guides the generation of the framework code. 2.1 Designing with Patterns The most common form of a design pattern is a document, such as a chapter in a pattern catalogue or a Web page. This form preserves the instructional nature of patterns, as a cache of known solutions to recurring design problems. Patterns in this form are easy to distribute and readily available to designers. They are also independent of software development tools since they are only applied during application design. Patterns also provide a common design lexicon, and communicate not only the structure of the design but also the reasoning behind it [2]. Patterns are not free from drawbacks, however. While useful as a design construct, they suffer from several problems after the design phase is complete. These problems make it difficult to 3

4 Client Component Operation() AddComponent() RemoveComponent() GetChild() Client LineComponent Draw() Move() Leaf Operation() Composite Operation() AddComponent() RemoveComponent() GetChild() children foreach g in children g.operation() ; Line Draw() Move() LineComposite Draw() Move() AddComponent() RemoveComponent() GetChild() children foreach g in children g.draw() ; foreach g in children g.move() ; (a) The structure of the Composite pattern [10]. (b) The class structure of a line drawing example. Figure 1: An application of the Composite design pattern to a line drawing program. implement patterns. Without a proper implementation, the forces addressed by the pattern may not resolved properly and the benefits of the pattern can be lost. The first problem is that patterns are written documents that are subject to human interpretation, and so are vulnerable to all of the ambiguities that are present in natural language. An incorrect interpretation of a pattern can lead to an incorrect implementation. The second problem is that some patterns are inherently complex, making them difficult to implement correctly. This can occur because of the type of problem the pattern is addressing. For example, patterns for distributed and concurrent programs are hard to implement correctly simply because the pattern domain is a difficult one. Even if the pattern domain is not difficult, pattern solutions are generally more complex than their non pattern counterparts. Design patterns try to introduce flexibility into a software system so that certain aspects can evolve over time, referred to as designing for change [10]. This flexibility is usually achieved by adding indirection between some classes and objects, which requires more code that must be written and maintained. The third problem is that the indirection introduced by a pattern can result in performance bottlenecks. The indiscriminant use of patterns can result in a slower application [19]. Lastly, the implementation of a pattern quickly takes on application specific characteristics. Although a pattern is applicable to a variety of problem domains (provided that the context of the design problem is the same), its implementation is necessarily specific to the application. To encompass this need for variability, a pattern is actually a family of solutions to a problem. Each member of the family shares a common base structure, but with some parameters that preserve the intent of the pattern while allowing it to be customized for its intended use. For example, consider the Composite design pattern, illustrated in Figure 1(a). The Composite pattern maintains part whole object hierarchies where individual parts and compositions of parts are treated uniformly. Individual part objects apply operations to themselves, and compositions apply operations to each of the parts they contain. Since both types of object are uniform, a composition can contain other compositions as children. Figure 1(b) shows the Composite applied to a line drawing example, where a user can manipulate an individual line (the Line objects) or a group of lines (the LineComposite objects). The Draw() method displays either a single line or a group of lines depending on the receiver object type, and Move() moves the receiver. This simple example helps demonstrate some of the possible drawbacks to patterns. If a simple 4

5 array of line objects is sufficient for the application, applying the Composite pattern is a poor choice. The pattern introduces extra classes (and all of the code therein) to provide unnecessary generality. We can also see where the pattern begins to immediately take on application specific characteristics. The Draw() and Move() methods are specific to this problem. The Composite pattern includes a set of application specific methods that should be applied to the parts in the hierarchy. Our particular use of the Composite is a variation where the child management methods are only available in the Composite class, which enforces safety at the cost of uniformity. The data structure for the children must also be determined, based on the needs of the application. If we assume that a user can group an unbounded number of lines, a linked list is the best choice. In spite of these problems, design patterns have gained popularity among software designers. Since the publication of the first pattern catalogue [10], the number of patterns related books and conferences has exploded. This has created a large body of work that designers can draw upon. 2.2 Patterns as Frameworks Rather than relegating design patterns to the design phase of programming, we would like to elevate them to programming constructs that provide reusable code. When considering reusable code in the object oriented domain, many developers think of frameworks. A framework is a set of classes that defines a reusable structure for a program component. This generic structure is defined in terms of primitive or hook methods. The programmer writes application specific implementations of the hook methods to write a program, using the rest of the framework design and structure. This is almost the complete opposite of a library, where the library supplies a set of application independent primitives and the programmer writes the program structure using them. The primary benefit of a framework is that it provides a design and partial implementation that can be reused over many programs. Rather than write a complete application, the programmer need only customize some parts of an existing structure. This requires less code and can reduce application development time [11]. Since a framework contains the structure of the program component, this structure must be stable before a framework is a good option. Although patterns are families of design solutions, the family members generally share a commonality that can be exploited in a framework. Differences between members can usually be captured using other patterns, such as the Strategy [10]. Unfortunately, a single static framework may not be capable of capturing all variations of a pattern. Some pattern variations redistribute their responsibilities, such as the two versions of the Composite pattern in Figure 1. In rare cases, a pattern variation may take a fundamentally different approach to solving the design problem. The Adapter pattern [10] has an object and class version, both of which cannot be captured in the same framework. Perhaps the largest problem in creating generic frameworks for design patterns is the need for application specific interfaces in the pattern implementation. The interface to the classes in a pattern is dictated partially by the pattern and partially by the context in which the pattern is applied. The child management methods in the Composite are required by the pattern, but the operation methods (Draw() and Move() in the example) are required by the application. Without these operations, the Composite serves no useful purpose. It is difficult to create a single framework that captures the structure of the pattern while allowing the programmer to add to the interfaces. In addition to the interface problem, static frameworks are limited to providing reusable imple- 5

6 mentations of the application independent parts of a pattern. This code may only be a small part of the overall implementation, limiting the code reuse benefits of the framework. For example, a framework for the Composite pattern can provide the child management methods but no more. The programmer must still write both the Line and LineComposite classes. Methods in the latter class are straightforward, enumerating over the children and invoking a method with the same name, as dictated by the Composite pattern. However, these methods are application specific. In summary, object oriented frameworks are a good mechanism for design and implementation reuse when the interfaces between components can be fixed. This is why frameworks are typically restricted to a particular application domain. Design patterns represent a family of solutions that must be adapted to the application, normally by adopting an application specific interface. This need for variability makes it difficult to create a single, generic framework for a pattern. 2.3 Programming with Generative Patterns Since a single generic framework is insufficient, we turn to a generative approach. We start with a generative pattern description, or design pattern template. A design pattern template is a programming construct that generates framework code for its associated design pattern. To preserve the idea that a pattern is a family of design solutions, each pattern template also has parameters that can be set by the programmer. The parameters may also include application specific interface information so that the pattern can be further adapted for its use. The programmer uses the parameters to guide the framework generation process to produce a framework implementing the pattern variation that is best suited to their application. Using our design pattern templates, a model for writing pattern based programs has emerged: 1. Identify the design patterns that are required for the application and select the corresponding design pattern templates. 2. Customize each pattern template by providing parameter values. Use the fully specified pattern template to generate framework code implementing the selected pattern structure. 3. Insert application specific code into the hook methods provided by the generated framework, and implement any additional code required for the application. To illustrate the process more concretely, consider the line drawing example program. The first step is to identify the required pattern and select the pattern template, in this case the Composite. The structure of the basic Composite template is that in Figure 1(a), which must be adapted to the structure in Figure 1(b) in the second step. This suggests the following, minimal set of pattern template parameters, with user supplied values for the line drawing example: Leaf class name The class names in the generated framework code should be meaningful. If we assume there is only one Leaf class, then the user can supply its name and the other class names can be generated automatically. For our example, the supplied class name is Line. The other classes are named by appending Composite and Component to this name. If we allow multiple Leaf classes, the user will have to explicitly name the other classes. 6

7 Child management methods location This parameter indicates in which classes the child management methods should be available. The choice is between uniformity and safety. In the line drawing example, we opt for safety. Application interface The user should supply a list of application specific methods for the Composite framework. These methods are present in all classes, and are hook methods in the Leaf class. The Draw() and Move() methods are specified using this parameter. Child data structure This parameter is a list of collections for storing the child objects, such as an array, hash table, or linked list. For the line drawing example, a linked list is selected. This list is one possible set of parameters for a Composite template, covering common alternatives. Additional parameters could be introduced to address other variations. Once the user has provided pattern template parameter values, the last part of the second step in our process is to use the template to generate framework code for the selected pattern structure. The Composite pattern template with the above parameter values yields the framework structure in Figure 1(b). In the third step, the user completes the framework by supplying hook method implementations. Once this is done, it can be instantiated and used to launch a computation. In a normal framework, creating and using the framework requires some knowledge about its design. To implement hook methods, the programmer would need to know which classes to subclass and which methods to override. To instantiate the framework, the correct objects must be created and composed, which may include classes other than those with hook methods. In contrast, the frameworks for a design pattern template are guided by the parameter values and are specific to the selected pattern structure, rather than being a generic solution. As well, we can generate additional support code for the frameworks. The generated code can make it easier to write code, simplify instantiation, provide additional support classes and methods, and encapsulate details. Each of these is described. Writing Code To write a program, the programmer must subclass framework classes and override hook methods with application specific code. Rather than having the user create these subclasses, we can automatically generate them based on class names supplied as template parameters. These classes can include default stubs for the hook methods, which the programmer fills in. Generating these default methods improves the usability of the framework and reduces the chances of programmer error. Also, since the user does not need to provide method signatures, there is less chance that they can introduce compilation or logic errors (by transposing two method arguments of the same type, for example) into the signature. Instantiating the Framework Since the framework generated for a pattern template is specific to the selected structure, some of the indirection that would normally be needed to accommodate framework variations is not necessary. Thus, our frameworks have a simpler object structure. This can result in improved performance, and can also make instantiating the framework easier. Support Classes and Methods We are not limited to generating just the framework code for a pattern template. We can also generate additional classes and methods based on our knowledge of the pattern. In essence, pattern templates allow more of the pattern structure to be considered 7

8 Mesh Class Mesh Element Abstract Mesh Class Mesh Class Thread Abstract Mesh Block Mesh Block Mesh Block Strategy (a) The user view of the Mesh framework. Bounded Mesh Element Array Mesh Element (b) The full object structure of the Mesh framework. Figure 2: An example of encapsulation in generated frameworks. as application independent even when it is specific to the problem. For example, each operation method in the Leaf class in a Composite has a counterpart method in the Composite class. The latter methods are straightforward, but are still application specific. In a normal framework, the user would have to write and maintain this code. When we generate the framework for the Composite pattern template, we can automatically generate these methods. In the line drawing example, for instance, we can automatically generate the complete LineComposite class. Encapsulation Even though the generated frameworks have a simplified object structure, we can further simplify them through encapsulation. Where appropriate, we use the Facade pattern [10] to hide structural detail. This Facade provides the user with a high level view of the framework and a constructor to create and compose the framework objects. The user is given access to only those classes that are germane to writing an application and launching a computation. Our Composite example does not use a Facade as the user builds an object structure with the framework. Instead, consider a framework for parallel mesh computations [16]. Figure 2(a) shows the user s view. This view consists of two classes: the Facade (Mesh Class) and the element that defines define the computation (Mesh Element). These two classes are all the user needs to create a parallel mesh application. The complete structure, managed by the Facade, is in Figure 2(b). Encapsulating the structural details of the frameworks allows programmers to concentrate on their application code. It also helps enforce correctness by preventing users from introducing errors into the structure. Assuming the structure is free from errors, the user can write a program knowing that the pattern is properly implemented. Encapsulation also prevents a program from drifting from its design, preventing later maintenance problems. Remarks Design pattern templates provide a generative pattern form that create customized frameworks from a pattern description based on user supplied parameter values. They are more flexible than static frameworks since they allow users to adapt the underlying pattern to the specific problem they are trying to solve. Pattern templates can also incorporate application specific interfaces into the generated framework. Further, with pattern templates we are not restricted to generating just basic framework code. We can create additional support code to make the frameworks easier to instantiate and use. 8

9 2.4 Unresolved Forces in Design Pattern Templates Like most programming constructs, design pattern templates must balance two forces: usability and flexibility. To make the pattern templates easy to specify and use, they should have few parameters. With fewer parameters, more of the structure must be fixed so some pattern alternatives will not be available, sacrificing flexibility. Finding a resolution to these forces for a given template must be done on a case by case basis. The design pattern template must be able to generate framework code for the most common alternatives in order to be useful to the majority of developers. However, this is only a partial resolution of these forces. It fails to address less common variations of a pattern structure. Some of these variations may represent optimizations that can be vital to meeting necessary performance targets. Adding to this problem is our encapsulation of the structural framework code during initial development. The solution is openness. Openness in this context suggests that the programmer should have access to the structural code generated for the pattern [22]. This openness is contrary to the encapsulation we advocated previously. Once again, we have two forces that must be resolved. To resolve openness and encapsulation, we turn to multiple programming layers in our development process. Multiple programming layers have been used successfully in other areas, such as computer networks, to reduce complexity through the separation of concerns. The layers in this pattern based programming process correspond to different stages of program development. The highest layer provides the pattern based development process derived in the previous section, encapsulating the structural code of the generated frameworks. We now add a fourth step to this process for our layered model: 4. If necessary, use the facilities at the lower layer to implement a pattern variation that cannot be specified using the design pattern template parameters. The structural code of the framework is available to the programmer so that it may be modified as needed. It is also possible to provide useful high level abstractions in the second layer. These abstractions are implemented by a third, lower level layer. For example, for pattern templates targeted at parallel programming, the structural framework code at the second, intermediate layer may be written in terms of high level communication and synchronization constructs. When changing the framework structure, it is usually not necessary to consider how these constructs are implemented. It is only if the communication and synchronization constructs become bottlenecks that their implementation need be examined. Intermediate layers can be added to the model as needed, based on the target application domain of the pattern templates. These layers gradually expose abstractions, rather than overwhelming the programmer with all of the details immediately. The lowest programming layer is the base programming language. 3 Meta templates Any concrete implementation of our design pattern based programming model will be limited by the set of patterns it supports. If a needed pattern is not available, then the programmer must implement it by hand. Since no tool can support a complete set of design patterns, this problem 9

10 has limited the acceptance of pattern based programming tools. Users will not invest the effort needed to learn a tool if they do not believe it will solve both their current and future problems. This problem is exacerbated by the continued work in mining new patterns. As these new patterns emerge, users will want to use them in their programs. However, most pattern based systems have no provision for extensibility by the user. Users must wait for the tool developers to incorporate these patterns into their systems. There must be a way to create new templates to extend existing systems with these new patterns. In addition, pattern templates must support framework evolution. As the frameworks are used in ways that are not anticipated by the initial pattern designer, weaknesses will become apparent and will need to be fixed. Openness at lower layers allows the programmer to perform this evolution, but it must be repeated for every occurrence of a framework, which is undesirable. Instead, it should be possible to modify an existing pattern template to generate different framework code. To address these problems, we introduce meta templates. A meta template provides a complete description of a design pattern template. It is used to generate the pattern template in much the same way that a pattern template is used to generate framework code. The generated pattern template is then incorporated into a programming system where it can be used in programs. The meta template itself is not used by the programmer when creating an application. With the addition of meta templates, any advanced user can also be a pattern designer. A new pattern template can be created by writing a meta template that describes it. Similarly, an existing template can be modified by changing its meta template. A pattern designer can evolve the framework code or add new template parameters that were not envisioned by the original pattern designer. Once the meta template is finished, it is used to create a pattern template that can be included in new applications. Programming systems that support meta templates will provide a fluid, extensible system that can be customized by its users. A meta template description includes the following: A description of the pattern template parameters. A description of the framework that is generated for the pattern template. This description must also detail how the pattern template parameter values affect the generated code. A description of a graphical representation of the pattern template. We expect some implementations of our pattern based programming process will have a graphical component. The remainder of this section discusses the first two parts of the meta template in more detail. 3.1 Pattern Template Parameters Pattern template parameters allow a programmer to customize the template and alter the framework code it generates. These parameters preserve the idea that a pattern (and its template) is not a single, static solution to a design problem, but is rather a family of solutions that can be applied in a particular context. The user must be able to customize the template to adapt it to the application. A meta template must describe the parameters that can be applied to the pattern template it represents. To simplify the meta template description, we list the different types of parameters: 10

11 Class Names This type of parameter is common to all pattern templates. Pattern template users should be able to specify class names in the generated frameworks for two reasons. First, the names of the classes should be meaningful in the context of the application. Second, it should be possible to use multiple instances of a pattern template in the same application, so we need a way to ensure that the frameworks will not have name clashes. As a result, each pattern template will have at least one class name that the programmer can specify, normally the name of the class (or classes) that export hook methods. If the framework has a Facade to hide the framework details, the programmer may supply its name as well. The structural class names are derived from one of the user supplied names, usually by prepending or appending extra text, to make them unique. For example, in our Composite template, the user selects the Leaf class name, and the remaining class names are derived from this name. Basic Parameters Other than class names, basic parameters are the most common pattern template parameter type. These parameters are an enumerated list of choices provided by the pattern designer. They can be used as Boolean switches or more elaborate choices. The example Composite template has two basic parameters, one determining which classes support child management methods and another selecting the data structure for storing child objects. Extended Parameters These parameters deal with the less common case where the parameter values have an arbitrary form. The pattern designer must provide extra information for each of these parameters, such as a way for the pattern template user to supply the parameter value and how different values affect the generated framework. The most common example of an extended parameter is a method signature. The method may be part of the application specific interface needed in a framework, which must be added to one or more framework classes. A default implementation of the method can be added as well. In the Composite template, the user can add a set of application specific methods. These methods should be added to all three classes in the generated framework. In the Composite class, these methods should iterate over the children and call a method with the same name, passing any arguments. In the Leaf class, these are hook methods and default stubs should be generated. Extended List Parameters These parameters handle the common situation where the pattern user supplies a list of values, which may be basic or extended parameters. A common example is a list of application specific methods that should be added to the interface of the framework classes, such as the application interface parameter in the Composite pattern template. These three parameter types are sufficient to cover a broad range of pattern template parameters. There may be useful specializations of extended parameter type, such as a parameter type especially for method signatures. In a Java environment, it may be useful to allow an interface to be specified as a list parameter for application specific methods. 3.2 Framework Generation After a pattern template user supplies values for the parameters, the template is used to generate object oriented framework code that implements the selected structure. The meta template must provide a way of describing the family of frameworks that can be created for a pattern template. 11

12 Our meta templates represent the family of frameworks using a source code template. A source code template is a set of annotated source code files, where the annotations indicate how the pattern template parameters alter the generated code. A code generator transforms the source code template based on the pattern template parameter values supplied by the user. The result of this transformation is a framework implementing the specified pattern. As with parameter types, we can enumerate the potential set of transformations that are required for a code generator. The necessary transformations are: Placeholder class names must be replaced by their final, unique names. For classes named by the user, this transformation simply replaces the complete name. For structural classes, which are hidden from the user at the highest layer, the final names are derived from one of the supplied class names, typically by combining one of the supplied names with a brief description of the structural class. In our Composite pattern template, the user only had to specify the Leaf class name, and the other class names were derived. Methods or variables may be selectively generated based on basic parameter values. This can be based on a single parameter setting or on a combination of several settings. In a pattern template for the Composite pattern, a basic parameter can indicate whether the part management interface should be implemented in all of the classes or only in the Composite class. The methods and necessary instance variables are included in the appropriate classes. New methods or portions of methods may be generated based on extended or list parameter values. For instance, the template user can specify a list of operation method signatures for the leaves for a Composite framework. Methods for each signature should be generated for all of the classes. In the Composite class, the implementation of the method should enumerate over the children and invoke the same method, with arguments, on each one. A portion of a method body may be selectively generated based on the values of basic parameters. For example, the user can select the data structure for the children in our Composite example. Different data structures may require different enumeration code to be generated for the Composite class. In each generated framework, there will be at least one class that contains hook methods that need to be implemented by the template user. Any tool based on our programming model must present this class to the user so they can enter method bodies. Depending on the tool, these classes may require additional processing. For example, a tool may provide a special editor for this class that prevents the user from editing certain key parts of the source file, such as the hook method signatures or the superclass. The Leaf class in a Composite framework, which contains the hook methods, may need additional processing to insert annotations for the editor. Not surprisingly, the most difficult parameters to handle in the code generator are extended parameters. Some of the code generation may require that the parameter be parsed for information, which may be specific to the pattern template. Thus, the pattern designer may need to include code generator extensions to properly parse extended parameters and generate the framework code. 12

13 Meta template Pattern Template Graphical Representation Parameter Descriptions Source Code Template Pattern Template Generator Graphical Display User specified Parameter Values Code Generator Framework Code Structural Code Hook Methods Application specific Code Figure 3: From meta templates to pattern templates to frameworks to programs. XSL stylesheet Meta template Parameter Description (XML document) Graphical Representation (GIF,JPG) XSLT processor Pattern Template Tool Configuration Graphical Components Source Code Template Figure 4: An example pattern template generator, based on XML documents and XSL stylesheets. 3.3 From Meta templates to Pattern Templates to Frameworks to Programs The relationship between all of the parts of our pattern based programming model is shown in Figure 3. The meta template, created by a pattern designer, contains all of the elements that are needed to describe the pattern template. A pattern template generator processes the meta template and outputs a pattern template, in an analogous manner to the way frameworks are constructed based on a source code template and template parameters. A tool or specification language is then responsible for obtaining values for the template parameters from the pattern template user. These values and the source code template from the meta template are used to produce a customized framework implementing the specified pattern structure. This is analogous to parameterized programming in C++, where different code is generated based on template parameters at compile time. An application is created by providing application specific implementations for the hook methods exported by the framework. Figure 4 shows one possible pattern template generator implementation, based on XML (Extensible Markup Language) and XSL (Extensible Stylesheet Language). The pattern template parameters and some of the graphical components are described using an XML document. To ensure that the XML file is properly formatted, a DTD (Document Type Definition) file is provided. The DTD file defines the XML document structure and the legal elements in the file. The XML pattern template description is converted into a pattern template using an XSL stylesheet. An XSLT processor takes this stylesheet and the XML document and outputs another document. This can be used to convert an XML description into a different type of document, such as an HTML file, a text file, or even another XML file. In Figure 4, the pattern description XML document is transformed into a tool configuration based on a supplied XSL stylesheet. The tool configuration is imported by the tool to support the new pattern template. As well, the graphical components of the pattern template are assembled from the parameter description and graphical representation. The parameter description may include labels and other 13

14 non graphical display components. The graphical representation typically includes images for the tool to display. The tool can select an image to display based on the current values of the pattern template parameters. 4 A Repository for Generative Patterns With our meta templates, we now have an explicit representation of a design pattern template that can be created and manipulated by a pattern designer. This allows programming system developers to create extensible tools that can be customized by their users based on their needs. Users do not need to rely on tool developers for new pattern templates or updates to existing templates. However, there is no reason to consider the users of a tool in isolation. If a tool is successful, then it will have an entire user community to draw upon. This community can exchange new and modified design pattern templates. This is similar to the open source community, where programmers contribute their knowledge (in the form of source code) to large software projects, which are in turn used by an even larger community. To facilitate the exchange of pattern templates, we envision a Web based pattern template repository. The repository would be a warehouse of meta templates, which supply not only a description of a pattern template but also a distribution format. Pattern designers can upload new meta templates into the repository. Other users can download and install pattern templates in their programming system, and use them in new programs. The advantages of this repository are threefold. First, it can be used by the entire user community. However, the community is not simply distributing meta templates, but rather is still distributing design experience. Pattern designers create new patterns for the community. Novices use these patterns to learn good object oriented design until some of them grow into pattern designers themselves. Second, the library of pattern templates can grow quickly, which is vital as new patterns emerge. Lastly, the generated frameworks will be used in a broader range of application domains. This will help uncover any limitations and make the frameworks more robust. If the meta templates are stored in a application independent format, they can be used to create pattern templates for any tool with an appropriate pattern template generator. An example of such a meta template is the format in Section 3.3. Now, a single repository can serve the users of any compatible pattern based tool. Existing tools can take advantage of new patterns, and new tools start with a large library of proven pattern templates. A large library of patterns also removes one of the barriers that has prevented widespread acceptance of pattern based tools. Earlier efforts in tool support had a limited selection of patterns, limiting the range of applications that could be written. As a result, programmers avoided the tools. Extensible tools with a large library of available pattern templates is one of the keys to convincing programmers to use pattern based programming tools. 5 Pattern based Programming Tools CO 2 P 3 S and MetaCO 2 P 3 S This section describes our implementation of tool support for both the pattern based programming model and the meta system. CO 2 P 3 S (Correct Object Oriented Pattern based Parallel Program- 14

15 ming System, pronounced cops ) is a pattern based programming system that supports a set of design pattern templates directed at parallel programs [15, 17, 16, 14]. It was created to address some of the weaknesses identified with similar tools in the field [22]. However, it can be generalized to pattern based programming systems for other domains. The MetaCO 2 P 3 S tool addresses extensibility, allowing pattern designers to create and change pattern templates [4]. 5.1 CO 2 P 3 S: A Pattern based Programming System CO 2 P 3 S is a parallel programming system that uses a combination of object oriented techniques, design pattern templates, frameworks, and programming layers to simplify the task of writing parallel object oriented programs. Currently, the system is targeted at parallel Java programs running on shared memory multiprocessors. The CO 2 P 3 S system is composed of three layers of abstraction that programmers can use when creating programs. These layers are: Patterns Layer This layer implements the process from Section 2.3. The programmer expresses the parallel structure of their application by selecting the appropriate design pattern templates. Once the templates have been been selected, the programmer specializes them by providing values for the template parameters. Once specialized, the templates are given to a code generator that creates a customized framework for the selected pattern structure. This generated code includes at least one class that provides hook methods. The programmer supplies application specific implementations of these hook methods to create a program. Figure 5 shows an application under development in CO 2 P 3 S, involving two pattern templates. The displayed template is the Two Dimensional Mesh template, which has been customized by parameter selection to be horizontally toroidal (indicated by the horizontal arcs) and to execute in an ordered fashion (indicated by the label on the bottom). The framework for this template, discussed in more detail in [17], is tailored by the parameter values. A key aspect of the frameworks generated at the Patterns Layer is that they separate the parallel structural code from the application specific user code. The parallel structural code implements the creation and management of concurrency, communication, and synchronization. The application code does not have to implement any parallel code for the framework structure to execute properly. The programmer can concentrate on the problem they are trying to solve and not on the structure of the framework that will execute it. After it is generated, the structural code is hidden away from the user, possibly using a Facade (Section 2.3). This prevents users from introducing errors into the program structure. To prevent further errors, CO 2 P 3 S generates a class that contains stubs for all of the hook methods. This class is shown to the user in a modified HTML viewer that allows method bodies to be inserted but does not allow the method signatures to be changed. Intermediate Code Layer This layer is an example of an intermediate programming layer from Section 2.4. It provides a high level explicitly parallel programming language based on a superset of an existing language, such as Java or C++ (currently we use Java). It provides high level abstractions for expressing parallelism and synchronization. 15

16 Figure 5: A screenshot of CO 2 P 3 S showing an example Mesh application. At this layer, we address openness (Section 2.4) by making the structural framework code generated at the Patterns Layer available. This structure is implemented in the high level language. The programmer can optimize the code or implement a pattern variation that is not supported by the template parameters. Alternately, the programmer can skip the Patterns Layer and write programs directly in this language. Native Code Layer This layer provides a base object oriented programming language with all of the libraries used to implement the higher level abstractions (the lowest layer from Section 2.4). The implementation of the parallel constructs at the Intermediate Code Layer can be accessed and tuned based on the architecture of the execution environment. The CO 2 P 3 S system currently supports four design pattern templates: Two Dimensional Mesh This template supports iterative computations over two dimensional data. Each element computes its new value based on the values in a neighbourhood around itself. This template has been used to implement reaction diffusion texture generation [25]. Distributor This template provides a data parallel style construct. Methods are invoked on a parent object, which invokes an identically named method on a fixed set of children in parallel. This template has been used to parallelize a simple IDA* search for the 15 puzzle [13]. Phases This template is a useful compositional mechanism that provides an ordered sequence of methods. Together with the Distributor, it has been used in the implementation of the Parallel Sorting by Regular Sampling algorithm [21]. Wavefront This template supports computations that must maintain data dependencies between elements. The elements can be arranged in a matrix. An element may only be processed when all of its dependencies have been satisfied. This template has been used to implement a parallel sequence alignment algorithm and to solve two of the Cowichan [24] problems [1]. Although the current pattern templates are targeted at parallel programs, pattern templates solving problems in the sequential domain, such as our Composite template, could also be added. However, parallel design patterns provide a good testbed for a pattern based programming system. Much like the patterns in [10], these design strategies have been understood for many years 16

17 and have been employed in countless programs. Despite their familiarity, parallel design patterns are harder to implement than other patterns. They must address difficult problems that are not present in sequential programming, such as communication and synchronization. In CO 2 P 3 S, these problems are addressed in the generated structural code for the pattern template. Thus, we require pattern template parameters to modify the communication and synchronization structure to satisfy the requirements of different problems. Like their sequential counterparts, parallel design patterns must still be adapted to their context. 5.2 MetaCO 2 P 3 S: Tool Support for Creating Patterns CO 2 P 3 S now includes the MetaCO 2 P 3 S tool that allows pattern designers to create and modify meta templates. An important characteristic of the resulting pattern templates is that they are first class, meaning that they are indistinguishable in form and function to the pattern templates included in a CO 2 P 3 S distribution. MetaCO 2 P 3 S provides facilities that permit pattern designers to easily work with meta templates. An example of a meta template is shown in Figure 6, which is the MetaCO 2 P 3 S tool editing one of the parameters of the Two Dimensional Mesh. On the left side of this figure, we can see the categories in the meta template for the generated framework code ( Class Names ), pattern template parameters ( Parameters ), and the user interface ( Constants for labels, and GUI Configuration for other graphical elements). On the right side is the description of one of the parameters for the Mesh pattern template, which allows a programmer to select either a four point or eight point mesh. This parameter is a basic parameter (from Section 3.1) with two possible values. The value will determine the signature of some of the hook methods in the generated framework. A pattern template generator takes a complete meta template description and creates the associated pattern template. Our meta templates are stored as XML files and translated as described in Section 3.3. The output of the template generator is a set of Java source files that are compiled and linked into the user interface to present the pattern template to the programmer. To generate framework code, CO 2 P 3 S uses a doclet. Doclets are programs that specialize the JavaDoc program. JavaDoc was originally created to generate HTML documentation from annotation tags in specially formatted comments and method signatures in Java source files. Doclets can customize the processing of these annotations and produce other output. Our doclet uses new annotation tags to direct a source code to source code transformation based on the pattern template parameter values. The output is the specialized framework for the pattern template. To test the coverage and correctness of MetaCO 2 P 3 S, we regenerated the templates from earlier versions of CO 2 P 3 S (the Mesh, Distributor, and Phases). CO 2 P 3 S now uses these templates. In addition, new templates have recently been added or are under development. The Wavefront template was added as part of an implementation of the Cowichan problem set [24], a series of seven problems that tests the range of applications that a parallel programming system can solve. Other research is under way to create distributed memory versions of all pattern templates to allow them to run on a network of workstations. A successful distributed Mesh has already been created using MetaCO 2 P 3 S [1]. 17

18 Figure 6: A screenshot of MetaCO 2 P 3 S showing the pattern template parameter for selecting a four point or eight point mesh. 6 Related Work Budinsky et al. generate code for the patterns in [10] using a Web based tool [5]. Like our tool, the programmer can supply some parameter values so that the code can be customized for different pattern variations. However, the system only produces an application independent skeleton that the programmer must extend. Further, users do not have the ability to add new patterns. PSiGene takes a different approach, narrowing the application domain to obviate the need for pattern variations [20]. This allows the tool to formalize the interfaces of the patterns. A program consists of a class model for the objects in the program and a series of pattern bindings. The bindings generate code that links the objects according to the selected pattern using a set of code templates. Unlike our work, PSiGene generates only one structure for each pattern, although that structure can sometimes be optimized. As well, PSiGene provide a large number of small patterns, each targeted at a particular problem, where our patterns can provide more substantial functionality. Commercial tools have also started including support for design patterns. Together Control- Center [6] operates in a similar method to PSiGene, but also supports the patterns from [10]. The programmer selects a design pattern and assigns the roles of the pattern to different classes. Appropriate code (including the class if it does not already exist) is created. While the patterns are not parameterized initially, the code is available to the programmer immediately after it is generated. In contrast, our work encapsulates this structure initially. ModelMaker supports a subset of the patterns in [10], again making the pattern code available once it is generated [23]. However, ModelMaker focuses on the object structure of the pattern, ignoring its intent. For example, ModelMaker supports the Decorator pattern but not the Proxy since the two share a similar object 18