Multi-Dimensional Separation of Concerns and IBM Hyper/J

Multi-Dimensional Separation of Concerns and IBM Hyper/J Technical Research Report Barry R. Pekilis Bell Canada Software Reliability Laboratory Electrical and Computer Engineering University of Waterloo January 22, 2002

Introduction The process of multi-dimensional separation of concerns (MDSOC) is intended to allow developers to partition overlapping concerns in new and existing software products along multiple dimensions of composition and decomposition[1][6][7][8]. A clean separation of concerns is purported to: reduce complexity; improve comprehensibility and traceability within and across software artifacts; limit the impact of change; facilitate evolution, adaptation, customization, and reuse; and simplify component integration. In practice, achieving all of these goals has, to date, eluded software developers. For one, sets of primary concerns for a given software product line tend to be highly context-sensitive and often vary greatly over time. Another hurdle relates to the loss of traceability when traversing the link from requirements to design to implementation. Within the various software artifacts, the units of change often do not match the units of abstraction. For example, consider the link between requirements and design. Despite being representations of the same conceptual entity, each artifact decomposes the software system differently. Requirements decompose by feature, while design and code decompose by object. Issues and differences such as these often leads to the scattering and tangling of individual concerns across modules of the software implementation. Scattering occurs when a single requirement affects multiple design and code modules. Tangling occurs when material pertaining to multiple requirements is interleaved within a single module. These two factors greatly complicate and contribute to limiting the ability of software developers to reuse code. Model of Conventional Software Software systems are intended to address problems or provide services within a specified domain. They are represented by a set of software artifacts such as requirements specifications, design documents, and implementation code. Each artifact is created using descriptive materials that may be viewed as units. What comprises each unit depends on the formalism used and the context it is used in (e.g., class, method, function, UML diagram). Modern software implementations are normally organized into modules to help software developers achieve the desired separation of concerns. This modular approach allows software developers to identify each major concern and localize its units into a corresponding module. Many dimensions of concerns are January 22, 2002 Page 1 of 10

important throughout an implementation s lifecycle. These dimensions of concerns are used to organize the units that make up a software product. Tyranny of Dominant Decomposition Most of the present software formalisms support, to some extent, separation of concerns, but with a tendency toward a single, main dimension. For example, the object-oriented paradigm encapsulates data concerns into classes, while other concern types are not effectively represented. This is referred to as the tyranny of dominant decomposition[6][7] because one dominant way of decomposing the software (e.g., by class) imposes a brittle structure that makes it difficult to encapsulate other dimensions of concerns. A Detailed Example A detailed example using the MDSOC approach for a personnel system is presented in [7]. The example is summarized in the companion presentation included with this report. Multi-Dimensional Separation of Concerns Hyperspace Hyperspaces is a MDSOC approach that supports the explicit identification of important concerns and the encapsulation and integration of those concerns. Formally, a hyperspace is defined as a tuple (U, M,H) in which: U is a set of units gathered into the hyperspace; M is a concern matrix which simultaneously organizes the units in U according to all concerns of importance; and H is a set of hypermodules which specify how to build components from the units in U. Concern A concern may be modeled as a predicate, c, over a set of units, U. The predicate is used to indicate whether a particular unit addresses a specific concern. The unit set induced by concern c may be represented by: U(c) ={u U c(u) }. Concerns are considered to overlap if their unit sets are not disjoint. Using this representation, a dimension of concern can be defined as a set of concerns whose unit sets partition U. This partitioning property is important because it implies that a point in multi-dimensional spaces projects onto exactly one coordinate in each dimension. A dimension s concerns cannot overlap and must cover all the units in U. January 22, 2002 Page 2 of 10

Identification of concerns is a process of selecting concerns and populating them with associated units. It may be performed in a top-down (e.g., design) or bottom up (e.g., evolution) fashion, depending on the phase of the software lifecycle. Software artifacts include UML design documents and Java code. After being identified and grouped, encapsulated concerns can be manipulated as first-class entities. Encapsulated concerns are integrated to create the actual software artifact that addresses the multiple concerns of the system. For example, in standard Java, classes are integrated by loading. Their relationships are determined by a combination of import specifications and the class path. Concerns other than interfaces and classes cannot be integrated by standard Java. None Each dimension d, has one concern called the None concern, N d. All units that do not address any concern in d address N d. The None concern may be defined as the set of all units unaffected by changes that may occur within its associated dimension. The None concern for a given dimension may be empty. Concern Matrix A concern matrix may be defined over a set of units, U, as a tuple (C, D) where: C is a set of concerns; and D is a set of concern dimensions such that: - every concern in C is in exactly one dimension in D; and - every dimension in D partitions U. Therefore, each unit in U will be located at exactly one point (i.e., address only one concern) in each dimension. The unit s coordinates in the concern matrix identifies the associated concern. Hyperslice A hyperslice may be described as a set of units, written in any formalism, that encapsulates concerns in one dimension other than the dominant one. Each hyperslice only contains those units that address a particular concern of interest. Hyperslices can overlap, thus supporting simultaneous decompositions along multiple dimensions. This allows a software implementation to be represented as a collection of hyperslices along as many dimensions as required. Formally, a hyperslice resides in a hyperspace as a tuple (U, M, H). Hyperslice are always defined as a declaratively complete concern (hs C). A hyperslice, hs, is considered closed under references of all kinds such that if a unit u 1 hs refers in some way to another unit, u 2 U, then u 2 hs. January 22, 2002 Page 3 of 10

Hypermodules A hypermodule comprises of a set of composable hyperslices, together with a set of composition rules (i.e., integration relationships) that specify how the hyperslices are to be composed. Hypermodules can be nested, allowing an entire software artifact to be modeled as a hypermodule. The hypermodule s composition rules indicate which units in the various hyperslices describe the same concepts and how these units must be integrated. For example, a rule may assert that classes in different hyperslices with the same name model the same concept and should be merged into a new, compound class with the same name and combined details. A hypermodule may be formally defined as a tuple (HS, CR) where: HS is a set of hyperslices; and CR is a set of composition relationships. A composition relationship is itself defined as a tuple (I, r, f, o), where: I is a tuple of input units from the hyperslices in HS; r is a correspondence relationship defining how the units in I are interrelated; f is a composition function, f :(I r) + > U indicating how to compose the units in I in accordance with r; and o is the output unit produced using f. Declarative Completeness Units pertaining to concerns other than the dominant one are often scattered across modules or tangled within a single module. Often, these units are still related in a number of ways. For example, one function unit may invoke another, or it may define or use a variable declaration unit, creating high levels of coupling. To decouple them, hyperslices are defined to be declaritively complete, that is they declare everything to which they refer. For example, each hyperslice must, at minimum, include a declaration for each function that its members invoke, as well as for any variables its members use. Implementation details for these declaration are not needed. The declaration may be abstract, specifying all or part, of the formal or informal properties the hyperslice relies on. Formally, a predicate, decl, is used on units to identify declarations. Sometimes, declarations are different to accommodate hyperslices that have different needs. To prevent false overlap, the implementation set of hyperslices may be defined as: I(hs) ={u hs decl(u)}. The implementation set of a concern is defined similarly. This property of declarative completeness means each hyperslice is self-contained with respect to January 22, 2002 Page 4 of 10

declarations. Suppose, for example, hyperslice X contains a unit u 1 for its method x() that use a accessor function y() from unit u 2 defined in hyperslice Y. To make X declaratively complete, it must contain another unit u 2decl which declares y(), without necessarily implementing it, so that u 1 uses u 2decl instead of u 2. This eliminates coupling between X and Y with the understanding that u 2decl will eventually be bound to an implementation in some other hyperslice. Correspondence To create a working implementation in the above example, some hyperslice must provide a unit that can be bound to u 2decl. This relationship is referred to as a correspondence and only occurs within the context of a particular software component or system. Therefore, the same declarative unit may be associated with different implementation units in different software products, as defined in the associated hypermodule. Once identified, corresponding units can be composed or integrated. For example, the composition of corresponding units u 2 and u 2decl results in u 2 being called by u 1 at runtime. Relationships Units, concerns, and hyperslices may be related in a variety of ways. For example, hyperslices may overlap and share units. In this case, sharing a unit in one hyperslice has a side effect on the other. Alternately, two hyperslices might be interrelated through one or more integration relationships that indicate how the two are to be combined. There are two distinct classes of relationships: context-insensitive and context-sensitive. Overlap is a context-insensitive relationship that exists as long as the associated hyperslices share common units. Integration is a context-sensitive relationship that only exists if the associated hyperslices are integrated in some context that is not inherent in their definition. Other common relationship types include generalizes, subsumes, and precludes. Composition To integrate a set of hyperslices, it is necessary to identify the required set of correspondence relationships among them and then determine how to compose to effect integration. Some examples of correspondence relationships include: binding of abstract declarations to implementations; correspondence of multiple implementations to integrate their respective capabilities; or elaboration - for example, a design unit might elaborate a requirement unit during model refinement. Composition rules are specifications for composition relationships They are succinct, and rely on January 22, 2002 Page 5 of 10

defaults and uniformity for composition mainly at the hyperslice level, rather than at the individual unit level. Composition details may vary greatly, depending on the artifact s formalism and how the formalism s constructs are treated as units and modules. Composition is based on commonality of concepts across units. That is, different units describing the same concept are composed into a single unit describing that concept more fully. This involves three steps: matching units in different hyperslices that describe the same concept; reconciliation of differences in these descriptions; and integration of units to produce a unified and whole implementation. Presently, composition cannot be fully automated and requires software developers to manually specify the details of the composition rules in the hypermodule. Composition Rules One approach to composition rules, based on subject-oriented programming[2][5], proposes rules be a combination of a concise, general rule, and a series of detailed, specific rules. The specific rules are used to explicitly state those exceptions to the general rule or to handle special cases not handled implicitly by the general rule. Application of the general rule may be automated since the general rule may be applied to either the entire composition or selectively portions of it. Detailed rules can then be employed to handle special issues such as: matching units with different names that describe the same concept; not matching units with the same names that describe different concepts; and reconciling different module structures (e.g., matching units nested at different depths in different hyperslices). IBM Hyper/J IBM Hyper/J is a tool that supports hyperspaces for the Java language formalism. The IBM Hyper/J installation manual and user guide[9] is provide with the Hyper/J binary and can be downloaded from <http://www.alphaworks.ibm.com>. The user manual provides details for running the tool, along with details for creating a hyperspace specification file, a concern mapping file, and a hypermodule. The experimental (i.e., binary) version of Hyper/J is written in standard Java and released as a standard jar file. The Hyper/J tool can operate in verbose mode. In this mode, the tool creates unparsed files and a dump January 22, 2002 Page 6 of 10

of the hyperspace that shows the dimensions and concerns created and the units that address each of those concern. Hyper/J also has a debug option to generate information for debugging erroneous hypermodule specifications, however, the output generated tends to be very large, detailed, and tedious to manually parse. A GUI would be a definite advantage for automatically identify errors through a WYSIWYG graphical interface. Unfortunately, Hyper/J source code and Hyper/J GUI is not available from IBM without special licensing arrangements. Hyper/J is presently intended for units at the granularity of declarations (e.g., classes, UML diagrams) rather than lower-level constructs (e.g., statements, expressions). The tool presently supports one language for defining units - Java 1. The Java language defines both declarator constructs (e.g., packages, interfaces, classes, methods) and statements. In Hyper/J, only a subset of these constructs are treated as units. A finer granularity might include all declarators and statements and thus allow the flexibility to compose any pieces of Java source. However, this would make matching and reconciliation enormously complex and difficult to analyze. A courser granularity simplifies the composition and the understandability of composed results at the cost of generality. The extension of Hyper/J to handle finer grained units in an organized fashion remains an issue of future research[1]. The Hyper/J tool itself works on, and generates, standard Java class files. All support for multidimensional separation of concerns occurs outside the standard Java language. Therefore, no special coding conventions or packages are required. Hyper/J can also be used with only class files (i.e., no source). However, this requires software developers have an intimate understanding of the software product and the actual program, interfaces, and method signatures from up-to-date documentation or class file dumps. To use Hyper/J, a software developer must write the necessary hypermodule. For each concern of importance, a hyperslice must be created and the appropriate composition rules specified. These rules direct Hyper/J on how to compose the specified hyperslices into a set of legal modules. The tool takes, as input: a project specification used to identify Java code units in a given hyperspace; a concern mapping used to describe how units are organized; and a hypermodule specification. Hyper/J is intended to allow software developers to identify and manipulate concerns, to focus in on 1. According to the literature, Hyper/J is being extended to incorporate Unified Modeling Language (UML). January 22, 2002 Page 7 of 10

particular dimensions of concern, and to create hypermodules by trial-and-error integration of concerns. Using the experimental version of Hyper/J makes this a time consuming process. The assistance of the GUI to highlight errors and relate them back to the original lines in the project specifications is essential for software products of considerable size. The software developer starts by creating the hypermodules. A set of concerns and a default (i.e., general) compositional relationship among those concerns is then chosen. Running Hyper/J creates the valid hyperslices for the concerns using automatic declaration completion. The tool then composes the hyperslices based on the software developer-specified general and specific compositional relationships. Class files produced by Hyper/J are standard Java class files and can be used as input to Hyper/J for further composition. Although these class files can be reused, once again, the software developer requires up-to-date implementation details such as updated interfaces and method signatures. At present, Hyper/J cannot extend Java library classes, however, this functionality is planned for a future release. Limitations A stated previously, the Hyper/J tool works on the granularity of whole members and does not support pulling apart code in method bodies. Instead, software developers must separately provide their own methods to accomplish any necessary processing for each concern of interest. The Hyper/J tool is then used to compose these methods as they are required. This leaves two choices for handling code for multiple concerns that are tangled in a single method body: first, the entire method to be excluded from composition by assigning the existing method to the None concern. Then, a new method must be separately provided by the software developer for each concern of interest; or second, if the Java source code is available, the software developer can attempt to modify and split up the original method into two or more methods. This has the obvious negative effect for other concerns that still require the original method. Neither option is desirable. The major limitations on the functionality of the Hyper/J tool are detailed in [9] and, therefore, not repeated here. Related Work This section will provide a very brief summary of work related to the MDSOC approach. Where appropriate, comparisons are made between concepts in MDSOC and the related work: January 22, 2002 Page 8 of 10

Subject-oriented programming[2][5] provides a model for object-oriented implementations where units are classes, methods, and instance variables. Systems are built as compositions of subjects where each is a class hierarchy modeling its domain from a particular point of view. This is similar to the role of a hyperslice in MDOSC. Like MDSOC, subject-oriented compositional rules provide considerable matching and reconciliation flexibility. The aspect-oriented programming (AOP)[4] paradigm is used to identify and separate non-functional, cross-cutting concerns (e.g. concurrency, distribution, persistence). AOP distinguishes between core classes, which encapsulate a system s functional requirements, from aspects, which encapsulate the non-functional, crosscutting requirements. Aspects are written with respect to core classes and are essentially orthogonal to one another. Individual aspect may be viewed as a hyperslice, and a set of aspects together with the core classes approximates a hypermodule. However, there is no central composition rule. Instead, each aspect contains its part of the rule, specifying how that aspect is to be woven into the base classes. This can make the handling of overlapping concerns (i.e., interaction among aspects) difficult to handle. Compositional contracts[3] have also been proposed for specifying reusable components. In this approach, each contract specifies a set of participant objects and their interactions, expressed as obligations. These are encapsulated to clearly separated them from other interactions involving the same object. A single object can participate in multiple contracts, but must satisfy all its obligations. There are many possible compositional rules for contracts. Contracts correspond to hyperslices and cut across classes. Conclusions Separation of concerns is a software engineering concept used to reduce the complexity of software system to manageable levels, in human terms, and allows software developers to focus on a smaller sets of details, while ignoring other less important details. To effectively employ the MDSOC approach. software developers must be able to identify, encapsulate, modularize, and manipulate multiple dimensions at any point in the software development lifecycle without suffering from the effects of invasive, modification, evolution, or rearchitecture. The stated ultimate goal of Hyper/J is to provide developers with the capability of on-demand remodularization. This means that under ideal circumstances, software developers must be able to choose the best modularization scheme, on the fly, for those concerns of importance and for the particular task at January 22, 2002 Page 9 of 10

hand. Unfortunately, at present, this is still an ambitious set of goals. No existing mechanism, including Hyper/J, fully satisfies them. Using Hyper/J is only a semi-automatic process. To use the tool effectively, a software developers must ultimately understand enough about the corresponding units to determine whether or not they are compatible and how best to integrate them. The issue of whether corresponding units are compatible and whether an implementation unit satisfies a declaration unit s requirements or whether a design unit satisfies a requirement involves both syntactic and semantic issues. How to characterize and check for such compatibility remains an issue of future research. This work is clearly at an early stage, largely unproven as yet. There is a considerable body of experience and related research now exists to support the claim that mutli-dimensional separation of concerns is one of the key software issues today. However, a fully automated approach remains elusive. References [1] M. Aksit (editor), Software architectures and component technology, International Series in Engineering and Computer Science, Volume 648, Kluwer Academic Publishers, October 2001. [2] S. Clarke, W. Harrison, H. Ossher, P. Tarr, Subject-oriented design: toward improved alignment of requirements, design, and code, OOPSLA 99, ACM, 1999, pp. 325-339. [3] I. Holland, Specifying reusable components using contracts, ECOOP 92, Lecture Notes in Computer Science, Volume 615, Springer, 1992, pp.287-308. [4] G. Kiczales, J. Lamping, A. Mendhekar, C. Maeda, C. Lopes, J.Loingtier, J. Irwin. Aspect-oriented programming, ECOOP'97, Lecture Notes in Computer Science, Volume 1241, Springer, 1997, pp. 220-242. [5] H. Ossher, M. Kaplan, W. Harrison, A. Katz, V. Kruskal, Subject-oriented composition rules, OOPSLA 95, ACM, October 1995, pp. 235-250. [6] H. Ossher, P. Tarr, Multi-dimensional separation of concerns using hyperspaces, IBM Research Report 21452, April, 1999. [7] H. Ossher, P. Tarr, Using multidimensional separation of concerns to (re) shape evolving software, Communications of the ACM, October 2001, pp.43-50. [8] P. Tarr, H. Ossher, W. Harrison, S. Sutton Jr., N degree of separation: mutli-dimensional separation of concerns, ICSE 99, ACM, 1999, pp.107-119. [9] P. Tarr, H. Ossher, Hyper/J User and Installation Manual, <www.research.ibm.com/hyperspace>, IBM, 2000. January 22, 2002 Page 10 of 10