INTEGRATING USER S REQUIREMENTS IN AUTOMATED CONCEPTUAL DATA WAREHOUSE DESIGN Opim Salim Sitompul and Shahrul Azman Mohd Noah 1 ) Abstract Conceptual design is the most important stage in data warehouse design since it represents a general view of data that could be understood by both users and system analysts and it is the key for the success of subsequent stages of the design process. In its implementation, data warehouse design rests on multidimensional model thereby many efforts have been directed to formulate a concise approach to develop this model. One of the approaches is called the transformationoriented methodology by which the entity-relationships (ER) model of a database is progressively transformed into the multidimensional model. A number of research works have shown that this methodology could be implemented into an automated tool. As a matter of fact, however, the automated tool could only provide the first cut of the design output, thus further refinements are still necessary to accommodate specific user s requirements. In this paper, we propose a refinement approach to facilitate intelligent interactions between such a tool called the DWDesigner and the user to make such refinement. Using this approach user could interactively refine measures, temporal dimensions, and dimension hierarchies of the multidimensional constructs. Keywords: data warehouse design, automated tool, multidimensional model. 1. Introduction Conceptual model is a preliminary design model that could help both users and system analysts to describe the data warehouse in general terms without theoretical or technical jargons. In this manner, users could participate in the design process using only general terms to propose ideas about the new system, whereas system analysts suggest design consideration using only users recognized terms. In addition, the conceptual model is also important as the foundation for the logical and physical design stages where modeling errors could be detected early and the schema could be extended easily [6, 18]. It is universally agreed that data warehouse implementation rests on multidimensional model; therefore many endeavors have been focused on the development of concise methodology to build this model (see e.g. [3], [6], [17], [13], [8]). As a design approach, most of the methodologies use ER model as a basis and by using the transformation-oriented methodology, this model is transformed into multidimensional model. 1 Faculty of Information Science and Technology, National University of Malaysia, 43600 UKM Bangi, Selangor, Malaysia
Although a number of methods supporting the aforementioned approach have been proposed, but the capacity of these methods to be successfully implemented in the form of computer aided software engineering (CASE) largely remains a question. Many tools developed (see e.g. [7, 18], [2], [17], [4, 5], [9]) to support the process of data warehouse design are passive and incapable of supporting the basic characteristics of data warehouse design [10]. The tools are unable to fulfill specific user requirements since they could only build the data warehouse model based on what have been stated explicitly in the design sources, thus could only produce the first cut of the model. For example, ER model that is commonly used as the design sources usually does not express explicitly some of the data warehouse s features such as measures and temporal dimensions. As a consequence, user intervention is a necessity in order to revise the initial model in order to obtain a model that will suit specific requirements. In this paper we will describe our approach to the refinement of multidimensional model, which is developed using an automated tool for data warehouse conceptual design called the DWDesigner[15]. The approach utilizes a command shell to facilitate intelligent interactions between the tool and the user to perform the refinement process. Using this approach user could interactively refine measures, temporal dimensions, and dimension hierarchies of the multidimensional constructs. The rest of this paper is structured as follows. In Section 2 we will describe the process of translating the ER model into multidimensional model using the transformation-oriented methodology with the intention to provide readers with some background on how the transformation process is performed. The multidimensional model resulted from this process is the first-cut of the conceptual data warehouse design that should be refined by user. Section 3 describes the process of integrating user s requirements into the resulting multidimensional model, illustrating how the refinement of each multidimensional construct is performed. In Section 4 we will show the result of the refinement process and discuss some important concepts related to the process. Section 5 provides the conclusions and future research works. 2. The Transformation Oriented Methodology Our approach to the conceptual data warehouse design is based on the transformation-oriented approach, which transforms the ER model into multidimensional model in five stages: translation of ER model into specification language model, transformation of specification language model into problem domain model, expansion of the problem domain model, transformation of the problem domain model into multidimensional model, and refinement of the multidimensional model. The first stage is a process of translating the ER model into a specification language in order to represent the application domain in a computer readable form necessary for the transformation process. The specification language resembles a class construct in which an entity is represented as a class with the name of the entity as the class name and the entity properties such as attribute, identifier, subclass, aggregation, and relationship are written as the class properties. The translation is guided by a set of syntax rules in order to record properties and semantic contents of the ER model [14]. A collection of entity classes representing the whole entities of the ER model is recorded into a text file. To illustrate this process, we will look at a portion of ER model for a university database (adapted from [1]) as shown in Figure 1.
Figure 1. Portion of ER model for a university domain The specification language model formulated based on the above ER model for the Student entity is given as in the following: CLASS "STUDENT" ATTRIBUTE (("Class": Integer)) IDENTIFIER NIL SUBCLASS ("GRAD_STUDENT") AGGREGATION NIL RELATIONSHIP (("Minor" "DEPARTMENT" "NIL" "(1 1)" "(1 n)")\ ("Major" "DEPARTMENT" "NIL" "(1 1)" "(1 n)")\ ("Registered" "CURRENT_SECTION" "(("Count": Integer))" "(1 n)" "(1 m)")\ ("Transcript" "SECTION" "(("Grade": Float))" "(1 n)" "(1 m)")) End-Class In the second stage, each entity and its property is transformed using a simple parser into a compound term representation, which takes a form of property entity values triplet; and is stored into a database as the initial problem domain model. For example, the Student entity in the specification language model above is transformed into the following initial problem domain. Has-Attribute STUDENT ((Class. Integer)) Has-Subclass STUDENT (GRAD_STUDENT) Has-Relationship STUDENT (((Name. Minor) (Participating-obj. DEPARTMENT) (Rel-Attribute. NIL) (First-constraint. (1 1)) (Second-constraint. (1 n))) ((Name. Major) (Participating-obj. DEPARTMENT) (Rel-Attribute. NIL) (First-constraint. (1 1))(Second-constraint. (1 n))) ((Name. Registered) (Participating-obj. CURRENT_SECTION) (Rel-Attribute. ((Count. Integer))) (First-constraint. (1 n)) (Second-constraint. (1 m))) ((Name. Transcript) (Participating-obj. SECTION) (Rel-Attribute. ((Grade. Float))) (First-constraint. (1 n)) (Second-constraint. (1 m)))
The initial problem domain model contains only facts that are created from the non-nil values of the entity s properties. In the subsequent stage, this model is then progressively analyzed using a set of synthesis and diagnosis rules and as a result new facts are augmented into the database. The basic constructs of the multidimensional model are built from the entity attributes, which are categorized into numeric, date/time, and other attributes. Entities with numeric attributes are candidates for fact schemes; in this case, the name of the fact scheme is taken from the entity name and its measures are obtained from the numeric attributes. The date/time and other attributes are the basis for creating temporal and other dimensions of the fact scheme. In addition, the system also considers getting additional dimensions from other entities connected to the fact entity through relationships. In this case, the system will recursively examine the existence of one-to-many relationship between the fact entity and the other entities. Dimensions obtained from those relationships represent sub-trees of dimension hierarchies. The last stage is the refinement process by which users will be able to integrate specific requirements by adding or modifying the multidimensional constructs such as measures, temporal dimension, and dimension hierarchies. Details on how the refinement could be performed are described in the following section. 3. The Refinement Process After completing the transformation process, the user will obtain a list of candidate fact schemes representing the multidimensional model. This model, however, is merely a direct transformation of the ER model whereby the multidimensional constructs such as measures, temporal dimensions, and dimension hierarchies are created automatically. Nevertheless, without user intervention the tool has no way of knowing about what features should be included in the final data warehouse model, thus the user should evaluate the resulting model in order to integrate those requirements. In this section we will look on how the refinement process is implemented in the system and as an example we will look at a student fact scheme generated by the tool from the university database as shown in Figure 2. Figure 2. A student fact scheme
The student fact scheme is represented in the form of a tree structure [3], which rooted on the Student fact and has one measure called Class. Nodes directly connected to the root are dimensions in the form of simple sub-trees containing one or more leaf nodes, or sub-trees with one or more branches. In the above example, there are six dimensions: five of them are simple sub-trees (BDate, Name, Sex, Ssn, and Address), and a sub-tree with one branch stemming from the DeptName node forming a dimension hierarchy. 3.1. Refining Measures Measure is the focus of interest in data warehouse design described through a set of atomic or derived attributes [16]. User could evaluate the applicability of a measure by evaluating it along the available dimensions with one or more aggregation functions, such as count, sum, average, minimum, and maximum. Since measure is commonly of numeric type, one important thing to be considered is summarizability (additivity). Summarizability ensures the correctness of a measure whenever it is aggregated (summarized) along a specific dimension hierarchy by avoiding double counting of data and avoiding addition of non-additive data [12]. For example, the Class measure of the Student fact scheme, which corresponds to the year a student in a university, could be evaluated using the count function. In this case, we can count the number of students for each class based on sex, age, and country of origin for each department or college. Further consideration on this measure shows that the counting of students for each class will lead to the number of students in the university, including undergraduate and graduate students. In this case, user could do the refinement by modifying the Class measure. User could refine the measure construct of the multidimensional model by interacting with the system in a series of question-answer type dialog as illustrated in Figure 3. Figure 3. Example refinement session for measure construct The dialog confirms user that he has chosen to modify the measure construct and asked whether he wants to modify it or need to see an explanation about the measure construct. If the user chooses E[xplain], a brief explanation about measure will be displayed. After that, the user could continue
with the modification by choosing [Y]es or exit from the dialog by choosing [D]one. Continuing the refinement process will cause the system ask the user to enter each measure in a specified format i.e. one or more (measure type) pairs separated by parenthesis. After entering the new measure, the system will reconfirm user on the input given. The option is [O]k or [N]o. If the user chooses [O]k, then system will override the old measure with the new provided measure, but if for some reason the user chooses [N]o then system will prompt the user to re-enter the new measure. 3.2. Refining Temporal Dimension Temporal dimension is a multidimensional construct that should be given more attention since time plays a very important role in data warehousing. It could be represented either as time interval that spans from a shorter to a longer time period or as a specific point of time resembling a snapshot of the data warehouse. Obtaining temporal dimension from an ER model is not always successful since this feature is commonly unavailable or not stated explicitly. From the Student fact scheme, for instance, the temporal dimension obtained is BDate, which has an attribute type of date/time. This dimension, however, neither resembles a time interval nor a time snapshot of a university data warehouse. Therefore, The user should modify this dimension to add time granularity such as month semester year to enable appropriate counting of the number of student in each department or college. An example of the refinement session is illustrated in Figure 4. Figure 4. Example of refinement session for temporal dimension As in refining measure, user could ask for an explanation or make the refinement directly. The explanation will describe the importance of temporal dimension construct in data warehouse design to capture the historical aspect of data and recommend user to have one. For the refinement, user has some choices of available temporal dimensions or could provide customize temporal dimension that fulfills the design requirements. In the above dialog, the user chooses the month semester year interval to count the number of students for each department or college.
3.3. Refining Dimension Hierarchies Dimension hierarchy is a structuring of dimensions to determine how fact instances may be aggregated and selected for decision-making process [3]. The aggregation functions applied to the dimension hierarchies can be classified into three sets of functions, namely functions that could be applied to data that can be added, data that can be used for average calculations, and constant data that can only be counted [11]. Those three aggregation functions could semantically used to keep track of what type of aggregate functions can be applied to a specific data. The automated tool obtains dimension hierarchies from other attributes of a fact entity and recursively add more hierarchy levels from other attribute of an entity that has many-to-one relationship with the fact entity. An attribute tree will grow from the attributes of this chain of entity-to-entity relationships; rooted at the identifier of the fact entity. The refinement of the dimension hierarchies produced by the tool could be done by performing three types of activities, i.e. pruning, grafting, and aggregating the attribute tree. Pruning and grafting is intended to remove dimensions that are not relevant to the fact scheme being considered. The relevance of a dimension to the fact scheme could be determined by evaluating the relationship between the fact and the dimension as well as among all dimensions in a hierarchy. Pruning the attributes trees is to remove a dimension and all its descendants, thus decreasing the number of available dimensions. Grafting a dimension on the other hand, is to remove the dimension but maintains all its descendants. Nodes directly connected to the deleted dimension are becoming new dimensions, thus increasing the number of dimensions in the multidimensional construct. Aggregating the attribute tree is a way to add new classification hierarchies to the fact scheme. The aggregation could be built from the existing attributes or by adding new attributes. The dialog in Figure 5 illustrates the refinement process of the dimension hierarchies for the Student fact scheme. Figure 5. Example of refinement session for dimension hierarchy In the above example, the user chooses to prune the Name dimension since this dimension is not important to the count of the number of students being considered. Then the user would consider grafting the Address dimension in order to maintain the City and State properties. Subsequently, the
user could add another hierarchy level by aggregating the City and State dimensions with a new dimension Country. As in the other refinements, user could either get a brief explanation about pruning, grafting, and aggregating or go directly for the refinement by choosing one of the three options available. Pruning and grafting can be performed on a single node or a set of nodes of the attributes tree, whereas aggregating can be performed by combining two or more dimensions into one aggregation level. As necessary, while aggregating the dimensions user can put additional dimensions level for finer or coarser granularities. The refinement process can be repeated to fulfill specific requirements. 4. Results and Discussion User intervention in order to refine the multidimensional model produced by the automated tool is indeed necessary due to the fact that the automated tool will only produce the multidimensional model based on the entity properties of the ER model. As an illustration, it has been shown in the previous section that user should refine the temporal dimension of the Student fact scheme since the automated tool would only suggest that the temporal dimension for the multidimensional model is BDate. This is because BDate is the only attribute of the Student entity that has the date/time field type. To illustrate how the automated tool generates the multidimensional model that fulfill specific user s requirement, Figure 6 shows an output from the DWDesigner for the Student entity before and after the refinement. Figure 6. Fact scheme of Student: (a) Before refinement (b) After refinement In order to arrive at the desired multidimensional model as shown in Figure 6, the steps taken during the refinement process are listed in the following:
a. Refining measures Modifying Class into Number-Of-Student b. Refining Temporal dimension Modifying BDate into Month Semester Year c. Refining dimension hierarchies - Pruning Name - Grafting Address - Pruning No, Street, AptNo, and Zip - Aggregating City, State and adding Country - Pruning DeptPhone, Office, CollegeOffice and Dean - Aggregating DeptName and CollegeName The result obtain from the refinement process are definitely depends on user preferences. For instance, users should determine what would be the focus of the analysis, how data will be analyzed in terms of time aggregation, what would be the dimensions used for analyzing the data, and how those data would be aggregated. Initial multidimensional data produced by the automated tool could be used as the basis to dynamically refine the data model to fulfill those requirements. Apparently, this would be beneficial both to users and system analysts since they can work together to formulate current as well as new requirements. Output from the DWDesigner illustrating result of the refinement process described above can be seen in Figure 7. Figure 7. Output from the DWDesigner for the refinement of the Student fact scheme
5. Conclusions and Future Research Works In this paper we have presented an approach to refine multidimensional model necessary for conceptual data warehouse conceptual design in order to fulfill user s requirements. Users could initiate a dialog with the system through a command shell to refine the existing multidimensional constructs such as measures, temporal dimensions and dimension hierarchies that have been automatically generated by the system. In modifying the dimension hierarchies, users could perform pruning and grafting the attribute tree as well as aggregating two or more dimensions. The automated tool developed generates the multidimensional model in the form of graphical output to enhance the readability of the multidimensional model. As an initial step to the development of CASE tool for data warehouse design, this research work provides a basis for developing the logical and physical design phases. The prototype of the automated tool has been developed using Allegro Common LISP version 6.2 from Franz Inc. Some issues that are still open for future research works could be given as in the following: o Input to DWDesigner is still manually translated from existing ER models. This could be enhanced by automatically acquiring the input from various operational databases; thereby it should be added with a data integration module. o The simple parser used in translating the specification language into the initial problem domain model could be enhanced by a capability to perform syntax analysis to ensure correct input is being provided to the DWDesigner. o As data warehouse is concern with integrating multiple databases, therefore, a module that can perform aspects of integration process at the schema level should be provided. Furthermore, with the introduction of the semantic web technology, unstructured database such as web pages should also be constructed for analysis and processing in supporting the data warehouse system. o As a design tool, the system developed is still in prototype form with simple graphical user interface and direct conversation with users during the refinement session. This interface could be enhanced with user-friendlier interface by providing users with point-and-click capability to perform the transformation process and to refine the multidimensional model 6. References [1] ELMASRI, R., NAVATHE, S.B, Fundamentals of database systems, 3 rd Edition, Reading, Mass.: Addison- Wesley, 2000. [2] FRANCONI, E., NG, G., The i.com Tool for Intelligent Conceptual Modeling, Proceedings of 7th International Workshop on Knowledge Representation Meets Databases (KRDB-2000), Berlin, Germany, 2000. [3] GOLFARELLI, M., MAIO, D., RIZZI, S., Conceptual design of data warehouses from E/R schemes, Proceedings of the 31st Hawaii International Conference on System Sciences, Kohala Coast, HI, 1998. [4] GOLFARELLI, M., RIZZI, S., WAND: A CASE Tool for Data Warehouse Design, Demo Proceedings of 17th International Conference on Data Engineering (ICDE 2001), Heidelberg, Germany, 2001. [5] GOLFARELLI, M., RIZZI, S., SALTARELLI, E. WAND: A CASE Tool for Workload-Based Design of a Data Mart, Proceedings Decimo Convegno Nazionale su Sistemi Evoluti Per Basi Di Dati, Portoferraio, Italy, 2002.
[6] HÜSEMANN, B., LECHTENBÖRGER, J., VOSSEN, G., Conceptual data warehouse design, Proceeding of the International Workshop on Design and Management of Data Warehouse (DMDW 2000), Stockholm, Sweden 2000. [7] MILLER, L., NILAKANTA, S. Data Warehouse Modeler: a CASE Tool for Data Warehouse Design, Proceeding of 31st Annual Hawaii International Conference on System Sciences, Kona, Hawaii, 1998. [8] MOODY, D., KORTINK, M.A.R., From enterprise models to dimensional models: A methodology for data warehouse and data mart design, Proc. of Int'l Workshop on Design and Management of Data Warehouses, Stockholm, Sweden, 2000. [9] NAGGAR, P, PONTIERI, L., PUPO, M., TERRACINA, G., VIRARDI, E., A Model and a Toolkit for Supporting Incremental Data Warehouse Construction, In Ciccheti, R. et al. (Eds.): DEXA 2002, LNCS 2453, Springer-Verlag Berlin Heidelberg, 2002. [10] NOAH, S.A., WILLIAMS, M., Intelligent object analyser for conceptual database design model. Jurnal Teknologi 39: 27-44, 2003. [11] PEDERSEN, T. B., JENSEN, C. S. Multidimensional Data Modeling for Complex Data, Proceedings of 1st International Conference on Data Engineering (ICDE 99), Sydney, Australia, 1999. [12] PEDERSEN, T. B., JENSEN, C. S., DYRESON, C. E., A Foundation for Capturing and Querying Complex Multidimensional Data, Information Systems, 26, 2000. [13] PHIPPS, C., DAVIS, K. C., Automating data warehouse conceptual schema design and evaluation. Design and Management of Data Warehouses 2002, Proceedings of the 4th Intl. Workshop DMDW'2002, Toronto, Canada, 2002. [14] SITOMPUL, O.S., NOAH, S.A.M., Translation of ER model into multidimensional model for data warehouse: an automated approach, Int. J. of Information Technology 3: 11-32, Bangi, Malaysia, 2002. [15] SITOMPUL, O.S., NOAH, S.A.M., Application of knowledge-based system in automated data warehouse design. Proceedings of the Knowledge Management International Conference and Exhibition (KMICE 2004), Penang, Malaysia, 2004. [16] TRUJILLO, J. PALOMAR, M., GOMEZ, J., SONG, I.-Y., Designing Data Warehouses with OO Conceptual Models, IEEE Computer, 34(12): 66 75, 2001. [17] TRYFONA, N., BUSBORG, F., CHRISTIANSEN, J. G. B., starer. Proceedings of the ACM 2nd International Workshop on Data Warehousing and OLAP, ACM Press, New York, 1999. [18] WU, L., MILLER, L., NILAKANTA, S., Design of Data Warehouse Using Metadata, Information and Software Technology, 43: 109-119, 2001;