DATA MODELING FUNDAMENTALS

Transcription

1 DATA MODELING FUNDAMENTALS A Practical Guide for IT Professionals Paulraj Ponniah

2

3 DATA MODELING FUNDAMENTALS

4

5 DATA MODELING FUNDAMENTALS A Practical Guide for IT Professionals Paulraj Ponniah

6 Copyright # 2007 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) , fax (978) , or on the web at Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) , fax (201) , or online at Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) , outside the United States at (317) or fax (317) Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at Library of Congress Cataloging-in-Publication Data Ponniah, Paulraj. Data modeling fundamentals: a practical guide for IT professionals / by Paulraj Ponniah. p. cm. ISBN-13: (cloth) ISBN-10: (cloth) 1. Database design. 2. Data structures (Computer science) I. Title. QA76.9.D26P dc Printed in the United States of America

7 To Daniel Arjun, my dear son-in-law and to Reisha and Shoba, my dear daughters-in-law

8

9 CONTENTS PREFACE ACKNOWLEDGMENTS xvii xxi I INTRODUCTION TO DATA MODELING 1 1 Data Modeling: An Overview 3 Chapter Objectives / 3 Data Model Defined / 4 What Is a Data Model? / 5 Why Data Modeling? / 6 Who Performs Data Modeling? / 9 Information Levels / 10 Classification of Information Levels / 11 Data Models at Information Levels / 13 Conceptual Data Modeling / 17 Data Model Components / 18 Data Modeling Steps / 20 Data Model Quality / 26 Significance of Data Model Quality / 27 Data Model Characteristics / 27 Ensuring Data Model Quality / 28 Data System Development / 29 Data System Development Life Cycle / 29 Roles and Responsibilities / 33 Modeling the Information Requirements / 33 Applying Agile Modeling Principles / 34 Data Modeling Approaches and Trends / 35 Data Modeling Approaches / 36 Modeling for Data Warehouse / 38 vii

10 viii CONTENTS Other Modeling Trends / 39 Chapter Summary / 41 Review Questions / 41 2 Methods, Techniques, and Symbols 43 Chapter Objectives / 43 Data Modeling Approaches / 44 Semantic Modeling / 44 Relational Modeling / 45 Entity-Relationship Modeling / 46 Binary Modeling / 46 Methods and Techniques / 47 Peter Chen (E-R) Modeling / 48 Information Engineering / 50 Integration Definition for Information Modeling / 51 Richard Barker s Model / 53 Object-Role Modeling / 55 extensible Markup Language / 57 Summary and Comments / 60 Unified Modeling Language / 61 Data Modeling Using UML / 61 UML in the Development Process / 64 Chapter Summary / 68 Review Questions / 68 II DATA MODELING FUNDAMENTALS 71 3 Anatomy of a Data Model 73 Chapter Objectives / 73 Data Model Composition / 74 Models at Different Levels / 74 Conceptual Model: Review Procedure / 76 Conceptual Model: Identifying Components / 77 Case Study / 81 Description / 81 E-R Model / 84 UML Model / 87 Creation of Models / 89 User Views / 90 View Integration / 92 Entity Types / 96 Specialization/Generalization / 98 Relationships / 98 Attributes / 100

11 CONTENTS ix Identifiers / 101 Review of the Model Diagram / 103 Logical Model: Overview / 104 Model Components / 104 Transformation Steps / 107 Relational Model / 109 Physical Model: Overview / 111 Model Components / 111 Transformation Steps / 112 Chapter Summary / 113 Review Questions / Objects or Entities in Detail 115 Chapter Objectives / 115 Entity Types or Object Sets / 116 Comprehensive Definition / 116 Identifying Entity Types / 120 Homonyms and Synonyms / 125 Category of Entity Types / 127 Exploring Dependencies / 130 Dependent or Weak Entity Types / 131 Classifying Dependencies / 132 Representation in the Model / 133 Generalization and Specialization / 134 Why Generalize or Specialize? / 136 Supertypes and Subtypes / 137 Generalization Hierarchy / 138 Inheritance of Attributes / 140 Inheritance of Relationships / 140 Constraints / 141 Rules Summarized / 144 Special Cases and Exceptions / 144 Recursive Structures / 145 Conceptual and Physical / 145 Assembly Structures / 147 Entity Type Versus Attribute / 148 Entity Type Versus Relationship / 148 Modeling Time Dimension / 149 Categorization / 150 Entity Validation Checklist / 153 Completeness / 153 Correctness / 154 Chapter Summary / 155 Review Questions / 155

12 x CONTENTS 5 Attributes and Identifiers in Detail 157 Chapter Objectives / 157 Attributes / 158 Properties or Characteristics / 158 Attributes as Data / 161 Attribute Values / 162 Names and Descriptions / 163 Attribute Domains / 164 Definition of a Domain / 164 Domain Information / 165 Attribute Values and Domains / 166 Split Domains / 167 Misrepresented Domains / 167 Resolution of Mixed Domains / 168 Constraints for Attributes / 169 Value Set / 169 Range / 170 Type / 170 Null Values / 170 Types of Attributes / 171 Single-Valued and Multivalued Attributes / 171 Simple and Composite Attributes / 171 Attributes with Stored and Derived Values / 172 Optional Attributes / 173 Identifiers or Keys / 175 Need for Identifiers / 175 Definitions of Keys / 175 Guidelines for Identifiers / 176 Key in Generalization Hierarchy / 177 Attribute Validation Checklist / 178 Completeness / 178 Correctness / 179 Chapter Summary / 180 Review Questions / Relationships in Detail 183 Chapter Objectives / 183 Relationships / 184 Associations / 184 Relationship: Two-Sided / 186 Relationship Sets / 187 Double Relationships / 187 Relationship Attributes / 189 Degree of Relationships / 190 Unary Relationship / 191 Binary Relationship / 191

13 CONTENTS xi Ternary Relationship / 193 Quaternary Relationship / 193 Structural Constraints / 194 Cardinality Constraint / 195 Participation Constraint / 198 Dependencies / 200 Entity Existence / 200 Relationship Types / 201 Identifying Relationship / 202 Nonidentifying Relationship / 204 Maximum and Minimum Cardinalities / 204 Mandatory Conditions: Both Ends / 206 Optional Condition: One End / 206 Optional Condition: Other End / 207 Optional Conditions: Both Ends / 208 Special Cases / 209 Gerund / 209 Aggregation / 210 Access Pathways / 211 Design Issues / 215 Relationship or Entity Type? / 215 Ternary Relationship or Aggregation? / 216 Binary or N-ary Relationship? / 216 One-to-One Relationships / 217 One-to-Many Relationships / 219 Circular Structures / 219 Redundant Relationships / 221 Multiple Relationships / 221 Relationship Validation Checklist / 222 Completeness / 223 Correctness / 224 Chapter Summary / 225 Review Questions / 225 III DATA MODEL IMPLEMENTATION Data Modeling to Database Design 229 Chapter Objectives / 229 Relational Model: Fundamentals / 231 Basic Concepts / 231 Structure and Components / 233 Data Integrity Constraints / 238 Transition to Database Design / 242 Design Approaches / 243

14 xii CONTENTS Conceptual to Relational Model / 243 Traditional Method / 244 Evaluation of Design Methods / 245 Model Transformation Method / 246 The Approach / 246 Mapping of Components / 249 Entity Types to Relations / 250 Attributes to Columns / 250 Identifiers to Keys / 252 Transformation of Relationships / 252 Transformation Summary / 267 Chapter Summary / 269 Review Questions / Data Normalization 271 Chapter Objectives / 271 Informal Design / 272 Forming Relations from Requirements / 272 Potential Problems / 273 Update Anomaly / 275 Deletion Anomaly / 275 Addition Anomaly / 276 Normalization Methodology / 276 Strengths of the Method / 277 Application of the Method / 277 Normalization Steps / 277 Fundamental Normal Forms / 278 First Normal Form / 278 Second Normal Form / 279 Third Normal Form / 281 Boyce-Codd Normal Form / 284 Higher Normal Forms / 285 Fourth Normal Form / 286 Fifth Normal Form / 287 Domain-Key Normal Form / 288 Normalization Summary / 290 Review of the Steps / 290 Normalization as Verification / 291 Chapter Summary / 292 Review Questions / Modeling for Decision-Support Systems 295 Chapter Objectives / 295 Decision-Support Systems / 296 Need for Strategic Information / 296

15 CONTENTS xiii History of Decision-Support Systems / 297 Operational Versus Informational Systems / 299 System Types and Modeling Methods / 299 Data Warehouse / 301 Data Warehouse Defined / 301 Major Components / 302 Data Warehousing Applications / 305 Modeling: Special Requirements / 305 Dimensional Modeling / 308 Dimensional Modeling Basics / 309 STAR Schema / 312 Snowflake Schema / 318 Families of STARS / 321 Transition to Logical Model / 322 OLAP Systems / 325 Features and Functions of OLAP / 325 Dimensional Analysis / 326 Hypercubes / 328 OLAP Implementation Approaches / 330 Data Modeling for OLAP / 332 Data Mining Systems / 334 Basic Concepts / 334 Data Mining Techniques / 338 Data Preparation and Modeling / 339 Data Preprocessing / 339 Data Modeling / 341 Chapter Summary / 342 Review Questions / 343 IV PRACTICAL APPROACH TO DATA MODELING Ensuring Quality in the Data Model 347 Chapter Objectives / 347 Significance of Quality / 348 Why Emphasize Quality? / 348 Good and Bad Models / 349 Approach to Good Modeling / 351 Quality of Definitions / 351 Importance of Definitions / 352 Aspects of Quality Definitions / 353 Correctness / 353 Completeness / 354 Clearness / 357 Format / 358

16 xiv CONTENTS Checklists / 358 High-Quality Data Model / 360 Meaning of Data Model Quality / 360 Quality Dimensions / 361 What Is a High-Quality Model? / 363 Benefits of High-Quality Models / 364 Quality Assurance Process / 365 Aspects of Quality Assurance / 365 Stages of Quality Assurance Process / 366 Data Model Review / 369 Data Model Assessment / 370 Chapter Summary / 373 Review Questions / Agile Data Modeling in Practice 375 Chapter Objectives / 375 The Agile Movement / 376 How It Got Started / 377 Principles of Agile Development / 378 Philosophies / 378 Generalizing Specialists / 379 Agile Modeling / 379 What Is Agile Modeling? / 380 Basic Principles / 380 Auxiliary Principles / 381 Practicing Agile Modeling / 381 Primary Practices / 381 Additional Practices / 382 Role of Agile DBA / 383 Agile Documentation / 383 Recognizing an Agile Model / 384 Feasibility / 384 Evolutionary Data Modeling / 385 Traditional Approach / 385 Need for Flexibility / 386 Nature of Evolutionary Modeling / 386 Benefits / 387 Chapter Summary / 388 Review Questions / Data Modeling: Practical Tips 391 Chapter Objectives / 391 Tips and Suggestions / 392 Nature of Tips / 392 How Specified / 392 How to Use Them / 392

17 CONTENTS xv Requirements Definition / 393 Interviews / 393 Group Sessions / 394 Geographically Dispersed Groups / 394 Documentation / 395 Change Management / 395 Notes for Modeling / 396 Stakeholder Participation / 396 Organizing Participation / 397 User Liaison / 397 Continuous Interaction / 398 Multiple Sites / 399 Iterative Modeling / 399 Establishing Cycles / 399 Determining Increments / 400 Requirements: Model Interface / 400 Integration of Partial Models / 401 Special Cases / 401 Legal Entities / 402 Locations and Places / 403 Time Periods / 405 Persons / 407 Bill-of-Materials / 409 Conceptual Model Layout / 409 Readability and Usability / 409 Component Arrangement / 410 Adding Texts / 416 Visual Highlights / 417 Logical Data Model / 417 Enhancement Motivation / 418 Easier Database Implementation / 418 Performance Improvement / 418 Storage Management / 419 Enhanced Representation / 419 Chapter Summary / 421 Review Questions / 421 Bibliography 423 Glossary 425 Index 433

18

19 PREFACE Do you want to build a hybrid automobile? First, you need to create a model of the car. Do you want to build a mansion? First, you need to have blueprints and create a model of the dwelling. Do you want to build a spaceship? First, you need to design a miniature model of the vehicle. Do you want to implement a database for your organization? First, you need to create a data model of the information requirements. Without a proper data model of the information requirements of an enterprise, an adequate database system cannot be correctly designed and implemented for the organization. A good data model of high quality forms an essential prerequisite for any successful database system. Unless the data modelers represent the information requirements of the organization in a proper data model, the database design will be totally ineffective. The theme of this book is to present the fundamentals and ideas and practices about creating good and useful data models data models that can function effectively as tools of communication with the user community and as database blueprints for database practitioners. THE NEED In every industry across the board, from retail chain stores to financial institutions, from manufacturing enterprises to government agencies, and from airline companies to utility businesses, database systems have become the norm for information storage and retrieval. Whether it is a Web-based application driving electronic commerce or an inventory control application managing just-in-time inventory or a data warehouse system supporting strategic decision making, you need an effective technology to store, retrieve, and use data in order to make the application successful. It is no wonder that institutions have adopted database technology without any reservations. In this scenario, the information technology (IT) department of every organization has a primary responsibility to design and implement database systems and keep them running. One set of special skills for accomplishing this relates to data modeling. Information technology professionals with data modeling skills constitute a significant group. Information technology professionals specializing in data modeling must be experts with a thorough knowledge of data modeling fundamentals. They must be well versed in the methodologies, techniques, and practices of data modeling. xvii

20 xviii PREFACE ADDRESSING THE NEED How can IT professionals desirous of acquiring data modeling skills learn the required techniques and gain proficiency in data modeling? Many seminar companies, colleges, and other teaching institutions offer courses in database design and development. However, such courses do not have data modeling as a primary focus. Very few courses, if any, concentrate just on data modeling. So, eager IT professionals are left with the choice of learning data modeling and gaining expert knowledge from books exclusively on this subject. How many such books should they read to learn the principles and concepts? This book intends to be the one definitive publication to fulfill the needs of aspiring data modelers, of those experienced data modelers desiring to have a refresher, and even of expert data modelers wishing to review additional concepts. In this volume, I have attempted to present my knowledge and insights acquired through three decades of IT consulting, through many years of teaching data-related subjects in seminar and college environments, and through graduate and postgraduate levels of studies. I do hope this experience will be of use to you. WHAT THIS BOOK CAN DO FOR YOU Are you a novice data modeler? Are you fairly new to data modeling but aspire to pick up the necessary skills? Alternatively, are you a practicing data modeler with experience in the discipline? Are you a generalizing specialist, meaning that you want to add data modeling as another skill to your arsenal of IT proficiency? Irrespective of the level of your interest in data modeling, this is the one book that is specially designed to cover all the essentials of data modeling in a manner exactly suitable for IT professionals. The book takes a practical approach in presenting the underlying principles and fundamentals, augmenting the presentation with numerous examples from the real world. The book begins in Part I with a broad overview of data modeling. In Chapter 1, you are introduced to all the essential concepts. Before proceeding into further details, you need to familiarize yourself with the data modeling techniques. Chapter 2 explores the leading techniques the approaches, the symbols, the syntax, the semantics, and so on. Part II of the book presents the fundamentals in great detail. It does not matter what your knowledge level of data modeling is. You will find this part interesting and useful. You are presented with a real-world case study with a completed data model. You are asked to study the anatomy of the data model and understand how the actual design and creation of the data model works. Part II also digs deeper into individual components of a data model with several real-world examples. In Part III, you will learn the transition from data model to database design. In recent times, decision-support systems have come to the forefront of computing. Part III describes decision-support systems such as data warehousing and data mining and guides you through data modeling methods for these systems. This is essential knowledge for modern data modelers. In Part IV of the book, you will find a chapter exclusively devoted to quality in the data model. Every data modeler aspires to create a model of the highest quality. This chapter is required reading. A new wave known as agile software development is on the rise

21 PREFACE xix producing great benefits. You will learn about this movement and gain insights into agile data modeling its principles and practices. Finally, are you looking for practical suggestions on data modeling distilled from years of experience of many practitioners? If so, the final chapter is for you. The book aptly concludes with such a chapter filled with numerous practical tips and suggestions. PAULRAJ PONNIAH Milltown, New Jersey April 2007

22

23 ACKNOWLEDGMENTS The authors listed in the bibliography at the end of the book greatly expanded and enhanced my understanding and appreciation for data modeling. I am deeply indebted to the authors, individually and collectively, for their insights and presentations. A great part of this book is a reinterpretation of their concepts and observations. I wish to express my special thanks to these authors. I must also record my gratitude to the several professional colleagues who had worked with me on various data modeling and database projects during my long IT consulting career. Also, thanks are due to the many students in my data modeling and database classes over the years. Interactions with my colleagues and students have shaped this book in a format especially suitable for the needs of IT professionals. xxi

24

25 I INTRODUCTION TO DATA MODELING 1

26

27 1 DATA MODELING: AN OVERVIEW CHAPTER OBJECTIVES. Introduce the process of data modeling. Present why data modeling is important. Explain how a data model represents information requirements. Describe conceptual, logical, and physical data models. Briefly discuss the steps for building a data model. Show the role of data modeling in system development. Provide an initial glimpse of data modeling history and trends James Watson and Francis Crick, working at Cambridge University, deduced the three-dimensional structure of DNA (deoxyribonucleic acid). In 1953, they published a brief paper describing their now-famous double helix model of DNA. This important milestone of creating a true model of DNA gave a tremendous boost to biology and genetics. For the discovery and creation of the double helix model, Watson and Crick shared the Nobel Prize for Physiology and Medicine in Well, what does Watson and Crick s achievement have to do with our current study? Essentially, they built a model. The model is a true representation of the structure of DNA something we find in the real world. Models are replicas or representations of particular aspects and segments of the real world. Building of models is quite common in many disciplines. When you think about it, the representation 5 þ 4 ¼ 9 is a mathematical model using symbols and logic. This model represents the fact that if you put five things together with four things of the same kind, you get nine things of the same kind. In physics, we create models to represent physical properties of the world. In economics, we create models of economic trends and forecast economic outcomes. Data Modeling Fundamentals. By Paulraj Ponniah Copyright # 2007 John Wiley & Sons, Inc. 3

28 4 CHAPTER 1 DATA MODELING: AN OVERVIEW Let us get a more vivid picture of what we mean by a model. Let us say that you are interested in buying a new home in one of the upcoming posh developments. You go to the sales office of the real estate developer. They point to a large open site where they plan to build the houses and to complete the development in 3 years. Right now, you cannot see any houses; you cannot observe the layout of roads and houses; all you can notice is a lot of vacant space and numerous trees. How can you get a picture of how the development will look in the future? How can you imagine how the house you want to buy will be structured? While you appear puzzled, the sales staff leads you into a large room. On a big table, in the middle of the room, sits a scale model of the development. They had created the model based on the requirements of the future homeowners in the community. You see roads, houses, swimming pools, tennis courts, and other amenities in the future development. These are not the real roads and houses. These are just components in the model. The model is a true representation of the real estate development. The sales people are able to point to different components in the model and communicate with you clearly and vividly. Now, you are able to understand, and you are happy. A model serves two primary purposes:. As a true representation of some aspects of the real world, a model enables clearer communication about those aspects of the real world.. A model serves as a blueprint to shape and construct the proposed structures in the real world. DATA MODEL DEFINED Data modeling is an integral part of the process of designing and developing a data system. While designing and developing a data system for an organization, you take into account all the information that would be needed to support the various business processes of the organization. If you are designing a data system for a banking institution, you have to provide data for the business processes of checking, savings, and loan account operations. If you are creating a data system for a medical center, you have to provide data for inpatient and outpatient services. You start with the analysis and gathering of details about which data elements would be needed for the business. You need to ensure that the results of requirements definition are completely implemented as the data content of the database that would support the organization. You start with planning and requirements definition. Based on these, you have to come up with the proper database system. During this process, you have to keep on communicating with the business stakeholders about the data elements, their structures, relationships among the structures, and the rules governing the structures and relationships. You have to make sure that these data elements are exactly the ones that are needed to support the business. The users must be able to understand clearly what you are designing and give their affirmation. Making the users understand the information content of the database system being built is one crucial aspect of the development process. The other significant aspect of the development process is your ability to create a database system that meets the information requirements exactly and conforms to what you have presented and described to your users. As database practitioners, what technique can we adopt to achieve these dual goals? How can

29 DATA MODEL DEFINED 5 we communicate with the users and keep them informed? How can we do what we promise to deliver and meet the information requirements exactly? What Is a Data Model? Data modeling provides a method and means for describing the real-world information requirements in a manner understandable to the stakeholders in an organization. In addition, data modeling enables the database practitioners to take these information requirements and implement these as a computer database system to support the business of the organization. So, what is a data model? A data model is a device that. helps the users or stakeholders understand clearly the database system that is being implemented based on the information requirements of an organization, and. enables the database practitioners to implement the database system exactly conforming to the information requirements. A data model, therefore, serves as a critical tool for communication with the users; it also serves as a blueprint of the database system for the developers. Figure 1-1 illustrates these two significant aspects of a data model. Notice how the data model serves the needs of the two groups: users and developers. Also notice the place of the data model between requirements definition and database system implementation. Data modeling is a technique for exploring the data structures needed to support an organization. A data model must record and indicate the content, shape, size, and rules of the data elements used throughout the scope of the various business processes of the organization. It would be a conceptual representation or replica of the data structures required in the database system. A data model focuses on what data is required and FIGURE 1-1 Data model: communication tool and database blueprint.

30 6 CHAPTER 1 DATA MODELING: AN OVERVIEW how the data should be organized. It does not necessarily reflect the operations expected to be performed on the data. Data modeling can be applied to representation of the information requirements at various levels. At the highest conceptual level, the data model is independent of any hardware or software constraints. At this level, the data model is generic; it does not vary whether you want to implement an object-relational database, a relational database, a hierarchical database, or a network database. At the next level down, a data model is a logical model relating to the particular type of database relational, hierarchical, network, and so on. This is because in each of these types, data structures are perceived differently. If you proceed further down, a data model is a physical model relating to the particular database management system (DBMS) you may use to implement the database. We will discuss these levels further. Why Data Modeling? You have understood that a data model is created as a representation of the information requirements of an organization. You have also noted that a data model functions as an effective communication tool for discussions with the users; it also serves as a blueprint for the database system. A data model, therefore, acts as a bridge from real-world information to database storing relevant data content. But, why this bridge? Why not go from real-world information to the database itself? Let us take a simple example. A business sells products to customers. We want to create a database just to support such sale transactions, nothing more. In our database, we need to keep data to support the business. In the real world of this business, data exists about customers, products, and sales of products to customers. Now, look at Figure 1-2, which shows these data elements for the business. The method for showing the information requirements as indicated in the figure is haphazard and arbitrary. If you are asked to depict the information requirements, you might do FIGURE 1-2 Sales: real-world information requirements.