INTERCONNECTING HETEROGENEOUS INFORMATION SYSTEMS

Size: px

Start display at page:

Download "INTERCONNECTING HETEROGENEOUS INFORMATION SYSTEMS"

Annice Evans
6 years ago
Views:

1 INTERCONNECTING HETEROGENEOUS INFORMATION SYSTEMS

2 The Kluwer International Series on ADVANCES IN DATABASE SYSTEMS Series Editor Ahmed K. Elmagarmid Purdue University West Lafayette, IN Other books in the Series: DATABASE CONCURRENCY CONTROL: Methods, Performance, and Analysis, Alexander Thomasian, ISBN: X TIME-CONSTRAINED TRANSACTION MANAGEMENT: Real-Time Constraints in Database Transaction Systems, Nandit R. Soparkar, Henry F. Korth, Abraham Silberschatz, ISBN: SEARCHING MULTIMEDIA DATABASES BY CONTENT, Christos Faloutsos, ISBN: REPLICATION TECHNIQUES IN DISTRIBUTED SYSTEMS, Abde/salam A. He/ai, Abdelsalam A. Heddaya, Bharat B. Bhargava, ISBN: VIDEO DATABASE SYSTEMS: Issues, Products, and Applications, Ahmed K. Elmagarmid, Haitao Jiang, Abdelsalam A. Helal, Anupam Joshi, Magdy Ahmed, ISBN: DATABASE ISSUES IN GEOGRAPHIC INFORMATION SYSTEMS, Nabil R. Adam and Aryya Gangopadhyay, ISBN: INDEX DATA STRUCTURES IN OBJECT-ORIENTED DATABASES, Thomas A. Mueck and Martin L. Polaschek, ISBN: INDEXING TECHNIQUES FOR ADVANCED DATABASE SYSTEMS, byelisa Bertino, Beng Chin Ooi, Ron Sacks-Davis, Kian-Lee Tan, Justin Zobel, Boris Shidlovsky and Barbara Catania ISBN: MINING VERY LARGE DATABASES WITH PARALLEL PROCESSING, Alex A. Freitas andsimon H Lavington, ISBN: DATA MANAGEMENT FOR MOBILE COMPUTING, Evaggelia Pitoura and George Samaras, ISBN: PARALLEL, OBJECT-ORIENTED, AND ACTIVE KNOWLEDGE BASE SYSTEMS, Joannis Vlahavas andnick Bassiliades, ISBN: DATABASE RECOVERY, Vijay Kumar and Sang H Son, ISBN: FOUNDATIONS OF KNOWLEDGE SYSTEMS: With Applications to Databases and Agents, Gerd Wagner, ISBN:

3 INTERCONNECTING HETEROGENEOUS INFORMA TION SYSTEMS by Athman Bouguettaya Queensland University oftechnology Brisbane, AUSTRALIA Boualem Benatallah Queensland University oftechnology Brisbane, AUSTRALIA Ahmed Elmagarmid Purdue University Lafayette, IN USA... " SPRINGER SCIENCE+BUSINESS MEDIA, LLC

4 ISBN DOI / ISBN (ebook) Library of Congress Cataloging-in-Publication Data A C.I.P. Catalogue record for this book is available from the Library of Congress. Copyright 1998 by Springer Science+Business Media New York Originally published by Kluwer Academic Publishers in 1998 Softcover reprint ofthe hardcover Ist edition 1998 AII rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher, Springer Science+Business Media, LLC Printed on acid-free paper.

5 Contents List of Figures List oftables Preface Acknowledgments Foreword xi xiii xv xix xxi 1. INTRODUCTION 2. MODELING ARCHITECTURE Terminology Global Schema Integration Federated Database Systems Loosely Coupled FOBS Tightly Coupled FOBS Multidatabase Language Approach CRITICAL ASPECTS OF INTEROPERABILITY Schema and Language Translation Schema Integration Schema and Data Heterogeneity 25 Types of Discrepancies 27 Examples of Semantic Conflict Resolution 30 Tools to Aid Conflicts Resolution Schema Integration Process Query Processing Query Processing in Centralized DBMS Query Processing in Distributed Database Systems Query Processing in Heterogeneous Database Systems 38 Major Issues in Multidatabase Query Optimization 38 Generic Architecture of a Multidatabase Query Processor 39 Proposed Solutions for Query Optimization Transactions Quasi-serializability Hierarchical Concurrency Control Value Dates 51

6 vi INTERCONNECTING HETEROGENEOUS INFORMATION SYSTEMS Extended Transaction Models RESEARCH AND COMMERCIAL PRODUCTS DAPlEX Data Definition Data Manipulation Higher level Capabilities Databases Integration IRIS Objects and Types Functions Pegasus Architecture Common Data Model Heterogeneous Object SQl Query Processing Translation and Integration Schema Integration Comandos Integration System System Architecture Common Data Model Translation and Integration Query Processing Federated Objectbase System System Architecture Common Data Model Translation and Integration Myriad Architecture Query Processing Federated Query Managers and Agents Query Optimization Transaction Management Translation and Integration Multibase System Architecture Common Data Model, Translation, and Integration Query Processing Mermaid System Architecture Common Data Model and Data Translation Query Processing SIMS Project Knowledge Representation or Modeling Selecting the Right Databases Access Planning Global Query Optimization InterBase 79

7 Contents VII 4.11 OMNIBASE Other Systems MIDDLEWARE CORBA The Common Object Request Broker Architecture 89 Static Invocation 90 Dynamic Invocation ORBs Interoperability Examples of CORBA Commercial ORBs OLE/COM Component Object Model (COM) OLE OLE/DB and ActiveX DCE OpenDoc ANSAware Other Relevant Middleware Technologies Java Technology HTMLlCGI Technology Message-Oriented Middleware and TP monitors Distributed Object Manager System Arjuna Conclusion RESEARCH TRENDS Information Sharing on the Web Distributed Naming Systems Information Retrieval 112 Network-based Information Retrieval 113 Internet Information Gathering 113 Multi-Agent Information Retrieval WWW Search Tools 114 Robots 115 Meta-Robots 116 Database-like WWW Languages Information Brokering Techniques Summary Transactional Workflows 121 Definition Background and Motivation Features of Workflow Systems 123 Types of Workflow Systems 123 Characteristics of a Workflow Management Systems 124 Workflow Management Coalition (WfMC) Modeling Techniques and Prototypes 126 Advanced Transaction Management (ATM) 126 Petri-net 127

8 viii INTERCONNECI1NG HETEROGENEOUS INFORMATION SYSTEMS Goal-Oriented Workflow Analysis and Infrastructure 127 Mentor 127 METEOR2 128 CBORD 128 WASA 129 ORCHESTRA 129 Other Techniques Limitations of Existing Systems Research Efforts Summary Advances in Data Warehousing Background and Motivation 134 Data Mining and Knowledge Discovery 135 On-Line Analysis and Processing 136 Architectural Requirements Basic Architectures Warehouse Consistency and View Maintenance 139 Levels of Consistency 139 Maintaining Data Consistency 140 View Maintenance for Warehouse 140 Concurrency Control Problem Warehouse Design Issues 143 Techniques for the View-Selection Problem Summary CASE STUDY 1: WEBFINDIT PROTOTYPE Introduction Design Overview Core Concepts 149 Coalitions 150 Service Links Documentation Co-databases WebTassili Language Rationale WebTassili Salient Features Evolution Features 160 Rules for Joining and Leaving WebFINDIT 160 Coalition Formation and Update 161 Service Formation and Update Implementing WebFINDIT Overview Browser Query Processor Co-database Server Broker Information Source Interface Scalability Evaluation Related Work 176

9 Contents ix 7.7 Conclusion CASE STUDY 2: A TRANSACTION-ORIENTED WORKFLOW ENVIRONMENT TOWE Architecture Process Model Scheduling Processes The Class Library Process Definition The Scheduler Class Implementation of Process Objects Recovery Implementation Conclusion 193 References 195 Index 213

10 List of Figures The Global schema approach. Federated approach. Language-based integration approach. Multidatabase query processor architecture. Pegasus functional architecture. Architecture ofcis. FBASE federation architecture. Myriad functional layers. Myriad schema architecture. Multibase schema architecture. Run time query processing subsystem. Mermaid schema architecture. Mermaid processes. OMG reference model. CORBA framework. Invocation ofa remote method. Location and activation ofa COM object. OLE technology. Architecture of DCE. DCERPC. OpenDOC technology. ANSAware system. Common characteristics of a workflow system. Workflow reference model. Basic architecture of data warehousing systems. Coalitions and service links. Example ofcoalitions and service links. WebFINDIT implementation architecture. Outline ofa typical co-database schema. Top level view of long-lived activity. Workflow view of a mortgage loan

11 XlI INTERCONNECfING HETEROGENEOUS INFORMATION SYSTEMS Activity tree for a home loan application. Library classes ofthe TOWE. Transfer process object. Finance process object Find_Credit_Union process object. OpenJIome-Loan-Account process object. TOWE system architecture

12 List of Tables Multidatabase functions, CDM and GAl. Translation, integration, and transaction management issues xiii

13 Preface Information systems are the backbone of many oftoday's computerized applications. Distributed databases and the infrastructure needed to support them have been well studied. However, this book is the first to address distributed database interoperability by examining the successes and failures, various approaches, infrastructures, and trends ofthe field. A gap exists in the way that these systems have been investigated among the real practitioners. This gap is more pronounced than usual partly because of the way businesses operate, the systems they have, and the difficulties created by systems' autonomy and heterogeneity. Telecommunications firms, for example, must deal with an increased demand for automation while at the same time continuing to function at their current level. While academics are focusing on investigating differences between distributed databases, federated databases, heterogeneous databases, and more general1y, among loosely connected and tightly coupled systems, those who have to deal with real problems right away know that the only relevant research is the one that wil\ ensure that their system work to produce reasonably correct results. This book covers the underlying principles and infrastructures needed to realize truly global information systems. The book discusses technologies related to Middleware, the Web, Workflows, Transactions, and Data Warehousing. It also overviews architectures with a discussion ofcritical issues. The book also gives an overview of systems that can be viewed as learning platforms. While these systems do not translate to successful commercial realities, they push the envelope in terms of research. Successful commercial systems have benefited from the experiments conducted in these prototypes. The book includes two case studies based on the authors' own work. The book is divided into eight chapters. The first chapter sets the scene in the sense that it provides a high level overview ofthe issues covered in the remaining chapters. The second chapter defines the terminology and summarizes more than two decades ofresearch. In the third chapter, we cover four important issues and expound on them. These are Schema and Language Translation, Schema Integration, Query Processing, and Transaction Processing. These four topics have been the focus of much study by academics for 20 years. In the fourth chapter, detailed coverage ofthe lessons learned from previous research projects conducted at industry and universities is presented.

14 XVI INTERCONNECTING HETEROGENEOUS INFORMATION SYSTEMS Chapter 5 looks at the standardization efforts for distributed computing and how these can be harnessed to support interoperable environments. Chapter 6 covers issues of particular importance, an understanding of which is needed in order to realize these systems in practice. These include Web interfaces and support, workflows, and data warehousing. Finally, in chapters 7 and 8, two case studies are presented: one examines WebFINDIT, a Web-based version of FINDIT. FINDIT is an earlier system built at the University ofcolorado. The other case study looks at TOWE, a system developed at the Queensland University oftechnology (Australia), which describes an approach based on a transactional workflow system. In these pages, we attempt to summarize and bring to attention the lessons learned over decades of research. It is only by canvassing past attempts and strategies that we can objectively harness the successes to rise to the challenge of providing truly universal interoperable information systems. The authors have played a role in the creation ofthis story and want to be part ofthe telling ofit. ATHMAN BOUGUETTAYA BOUALEM BENATALLAH AHMED ELMAGARMID

15 In memory ofmy brother, friend, and classmate Abdelhak. Athman Bouguettaya In memory ofmy parents, Kheira and Mohamed. Boualem Benatallah To my mentor, teacher and bestfriendforover 25 years, Kenneth C. Schraut, Professor Emeritus of Mathematics ofthe University ofdayton. Ahmed Elmagarmid

16 Acknowledgments Many people helped tremendously. We want to thank Roger King and Michael Novak from the University of Colorado (USA) for the early discussion on the book structure and for providing the first author with the computing facilities while on sabbatical leave. We also wish to thank the graduate students of the 1997 and 1998 classes from the Queensland University of Technology who helped in one way or another towards this book. In particular, we would like to thank Atma Beeharry, Mourad Ouzzani, Ong Cheng Keat, Christine Fong, Hien Lieu, Lily Hendra, Chee Seng, Kevin Kucks, and Manuel Dos Santos. Our special thanks go to our colleagues David Edmond and Arthur ter Hofstede for helping with the proofreading. We would also like to thank Mike Papazoglou (Tilburg Univ.) and Alex Delis (Polytechnic Univ.) and Mostafa Haghjoo (Tehran Univ.) for their contribution to Chapter 8. Athman Bouguettaya would like to thank his wife, Urn Zakaria, for her support and patience. He would also like to apologize to his small children, Zakaria. Ayoub, and Mohamed Islam for missing their father during the preparation of this manuscript. He would also like to thank his parents, brothers and sisters for all their love and support. Finally, we are grateful to Allah for His guidance and for giving us the strength to finish this project.

17 Foreword The emergence of a true global economy has highlighted one of the most critical problems facing the database research and development community - the interoperation of heterogeneous database systems. Individual corporations have data in highly dispersed locations. Corporations must also be able to share data with governments, with libraries and universities, and with each other. This demand for effective tools to interconnect databases in a meaningful way is largely unmet, even though researchers have been working in the area for a couple of decades. The problem is particularly difficult for a number of reasons, including the highly complex and diverse nature of database products, the fact that detailed application semantics are embedded in database schemas, and the need to gracefully evolve database interconnections. The authors ofthis book have taken a close look at the existing research literature and provide an intuitive overview ofpublished results in this area. They also describe a number of prototype heterogeneous database systems, as well as several emerging standards for specifying the location-independent management of data. This is an excellent book for a specialized undergraduate or graduate course on distributed databases. Chapters 2 and 3 give a solid overview ofthe problem domain, and chapter 6 gives a discussion of likely future trends in heterogeneous data management; these three Chapters would be useful as a supplementary text in an introductory class. The book also gives application specialists an understanding of the current state of heterogeneous database technology, as well as a solid idea of where this critical area ofsoftware development is headed. Roger King, Professor University of Colorado

PARALLEL, OBJECT -ORIENTED, AND ACTIVE KNOWLEDGE BASE SYSTEMS

PARALLEL, OBJECT -ORIENTED, AND ACTIVE KNOWLEDGE BASE SYSTEMS The Kluwer International Series on ADVANCES IN DATABASE SYSTEMS Series Editor Ahmed K. Elmagarmid Purdue University West Lafayette, IN 47907