Handbook of Research on Innovations in Database Technologies and Applications: Current and Future Trends

Transcription

1 Handbook of Research on Innovations in Database Technologies and Applications: Current and Future Trends Viviana E. Ferraggine Universidad Nacional del Centro de la Provincia de Buenos Aires, Argentina Jorge H. Doorn Universidad Nacional del Centro de la Provincia de Buenos Aires, Argentina Laura C. Rivero Universidad Nacional del Centro de la Provincia de Buenos Aires, Argentina Volume II Information science reference Hershey New York

2 Director of Editorial Content: Managing Editor: Assistant Managing Editor: Cover Design: Printed at: Kristin Klinger Jamie Snavely Carole Coulson Lisa Tosheff Yurchak Printing Inc. Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue, Suite 200 Hershey PA Tel: Fax: Web site: and in the United Kingdom by Information Science Reference (an imprint of IGI Global) 3 Henrietta Street Covent Garden London WC2E 8LU Tel: Fax: Web site: Copyright 2009 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Handbook of research on innovations in database technologies and applications: current and future trends / Viviana E. Ferraggine, Jorge H. Doorn, and Laura C. Rivero, editors. p. cm. Includes bibliographical references and indexes. Summary: "This book provides a wide compendium of references to topics in the field of the databases systems and applications"--provided by publisher. ISBN (hardcover) -- ISBN (ebook) 1. Database management. 2. Database design--economic aspects. 3. Technological innovations. I. Ferraggine, Viviana E., II. Doorn, Jorge H., III. Rivero, Laura C., QA76.9.D3H dc British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher. If a library purchased a print copy of this publication, please go to for information on activating the library's complimentary electronic access to this publication.

3 300 Chapter XXXIII Internet Map Services and Weather Data Maurie Caitlin Kelly Pennsylvania State University, USA Bernd J. Haupt Pennsylvania State University, USA Ryan E. Baxter Pennsylvania State University, USA INTRODUCTION Internet map services (IMSs) are redefining the ways in which people interact with geospatial information system (GIS) data. The driving forces behind this trend are the pervasiveness of GIS software and the emerging popularity of mobile devices and navigation systems utilizing GPS (Global Positioning System), as well as the ever-increasing availability of geospatial data on the Internet. These forces are also influencing the increasing need for temporal or real-time data. One trend that has become particularly promising in addressing this need is the development of IMS. IMS is changing the face of data access and creating an environment in which users can view, download, and query geospatial and real-time data into their own desktop software programs via the Internet. In this section, the authors will provide a brief description of the evolution and system architecture of an IMS, identify some common challenges related to implementing an IMS, and provide an example of how IMSs have been developed using real-time weather data from the National Digital Forecast Database (NDFD). Finally, the authors will briefly touch on some emerging trends in IMS, as well as discuss the future direction of IMS and their role in providing access to real-time data. BACKGROUND The origins of IMS can be traced to the geospatial data sharing initiatives of the 1990s when spatial data sharing began in earnest. In the early 1990s, the World Wide Web (WWW) and browsers altered users perception of the Internet. Suddenly users were able to see images and interact with the Internet through graphics and scripts and even Copyright 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.

4 access digital data. In the United States the creation of the National Spatial Data Infrastructure (NSDI) initiated an effort to develop standards for sharing data. By 1995 the vision for the NSDI to promote the sharing of data within the federal government and enhance sharing with state and local governments was solidified (Federal Geographic Data Committee, 1995). Spatial data, which is defined as any data containing a locational component, have become widely available to the public through efforts of spatial data clearinghouses, governmental Web sites, and even nonprofit organizations, many of which were early NSDI initiatives. Through these spatial data clearinghouses, static spatial data were accessed traditionally via file transfer protocol (FTP). In cases such as these, the onus was on the user or client to identify, download, and manipulate data to comply with the desktop GIS environment. Temporal and real-time data have a more complex history. This type of information, which includes such diverse data types as traffic, hydrologic, and weather data, is difficult to maintain and provide access to due to its dynamic nature. For example, weather data, which encompasses everything from radar to precipitation to wind speed, are changing continuously and present the user with numerous data formats and types with which to contend (Van der Wel, Peridigao, Pawel, Barszczynska, & Kubacka, 2004). The past few years have seen advances on this front. For example, the National Weather Service (NWS) of the U.S. National Oceanic and Atmospheric Administration (NOAA) has developed several online applications that allow viewing of real-time weather information in a Web-based GIS environment. In addition, Figure 1. Diagram of early IMS architecture some data sets are being provided in a format that can be integrated into desktop GIS programs and tools have been created to convert the format and automate update processes within these programs (Liknes, Hugg, Sun, Cullen, & Reese, 2000). However, these tools do not allow for seamless integration of remote real-time databases into the desktop environment, an issue that the emergence of IMS has addressed. IMS initiatives began in the mid 1990s. As the amount and availability of spatial and temporal data increased, so did the desire for increased interactive capabilities. Initially faced with the same historical constraints as in the past large file size, long download times, and outdated data data sites, programmers, and software companies began working toward providing users with the capability to bring data to their desktops without downloading the data: GIS over the Web became the new trend (Gold, 2006). The early steps toward visualization and subsequent desktop utilization of data via the Internet were basic. These applications were comprised of a set of data, predefined by the developer, which users could view via an interactive map interface that was built within a standard HTML (hypertext markup language) page. Analysis and navigation functionality, symbology, and compatibility with client browser software were controlled by the developer and server architecture (Figure 1). In many cases, the initial applications were environmental in nature, focusing on watershed health or water quality in a specific geographic area (Abel, Taylor, Ackland, & Hungerford, 1998). However, there were few if any customization capabilities and limited download capabilities in the first few years of the IMS movement. The user was simply viewing the data and turning data layers on and off. Another component of early IMS development was the use of a map as part of a search engine to access data (Kraak, 2004). As groundbreaking as these Web GIS applications were, they still had limited utility for those who wanted to acquire data and utilize them on their own desktops or who desired more than predefined static data sets. 301

5 The next step in the evolution of this field would be the development of the capability for users to access exactly the data they needed without ever having to download a single file. Figure 2. Diagram of the architecture for a map service Evolution of IMS from Web GIS application to Map Service There are several different architecture types and performance enhancing capabilities that have emerged to support spatial data distribution and Web-based access over the past few years. The specific development and deployment architecture from the type of software, database, and hardware used are essentially a reflection of the internal capabilities of the developers and their organization. However, several major developments have impacted the performance of IMS and have enabled increased functionality and integration of data. From the basic interface of the early Web GIS applications have grown a host of new tools and capabilities that have made data not only viewable but also intrinsically more usable. The evolution of these basic interfaces and capabilities has occurred for a number of reasons. As stated earlier in this article, the Internet itself is an ever-changing landscape in which developers are consistently pushing the technology envelope and users are expecting bigger, better, and faster services and resources. Second, existing Web-based GIS services such as those provided by spatial data clearinghouses or services through sites such as Google Earth have created an environment in which people are more spatially aware. With this awareness comes the expectation of greater access to more relevant and timely data. Third is the extension of the traditional database model to include spatial data. Databases such as Oracle were most commonly utilized to store and access functional or business-related data. However, in the past decade, these databases have come to incorporate spatial and dynamic data and serve as the backbone for the development of access tools and IMS. Finally, there was the evolution of the IMS from a simple Web map that provided only 302 a static image of a data set to services that allow interaction and querying data. In the earliest incarnations of IMS development, data were housed within the application environment stored predominantly in a flat file structure. Commonly, data were stored on a server and accessed by the Internet mapping server directly. This architecture was adequate for data of relatively small sizes (kilobytes and megabytes). However, file-based data retrieval imposed constraints on performance once individual files and collections grew to sizes in the gigabytes and terabytes. Today, with the exponential growth in availability of detailed vector data (parcels, transportation networks, etc.) and especially high-resolution aerial photography, file sizes have rendered file-based storage impractical and prohibitively slow. Advancements in database technology, for example, the Environmental Systems Research Institute (ESRI) Spatial Database Engine (SDE), which manages data within the data tables in a relational database management system (RDBMS), have enabled traditional databases, such as Oracle and DB2, to house large quantities of both raster and vector spatial data. Also, database vendors themselves have developed spatial data models to extend their products, such as Oracle Spatial and DB2 Spatial Extender. Techniques such as the use of pyramids and new indexing procedures to manage and retrieve data within an RDBMS or SDE environment have also enabled substantially increased performance and speed of IMS (Chaowei, Wong, Ruixin, Menas, & Qi, 2005). In addition, the use of object-oriented

6 databases (OODBMS), which combine both data and function, are on the rise (Alam & Wasan, 2006). The potential impact of OODBMS is still unknown since it is found most commonly within the commercial or business environment, but as IMS becomes more reliant on database technologies, it is possible that OODBMS will increase the functionality of IMS. By taking advantage of the data retrieval performance and efficiency of databases, Internet mapping servers are able to deploy applications and services containing terabytes of spatial data within acceptable time frames. In addition, as database management has become more mainstreamed, smaller organizations have developed the capability to build higher functioning IMSs. An example of this type of database is MySQL, which is a more user-friendly and affordable open-source database that utilizes the structured query language (SQL). Building on the advances in databases and software, the next challenge was to bring the data from the browser environment onto the desktop without requiring that users download the source data files to their own computers. There are several types of IMSs that have emerged to fill this need. The Open Geospatial Consortium, which is a nonprofit, international organization, has created several types of standards for IMS development including Web mapping services (WMSs), Web feature services (WFSs), and Web coverage serfigure 3. Example of the integration of remote temporal data, in this instance NDFD weather data for the continental United States (CONUS), from the NWS/NOAA vices (WCSs). In addition, the two most common types of services are the feature service and image service. The image service allows the user to view a snapshot or picture of the data via a standard image format, such as JPEG or PNG. This type of service is particularly meaningful for those utilizing raster data, aerial photography, or other static data sets. The feature service is the more intelligent of the two as it streams raw spatial geometries (points, lines, polygons) to the client and allows the user to determine the functionality and symbology of the data. IMS is a particularly effective way of serving temporal real-time data since it allows for real-time updates of source data on the server side and customization on the client side. Users can access services through their desktop GIS software with no need to utilize a browser or use a search engine. IMS and the Challenge of Weather Data Real-time or temporal data by nature present numerous challenges to the IMS developer and database administrator as they bring with them the aspect of integrating the dimension of time. Weather or climate data are one of the most complex data types in the temporal data environment. In considering the development and deployment of a weather-based IMS, the unique data format, frequency of updates, and numerous data types, that is, those taken as surface points as well as those taken by satellite or radar must be considered (Liknes et al., 2000). However, the significance of this data in areas such as emergency management cannot be underestimated. Just as important is the timeliness and speed with which the information is updated. There are numerous examples of the use of IMS to facilitate the sharing and display of time-sensitive data. These range from traffic management to earthquake early-warning systems and damage distribution maps (Erdik, Fahjan, Ozel, Alcik, Mert, & Gul, 2003). Climate and weather data pose their own unique problems and often present a heavy burden for those involved in developing 303

7 and serving temporal data (Shipley, Graffman, 7 Ingram, 2000). Working with weather data proves to be difficult for several reasons. First there is the vast amount of output produced by weather stations and apparatus and high-resolution large-scale and regional climate forecast models that produce frequent output in different data formats (such as NetCDF); furthermore, there is the issue of timely data availability and at times unknown data quality. In addition, short-lived climate events like hurricanes and tornados are particularly challenging. The reasons for this are not related to the amount of data, data type, or the frequency of recurrent data updates. It is the temporary nature of the data availability and the immediacy of the event that impact the IMS developer when an extreme climate event occurs. Finally, the types of weather data vary. There are point data that include surface observations like lightning strikes and tornados, wind speed and gusts, and radar such as NEXRAD (Next-Generation Radar). Each type of data can require its own specific manipulation to prepare it for integration in a spatial data environment. In the example illustrated by Figure 3, regularly updated weather data are automatically downloaded from given Internet FTP locations at predetermined times. The downloaded data are checked for completeness. Once the files are stored on a local server, they are degribbed (unpacked). The GRIB2 data files (e.g., temperature, apparent temperature, minimum and maximum temperature, probability of precipitation, precipitation, snow amount, dew point temperature, relative humidity, weather features, sky cover, wave height, wind speed and direction, wind gusts, etc.) contain the actual weather data in shapefile format. Each data set includes metadata and information about each available data layer for each forecasted time (up to 40 possible layers for up to 7 days). The data are then automatically uploaded into a relational spatial database, where they reside until new data are downloaded from the NWS/NOAA. In addition, map services are defined from updated map configuration files (AXL files) in order to provide the user with a map service that shows date and time. 304 The level of effort in accessing and incorporating temporal data into an IMS environment is difficult to determine by the simplified explanation provided here. Suffice it to say that the actual process of integrating this data requires approximately 5,000 lines of code to ensure that the data downloaded and errors resulting from interrupted dataflow during the data download are reported, and to ensure data availability and completeness, data preparation, automated database upload, that AXL file modifications and updated IMS services work flawlessly, and that possible foreseeable errors are covered by backup error procedures. FUTURE TRENDS There are numerous potential influences on spatial and temporal data and IMS development. Object-oriented database technologies are poised to challenge the relational database approach and influence the spatial database realm. As suggested previously, OODBMS has been utilized in business environments but have applicability in the spatial environment as well. In addition, significant work has been done with XML and the geography markup language (GML) that could influence the development of IMS. The creation of the NetCDF markup language or NcML could have a significant impact on weather and geoscience-related data exchange. Societal changes in the way people perceive spatial information and utilize spatially based services to access real-time data will increase the need for faster and more effective IMSs. Most recently, Google developed Google Earth that allows users to find imagery satellite or aerial photography of almost any area on earth via the Internet. As the technology that supports IMS improves, the integration of temporal data will increase. Database management software will provide the framework for managing large data collections and supporting the processing of temporal data. Performance factors have improved dramatically in a short period and will continue to do so until issues of loading time become obsolete. Emerging trends such as

8 image and feature services and the Open GIS efforts to develop advanced open-source WMS and WFS will allow the users greater flexibility and customization options and eventually the line between remotely stored and served data and the desktop will disappear. These improvements and the increased focus on emergency management and climate and weather information will place IMS in the forefront of providing support and information to stakeholders in need of data. response and early warning system. Bulletin of Earthquake Engineering, 1(1), CONCLUSION Liknes, G., Hugg, R., Sun, J., Cullen, W., & Reese, C. (2000, June). Techniques for conversion of weather data into ESRI formats. Proceedings of the ESRI International Users Conference, San Diego, CA. In conclusion, advances in serving data via the Internet, specifically Internet map services and Web-based GIS, are meeting the challenge of temporal data dissemination. In a period of only a decade, spatial and temporal data have moved from the realm of FTP to becoming fully integrated into the desktops of users in every walk of life. Spatial and temporal data that had heretofore required download time, manipulation, and consistent monitoring can now be viewed via a browser or imported into your desktop software with a click of a button. REFERENCES Abel, D. J., Taylor, K., Ackland, R., & Hungerford, S. (1998). An exploration of GIS architectures for Internet environments. Computers, Environment, and Urban Systems, 22(1), Alam, M., & Wasan, S. K. (2006). Migration from relational database to object oriented database. Journal of Computer Science, 2(10), Chaowei, P. Y., Wong, D., Ruixin, Y., Menas, K., & Qi, L. (2005). Performance-improving techniques in Web-based GIS. International Journal of Geographical Information Science, 19(3), Erdik, M., Fahjan, Y., Ozel, O., Alcik, H., Mert, A., & Gul, M. (2003). Istanbul earthquake rapid Federal Geographic Data Committee. (1995). Development of a national digital geospatial data framework. Washington, DC: Author. Gold, C. M. (2006). What is GIS and what is not? Transactions in GIS, 10(4), Kraak, M. J. (2004). The role of the map in a Web-GIS environment. Journal of Geographical Systems, 6(2), Shipley, S. T., Graffman, I. A., & Ingram, J. K. (2000, June). GIS applications in climate and meteorology. Proceedings of the ESRI User Conference, San Diego, CA. Van der Wel, F., Peridigao, A., Pawel, M., Barszczynska, M., & Kubacka, D. (2004, June). COST 719: Interoperability and integration issues of GIS data in climatology and meteorology. Proceedings of the 10th EC GI & GIS Workshop: ESDI State of the Art, Warsaw, Poland. KEY TERMS Degrib: NDFD GRIB2 Decoder is the starting point for decoding the NDFD and various World Meteorological Organization (WMO) GRIB2 files. A degribbed file is a packed binary file or archive (GRIB2) like any other compressed archive; for more information see weather.gov/ndfd. Feature Service: A feature service is a type of map service that delivers raw geometry features via the Internet to clients through desktop software or applications. 305

9 Image Service: An image service is a type of map service that generates a snapshot view of the spatial data requested by a client. The snapshot is stored in a standard image file, such as a JPEG, which is subsequently delivered to the client. the interface. The NetCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data that include metadata. Map Service: A map service is a special type of Web-based service that allows spatial information, that is, maps, to be accessed remotely over the Internet and displayed and manipulated in a client software program. NEXRAD: Next-Generation Radar is an NWS network of about 160 Doppler weather surveillance radars (WSRs) operating nationwide. NDFD: The National Digital Forecast Database is a database put together by the National Weather Service (NWS) to provide forecasts of sensible weather elements (e.g., cloud cover, maximum temperature) on a seamless grid. The NDFD data sets are currently given out to the public as GRIB2 files. NetCDF: Network Common Data Form is an interface for array-oriented data access and a library that provides an implementation of 306 Open GIS: Open GIS is the full integration of geospatial data into mainstream information technology by defining a standard syntax for requesting IMS. Particular formats include Web map services (WMSs), which are types of image services, and Web feature services (WFSs), which are types of feature services. SDE: Spatial Database Engine is a software product from Environmental Systems Research Institute (ESRI) that acts as a middleware between the database and applications allowing GIS data to be stored in and retrieved from a relational database, such as Oracle or DB2.