Web Data Management: An Introduction to Semistructured and XML Databases

Transcription

1 Web Data Management: An Introduction to Semistructured and XML Databases V. CHRISTOPHIDES I FUNDULAKI Department of Computer Science University of Crete ICS - FORTH, Heraklion, Crete 1 A bit of History Research: Internet industry: 1950 s: Lisp [Mac Carthy] 1957: Sputnik launches ARPA 1960 s: Tree languages [Buchi] 1972: First demonstration of ARPANET 1970 s: Relational DBs [Codd] 1989: Number of hosts breaks 100, : Graphlog [Univ. Toronto] 1991: CERN releases the World Wide Web HTML as the support for information 1994: O2 extensions [INRIA] 1997: 20 Million Hosts,1 Million Web sites 1995: Tsimmis & OEM [Stanford] 1998 : W3C releases XML to represent 1995: UnQL [UPenn] information on the Web Need to handle irregular Web data. Use graph data models.? XML provides a syntax for irregular textual Web information. 2 1

2 Documents vs Databases Document world Database world plenty of small documents usually static implicit structure section, paragraph, toc, tagging human friendly content form/layout, annotation paradigms Save as, WYSIWYG metadata author name, date, subject a few large databases usually dynamic explicit structure types records machine friendly content data, methods paradigms Data Independence, Transaction Management, Query Languages metadata schema description 3 What to do with them Documents Database editing spell-checking updating cleaning counting words retrieving (IR) printing querying composing/transforming 4 2

3 Query Languages Document Retrieval Claude Monet and San Diego Museum of Art Database Querying select p from Artists a, a.artwork p where a.first = Claude and a.last = Monet and p.located = San Diego Museum of Art 5 How the Web is Today? Information and its presentations are mixed up in the form of HTML documents all intended for human consumption many generated automatically by applications Easy to fetch any Web page, from any server, any platform access through a uniform interface 6 3

4 The Secrets of HTML Success Everybody can write it: HTML is simple HTML is textual: it is human readable, you can use any editor,... Hypertext Links Everybody can read it: HTML is portable on any platform The browser is the universal application Everybody can search it: Keyword-based Search Engines: high recall, low precision It connects pieces of information together Through hypertext links 7 What s Wrong with HTML If written properly, normal HTML markup may reflect document presentation, but it cannot adequately represent the semantics & structure of data Artist Name Artifact Title <B>MONET, Claude<B><BR> Date Haystacks at Chailly at Sunrise<BR> 1865<BR> Dimensions Oil on canvas<br> Material 30 x 60 cm (11 7/8 x 23 3/4 in.)<br> San Diego Museum of Art <BR> Image <P> Reference Museum <IMG SRC= m/monet/hayricks.jpg > 8 4

5 HTML Document Presentation 9 But Modern Web Applications Need More! Infomediaries: Community Web Portals Digital Museums & Libraries Electronic commerce: On-line Catalogs & Procurement Comparison Shoppers Market Places Virtual Enterprises Scientific applications: E-learning Data & Knowledge Grids Advanced Information Management finding, extracting, representing, interpreting, maintaining Flexible, Quick Interoperation: the ability to uniformly share, interpret and manipulate heterogeneous information applications cannot consume HTML More than HTML documents: Data on the Web More than Web browsers: Web-enabled Applications 10 5

6 Paradigm Shift on the Web New Web standard XML: XML generated by applications XML consumed by applications Data exchange: across platforms across organizations XML Data application application object-relational Integrate WEB (HTTP) Transform Warehouse Web: from collection of documents to Web data published as documents relational data application legacy data 11 XML Data Representation: The Document View <ARTIST> Element Name Element Content <NAME> <FIRST>Claude</FIRST> <LAST>Monet</LAST> </NAME> <ARTWORK> <ARTIFACT> <TITLE>Haystacks at Chailly at Sunrise</TITLE> Attribute Name <DATE>1865</DATE> <MATERIAL>Oil on canvas</material> <DIM Metric= cm > <HEIGHT>30</HEIGHT><WIDTH>60</WIDTH></DIM> <DIM Metric= in > <HEIGHT>11 7/8</HEIGHT><WIDTH>23 3/4</WIDTH></DIM> <LOCATION>San Diego Museum of Art</LOCATION> <IMAGE File= /> </ARTIFACT> </ARTWORK> </ARTIST> Empty Element Attribute Value 12 6

7 XML Data Representation: The Database View ARTIST NAME ARTWORK FIRST LAST ARTIFACT Claude MONET TITLE DATE Haystacks 1865 Oil on canvas DIM DIM IMAGE...hayricks.jpg H W H W /8 3/4 MATERIAL LOCATION San Diego Mus. 13 The Secrets of XML Popularity It looks like HTML... Simple, familiar, easy to learn, human-readable Universal and portable Supported by the W3C: trusted and quickly adopted by the industry but it s more than HTML! flexible: you can represent any information extensible: you can represent it the way you want! Increasing precision in XML specifications Well-Formed: already better than plain text Valid: Structure conforms to a DTD or an XML Schema <?XML!> 14 7

8 Well-Formed XML An object is said to be a well-formed XML document if it meets all the well-formedness constraints (WFCs) of the XML syntax: tags (etc.) are syntactically correct every tag has an end-tag tags are properly nested there exists a root By definition if a document is not well-formed, it is not XML This means that there is no an XML document which is not wellformed, and XML processors are not required to do anything with such documents 15 Valid XML A well-formed document is valid only if it contains a proper DTD (or Schema) and if the document obeys the constraints of that DTD (or Schema) and therefore the XML Validity Constraints (VCs) only declared tags (element or attribute names) are used all tag occurrences conform to specified content models Examples: The following XML Document is well-formed but not valid <ARTIST>Claude Monet</ARTIST> The following XML Document is not even well-formed <FIRST>Claude</FIRST><LAST>Monet</LAST> 16 8

9 XML Document Type Definition (DTD) <!DOCTYPE artist [ <!ELEMENT artist (name, born, death, artwork, nationality?, influences)> <!ATTLIST artist oid ID #REQUIRED> <!ELEMENT name (first, last)> <!ELEMENT first (#PCDATA)> <!ELEMENT last (#PCDATA)>... <!ELEMENT artwork (artifact+)> <!ELEMENT artifact (title, date, material, dim*, location, image)> <!ELEMENT title (#PCDATA)>... <!ELEMENT dim (height, width)> <!ATTLIST dim metric (cm in) cm > <!ELEMENT location (#PCDATA)> <!ELEMENT image EMPTY> <!ATTLIST image file ENTITY #REQUIRED> <!ELEMENT influences (PCDATA aref)*> <!ELEMENT aref EMPTY> <!ATTLIST aref oref IDREF #IMPLIED> ]> 17 XML Anatomy 18 9

10 Is XML the Solution to Interoperability? Document = medium for exchanging information Still need to agree on: DTDs or Schemas Meaning of tags Operations on data Meaning of operations ARTIST ARTIST NAME ARTWORK NAME ARTWORK FIRST LAST ClaudeMONET TITLE DATE Hayst1865 acks MATERIAL ARTIFACT DIM Oil on canvas DIM IMAGE H W H W hayricks.jpg /83/4 LOCATION San Diego Mus. Communication FIRST LAST ClaudeMONET TITLE DATE Hayst1865 acks MATERIAL ARTIFACT Application 1 Application 2 DIM Oil on canvas DIM IMAGE H W H W hayricks.jpg /83/4 LOCATION San Diego Mus. 19 Large Scale Interoperation on the Web? Communication Partner using DTD B?? XML-based Communication using DTD A Communication Partner using DTD C Sender using DTD A Recipient using DTD A 20 10

11 Interoperability is still an Open Issue! Semantic discrepancies : Synonymy & Polysemy & Taxonomy <ARTIFACT> vs. <ARTEFACT> is <ARTWORK> paintings or songs? how < Style= Impressionism > is related to < Style= Pointillism >? Structural discrepancies : Aggregation <NAME><FIRST>Claude</FIRST><LAST>Monet</LAST></NAME> vs<name>claude Monet</NAME> Type <ARTIFACT Kind= Painting >... </ARTIFACT> vs<painting>claude Monet</PAINTING> Syntactic discrepancies : <ARTIST Name= Claude Monet >... </ARTIST> vs<artist> <NAME>Claude Monet</NAME>... </ARTIST> More than Web Data: Semantics on the Web More than Web Applications: Web Services 21 The Semantic Web Vision: A Web of Meaning The Next Generation Web aims to provide infrastructure for expressing information in a precise, human-readable, and machineinterpretable form Enable both syntactic and semantic/ structural interoperability among independently-developed Web applications, allowing them to efficiently perform sophisticated tasks for humans Enable Web resources (data & applications) to be accessible by their meaning rather than by keywords and syntactic forms Conceptual Navigation & Querying Inference Services (Picasso is an Artist) Techniques Artists Semantic Relationships Artifacts Museums 22 11

12 Web Innovation: From Web Sites to Web Services Packets Connect TCP/IP Browse/ Present Integrate/ Web Sites Today Collaborate Transact Program/ Web Services HTML XML UDDI, SOAP, RosettaNet 1970s 1980s 1990s Middleware Evolution & Interoperability 24 12

13 About the W3C and the XML Activities Membership organization Core XML WG Different types of groups inside W3C: extensible Markup Language (XML 1.0), namespaces, Infoset Working groups XML Linking WG Interest groups XML Pointer Language (Xpointer), Coordination groups XML Linking language Status for W3C documents: XML Schema WG Working draft XML Query WG Last Call XML Data Model, Algebra and Candidate/proposed Query Language recommendation Document Object Model WG Recommendation ~ Standard XSL WG XPath (with XML Linking WG) Transformation and stylesheet language (XSLT/XSL) 25 W3C XML Related Specifications W3C rec industry std W3C draft Open std XML Core XML 1.0 XML namespaces Xfragment RDF Canonical Xpath APIs JDOM JAXP DOM 1 DOM 2 DOM 3 XML signature XSLT XSL Xpointer XML query. Xlink XML base Infoset XHTML events MathML SMIL 1 & 2 SVG... XHTML 1.0 SAX 1 SAX 2 XML schema SOAP UDDI Biztalk FinXML CSS 1 XML-RPC ebxml IFX dirxml CSS 2 WSDL WDDX XMI CSS 3... FpML Style Protocols Web Services Application areas Xforms Modularized XHTML 100's more... XHTML basic Ian GRAHAM 26 13

14 XML is Just the Beginning... We now want to build advanced Web applications There is an urgent need for XML tools Designing XML tools is a data management problem: XML 1.0 to describe structured documents = Syntax for trees XML data models to describe the information content = Data model for trees XML schemas to describe the structure of information = Data definition language for trees XML languages to describe information processing = Data manipulation language for trees 27 Why is Database Work Important? Databases are large collections of data Even basic APIs to XML will fail on large XML documents Other reasons -- all the good things that the DB community brought the world: Data models, integrity constraints, and schemas Query languages, optimizers, fast joins Views, Updates Concurrency Federated database systems The emergence of XML underscores the importance of semistructured data 28 14

15 XML Main Characteristics: Semistructured Data Data Schema is not what it used to be: not given in advance (self-describing, schema-less), descriptive, not prescriptive (designed by document, not db experts), partial (documents and data mixed together), rapidly evolving (without notice), may be large (compared to the size of the data) Data Types are not what they used to be: elements and attributes are not strongly typed missing or additional attributes multiple attributes elements in the same collection may have different types i.e., heterogeneous collections attributes with different types in different elements 29 Schemas are Useful for Data readers What info is in a given collection? Thus, what queries might make sense? Data writers What should I call this piece of info? Is it okay to put this kind of data here? Efficient/effective data manipulation Optimize query processing Facilitate integration of multiple data sources Improve storage Construct indexes, statistics Forbid certain types of updates 30 15

16 Needs also for a DB Paradigm Shift Managing semistructured/xml data requires rethinking the design of components of a DBMS: How do we model it? directed labeled trees with references (i.e. graphs) How do we query it? a new standard based on functional languages and regular path expressions How do we store the data? looking for structural patterns How do we optimize queries? beginning to understand (algebras, indexes, etc.) What about Integrity constraints, views, updates,,? 31 Towards a Convergence Databases: relax rigid constraints imposed by schemas Move to a dynamic type system: semistructured data SGML, Document Management Origins HTML, Web Pages Documents: enrich formatting instructions with structuring/ semantic information Add types to documents: XML Semistructured Data ~ = XML ANS.1, ACeDB Scientific Data Formats Semi-structured data models for data integration 32 16

17 What This Course is About What the database community has done: Semistructured data model: SSD-exps, labeled graphs OEM, UnQL and YATL Schemas, Storage, Query Optimization What the Web community has done: Data formats and APIs: XML 1.0, DOM Transformation and Stylesheet languages (XSLT/XSL) Where they meet and where they differ Comparison to relational and object-oriented data models Present emerging XML technology as a data management issue XML Data models XML Data Definition (Schema) Languages XML Data Manipulation (Query) languages 33 Your Research Projects XML Data Semantics Type Systems Structural & Integrity Constraints Incremental Validation XML Query Processing XQuery Algebras Tree Query Pattern Containment & Minimization XPath Engines Stream-based Query Processing XML Query Optimization Storage Schemes Labelling & Indexing Schemes Structural Joins & Cost Models Data Statistics & Compression Benchmarks, Real & Synthetic Data XML Data Management Updates, Evolution & Versioning Access Control & Active Rules Data Publishing & Relational Databases Warehouses & View Maintenance XML Database Systems Commercial DBMS Native DBMS 34 17

18 Retrospective 1960 s: Data Centric 1970 s: Process Centric 1980 s: Object Oriented 1990 s: Component Based 2000 s: XML? 35 Data was our First Focus Record Layouts Printer Layouts System Flow Charts Decision Tables 60 s Data Batch Jobs were a Series of small Programs 36 18

19 Then we Focused on Logic GOTO-Less Programming Structured Programming Top-Down Design 60 s Data 70 s Logic Programs Became Very Large 37 Object Oriented Programming Focused on Runtime Behavior Common Terms for Analysis and Design Tightly Coupled Code 60 s Data 70 s Logic Code Reuse was the Holy Grail, Rarely Achieved 80 s OO 38 19

20 Component Programming Shifted the Focus to Interfaces Code Reuse IDE-Based Composition Limited Acceptance 60 s Data 90 s Comp 70 s Logic Serialization Tied to Code 80 s OO 39 XML Returns the Focus to Data XML Wrappers for Incompatible Systems Industry-Specific Markup Languages XML for Persistent Data and Composition 00 s XML 90 s Comp 70 s Logic XML Enables Middleware for Application-Specific Data 80 s OO 40 20