XML Databases 10. XML Storage 1 Overview

Transcription

1 XML Databases 10. XML Storage 1 Overview Silke Eckstein Andreas Kupfer Institut für Informationssysteme Technische Universität Braunschweig

2 10. XML Storage Motivation 10.2 Text-based storage 10.3 Model-based storage 10.4 Schema-based storage 10.5 Conclusion 10.6 Overview and References XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 2

3 10.1 Motivation Applications require different types of XML documents Structure vs. content Regular vs. irregular Thus, XML documents are Data-centric Document-centric or somewhere in-between Questions Storage of XML documents Efficient processing of queries on the stored documents or data There are several methods for storage 1 st goal: Learn and understand methods 2 nd goal: Classify methods Principles Advantages and disadvantages Usage XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 3

4 10.1 Motivation Characterisation of XML documents: Data-centric documents Structured, regular E.g. product catalog, order, invoice Document-centric documents Unstructured, irregular E.g. scientific article, book, , web page Semi-structured documents Data-centric and document-centric parts E.g. publications, Amazon, MS Press (example chapters) XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 4

5 10.1 Motivation Requirements for the physical layer: Order preserving and lossless storage of XML documents Efficient access to XML documents or parts thereof Quick response time for Queries Update operations Indexing Transaction processing Support of XPath and XQuery Support of SAX and DOM for applications XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 5

6 10.1 Motivation Storage approaches for XML documents Text-based Storage as character data Model-based On top of the relational DBMS Inside the relational DBMS Schema-based Mapping to (object-)relational databases Deriving the database schema from the XML structure Using user defined mapping procedures XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 6

7 10.1 Motivation Text-based storage Schema-based storage Model-based storage Examples XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 7

8 10. XML Storage Motivation 10.2 Text-based storage Index structures 10.3 Model-based storage 10.4 Schema-based storage 10.5 Conclusion 10.6 Overview and References XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 8

9 10.2 Text-based storage The whole XML document text is stored as character data File in the file system CLOB (Character-Large-OBject) in the DBS Operations on documents as a whole are very efficient Reading and writing the whole document But the content is monolithic and opaque with respect to the relational query engine (query can't inspect a fragment) Getting granular access requires additional support Full text index Path index XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 9

10 Index structures Index structures for XML documents allow efficient access for specific queries Different types of indexes are optimized for different types of queries Generate redundancy Index has to be up-to-date by propagating data changes Index structures can be storage structures as well They define the storage method XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 10

11 Index structures Types of index structures Value index Indexes atomar values of an XML document, like element content or attribute values Index format for structured parts of XML documents Already known from databases (B-trees, hash index, ) Full text index Indexes single words from the full text Index format for unstructured parts of XML documents Already known from Information Retrieval (inverted lists, tries, suffix trees, ) Path index Indexes subtrees/paths in an XML document Index format for semistructured parts of XML documents Already known from object-databases (access support relations, ) XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 11

12 Index structures B-tree as value index for an XML fragment document [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 12

13 Index structures Full text index Not limited to exact matches Keyword-based search and boolean retrieval Pattern search (with regular expressions) Use of Statistical, word-based methods Stop word removal Elimination of uncommon items Linguistic methods Normalization of words (e.g. capitalisation, hyphenation,) Word decomposition by rules (engl.) or dictionaries (german) Stemming Knowledge-based methods Use of ontologies and thesauri to search for synonyms, hypernyms and hyponyms XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 13

14 Index structures Inverted list as full text index for XML word occurrence word position in the text [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 14

15 Index structures word occurrence [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 15

16 Index structures Path index Structure information must be identifiable and reconstructable Assigning the markup to the content as well as Representing the hierarchical nesting and order of elements/attributes Especially suited for keyword search with regard to structure or path expressions FOR $b IN //book WHERE CONTAINS($b/author,"Benjamin") RETURN $b XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 16

17 Index structures Types of path indexes Nested path index Access to root node from every node Multi-index Accessing parent nodes Join-index Access parent and child nodes Access Support Relations (ASR) Generalization of indexes above, by listing all paths in a table [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 17

18 Index structures Conclusion Efficient query processing on XML documents requires different types of index structures Value index For efficient access to structured parts Keyword search, value search Full text index For efficient access to unstructured parts Path index Using the document structure Navigating queries XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 18

19 10.2 Text-based storage Summary text-based storage Schema definition: not required Document reconstruction: documents stay in their original format Queries: Information retrieval queries Processing the markup of the queries XML queries possible Special features: Full text functions Efficiency: Character string must be parsed on every access with XML processors expensive No concurrency on read or write no parallel processing Usage: For document-centric XML applications Suitable to only a limited extent also for semi-structured applications XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 19

20 10. XML Storage Motivation 10.2 Text-based storage 10.3 Model-based storage On top of the relational DBMS Inside the relational DBMS 10.4 Schema-based storage 10.5 Conclusion 10.6 Overview and References XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 20

21 Model-based storage Idea: generic storage of the graph structure XML elements, XML attributes, are nodes of a graph Nesting of elements defines edges Nodes get an (internal) ID based on graph traversal Using relations or object classes to store elements and attributes Elements ID Element name Value Reference to preceeding Rank Attributes ID Attribute name Value Reference to element Document structure can be restored completely Extension for data type adapted storage is possible XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 21

22 Model-based storage The EDGE approach [FK99] XML documents Variant BINARY: horizontal partition of EDGE based on label [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 22

23 Model-based storage XML queries XML queries (XPath, XQuery) are mapped to SQL queries (taking storage structures into account) Result of XML query is generated from result of database query "Labeling" of the result tuples Result is in XML format [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 23

24 Model-based storage Example: list bargain buy with prices SELECT a.content, b.content FROM Edge a, Edge b WHERE (a.label = 'price') AND (a.content < 10.00) AND (b.label = 'description') AND (b.parent = a.parent) AND (a.key = b.key) [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 24

25 Model-based storage DOM-based storage Information from the Document Object Model are stored in the database Storage alternatives (Object-)relational databases Object-oriented databases Developing own data structure [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 25

26 Model-based storage DOM-based storage example Node type: ELEMENT Node type: ATTRIBUTE Node type: TEXT [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 26

27 Model-based storage XML Queries XML queries (DOM method invocations) are mapped to SQL queries (taking storage structures into account) Result of method invocation is generated from result of database query [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 27

28 Model-based storage Summary model-based storage on top of the DBMS Schema definition: not required for storage Document reconstruction: Possible, but expensive Queries: XML queries possible Adapted database queries Special features: Querying many elements/attributes is expensive Efficiency: Navigation from the given context is efficient Restoring the document and evaluating path expressions is inefficient Usage: For data- and document-centric as well as for semi-structured XML applications XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 28

29 10. XML Storage Motivation 10.2 Text-based storage 10.3 Model-based storage On top of the relational DBMS Inside the relational DBMS 10.4 Schema-based storage 10.5 Conclusion 10.6 Overview and References XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 29

30 XML data type SQL/XML Storing XML documents inside the database as values of type XML <City> <City> <City> <Name> Braunschweig </Name> <Zip>38100</Zip> <Zip>38106</Zip> <State> Niedersachsen </State> </City> Generating XML documents using SQL/XML functions Datatype XML with belonging functions Mapping between SQL and XML Embedding XQuery in SQL Mapping between SQl and XML SQL XQuery SQL database XML datatype [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 30

31 XML data type A value of the data type XML can contain whole XML document XML element a set of XML elements All XML publishing operators from chapter 9.4 create values of the data type XML, not a string XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 31

32 XML data type XML(SEQUENCE) NULL or document node Untyped elements & attributes, elements not NULL XML(CONTENT(ANY)) 1 element child Validated against schema XML(CONTENT(UNTYPED)) XML(CONTENT(XMLSCHEMA)) 1 element child XML(DOCUMENT(ANY)) Validated against schema 1 element child XML(DOCUMENT(UNTYPED)) XML(DOCUMENT(XMLSCHEMA)) [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 32

33 XML data type Specification of XML type XML [({DOCUMENT CONTENT SEQUENCE} [({ANY UNTYPED XMLSCHEMA schema name})])] Modifiers are optional Primary type modifier DOCUMENT (XML document) CONTENT (XML element) SEQUENCE (sequence of XML elements) Secondary type modifier UNTYPED XMLSCHEMA (typed) ANY (may be typed) [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 33

34 XML data type Create a table that is an XML data type in itself CREATE TABLE XMLDOCUMENT OF XMLTYPE; Create a table containing an XMLType data type column CREATE TABLE XML ( ID NUMBER NOT NULL, XML XMLTYPE, CONSTRAINT XPK PRIMARY KEY (ID) ); [Pow07] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 34

35 XML data type Example: Definition of an XML type column CREATE TABLE Groups ( ID INTEGER, Name XML ); ID Name 123 <Groups>Annabelle</Groups> 234 <Groups>Magdalena, Marius</Groups> 345 <?xml version 1.0?> <Groups> <Person>Patrick</Person> <Person>Robert</Person> </Groups> 654 <Groups>Rebecca</Groups> <Groups>Torben</Groups> XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 35

36 XML data type Characteristics Allowed values: XML documents (including prolog) XML content according to XML 1.0 (includes pure text comments, PI?) NULL No comparison possible (compare CLOB in SQL) User can define an order, if comparison is necessary No corresponding type in programming languages for embedding in SQL available Standard defines operators to convert to other SQL data types [Kud07] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 36

37 XML data type Parsing & Serialization XMLParse: Parses a string value using an XML parser Produces value whose specific type is <City> <Name> Braunschweig </Name> <Zip>38100</Zip> <Zip>38106</Zip> <State> Niedersachsen </State> </City> XML(DOCUMENT(ANY)), or CONTENT, or XMLSerialize Transforms an XML value into a string value (CHAR, VARCHAR, CLOB, or BLOB) <City> <Name> Braunschweig </Name> <Zip>38100</Zip> <Zip>38106</Zip> <State> Niedersachsen </State> </City> [Mel05] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 37

38 XML data type Validation of XML Is like integrity constraints in DBs Requires an XML Schema XML Schemas may be registered with the SQL-server Implementation-defined mechanism Known by SQL name & by target namespace URI Schema does need a unique name Used by XMLValidate(), IS VALID, and to restrict values of XML(DOCUMENT-or-CONTENT(XMLSCHEMA)) [Mel05] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 38

39 XML data type Schema registration Register XMLSCHEMA ' FROM 'file://c:/xml_schemata/grussschema.xsd' AS GrussSchema COMPLETE ; CREATE TABLE Dokument_XML (Dokument_XML_Nr CHAR (4) NOT NULL PRIMARY KEY, Dokument XML, CONSTRAINT validieren CHECK (Dokument IS VALIDATED ACCORDING TO XMLSCHEMA ID GrussSchema ) ) ; XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 39

40 XML data type Schema definition Syntax XML(CONTENT(XMLSCHEMA) <schema> [<elements>])) <schema> := URI <namespace> [LOCATION <loc>] NO NAMESPACE [LOCATION <loc>] ID <registered schema name> <element> := [NAMESPACE <namespace>] ELEMENT <element name> [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 40

41 XML data type New functions and predicates: XMLValidate Validates an XML value against an XML Schema (or target namespace), returning new XML value with type annotations IS VALID Tests an XML value to determine whether or not it is valid according to an XML Schema (or target namespace); return true/false without altering the XML value itself IS DOCUMENT determines whether an XML value satisfies the (SQL/XML) criteria for an XML document IS CONTENT determines whether an XML value satisfies the (SQL/XML) criteria for XML content [Mel05] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 41

42 XML data type Benefits of schema registration Security issues Schemas cannot disappear without SQLserver knowing about it Schemas cannot be hijacked (altered in inappropriate ways) without SQL-server knowing about it Documents cannot be marked valid against schemas unless SQL-server knows about them [Mel05] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 42

43 XML standard Predefined schemas (build-in namespaces) xs: xsi: sqlxml: More depending on the DB implementation Completely supported per XML+Namespaces: XMLElement, XMLForest, XMLTable Default namespace, explicit namespace (prefix) Declare namespace within scopes of WITH clause, column definitions, constraint definitions, insert/delete/update statements, compound statements [Mel05] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 43

44 SQL/XML standard SQL/XML standard published as ISO/IEC :2003 Mappings and Publishing Functions ISO/IEC :2006 Adds XQuery, including Data Model, Validation ISO/IEC :2008 Updates [Mel05] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 44

45 SQL/XML standard SQL/XML:2003 plus Additional publishing functions XQuery data model More precise XML type (modifiers) XMLQuery, XMLTable XMLValidate, IS VALID XMLExists, IS DOCUMENT, IS CONTENT Casting between XML type and SQL types [Mel05] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 45

46 SQL/XML standard Overview of some operators for the XML type XMLELEMENT creates an XML element node XMLFOREST creates a sequence of XML element nodes from a table XMLCOMMENT creates an XML comment node XMLTEXT creates a text node XMLPI creates a processing instruction XMLAGG aggregates XML values of a group XMLCONCAT concatenates XML type values XMLTRANSFORM applies an XSL to a document [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 46

47 SQL/XML standard... Overview of some operators for the XML type XMLPARSE a well-formed SQL text to XML value XMLSERIALIZE converts an XML value to a SQL text XMLDOCUMENT creates an XML document node from an XML value XMLVALIDATE validates an XML value with a schema XMLQUERY evaluates an XQuery expression XMLTABLE transforms an XQuery result to a SQL table XMLITERATE transforms an XQery sequence to a SQL table [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 47

48 SQL/XML standard Review of SQL/XML Two components A data type XML to store XML data Functions to map relational structures to XML Only construction operators No extraction of values or search But construction operators are based on XQuery Mapping of tables, schemas, catalogues ignores some information from the relational schema UNIQUE REFERENCES CHECK Further extensions are expected [Kud07] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 48

49 Model-based storage Summary model-based storage inside the DBMS Schema definition: not required for storage Document reconstruction: Possible without problems Queries: XML queries possible Special features: Full integration of SQL and XML Efficiency: Vendor specific Optimized data structures possible Usage: For data- and document-centric as well as for semi-structured XML applications XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 49

50 10. XML Storage Motivation 10.2 Text-based storage 10.3 Model-based storage 10.4 Schema-based storage Automatic mapping User-defined mapping 10.5 Conclusion 10.6 Overview and References XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 50

51 Schema-based storage Motivation XML content shall be stored in a conventional database Accepting the loss of native access DB schema is derieved from a DTD or an XML schema Problem Generate DB schema automatically Thereby use as much structure information as possible General approach for mapping from a DTD Transform DTD into a tree representation Nodes: element types, attributes, etc. (type layer!!!) Edges: nesting relationships of element types and their restrictions Traverse tree in order to transform nodes and edges into database tables (according to certain rules) XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 51

52 Schema-based storage Generating the DB schema for a DTD: Rules to map element types: XML element type column of a table Sequence of element types columns of a table Alternative of element types column of a table Element type with quantifier? column with null values Element type with quantifier +,* set/list of columns (SET OF, LIST OF) Nested element types TUPLE OF Rules to map attributes: XML attribute column of a table IMPLIED null values allowed REQUIRED null values not allowed Default value DEFAULT constraint XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 52

53 Schema-based storage Mapping to relational databases DTD is usually required Queries use SQL functionality RDBMS data types are used (e.g. prices are NUMERIC) Problem: Mapping of collection types Subdivide into additional relations Example: Comment: Customer_Info: Feedback: Comment_ID Customer_info Feedback C0001 F0001 ID Fname Lname C0001 Charles Sanchez ID Type Content F001 opinion Darjeeling Special XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 53

54 Schema-based storage Mapping to object relational databases DTD is usually required Queries use SQL functionality "Natural" mapping to tuple types, collection types In case of irregular document structure databases contain many null values. Comment: Comment_ID <Customer_info> <Feedback> Fname Lname Charles Sanchez Type Content opinion Darjeeling Specia XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 54

55 Schema-based storage [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 55

56 Schema-based storage Mapping of recursive data definitions DTDs can be recursive Infinite recursion is impossible on instance layer of a database Procedure: Marking the nodes Subdividing into separate tables Use primary and foreign keys in RDBMS Use reference types in ORDBMS <!ELEMENT book (front, body, references)> <!ELEMENT references (book+)> XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 56

57 Schema-based storage Mapping of element sequences Sequence can be important Use an additional attribute in these cases Example: <lecture> <lesson>introduction</lesson> <lesson>xml basics</lesson> Order Lesson 1 Introduction 2 XML basics XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 57

58 Schema-based storage Mapping of alternatives XML allows to specify alternatives Example: <!ELEMENT car (compactcar sedan van)*> Three possible storage variants Each alternative is stored as separate table column Subdivide alternatives in separate tables Use a table column of type XML type XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 58

59 Schema-based storage Variant 1 all alternatives in one table Problem: many null values (wasting storage space) [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 59

60 Schema-based storage Variant 2 subdivided into multiple tables For queries, combination of tables is needed [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 60

61 Schema-based storage Variant 3 Using column type XML XML type allows XML queries or DOM methods [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 61

62 Schema-based storage Mapping with STORED (Semistructured TO RElational Data) Basic idea: Use data mining techniques on the XML structure to find a good mapping to tables [DFS99] Input XML documents (or an average sample of the collection) Query workload Restrictions of storage space, number of tables, No DTD or XML schema is required! Output Relational schema STORED-queries: Mapping instructions for XML documents to DB tables Procedure Determine the XML subtrees with the largest support in the collection and in the queries These subtrees are materialised in tables Irregular data is stored in overflow tables according to the EDGE approach XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 62

63 Schema-based storage Mapping with STORED example Subtrees with high support XML documents shown as tree structure [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 63

64 Schema-based storage Schema-based storage with automatic mapping Advantages Queries, data types, aggregation functions, views Integration in other databases when storing structured data Disadvantages Large schema, sparsely filled databases (many null values) No flexible data types, storage of alternatives has problems Less flexible queries No information retrieval queries possible without additional extensions No full text operations for semi- or unstructured data Usually native access is not possible any more XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 64

65 Schema-based storage Summary Schema-based storage with automatic mapping Schema definition: Is usually required and analysed not required, e.g. for STORED Document reconstruction: Limited (requires logging of the mapping process) Queries: Database queries XML queries possible,but lack the XPath horizontal axes, e.g. following, preceding-sibling Special features: Federation with existing databases is possible Efficiency: High efficiency by using the DB-engine Usage: For data-centric XML applications, but with limited nesting XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 65

66 User-defined Mapping User defined mapping Idea In all previously shown methods it is not possible to affect the storage in the DB With user defined mappings the user defines the storage structure The structure of XML documents and database schema can be designed independently from each other Also possible: storing XML documents in existing databases Annotation of DTD and XML schema, respectively In many cases the mapping definition is combined with existing schema information Only limited XML queries possible Logging of the mapping process from XML documents to databases For a given query all relevant data has to be stored (lossless mapping) XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 66

67 User-defined Mapping Mapping solutions with different specializations Algorithms, middleware, commercial applications, Varying amount of required input or user decisions Many algorithms create different database schemas Two phases Mapping Assign a place for each node type in the DB Shredding Import the XML data as DB tuples XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 67

68 User-defined Mapping The shredder can be part of the DB Usually requires an XML schema In the IBM Data Studio, the shredder is part of the "annotated XML schema decomposition" Direct approach in DB2: register the XML schema and call the stored procedure: register xmlschema from dec_files/custacc.xsd as cust_schema ; complete xmlschema cust_schema enable decomposition ; call SYSPROC.XDBDECOMPXML ('VRODRIG', 'CUST_SCHEMA',?,?, 1, null, null, null) XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 68

69 User-defined Mapping Shredding without XML schema in DB2 XMLTABLE function in combination with an INSERT INSERT INTO ENVELOPEXT (MAILFROM, MAILTO, MAILDATE, SUBJECT) SELECT MAILFROM, MAILTO, MAILDATE, SUBJECT FROM XMLTABLE( XMLNAMESPACES(' AS " "), '$doc/ mails/mail' (: some xquery-expression :) PASSING xml-source AS "doc" COLUMNS MAILFROM VARCHAR (100)PATH 'envelope/from', MAILTO VARCHAR (100)PATH 'envelope/to', MAILDATE VARCHAR (30) PATH 'envelope/ date', SUBJECT VARCHAR (100)PATH 'envelope/subject') AS T; ibrary/techarticle/dm-0801ledezma/ XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 69

70 User-defined Mapping Example: XML document mapping instruction [Tür08] XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 70

71 User-defined Mapping Mapping instruction Example syntax for XML-DBMS (Roland Bourret) <ClassMap> <ElementType Name="sales:SalesOrder"/> <ToClassTable> <Table Name="Sales"/> </ToClassTable> <PropertyMap> <Attribute Name="SONumber"/> <ToColumn> <Column Name="Number"/> </ToColumn> </PropertyMap> </ClassMap> Connection between elements and tables Connection between elements/attributes and table columns XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 71

72 User-defined Mapping Remarks Many different mapping languages or schema annotations Automatic mappings usually have an internal mapping language Remember the mapping constructs from last lecture (9). The SQL/XML annotations are a mapping language, too. DB2 uses similar annotations as SQL/XML see next slide: XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 72

73 Mapping SQL tables CREATE TABLE Account ( Name CHAR(20), Balance NUMERIC(12,2), ); Name Balance Joe 2000 Jim 3500 [Tür08] Mapping SQL table columns to XML elements Mapping table rowstoxml <row> elements <ACCOUNT> <row> <NAME>Joe</NAME> <BALANCE>2000</BALANCE> </row> <row> <NAME>Jim</NAME> <BALANCE>3500</BALANCE> </row> </ACCOUNT> <xsd:complextype xmlns:db2-xdb= " name="row.account"> <xsd:sequence> <xsd:elementname="name" type="char_20" db2-xdb:rowset="account" db2-xdb:column="name"/> <xsd:element name="balance" type="numeric_12_2"/> db2-xdb:rowset="account" db2-xdb:column="balance"/> </xsd:sequence> </xsd:complextype> <xsd:complextype name="table.account"> <xsd:sequence> <xsd:element name="row" type="row.account"/> </xsd:sequence> </xsd:complextype> <xsd:element name="account" type="table.account"/> XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 73

74 User-defined Mapping Summary schema-based storage with user defined mapping Schema definition: Depends on mapping language Document reconstruction: Not possible in most cases (requires logging of the mapping process) Queries: Database queries XML queries in rare cases only! Special features: Integration with existing databases is possible Efficiency: High efficiency by using the DB-engine Usage: For data-centric XML applications XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 74

75 10. XML Storage Motivation 10.2 Text-based storage 10.3 Model-based storage 10.4 Schema-based storage 10.5 Conclusion 10.6 Overview and References XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 75

76 10.5 Conclusion Different methods for storage of XML documents Text-based Storing whole XML documents as string Can use full text index or path index Model-based Generic mapping of the tree structure Schema-based Detect and analyse the structure of the XML documents Derive a DB schema from the structure Hybrid approaches A combination of some of those methods No algorithm has the optimal solution for all kind of XML documents Reasonable solution is heavily dependent on the application XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 76

77 10.6 References "XML und Datenbanken" [Tür08] Can Türker Lecture, University of Zurich, 2008 "XML und Datenbanken" [KM03] M. Klettke, H. Meier dpunkt.verlag, 2003 "Generierung eines adaptiven Datenbankschemas für datenzentrierte XML- Dokumente" [Bus08] Carsten Busche Diplomarbeit, TU Braunschweig, 2008 [FK99] D. Florescu, D. Kossmann: Storing and Querying XML Data using an RDBMS. IEEE Data engineering Bulletin (DEBU), Volume 22(3), Seiten 27-34, [DFS99] A. Deutsch, M.F. Fernández, D. Suciu: Storing Semistructured Data with STORED. Proceedings of the 1999 ACM SIGMOD international conference on Management of data, Seiten , ACM, XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 77

78 10.6 Overview Introduction and Basics 1. Introduction 2. XML Basics 3. Schema Definition 4. XML Processing Querying XML 5. XPath & SQL/XML Queries 6. XQuery Data Model 7. XQuery XML Updates 8. XML Updates & XSLT Producing XML 9. Producing XML Storing XML 10. XML storage 11.Relational XML storage 12. Storage Optimization Systems 13. Technology Overview XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 78

79 Questions, Ideas, Comments Now, or... Room: IZ 232 Office our: Tuesday, 12:30 13:30 Uhr or on appointment XML Databases Silke Eckstein Institut für Informationssysteme TU Braunschweig 79