Michael I. Schwartzbach Computer Science, University of Aarhus

Transcription

1 Databases 2009 Michael I. Schwartzbach Computer Science, University of Aarhus

2 XML vs. Tables In principle, p XML trees could replace tables: a more general data model XQuery is more expressive than SQL In practice, this will not happen soon: huge amounts of legacy data an enormous installed base no comparably efficient engines id course grade Math 101 C Biology 101 C Statistics 101 D Math 101 A Impedance mismatch XML 101 A Physics 101 B XML 102 A 2

3 Databases may support XML at different levels: XML views generates views of tables in XML format XML as LOBs allows storage and external processing XML as shredded documents stores XML as tables and uses SQL in place of XQuery Native XML directly supports XML trees and XQuery 3

4 Overview Publishing XML output from databases SQL/XML Storing XML input in databases LOBs shredding purexml Example: Housebrokers 4

5 XML Views Creating a fixed XML view of a table: room Turing Ada Aud-E 286 capacity <table> <row room= Turing-222 capacity= 4 /> <row room= Ada-333 capacity= 26 /> <row room= Aud-E capacity= 286 /> </table> Which view is better? <table> <row> <room>turing-222</room> <capacity>4</capacity> </row> <row> <room>ada-333</room> <capacity>26</capacity> </row> <row> <room>aud-e</room> <capacity>286</capacity> </row> </table> Any view allows integration ti with XML and XQuery 5

6 SQL/XML An extension of SQL that allows XML output New primitives: XMLDOCUMENT XMLELEMENT XMLATTRIBUTES XMLTEXT XMLNAMESPACES XMLCONCAT XMLFOREST XMLAGG These are used to build XML trees in queries 6

7 XML Constructors XMLDOCUMENT(...) constructs a document from a root element XMLELEMENT(NAME "name",,...) constructs an element node from a set of attributes and a sequence of child elements e e XMLATTRIBUTES(value AS "name",...) constructs a set of attributes from name/value pairs XMLTEXT(value) constructs t a character data node 7

8 XML Namespaces XMLNAMESPACES(DEFAULT "value") constructs a default namespace declaration XMLNAMESPACES("value" AS "prefix") constructs an explicit namespace declaration 8

9 XML Combinators XMLCONCAT(...) combines a number of XML arguments into a single XML sequence XMLFOREST(a 1 AS n 1,..., a k AS n k ) is syntactic sugar for: XMLCONCAT( XMLELEMENT(NAME n 1, a 1 ),... XMLELEMENT(NAME n k, a k ) ) 9

10 XML Aggregation XMLAGG(...) is an aggregation function It combines a set of XML values by concatenating them into a sequence This is not a true aggregation function since it is associative but not commutative To compensate, the order may be controlled: XMLAGG(... ORDER BY keys) 10

11 Programmable Views SELECT SQL/XML queries for generating the two XML views And countless other variations are similarly possible... XMLDOCUMENT( XMLELEMENT(NAME "table", XMLAGG( XMLELEMENT(NAME "row", XMLATTRIBUTES(room AS "room", capacity AS "capacity") ") ) ) ) ) FROM Rooms; SELECT XMLDOCUMENT( XMLELEMENT(NAME "table", XMLAGG( XMLELEMENT(NAME "row", XMLELEMENT(NAME "room", room), XMLELEMENT(NAME "capacity", capacity) ) ) ) ) FROM Rooms; 11

12 SQL Queries With XML Answers Generate the business cards for those students that are invited to meetings: SELECT XMLDOCUMENT( XMLELEMENT(NAME "cards", XMLNAMESPACES(DEFAULT ' XMLAGG( XMLELEMENT(NAME "card", XMLELEMENT(NAME "name", name), XMLELEMENT(NAME "title", 'PhD student'), XMLELEMENT(NAME " ", ) ) ) FROM People, Participants WHERE group= phd' AND userid=pid; 12

13 Overview Publishing XML output from databases SQL/XML Storing XML input in databases LOBs shredding purexml Example: Housebrokers 13

14 Storing XML An XML document may be stored in a database in three different manners: As text files (LOBs) As tables (shredding) As native XML (e.g. purexml) 14

15 XML LOBs An XML document may be stored as a LOB This is ideal and efficient for an XML repository : storing entire XML documents retrieving entire XML documents But it does not support XML queries 15

16 XML Shredding XML trees may be represented as tables One table for each element kind: one row for each element node a unique XML key for each node (in document order) an attribute with the parent key (NULL for the root) XML attributes become row attributes Chardata nodes become row attributes Some elements are inlined as row attributes: t always exactly one occurrence only a single character data child node 16

17 Shredding Example (1/3) The XML student records again: <students> <student id="100026"> <name>joe Average</name> <age>21</age> <major>biology</major> <results> <result course="math 101" grade="c-"/> <result course="biology 101" grade="c+"/> <result course="statistics 101" grade="d"/> </results> </student> <student id="100078"> <name>jack Doe</name> <age>18</age> <major>physics</major> <major>xml Science</major> <results> <result course="math 101" grade="a"/> <result course="xml 101" grade="a-"/> <result course="physics 101" grade="b+"/> <result course="xml 102" grade="a"/> </results> </student> </students> 17

18 Shredding Example (2/3) CREATE TABLE Student ( xmlkey INT PRIMARY KEY, parent INT REFERENCES Students(xmlkey), CREATE TABLE Students ( id CHAR(6), xmlkey INT PRIMARY KEY name VARCHAR(40), ) age INT ) CREATE TABLE Results ( xmlkey INT PRIMARY KEY, parent REFERENCES Student(xmlkey) ) ) grade VARCHAR(2) CREATE TABLE Result ( xmlkey INT PRIMARY KEY, parent INT REFERENCES Results(xmlkey), course VARCHAR(40), grade VARCHAR(2) ) CREATE TABLE Major ( xmlkey INT PRIMARY KEY, parent INT REFERENCES Student(xmlkey), chardata VARCHAR(40) ) 18

19 Shredding Example (3/3) Students xmlkey 101 Student xmlkey parent id name age Joe Average Jack Doe 18 Major Results xmlkey parent xmlkey parent chardata Biology Physics XML Science Result xmlkey parent course grade Math 101 C Biology 101 C Statistics 101 D Math 101 A XML 101 A Physics 101 B XML 102 A 19

20 Using Shredded XML Queries must be rephrased from XQuery to SQL in principle, this can be done automatically Results can be in different forms: other shredded XML documents using SQL real XML output using SQL/XML The XML document can be reconstructed: xmlkeys and parents encode the tree structure t an SQL/XML query can be systematically derived 20

21 An Example Query <doubles> { for $s in doc("students.xml")//student let $m := $s/major where count($m) ge 2 order by $s/@id return <double> } </doubles> { $s/name/text() } </double> SELECT XMLDOCUMENT( XMLELEMENT(NAME "doubles", XMLAGG( XMLELEMENT(NAME "double", Student.name) ORDER BY Student.id ) ) ) FROM Student, Major WHERE Student.xmlkey = Major.parent GROUP BY Student.id, Student.name HAVING COUNT(Major.xmlkey) >= 2; 21

22 Native XML: purexml Extend SQL tables with the column type XML XML data are stored directly as XML trees Extends SQL/XML with specific functions: XMLQUERY XMLEXISTS XMLCAST XMLPARSE XMLSERIALIZE XMLVALIDATE 22

23 XML Queries XMLQUERY('...') evaluates an XQuery expression returns an XML result XMLQUERY('...' PASSING value AS name) further defines $name as a variable in XQuery used for passing parameters SQL columns in scope are always implicit parameters XMLEXISTS('...') similar to XMLQUERY but is a predicate yields true if the generated sequence in non-empty 23

24 XML Casting XMLCAST(value AS type) casts an XML value into an SQL type casts an SQL value into XML Examples: XMLCAST(XMLQUERY('42') AS INT) XMLCAST(XMLQUERY('"foo"') AS CHAR(3)) XMLCAST(XMLQUERY('"foo"') AS VARCHAR(10)) XMLCAST(XMLQUERY('"foo"') AS bar) XMLCAST(NULL AS XML) XMLCAST('foo' AS XML) 24

25 XML Parsing and Serializing XMLPARSE(DOCUMENT value) parses the string value into an XML tree XMLSERIALIZE(value AS type) unparses the XML value into an SQL type the type must be VARCHAR or CLOB 25

26 XMLVALIDATE( ) DOCUMENT value XML Validation ACCORDING TO XMLSCHEMA ID name validates the unparsed XML value against the XML Schema the schema must be uploaded d and named in advance an runtime errors occurs if the document is invalid If a specific root element is required: XMLVALIDATE(... ELEMENT name) 26

27 XML Indexes XML trees can also be given indexes These are based on (simple) XPath expressions CREATE TABLE AllStudents ( university VARCHAR(20), students XML ); CREATE INDEX AllIndex ON AllStudents(students) GENERATE KEY USING XMLPATTERN '/students/student/name' AS SQL VARCHAR(40); 27

28 Overview Publishing XML output from databases SQL/XML Storing XML input in databases LOBs shredding purexml Example: Housebrokers 28

29 The Housebrokers Example A data model that mixes XML and tables Table data: data about the listed houses data about the prospective buyers XML data: feedback from buyers having viewed a house 29

30 Houses CREATE TABLE HOUSES ( ID INT PRIMARY KEY ADDRESS VARCHAR(200) NOT NULL, SELLER VARCHAR(200) NOT NULL, VARCHAR(50) NOT NULL, PASSWORD CHAR(8) NOT NULL, SQ_METERS_HOUSE INT NOT NULL, SQ_METERS_GROUND INT NOT NULL, NO_OF_ROOMS INT NOT NULL, ASKING_PRICE INT NOT NULL, LOWEST_PRICE INT NOT NULL, SOLD_DATE DATE WITH DEFAULT NULL, SOLD_PRICE INT WITH DEFAULT NULL, SOLD_TO_ID INT WITH DEFAULT NULL, OFFERED_DATE DATE NOT NULL, ZIPCODE CHAR(4) WITH DEFAULT NULL ); 30

31 Prospects CREATE TABLE PROSPECTS ( ID INT NOT NULL, HOUSE_ID INT NOT NULL, CREATED_DATE DATE NOT NULL, UPDATED_DATE DATE NOT NULL, ED_DATE DATE WITH DEFAULT NULL, FEEDBACK XML ); Notes from house showings are in a looser format 31

32 Feedback in XML Format <feedback xmlns=" <buyer> <name>jo Smith</name> </buyer> <offer>456870</offer> <repair status="open"> <part_of_house>garage door</part_of_house> <estimated_price>150</estimated_price> </repair> <movable status="open">garden furniture</movable> <question> <text>is adsl installed?</text> <answer>yes, 2MB connection</answer> </question> </feedback> 32

33 XML Schema For Feedback (1/4) <schema xmlns=" xmlns:b=" targetnamespace=" elementformdefault="qualified"> <element name="feedback"> <complextype> <sequence> <element ref="b:buyer"/> <element name="offer" type="nonnegativeinteger" minoccurs="0"/> <choice minoccurs="0" maxoccurs="unbounded"> <element ref="b:repair"/> <element ref="b:movable"/> </choice> <element ref="b:question" minoccurs="0" maxoccurs="unbounded"/> </sequence> </complextype> </element> 33

34 XML Schema For Feedback (2/4) <element name="buyer"> <complextype> <sequence> <element name="name" type="string"/> <element name=" " type="string"/> </sequence> </complextype> </element> <element name="repair"> <complextype> <sequence> <element name="part_of_house" type="string"/> <element name="estimated_price" type="nonnegativeinteger"/ <element name="comment" type="string" minoccurs="0"/> </sequence> <attribute name="status" type="b:status_type" use="required"/> </complextype> </element> 34

35 XML Schema For Feedback (3/4) <element name="movable"> <complextype> <simplecontent> <extension base="string"> <attribute name="status" type="b:status_type" use="required"/> </extension> </simplecontent> </complextype> </element> <element name="question"> <complextype> <sequence> <element name="text" type="string"/> <element name="answer" type="string"/> </sequence> </complextype> </element> 35

36 XML Schema For Feedback (4/4) <simpletype name="status_type"> <restriction base="string"> string <enumeration value="accepted"/> <enumeration value="rejected"/> <enumeration value="open"/> </restriction> </simpletype> </schema> 36

37 The Application (1/2) A standard Web architecture: Browser Java Servlets DB2 37

38 The Application (2/2) Present status of prospects Update a feedback form 38

39 XML for Feedback XML feedback form Updater Java Servlets DB2 table data SQL/XML Presenter Java Servlets DB2 39

40 Representing Feedback XML The feedback XML is sent back and forth shredded representation is no good The feedback XML is queried with XQuery LOB representation is no good Native XML supports both uses efficiently purexml saves the day 40

41 SQL/XML For The Updater Fetching the feedback form: SELECT HOUSE_ID, XMLQUERY( 'declare default element namespace \" /feedback' ) FROM PROSPECTS WHERE ID=id; Storing the updated feedback form: INSERT INTO PROSPECTS(HOUSE_ID, FEEDBACK) VALUES (id, XMLPARSE(DOCUMENT XMLVALIDATE(DOCUMENT feedback ACCORDING TO XMLSCHEMA ID HousebrokersFeedback))) 41

42 SQL/XML For The Presenter Generating the table of status information: SELECT id AS feedback_id, XMLCAST(XMLQUERY( 'declare default element namespace\" /feedback/offer' ) AS INT) AS offer, XMLCAST(XMLQUERY( 'declare default element namespace \" /feedback/(offer - sum(repair/estimated_price))' ) AS INT) AS net_offer, XMLCAST(XMLQUERY( 'declare default element namespace \" count(/feedback/question) - count(/feedback/question/answer)' ) AS INT) AS remaining_questions FROM PROSPECTS WHERE house_id = id; 42