XML Query Languages. Yanlei Diao UMass Amherst April 22, Slide content courtesy of Ramakrishnan & Gehrke, Donald Kossmann, and Gerome Miklau

Transcription

1 XML Query Languages Yanlei Diao UMass Amherst April 22, 2008 Slide content courtesy of Ramakrishnan & Gehrke, Donald Kossmann, and Gerome Miklau 1

2 Querying XML How do you query a directed graph? a tree? A common approach: define some sort of a template describing traversals from the root of the directed graph. In XML, the basis of this template is XPath. XPath navigates through the tree to select data. The complete query language is XQuery. XQuery both selects data and constructs output. 2

3 XML Path Language (XPath) 1.0 XPath 1.0: Navigates through hierarchical XML document structure Applies selection conditions Retrieves nodes Input: a tree of nodes Node type: Document, Element, Attribute, Text Output: often a set of nodes An ordered language It can query in an order-aware fashion. It always returns nodes in document order. 3

4 Sample Data for Queries <bib> <book> <publisher> Addison-Wesley </publisher> <author> Serge Abiteboul </author> <author> <first-name> Rick </first-name> <last-name> Hull </last-name> </author> <author> Victor Vianu </author> <title> Foundations of Databases </title> <year> 1995 </year> </book> <book price= 55 > <publisher> Freeman </publisher> <author> Jeffrey D. Ullman </author> <title> Principles of Database and Knowledge Base Systems </title> <year> 1998 </year> </book> </bib> 4

5 Data Model for XPath Ordered Tree: ordered among element and text nodes; unordered among attribute nodes bib Document Node Element Node book book Attribute Node publisher author author author... price... Text Node Addison- Wesley Serge Abiteboul first Rick last Kull Victor Vianu 50 5

6 XPath: Simple Expressions /bib/book/year child operator / Result: <year> 1995 </year> <year> 1998 </year> /bib/paper/year Result: empty (there were no papers) /bib/(book paper)/year OR operator 6

7 XPath: Restricted Kleene Closure //author Result:<author> Serge Abiteboul </author> <author> <first-name> Rick </first-name> <last-name> Hull </last-name> </author> <author> Victor Vianu </author> <author> Jeffrey D. Ullman </author> /bib//first-name descendant operator // Result: <first-name> Rick </first-name> 7

8 XPath: Text Nodes /bib/book/author/text() Result: Serge Abiteboul Victor Vianu Jeffrey D. Ullman Rick Hull doesn t appear because he has firstname, lastname elements. 8

9 XPath: Wildcard //author/* Result: <first-name> Rick </first-name> <last-name> Hull </last-name> * Matches any element name 9

10 XPath: Attribute Nodes Result: means that price has to be an attribute 10

11 XPath: Predicates Positional predicate /bib/book/author[1] Result: <author> Serge Abiteboul </author> <author> Jeffrey D. Ullman </author> 11

12 XPath: Predicates Predicate using a nested path expression /bib/book/author[first-name] Result: <author> <first-name> Rick </first-name> <last-name> Hull </last-name> </author> /bib/book/author[first-name = Rick ] 12

13 XPath: More Predicates More nested paths. operator /bib/book/author[firstname][address[city][.//zip]]/lastname Result: <lastname> </lastname> <lastname> </lastname> [./city] 13

14 XPath: Summary /bib matches a bib element under root /bib/paper matches a paper in bib /bib/* matches any child element in bib /bib//paper matches a paper in bib, at any depth //paper matches a paper at any depth //(paper book) matches a paper or a book /bib/book/@price matches price attribute in book, in bib /bib/book[@price<60]/author matches an author of a book costing less than $60, in bib /bib/book/author[1]/lastname matches the lastname of the first author of each book in bib /bib/book/author[lastname] matches an author with a lastname in each book in bib 14

15 XQuery 1.0 XQuery: the SQL for XML. A full programming language that can retrieve fragments from XML transform XML data to XML data in arbitrary ways Currently being widely used for querying XML data sets routing and transforming XML messages in Web services creating logical views in data integration 15

16 XQuery Data Model Instance of the data model: a sequence composed of zero or more items Item: node or atomic value Node: document element attribute text namespace processing instruction comment Atomic value: an indivisible value, e.g., string, boolean, ID, IDREF, decimal, URI,

17 XML nodes XML nodes: have unique node identifiers. have children and an optional parent (conceptually a tree ). are ordered based of the topological order in the tree ( document order ). Nodes can be compared directly by: (i) identity, (ii) document order, or (iii) value (after converted from node to value, explicitly using data() or implicitly) 17

18 XQuery Expressions XQuery Expr := Constants Variable FunctionCalls ComparisonExpr ArithmeticExpr LogicExpr PathExpr FLWORExpr ConditionalExpr QuantifiedExpr TypeSwitchExpr InstanceofExpr CastExpr UnionExpr IntersectExceptExpr ConstructorExpr ValidateExpr Expressions can be nested with full generality! Functional programming heritage. 18

19 Examples of Path Expressions doc( bibliography.xml )/bib $x/bib/book/year $x/chapter[figure] $x//author $x/book/chapter[5]/section[2] 19

20 Simple Iteration Expression Syntax : for variable in expression1 return expression2 e.g. for $x in doc( bib.xml )/bib/book return $x/title Semantics: bind the variable $x to each node returned by expression1 for each such binding, evaluate expression2 concatenate the resulting sequences 20

21 FLWOR expressions FLWOR (For-Let-Where-Order by-return) is a high-level construct that selects data and further restructures data for output: for $x in expression1 /* similar to FROM in SQL */ [let $y := expression2 ] /* no analogy in SQL */ [where expression3 ] /* similar to WHERE in SQL */ [order by expression4 (ascending descending)? (empty (greatest least))? ] /* similar to ORDER BY in SQL */ return expression5 /* similar to SELECT in SQL */ 21

22 Example FLOWR Expression for $x in doc( bib.xml )/bib/book // iterate, bind each item to $x let $y := $x/author // no iteration, bind a sequence to $y where $x/title= XML // filter each tuple ($x, $y) order by $x/@year descending // order tuples return count($y) // one result per surviving tuple The for clause iterates over all books, binding $x to each book in turn. For each binding of $x, the let clause binds $y to all authors of this book. The result of for and let clauses is a tuple stream in which each tuple contains a pair of bindings for $x and $y, i.e. ($x, $y). The where clause filters each tuple ($x, $y) by checking predicates. The order by clause orders surviving tuples. The return clause returns the count of $y for each surviving tuple. 22

23 FOR versus LET for $x in doc("bib.xml")/bib/book return <result> $x </result> let $x := doc("bib.xml")/bib/book return <result> $x </result> Returns: <result><book>...</book></result> <result><book>...</book></result> <result><book>...</book></result>... Returns: <result> <book>...</book> <book>...</book> <book>...</book>... </result> 23

24 Getting Distinct Values from FOR Distinct values: The for expression evaluates to a sequence of nodes. The fn:distinct-values function converts nodes to atomic values and removes duplicates. for $a in doc( bib.xml )/book/author return $a versus for $a in distinct-values(doc( bib.xml )/book/author) return <author-name> {$a} </author-name> 24

25 More Examples of WHERE Selection conditions for $b in doc("bib.xml")/bib/book where $b/publisher = Addison Wesley" and $b/@year = 1998 return $b/title for $b in doc("bib.xml")/bib/book where empty($b/author) return $b/title for $b in doc("bib.xml")/bib/book where count($b/author) = 1 return $b/title Aggregates over a sequence: count, avg, sum, min, max 25

26 General Comparisons General comparison operators (=,!=, <, >, <=, >=): Existentially quantified comparisons, applied to operand sequences of any length. Comparison is true if one value from a sequence satisfies the comparison. for $b in doc( bib.xml )/bib/book where $b/author = JefferyUllman return $b/author (1, 2) = (2, 3)? 26

27 Value Comparisons Value comparisons (eq, ne, lt, le, gt, ge): Used to compare single values. If the left (right) operand does not carry a single value, a runtime error is thrown. for $b in doc( bib.xml )/bib/book where $b/author eq JefferyUllman return $b/author 27

28 Node Comparisons Node comparison by identity: for $b in doc( bib.xml )/bib/book where $b/author[last = Ullman ] is $b/author[first = Jeffery ] return $b/title Node comparison by document order: for $b in doc( bib.xml )/bib/book where $b/author[. = JefferyUllman ] << $b/author[. = JenniferWidom ] return $b/title 28

29 String Operations Functions for string matching fn:contains(xs:string, xs:string) fn:starts-with(xs:string, xs:string) fn:ends-with(xs:string, xs:string) fn:substring-before(xs:string, xs:string) fn:substring-after(xs:string, xs:string) fn:matches($input as xs:string, $pattern as xs:string) for $a in doc( bib.xml )//author where contains($a, Ullman") return $a Use all data of $a (i.e., all descendant text nodes)! <author> <name>jeffery Ullman</name> </author> <author> <first>jeffery</first> <last>ullman</last> </author> 29

30 Joins in XQuery Joins A tuple here is ($b, $p), a unique combination of bindings of $b, $p. for $b in doc( bib.xml )//book, $p in doc( publishers.xml")//publisher where $b/publisher = $p/name return ($b/title, $p/name, $p/address) 30

31 Node Constructors Node constructor: constructs new nodes elements, attributes, text, documents, processing instructions, comments.. Direct element constructor: a form of element constructor in which the element name is a constant <book isbn="isbn "> <title>harold and the Purple Crayon</title> <author> <first>crockett</first> <last>johnson</last> </author> </book> 31

32 Element Construction in RETURN for $p in doc( report.xml )//person return ( <result> first person is Bob /* literal */ </result>, <result> { $p/name } /* {evaluated content} */ </result>, <result> { $p/name/text()} </result>, <result> Yesterday {$p/name} visited us. </result> ) Braces "{ }" are used to delimit enclosed expressions, which will be evaluated and replaced by their values. If return has > 1 top element, use ( ) and comma operator. 32

33 More on Element Construction Elements may contain attributes. Value of an attribute can be: a string of characters enclosed in quotes, e.g., <book year= 2005 > evaluated using enclosed expressions specified in { } <bib> { for $b in doc("bib.xml")/bib/book where $b/publisher = "Addison-Wesley" and $b/@year > 1991 return <book year="{ $b/@year }"> { $b/title } </book> } </bib> The largest wrapping tag creates well-formed XML. 33

34 Nested FLWOR <author_list> { for $a in distinct-values(doc( bib.xml )/book/author) order by $a return <author> <name> {$a} </name> <books> { for $b in doc( bib.xml )/book[author = $a] order by $b/title return $b/title } </books> </author> } </author_list> The nested FLOWR effectively implements group books by author. No Group By in XQuery! 34