XML Overview COMP9319

Transcription

1 XML Overview COMP9319 Raymond Wong

2 XML XML (extensible Markup Language) is a standard developed by W3C (World Wide Web Consortium) and endorsed by a host of industry heavyweights such as IBM, Microsoft, SAP, Software AG, General Motors,...

3 Links

4 About XML XML is just a markup language defined by W3C (officially in Feb 98) It s a simplified version of SGML HTML4.0 XML SGML HTML for presentation markup, XML for content markup 4

5 Semistructured Data / XML Semistructured => loosely structured (no restrictions on tags & nesting relationships) no schema required XML under the semistructured umbrella self-describing the standard for information representation & exchange 5

6 Storage format vs presentation format - The power of markup Traditional Database or Spreadsheet Raymond, Wong, wong, 5932, John, Smith, jsmith, 1234,... HTML <br> <font size=1 color= ff003a > <ul> <li> <b> Raymond Wong </b> </li> <li> Login: wong </li> <li> Phone: <i> x5932 </i> </li> </ul> </font> XML <Staff> <Name> <FirstName> Raymond </FirstName> <LastName> Wong </LastName> </Name> <Login> wong </Login> <Ext> 5932 </Ext> </Staff> 6

7 The Family of XML Technologies Document structures Definition of gst, price, XMLSchema / DTD <Product> <Product> <product_id> m101 </product_id> <product_id> <Product> m101 </product_id> <name> Sony walkman </name> <name> <product_id> Sony walkman m101 </name> </product_id> <currency> AUD </currency> <currency> <name> AUD Sony </currency> walkman </name> <price> </price> <price> <currency> </price> AUD </currency> <gst> 10% </gst> <gst> 10% <price> </gst> </price> </Product> </Product> <gst> 10% </gst> </Product> Presentation format info XML Stylesheet XML Stylesheet XML Stylesheet 7

8 Why need to query XML data To extract data from large XML docs To exchange data (data- or query-shipping) To exchange data beteen different user communities or ontologies or schemas To integrate data from multiple XML sources 8

9 XML data file can be modeled in a tree form <Staff> <Name> <FirstName> Raymond </FirstName> <LastName> Wong </LastName> </Name> <Login> wong </Login> <Ext> 5932 </Ext> </Staff> Staff Name Login Ext wong 5932 FirstName LastName Raymond Wong 9

10 Answering queries requiring navigation of the data tree dept Query: All phones? Query: Profs phones? Query: Peter s phone? phone reception prof members Query: Susan s phone??? prof name phone name peter susan 10

11 Example query in XPath <Name>The Bamboo Restaurant</Name> <Name>Chen's Seafood Restaurant</Name> <xql:attribute Name="Thai Palace" /> <Name>Nice Noodles</Name> 11

12 XML Terminology tags: book, title, author, start tag: <book>, end tag: </book> elements: <book> </book>,<author> </author> elements are nested empty element: <red></red> abbrv. <red/> an XML document: single root element well formed XML document: if it has matching tags 12

13 More XML: Attributes <book price = 55 currency = USD > <title> Foundations of Databases </title> <author> Abiteboul </author> <year> 1995 </year> </book> 13

14 More XML: Oids and References <person id= o555 > <name> Jane </name> </person> <person id= o456 > <name> Mary </name> <children idref= o123 o555 /> </person> <person id= o123 mother= o456 > <name>john</name> </person> 14

15 More XML: CDATA Section Syntax: <![CDATA[...any text here ]]> Example: <example> <![CDATA[ some text here </notatag> <> ]]> </example> 15

16 More XML: Entity References Syntax: &entityname; Example: <element> this is less than < </element> Some entities: < < > & &apos; " & > & Unicode char 16

17 More XML: Processing Instructions Syntax: <?target argument?> Example: <product> <name> Alarm Clock </name> <?ringbell 20?> <price> </price> </product> 17

18 More XML: Comments Syntax <!--... Comment text... --> 18

19 XML Namespaces (1/99) name ::= [prefix:]localpart <book xmlns:isbn= > <title> </title> <number> 15 </number> <isbn:number>. </isbn:number> </book> 19

20 XML Namespaces syntactic: <number>, <isbn:number> semantic: provide URL for schema <tag xmlns:mystyle = > defined here <mystyle:title> </mystyle:title> <mystyle:number> </tag> 20

21 Differences from traditional DB What sets semistructured/xml data servers apart from RDBMS or OODB is the lack of typing. This affects mostly the way the data is stored and indexed. Also, Web data are inherently distributed 21

22 Implementing XML Repository Repository backend plain text file relational database object database tailor-made, specialized XML database Type information even partial typing information can be used to improve the storage 22

23 Text files it s the simplest way to store easy to handle widely available have to check out an entire doc in order to retrieve a datum simultaneously access/update access/modify an item from a large catalog collection 23

24 Relational databases existing, proven technology to provide full database management it s not easy and efficient to manage XML data in traditional RDBMS 24

25 An Example (using RDBMS) assume no typing information data can be an arbitrary graph let s use two tables for the XML instances: one to store all edge information one to store values 25

26 The two tables Ref(src, label, dst) Val(oid, value) Suppose a simple query like: family/person/hobby in XPath 26

27 The same query in SQL select v.value from Ref r1, Ref r2, Ref r3, Val v where r1.src = root AND r1.label = family AND r1.dst = r2.src AND r2.label = person AND r2.dst = r3.src AND r3.label = hobby AND r3.dst = v.oid This is a 4-way join!!! It s very inefficient though index on label can help a lot. 27

28 Efficiency problem even simple query will have a large no of joins RDBMS organizes data based on the structure of tables and type info => clustering, indexing, query optimization are not working properly for XML data Also #ways to traverse path expressions are much more than that on tables 28

29 XML Parsers There are several different ways to categorise parsers: Validating versus non-validating parsers Parsers that support the Document Object Model (DOM) Parsers that support the Simple API for XML (SAX) Parsers written in a particular language (Java, C, C++, Perl, etc.) 29

30 Non-Validating Parsers Speed and efficiency It takes a significant amount of effort for an XML parser to process a DTD and make sure that every element in an XML document follows the rules of the DTD. If we only want to find tags and extract information we should use a nonvalidating parser 30

31 Using an XML Parser Three basic steps in using an XML parser Creating a parser object Passing the XML document to the parser Processing the results Generally, writing out XML is not in the scope of parsers (though some may implement proprietary mechanisms) 31

32 The SAX Parser SAX parser is an event-driven API An XML document is sent to the SAX parser The XML file is read sequentially The parser notifies the class when events happen, including errors The events are handled by the implemented API methods to handle events that the programmer implemented 32

33 SAX Parser Events A SAX parser generates events at the start and end of a document, at the start and end of an element, when it finds characters inside an element, and at several other points User writes the code that handles each event, and decides what to do with the information from the parser 33

34 Example Event Handlers startelementhandler endelementhandler chardatahandler CDATASectionHandler CommentHandler PIHandler etc... 34

35 When to (not to) use SAX Ideal for simple operations on XML files E.g. reading and extracting elements Good for very large XML files (c.f. DOM) Not good if we want to manipulate XML structure Not designed for writing out XML 35

36 DOM Document Object Model Set of interfaces for an application that reads an XML file into memory and stores it as a tree structure The abstract API allows for constructing, accessing and manipulating the structure and content of XML and HTML documents 36

37 What a DOM Parser Gives When you parse an XML document with a DOM parser, you get back a tree structure that contains all of the elements of your document The DOM provides a variety of functions you can use to examine the contents and structure of the document 37

38 DOM Parser produces a memory tree (DOM Tree) after parsing <Staff> <Name> <FirstName> Raymond </FirstName> <LastName> Wong </LastName> </Name> <Login> wong </Login> <Ext> 5932 </Ext> </Staff> DOM Parser Staff Name Login Ext FirstName Raymond LastName Wong wong

39 Why to Use DOM Task of writing parsers is reduced to coding against the DOM Tree API Domain-specific frameworks will be written on top of DOM 39

40 You can build a DOM parser using a SAX parser <Staff> <Name> <FirstName> Raymond </FirstName> <LastName> Wong </LastName> </Name> <Login> wong </Login> <Ext> 5932 </Ext> </Staff> SAX Parser DOM Tree Builder Staff Name Login Ext APPLICATIONS DOM API FirstName wong 5932 LastName Raymond Wong 40

41 Creating a DOM Tree A DOM implementation has a method to pass an XML file and returns a Document object that represents the root element of a whole document On the Document objects, DOM standard interface can be used to interact with XML structure 41

42 DOM Tree Document Document Type Element Attribute Attribute Element Element Text Text Element Entity Reference Text Text Comment Text 42

43 Normalizing a Tree Normalizing a DOM Tree has two effects: Combine adjacent textual nodes Eliminate empty textual nodes We can apply a normalize() method to the document element 43

44 DOM Interfaces and Classes Node DocumentFragment Figure as from The XML Companion - Neil Bradley Document Text CDATASection CharacterData Attr Comment Element DocumentType Notation Entity EntityReference NodeList NamedNodeMap DocumentType ProcessingInstruction 44

45 DOM Classes DOM classes Node The base data type of the DOM Element Represents element Attr Represents an attribute of an element Text The content of an element or attribute Document Represents the entire XML document. A Document object is often referred to as a DOM tree 45

46 Node Methods Three categories of methods Node characteristics name, type, value Contextual location and access to relatives parents, siblings, children, ancestors, descendants Node modification Edit, delete, re-arrange child nodes 46

47 e.g., Node Navigation methods Every node has a specific location in tree Node interface specifies methods to find surrounding nodes Node getfirstchild(); Node getlastchild(); Node getnextsibling(); Node getprevioussibling(); Node getparentnode(); NodeList getchildnodes(); 47

48 DOM vs. SAX If your document is very large and you only need a few elements - use SAX If you need to process many elements and perform manipulations on XML - use DOM If you need to access the XML many times -use DOM 48

49 What Would You Choose for Processing an XML document in a server? Processing an XML document in a remote client? Direct access to element in the XML file (e.g., index based on paths) A visual tool for traversal over the document tree? 49

50 Summary - DOM DOM is a large API designed for complex environments A W3C standard, developed by W3C working groups Implemented by products like Xerces Represents a document tree fully held in memory Has to have the same API on multiple languages Reading and changing the document is non-intuitive Fairly heavyweight to load and store in memory 50

51 Summary - SAX SAX is a lightweight API designed for fast reading Public domain API from David Megginson and XML-DEV mailing list Implemented by products like Xerces Callback mechanism reports when document elements are encountered Lightweight since the document is never entirely in memory Does not support modifying the document Does not support random access to the document Fairly steep learning curve to use correctly 51

52 What is Expat? Expat is a C parser library. It's the underlying XML parser for Mozilla, perl's XML::Parser, and other open-source XML parsers. It's very fast, reliable & robust. Learn more from esp. the introduction article (useful for your first lab and assignment) 52

53 Key functions in Expat XML_ParserCreate Create a new parser object. XML_SetElementHandler Set handlers for start and end tags. XML_SetCharacterDataHandler Set handler for text. XML_Parse Pass a buffer full of document to the parser 53

54 The sample file outline.c void main (int argc, char **argv) { XML_Parser p = XML_ParserCreate(NULL); if (! p) { fprintf(stderr, "Couldn't allocate memory for parser\n"); exit(-1); } XML_SetElementHandler(p, start, end); 54

55 The main routine (con t) for (;;) { int done; int len; len = fread(buff, 1, BUFFSIZE, stdin); if (ferror(stdin)) { fprintf(stderr, "Read error\n"); exit(-1); } done = feof(stdin); if (! XML_Parse(p, Buff, len, done)) { fprintf(stderr, "Parse error at line %d:\n%s\n", XML_GetCurrentLineNumber(p), XML_ErrorString(XML_GetErrorCode(p))); exit(-1); } if (done) break; } } /* End of main */ 55

56 The startelement Handler char Buff[BUFFSIZE]; int Depth; void start(void *data, const char *el, const char **attr) { int i; for (i = 0; i < Depth; i++) printf(" "); printf("%s", el); for (i = 0; attr[i]; i += 2) { printf(" %s='%s'", attr[i], attr[i + 1]); } printf("\n"); Depth++; } /* End of start handler */ 56

57 The endelement Handler void end(void *data, const char *el) { Depth--; } /* End of end handler */ 57

58 XPath (11/99) Building block for other W3C standards: XSL Transformations (XSLT) XML Link (XLink) XML Pointer (XPointer) XPath 2.0 XQuery Was originally part of XSL 58

59 Example for XPath Queries <bib> <book> <publisher> Addison-Wesley </publisher> <author> Serge Abiteboul </author> <author> <first-name> Rick </first-name> <last-name> Hull </last-name> </author> <author> Victor Vianu </author> <title> Foundations of Databases </title> <year> 1995 </year> </book> <book price= 55 > <publisher> Freeman </publisher> <author> Jeffrey D. Ullman </author> <title> Principles of Database and Knowledge Base Systems </title> <year> 1998 </year> </book> </bib> 59

60 Data Model for XPath bib The root element book book publisher author.... Addison-Wesley Serge Abiteboul 60

61 XPath: Simple Expressions /bib/book/year Result: <year> 1995 </year> <year> 1998 </year> /bib/paper/year Result: empty 61

62 XPath: Restricted Kleene Closure //author Result:<author> Serge Abiteboul </author> <author> <first-name> Rick </first-name> <last-name> Hull </last-name> </author> <author> Victor Vianu </author> <author> Jeffrey D. Ullman </author> /bib//first-name Result: <first-name> Rick </first-name> 62

63 XPath: Text Nodes /bib/book/author/text() Result: Serge Abiteboul Victor Vianu Jeffrey D. Ullman Rick Hull doesn t appear because he has firstname, lastname Functions in XPath: text() = matches the text value node() = matches any node (= * or text()) name() = returns the name of the current tag 63

64 XPath: Wildcard //author/* Result: <first-name> Rick </first-name> <last-name> Hull </last-name> * Matches any element 64

65 XPath: Attribute Nodes Result: means that price is has to be an attribute 65

66 XPath: Qualifiers /bib/book/author[firstname] Result: <author> <first-name> Rick </first-name> <last-name> Hull </last-name> </author> 66

67 XPath: More Qualifiers /bib/book/author[firstname][address[//zip][city]]/lastname Result: <lastname> </lastname> <lastname> </lastname> 67

68 XPath: More Qualifiers < 60 ] /bib/book[author/@age < 25 ] /bib/book[author/text()] 68

69 XPath: More Details We can navigate along 13 axes: ancestor ancestor-or-self attribute child descendant descendant-or-self following following-sibling namespace parent preceding preceding-sibling self 69