Semistructured data, XML, DTDs

Size: px

Start display at page:

Download "Semistructured data, XML, DTDs"

Hilda Hines
6 years ago
Views:

1 Semistructured data, XML, DTDs Introduction to Databases Manos Papagelis Thanks to Ryan Johnson, John Mylopoulos, Arnold Rosenbloom and Renee Miller for material in these slides

2 Structured vs. unstructured data 2 Databases are highly structured Well-known data format: relations and tuples Every tuple conforms to a known schema Data independence? Woe unto you if you lose the schema Plain text is unstructured Cannot assume any predefined format Apparent organization makes no guarantees Self-describing: little external knowledge needed... but have to infer what the data means

3 Semi-structured data 4 Observation: most data has some structure Text: sentences, paragraphs, sections,... Books: chapters Web pages: HTML Idea of semistructured data: Enforce well-formatted data => Always know how to read/parse/manipulate it Optionally, enforce well-structured data also => Might help us interpret the data, too Pro: highly portable Con: verbose/redundant

4 XML 6

5 XML: designed for data interchange 7 <books search-terms= database+design > <book> <title>database Design for Mere Mortals </title> <author>michael J. Hernandez</author> <date>13/03/2003 </date> </book> <book id= B2 > <title>beginning Database Design</title> <subtitle>from Novice to Professional</subtitle> <author>clare Churcher</author> </book> </books>

6 Features of XML 8 Intentionally similar syntax to HTML Tree-structured (hierarchical) format Elements surrounded by opening and closing tags Attributes embedded in opening tags => <tag-name attr-name= attr-value >data</tag-name> But with important differences Strictly well-formed (must close all tags, etc.) Tag/attribute names carry no semantic meaning Data-only format: no implied presentation Valid identifiers (for elements and attributes) [a-z][a-z] Both descendants of SGML

7 XML terminology 9 <?xml version= 1.0?> attributes <PersonList Type= Student Date= > <Title Value= Student List /> <Person> </Person> <Person> </Person> </PersonList> elements Empty element Root element Element (or tag) names Elements are nested Root element contains all others

8 Content of Person XML terminology (cont.) Opening tag <Person Name = John Id = s > John is a nice fellow <Address> <Number>21</Number> <Street>Main St.</Street> </Address> </Person> standalone text, not very useful as data, non-uniform Child of Address, Descendant of Person Nested element, child of Person Parent of Address, Ancestor of Number Closing tag: What is open must be closed

9 Example XML Document 11 <?xml version= 1.0?>  <Students> <Student StudId= > <Name><First>John</First><Last>Doe</Last></Name> <Status>U2</Status> <CrsTaken CrsCode= CS308 Semester= F1997 /> <CrsTaken CrsCode= MAT123 Semester= F1997 /> </Student> <Student StudId= > <Name><First>Bart</First><Last>Simpson</Last></Name> <Status>U4</Status> <CrsTaken CrsCode= CS308 Semester= F1994 /> </Student> </Students>

10 XML Document is a Tree 12

11 Two kinds of XML Documents 13 Well-Formed XML Just need to use proper nesting Can invent your own tags Any tag can go anywhere Valid XML Can invent tags, but you define what they are and where they can go A DTD (document type definition) specifies these rules

12 Rules for well-formed XML Must have a root element Every opening tag must have matching closing tag Elements must be properly nested <foo><bar></foo></bar> is a no-no An attribute name can occur at most once in an opening tag. If it occurs: It must have an explicitly specified value (Boolean attrs, like in HTML, are not allowed) The value must be quoted (with or ) XML processors not allowed to fix ill-formed documents (HTML browsers expected to!)

13 XML, text, and whitespace 16 Adjacent non-tag chars parsed as text nodes Parser never ignores whitespace Leading and trailing space left with its text node Whitespace between tags produces empty text nodes Example: <foo> hi<bar> ho </bar> </foo> foo hi bar \n ho \n

14 Example: Well-Formed XML 17 <?xml version = 1.0 standalone = yes?> <BARS> <BAR><NAME>Joe s Bar</NAME> <BEER><NAME>Bud</NAME> Root tag <PRICE>2.50</PRICE></BEER> <BEER><NAME>Miller</NAME> <PRICE>3.00</PRICE></BEER> </BAR> <BAR> </BAR> </BARS> Tags surrounding a BAR element A NAME subelement A BEER subelement Nesting rule for tags must be obeyed

15 Checking your XML 18 xmllint command on cdf. By default, checks if well-formed --debug Outputs an annotated tree of the parsed document

16 Problems with well-formed XML 19 If a program will process XML, good to know things like: What tags are allowed What order, nesting What attributes for each tag What s mandatory or optional A DTD specifies exactly this

17 DOCUMENT TYPE DEFINITION (DTD) 20

18 Document type definition (DTD) 21 Enforces more than well-formed -ness Which entities may (or must) appear where Attributes entities may (or must) have Types attributes and data must adhere to DTD is separate from XML it constraints May be embedded in separate section Most often linked (referenced externally) Validation: checking XML against its DTD(s) Important for interpreting/validating data Not necessary for parsing

19 DTD building blocks 22 Elements (<an-element>...</an-element>) Must always close tags If no contents: <empty-element/> Attributes (<... an-attr=......>) Text PCDATA (parsed character data) Mixed text and markup Use entities to escape >, etc. which should not be parsed CDATA ([non-parsed] character data) Plain text data Tags not parsed, entities not expanded Entities ( special tokens) e.g. < > & " ' HTML defines lots of others (e.g. )

20 DTD elements 23 <!ELEMENT $e...> $e is the element name " " may contain any of: Nothing: <!ELEMENT $e EMPTY> Anything: <!ELEMENT $e ANY> Text data: <!ELEMENT $e (#PCDATA)> Always parsed (#CDATA not allowed here) Child elements: <!ELEMENT $e (...)> Any child referenced must also be declared Child elements may themselves have children Mixed content: <!ELEMENT $e (#PCDATA...

21 DTD elements: children 24 Base construct: sequence (,) <!ELEMENT $e (a)> <!ELEMENT $e (a, b, c,...)> Comma ","defines order of which children must appear in XML Either-or content ( ) <!ELEMENT $e (a b...)> Exactly one of the options must appear in the XML Constraining child cardinality <!ELEMENT $e (a, b+, c*, d?)> not followed by any of +, *,? : exactly one (e.g., a) +: at least one (e.g., b) *: zero or more (e.g., c)?: at most one (e.g., d)

22 DTD elements: Example 25 <!ELEMENT resume ( bio,interests,education, experience,awards,service)> <!ELEMENT bio ( name, addr, phone, ?, fax?, url?)> <!ELEMENT interests (interest+)> <!ELEMENT education (degree*)> <!ELEMENT awards ((award honor)*)>... Sequences and either-or can both nest

23 DTD elements: Another Example 26 <!DOCTYPE BARS [ ]> <!ELEMENT BARS (BAR*)> <!ELEMENT BAR (NAME, BEER+)> <!ELEMENT NAME (#PCDATA)> <!ELEMENT BEER (NAME, PRICE)> <!ELEMENT PRICE (#PCDATA)> NAME and PRICE are text A BARS object has zero or more BAR s nested within A BAR has one NAME and one or more BEER subobjects A BEER has a NAME and a PRICE

24 DTD Attributes 27 <!ATTLIST $e $a $type $required> Declares an attribute $a on element $e $type may be any of character data: CDATA one of a set of values: (v1 v2...) unique identifier: ID references to one/many ID token(s) of other attributes: IDREF[S] valid xml name (or list of names): NMTOKEN[S] entity (or entities): ENTITY/ENTITIES $required may be required (not required): #REQUIRED (#IMPLIED) fixed value (always the same): #FIXED $value default value (used if none given): $value

25 DTD attributes: examples 28 <!ATTLIST person sin ID #REQUIRED spouse IDREF #IMPLIED name CDATA John Doe trusted (yes no) no species #FIXED homo sapiens alive (yes no) #IMPLIED > #IMPLIED unless specified otherwise

26 DTD attributes: ID[REF][S] 29 ID attribute type Uniquely identifies an element in the document (like keys) Error to have two Like HTML id attribute, but can have any name IDREF Refers to another element by ID (like foreign keys) Error if corresponding ID does not exist Like HTML href attribute, but no # needed IDREFS List of IDREF attributes, space-separated Problem: only one global set of IDs

27 Example: The DTD 30 <!DOCTYPE BARS [ <!ELEMENT BARS (BAR*, BEER*)> <!ELEMENT BAR (SELLS+)> <!ATTLIST BAR name ID #REQUIRED> <!ELEMENT SELLS (#PCDATA)> <!ATTLIST SELLS thebeer IDREF #REQUIRED> <!ELEMENT BEER EMPTY> <!ATTLIST BEER name ID #REQUIRED> <!ATTLIST BEER soldby IDREFS #IMPLIED> ]>

28 Example: The XML Document 31 <BARS> <BAR name = JoesBar > <SELLS thebeer = Bud >2.50</SELLS> <SELLS thebeer = Miller >3.00</SELLS> </BAR> <BEER name = Bud soldby = JoesBar SuesBar /> </BARS>

29 DTD entities 32 The XML equivalent of #define <!ENTITY $name $substituted-value > Can t take parameters, though Used just like other entities <politician-speak> I vow to lead the fight to stamp out &buzz-word; by instituting powerful new programs that will... </politician-speak> Pick your favorite substitution: <!ENTITY buzz-word communism > <!ENTITY buzz-word racism > <!ENTITY buzz-word terrorism > <!ENTITY buzz-word illegal file sharing > Not heavily used: better templating methods exist

30 Embedded vs. External DTD Specified as part of a document <?xml version= 1.0?> <!DOCTYPE Book [ ]> <Book> </Book> Reference to external (stand-alone) DTD <?xml version= 1.0?> <!DOCTYPE Book > <Book> </Book>

31 EXAMPLE: Emdedded DTD 34 <?xml version = 1.0 standalone = no?> <!DOCTYPE BARS [ <!ELEMENT BARS (BAR*)> <!ELEMENT BAR (NAME, BEER+)> The DTD <!ELEMENT NAME (#PCDATA)> <!ELEMENT BEER (NAME, PRICE)> <!ELEMENT PRICE (#PCDATA)> ]> <BARS> <BAR><NAME>Joe s Bar</NAME> <BEER><NAME>Bud</NAME> <PRICE>2.50</PRICE></BEER> <BEER><NAME>Miller</NAME> <PRICE>3.00</PRICE></BEER> </BAR> <BAR> </BARS> The document

32 EXAMPLE: External DTD 35 Assume the BARS DTD is in file bar.dtd: <?xml version = 1.0 standalone = no?> <!DOCTYPE BARS SYSTEM bar.dtd > <BARS> <BAR><NAME>Joe s Bar</NAME> <BEER><NAME>Bud</NAME> <PRICE>2.50</PRICE></BEER> <BEER><NAME>Miller</NAME> <PRICE>3.00</PRICE></BEER> </BAR> <BAR> </BARS> Get the DTD from the file bar.dtd

33 Limitations of DTDs 36 Don t understand namespaces Very limited typing (just strings and xml names) Very weak referential integrity All ID / IDREF / IDREFS use global ID space Can t express unordered contents conveniently How to specify that a,b,c must all appear, but in any order? All element names are global Is <name> for people or companies? can t declare both in the same DTD

34 XML Schema 37 Designed to improve on DTDs Advantages: Integrated with namespaces Many built-in types User-defined types Has local element names Powerful key and referential constraints Disadvantages: Unwieldy, much more complex than DTDs We won t cover XML schema in class

35 What is Next? 38 XML Query Languages XPATH XQUERY

XML. Document Type Definitions XML Schema. Database Systems and Concepts, CSCI 3030U, UOIT, Course Instructor: Jarek Szlichta

XML. Document Type Definitions XML Schema. Database Systems and Concepts, CSCI 3030U, UOIT, Course Instructor: Jarek Szlichta XML Document Type Definitions XML Schema 1 XML XML stands for extensible Markup Language. XML was designed to describe data. XML has come into common use for the interchange of data over the Internet.