Week 2: Lecture Notes. DTDs and XML Schemas

Transcription

1 Week 2: Lecture Notes DTDs and XML Schemas In Week 1, we looked at the structure of an XML document and how to write XML. I trust you have all decided on the editor you prefer. If not, I continue to recommend Notepad++ or Vim. We did discuss DTDs for validating XML and in the header of many XML documents, we see a standard link to the DTD for the W3C. DTDs are simple and easy to write, but they use their own language and offer limited features. Schemas are a more powerful standard for validation: much more flexible, featurerich and written in XML. However, schemas are also much more complicated and there are other alternatives which we will look at in the assignments. For now, we will concentrate on DTDs and schemas, but first we must examine the reasons for validation. Why validate? There are many reasons to validate a document, and while validation of XML is optional, it s simply achieved with the document type declaration section of an XML document as discussed briefly last week. There may be cases where maintaining the structure of an XML document is vital, and in such cases we can use validation as a check on the document before it is accepted for processing. For example, in the pay_record from last week, payroll data is vital for the smooth functioning of an organisation. If such data is stored in XML format (and increasingly XML-formatted documents play a large part in electronic data interchange), then it is desirable to validate such documents before they are processed. The XML documents are validated by an XML parser, and any documents not conforming to some specified structure can be flagged as invalid before they are routed from the parser to an application for processing. Data which is found to be invalid or corrupted is cheaper to fix if it is discovered before it is processed, rather than halfway through a processing cycle of a critical application. Discovery at that later stage could involve complicated back-out procedures or costly data restores. Also, if the data structure and verification can be handled by an XML parser, then the application which processes it can be confident of the data it s receiving, thus reducing the need for costly checking within the application itself. With the increase of e-commerce applications, business-to-business data is passed over the Internet from one application to another. Obviously, there is room for data corruption at many stages along the way, and validation by XML parsers can help maintain the integrity of the data flowing through such e-commerce systems. Such validation forms a vital link between systems. 1/19

2 Document Type Definitions A DTD can be contained in two places: within the document type declaration of the XML document itself (an internal DTD), or within an external file indicated in the document type declaration. Let s discuss a simple example showing both methods. Recall from the previous week that a document type declaration looks like the following in an XML document: <!DOCTYPE root-element SYSTEM "file" [ ] > If an XML document contains an internal DTD, then the document type declaration looks like this, with the DTD contained in between the square brackets "[ ]" after the root-element: <!DOCTYPE root-element [ DTD specification ] > If an XML document has an external specification, then the document type declaration looks like this, where the name of the file containing the DTD specification appears after the keyword SYSTEM: <!DOCTYPE root-element SYSTEM "file-name" > Now the name of the file can also contain path information, and is a relative path from the current XML document to the DTD specification. If we take a small portion of the pay-record from last week say the address subcomponent and make it into an XML document, we would have the following, which we can use for a small example (see Listing 1): <?xml version="1.0"?> Listing 1: address.xml <address> <street>12 City Road</street> <city>manchester</city> <state>england</state> <zip>m70 2PP</zip> </address> Now we can add an internal DTD to address.xml to allow verification of the structure. This is shown in Listing 2 below: 2/19

3 Listing 2: address_internal.xml <?xml version="1.0"?> <!DOCTYPE address [ <!ELEMENT address (street, city, state, zip)> <!ELEMENT street (#PCDATA)> <!ELEMENT city (#PCDATA)> <!ELEMENT state (#PCDATA)> <!ELEMENT zip (#PCDATA)> ] > <address> <street>12 City Road</street> <city>manchester</city> <state>england</state> <zip>m70 2PP</zip> </address> The same XML document with an external DTD specification is shown in Listing 3: Listing 3: address_external.xml <?xml version="1.0"?> <!DOCTYPE address SYSTEM "address.dtd"> <address> <street>12 City Road</street> <city>manchester</city> <state>england</state> <zip>m70 2PP</zip> </address> Notice the DTD has now been moved to an external file called address.dtd. The content of address.dtd is shown in Listing 4. Listing 4: address.dtd <!ELEMENT address (street, city, state, zip)> <!ELEMENT street (#PCDATA)> <!ELEMENT city (#PCDATA)> <!ELEMENT state (#PCDATA)> <!ELEMENT zip (#PCDATA)> We will discuss the advantages of internal versus external DTD later. For now, we need to look closely at the DTD specification as shown in Listing 4. 3/19

4 DTD element syntax What we will discuss in this section is the basics of the DTD syntax, and cover the more common elements of the DTD specification. Your textbook and the Web references below provide a more detailed coverage. Since a DTD is used to define the structure of an XML document, the basic building blocks of DTD documents are also represented in the basic building blocks of XML documents, which are: elements, tags, attributes, entities, PCDATA and CDATA. Elements are the main building blocks of both XML and XHTML documents. Examples of XHTML elements are body and table. Examples of XML elements could be address and name. Elements can contain text, other elements or be empty. Tags are used to mark up elements. A starting tag like <name> marks up the beginning of an element, and an ending tag like </name>, marks up the end of an element. Attributes provide extra information about elements. Attributes are always placed inside the starting tag of an element. Attributes always come in name/value pairs. The following salary element has additional information about currency of the salary: <salary currency="gbp">32000</salary> Now the DTD in Listing 4 contains only ELEMENT declarations because the XML document it is defining in Listing 3 contains only XML elements in a fairly simple structure. Each of these XML elements is specified in the DTD with an element declaration. An element declaration has the following syntax: <!ELEMENT element-name content-model > The element-name is the name of the tag used to mark up the associated element in the XML document. The content-model specifies the model (or rules) of what the content of the XML element looks like. For each element of the XML document, there is a corresponding element declaration in the DTD (usually). The content-model can specify content or structure. For example, in the XML document in Listing 3, the body of the document contains the specification of the address element: 4/19

5 <address> <street>12 City Road</street> <city>manchester</city> <state>england</state> <zip>m70 2PP</zip> </address> where address is the root element in the document. The root element contains further structure and any element declaration in the DTD must reflect this. Thus, the first element declaration in the DTD is: <!ELEMENT address (street, city, state, zip)> This specifies the structure of the address element (the root element) as containing the subelements street, city, state and zip. Each of these elements contains no further subelements, thus in the corresponding element declarations, the contentmodel specifies content: <!ELEMENT street (#PCDATA)> <!ELEMENT city (#PCDATA)> <!ELEMENT state (#PCDATA)> <!ELEMENT zip (#PCDATA)> For element declarations which specify structure, there must follow element declarations for each of the subelements declared. Now in the content element declarations above, street, city, state and zip are declared with the content #PCDATA. This stands for parsed character data. For example, in the content of the <street> tag in the sample that follows, 12 City Road is normal character data that is parsed by the XML parser, spaces and all: <street>12 City Road</street> The following Table 1 identifies the possible declarations types for content and structure in element declarations. 5/19

6 Table 1: Structure/content options ELEMENT declaration #PCDATA Content EMPTY ANY sequences Structure either/or mixed content Parsed character data Element can be defined to contain no data Element can be defined to contain any content, including other elements Specifies a sequence of subelements Specifies alternatives for subelements Specifies both content and structure As well as defining content as #PCDATA, we can define empty content for an element. Empty elements are declared with the EMPTY keyword as follows, for the <married /> tag from the pay_record XML document from Week 2: <!ELEMENT married EMPTY > Such tags contain no data; just their very existence implies data. Elements declared with the category keyword ANY can contain any combination of parsable data. The example that follows says that the element position can contain any format, both content and structure, as long as it s well-formed: <!ELEMENT position ANY > This rule does tend to defeat the purpose of validating an XML document. <!ELEMENT address (street, city, state, zip)> The example above shows an element declaration for a structure sequence. That is, the element address consists of a sequence of subelements, street, city, state and zip. We can also declare the structure with an either/or type structure. This means that we can specify alternatives for structure. For example, in the address element, we may allow the pay_record to have either a street address or a post office box number. We can specify these alternatives as: 6/19

7 <!ELEMENT address ((street pobox), city, state, zip)> <!ELEMENT street (#PCDATA)> <!ELEMENT pobox (#PCDATA)> <!ELEMENT city (#PCDATA)> <!ELEMENT state (#PCDATA)> <!ELEMENT zip (#PCDATA)> The pipe symbol is used to indicate either one or the other. Thus, either of the following XML document address elements is valid if the above DTD is specified. or <address> <street>12 City Road</street> <city>manchester</city> <state>england</state> <zip>m70 2PP</zip> </address> <address> <pobox>14428 White City</pobox> <city>melbourne</city> <state>victoria</state> <zip>8001</zip> </address> Finally, we can also specify a mixed-mode content/structure with something structured as follows: <!ELEMENT workdays ( #PCDATA (day+) )> <!ELEMENT day (#PCDATA) > This specifies that the workdays element can contain either directly parsable character data, or further structure in the form of day child elements. Thus, the above DTD declarations will validate either of the forms below. or <workdays> N/A </workdays> <workdays> <day>monday</day> <day>wednesday</day> <day>friday</day> </workdays> Note the use of the + symbol in the DTD declaration above. It stands for one or more. A list of possible symbols and their use is given in Table 2 below. 7/19

8 DTD attribute syntax Table 2: Symbols and their uses Symbol Definition * Data will appear zero or more times Comma, Provides for separation of items Parentheses Used to contain the content-model for an () element, and used for grouping Separates possible choices + Data will appear one or more times? Data will appear either zero or one times Now as well as providing information to check the validation of XML elements in the document, the DTD also lets us specify information to validate and check attributes information within the elements. Recall that an attribute is declared in the XML document as: <salary currency="gbp">32000</salary> The attributes in the XML document are validated with attribute list declarations in the DTD. A corresponding declaration for the above XML element would be: <!ATTLIST salary currency CDATA #IMPLIED > In the above ATTLIST declaration, salary is the corresponding element that the attribute belongs to, and currency is the name of the attribute. CDATA is the type of data in this case, character-data and the #IMPLIED indicated that the value of the attribute is optional. The general form of the attribute-list declaration is <!ATTLIST element-name attribute-name attribute-type default-behaviour default-value > The element-name is the name of the element to which the attribute belongs. The attribute-name is the name of the attribute. The attribute-type specifies the type of the attribute which can be one of about 10 different types. The default-behaviour is one of three different types dictating how the parser treats the attribute, and the default-value is just a value which acts as a default for the attribute in the absence of a supplied value. Although there are 10 different attribute-types, the only one we discuss here is the CDATA. For the rest, please refer to the relevant table in this week s textbook reading. CDATA allows a valid stream of character data to be assigned to the attribute, as in AUD in the above example. The default-behaviour can be one of three types: 8/19

9 #REQUIRED means that the value of the attribute must be specified; #IMPLIED means that the value of the attribute is optional and need not be specified; and #FIXED means that the value is optional, but if specified, must have a specific value. The default-behaviour may be omitted. Also, a default-value can be specified. For example, in the salary element above, if the currency attribute is omitted (if it s optional), we may want to specify the default value as USD, indicating the currency type is in US dollars. We would specify this as: <!ATTLIST salary currency CDATA #IMPLIED "USD" > More detailed information can be found in the textbook in this week s textbook reading. You may care to alter Listing 1 above and change <city> to <town> and similarly </city> to </town> and watch what happens. If you are using Notepad++, then it will object to the change, but will still allow the save. Browsers will ignore the errors; browsers are designed to be tolerant. However, the validator at will identify the error. This illustrates the need for a formal validation: Editors will save an errant file; browsers will display it. However, if we are trying to read from or write to a database on the server, then the program will fail. Internal versus external DTDs We originally talked about internal versus external DTDs. An external DTD allows a user to provide a verification tool for a whole range of XML applications that want to use a standard format to pass data between them. The use of internal DTDs allows an XML document to define its own structure. In this case, the structure of the document can be passed to an application along with the data as well. This type of structure verification is good for applications that are not rigid in structure and perhaps accept a wide range of different types of data. In such cases, no predefined structure is required of the application, and the structure can be determined at the time the document is parsed. However, for the correct validation of data that is passed between more formal applications, such as e-business systems, an external DTD provided by the application builders provides for verification that is more formal and is able to better filter out data that would cause the application to fail. 9/19

10 XML Schemas XML schemas are a much more extensive specification, so of necessity, we can only provide an introduction in this week. As indicated earlier, there are some limitations in the DTD specification just previously covered. Indeed, apart from specifying data to be #PCDATA, we can do little other validation. XML Schemas have been designed to go beyond the capabilities of DTD to provide a far more comprehensive possible validation of an XML document. With XML Schemas, there is an objectoriented type thinking associated with it, insofar as XML documents are considered instances of an XML Schema. Thus, an XML Schema is considered a class from which the instances (XML documents) are derived. XML Schemas are also different in that they use XML syntax to define the schema document. This is different than the DTD declarations that we have just seen. XML Schema language is also referred to as XML Schema Definition (XSD), thus we will be discussing XSD documents. The purpose of an XML Schema is to define the valid building blocks of an XML document similar to DTDs already covered. XSD documents: define elements that can appear in a document; define attributes that can appear in a document; define which elements are child elements; define the order of child elements; define the number of child elements; define whether an element is empty or can include text; define data types for elements and attributes; and define default and fixed values for elements and attributes. Let s look at a simple schema for the address.xml file in Listing 1. 10/19

11 Listing 5: address.xsd <xsd:schema xmlns:xsd=" > <xsd:element name="address"> <xsd:complextype> <xsd:sequence> <xsd:element name="street" type="xsd:string"/> <xsd:element name="city" type="xsd:string"/> <xsd:element name="state" type="xsd:string"/> <xsd:element name="zip" type="xsd:integer"/> </xsd:sequence> </xsd:complextype> </xsd:element> </xsd:schema> The above XML schema is a fairly simple schema definition for the address.xml file. The XML Schema is itself stored in a file with an extension of.xsd, in this case, address.xsd. Let s look a little more closely at the schema definition above. The schema is in the format of an XML document, which always begins with the root element schema. Notice that all the elements in the schema definition are prefixed with xsd:, which is associated with the XML Schema namespace through the declaration of the schema element: <xsd:schema xmlns:xsd=" > The prefix xsd: is used by convention to denote the XML schema namespace, although any prefix can be used, and in many cases you will see examples using the xs: prefix. The same prefix, and hence the same association, also appears on the names of all the elements which are part of the schema definition used. In this case, the schema recommendation 2001 is located at Schema. The purpose of the association is to identify the elements and simple types as belonging to the vocabulary of the XML Schema language rather than the vocabulary of the schema author. Recall from last week that while the namespace identifies a unique URI, the actual file associated with the XML Schema namespace is not physically searched by an XML validator. After the schema element is specified, the root element of the XML document is then specified using the schema element declaration element. In a schema, XML elements can be specified as either complex elements or simple elements. We will briefly discuss complex types first, as the root element in an XML document is usually a complex type as in Listing 5 above. Complex schema elements A complex element is an XML element that contains other elements and/or attributes. There are four kinds of complex elements: empty elements; elements that contain only other elements; 11/19

12 elements that contain only text; and elements that contain both other elements and text. Each of these elements may contain attributes as well. We can define a complex element in an XML schema in different ways: The address element can be declared directly by naming the element, like this: <xsd:element name="address"> <xsd:complextype> <xsd:sequence> <xsd:element name="street" type="xsd:string"/> <xsd:element name="city" type="xsd:string"/> <xsd:element name="state" type="xsd:string"/> <xsd:element name="zip" type="xsd:integer"/> </xsd:sequence> </xsd:complextype> </xsd:element> If we use this method, only the address element can use the specified complex type (this is generally the method we ll use in these weeks). Notice that the child elements street, city, state and zip are surrounded by the <sequence> element. This means that the child elements must appear in the same order as they are declared: street, city, state and zip. Alternatively, the address element can have a type attribute that refers to the name of the complex type to use: <xsd:element name="address" type="addresstype" / > <xsd:complextype name="addresstype"> <xsd:sequence> <xsd:element name="street" type="xsd:string"/> <xsd:element name="city" type="xsd:string"/> <xsd:element name="state" type="xsd:string"/> <xsd:element name="zip" type="xsd:integer"/> </xsd:sequence> </xsd:complextype> If you use this method, several elements can refer to the same complex type. Now, we have already mentioned the use of the element sequence. Sequence is known as an indicator. Indicators are used to control how elements are to be used within XML documents. There are seven different types of indicators, grouped below into three categories (see appropriate textbook table from this week s reading for more details) order indicators 12/19

13 o all o choice o sequence occurrence indicators o maxoccurs o minoccurs group indicators o group name o attributegroup name The <all> indicator specifies by default that the child elements can appear in any order and that each child element must occur once and only once: <xsd:complextype> <xsd:all> : </xsd:all> </xsd:complextype> The <choice> indicator specifies that either one child element or another can occur <xsd:complextype> <xsd:choice> <xsd:element name="street" type=" xsd:string "/> <xsd:element name="pobox" type=" xsd:string "/> </xsd:choice> </xsd:complextype> The <sequence> indicator specifies that the child elements must appear in a specific order, as previously discussed. The behaviour of other indicators can be found in your textbook and Web references. Simple schema elements A simple element is an XML element that can contain only text. It cannot contain any other elements (children) or attributes. The syntax for defining a simple element is: <xs:element name="xxx" type="yyy"/> where "xxx" is the name of the element and "yyy" is the data type of the element. For example: <xsd:element name="city" type="xsd:string"/> Notice the type of the element above is declared as string, as opposed to PCDATA in DTDs. In XML Schemas, there is a wide choice of types from which to choose. For example, in Listing 5 we used integer for the zip element: 13/19

14 <xsd:element name="zip" type="xsd:integer"/> There are over 40 data types that you can choose from when assigning a data type to an element specification (see the appropriate table in this week s textbook reading for more items and detail), but the most common ones are: xsd:string; xsd:decimal; xsd:integer; xsd:float; xsd:boolean; xsd:date; and xsd:time Declaring attributes in XML As in DTDs, we can declare attributes belonging to the XML elements. Simple elements cannot have attributes. If an element has attributes, it is considered to be of complex type. However, the attribute itself is always declared as a simple type. This means that an element with attributes always has a complex type definition. The syntax for declaring an attribute (as per simple type) is: <xsd:attribute name="xxx" type="yyy"/> where "xxx" is the name of the attribute and "yyy" is the data type of the attribute. The data type is chosen from the same list of data types as per simple types. For example, to declare the attribute currency for the salary element in the XML document from Week 1, <salary currency="gbp">32000</salary> we would use: <xsd:element name="salary"> <xsd:complextype> <xsd:simplecontent> <xsd:extension base="xsd:integer"> <xsd:attribute name="currency" type="xsd:string" /> </xsd:extension> </xsd:simplecontent> </xsd:complextype> </xsd:element> Referencing and XML Schema from XML To reference a DTD from an XML document, we used the <!DOCTYPE > declaration as shown in Listing 3. However, to reference an XML Schema that defines the structure of the current XML document is quite different. If we take the 14/19

15 XML Schema as shown in Listing 5 to define and verify instance documents, then an instance document that references the XML Schema is: Listing 6: Referencing XML Schemas <?xml version="1.0"?> <address xmlns:xsi=" xsi:nonamespaceschemalocation=" ddress.xsd" > <street>12 City Road</street> <city>melbourne</city> <state>victoria</state> <zip>8001</zip> <country region="4">australia</country> </address> The corresponding file address.xsd is shown below Listing 7: The schema for Listing 6 <xsd:schema xmlns:xsd=" <xsd:element name="address"> <xsd:complextype> <xsd:sequence> <xsd:element name="street" type="xsd:string"/> <xsd:element name="city" type="xsd:string"/> <xsd:element name="state" type="xsd:string"/> <xsd:element name="zip" type="xsd:integer"/> <xsd:element name="country"> <xsd:complextype> <xsd:simplecontent> <xsd:extension base="xsd:string"> <xsd:attribute name="region" type="xsd:integer"/> </xsd:extension> </xsd:simplecontent> </xsd:complextype> </xsd:element> </xsd:sequence> </xsd:complextype> </xsd:element> </xsd:schema> The main difference you ll notice is the referencing information to the XML Schema that is contained as namespace attributes in the root element of the XML document, in this case, address. The line xmlns:xsi=" 15/19

16 makes XMLSchema-instance namespace available with the prefix xsi. The line that follows uses the nonamespaceschemalocation attribute to tell the XML validator where to find the associated XML Schema: xsi:nonamespaceschemalocation=" ddress.xsd" > In this case, we have used the nonamespaceschemalocation since we haven t set up a default namespace for the XML document. The location of the XML Schema in this case would be on the Laureate Student server at URL We could have set up a default namespace, and then used the schemalocation attribute to specify where the schema was located. This attribute takes two values: the default namespace and the location of the schema. This is discussed in more detail in your text. Verifying XML and XML Schema documents Just as with DTDs, you ll need an XML parser that implements the XML Schema standard to see the results of all these things we have discussed. Microsoft XML Notepad mentioned in Week 1 won t do this effectively. However, there are XML parsers which do implement the XML Schema standard we have been discussing. Some of these are open source, and a list of available utilities is available from the XML Schema home page, referenced in the Web references below. One such validator is the XSV XML online validator. With this tool, you can supply a URL to the XML document, which in turn contains a URL to the XML Schema used, and the tool verifies the XML document online and returns the results as a Web page. The URL to this tool is supplied in the Web references below. You can try verifying the XML document shown in Listing 6 above. Copy listings 6 and 7 above into the validator at DTDs versus XML Schemas Now, we have briefly discussed both XML Schema and DTDs for validating XML documents. As the XML Schema specification is extensive, only a very brief introduction has been given. However, as is obviously indicated, defining an XML Schema for a document is a complicated task. In contrast, DTDs are relatively simple. Yet, despite the complexity, XML Schemas give the user much greater control over the format of the document, and the specification of the types of data that the XML document can use. This allows the user to specify the XML structure and allowable data in far more detail, thus allowing for the possibility of greatly reducing undetected errors creeping into documents as they pass through various applications. A few points to make on the subject are: XML Schemas are the successors of DTDs. 16/19

17 XML Schemas are extensible to future additions. XML Schemas are richer and more useful than DTDs. XML Schemas are written in XML. XML Schemas support data types. XML Schemas support namespaces. One of the greatest strengths of XML Schemas is the support for data types. With data type support in XML Schemas, it is easier to describe permissible document content; it is easier to validate the correctness of data; it is easier to work with data from a database; it is easier to define data facets (restrictions on data); it is easier to define data patterns (data formats); and it is easier to convert data between different data types. Another great strength about XML Schemas is that they are written in XML; thus, you don t have to learn another language; you can use your XML editor to edit your schema files; you can use your XML parser to parse your schema files; you can manipulate your schema with the XML DOM; and you can transform your schema with XSLT. XML Schemas help secure data communication. When data is sent from a sender to a receiver, it is essential that both parts have the same expectations about the content. With XML Schemas, the sender can describe the data in a way that the receiver will understand. For example, a date like might (in some countries) be interpreted as 3rd November or (in some other countries) as 11th March, but an XML element with a data type such as: <date type="date"> </date> ensures a mutual understanding of the content because the XML data type date requires the format YYYY-MM-DD. This is, of course, the international standard ISO Summary This week has briefly covered both DTD and XML Schema for XML document validation. Without the use of either of these mechanisms for validation, an XML parser only requires that XML documents be well-formed. However, being wellformed is not enough. A well-formed XML document is a document that only conforms to the XML syntax rules. That is, it must begin with the XML declaration; it must have one unique root element; all start tags must match end tags; XML tags are case sensitive; 17/19

18 all elements must be closed; all elements must be properly nested; and all attribute values must be quoted. However, even if documents are well-formed, they can still contain errors, and those errors can have serious consequences. A simple example might be ordering two containers of parts instead of two parts. With DTDs and XML Schemas, most of these errors can be caught by your validating software. In particular, XML Schemas are based on a new standard designed to supersede DTDs. Although the XML Schema standard is far more complex and extensive than DTDs, it provided for a far more extensive validation. However, due to the simplicity of the DTDs, it will most likely continue to be used for those only requiring simple validation of the XML document. 18/19

19 DTD references DTD Tutorial An Introduction to XML Document Type Definitions XML DTDs and valid XML documents Schema references XML Schema Tutorial Using W3C XML Schema Schematron, a Schema alternative Schematron Relax NG, another Schema alternative Specifications XML Schema Part 0: Primer XML Schema home page (includes a list of utilities) Validators XSV, an open source XML Schema validator ftp://ftp.cogsci.ed.ac.uk/pub/xsv/xsv31.exe XSV XML validator, online form based interface FreeFormatter, XML validator 19/19