READY, SET, XML! USING ORACLE XML DATA John Jay King, King Training Resources

Transcription

1 READY, SET, XML! USING ORACLE XML DATA, King Training Resources Oracle and XML XML has become the "Esperanto" of the IT industry quickly. Oracle has been at the forefront of this trend incorporating XML documents into its database products. XML data may be stored in the Oracle database in two ways: Structured ("shredded" into tables and columns) and Unstructured (XML document). In this session attendees will learn how to store/maintain/retrieve XML data in the Oracle database in both Structured and Unstructured forms. Use of the XML DB WebDAV interface will be explored to show how Oracle data may easily be shared with non-oracle applications. Oracle devotes two manuals to the use of XML in the database: "XML Database Developer's Guide - Oracle XML DB" and "XML API Reference - XDK and Oracle XML DB" both are included in the online Oracle Technet documentation. XML-related features have been greatly enhanced in both Oracle 10g and Oracle9i Release 2 and have been re-badged as "XML DB" in Oracle's documentation. XML DB allows native XML database activity in addition to the standard ORDBMS functionality. Oracle allows XML documents to be stored in the database in two ways: Unstructured XMLType where XML is stored as a string of bytes in a CLOB database object, and Structured XMLType with the XML document shredded into relational objects in the ORDBMS (SQL 99 standard objects). XMLtype Datatype The XMLType datatype was introduced with Oracle9i to provide an alternative to storing XML documents as CLOB data. In Oracle9i Release 1 the "SYS" schema needed be used whenever referencing SYS.XMLtype data. The need for specifying the SYS schema is eliminated in Oracle9i Release 2 and later releases. Oracle's XMLType is object-oriented and provides several member functions including: createxml (Create XMLType instance), existsnode (Checks if XPath can find valid node), extract (Uses XPath to return XML fragment), isfragment (Checks if document is a fragment), getclobval (Gets document as a CLOB), getstringval (Gets value as a string), and getnumberval (Gets numeric value as a number). XML Functions In addition to the functions added to support the XMLType Oracle provides several other XML-related functions. Some of Oracle's XML functions were part of the original XML inclusion effort and are Oracle-specific. Later additions support the proposed ISO/ANSI "SQLX" standard for XML database functions. SQL provides several functions specifically for dealing with XML data including: SYS_DBURIGEN(ts) to generate DBURITYPE URL used to obtain XML data from the database; SYS_XMLGEN(exp) to convert specified database row and column into an XML document; SYS_XMLAGG(exp) to generate single XML document from aggregate of XML data specified by "exp"; XMLELEMENT(name,exp) generates XML element using name and expression as data; and XMLATTRIBUTES(exp,list) generates XML attributes using expression. XMLELEMENT and XMLATTRIBUTES illustrate Oracle s support of the ANSI/ISO SQLX standard. SYS_XMLGEN SYS_XMLGEN uses a single input expression representing a particular row/column (scalar value or user-defined type) and a single XML element representing scalar values is returned containing XML elements representing each of a user-defined type s data items. The function returns an instance of SYS.XMLType data that is an XML document. The example below uses getstringval since SYS.XMLType data returns as CLOB and is not displayable by SQL*Plus: select sys_xmlgen(ename).getstringval() Name from emp where job = 'ANALYST' NAME <?xml version="1.0"?> <ENAME>FORD</ENAME> <?xml version="1.0"?> <ENAME>SCOTT</ENAME>

2 SYS_XMLAGG SYS_XMLAGG aggregates all XML documents (or fragments) in an expression to produce a single document. ROWSET is the default root element tag name used. The SYS.XMLGenFormatType function may be used to change a tag name if desired. The example below uses the SYS_XMLGEN function to generate an XML document (example uses getclobval since SYS.XMLType data returns as CLOB): select sys_xmlagg(sys_xmlgen(ename)).getclobval() emps from emp where deptno = 10 EMPS <?xml version="1.0"?> <ROWSET> <ENAME>KING</ENAME> <ENAME>CLARK</ENAME> <ENAME>MILLER</ENAME> </ROWSET> XMLELEMENT XMLELEMENT(name,exp) generates an XML element using the specified name and expression as data: select xmlelement("employee", xmlelement("empid",empno), xmlelement("empname",ename)) myxml from emp <employee> <empid>7369</empid> <empname>smith</empname> </employee> <employee> <empid>7499</empid> <empname>allen</empname> </employee> <employee> <empid>7521</empid> <empname>ward</empname> </employee> <employee> <empid>7566</empid> <empname>jones</empname> </employee> <employee> <empid>7654</empid> <empname>martin</empname> </employee> XMLATTRIBUTES Generates XML attributes using an expression list: select xmlelement("employee", xmlelement("emp", xmlattributes(empno as "empno", ename as "ename")), xmlelement("job",job), xmlelement("hiredate",hiredate), xmlelement("pay", xmlattributes(nvl(sal,0) as "sal", nvl(comm,0) as "comm")) ) as myxml from emp; <employee> <emp empno="7782" ename="clark"/> <job>manager</job> <hiredate>09-jun-81</hiredate> <pay sal="2450" comm="0"/> </employee> *** More like the above *** OTHER XML FUNCTIONS XMLColattval creates series of XML fragments using an element name of "column" and column names and values as attributes. XMLConcat concatenates a series of XMLType objects (opposite of XMLElement). XMLForest creates XML fragments from a list of arguments/parameters. XMLSequence creates a Varray of XMLType instances. XMLTransform uses input XMLType and XSL style sheet (also XMLType) to create a new XMLType. UpdateXML uses an XMLType and an XPATH reference and returns an updated XMLType. Structured XML Data With "structured XML" the XML document is shredded into database objects (relational columns and rows). Structured XML requires that documents must conform to a registered XML Schema. Oracle9i Release 2 and later releases allow the registration of an XML Schema to be used in validating and populating XML data. Structured Storage has several advantages: memory management is better than with CLOB, storage requirements are reduced, relational-style indexing is possible, and partial or in-place updates are possible. Because of the complexity involved in "shredding" and "unshredding" the data, adding and retrieving XML documents is slower then when using Structured Storage. XML Schemas are used to validate data entered into the database and used when formatting output. XML Schemas are powerful, but do not provide 2

3 some features we take for granted in the database such as UNIQUE key and FOREIGN key constraints. By default, Oracle does not completely validate documents as they are inserted into the database; restriction facets such as minlength, maxlength, and patterns are ignored in Oracle 9i R2, but Oracle 10g enforces more. Schema validation may be enabled for individual schemas via Check Constraints or Triggers. Complete XML Schemas may be registered with XML DB to validate documents and to define how documents will be stored by the database. XML Schemas are registered in the XML DB Repository. The XML DB Repository provides the mechanism for associated URIs with XPath notation to access XML data. The XML DB Repository also supports interaction with HTTP, FTP, and WebDAV clients. Unstructured XML Data XMLtype may be used to store an entire document (unparsed) in the database. This is generally quite a bit faster when loading and unloading database data, but slower when accessing data using traditional relational database tools. An unstructured XML document will generally require more space in the database than the equivalent data in an RDBMS structure. Unstructured XML documents are also more difficult to retrieve data from when using non-xml tools. XML Schema Support The W3C (World-Wide Web Consortium) XML Schema standard provides a middle ground between data modeling and document modeling. XML schemas may be used to automatically create tables and types, or to validate updates and inserts. XML schemas may be used as the basis for XMLType tables and columns (but, schemas are not required to store XMLType data). XML schemas must be registered in the database where they are stored as CLOBs; once registered, XML schemas may be referenced using URL notation. Registered XML schemas may be used to map XML documents to both structured and unstructured database storage. Beginning with Oracle9i Release 2 some of the XML schema related activity permitted includes: XMLType objects may be built based upon XML schema and XMLType objects may be validated using XML schema; tables may be created based upon XML schema automatically creating storage structures (does not require specific column definition), XML schemas may registered in the database using the DBMS_XMLSCHEMA package; registered schemas may be shared; schema registration may create Java beans and default tables too. Using Schemas allows pre-parsing ("shredding") of incoming XML documents and their direction to the appropriate tables. XML documents and instances may be validated using XMLType s XMLIsSchemaValid() method and the extractvalue() may be used to extract part of the XML document. REGISTERING SCHEMA Schemas must be created and tested (use an appropriate XML editor), then register them with DBMS_XMLSCHEMA. Here is an example of a PL/SQL block registering a Schema: begin dbms_xmlschema.registerschema('mybooks.xsd', '<?xml version="1.0" encoding="utf-8"?> <xs:schema xmlns:xs=" elementformdefault="qualified"> <xs:element name="mybooks"> <xs:complextype> <xs:sequence> <xs:element ref="book" maxoccurs="unbounded"/> **** rest of schema definition **** </xs:schema>',true,true,false,false); end; SCHEMA WITH XMLTYPE TABLES To create a table using the schema, type the following: create table mybookstype of xmltype xmlschema "mybooks.xsd" element "mybooks"; SCHEMA WITH XMLTYPE COLUMNS To create an XMLType column using the schema definition: create table mybooks (id number, books xmltype) xmltype column books xmlschema "mybooks.xsd" Element "mybooks"; 3

4 UNSTRUCTURED XML DATA XMLType data stored without a schema is unstructured and stored as CLOB. The first example shows the creation of a table using unstructured data and the second show a table with an unstructured XMLType column., CREATE TABLE xml_books OF XMLTYPE; CREATE TABLE xml_books ( books XMLTYPE ); UNSTRUCTURED INSERT Data inserted into an unstructured XMLType table or column is stored in CLOB form as shown below: INSERT INTO xml_books (books) VALUES (XMLTYPE(' <name>learning XML</name> <author>eric T. Ray</author> <publisher>o&apos;reilly</publisher> <isbn> </isbn> </book> ')); PL/SQL XML INSERT The following lines show the creation of a procedure used to insert XML data into the Oracle database: CREATE or REPLACE PROCEDURE insertpurchaseorderxmlorder IS PurchaseOrderXML CLOB; -- CLOB to hold XML BEGIN PurchaseOrderXML := <?xml version="1.0" encoding="utf-8" standalone="yes"?> <PurchaseOrder> <POID>1234</POID> <Date> </Date> <CustomerID>AA1234></CustomerID> <Company>King Training Resources</Company> <!-- other elements --> </PurchaseOrder>'; -- Insert the Purchase Order XML into an XMLType column INSERT INTO purchaseordertable (purchaseorder) VALUES (XMLTYPE(PurchaseOrderXML)); EXCEPTION WHEN OTHERS THEN raise_application_error(-20101, 'Error loading purchaseordertable, SQLCODE=' SQLERRM); END insertpurchaseorderxmlorder; UNSTRUCTURED UPDATE Unstructured Updates look like something like the following example: update xml_books xstuff set books = xmltype(' <name>learning XML</name> <author>eric T. Ray</author> <publisher>o Reilly</publisher> <isbn> </isbn> </book>') where xstuff.books.extract('/mybooks/book/text()').getstringval() = 'Wiley' UNSTRUCTURED DELETE Unstructured Delete may also use XPath notation to identify parts of the document to be deleted. delete from xml_books xstuff where xstuff.books.extract('/mybooks/book/text()').getstringval() = 'Wiley'; STRUCTURED XML DATA XMLType data stored using a schema is structured and is taken apart (shredded) into relational objects using the structure dictated by the schema. The first example below shows a table being created using structured data, the second example shows a table with a structured XMLType column, and the final example shows the creation of a table with an unstructured XMLType column using a schema. 4

5 CREATE TABLE xml_books OF XMLTYPE XMLSCHEMA 'mybooks.xsd' ELEMENT 'mybooks'; CREATE TABLE xml_books ( info XMLTYPE, desc VARCHAR2(100)) XMLTYPE COLUMN info STORE AS OBJECT RELATIONAL XMLSCHEMA 'mybooks.xsd' ELEMENT 'mybooks'; CREATE TABLE xml_books ( info XMLTYPE, desc VARCHAR2(100) ) XMLTYPE COLUMN info STORE AS CLOB XMLSCHEMA 'mybooks.xsd' ELEMENT 'mybooks'; INSERT USING SCHEMA When inserting data, XML document using schemas are validated. INSERT INTO mybookstype VALUES ( xmltype.createxml('<?xml version="1.0"?> <mybooks xmlns:xsi=" xsi:nonamespaceschemalocation="mybooks.xsd"> <name>definitive XML Schema</name> <author>prescilla Walmsley</author> <publisher>prentice-hall</publisher> <isbn> </isbn> </book> <name>definitive XSLT and XPATH</name> <author>g. Ken Holman</author> <publisher>prentice-hall</publisher> <isbn> </isbn> </book> </mybooks> ')); SELECTING USING XPATH XPath notation may be used to select sets of data or particular rows. select value(xml) from mybookstype xml where existsnode(value(xml), '/mybooks/book[publisher="wiley"]') = 1 UPDATE USING XPATH XPath may be used to alter portions of the structured XML document (piece-wise update) as shown below: update mybookstype xstuff set value(xstuff) = updatexml(value(xstuff), '/mybooks/book/publisher/text()','shannon') where existsnode(value(xstuff), '/mybooks/book[publisher="wiley"]') = 1 DELETE USING XPATH Delete may also use XPath notation to identify portions of the XML document to be deleted. delete mybookstype xstuff where existsnode(value(xstuff), '/mybooks/book[publisher="wiley"]') = 1 Structured Storage Review With structured storage an XML document is shredded into database objects. Documents must conform to a registered XMLSchema and XML DB will use the XML Schema to generate SQL. Structured Storage has several advantages: memory management is better than with CLOB, storage requirements are reduced, indexing using standard b-tree indexes is possible using structured storage. Both structured and unstructured storage may use text-based indexes and function-based indexes. Partial or in-place updates are possible with structured storage while unstructured documents must be updated all at once. Due to the overhead associated with "shredding" and "unshredding" structured data, adding and retrieving XML documents to the database is slower than when using unstructured data. 5

6 XML DB Components Oracle XML DB has several standard components: XMLType provides a data type defining the column or table as XML data and including methods to allow operations on the XML such as XSL transformations and validation via XML Schema. XMLSchema allows XML Schemas to be registered with XML DB to validate documents and to define how documents will be stored by the database. XML DB Repository provides a mechanism for associated URIs with XPath notation to access XML data and supports interaction with HTTP, FTP, and WebDAV clients. SQL/XML includes many operators that are part of the emerging SQL/XML standard. Schema Evolution Before Oracle 10g once an XML schema is registered with Oracle XML DB it cannot be modified (evolved) since there might be XMLTypes that depend upon the schema. Oracle 10g support schema "evolution" using a PL/SQL procedure named CopyEvolve() which is part of the DBMS_XMLSCHEMA package. CopyEvolve() copies an XMLType document to temporary tables, drops and re-registers the XML schema, then copies the XMLType data into the new XMLTypes. This allows schemas to be modified while allowing existing XMLTypes to stay valid. If the XMLTypes that depend upon the schema may be eliminated: drop the XMLType tables dependent upon the schema, drop the XMLType tables, delete the schema, and finally register the new schema's URL. CopyEvolve() has some limits: indexes, triggers, and constraints dependent upon the schemas are lost and must be recreated. Changing top-level element involves additional processing detailed in the Oracle documentation. The CopyEvolve() may not be used if there is a table with an object-type column that has an XMLType attribute that is dependent on any of the schemas to be evolved. Using XML DB features XML DB includes many operators that are part of the emerging SQLX XML standard. These operators are used to query and access XML documents as part of SQL. SQLX-based operators are also used to generate XML using the SQL SELECT statement. WebDAV WebDAV is an IETF ( standard set of HTTP extensions allowing an HTTP Server to serve files to a WebDAVenabled client. Any WebDAV-enabled product can read and update XML content stored in the XML DB Repository. Since both Microsoft Office (Office XP and beyond) and Oracle support WebDAV, they work together automatically. Some other WebDAV-enabled products: Microsoft Internet Explorer, Altova XMLSpy, Macromedia MX and others. XML s promise of portable data is greatly facilitated by WebDAV. Oracle s XDB and WebDAV Demo may be downloaded from Look under XML for the XDBBasicDemo then download it and any other software that might be useful. The download site includes an XML editor, and an FTP package that are not required, but make installation and execution of the demo easier. Try WebDAV by following the well-laid-out instructions. About the Author John King is a Partner in King Training Resources, a firm providing instructor-led training since 1988 across the United States and Internationally. John specializes in application development software on a variety of platforms including Web, Unix, Linux, IBM mainframe, and personal computers. John has worked with Oracle products and the database since Version 4 and has been providing training to Oracle application developers since Oracle Version 5. John develops and presents customized courses in a variety of topics including Oracle, DB2, UDB, Java, XML, C#, and various programming languages. He has presented papers at many industry events including: IOUG-A Live!, UKOUG Conference, EOUG Conference, AUSOUG Conferences, RMOUG Training Days, MAOP-AOTC, NYOUG, and the ODTUG conference. - King Training Resources 6341 South Williams Street Littleton, CO U.S.A. Phone: (within the U.S.) Fax: john@kingtraining.com Website: 6