Using Oracle XML DB to Optimize Performance and Manage Structured XML Data

Transcription

1 Using Oracle XML DB to Optimize Performance and Manage Structured XML Data

2 I want to improve the performance of my application... Can I copy Java code to an HTML Extension? I coded it this way... Here s where you can find the latest release. Is the app customizable? How does restricted task reassignment work? The best way to migrate Oracle E-Business Application Suite Tier servers to Linux is... Just watch the live webcast on virtualization. Where can I find technical articles on logging in Java ME? Oracle Technology Network. It s code for sharing expertise. Come to the best place to collaborate with other IT professionals. Oracle Technology Network is the world s largest community of developers, administrators, and architects using industrystandard technologies with Oracle products. Sign up for a free membership and you ll have access to: Discussion forums and hands-on labs Free downloadable software and sample code Product documentation Member-contributed content Take advantage of our global network of knowledge. JOIN TODAY Go to: oracle.com/technetwork Copyright 2010, Oracle and/or its affiliates. All rights reserved. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners

3 Using Oracle XML DB to Optimize Performance and Manage Structured XML Data... 1 Reviewing XML Schema and XML Documents in a Web Browser... 2 Using XQuery and SQL/XML to Query XML Documents... 7 Start SQL Developer to Examine the Storage Table for the Purchase Order XML Documents... 7 Run SQL/XML and XQuery Queries against the PURCHASEORDER XMLType Create Indexes to Improve Query Performance Using XQuery Pragma ora:defaulttable to Improve Repository Query Performance Equi-Partitioning of Object-Relational Storage Create Partitioned XMLType Table Improving Query Performance of Partitioned XMLType Table through Index Creation Using Parallelism in Query Execution Summary... 36

4 Using Oracle XML DB to Optimize Performance and Manage Structured XML Data Purpose This tutorial shows you how to store, index, and query structured XML data in Oracle XML Database. Time to Complete Approximately 40 minutes. Overview XML schemas are used by many industries to define the XML structures used for information exchange. XML documents conforming to these XML schemas are highly structured. This tutorial will show you how the rich capabilities of Oracle XML DB enable storage, indexing, query, optimized performance, and management of structured XML data. It covers the following topics: Using the object-relational storage model to store, index, and query structured XML data Using the Oracle ConText full-text index to improve full-text query performance Using equi-partitioning and parallelism on the object-relational storage model to improve performance and scalability Prerequisites Before you perform this tutorial, you should first complete the following steps: 1. Check your Oracle Database 11g Release 2 ( ) installation 2. Check your Oracle SQL Developer (version 2.1) installation 3. Check the files in your working directory (/HOL/Part1/files) 1

5 Reviewing XML Schema and XML Documents in a Web Browser An XML Schema and over a hundred sample XML documents have been supplied as part of the OE sample schema in Oracle Database 11g Release 2. In this section, you will review their content using a web browser. 1. Open a browser and enter the URL for the OE folder in the XML DB repository. 2. Enter the user name OE and password oe in the popup dialog box, then click the OK button. 3. Now click the purchaseorder.xsd hyperlink in the browser window to view the XML schema. 2

6 3

7 4. The XML schema will now appear in the browser window as shown below. Notice the highlighted annotation xdb:defaulttable= PURCHASEORDER, which specifies that all XML documents conforming to this schema will be physically stored in the PURCHASEORDER table. 5. XML documents stored in the XML DB repository can also be viewed via a web browser. Now click the PurchaseOrders folder hyperlink. 4

8 6. The browser window will show a hyperlinked sub-folder 2002/. Click the hyperlink. 7. A number of hyperlinked sub-folders will appear in the browser window as shown below. Click Dec/ hyperlink. 8. The browser window will show a number of XML documents of purchase orders. Click one of the hyperlinked XML documents to view the content. 5

9 9. Review the formatted purchase order XML document below. Notice the highlighted annotation xsi:nonamespaceschemalocation= specifying the XML schema location URI. This is referencing the xsd file that we have reviewed earlier in a browser window. 6

10 Using XQuery and SQL/XML to Query XML Documents In the last section, you browsed XML documents stored in the XML repository via a web browser. Since each of these XML documents is physically stored as a row of an XMLType table, queries can be executed against either the XMLType storage table or the repository. In this section, you will use XQuery and SQL/XML to query XML documents stored in the PURCHASEORDER table. Start SQL Developer to Examine the Storage Table for the Purchase Order XML Documents Perform the following steps: 1. Click the SQL Developer icon on the desktop to start the application. 2. Create a database connection as oe user by performing the following steps. a. In the Connections tab, right-click Connections and select New Connection. b. The New / Select Database Connection window appears. Enter the following details, and click Test to make sure that the connection has been set correctly. Select the Save Password check box, the password is saved. Click Connect after he test status shows success. Connection Name: oe UserName: oe Password:oe Hostname: localhost Port: 1521 SID: orcl 7

11 8

12 3. Set the Autotrace parameters. Perform the following steps: a. Go to Tools > Preferences. b.expand Database, and select Autotrace Parameters. c. Make sure to select the following check boxes and click OK. Object_Name Cost Cardinality Predicates 9

13 4. Examine the oe user schema. Perform the following steps: a. Click the + icon to the left of the oe connection to expand it. b. Expand Tables, and select PURCHASEORDER. On the right window pane, select the Details tab to examine the values of the NUM_ROWS and the TABLE_TYPE parameters. Notice the highlighted TABLE_TYPE parameter with value XMLTYPE. 10

14 c. Click the Files tab in the left window pane to open the SQL script 1_3_queries.sql for display on the right window pane. On the right window pane, select the query Q0 and the Run Statement button to execute a query against the USER_XML_TABLES data dictionary view. You can see below that The PURCHASEORDER XMLType table is stored with the OBJECT-RELATIONAL storage option. The XMLSCHEMA is (the same as specified in a conforming XML document we reviewed earlier). Run SQL/XML and XQuery Queries against the PURCHASEORDER XMLType Perform the following steps: 11

15 1. On the left window pane, click the Files tab to open the 1_3_queries.sql script. On the right window pane, select the Q1 query and click the Run Statement button to execute the query. The Query Result tab below will display the results. Notice XMLSerialize(), XMLQueru(), and XMLExists() SQL/XML functions along with XQuery expressions as their parameters in the query. 12

16 2. Scroll down the same file to find the Q11 query, which uses the XMLTable() SQL/XML function to map structured data in the purchase order XML documents into a master-detail table for ROLLUP calculations. Notice the output in the Query Result tab to find a sub-total calculated for the A10 cost center in row No You can now run Autotrace on these queries to see of their execution plans are optimal. Using Q1 as an example, you can see below that the execution plan involved an inefficient TABLE ACCESS FULL operation. 13

17 Create Indexes to Improve Query Performance In the following steps you will learn how to create indexes to improve XML query performance. 14

18 1. On the left window pane, click the Files tab to open the 1_2_createIndexes.sql script. On the right window pane, click the Run Script button to create B-tree indexes. Notice how the CREATE INDEX statements below used XMLCast() and XMLQuery() SQL/XML functions along with XPath expressions to create B-tree indexes on the leaf nodes. Two composite indexes and one full-text index are also created for the columns of the LINEITEM_TABLE nested table. 15

19 2. To examine the created indexes, scroll to the bottom of the same file to select the query shown below. Click the Run Statement button to see that you have successfully created B-tree indexes and one full-text index (the DOMAIN index) on the PURCHASEORDER table and its associated LINEITEM_TABLE. 3. Check the execution plan again by first selecting the Q1 query in 1_3_queries.sql file and then click the Autotrace button. You can see below how the query execution plan has improved where TABLE ACCESS FULL operation has been replaced by more efficient TABLE ACCESS BY INDEX ROWID and INDEX RANGE SCAN operations. 16

20 17

21 4. Similarly, the query execution plan for Q5 also used more efficient INDEX RANGE SCAN operations. Notice that the DOMAIN INDEX (i.e., the full-text index on purchase order item description) is also used to execute the ora:contains() XQuery function in the XMLExists() SQL/XML function. 18

22 Using XQuery Pragma ora:defaulttable to Improve Repository Query Performance In previous releases, to obtain optimal performance for XQuery expressions that use fn:doc and fn:collection over Oracle XML DB Repository resources, you needed to write explicit joins with RESOURCE_VIEW. In the release, the new XQuery pragma ora:defaulttable will perform the necessary joins automatically. In the following steps you will learn how to use this new pragma. 1. On the left window pane, click the Files tab to open the 1_4_defaultTableQueries.sql script. On the right window pane, select the CASE 1 query and click the Autotrace button to check the execution plan. Since XQuery fn:collection() function is used to specify an XML document as an XM LDB repository resource, the query execution engine has no immediate knowledge that the physical storage of this document is object-relational with applicable indexes. As a result, the query plan uses the very inefficient COLLECTION ITERATOR PICKLER FETCH operations. -- CASE 1: In previous releases, repository queries using fn:collection will result -- in poor query plan SELECT XMLQuery ('for $doc in fn:collection("/home/oe/purchaseorders/2002/sep") let $val := $doc/purchaseorder/lineitems/lineitem/part[@id = " "] return $val' RETURNING CONTENT) FROM DUAL; 19

23 2. Prior to release , you had to manually rewrite the query to explicitly relate the XML DB repository resource to the underlying physical storage table as shown in CASE 2 of the same SQL script file. Notice how the query execution plan has improved to use INDEX RANGE SCAN operations. -- CASE 2: Manually rewriting the repository query to use under_path() instead will -- result in optimized query execution SELECT XMLQuery ('let $val := $DOC/PurchaseOrder/LineItems/LineItem/Part[@Id = " "] return $val' PASSING OBJECT_VALUE AS "DOC" RETURNING CONTENT) FROM RESOURCE_VIEW rv, purchaseorder p WHERE ref(p) = XMLCast(XMLQuery ('declare default element namespace " (: :) fn:data (/Resource/XMLRef)' PASSING rv.res RETURNING CONTENT) AS REF XMLType) AND under_path(rv.res, '/home/oe/purchaseorders/2002/sep') = 1; 20

24 21

25 3. In the new release, instead of manually rewriting the query, you can simply use the new ora:defaulttable pragma in your XQuery expression to give a hint about the underlying physical storage table (i.e., the PURCHASEORDER table in this example) to the query processing engine. Notice how the execution plan is now as good as the manually rewritten query in the previous step. -- CASE 3: Starting in , using the new ora:defaulttable pragma along with -- fn:collection() will offer optimized query execution SELECT XMLQuery ('for $doc in (# ora:defaulttable PURCHASEORDER #) { fn:collection ("/home/oe/purchaseorders/2002/sep ") } let $val := $doc/purchaseorder/lineitems/lineitem/part[@id = " "] return $val ' RETURNING CONTENT) FROM DUAL; 22

26 Equi-Partitioning of Object-Relational Storage Equi-partitioning for object-relational storage was introduced in Oracle Database 11g Release 2 to help simplify XML data lifecycle management and performance. In this section, you will learn how the object-relational storage table can be equipartitioned. Create Partitioned XMLType Table 1. Create an XMLType table with partitioned object-relational storage, and populate the table with data selected from the PURCHASEORDERS table. Run the script 2_1_createTable.sql. Perform the following steps: a. Cick in the Files tab,. b. Browse to the location of your working directory \HOL\Part1\files, and select the file 2_1_createTable.sql. Then, right-click Open. c. Select the oe connection from the drop down box on the right. d. The code is displayed in the SQL Worksheet. Click the Run Script icon. Note that three list partitions are specified based on the partitioning key COST_CENTER, which corresponds to the value of /PurchaseOrder/CostCenter node in the purchaseorder XML document. Note also the table partitions for the ordered collection table (OCT) LINEITEM_PT_TABLE are equi-partitioned in the same way as their base table and given their own partition names (i.e., lineitem_a10, lineitem_r20, lineitem_s30). CREATE TABLE PARTITIONED_PO OF XMLType XMLSCHEMA " ELEMENT "PurchaseOrder" VARRAY "XMLDATA"."LINEITEMS"."LINEITEM" STORE AS TABLE lineitem_pt_table ((PRIMARY KEY (NESTED_TABLE_ID, SYS_NC_ARRAY_INDEX$))) PARTITION BY LIST (XMLDATA.COST_CENTER) (PARTITION A10 VALUES ('A10') TABLESPACE PT01 VARRAY "XMLDATA"."LINEITEMS"."LINEITEM" STORE AS TABLE lineitem_a10, PARTITION R20 VALUES ('R20') TABLESPACE PT02 VARRAY "XMLDATA"."LINEITEMS"."LINEITEM" STORE AS TABLE lineitem_r20, PARTITION S30 VALUES ('S30') TABLESPACE PT03 VARRAY "XMLDATA"."LINEITEMS"."LINEITEM" STORE AS TABLE lineitem_s30 ); INSERT INTO PARTITIONED_PO SELECT OBJECT_VALUE FROM PURCHASEORDER; 23

27 Note:Throughout this tutorial, to execute the script files in SQL Developer, you must follow the above steps a through c. If you want to run a single statement at the mouse pointer, click the Execute Statement icon. Alternatively, move the cursor over the statement, and press F9. e. Right click the Tables folder to select the Refresh menu item. 24

28 f. Click the newly created XMLType table PARTITIONED_PO to find more information on the right pane of the Oracle SQL Developer window. g. Click the Partitions tab to display table partitions. 25

29 h. Notice the partition names as you have specified. Click the Actions button and move the cursor to select the Gather Statistics menu item. i. In the Gather Statistics popup dialog, click the Apply button. 26

30 j. Once statistics are gathered, you can see that the 132 rows of purchase orders are now distributed among three partitions with 32, 43, and 57 rows each. k. To run a test query on the partitioned XMLType table, use Oracle SQL Developer to open 2_3_queries.sql file in the \HOL\Part1\files directory. Select the Q1 query and click the Autotrace button to examine the query execution plan. Notice the PARTITION LIST SINGLE operation, PARTITION_START and PARTITION_STOP of 2 as a result of the partition pruning optimization to access only the R20 partition. select XMLSerialize (Content xmlquery('/purchaseorder/reference' passing object_value returning content) as varchar2(64)) from partitioned_po where xmlexists('/purchaseorder[user="lsmith"][costcenter="r20"]' passing object_value) / l. Click the newly created XMLType table LINEITEM_PT_TABLE to find more information on the right window pane. 27

31 m. Click the Partitions tab to display table partitions for the ordered collection table (OCT) LINEITEM_PT_TABLE. With equipartitioning support, OCTs are partitioned in the same way as their base table. In the above topic, you learned how to create a list partitioned XMLType table with object-relational storage, populated data into this table, and checked the query execution plan for partitioning-specific optimizations. 28

32 Improving Query Performance of Partitioned XMLType Table through Index Creation As you have learned earlier in the lab, you can improve the performance of your queries by creating indexes. In this section, you will create locally managed B-Tree indexes and a globally managed full-text index on the partitioned XMLType table. You will then run the SQL/XML, XQuery queries against the partitioned object-relational table to see the autotrace result and note the performance improvement with more efficient query execution operations. Perform the following steps: 1. In your SQL Developer session, connect as OE user. Then, execute the script 2_2_createIndexes.sql. Note the use of keyword LOCAL to create B-tree indexes local to each partition. Note also the absence of the LOCAL keyword for the ConText Full-Text index (i.e., ipt_description_text) to create a globally managed index. -- B-tree indexes for Equi-Partitioned O-R storage drop index ipt_user; drop index ipt_costcenter; drop index ipt_ref; drop index ipt_lineitempartnumber; drop index ipt_partnumber; drop index ipt_description_text; create index ipt_user on partitioned_po (XMLCast(XMLQuery('/PurchaseOrder/User' PASSING OBJECT_VALUE RETURNING Content) as varchar2(10))) LOCAL / create index ipt_costcenter on partitioned_po (XMLCast(XMLQuery('/PurchaseOrder/CostCenter' PASSING OBJECT_VALUE RETURNING Content) as varchar2(4))) LOCAL / create index ipt_ref on partitioned_po (XMLCast(XMLQuery('/PurchaseOrder/Reference' PASSING OBJECT_VALUE RETURNING Content) as varchar2(30))) LOCAL / create index ipt_lineitempartnumber on LINEITEM_PT_TABLE (ITEMNUMBER, PART.PART_NUMBER, NESTED_TABLE_ID) LOCAL / create index ipt_partnumber on LINEITEM_PT_TABLE (PART.PART_NUMBER, NESTED_TABLE_ID) LOCAL / create index ipt_description_text on LINEITEM_PT_TABLE (DESCRIPTION) indextype is CTXSYS.CONTEXT parameters ('transactional') / commit; call dbms_stats.gather_table_stats(user,'partitioned_po') / call dbms_stats.gather_table_stats(user,'lineitem_pt_table') / 29

33 Once the indexes are created, you can scroll to the bottom of the script to query the data dictionary view USER_INDEXES. Notice the indexes created with the LOCAL keyword are showing YES in the PARTITIONED column. 30

34 2. Now that you have created indexes, you can observe the improved query execution plan with INDEX RANGE SCAN operations for a specific partition. Open the file 2_3_queries.sql file, select and drag the mouse to highlight the Q2 query. Click the Autotrace button to generate a query execution plan. Note the usage of the index IPT_PARTNUMBER and the value of PARTITION_START and PARTITION_STOP pointing to a single partition. Note also the highlighted query execution operation TABLE ACCESS BY LOCAL INDEX ROWID. Code in the Q2 query: select XMLSerialize (Content xmlquery('/purchaseorder/reference' passing object_value returning content)) from partitioned_po where xmlexists('/purchaseorder[user="lsmith"][costcenter="a10"]/lineitems/lineitem[part/@id=" "]' passing object_value) / 31

35 3. View the execution plan of the query that lists the references for LineItem 20 with a description containing picnic on a particular purchase order. Select query Q5 in 2_3_queries.sql file and click the Autotrace button. Notice the usage of IPT_DESCRIPTION_TEXT index in the query execution plan. However, you can see there are now two TABLE ACCESS BY GLOBAL INDEX ROWID operations. It is because that the IPT_DESCRIPTION_TEXT index is a globally managed index. Code in Q5: SELECT XMLSerialize(Content XMLQuery('/PurchaseOrder/Reference' passing object_value returning content)) FROM PARTITIONED_PO WHERE XMLExists('/PurchaseOrder/LineItems/LineItem[@ItemNumber=20]/Description/text() [ora:contains(.,"picnic") > 0]' passing object_value) / In the above section, you learned how to create B-Tree indexes on partitioned object-relational storage. You also learned how to observe the improved query performance by viewing the query execution plan. 32

36 Using Parallelism in Query Execution Equi-partitioning and local indexes allow parallel queries and DMLs to further improve performance. Perform the following steps: 1. In Oracle SQL Developer, open the 2_4_alterTable.sql file in the \HOL\Part1\files directory. Choose oe as the DB connection. Click the Run Script button to execute the ALTER TABLE command. This enables parallel query and DML on the PARTITIONED_PO table. ALTER TABLE PARTITIONED_PO PARALLEL; 33

37 2. In Oracle SQL Developer, click the 2_3_queries.sql tab to select the Q6 query. Click the Autotrace button to generate the query execution plan. Note the parallel query operators and the usage of locally managed IPT_PARTNUMBER index. 3. Parallel query execution can make a tremendous difference in business intelligence calculations. Query Q11 in the 2_3_queries.sql file is such an example that aggregates total dollar amount for each cost center and requestor. Once you selected the Q11 query and click the Autotrace button, you can use the scrollbar to see the complete query execution plan with extensive use of parallel query operators (i.e., PX operators). 34

38 35

39 Summary In this tutorial, you learned how to: Review an XML Schema and XML documents in a web browser Use XQuery and SQL/XML to query XML documents Use XQuery pragma ora:defaulttable to improve repository query performance Equi-partition of XMLType table stored in the object-relational format Improve XML query performance through Index creation Use parallelism in query execution 36

40 Using Oracle XML DB to Aggregate Facebook, Twitter, YouTube, Blog, and News

41 Using Oracle XML DB to Aggregate Facebook, Twitter, YouTube, Blog, and News... 1 Examining a Table to See Stored Information about RSS Feeds... 2 Examining RSS Items Stored in a Partitioned XMLType Table with the Binary XML Storage Option... 4 Creating a Locally Managed Unstructured XMLIndex Index on RSS Items Creating an Aggregated RSS View with Normalized RSS Items from Diverse RSS Data Feeds Reading the Aggregated RSS Feed with a Firefox 3.5 or a Windows Internet Explorer 8 Browser Creating an XMLIndex Structured Component on RSS Items Creating a New Aggregated RSS View with Normalized RSS Items from Diverse Data Feeds Summary... 28

42 Using Oracle XML DB to Aggregate Facebook, Twitter, YouTube, Blog, and News Purpose This tutorial shows you how to use binary XML storage model, XMLIndex, XQuery, SQL/XML, and many other salient capabilities of Oracle XML DB to aggregate and query unstructured XML data from diverse Web 2.0 data sources. Time to Complete Approximately 40 minutes. Overview Facebook, Twitter, YouTube, blogs, news, and other social media feeds are available as either ATOM 1.0 or RSS 2.0 semistructured XML data. This tutorial covers how Oracle XML DB can store, normalize, index, query, search, transform, and aggregate ATOM 1.0 and RSS 2.0 feeds from diverse data sources. It covers the following topics: Using the binary XML storage model and unstructured XMLIndex to store, index, and query semistructured XML data Using the binary XML storage model and structured XMLIndex to store, index, and query semistructured XML data Using the Oracle ConText full-text index as a secondary index to provide full-text query capability Using partitioning, parallelism, and XML indexes to improve performance and scalability Prerequisites Before you perform this tutorial, you should first complete the following steps: 1. Check your Oracle Database 11g Release Check your Oracle SQL Developer (version 2.1) installation 3. Check the files in your working directory (\HOL\Part2\files) 4. Check your Firefox browser version 3.5 or Windows Internet Explorer 8 installation Introduction Oracle Database 11g Release 2 XML DB supports a binary XML storage model, structured and unstructured component of XMLIndex index, partitioning of binary XML storage model, and locally managed structured and unstructured XMLIndex. This hands-on lab uses the real world use case of an RSS feed aggregator to demonstrate these versatile and high performance capabilities. 1

43 Examining a Table to See Stored Information about RSS Feeds An RSS (Real Simple Syndication) feed is an URL pointing to an XML document in one of the many variants of RSS formats. The XML document of an RSS feed contains news items. In this section, you will examine a table to see the stored information about subscribed RSS feeds. 1. Create a connection with user rss/rss. Click the Connect button to connect as user rss. 2. Open and execute the script 2_3_VerifyFeeds.sql to verify that a list of RSS feeds has been inserted. 2

44 3

45 Examining RSS Items Stored in a Partitioned XMLType Table with the Binary XML Storage Option Different RSS formats use different XML structures to represent feed items. This type of highly varied XML structures is best stored with the binary XML storage model. By taking advantage of the binary XML storage model along with the SECUREFILE LOB storage format, you can gain improved performance while reducing storage space. 1. In Oracle SQL Developer, click the Connections tab on the left window pane to select and open rss Tables NEWSITEMSTABLE. 4

46 2. Click the Data tab on the right window pane to see a list of RSS items stored in the NEWSITEMSTABLE. 5

47 3. Click the Constraints tab, you will see there is a TITLE_IS_UNIQUE constraint on the VC_TITLE virtual column. 6

48 4. With a unique constraint on the virtual column of news item title, Oracle XML DB can enforce the uniqueness of a news item title. You can execute 3_3_InsertTestItems.sql to insert news items with duplicate item titles to confirm the existence of a unique constraint on the news item title virtual column. 7

49 5. Expand the rss connection and the Tables icon to click the NEWSITEMSTABLE to open a new tab on the right pane. Click the Partitions sub-tab and the Actions dropdown list to select the Gather Statistics Action. Specify 100% statistics sampling by selecting the drop down list. 8

50 6. Click the Refresh button to see the collected statistics for the partitions of NEWSITEMSTABLE. Notice how each partition has its corresponding name, number of rows, number of blocks, and the range partitioning key information. 9

51 Creating a Locally Managed Unstructured XMLIndex Index on RSS Items When RSS items are stored in the binary XML storage format, creating an XMLIndex Unstructured Component on RSS items will provide much improved query performance. To allow full text searches on RSS item titles, a secondary text index can be created on the path table of an unstructured XMLIndex. Detailed information on this topic can be found in a best practices white paper for optimal XML query performance. 1. As user RSS, execute the script 4_1_CreateUnstructXMLIndex.sql to create an XMLIndex Unstructured Component index using a subset of XPath pointing to news item titles. A secondary Oracle Text Context index is also created. Notice how Oracle XML DB handles multiple namespaces used by different RSS formats. Also notice the LOCAL keyword specified for creating locally managed unstructured XMLIndex. 10

52 2. Expand Connections rss Tables to open on the right window pane a tab for NEWSITEM_BIX_PATH_TABLE, which is the main storage table for the XMLIndex Unstructured Component. 11

53 3. Click the Partitions tab and the Refresh button to see how locally managed Unstructured XMLIndex index data have been stored in their corresponding local partitions. 12

54 Creating an Aggregated RSS View with Normalized RSS Items from Diverse RSS Data Feeds The main function of a RSS (Real Simple Syndication) aggregator is to aggregate and normalize RSS items from multiple feeds using diverse RSS formats. In this section, you will see how you can create an RSS view normalizing diverse RSS formats. 1. As user rss, execute the script 5_1_CreateUnstructRSSViews.sql to create two relational views newsitemsrelview_1 and newsitemsrelview_2, which normalize RSS items using diverse RSS formats. Notice the following, The UNION ALL clause in the SQL statements for creating these views. To create newsitemsrelview_1, the ora:contains() XQuery function was used for full text searches on RSS item titles. To create newsitemsrelview_2, the VC_DATE virtual column was used in the WHERE clause to filter RSS item titles. 13

55 14

56 2. You can check out the query execution plan for the query that generates a normalized RSS feed from newsitemsrelview_1. In the same script file 5_1_CreateUnstructRSSViews.sql, select the CASE 1 SQL/XML query and click the Autotrace button to see the execution plan in the window pane below. Notice in the normalized query result in optimized query execution where the Unstructured XMLIndex and the secondary text index are used. Notice the TABLE ACCESS BY LOCAL INDEX ROWID and the TABLE ACCESS BY GLOBAL INDEX ROWID operations associated with the locally managed unstructured XMLIndex and the globally managed full-text index respectively. 15

57 3. You can do the same for newsitemsrelview_2. In the same script file 5_1_CreateUnstructRSSViews.sql, select the CASE 2 SQL/XML query and click the Autotrace button to see the execution plan in the window pane below. Notice PARTITION RANGE SINGLE operation that accessed only partition 2 of the NEWSITEMSTABLE. 16

58 4. Open and execute 5_4_CreateRSSResouce.sql to create two Oracle XML DB repository resources representing two normalized RSS feeds with aggregated and normalized RSS items. 17

59 18

60 Reading the Aggregated RSS Feed with a Firefox 3.5 or a Windows Internet Explorer 8 Browser There are several news readers capable of processing RSS 2.0 format. You can use either a Firefox 3.5 or a Windows Internet Explorer 8 browser to read the normalized RSS feed. 1. Open an Internet Explorer 8 browser and point to the URL folder. You will see two XML documents in the folder. Click one of the XML document links, you will see a list of RSS items aggregated from multiple RSS feeds. Enter rss/rss when prompted for User Name and Password. 19

61 20

62 Creating an XMLIndex Structured Component on RSS Items Alternatively, with structured islands in RSS items, creating an XMLIndex Structured Component index on RSS items will also improved query performance. To allow full text searches on RSS item titles, a secondary text index can be created on the content table of a Structured XMLIndex. 1. As user RSS, open and click no the Run Script button to execute the script 4_2_CreateStructXMLIndex.sql, which creates an XMLIndex Structured Component index on structured islands of RSS item title, URI, description, and pubdate. A secondary text index is also created on the item title column of the relational content table. Notice the following, Oracle XML DB can handle multiple namespaces used by different RSS formats. The LOCAL keyword is specified to create locally managed Structured XMLIndex. The PARALLEL keyword is specified to build the index using parallelism. 21

63 2. Expand Connections rss Tables to open on the right window pane a tab for NEWSITEM_SIX_ALL_TABLE, which is the index content table for the XMLIndex Structured Component. 22

64 3. Click the Partitions tab and the Refresh button to see how locally managed Structured XMLIndex index data have been stored in their corresponding local partitions. 23

65 Creating a New Aggregated RSS View with Normalized RSS Items from Diverse Data Feeds Similar to what you have gone through in a previous step using Unstructured XMLIndex, you will see how you can create an RSS view normalizing diverse RSS formats. 1. As user RSS, execute the script 5_2_CreateStructRSSViews.sql to create a relational view newsitemsrelview_1 and newsitemsrelview_2, which normalize RSS items. Notice the following, For VIEW 1, the SQL CONTAINS() function for full text searches is used in the WHERE clause for the TITLE column. This takes advantage of the secondary Oracle Text Context indexes created on the TITLE column of the index content table. For VIEW 2, the VC_DATE virtual column is used in the WHERE clause to return only more recent RSS items. 24

66 25

67 2. You can check out the query execution plan for the query that generates a normalized RSS feed from newsitemsrelview_1. In the same script file 5_1_CreateUnstructRSSViews.sql, select the CASE 1 SQL/XML query and click the Autotrace button to see the execution plan in the window pane below. Notice in the normalized query result in optimized query execution where the Structured XMLIndex (i.e., NEWSITEM_SIX_ALL_TAB) is used. 26

68 3. You can do the same for newsitemsrelview_2. In the same script file 5_1_CreateUnstructRSSViews.sql, select the CASE 2 SQL/XML query and click the Autotrace button to see the execution plan in the window pane below. Notice the PX operations indicating parallel query execution and the PARTITION RANGE SINGLE operation accessed only partition 2 of the NEWSITEMSTABLE indicating partition pruning. 27

69 Summary In this tutorial, you learned about the following: Creating a table to store information about RSS feeds Creating a partitioned table to store RSS news items Creating both a Unstructured and a Structured XMLIndex indexes that are locally managed on partitions of corresponding news Items Creating an RSS View with Aggregated and Normalized News Items from Diverse RSS feeds 28

70 Document-Centric Apps with Oracle XML DB and Database- Native Web Services

71 Document-Centric Apps with Oracle XML DB and Database-Native Web Services... 2 Starting the XFILES Demo Application... 3 Exploring Content Management Capabilities of Oracle XML DB Repository... 5 Locking resources... 5 Verifying the resource lock status... 7 Unlocking resources Versioning resources Checking out resources Updating and uncheckout resources Using XML Index- and XML Schema-Based Intelligent Search Creating and populating an XMLType table with the default binary XML storage Creating an unstructured XMLIndex over the created XMLType table Using XML index-based intelligent search Using XML schema-based intelligent search Developing AJAX-Based Applications by Using Database-Native Web Services Examining the XML index-based search web page Examining relevant Javascript files Examining SOAP request/response, XML, and XSL content Summary

72 Document-Centric Apps with Oracle XML DB and Database-Native Web Services Purpose This hands-on lab shows you how to build document-centric applications by using Oracle XML DB and database-native web services. Time to Complete Approximately 40 minutes. Overview Oracle XML DB repository is optimized for handling document-centric XML data. It contains resources that are either folders or files. It enables access to content in the database via HTTP/S and WebDAV. Database-native SOAP web services support allows execution of SQL and XQuery queries and invocation of PL/SQL stored functions and procedures. This tutorial uses the XFiles demo application to demonstrate how repository and Web services capabilities are used to manage and publish document-centric content. The session covers The content management capabilities of the repository XML index- and XML schema-based intelligent search Developing AJAX-based applications by using database-native web services Prerequisites Before you perform this tutorial, you should first complete the following steps: 1. Check your Oracle Database 11g Release 2 installation. 2. Check your Oracle SQL Developer (version 2.1) installation 3. Check the files in your working directory (\HOL\Part3\files) 2

73 Starting the XFILES Demo Application XFILES is an AJAX-based demo application supporting content management and content search capabilities. In this hands-on lab, you will use this application along with Oracle SQL Developer to learn about developing document-centric applications with Oracle XML DB and database- native web services. 1. Open your browser and enter the following URL: Click the Login button at the upper right corner. 2. A dialog will pop-up as shown below. Enter HOL3 for User Name and hol3 for Password. Click the OK button to login. 3

74 3. The browser window will refresh to display Current User : HOL3. Click the Home button on the upper right corner. 4. The browser display will show the home directory for user HOL3 containing a sampledata sub-folder. Click the sampledata link to explore the folder hierarchy. 4

75 Exploring Content Management Capabilities of Oracle XML DB Repository XFILES exposes content management capabilities of Oracle XML DB repository by invoking repository services via PL/SQL functions and procedures. Locking resources 1. Drill down the folder hierarchy (home HOL3 sampledata XML Plays Comedies) until you reach the Comedies folder to see a list of XML documents of Shakespeare s plays. Select the first file on the list (A Midsummer Night s Dream.xml) and then click the Do Action button on the right to see a dropdown menu with a list of Content Management operations available for Oracle XML DB repository resources. Select the Lock Resources item from the dropdown menu. 2. Click the OK button to confirm the request of locking the selected document. 5

76 3. An information dialog will pop-up to confirm the completion of the document locking operation. Click the OK button to dismiss the information dialog. 4. You will see that the status of this file has changed. 6

77 Verifying the resource lock status 1. To verify the lock status, you can use another WebDAV client to see if you can edit the same XML document. Open the Windows Explorer program and select Map Network Drive from the Tools dropdown menu. 7

78 2. Click the Sign up for online storage link and follow the steps below to map an Oracle XML DB repository folder to a network place shortcut. Click the Next > button. Select Choose another network location and then click the Next > button. 8

79 Enter for the network address entry and click the Next > button. 9

80 Enter HOL3 for the User name and hol3 for the Password in the popup dialog and click the OK button. Click the Next > button to confirm the network place naming below. 10

81 Click the Finish button to create a new network place. The network place window will popup to display the sampledata sub-folder in the Oracle XML DB repository. 11

82 You can now go back to the Windows Explorer window to open the sub-folders of HOL3 on localhost on the left window pane until it reaches the Comedies folder. You will see a list of XML documents displayed on the right window pane. 12

83 3. After mapping the repository folder to a network place shortcut, you can now run the Windows Office Word 2007 program and select the Open dropdown menu item. In the Open dialog box, select My Network Places on the left and click the HOL3 on localhost link. Drill down the sub-folder path of sampldata XML Plays Comedies to select A Midsummer Summer Night s Dream.xml, which was locked in a previous step. 13

84 Because the XML document was locked, you should see the File In Use popup dialog. Click Cancel to dismiss the dialog box. 14

85 Unlocking resources Since resource locking and unlocking are services supported by Oracle XML DB repository via PL/SQL functions and procedures, you can invoke them directly by using the Oracle SQL Developer tool. 1. Click the SQL Developer icon on the desktop to start the application. 2. On the left window pane, find and open the 2_1_UnlockResource.sql file under HOL Part3 files directory. 3. Choose hol3 as the DB connection on the right window pane and click the Run Script button to unlock the XML document. 15

86 4. Go back to the browser window and click the Comedies link to refresh the window. You will see the status for the XML doc has changed to unlocked. Versioning resources Oracle XML DB repository also supports versioning. A resource can be put under version control, checked in, checked out, etc. 16

87 1. On the left window pane, find and open the 2_2_makeVersioned.sql file under HOL Part3 files directory. 2. Choose hol3 as the DB connection on the right window pane and click the Run Script button to put the XML document under version control. 3. Go back to the browser window running XFILES and click the Comedies link to refresh the window. You will see the change of status for the XML document to a versioned icon. 17

88 Checking out resources You will learn to use the resource check-out service of Oracle XML DB repository. 1. On the left window pane of Oracle SQL Developer, find and open the 2_4_checkOutResource.sql file under HOL Part3 files directory. 2. Choose hol3 as the DB connection on the right window pane and click the Run Script button to check out the XML document. 18

89 3. Go back to the browser window running XFILES and click the Comedies link to refresh the window. You will see the change of status for the XML document to a checked-out icon. Updating and uncheckout resources You will learn to update and undo checkout of a resource of Oracle XML DB repository. 1. On the left window pane, find and open the 2_5_updateResource.sql file under HOL Part3 files directory. 19

90 2. Choose hol3 as the DB connection on the right window pane and click the Run Script button to update the XML document to one with an <empty/> element. 20

91 3. Go back to the browser window running XFILES and click the < > icon to display the updated XML document. You will see in a separate browser window that the content has been update to <empty/>. 4. To undo the content change, you can un-checkout the resource. On the left window pane of Oracle SQL Developer, find and open the 2_6_uncheckoutResource.sql file under HOL Part3 files directory. 21

92 5. Choose hol3 as the DB connection on the right window pane and click the Run Script button to un-checkout the XML document. 22

93 6. Go back to the browser window running XFILES and click the Comedies link to refresh the window. You will see that the XML document is no longer in the checked out state. 23

94 Using XML Index- and XML Schema-Based Intelligent Search XML documents stored in the Oracle XML DB can be structured or unstructured. A structured XML document conforms to an XML schema registered with the Oracle XML DB. The XML schema registration operation can also initialize underlying objectrelational storage for conforming XML instances. In contrast, an unstructured XML document is often stored in the CLOB or the binary XML format. Efficient querying operation of unstructured XML documents requires the creation of XMLIndex indexes. You will learn how to use XML index information to guide your queries over unstructured XML documents. Likewise, you can use XML schema information to guide your queries over structured XML documents. Creating and populating an XMLType table with the default binary XML storage Before using an XML index to guide your queries, you need to create an XMLType table and populate the table with data copied from the PURCHASEORDER table owned by user OE. 1. On the left window pane of Oracle SQL Developer, find and open the 3_1_createBixTable.sql file under HOL Part3 files directory. 2. Choose hol3 as the DB connection on the right window pane and click the Run Script button to create an XMLType table using the default binary XML storage format and populate the table by copying data from the PURCHASEORDER table belonging to user OE. 24

95 Creating an unstructured XMLIndex over the created XMLType table You will now create an unstructured XMLIndex over the XMLType table created in the previous step. 25

96 1. On the left window pane of Oracle SQL Developer, find and open the 3_2_createXMLIndexes.sql file under HOL Part3 files directory. 2. Choose hol3 as the DB connection on the right window pane and click the Run Script button to create an unstructured XMLIndex po_sl_xmlindex_bix_ix. 26

97 Using XML index-based intelligent search You will use the XFILES application to conduct XML index-based intelligent search. 1. Click the Home button to return to the home folder for user HOL3. 27

98 2. Select the XML Indexes item from the dropdown list and then click the Search button on the right. 3. You will see the browser refreshes to display the XMLIndex you have just created. Click the PO_SL_BIX_TABLE link. 4. The browser window will refresh to display predicate fields for index-based intelligent search. Click the small icon the left of User to display additional icons to its right. 28

99 5. Click the edit icon to the right of User to display an editable text field. Enter SBELL into the text field. 6. Click the checkbox on the left. You will see status at the bottom showing 13 matches. 7. Click the Show Query Details button on the right to display a Current SQL popup dialog with the actual query that returned 13 matches. Click the (X) icon to dismiss the Current SQL dialog box. 29

100 8. Click the Logout icon to logout user HOL3. 30

101 Using XML schema-based intelligent search You will use the XFILES application to conduct XML schema-based intelligent search. 1. Click the Login button. 2. Enter OE for User name and oe for Password. Click the OK button to login. 3. Select XML Schemas from the Search options on the right of the browser window. 31

102 4. Click the Search icon to the right of the Search option. 5. You will see the browser window displaying an item in the Local XML Schemas area. Click the schema link. 6. The browser window will refresh to display predicate fields for XML schema-based intelligent search. Click the small icon to the left of LineItems to expand further to show its child nodes until you reach the Part node and its child nodes. 32

103 7. Click the Edit icon to the right of the Id attribute node. 8. Enter a Part ID into the text field and click the check box to the left afterward. 33

104 9. Note the status bar at the bottom displaying the query result for the specified Part ID returned 11 matches. 10. Click the Show Query Details icon on the right side of the browser window to display the actual SQL query statement used. 11. In the popup window, note how the XMLExists() clause used the Part ID you entered earlier. Click the (X) icon to dismiss the popup window. 34

105 12. Click the Logout icon to logout user OE. 35

106 Developing AJAX-Based Applications by Using Database-Native Web Services AJAX stands for Asynchronous JavaScript and XML. It is a programming model that takes advantage of the XMLHTTPRequest object supported by a web browser s Javascript engine to asynchronously transfer XML data to a web browser without reloading whole web pages. Once transferred, a browser can use its built-in XSLT engine to transform the XML data into HTML display according to an XSL style sheet. This programming model provides more efficient data loading and better user experience with web applications. The AJAX programming model often uses a standard web service protocol for its asynchronous XML data transfer. Likewise, to further simplify the XML data fetching tasks, Oracle Database 11g Release 1 introduced Oracle XML DB native web services to allow Oracle XML DB serving as a SOAP protocol end point. In the steps below, you will use the XFILES AJAX-based application to learn how a Javascript program uses the AJAX programming model to fetch XML data generated by Oracle XML DB via its native web services. You will examine the key components for implementing the AJAX programming model. Examining the XML index-based search web page Using the XML index-based search example covered earlier, you will see how the web page invokes a Javascript program based on user action. 1. Click the Login icon to login as user HOL3. 2. Enter HOL3 for User name and hol3 for Password. 36

107 3. Select XML Indexes from the Search options. 4. Click the Search icon to display the XMLIndex search page. 5. Move your mouse cursor to the PO_SL_BIX_TABLE link and note the browser Status Bar display showing an URL. Click the link to display the XML index-search page. 37

108 6. In the new page, notice the browser URL is now H_TABLE&indexName=PO_SL_XMLINDEX_BIX_IX&uploadFolder=&targetColumn=object_value&stylesheet=/XFILES/XML Search/xsl/xmlIndex.xsl 7. In the browser menu, select Source from the View dropdown menu. 38

109 8. In the HTML source file, notice the Javascript file xmlindexsearch.js in the <head> section. Examining relevant Javascript files You will use Oracle SQL Developer to see how how Javascript was used to implement XML index-based search. 39

110 1. In the Files tab on the left window pane of Oracle SQL Developer, find and open the xmlindexsearch.js file under the HOL XFILESV src XMLSeach js directory. 40

111 2. On the right window pane, you will see how this Javascript handles the loading of a XSL stylesheet, constructing a SOAP request, invoking SOAP response processing, and so on. Note the invocation of soaprequestmanager() and processresponse(). 3. In the Files tab on the left window pane of Oracle SQL Developer, find and open the common.js file under the HOL XFILESV src common js directory. 41

112 4. On the right window pane, you can enter soaprequestmanager in the Search text box to find the Javascript code implementing this function. 5. You can also find the implementation of a Javascript function invoking the XMLHttpRequest by entering the text in the Search text box. 42

113 Examining SOAP request/response, XML, and XSL content You will go back to the XFILES browser window to examine the corresponding SOAP request/response, the XML data transferred, and the XSL style sheet. 1. Click the? icon on the right and select Show Request from the dropdown lost. 43

114 2. Note the popup window will display the corresponding SOAP web service request. Click the (X) icon to dismiss the popup window. <Envelope xmlns=" <Body> <CXMLTYPE-GETXMLINDEXROOTNODEMAPInput xmlns= " <P_PATH_TABLE_OWNER-VARCHAR2-IN>HOL3</P_PATH_TABLE_OWNER-VARCHAR2-IN> <P_PATH_TABLE_NAME-VARCHAR2-IN>PO_SL_BIX_PATH_TABLE</P_PATH_TABLE_NAME-VARCHAR2-IN> </CXMLTYPE-GETXMLINDEXROOTNODEMAPInput> </Body> </Envelope> 3. Select Show Response from the dropdown list. 44

115 4. Note the popup window showing the actual SOAP web service response containing the response below. Click the (X) icon to dismiss the popup window. <?xml version="1.0"?> <soap:envelope xmlns:soap=" <soap:body> <GETXMLINDEXROOTNODEMAPOutput xmlns= " <RETURN> <NodeMap xmlns=" <Namespaces> <Namespace Prefix="ns1"> </Namespaces> <Element xmlns=" ID="4E36"> <Name>PurchaseOrder</Name> <Attrs> <Attr ID="5695">ns1:noNamespaceSchemaLocation</Attr> </Attrs> <Elements> <Element ID="742E">CostCenter</Element> <Element ID="25E7">User</Element> <Element ID="4F8C">Reference</Element> <Element ID="7D80">ShippingInstructions</Element> <Element ID="32E0">LineItems</Element> <Element ID="7A06">Reject</Element> <Element ID="76E2">Requestor</Element> <Element ID="2BF1">SpecialInstructions</Element> <Element ID="5213">Actions</Element> </Elements> </Element> </NodeMap> </RETURN> </GETXMLINDEXROOTNODEMAPOutput> </soap:body> </soap:envelope> 45

116 5. Select Show XML from the dropdown list. 46

117 6. Note the actual XML data without the SOAP protocol tags. Click the (X) icon to dismiss the popup window. 7. Select the Show XSL item from the dropdown list. 8. You will see the XSL file used to transform the XML data into HTML for rendering by the browser. Click the (X) icon to dismiss the popup window. 47

118 48