XML for Java Developers G Session 10 - Main Theme XML Information Retrieval (Part II) And XML-Based Frameworks (Part I)

Transcription

1 XML for Java Developers G Session 10 - Main Theme XML Information Retrieval (Part II) And XML-Based Frameworks (Part I) Dr. Jean-Claude Franchitti New York University Computer Science Department Courant Institute of Mathematical Sciences 1 Agenda Summary of Previous Session XML Object Persistence Advanced XQuery Concepts Presentation Oriented Publishing (POP) Frameworks Web Services Assignment 5a-c (due next week) 2 1

2 Summary of Previous Session Summary of Previous Session Applications of XML to Database Technology XML Query Languages XPath XML Queries XQuery: A Query Language for XML XML Query Engines XML Registries API Web Services Support in J2EE /.Net Assignment 5a+5b 3 Part I XML Object Persistence Also See: 4 2

3 Object Persistence Need for objects to exist beyond an application s lifetime Objects Objects Objects Application Data Store Limitations of the OOP / RDBMS direct coding approach Difficult to handle data structure changes Difficult to reuse persistence logic Solution is to use a Persistence Framework 5 Direct JDBC JDBC is NOT a Persistence Framework JDBC is a database connection utility Ok for Window on data style application Ok when business logic is entrenched on database rows Application JDBC SQL 6 3

4 XML Object Persistence ( SOAP Competitors" in Started as SODL and XMOP Simple Object Definition Language XML Metadata Object Persistence XML and JavaBeans interoperability (e.g., BML, Coins, etc.) XML and EJB integration XML serialization for Java (e.g., Koala, etc.) SOAP - XML-RPC protocol Ability to invoke (persistent) objects methods remotely 7 Transferring Data Between Databases and Java Objects via XML Example Project: XSU ( Transform data retrieved from object-relational database tables into XML Extract data from a XML document and insert the data into the appropriate columns/attributes of a table Extract data from a XML document and apply this data to updating or deleting values of the appropriate columns/attributes Implementation DB: Oracle 9i Release 1 Server XSU Utility/Tool + Castor Alternatives to Castor: XML DB (Oracle 9i Release 2) 8 4

5 XSU Architecture XSU Castor XML Oracle 9i Sample Data XML document Java classes Objects 9 Persistence Broker Design Pattern Enables full separation of business and persistence logic Easily customizable for different applications Enables added functionality (e.g., object caching) 10 5

6 Persistence Broker Architecture Simple architecture Using a Persistence Broker User Interface Classes User Interface Classes Business classes Business classes Persistence Broker Cache Database Database 11 Applications Persistent Queries Objects Persistent Objects Persistent Objects Persistence Broker Persistent Identities Objects Persistent Objects Persistent Objects Cache Persistent Objects Caches persistent objects Looking up cached instances Retrieving Storing and deleting Deleting storage and deletion Delegating queries from the application to lower level Contains mappings between classes and tables Returning the field values for retrieved objects Specifying the criteria field values Handles executing SQL statements Persistent Queries Objects Persistent Field Objects Values Persistent SELECT Objects Queries Persistent Objects Persistent Objects Persistence Mapping Persistent Field Objects Values Persistent Field Objects Values Query Mechanism Persistent ACTION Objects Queries Persistent Objects Persistent Field Objects Values Persistent Field Objects Values Persistent Column Objects Values Reflection Conversion Accesses object fields dynamically Specifying the field values of objects to store or delete Converts object field values to table column values and vice versa Database-specific SQL queries Persistent SELECT Objects Results Database 12 6

7 Object Persistence Layer in J2EE Abstracts persistence details from the application layer object-level querying and creation results are objects Objects J2EE & Web Services Objects Persistence Layer object creation and updates through object-level API results are returned as raw data rows JDBC SQL API uses SQL or database specific calls 13 TopLink Runtime Architecture Application Server Data Source Application TopLink Mappings JSP, Servlet, Struts, etc. App Logic Session Cache Query Data Access JDBC Entities Java Objects EJB Entity Beans CMP/ BMP J2EE Container TX JTA 14 7

8 TopLink & XML / J2EE XML documents can be represented at different levels of abstractions in J2EE applications: Parsed document (DOM, SAX ) - parser Unmanaged Java objects from non-transactional data source data converter Managed Java objects from a transactional data source persistence manager Developers do not need to work with low level XML documents and manually code persistence manager functionality 15 Three Levels of XML Representation DOM Unmanaged Object Managed Object/EJB Persistence Manager O-X Data Converter XML Parser File XML Document JDBC J2C Web Service BPM JMS XDB EIS 16 8

9 Sun JDO Java Data Objects (JDO) First standardized, completely object oriented approach to object persistence. Developed as Java Specification Request 12 under the support of the Java Community Process. Application programmers can use JDO to directly store their Java domain model instances into the persistent store. JDO fills a large gap in the area of database programming. 17 Benefits of Using JDO Transparent persistence Database independence Portability Ease of use High performance Integration with EJB 18 9

10 JDO Architecture 19 Software Stacks Supported MyApplication Persistence Manager (XJDO Implementation) Java-based Data Store Access (exist XML DB Driver Class Impl) Data Store (exist Native XML DB) MyApplication Persistence Manager (LiDO JDO Implementation) Java-based Data Store Access (mysql-connector-java ) Data Store (My SQL) MyApplication Persistence Manager (LiDO JDO Implementation) Java-based Data Store Access (LIBeLIS file db connector) Data Store (LIBeLIS' FileDB) 20 10

11 exist Native XML DB exist is an Open Source, native XML DB exist features Completely written in Java Lightweight Efficient Index-based XPath query processing Extensions for keyword search Tight integration with existing XML development tools Can be deployed two ways: A stand-alone server process, inside a servlet-engine Directly embedded into an application. Other popular native XML DB: Apache Xindice 21 Conversion Process for Retrieved Objects exist XML Database XML Document Stored in XML Database <xobj package= xobj.test.b eans class= Person > <field name= firstname value= Luann /> <field name = lastname value= Ahrens /> Document object when retrieved from XML database as DOM org.w3c.document Element Name= xobj Attr Package= xjdo.test.beans Object Instance Returned to Application Developer xjdo.test.beans.person firstname= Luann lastname= Ahrens address= 555 NoWhere St

12 XJDO and XSLT Compatibility Possible to use the data in XML data base for other means without using the XJDO application Data can be accessed by other than via Java applications using XJDO Using XSLT and XPath, the XJDO XML format can be transformed to a different XML format <xobj package= xjdo,test.beans name= Person > <field name= fname value= Luann /> <field name= lname value= Ahrens /> <field name= address value= 555 NoWhere St. /> </xobj> <xjdo.test.beans.person> <firstname>luann</firstname> <lastname>ahrens</lastname> <address>555 NoWhere St.</address> </xjdo.test.beans.person> XSLT Transformation 23 Part II Advanced XQuery Concepts 24 12

13 XQuery Today XQuery: The W3C XML Query Language DOM+XPath+XSLT applications can now be implemented in just one language: XQuery XQuery is expressive, concise, easy to learn XQuery is implementable, and optimizable XQuery supports integration of data from multiple sources Several implementations of XQuery are currently available XQuery provides preliminary support for update 25 XQuery Design Goals Language Expressive power Functionality derived from XML-QL, XQL, SQL, OQL Applicable to the many different types of XML data Implementation based on published use-cases XQuery Engine Implementations Can be implemented on top of traditional databases, XML repositories, XML programming libraries, etc. Queries may combine data from many sources Minimalist design Small, easy to understand, clean semantics 26 13

14 Querying Heterogeneous Data DOM DOM SAX SAX DBMS XML W3C XML Query Data Model XQuery W3C XML Query Data Model DBMS XML Java Java COBOL COBOL 27 XQuery Expressions XQuery is a functional language Each query is an expression Expressions can be easily combined Structure of a query Namespace declarations (optional) Function definitions (optional) Query expression: may include many expressions 28 14

15 XQuery Expressions Path expressions: /a//b[c = 5] FLWR expressions: FOR... LET... WHERE... RETURN Element constructors: <a>... </a> Variables and constants: $x, 5 Operators and function calls: x + y, -z, foo(x, y) Conditional expressions: IF... THEN... ELSE Quantifiers: EVERY var IN expr SATISFIES expr Sorted expressions: expr SORTBY (expr ASCENDING,... ) Updates expressions: INSERT, REPLACE, DELETE 29 Sample Document <books> <book year= 2002"> <title>xml and Java</title> <author> <last>maruyama</last> <first>h.</first> </author> <publisher>addison- Wesley</publisher> <price> 34.99</price> </book> </books> 30 15

16 Element Constructors # Element constructors look like the XML they construct <book year= 2002"> <title>xml and Java</title> <author> <last>maruyama</last> <first>h.</first> </author> <publisher>addison- Wesley</publisher> <price> 34.99</price> </book> 31 Path Expressions <books> <book year= 2002"> # XQuery uses the abbreviated syntax <title>xml and Java</title> # of XPath for path expressions <author> document( books.xml ) <last>maruyama</last> <first>h.</first> /books/book/author </author> <publisher>addison- //author[last= Maruyama Wesley</publisher> and first= H. ] <price> 34.99</price> </book> document( books.xml )//author </books> 32 16

17 Path Expressions - Extensions # Range expressions /books/book/author[1 TO 2] # BEFORE and AFTER //book[ author[last= Maruyama ] BEFORE author[last= Clark ] ] # Namespaces NAMESPACE rev = " //rev:rating # Dereference //publisher/web/@href->html 33 FLWR Expressions FOR - LET - WHERE - RETURN Similar to SQL s SELECT - FROM - WHERE FOR $book IN document("books.xml")//book WHERE $book/publisher = "Addison-Wesley" RETURN <book> { $book/title, $book/author } </book> 34 17

18 FOR vs. LET FOR iterates on a sequence, binds a variable to each node LET binds a variable to a sequence as a whole FOR $book IN document("books.xml")//book LET $a := $book/author WHERE contains($book/publisher, "Addison-Wesley ) RETURN <book> { $book/title, <count> Number of authors: { count($a) } </count> } </book> 35 Inner Joins FOR $book IN document(" $quote IN document(" WHERE $book/isbn = $quote/isbn RETURN <book> { $book/title } { $quote/price } </book> SORTBY (title) 36 18

19 Outer Joins FOR $book IN document("books.xml")//book RETURN <book> { $book/title } { FOR $review IN document("reviews.xml")//review WHERE $book/isbn = $review/isbn RETURN $review/rating } </book> SORTBY (title) 37 Combining Expressions <bibliography> { FOR $book IN document("books.xml")//book RETURN <book> { $book/author, $book/title } </book> SORTBY (author, title) } </bibliography> 38 19

20 Combining Expressions <bibliography> Expression </bibliography> 39 Combining Expressions <bibliography> Expression </bibliography> Can be extended as: <bibliography> { FOR $book IN Expression RETURN Expression } </bibliography> 40 20

21 Combining Expressions (continued) <bibliography> { FOR $book IN Expression RETURN <book> { Expression, Expression } </book> SORTBY (Expression, Expression) } </bibliography> 41 Combining Expressions (example) <bibliography> { FOR $book IN document("bib.xml")//book RETURN <book> { $book/author, $book/title } </book> SORTBY (author, title) } </bibliography> 42 21

22 Functions Built-in functions max(), min(), sum(), count(), avg() distinct(), empty(), contains() the normative set has not yet been fixed User-defined functions Defined in XQuery syntax May be recursive May be typed Extensibility mechanisms will be added 43 Functions (continued) FUNCTION depth(element $e) RETURNS integer { -- An empty element has depth 1 -- Otherwise, add 1 to max depth of children IF empty($e/*) THEN 1 ELSE max(depth($e/*)) + 1 } depth(document("partlist.xml")) 44 22

23 Data Types W3C XML Schema simple types string "Hello" boolean true, false integer 47, -369 float -2.57, 3.805E-2 Type constructor functions date(" ") Operators and functions to be defined Bibliography Example <?xml version="1.0"?> <books> <book> <title> XML and Java </title> <author> <lastname> Maruyama </lastname> <firstname> Hiroshi </firstname> </author> <pubinfo> <publisher> Addison Wesley </publisher> <price> </price> <year> 2002 </year> </pubinfo> </book> </books> 46 23

24 Books by Author <?xml version="1.0"?> <books> <book> <title> XML and Java </title> <author> <lastname> Maruyama </lastname> <firstname> Hiroshi </firstname> </author> </name> <pubinfo> <publisher> Addison Wesley </publisher> <price> </price> <year> 2002 </year> <author> </pubinfo> </book> </books> <booksbyauthor> <author> <name> <last> Maruyama </last> <first> Hiroshi </first> <book> XML and Java (1 st Edition) </book> <book> XML and Java (2 nd Edition) </book>... </booksbyauthor> 47 Inverting the Hierarchy <?xml version="1.0"?> <books> <book> FOR $a IN distinct(document( books/books.xml")//book/author) <title> LET $b XML := and document( books/books.xml")//book[author Java </title> = $a] <author> RETURN <lastname> Maruyama </lastname> <firstname> <book> Hiroshi </firstname> </author> { $a } <pubinfo> { $b/title SORTBY (.) } <publisher> Addison Wesley </publisher> </book> <price> </price> SORTBY(author/last, <year> 2002 </year> author/first) </pubinfo> </book> </books> 48 24

25 INSERT, DELETE, REPLACE INSERT FOR $e IN /emp INSERT <n_skills> count($e/skill) </n_skills> BEFORE $e/skill[1] REPLACE FOR $e IN /emp WHERE $e/empno = "1234 REPLACE $e/job WITH <job> Software Architect"</job> 49 INSERT, DELETE, REPLACE (continued) DELETE FOR $e IN /emp/[job = "Programmer"], $s IN $e/skill WHERE $s/rating < 4 OR $s/cert_date < date( ") DELETE $s 50 25

26 Current Limitations on Update No distributed update - single data source No updates on views 51 Advanced XQuery Concepts Mainstream XQuery Engines Software AG s QuiP H. Katz XQEngine API for XML Databases Supported in exist and ozone-db Experiment with Complex Queries and QuiP 52 26

27 XQuery Additional Information W3C XQuery W3C XML Query Use Cases W3C XML Query Requirements W3C XML Query Data Model W3C XML Query Algebra 53 Part III Presentation-Oriented Publishing Frameworks 54 27

28 Client-Side POP IE5 Server-Side POP Cocoon & XSP Rocket CPAN s Perl Framework References POP Frameworks Apache Cocoon XML based publishing Framework An Apache Software Foundation open source project Written in Java, runs mostly as a servlet Started as a simple servlet based XSL styling engine for site Current version is in the second generation of evolution Designed for scalability ( uses SAX processing ) -- can process huge documents using small amount of memory 56 28

29 Apache Cocoon (continued) Cocoon promotes the separation of Content, Logic, Presentation and Management in website design. 57 Cocoon Servlet Dirs. & Files Directory for auto mounting sub-sitemaps Cocoon Configuration file A sub-sitemap directory Main sitemap file Directory for log files 58 29

30 Cocoon Sitemap Sitemap Goal Used to de-couple the exposed URI space from the actual location of resources Allows easily changeable specification of processing steps Sitemap Contents Component declarations generators, transformers, serializers,... Resource declarations named collection of pipeline components Pipeline declarations sequential arrangement of components for processing 59 Cocoon Sitemap (continued) A sitemap is an XML file Sitemaps are hierarchical -- A sitemap can point, explicitly or implicitly, to subsitemaps A sitemap is translated into a java program and is compiled into bytecode Changes to sitemaps can be loaded dynamically and asynchronously 60 30

31 Sample Cocoon Pipeline <map:pipeline> <map:match pattern="hello.html"> <map:generate src="docs/samples/hello-page.xml"/> <map:transform src="stylesheets/page/simplepage2html.xsl"/> <map:serialize type="html"/> </map:match> <map:match pattern="images/**.png"> <map:read src="resources/images/{1}.png" mimetype="image/png"/> </map:match> <map:handle-errors> <map:transform src="context://stylesheets/system/error2html.xsl"/> <map:serialize status-code="500"/> </map:handle-errors> </map:pipeline> 61 Cocoon Request Processing Request is dispatched to matching pipeline Basic pipeline operation The generator generates XML content Zero or more transformers transform the content The serializer writes it to the output stream Different Kinds of generators File, Directory, XSP, JSP, Stream, Different Kinds of transformers XSLT, I18N, Log, Different Kind of Serializers HTML, XML, Text, PDF, SVG,

32 Dynamic Content Generation from XSP <!-- A simple XSP Page --> <xsp:page language="java xmlns:xsp= > <page> <title>a Simple XSP Page</title> <content> <para>dynamically generated list:</para> <ul> <xsp:logic> for (int i=0; i < 3; i++) { <li> Item <xsp:expr>i</xsp:expr> </li> } </xsp:logic> </ul> </content> </page> </xsp:page> 63 Part IV Web Services 64 32

33 Web Services Summary Web Services are described in WSDL exchange SOAP messages use HTTP as transport optionally, registered in UDDI registry Interaction can happen using RPC style or Document Exchange style, synchronously or asynchronously Knowledge of Implementation language or deployment platform is not required for interaction. Web Services promise Interoperability. 65 SOAP Summary XML based protocol for exchange of information Encoding rules for datatype instances Convention for representing RPC invocations Designed for loosely-coupled distributed computing Used with XML Schema Transport independent SOAP with Attachments allow arbitrary data to be packaged. SOAP Message Structure SOAP Envelope Header Entries [Header Element] Body Element [Fault Element] 66 33

34 WSDL Summary A WSDL document describes What the service can do Where it resides How to invoke it WSDL are like IDL but lot more flexible and extensible Defines binding for SOAP, HTTP GET/POST and MIME WSDL descriptions can be made available from an UDDI registry WSDL Document Structure WSDL Document [Types] {Messages} {PortTypes} {Bindings} {Services} 67 Web Services Platform Environment, tools, libraries and other resources for: Development of web services top down -- start with user code and generate WSDL bottom up -- start with WSDL and generate interfaces middle out -- a combination discovery of service interfaces Deployment of web services publishing of services pre and post processing of SOAP requests, responses processing requests for WSDL descriptions Consumption of Web Services discovery of Web Service invocation of/interaction with Web Services 68 34

35 Java Platforms and Standards Java Platforms for Web Services Apache SOAP 2.2, Apache Axis, UDDI4J, Cocoon JWSDK ( Reference Implementation from Sun ) etc. Java Standards for Web Services JAXM ( Java XML Messaging ) JAX-RPC ( Java Web Service Client Prog. model) JSR 109 ( Java Web Service Prog. Model, deployment,...) JAXR ( Java API to access Web Service Registries ) J2EE 1.4 ( J2EE platform for Web Services ) Java standards promise portability 69 Current State of Java Platforms and Standards Handle synchronous RPC style Web Services quite well Not so good at supporting Web Services with document exchange style, especially with asynchronous interface Have difficulty dealing with large data Many approaches to build and consume Web Services ( but must adhere to wire protocols ) Standards in many areas are still emerging routing, reliable messaging security transactioning

36 Using Cocoon as a Presentation Framework for Web-Services WS Registry Cocoon Servlet Container Web Services Client Tier Web Tier Enterprise Apps 71 Cocoon as a Processing Framework for Web Services (continued) WS Registry Servlet Container Cocoon Web Services Client Tier Web Tier P2 P1 Enterprise Apps 72 36

37 Sample Implementation Use Google Web Services API from to create a simple web application that accepts a search string from user and displays the list of result entries. Sample is based on the XSP contributed by Ugo Cei in Cocoon-dev mailing list Application components: sitemap.xmap -- Sitemap for this application index.html -- HTML file to accept search string google.xsp -- XSP file that makes the SOAP call to Google Web Service using SOAP logicsheet. search-results.xsl -- XSL stylesheet to transform SOAP response from Google Web Service to HTML page Downloadable from: 73 Application Architecture Browser Cocoon Servlet Google WS index.html Maps request to index.html /search?q=... Search results in HTML executes google.xsp SOAP request SOAP response Applies search-results.xsl 74 37

38 Part V Conclusions 75 Summary Bindings approaches are currently implemented between XML and JavaBeans/EJBs Software AG s Quip implements complex query processing as per XQuery 1.0 Server-side POP is the approach of choice for XML processing 76 38

39 Readings Readings XML and Java: Chapter 11, Appendix D Developing Java Web Services: Chapters 6, 9, Handouts posted on the course web site Project Frameworks Setup (ongoing) Howard Katz s XQEngine, Software AG s QuiP Apache s Web Server, TomCat/JRun, and Cocoon Apache s Xerces, Xalan, Saxon Antenna House XML Formatter, Apache s FOP, X-smiles Publishing Systems at Visibroker 4.5 (or BES 5.2.1), WebLogic , WAS 5.0 POSE & KVM (See Session 3 handout) 77 Assignment Assignment #5: #5a+b continued from last week #5c: This part of the project focuses on the discovery/design/development of the application presentation layer architecture using XML-Based presentation frameworks. The discovery/design/development process should adhere to the following steps: (a) Determining how W3C recommendations will be applied (e.g., determine when DOM is appropriate, etc.), (b) Using the details and syntax of core W3C recommendations (XML, XSL, DOM, XPath, namespaces, DTDs, XML well-formedness, Schemas, etc.), (c) Refining the application presentation layer architecture design as necessary based on data model, data integration, data rendering, and data query issues 78 39

40 Assignment (continued) Assignment #5: #5c: This part of the project also focuses on the discovery/design/development of the non-presentation specific application architecture using XML-Based frameworks. The discovery/design/development process should follow the same steps as the ones listed in assignment #5a. Additionally, particular care should be taken at determining the implications of a given architecture on the overall XML design related considerations More specific project related information, and extra credit assignments will be provided during the session 79 Next Session: XML-Based Frameworks (Part II) and Additional XML Services XML and Application Servers Presentation Oriented Publishing (POP) Frameworks Message Oriented Middleware (MOM) Frameworks Web Services XML Messaging XML Security 80 40