XML Programming in Java

Transcription

1 Mag. iur. Dr. techn. Michael Sonntag XML Programming in Java DOM, SAX XML Techniques for E-Commerce, Budapest Michael Sonntag 2005 Institute for Information Processing and Microprocessor Technology (FIM) Johannes Kepler University Linz, Austria

2 ??? Questions? Please ask immediately!??? Michael Sonntag 2005

3 XML Programming in Java ebxml, SOAP, Security Metadata,... Java HTML XML XML Schema XML XSLT FO XML Namespace XPath Michael Sonntag XML Techniques for E-Commerce: Programming in Java 3

4 Content Introduction Stages and methods of XML handling "Manual" handling DOM: "Static" handling Parsing Tree traversion Modifying content Writing SAX: "Dynamic" handling Handling event Handlers Filtering and modifications Michael Sonntag XML Techniques for E-Commerce: Programming in Java 4

5 Introduction To be useful, XML must be rather easy to parse Regardless of the target environment, e. g. character encoding Regardless of the programming language Advantage of XML: Even with very simple methods (string processing), XML can be used successfully The better the support, the easier for the programmer! Biggest "problem" of XML: Needs Unicode support Today no longer a problem on most systems! Main areas of use: Data storage: Small but highly structured data» Small: A XML file is no database! Communication content: Any size and type of data to be exchanged between individual systems Portability» Within: sometimes, but usually proprietary formats (objects, etc.) Michael Sonntag XML Techniques for E-Commerce: Programming in Java 5

6 The three main stages of XML handling Parsing the input» Support very diverse Directly manipulating XML is not very efficient or easy» Directly = String manipulation Must be converted into objects or structured data before use Using the content» According to the parser model; main area of the application Searching for content & manipulating (add, change, delete) it Writing to the output» Support not that good, but rather easy Creating new XML from the structured data or objects For further processing Also called: "Result tree serialization"» "object serialization": Making a flat format out of a net of objects! Michael Sonntag XML Techniques for E-Commerce: Programming in Java 6

7 Methods of XML handling XML can be handled manually (=directly as strings) Good and easy for writing simple files Very bad for parsing and manipulation (e.g. foreign charsets!) XML can be handled as a tree of objects (=DOM) Slow parsing (high startup time until everything parsed) Easy to search (especially repeatedly for different data each time) Easy to manipulate large parts (whole subtrees) Easy to change the structure drastically XML can be handled as a stream of events (=SAX) Very fast parsing Easy filtering (changing few data and writing it out immediately)» Structure must stay mostly the same, however! Easy searching for certain data once Especially bad for repeated searching of different data Michael Sonntag XML Techniques for E-Commerce: Programming in Java 7

8 Manual handling: Writing Creating XML by just writing it out as text Very easy, very fast No support for checking validity» Proper nesting, closing of tags, etc. must all be done manually Character set may be difficult to handle Typically done by hierarchically calling methods Example: public String toxml() { StringBuffer res=new StringBuffer(); res.append("<elem"); if(value!=null) res.append(" attribute=\""+value.tostring()+"\""); res.append(">"); res.append(childelem.toxml()); res.append("</elem>"); return res.tostring(); } Michael Sonntag XML Techniques for E-Commerce: Programming in Java 8

9 Not really a good idea: Manual handling: Parsing and modifying Document type or schema impossible to verify (validation), Namespaces are difficult ("inheritance" of namespaces), Solving entity references, etc. Only usable for finding a specific part in an exactly known format (e. g. the second "file" element) Better avoid this! Reason: Very complicated and still extremely unreliable» Different character encodings,» Small format changes require large modifications» This is like a binary format: But we wanted to do away with this! Use one of the available parsers/apis/systems» SAX or specialised libraries (e.g. Apache Commons Digester)» Fast, reliably, efficiently, easy to learn, Michael Sonntag XML Techniques for E-Commerce: Programming in Java 9

10 DOM = Document Object Model DOM Overview API for valid HTML and well-formed XML documents» HTML probably only at least version 4.1 (XHTML works as XML) Specialties: No entities, no character references (replaced by their text content) Serialization may differ from original (see above)» But "semantically" the same! DOM is a specification of language-independent interfaces Language independent API Implementation is not touched at all! Different levels: Level 1 = Core document model Level 2: Stylesheets, traversal, events, Level 3 (in the work): Loading, saving, validation, views, XPath,» Core and Load&Save are recommendations now! Michael Sonntag XML Techniques for E-Commerce: Programming in Java 10

11 DOM Modules DOM is separated into several modules Which can be omitted (except Core, obviously!) Modules: Core: Elements, attributes, document, node, text, XML: CDATASection, PI, DocType, HTML: Specifies the HTML elements Others: Views, stylesheets, CSS, CSS2 Events: Events, UI, mouse, mutation, HTML» Based on the "Listener" pattern Range: Defining spans and positions within a document Traversal: "Going" through parts of the document» NodeIterator: Flattened view of nodes in document order (prev, next)» TreeWalker: Hierarchical view of nodes (+parent, first-/nextchild, )» Both can be filtered to leave out "uninteresting" nodes Michael Sonntag XML Techniques for E-Commerce: Programming in Java 11

12 DOM Java Implementation Java: org.w3c.dom.* package in J2SE <5.0 Only the Core and XML modules are implemented! Especially missing: Traversal» However, most TreeWalker methods are contained in class "Node" JDK <1.5 contains old versions! Many problems through this!» Place the new jars into "../jre/lib/endorsed" More implemented in Apache Xalan, Xerces, J2SE 5.0 Complete DOM Level 2, almost complete Level 3 L&S support Includes also other classes needed» E. g. actual parsers (DOM is only a set of interfaces!) Weak typing: Everything is a "Node", elements are "Element" <Address> is not "Class Address", but just "Class Element", which happens to have in its field "name" the string "Address" as value!» See JAXB (Java API for XML Binding)! Michael Sonntag XML Techniques for E-Commerce: Programming in Java 12

13 DOM Nodes (1) In DOM, an XML document is a tree made of "Nodes" One special root node (=the "document node")» NOT the document element, but rather the XML file as a whole! Each node has one parent and 0..N child nodes Some nodes are not in the tree» Element attributes are nodes, but not children of their element Each node has the following: Local name, namespace URI, prefix, Node name: Elements, attributes,... = prefixed name; Others: Type» Examples: <cf:age> = "cf:age", <!-- ABC --> = "#comment", "ABC" = "#text", <?php echo;?> = "php" Node value: null or content» Example: <cf:age> = null, <!-- ABC --> = "ABC", "ABC" = "ABC", <?php echo;?> = "echo; " Attributes: A list of attributes for elements; null for everything else Michael Sonntag XML Techniques for E-Commerce: Programming in Java 13

14 DOM Nodes (2) Order of children is document order (=textual order) Example: <?xml version="1.0"?> <?xml-stylesheet type="text/css" href="xml-rpc.css"?> <! Example XML file --> <!DOCTYPE demo SYSTEM "demo.dtd"> <demo> <content>abc</content> </demo> Document node has 4 children in this order: XML prologue is NOT included!» Processing instruction node for the stylesheet declaration» Comment node» Document type node for the external document type declaration» Element node for the root <demo> element This has again 3 children: Text for linebreak, element, text for space Michael Sonntag XML Techniques for E-Commerce: Programming in Java 14

15 DOM Parsing XML Core Parsing is not contained in DOM Core: MANY versions! Use any of the available parsers for this» Some maybe based on SAX (see below); reuses some SAX interfaces Typical way: javax.xml.parsers DOM: DocumentBuilder, which is created by a DocumentBuilderFactory DocumentBuilderFactory: Contains several very general settings for parsing» E. g. namespaces, comment ignoring, validating, Produces appropriate DocumentBuilder implementations DocumentBuilder: Creating empty documents Parsing from various sources (File, InputSource, InputString, URI) Setting the entity resolver (for obtaining external entities) Setting the error handler (notifications on parsing problems) Michael Sonntag XML Techniques for E-Commerce: Programming in Java 15

16 DOM Parsing Level 3 Load&Save Most things now standardized (e.g. parameters) Get a DOMImplementationLS This is still implementation-dependant! Get a LSParser and configure it Parse the input DOMImplementationRegistry registry=domimplementationregistry.newinstance(); DOMImplementationLS impl=(domimplementationls)registry.getdomimplementation("ls");» Bootstrapping of the whole process LSParser builder=impl.createlsparser(domimplementationls.mode_synchronous,null); DOMConfiguration config = builder.getdomconfig(); config.setparameter("error-handler",this); config.setparameter("validate",boolean.true); config.setparameter("schema-type",w3c_xml_schema); document=builder.parseuri(filename); Michael Sonntag XML Techniques for E-Commerce: Programming in Java 16

17 DOM Tree traversing Tree traversing can be done completely manual and freely: Element docelem=document.getdocumentelement(); Element comp=(element)docelem.getfirstchild().getnextsibling(); String zip=((element)comp.getfirstchild()).getattribute("zip"); Document: <doc><name/><contact><address ZIP="4040"> doc name contact address ZIP Other possibility: DOM traversal» org.w3c.dom.traversal package (JDK 1.5 or Apache Xerces required!) Michael Sonntag XML Techniques for E-Commerce: Programming in Java 17

18 DOM Manipulation Removing nodes from the tree Node.removeChild Adding nodes to the tree Node.appendChild Node.insertBefore Changing node content/attributes/children,» "Node attributes" are different from XML attributes! Node.replaceChild Node.setNodeValue (Text, attribute value, etc.) The node name itself cannot be changed!» Remove node and add new one (take care to move children also)! Element.setAttribute» These are the XML attributes Namespace functions are available, but omitted here! Michael Sonntag XML Techniques for E-Commerce: Programming in Java 18

19 DOM Writing Writing must be done manually or using other classes DOM core does NOT provide a way for this (e.g. no "toxml()")! Typically: org.apache.xml.serializer.* Or: javax.xml.transform.* and javax.xml.transform.dom.* Or: DOM Level 3 Load&Save Example: Parse an order file Extract all product numbers and the delivery address name Extract all plain text (from all elements!) Please note: Two versions provided With transformer» Will not print a DTD (DTD's cause LOTS of problems!); schema works With DOM Level 3 Load&Save (very simple; see file!)» Uses XPath for name extraction DOMCoreSample.java, DOMLSSample.java, build.xml Michael Sonntag XML Techniques for E-Commerce: Programming in Java 19

20 SAX Overview SAX = Simple API for XML SAX handles XML as a stream of events E. g. Element "addr" starts, text character, attribute "ZIP" found, Basic idea: Fast content access Output is again missing! But rather easy: Each event writes out what it is about But still needs a parser Main interface: ContentHandler 11 methods for parsing XML document content Additional interface: DTDHandler For handling DTDs No schema support! Namespace support is available in SAX2 (=SAX Version 2) Michael Sonntag XML Techniques for E-Commerce: Programming in Java 20

21 SAX Configuration Additional "things" can be configured as "features":» Or for determining additional information, e. g. whether the document is marked as "standalone" or not» setfeature(boolean) Whether to process external entities (general, parameter) Namespace processing Use of extended interfaces» Attributes2, Locator2, EntityResolver2 (all in org.xml.sax.ext)» These expose additional information Whether validation should be done URI handling (base URIs, etc.) Some interfaces must be set by using "properties"» setproperty(string,object) Declaration handler, lexical handler, Michael Sonntag XML Techniques for E-Commerce: Programming in Java 21

22 SAX Events Whenever something "interesting" happens, a method from your application is called Interface defines the "interesting" events Application decides what to do with the events it receives This is the "Observer" design pattern! No going back: Processing is only from start to end No hierarchy tracking: Events are just "singular" items! All events are conceptually at the same level» No depth provided (count manually if required!)» No access to the "parent", preceding, element possible If start and end tag do not match, an exception is thrown How deep, within which other element, : Application!» If needed, use a stack (e. g. java.util.stack) Michael Sonntag XML Techniques for E-Commerce: Programming in Java 22

23 SAX ContentHandler (1) Characters: Textual content encountered Might be all till the next event, or only some portion (no guarantee!)» Accumulate in a buffer and act in the endelement event StartDocument: First call (start of file) EndDocument: Last call (end of file) StartElement: Beginning of an element Provides namespace (if configured), local name and qualified name Also provides all attributes from the file» Not those from DTD and which are missing (e. g. #IMPLIED, default)!» No ordering available! EndElement: End of an element Provides namespace (if configured), local name and qualified name Does NOT provide the attributes (again)» These are available only in the StartElement event! Michael Sonntag XML Techniques for E-Commerce: Programming in Java 23

24 SAX ContentHandler (2) ProcessingInstruction: When a PI is found Reports target and data SkippedEntity: Whenever an entity is skipped When the declaration is missing (e. g. external DTD && not loaded) Normal entity references are replaced and not reported here! Start-/EndPrefixMapping: Manual namespace processing Required only in very special circumstances IgnorableWhitespace: Whitespaces which should be ignored Validating parser: Whitespaces where only elements should be SetDocumentLocator: Providing a mechanism for determining (later) where an event appeared E. g. for custom error messages ("in line 10 at character 3") Michael Sonntag XML Techniques for E-Commerce: Programming in Java 24

25 SAX ContentHandler Example xmlreader.setcontenthandler(this); Register this class as the content handler public void characters(char[] ch,int start,int length) { text.append(new String(ch,start,length)); } All textual data (regardless of nesting!) is added to 'text' public void startelement(string uri, String localname, String qname, Attributes atts) { if("product".equals(localname) && atts.getindex("productnumber")>=0) { prdouctnumbers.add(atts.getvalue("productnumber")); } } Attribute 'productnumber' of element 'product' is added to the vector 'productnumbers'» Wherever this appears in the document! Michael Sonntag XML Techniques for E-Commerce: Programming in Java 25

26 SAX ErrorHandler Instead of throwing an exception on errors, the error handler is called if registered» Exception: SAXException (but ONLY thrown for fatal errors!) Fatal error: Parsing cannot continue in any meaningful way A kind of very simple logging Replacement for catch-clauses 3 Methods; all receive the exception as parameter Warning: Parsing can/will continue Error: Error according to XML specification» Example: Document is not valid Doesn't match its DTD, but is still well-formed XML Fatal error: Fatal error according to XML specification» Example: Document is not well-formed» Events may or may not be reported afterwards» The resulting document should not be used at all Michael Sonntag XML Techniques for E-Commerce: Programming in Java 26

27 SAX EntityResolver When external entities are encountered, this resolver is called (if it is registered) E. g. external DTD, external parameter entities Method: resolveentity(publicid,systemid) Provides full data on both IDs If null is returned, the parser should try to find it by itself Or not at all, if it does not want to Will usually try to open it as an URI Must return an InputSource otherwise This encapsulates public and system ID, encoding and a character or a byte stream Michael Sonntag XML Techniques for E-Commerce: Programming in Java 27

28 SAX EntityResolver Example xmlreader.setentityresolver(this); For finding external files (entities, DTDs, schemas, ) public InputSource resolveentity(string publicid, String systemid) { System.out.println("Entity to resolve: "+publicid+ " (System-ID: "+systemid+")"); return null; } Example: XML file contains "<!DOCTYPE message SYSTEM "Message.dtd">" Message: "Entity to resolve: null (System-ID: file:///c:/message.dtd)"» The systemid is already extended (by default) with the base URI Which happened here to be the directory "C:\" Michael Sonntag XML Techniques for E-Commerce: Programming in Java 28

29 SAX Missing parts Comments: They are ignored and stripped away! Unskipped entities: Resolved to character data CDATA sections: Resolved to character data See LexicalHandler for all three! DTD information See DTDHandler and DeclHandler! XML declaration and its parts Intended for SAX 2.1 (not yet finished!) Several minor things of no syntactic importance, but which prohibit exactly reproducing the document E. g. character references are currently silently changed to text Michael Sonntag XML Techniques for E-Commerce: Programming in Java 29

30 SAX Filtering Filters: For chaining of several handlers XML SF 1 Can be either passing on of events, or the previous stage posing as a complete XML parser XMLFilter: Just contains a parent "parser" (XMLReader) XMLFilterImpl: Helper class that filters (everything stays unchanged)» Required methods must me overwritten» Parent method must be called Model for picture above: SF 2 is parent of SF 1, which is original receiver Application is parent of SAX 2 (and final receiver) Michael Sonntag XML Techniques for E-Commerce: Programming in Java 30 SF 2 App.

31 SAX Writing Writing must be (again) done manually / using other classes SAX does NOT provide a way for this (e.g. no "logging" of events) Typically: Using a filter and doing manual writing at the end Or: org.apache.xml.serializer.* Or: javax.xml.transform.* and javax.xml.transform.sax.* Example will NOT work correctly regarding the DTD with SAX 1.0! Example how difficult writing XML can be even with existing software! That's why DTD's should not be used (Schemas no problems!) Also, DTD content (e. g. element declaration) is an SAX extension» In SAX 2.0 this works! <!-- Test file for DOM and SAX --> [ <!ELEMENT doc (name,company)> <!DOCTYPE doc]> SAXSample.java, build.xml Michael Sonntag XML Techniques for E-Commerce: Programming in Java 31

32 SAX vs. DOM SAX: Models the parser Push model: Information is supplied, program discards uninteresting Much more efficient Suitable for stream processing Good for manipulating small parts Good for manipulating parts located closely together Requires few memory Once data is passed, there's no going back Processing can start before the document is fully available Modification only possible "in-time" "Private" or custom object hierarchy can be built easily DOM: Models the document Pull model: Program extracts data Much more "natural" Suitable for storage processing Good for manipulating large subtrees Good for manipulating data wide apart Requires large amount of memory Any data is available all the time Document must be fully parsed before work can start at all Allows repeated modification Predefined hierarchy of objects only Michael Sonntag XML Techniques for E-Commerce: Programming in Java 32

33 Literature DOM: SAX: Java2EE 1.4 Tutorial : Moller/Schwartzbach: The XML Revolution Harold: Processing XML with Java Harold: Advanced XML Programming xml/advanced_xml_programming.html Michael Sonntag XML Techniques for E-Commerce: Programming in Java 33