Web Technologies. XML data processing (I) DOM (Document Object Model) Dr. Sabin Buraga profs.info.uaic.ro/~busaco/

Transcription

1 Web Technologies XML data processing (I) ⵄ DOM (Document Object Model)

2 The golden rule is that there are no golden rules. George Bernard Shaw

3 How can we process the XML documents? Dr. Sabin Buraga profs.info.uaic.ro/~busaco/

4 Types of XML processing manual processing e.g., regular expressions

5 Types of XML processing object-oriented processing DOM (Document Object Model) non-dom

6 Types of XML processing event-driven processing SAX (Simple API for XML) XPP (XML Pull Parsing) see next lecture

7 Types of XML processing simplified processing Simple XML see next lecture

8 Types of XML processing specific processing via specialized APIs in order to process particular document types e.g., RSS, SOAP, SVG,

9 XML (parsers) processors without validation verifying only if the document is well formed Expat, libxml, MSXML,...

10 XML (parsers) processors with validation verify if the document is valid, according to a XML validation method e.g., DTD Apache Xerces, JAXP, libxml, MSXML,...

11 DOM introduction DOM interfaces DOM Core DOM level 2 DOM level 3 DOM level 4 implementations DOM in the Web browser

12 dom: intro Goal: object-oriented standardized processing of XML and/or HTML documents

13 dom: features An abstract API for XML/HTML

14 dom: features An abstract API for XML/HTML platform- and language-independent standardized by the Web Consortium W3C

15 dom: features Defines a logical tree-like structure of XML documents document = hierarchy of a set of objects based on which we can access/modify XML data

16 dom: specification levels DOM 1 (1998) DOM Core for XML DOM HTML for standardized processing of Web pages usually, on client-side (browser)

17 dom: specification levels DOM 2 (2001) multiple recommendations concerning various functionalities: namespaces, styling, event handling, etc.

18 dom: specification levels DOM 3 (2004) specific functionalities provided by modules (some of them already standardized) XPath, tree traversal, validation, (asynchronous) load & save,

19 dom: specification levels DOM 4 (2015) interface restructuring + new functionalities

20 dom: specification levels DOM Living Standard dom.spec.whatwg.org HTML5 specific in continuous development (latest update: 16 April 2018)

21 dom: interfaces An abstract manner to access and modify the XML document s internal representation

22 dom: interfaces An abstract manner to access and modify the XML document s internal representation data is encapsulated by objects, hidden and/or protected from direct external processing

23 dom: interfaces Do not require a concrete, specific implementation DOM provides implementation-independent interfaces for data access/processing

24 dom: interfaces These interfaces are specified by IDL (Interface Description Language) according to CORBA (Common Object Request Broker Architecture) current specification (March 2016):

25 dom: interfaces IDL Defining object types by specifying their interfaces (properties + public methods) purely declarative

26 dom: interfaces IDL Providing support for multiple inheritance via interfaces specifying modules, interfaces, methods, types, attributes, exceptions, constants (W3C Recommendation, 2016) heycam.github.io/webidl/ (draft, 10 April 2018)

27 Short integer UShort Long ULong scalar real Float Double IDL data types structured other Struct Sequence Boolean Char String Enum reference Union Any

28 dom: interfaces example Specifying the NodeList interface interface NodeList { Node item (in unsigned long index); readonly attribute unsigned long length; };

29 dom: interfaces example Specifying the NodeList interface interface NodeList { Node item (in unsigned long index); readonly attribute unsigned long length; }; a method having a parameter; result: a value of Node type a read-only property of unsigned long integer

30 dom: interfaces example Specifying the Attr interface interface Attr : Node { readonly attribute DOMString name; readonly attribute boolean specified; attribute DOMString value; };

31 dom: interfaces example Specifying the Attr interface interface Attr : Node { readonly attribute DOMString name; readonly attribute boolean specified; attribute DOMString value; }; Attr extends Node 3 properties

32 dom: core A document hierarchy of node-objects that can implement (specialized) interfaces

33 dom: core A document hierarchy of node-objects that can implement (specialized) interfaces nodes have descendants or are leaf nodes tree traversal is performed in preorder, depth-first (re)visit XML Infoset

34 dom: core The access to data node lists, attributes, values, is performed by using specific methods of each node type from the tree

35 dom: core node types Document Document Fragment Element Attr Text Element, ProcessingInstruction, Comment, DocumentType Element, ProcessingInstruction, Comment, Text, CDATASection, Element, Text, Comment, CDATASection, EntityReference, Text, EntityReference (leaf node of the DOM tree)

36 dom: core Fundamental interfaces: DOMException managing the processing exceptions

37 dom: core Fundamental interfaces: DOMImplementation providing details about current implementation

38 dom: core Fundamental interfaces: DocumentFragment : Node access to a tree fragment (minimal object without a root node) DOMNode DOMDocumentFragment

39 dom: core Fundamental interfaces: Document providing access to a document for reading and/or changing

40 dom: core Fundamental interfaces: Document properties doctype, implementation, documentelement

41 dom: core Fundamental interfaces: Document properties doctype, implementation, documentelement access to root-element

42 dom: core Fundamental interfaces: Document methods createelement(), createtextnode(), createattribute(), getelementsbytagname(), getelementsbytagnamens(), getelementbyid(), createelementns(), importnode(), etc.

43 dom: core Fundamental interfaces: Node access to tree nodes node types e.g., Document, Element, CharacterData (Text, Comment, CDATASection), DocumentFragment, DocumentType, EntityReference, processed in a similar way

44 dom: core Fundamental interfaces: Node properties: nodename nodevalue nodetype parentnode parentelement firstchild lastchild previoussibling nextsibling

45 Node types (nodetype dom: property) core Value ELEMENT_NODE 1 ATTRIBUTE_NODE 2 TEXT_NODE 3 CDATA_SECTION_NODE 4 ENTITY_REFERENCE_NODE 5 ENTITY_NODE 6 PROCESSING_INSTRUCTION_NODE 7 COMMENT_NODE 8 DOCUMENT_NODE 9 DOCUMENT_TYPE_NODE 10 DOCUMENT_FRAGMENT_NODE 11 NOTATION_NODE 12

46 dom: core Fundamental interfaces: Node methods insertbefore(), appendchild(), replacechild(), removechild(), clonenode(), haschildnodes(), isequalnode(), issamenode()

47 dom: core Fundamental interfaces: Element facilitating the access to XML elements DOMNode DOMElement

48 dom: core Fundamental interfaces: Element properties: tagname id attributes

49 dom: core Fundamental interfaces: Element methods getattribute(), getattributens(), getattributenames(), getattributenode(), setattribute(), setattributens(), removeattribute(), removeattributens(), hasattribute(), hasattributens(),

50 dom: core Fundamental interfaces: Attr : Node access to the attributes of an element DOMNode DOMAttr

51 dom: core Fundamental interfaces: NodeList access to collections of nodes through indexes NamedNodeMap access based on keys (in case of HTML, for attribute lists) property: length methods: item() getnameditem() getnameditemns() setnameditem() removenameditem() etc.

52 dom: html DOM HTML extends DOM Core specializing some interfaces + providing object-oriented support for HTML document processing standardizing the Web page processing (e.g., inside a Web browser)

53 dom: core in the HTML case, the element names are available in capitals an HTML document and its DOM tree rendered via Live DOM Viewer software.hixie.ch/utilities/js/live-dom-viewer/

54 dom: core Why does this node appear? an HTML document and its DOM tree rendered via Live DOM Viewer software.hixie.ch/utilities/js/live-dom-viewer/

55 The DOM tree associated with the HTML document can be accessed/altered via the document object Instance of the class that implements the HTMLDocument interface

56 interface HTMLDocument : Document { attribute DOMString title; // document title readonly attribute DOMString referrer; // URL of the referring resource readonly attribute DOMString domain; // document domain readonly attribute DOMString URL; // absolute URL of the document attribute HTMLElement body; // access to <body> element readonly attribute HTMLCollection images; // list of all images readonly attribute HTMLCollection links; // list of all hyperlinks readonly attribute HTMLCollection forms; // list of all forms attribute DOMString cookie; // access to cookies // emits an exception if a value is assigned void open (); // opens a writing stream (alters the current DOM) void close (); // closes writing flow & forces content rendering void write (in DOMString text); // writes a string (e.g., HTML markups) void writeln (in DOMString text); // idem, but insert also a new line NodeList getelementsbyname (in DOMString numeelement); // provides a list of elements according to a tag name };

57 dom: html HTMLCollection interface represents a HTML node list a node could be accessed by using a numerical index or an identifier specified by the id attribute interface HTMLCollection { readonly attribute unsigned long length; // provides the number of list items Node item (in unsigned long index); // access to a node via an index Node nameditem (in DOMString name); // access to a node by its name };

58 HTMLElement interface extends the general interface provided by DOM Level 2 each specific HTML element derives from it Node Element HTML Element HTML DivElement An interface specific to each HTML element

59 <!DOCTYPE html> <html> <body> <p>web</p> <div> <img src="web.png"/> </div> </body> </html> HTML ParagraphElement HTML HtmlElement HTML BodyElement HTML DivElement Text HTML ImageElement

60 <!DOCTYPE html> <html> <body> <p>web</p> <div> <img src="web.png"/> </div> </body> </html> HTML ParagraphElement HTML HtmlElement HTML BodyElement HTML DivElement Text HTML ImageElement

61 // a generic HTML element interface HTMLElement : Element { attribute DOMString id; // identifier associated to this element attribute DOMString title; // explanatory title attribute DOMString lang; // content language attribute DOMString classname; // CSS class name used for rendering }; // specifying a Web form interface HTMLFormElement : HTMLElement { readonly attribute HTMLCollection elements; // HTML elements included by this form readonly attribute long length; // number of form fields attribute DOMString action; // URI of the resource that process the data attribute DOMString enctype; // MIME type used for data encoding // (application/x-www-form-urlencoded) attribute DOMString method; // used HTTP method (GET or POST) void submit(); // sends data to URI specified by action }; // DOM interface for <img/> element (e.g., raster graphical content: GIF, JPEG, PNG) interface HTMLImageElement : HTMLElement { attribute DOMString alt; // alternative text describing the content attribute DOMString src; // URL of the image resource };

62 dom: html Specific aspects: innerhtml mutable property providing the HTML markup of an Element node not recommended

63 dom: html Specific aspects: textcontent property that gets/sets the textual content of a node and its possible descendants

64 dom: level 2 Extends the DOM1 functionalities creating a Document object copying a node form a document to another and many others DOM 2 Core specification

65 dom: level 2 Extends the DOM1 functionalities other features: controlling how the CSS stylesheets are applied handling various events specifying filters and iterators (complex DOM tree traversals)

66 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 2 Support for stylesheet processing StyleSheet StyleSheetList MediaList DocumentStyle details at

67 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 2 Support for stylesheet processing for HTML5, is based on a specific object model: CSSOM (CSS Object Model) draft specification 12 April 2018 drafts.csswg.org/cssom/

68 (instead of) break Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ geek-and-poke.com

69 dom: level 2 Event handling defining callbacks to be executed when an event occurs event = action triggered within the run-time environment that can conduct to a reaction from a program

70 dom: nivelul 2 Event handling hierarchical description of the event flow capture versus bubble

71 dom: nivelul 2 Event handling hierarchical description of the event flow event handling can be performed from the root to target-object capture phase

72 dom: nivelul 2 Event handling hierarchical description of the event flow event handling can be made when the event is propagated from the emitting object to the superior entities bubbling phase

73 event flow (T. Leithead et al., 2012) also, study W. Page, An Introduction to DOM Events (2013)

74 dom: level 2 Event handling a standardized set of events will be used

75 dom: level 2 Event types: UI events context: user interaction mouse: click, mousedown, mouseup, mouseover, mousemove keyboard: keypress, keydown, keyup usually, used in the HTML context

76 dom: level 2 Event types: regarding the interaction with the browser HTML specific (concerning a Web document or form) load, unload, abort, error, select, submit, focus, blur, resize, scroll

77 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 2 Event types: regarding the modification of document structure (mutation events) subtreemodified, nodeinserted, noderemoved, attrmodified,

78 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 2 Event types: regarding the modification of document structure (mutation events) currently, these are considered obsolete alternative: using MutationObserver (DOM 4)

79 dom: level 2 Event processing available interfaces: EventTarget EventListener Event minimum: UIEvent and MouseEvent

80 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 2 DOM tree traversal optional interfaces are specified: TreeWalker NodeIterator Filter

81 dom: level 3 Extends DOM 2, providing interfaces for XML manipulation via DOM modules a module offers a particular feature

82 dom: level 3 Core module includes the fundamental interfaces to be implemented by all DOM implementations conforming to the standard

83 dom: nivelul 3 depends on available modules: XML, HTML, XPath, Traversal, Range, Validation, Events, Views, Load & Save, Stylesheet,

84 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 3 Important new interfaces: DOMStringList, NameList, TypeInfo, UserDataInfo, DOMImplementationList, DOMImplementationSource, DOMLocator, DOMConfiguration

85 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 3 Modified (extended) interfaces: Document, Node, Attr, Element, Text, Entity

86 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 3 DOM 3 standardized modules DOM Load & Save available interfaces: LSParser, LSInput, LSSerializer, LSOutput

87 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 3 DOM 3 standardized modules DOM Validation provides functionalities for (automatic) creation/editing of documents conforming to a validation scheme

88 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 4 Unifies DOM3 Core, Element Traversal, Selectors API level 2, DOM3 Events new interfaces: ParentNode, ChildNode, Elements, support for specifying the programmer-defined events via CustomEvent interface

89 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 4 Element interface includes new methods: getelementsbyclassname () and matches ()

90 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: level 4 Selectors API access to various data via CSS selectors by using query() queryall() queryselector() queryselectorall() // all <li> elements selected via CSS (items of NodeList type) var elements = document.queryselectorall ("ul.menu > li"); for (var i = 0; i < elements.length; i++) { process (elements.item (i)); // processing each node }

91 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ CSS3 selectors example using the Web browser console: document.queryselectorall ("section[id^=\"week\"]:nth-child(odd) > h2");

92 JSXML JavaScript library: jsxml.net Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: implementations DOMDocument PHP class php.net/manual/en/book.dom.php HXT (Haskell XML Toolbox): DOM processing in Haskell wiki.haskell.org/hxt JAXP (Java Architecture for XML Processing) part of J2SE (javax.xml.*) docs.oracle.com/javase/tutorial/jaxp/ jsdom Node.js module: github.com/jsdom/jsdom

93 dom: implementations QDOM from Qt (C++): doc.qt.io/qt-5/qdomdocument.html libxml XML processor written in C: xmlsoft.org foundation for C++, Perl, PHP, Python, Ruby, libraries MSDOM XML processing on client/server side in C++ part of MSXML SDK msdn.microsoft.com/en-us/library/ms763742(v=vs.85).aspx org.w3c.dom.document DOM interface (Android) developer.android.com/reference/org/w3c/dom/document.html script::dom Rust module: doc.servo.org/script/dom/

94 dom: implementations Xerces DOM API XML processing platform (C++ and Java): xerces.apache.org XmlDocument.NET Framework class (C# et al.) docs.microsoft.com/en-us/dotnet/api/system.xml.xmldocument XMLDocument Objective-C / Swift class developer.apple.com/documentation/foundation/xmldocument XML::DOM Perl: search.cpan.org/perldoc?xml::dom based on Expat: libexpat.github.io xml.dom Python module, part of PyXML docs.python.org/3/library/xml.dom.html

95 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: specific apis (examples) node-xmpp Node.js (JavaScript) library for XMPP xmppjs.org OpenDocument Format (ODF) SDKs ODF document processing in C#, Java, PHP, Python, opendocumentformat.org/developers/ Open Street Map API access to XML data provided by the OpenStreetMap cartographic free service wiki.openstreetmap.org/wiki/osm_xml wiki.openstreetmap.org/wiki/api

96 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ dom: specific apis (examples) SVG DOM API SVG document processing in Java (Apache Batik): xmlgraphics.apache.org/batik/ developer.mozilla.org/web/api/document_object_model#svg_interfaces TwiML (Twilio Markup Language) API accessing Twilio telephony services via C#, Node.js, PHP, Python, Ruby, Swift libraries: Xamarin.Forms creating XAML user-interfaces for Android, ios, and Windows via C#

97 dom: implementations XML/HTML document processing DOM tree program processor XML HTML data XML API

98 Case study: getting info about the list of project proposals study the code examples associated to this lecture

99 try { $doc = new DomDocument; // a DOM object instance $doc->load ("projects.xml"); // loading the XML document // showing info about projects: title & class (if exists) $projs = $doc->getelementsbytagname("project"); foreach ($projs as $proj) { $titles = $proj->getelementsbytagname("title"); foreach ($titles as $title) { echo "Project: ". $title->nodevalue; } // verifying if the project class is specified if ($proj->hasattribute("class")) { echo " of class ". $proj->getattribute("class"); } } } catch (Exception $e) { die ("Sorry, an exception occurred."); } // processing the <title> node-elements DOM processing in PHP

100 import urllib import xml.dom from xml.dom.minidom import parse try: doc = urllib.urlopen('projects.xml') # loading the document dom = parse(doc) # parsing # processing the <project> elements + providing info about each of them for proj in dom.getelementsbytagname('project'): print 'Project ' + proj.childnodes[1].firstchild.nodevalue + \ ' has class: ' + proj.getattribute('class') except Exception: print 'Sorry, an exception occurred.' DOM processing in Python

101 # a Perl program counting the projects of 'S' class Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ # instantiating the XML processor $parser = new XML::DOM::Parser; # processing the document $doc = $parser->parsefile ('projects.xml'); $projs = $doc->getelementsbytagname ('project'); $proj_number = $projs->getlength; $projs_s = 0; # initially, 0 projects of 'S' class for (my $i = 0; $i < $proj_number; $i++) { # processing all projects $proj = $projs->item ($i); # getting the project class $class = $proj->getattributenode ('class')->getvalue; if ($class eq 'S') { $projs_s++; } } print "\nthere are $projs_s projects of S class.\n"; $doc->dispose (); # freeing the memory DOM processing in Perl

102 using System.Xml; try { doc = new XmlDocument(); // instantiating an XML document doc.load("projects.xml"); // in order to be loaded // showing info about projects: title and class XmlNodeList projs = doc.getelementsbytagname("project"); foreach (XmlElement proj in projs) { // selecting the <title> nodes via an XPath expression XmlNodeList titles = proj.selectnodes("./title"); foreach (XmlElement title in titles) { Console.Write("Proiect: {0} ", title.innerxml); } if (proj.hasattribute("class") == true) { // the project class is specified? Console.WriteLine("de clasa '{0}'.", proj.getattribute("class")); } } catch ( Exception e ) { // an exception occurred } DOM processing in C# (.NET)

103 import org.w3c.dom.*; import javax.xml.parsers.*; DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); DocumentBuilder docb = dbf.newdocumentbuilder(); Document doc = docb.parse("projects.xml"); // exploring the DOM tree (recursively) explore (doc.getdocumentelement()); static private void explore (Node node) { // showing the names of element nodes + number of attributes if (node.getnodetype() == Node.ELEMENT_NODE) { System.out.println ("Element '" + node.getnodename() + "' has " + node.getattributes().getlength() + " attributes."); } Node child = node.getfirstchild(); if (child!= null) { explore (child); } child = node.getnextsibling(); if (child!= null) { explore (child); } } DOM processing in Java

104 dom: demo Dr. Sabin Buraga profs.info.uaic.ro/~busaco/

105 XML documents processing through DOM in a Web client?

106 XML document processing in the Web browser Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ user interface browser engine rendering engine data persistence net work JS interpreter XML parser display back-end generic + particular XML processing: SVG, RSS,

107 dom: browser The visualization/processing of HTML and XML documents without validation is performed via DOM by using the JavaScript (ECMAScript) programs interpreted by the Web browser details in course Dezvoltarea aplicațiilor Web la nivel de client (3 rd year) profs.info.uaic.ro/~busaco/teach/courses/cliw/web-film.html

108 dom: browser inspecting the DOM tree associated to an HTML document by using the Web browser tools

109 advanced Dr. Sabin Buraga profs.info.uaic.ro/~busaco/ using createelement() appendchild() getelementbyid() generating a Web form via JavaScript jsfiddle.net/busaco/0wvn3fha/

110 dom: browser Also, there is support for asynchronous data transfer between the client (browser) and Web server AJAX Asynchronous JavaScript And XML via XMLHttpRequest object and/or Fetch API see upcoming lectures

111 conclusion ⵄ DOM: features, specification levels, examples

112 Client application SAX processor Handler instantiation notifying start processing: parse ( ) event occurred start tag calling handler event occurred end tag calling handler etc. Processing sending SAX events <projects> <project class="s"> </project> </projects> next episode: XML processing via SAX + simplified methods