exist: An Open Source Native XML database

Transcription

1 exist: An Open Source Native XML database Advisor: Yin-Fu Huang Student: Shing-Hang Wang MM & DB Lab. 1 Introduction! exist is an Open Source effort to develop a native XML database system.! It featuring efficient, index-based query processing, automatic indexing, extensions for full-text search, XUpdate support.! Provides schema-less storage of XML documents in hierarchical collections MM & DB Lab. 2

2 Architecture Overview MM & DB Lab. 3 Database Features! Data Storage " Native XML data store based on B+-trees and paged files. Document nodes are stored in a DOM tree.! Collections " Documents are managed in hierarchical collections, similar to storing files in a file system! Updates " Document-level and node-level updates.! Authorization Mechanism " Unix-like access permissions for users/groups at collection- and document-level MM & DB Lab. 4

3 Database Features! Multi-User Access " Concurrent read/write access supported. Database manages concurrency at the level of the basic database operations.! Deployment " exist may be deployed as a stand-alone database server, as an embedded Java library or as part of a web application (running in the servlet engine).! Backup/Restore " Backup/restore functionality is provided via Java admin client or Ant scripts. Allows full restore of a database including user/group permissions.! XML Standards " XPath, Xquery, XUpdate, XSL/XSLT.! Network Protocols " HTTP/REST, XML-RPC, SOAP, WebDAV MM & DB Lab. 5 Numbering Schemes! The major benefit of Numbering Schemes is that relationships can be determined in efficient.! K-ary Tree MM & DB Lab. 6

4 Numbering Schemes! Instead the number of children a node may have is recomputed for every level of the tree. " Two nodes x and y of a tree, size(x) = size(y) if level(x) = level(y) MM & DB Lab. 7 Index and Data Organization! exist uses four index files at the core of the native XML storage backend: " collections.dbx - manages the collection hierarchy " dom.dbx - collects nodes in a paged file and associates unique node identifiers to the actual nodes " elements.dbx - indexes elements and attributes " words.dbx - keeps track of word occurrences and is used by the fulltext search extensions! All based on B + -trees MM & DB Lab. 8

5 collections.dbx! Manages the collection hierarchy and maps collection names to collection objects.! An important point to note is that the indexes for elements, attributes and keywords are organized by collection and no by document collection XML collection collection XML MM & DB Lab. 9 dom.dbx! The XML data store(dom.dbx) represents the central component of exist's native storage architecture.! All document nodes are stored according to the W3C's DOM (Document Object Model).! The data store is backed by a multi-root B+-Tree in the same file.! Associate the unique node identifiers of top-level elements in a given document to the node s storage address in the data pages MM & DB Lab. 10

6 dom.dbx! However, the query engine will process most types of XPath without accessing dom.dbx MM & DB Lab. 11 elements.dbx! Element and Attribute names are mapped to unique node identifiers in file elements.dbx.! Each entry in the B + -tree index consists of a key - <collection-id, name-id>! And each key correspond an array containing an list of <document-id, node-id> Attribute or Element MM & DB Lab. 12

7 word.dbx! words.dbx corresponds to an inverted index as found. with the set of documents in which it has been found and the exact position where it occurred! Using<collection-id, keyword> pair for key, like element.dbx! Each entry in the value list points to a text or attribute node where the keyword occurred. <collection-id, keyword> Keyword occurred Keyword occurred Like node-id MM & DB Lab. 13 Query Language Implementation! First decompose a given path expression into a chain of basic steps! Example Query " /PLAY//SPEECH[SPEAKER= HAMLET ]! <1> Load the root elements "PLAY" for all documents in the input document set.! <2> The set of "SPEECH" element is retrieved for the input documents via an index lookup from file elements.dbx.! <3> Ancestor-descendant join algorithm is applied to the two sets (use <document-id, node-id>). And repeat.! <4> While value need to be compared!like "HAMLET""#it must retrieved value from dom.dbx MM & DB Lab. 14

8 Fulltext Search! The fulltext index is required by exist's fulltext extensions.! " =" operators. " node-set = 'string of keywords'! selects context nodes containing any of the keywords in the right-hand argument.! The fulltext index is stored in file words.dbx MM & DB Lab. 15 Performance MO)712P,&5Q &M%-7 &M%-7N &L7&*-%#*- M%*3%4& K)L&*!"#$%&'=!!+&*5&./J5)")/<!!45&3%7'G5#(&./3%5&47#5-/< %&,, $&$, $)&(' '$&+(!"#$%&'+&*5&-!+&*5&./I&-7&5*/<!7%7(& )&-. )&'% $&%. -&*+!"#$%&'()*+,)+&-!()*+,)+&./0*+(%-1/<!7%7(& $&,* )&.- %,&$+ +&*!"#$%&'=!!45&3%7!G41)5)47&5*)"&./H&4&F7%#*%-7/< %&$' )&'$ '-&), *$&,+!"#$%&'4#*7)%*-?=!!4#""&*792/F5&3%47)D(&/B< '&-. )&' '*&-* %$&,.!"#$%&'=!!45&3%7./C)D(&92E()5;/<,&,- )&%* )&%+ %%&-'!"#$%&'=!!()*+,)+&-!()*+,)+&./0*+(%-1/<!7%7(&'-7)57-6>%71?=9/@A*32875&&7/B< $&(% )&%' $-&,- %'&(,!"#$%&'()*+,)+&-!()*+,)+&./0*+(%-1/2)*3245&36%7-!45&3%7./8%*)75)92:5)*;/< *&$( )&*+ )&$$ $%&'( MM & DB Lab. 16