Semistructured Data Management Part 2 - Graph Databases

Transcription

1 Semistructured Dt Mngement Prt 2 - Grph Dtbses 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 1 1

2 Tody's Questions 1. Schems for Semi-structured Dt 2. Grph Dtbses nd Schem Grphs 3. Schem Extrction nd Indexing Semistructured Dt 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 2 2

3 Wht Do You Think? Why re schems importnt? How re schems defined? 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 3 3

4 1. Providing Dt with Context Exmple: Serching biologicl dtbses Without context (like Google, Gnutell) Serching for dt on "nglerfish" Results will be precise This seems esy, but the sme for "leech" Orgnism leech Authors: "Bleech", "Leechmn", Protein sequences: MNTSLEECHMPKGD Serch for "257" Semi-structured dt llows to mrkup SwissProt: <Species> leech </Species> EMBLChnge: <Orgnism> leech </Orgnism> Schems (structured dt) provide greed-upon dt structures 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 4 The exmple illustrtes tht dt without context is difficult or impossible to interpret. Therefore context needs to be provided either in the form of mrkup, tht is directly encoded intot the dt, then we spek of semi-structured dt, or in the form of schems which define structured dt type, where ech structurl element hs its proper interprettion. Schems cn be provided both for semistructured dt (see the cse XML documents nd their DTDs) nd for structured dt, where no schem informtion is encoded into the dt itself (see the cse reltionl dt nd their reltionl dtbse schems). In the following we will consider the cse of schems for semi-structured dt. 4

5 "To Schem or not to Schem?" Benefits of schems (e.g. DTD) Agreement on dt structures (mrkup), thus greed interprettion Incresed dt consistency e.g. integrity constrints Optimizing query evlution nd dt storge e.g. reltionl dt storge for semi-structured dt e.g. construction of indices Understnding of dt structure e.g. browsing of dtbses, site mp Benefits of schem-less dt (e.g. well-formed XML) incresed flexibility e.g. dding or dropping dynmiclly structurl elements such s ttributes self-contined dt e.g. complete context directly encoded into dt (mrkup) 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 5 Looking t Dt on the Web we cn mke the following observtions: The dt is frequently schem-less. HTML nd XML documents come long structured in n d hoc mnner. Structurl properties (type informtion resp. clssifiction) re implicit, hidden in the textul content of lbels of structurl elements, like links, titles, column nmes in tbles etc. Hving no schem hs the dvntge tht users cn exchnge nd process dt without requiring common context (the schem). Also it becomes esier to del with the frequent chnges of schems on the Web. Of course the processing becomes less efficient nd importnt met informtion is possibly lost. Both reson nd consequence of dt being schem-less is tht the dt is irregulr, i.e. documents describing the sme or similr content cn be structured differently for every dt instnce. 5

6 Exmple DB schem-less dt Answering /DB// requires to visit every lef node Storing s n edge tble requires to perform for every nvigtion step rndom tble ccess If we knew tht the dt follows the schem shown the dt could be efficiently stored in one tble nd efficiently queried DB schem 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 6 Schems on the other side re crucil for efficient dt processing: For storge lyout the dtbse schem is used to prtition the dt: We hve ctully seen for the problem of reltionl XML storge, tht knowing schem is beneficil for determining more efficient storge schemes (both in terms of storge cost nd dt processing cost). For indexing nd efficient query ccess the dtbse schem is used to identify frequently used ccess pths: Knowing the schem llows to reson bout queries without evluting them. In conventionl dtbses this corresponds to the process of semnticlly checking query. In this process it is checked whether query is correct with respect to the dtbse schem, e.g. whether the ttributes nd reltions used exist t ll etc. As result, it cn be determined if n nswer to the query is possible t ll. If queries re semnticlly correct, by using the schem we my be ble to decide which prts of dtbse need to be ccessed t ll in order to find n nswer. The schem my lso help modify the query by refining it to query tht is more precise nd thus requires less dt ccess. Schems re lso extremely helpful in interfcing to users of the dtbse, s the schem helps to summrize wht kind of dt nd dt structures cn be expected in dtbse. This informtion in turn cn be exploited to build more efficient humn-computer interfces, such s visul query interfces or browsing interfces. So the problem we hve to fce is how cn we pply techniques tht tke dvntge of the vilbility of schems without hving priori dtbse schems defined? 6

7 Schem Extrction Turning dt with schem into schem-less dt: trivil requires embedding of schem informtion into dt, see XML Turning schem-less dt into dt with schem: schem extrction detect the structurl regulrities in given dtbse exploit the embedded schem informtion no well-defined solution: schems might only prtilly represent the structure -> no unique schem non-trivil lgorithmic tsk Study the problem for generic dt model: grph model more generl thn specific models, such s XML, RDF, reltionl void the "bells nd whistles" built into the models focus on the common lgorithmic issues Bsic primitive on the Web: nvigtion -> directed grphs 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 7 The nswer to the question of how to mke schems vilble, is tht the schems re to be extrcted from the existing dtbse. For semi-structured dt models, such s XML, schemtic informtion is directly embedded into the dt. Schemtic informtion is (met-) dt tht is ssocited with dt in order to mke n interprettion possible, such s it is for exmple done in XML by enclosing content into element tgs. Exploiting this embedded schemtic informtion mkes is possible to deduce from dtbses regulrities in their occurrence nd derive from tht -posteriori schem informtion. This process is clled schem extrction. Schem extrction is non-trivil tsk: first it requires creful definition of wht is schem. This definition should llow some flexibility, since schem should only describe only those prts of semi-structured dtbse tht re ctully regulrly structured (otherwise the schem would just be copy of the dtbse itself). Extrcting schem is lso nontrivil lgorithmic tsk. Given the difficulty of the tsk nd for wider pplicbility of the results the problem hs been mostly investigted for generliztions of existing dt models, tht concentrte on their essentil properties. A very bstrct model tht hs been used for tht purpose is the grph dt model, which focuses essentilly on nvigtion (or referencing) the most importnt wy to link together informtion on the Web. 7

8 2. Grph Dt Model Grph Dt Model techniclly simpler to tret thn for exmple XML dtbses nd schems re treted in the sme frmework: grphs Grph Dt Model Definition dt grph is lbeled, rooted grph lef nodes contin tomic nd typed dt vlues Formlly: A dt grph D=(V, E, R) is lbeled rooted grph, where V N is finite set of nodes, E V x L x V is set of lbeled edges, R V is set of root nodes nd All nodes in V re rechble from some root in R. Formlly: A V is set of lef nodes. C A x U x T is the dt stored in D, where U is the domin of ll dt vlues nd T is the domin of dt types. The nodes A re clled tomic nodes 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 8 The lest common denomintor with respect to structurl properties of ny dt model is the fct tht we cn view ll dt s lbeled grph. This is true for XML/HTML documents where the tree structure cn be seen s grph nd the lbels re determined through the element types, s well s when looking t the Link structure of the Web, where the lbels re derived from the textul content of the links. For processing lrge quntities of Web dt we need however to be ble to pply techniques from dtbse mngement. These exploit typiclly some regulrity in the dt, which re normlly cptured by dtbse schems. The pproch we will introduce in the following is: We consider Web dt s dt grph We construct grphs tht cn be used s schems for the dt grph, by extrcting ny regulr (i.e. repeting) structures in the dt grph We use these schem grphs for efficient processing of queries. A grph dtbse consists not only of structure (the lbeled grph) but lso of dt content. The dt content is ttched to lef nodes of the dt grph, i.e. nodes with no outgoing edges. Dt content cn be of ny tomic type, like integer, string etc.. We do not further specify the types here, we ssume tht finite number of types is given. Since the nodes contin tomic dt they re clled tomic nodes. In some cses, when it will ply no role, we will omit the dt content nd focus on the structurl prt of grph dtbses only. 8

9 Exmple: Grph Dtbse D root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce" "dminstr." "PR" "undergrd" "grd" "postgrd" "web" 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 9 This is n exmple of dt grph. Compred to n XML structure there exist some subtle differences. Insted of the nodes the edges re lbeled. The grph cn hve multiple roots nd it is not tree, but generl directed grph. The lbels of the edges re used in order to determine the mening of the content of tomic nodes. 9

10 Grph Dtbses Properties Any dtbse cn be interpreted s grph dtbse, e.g. reltionl, XML, object-oriented Extends nd restricts the XML dt model Extension s rbitrry grphs re llowed (if one ignores ID/IDREF nd Xlink) Restriction: no order, no dt types etc. Useful bstrction to study specific spects of processing semi-structured dt relted to the processing of pths Pth indexing, pth queries, schems Estblishes n ppliction of utomt theory to dt mngement e.g. XPth epressions re regulr expressions; regulr expressions re equivlent to finite stte utomt 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 10 Any dtbse cn be interpreted s grph dtbse. On the other side the grph dtbse model is much weker in terms of expressible constrints nd behvior. The reltionship between XML nd the grph dtbse model cn be described s follows. -In one sense the grph dt model is n extension of XML s the core XML model only considers trees wheres grph dtbses llow generl grphs. This is true for core XML. However, pplictions on top of XML (such s XLink, the successor of html links) support mechnisms to model grphs through referencing. -In nother sense the grph dt model is more limited. Besides number of minor detils, like dt types nd constrints, the min restriction is the lck of ordering of elements (resp. nodes) which is inherent in XML documents. The grph dtbse model is however useful bstrction to investigte mny questions nd techniques tht hve to do with the processing of semi-structured dt, in prticulr spects relted to the use of pths in the dt grph (e.g. processing of pth queries s they cn be formulted in XPth). It is lso defined in wy tht it mkes stndrd utomt theory immeditely pplicble, which llows to derive number of importnt properties nd lgorithms both of prcticl nd theoreticl interest. 10

11 Structurl Properties of Grph Dtbses The min structurl property is the existence of pths tht strt from the root We cn enumerte them to cpture the type of the dt grph project... progrmmer.. sttisticin.. progrmmer.. sttisticin.. progrmmer.. sttisticin /4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 11 A simple method to look t the structure of grph dtbse is to nlyze which pths occur in it. We could for exmple enumerte them nd the set of possible pths describes the type of the dt grph. However, this is redundnt representtion. 11

12 A Possible Schem Using trie s4 progrmmer s2 s1 s5 sttisticin s3 projects s6 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 12 A representtion of the set of pths is by constructing nother grph tht represents ll these pths, but does not repet ny of these pths s it is the cse in the dt grph. Since the set of possible pths is set of strings over the lphbet consisting of ll element nmes, we might use trie structure in order to represent this set nd use the trie s schem. This is ctully lredy very close to the notion of schems for grph dtbses tht we will develop. 12

13 Exmple Schem Grph S1 The following grph cptures possible reltionships nodes tht the dt grph hs progrmmer s2 s1 s4 sttisticin projects s3 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 13 A more compct representtion would combine those nodes tht shre common properties. For exmple, ll string contents re of the sme nture nd cn be considered s common node. Nodes tht re reched by the sme pths lso cn be combined. 13

14 Simultion Why is S1 schem for D? For every node d in D reched by pth p strting from the root there exists corresponding node s in S1 rechble by the sme pth nd the types of the lef nodes re the sme in cse d is lef node Reltionship R between nodes of D nd S1: d R s S1 simultes D, denoted s D < S1 or D < R S1 Formlly: Simultion: Given Grphs G 1, G 2 nd reltion R V 1 V 2, then R is simultion if for ll lbels l L nd for ll x 1, y 1 V 1 nd for ll x 2 V holds: If x 1 fi l y 1 nd x 1 R x 2 then there exists y 2 V 2 such tht y 1 R y 2 nd x 2 fi l y 2 We write then G 1 < G 2 nd sy there exists simultion of G 1 using G 2 Formlly: Rooted Simultion: for the roots r 1 nd r 2 it holds r 1 R r 2 ; then G 1 < root G 2 Typed Simultion: for ll x,y: if x R y nd y is n tomic type then x must be n tomic node with content of tht type Wildcrds nd lternte lbels: x R _ nd x R (x y ) A schem grph S is sid to be schem for dt grph D if there exists rooted, typed simultion of D using S; S my contin wildcrds nd lternte lbels 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 14 We wnt now to formlize wht it mens tht grph is schem grph for dt grph. To tht extent we introduce the notion of simultion. A simultion is chrcterized by reltion R. The definition of simultion expresses, tht whenever we hve in the dt grph n edge nd the node from which the edge emerges is relted through the simultion to node in the schem grph, then in the schem grph there exists n edge to node tht is relted to the node to which the edge in the dt grph connects to. The simultion reltionship induces prtil order on grphs. To mke schem grphs more flexible, we extend first the definition of dt grph in two wys. We llow not only single lbels, but lso sets of lbels to occur t the edges in the schem grph. Either we llow ny possible edge ( wildcrd, using the sme nottion s for pth queries) or specific set of tbles (lternte lbels). The definition of the simultion condition hs to be extended in the strightforwrd mnner: If x 1 -> l y 1 nd x 1 R x 2 then there exists y 2 ιν V 2 such tht y 1 R y 2 nd x 2 > L y 2 nd l is contined in the set of lbels specified by L. A further extension is required to del with the typed contents of dt grph. For tht purpose we introduce specil nodes tht re used to represent tomic content of specific type. For ech type of tomic content occurring in the dt grph there exists exctly one node in the schem grph. Hving extended the notion of dt grphs to schem grphs, we hve lso to extend the definition of simultion, to tke cre of two properties of dt grphs, nmely roots nd tomic nodes. We require tht roots of the dt nd schem grph re relted by the reltion R (rooted simultion). And we require tht tomic nodes in the dt grph re relted to tomic type nodes of the sme type in the schem grph. This completes the definition for schem grph for semi-structured dtbse D! 14

15 Exmple Schem Grph S2 root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 R p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce" "dminstr." "PR" "undergrd" "grd" "postgrd" "web" progrmmer sttisticin t1 t2 _ projects 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

16 Exmple Schem Grph S2 root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 R p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce" "dminstr." "PR" "undergrd" "grd" "postgrd" "web" progrmmer sttisticin t1 t1 t2 _ projects 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

17 Exmple Schem Grph S2 root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 R p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce" "dminstr." "PR" "undergrd" "grd" "postgrd" "web" progrmmer sttisticin t1 t1 t2 _ projects 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

18 Exmple Schem Grph S2 root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 R p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce" "dminstr." "PR" "undergrd" "grd" "postgrd" "web" progrmmer sttisticin t1 t2 _ projects 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

19 Exmple Schem Grph S2 root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 R p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce" "dminstr." "PR" "undergrd" "grd" "postgrd" "web" progrmmer sttisticin R t1 t2 _ projects 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

20 Exmple Schem Grph S2 root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce" "dminstr." "PR" "undergrd" "grd" "postgrd" "web" progrmmer sttisticin R t1 t2 _ projects 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

21 Exmple Schem Grph S2 root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 R p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce" "dminstr." "PR" "undergrd" "grd" "postgrd" "web" progrmmer sttisticin R t1 t2 _ projects 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

22 Asymmetry of the Simultion Reltionship Exmple D < S2 nd S2 < D (bisimultion) D < S1 but not S1 < D For every node d in D reched by pth p strting from the root there exists corresponding node s in S1 rechble by the sme pth nd the types of the lef nodes re the sme in cse d is lef node; but not vice vers! d1 s1 t1 b d2 d3 s2 s3 t2 D S1 S2 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 22 One might wonder why the simultion reltionship is symmetric. The reson is tht the "lrger", the schem grph, my contin more structures thn the dt grph. In other words the dt grph hs to conform structurlly to the schem grph but not vice vers. In cse simultion exists on both directions, we re tlking of bisimultion. Hving bisimultion mens tht the grphs re structurlly equivlent (contin exctly the sme kinds of pths). 22

23 Clssifiction by Schem Grphs The reltionship R between nodes from the dt grph ("dt instnces") nd nodes from the schem grph ("dt clsses") implies clssifiction of the nodes of the dt grph Exmple: dt grph D nd schem grph S1 R Clss s1 s2 s3 s4 Instnces root c1 c2 e1, e2, e3, e4 p1,, p9 Attention Clssifiction is not unique, i.e. dt node my belong to multiple clsses Clssifiction cn be mbigue, i.e. dt node my belong to one or more of severl clsses 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 23 The nturl wy to clssify dt instnces, which re the nodes of the dt grph, is now to consider the node in the schem grph to which they re relted vi the simultion s the clss to which the nodes in the dt grph belong to. 23

24 Exmple: Multiple Clssifiction progrmmer s2 s1 sttisticin s3 s4 projects s5 _ 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 24 R Clss Instnces s1 root s2 c1 s3 c2 s4 e1, e2, e3, e4 s5 e1, e2, e3, e4 p1,, p9 Multiple clssifiction occurs when the sme node in the dt grph cn be reched by different pths nd these pths re ssocited with different nodes (clsses) in the schem grph. We illustrte this by slightly modifying our schem grph exmple. Note tht this chnge does not ffect the property tht the schem grph is simultion of the dt grph. 24

25 Exmple: Ambigue Clssifiction projects s2 progrmmer s1 s4 sttisticin s3 s6 R1 Clss s1 s2 s3 s4 s6 Instnces root c1 c2 e1, e2, e3, e4 p1,, p9 e2 R2 empty 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 25 Clss s1 s2 s3 s4 s6 Instnces root c1 c2 e1, e2, e3, e4 p1,, p9 For every node d in D reched by pth p strting from the root there exists corresponding node s in S1 rechble by the sme pth nd the types of the lef nodes re the sme in cse d is lef node Wheres multiple clssifiction ws required in the previous exmple in order to stisfy the simultion property, it cn even led to mbiguity in clssifiction. Using the sme schem grph, different simultions re possible leding to different clssifictions. In the exmple we could hve for the simultion reltion R, tht the instnce c2 is member of clss s3 nd s6 or only of clss s3. In both cses we would hve correct simultion reltionship since the definition of simultion only sttes tht there exists t lest one successful continution in the schem grph, nd tht the nodes need to be relted to ll possible successful continutions. This is of course n undesirble sitution: given dtbse schem we cnnot uniquely determine to which clss dt object belongs. Consider for exmple sitution where the dtbse schem is used to orgnize the storge nd dt is clssified differently t the time when it is inserted to the dtbse nd when it is serched for. It could not be found. To remedy this sitution we hve to find possibility to gurntee unique clssifiction. 25

26 Mximl Simultions Given two simultions R 1 nd R 2 between dt grph D nd schem grph S the following holds D < R1 S nd D < R2 S then D < R1 R2 S Consequently there exists mximl simultion from D to S Unmbiguous Clssifiction: dt object belongs to clss if it is ccordingly clssified by mens of the mximl simultion The mximl simultion cn be computed s fixpoint itertion Formlly: Input dt grph D; schem grph S; R := ; R := { (o, c) o in D, c in S }; while R R R := R; R := { (o, c) either o is tomic vlue nd c it's type or for ll o fi l o D there exists (o,c ) R nd c fi l c S } 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 26 In order to rrive to unique clssifiction we first mke n importnt observtion on simultions: if two (different) simultions re given by mens of two reltions, then tking the union of the reltions we obtin gin simultion. This fct ws lredy illustrted in the previous exmple where one simultion ws subset of the other. Due to this property mximl simultion must exist, which is the lrgest reltion R tht defines simultion between dt grph D nd schem grph S. This lrgest reltion is simply the union of ll possible reltions tht define simultions. Given tht, we hve identified unique simultion mong the set of ll possible simultions. In order to void mbiguity we therefore decide to clssify dt objects bsed on this unique mximl simultion nd to thus void mbiguity. The mximl simultion cn be computed s fixpoint s follows: we strt from the totl reltion between D nd S, i.e. ll dt objects nd clsses re relted. Then we stepwise eliminte those pirs in R tht violte the simultion condition. Tht is, whenever pir should be contined in R we check whether link exists in the dt grph. In prctice there exist more efficient lgorithms to perform clssifiction. This clssifiction method is for exmple required when we wnt to mtch n existing dtbse to given schem in the best possible wy. 26

27 Exmple Exmple of dt nd schem grph tht would required multiple itertion steps for computing the mximl simultion 1 c c2 c c4 c5 b c c b c6 c 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 27 Actully the clssifiction process in most prcticl cses completes in one step. This is n exmple where convergence does not occur in one step. The reson is, tht in order to clssify the nodes t the first level correctly the clssifiction t the second level hs to be performed in first itertion. Only fter we «know» to which clsses 5,6,7 belong we cn correctly relte 2,3,4 to c2 nd c3. 27

28 Exmple: Round 1 Exmple of dt nd schem grph tht would required two itertion steps for computing the mximl simultion 1 c c2 c c4 c5 b c c b c6 c 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

29 Exmple: Round 2 Exmple of dt nd schem grph tht would required two itertion steps for computing the mximl simultion 1 c c2 c c4 c5 b c c b c6 c 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

30 Exmple: Round 3 Exmple of dt nd schem grph tht would required two itertion steps for computing the mximl simultion 1 c c2 c c4 c5 b c c b c6 c 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

31 Refining Schems Refining Schems is needed when schem is dpted over time more informtion on the dtbses becoming vilble progrmmer sttisticin S2 t1 projects t2 _ refine progrmmer s2 S1 s1 s4 sttisticin projects s3 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 31 Up to now we hve only considered the sitution where schem is given nd investigted the question of how dtbse conforms to this schem by introducing the notion of simultion. In prctice the sitution will occur tht one strts from firly simple, nd very generl schem, nd refines it over time, by dding structurl informtion tht is observed from the dt. Of course, while doing tht we would prefer not to loose conformnce of the existing dt with the new schem. Therefore we require tht the refined schem must conform to ll dtbses tht conform to the originl schem. A refined schem is clled to be subsumed the schem tht it refines. Subsumption expresses tht the refined schems contins not only the constrints imposed by the originl schem but lso some dditionl ones tht hve been dded. 31

32 Schem Subsumption Wht does it men tht one schem is refinement of nother? All dtbses conforming to the refined schem S' conform lso to the more generl schem S This is exctly the cse if S' < S! In other words: if we cn simulte schem S' using S then S' is refinement of S, or we sy S subsumes S' Defines prtil order on ll schems! Queries ginst more generl schems cn lso be nswered ginst refined schems, but not vice vers Formlly: for simultion reltionships R nd R : G1< R G2 nd G2 < R G3 then G1< R o R G3 where x (R o R ) z iff exists y such tht (x R y nd y R z) Consequence: if S' < S nd D < S' then lso D < S 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 32 Formlly: we observe tht for simultions the following property holds: if we compose two reltions defining simultions the resulting reltion gin defines simultion reltion. The composition of reltions is defined in n nlogous wy to the join of binry reltions with one mtching ttribute. We use this property of composition of reltions then s follows: if S' is schem for D (i.e. D<S') nd S subsumes S' (i.e. S'<S) then we cn conclude tht D<S nd therefore S is lso schem for D. Thus we hve well defined notion of schem subsumption. Remrk: schem subsumption is not only considered s n importnt concept for grph dtbses, but s well for other dt models, in prticulr the reltionl model. There it is known tht deciding whether one schem subsumes nother is known to be n importnt (nd hrd) problem. 32

33 Exmple: S1 < S2 R (D<S1) Clss s1 s2 s3 s4 Instnces root c1 c2 e1, e2, e3, e4 p1,, p9 R' (S1<S2) Clss t1 t2 Clsses s1, s2,s3 s4 R o R' (D<S2) Clss t1 t2 Clsses root, c1, c2 e1, e2, e3, e4 p1,, p9 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 33 We illustrte schem subsumption by the two schems S1 nd S2 we hve devised for D. The tble illustrtes on the one hnd how the nodes in D re clssified, nd on the other hnd the reltion between nodes of S1 nd S2 tht specifies the simultion between the two schems. Schem S2 is obviously less refined s it induces corser clssifiction of dt. It llows to ccommodte more different dt grphs thn S1. 33

34 Summry Why re schems useful for dtbses? Which chrcteristics of dt grph is cptured by schem grph? Wht is the concept of rooted, typed simultion needed for? Does there exist more thn one schem grph for dt grph? Wht is the difference between the notions non-unique nd mbigue clssifiction? Wht cn we obtin from mximl simultion? Wht is schem subsumption? Why is the notion of schem subsumption importnt? 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

35 3. Schem Extrction Construction of Schem Grph from the Dt Grph Automticlly There exists no unique schem grph Gol: construct schem grph tht is Accurte: every pth tht occurs in the schem grph occurs in the dt grph nd vice vers Concise: every pth occurs only once This schem grph is clled Dt Guide 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 35 Up to now we hve looked t the questions (1) how to clssify dt given schem grph nd (2) how to compre two schem grphs In both cses the ssumption ws tht schem is given. If we look t the sitution we encounter on the Web we will often hve very lrge nd complex semi-structured dtbses without schems, nd it will be very difficult to devise correct schem mnully. Therefore we go now one step further nd look t the problem whether schem cn lso be utomticlly generted. Being ble to generte schems utomticlly cn hve mny benefits. Besides the usul benefits lredy mentioned erlier, this cn be prticulrly useful for dt integrtion, where n extrcted schem my be used to define mppings between the dt in different heterogeneous dtbses. Of course, we hve lso seen tht mny different schems for the sme dtbse my exist. For n utomtic schem extrction from dtbse we need thus well-defined criterion for determining which schem to generte out of the set of possible schems. To tht end we require the following: -The schem should be ccurte. Tht mens pth tht occurs in the schem must lso be present in the dt grph (the mechnism must not "invent" pths -> correctness of the schem) nd ny pth tht occurs in the dt grph must be represented in the schem (the mechnism must not "forget" dt pths, -> completeness of the schem) -The schem should be concise: the schem should contin no redundncy, i.e. every pth tht occurs in the schem should occur only once. Actully observing these criteri leves only one possible schem to be constructed, which is then clled the dt guide. 35

36 Dt Guide Construction Nodes of the dt guide correspond to subsets of dt grph nodes The set contining the dt grph root is the root of the dt guide For ech dt guide node constructed so fr nd for ech edge lbel of n edge leving some node from the node set of the dt guide node do the following form the set of nodes in the dt grph tht re reched by this edge lbel either the set exists lredy in the dt guide: crete lbeled edge to it otherwise crete new dt guide node nd connect it by the edge Continue till no more new dt guide nodes re creted 1 dg := new node in DG; R := {({root}, dg)}; Mke({root}, dg); 2 Mke(s1, d1) 3 { p := { (l, o) o -> l o in D nd o in s1}; 4 for ech l occuring in p 5 s2 := {o (l, o) in p }; 6 if exists (s2, d2) in R dd n edge d1 -> l d2 to DG 7 else d2 := new node in DG; 8 R := R union {(s2, d2)}; 9 dd n edge d1 -> l d2 to DG; 10 Mke(s2, d2); } 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 36 We now give n lgorithm for constructing the dt guide nd we will nlyze its properties subsequently. The bsic ide of the lgorithm is to construct, strting from the root of the dt grph, for every possible pth in the dt grph the set of dt objects tht re reched by exctly this pth. Let us nlyze the lgorithm in more detil. It strts by creting root node in the dt guide. The root node in the dt guide is relted to the root node of the dt grph, nd we store this reltionship in R (line 1). The reltionship R is the simultion reltionship. Then we strt the recursive procedure of constructing the dt guide by clling Mke (line 2) with given set of nodes (clss) nd the corresponding representtion of it in the dt guide. When Mke is clled we investigte ll lbeled edges tht strt from node of the clss we re currently looking t (s1) nd store them in set p (line 3). For ech different lbel tht we find in p we hve now to check how to represent the new pth (tht is the extension of the pth we hve reched so fr by n edge with lbel l) whether we hve lredy clss in R tht represents exctly the nodes tht cn be reched by this pth (these re the nodes s2 from line 5) or whether new clss needs to be creted in the dt guide. If lredy clss exists (line 6) then we hve simply to dd new edge to the dt guide. If it does not exist (line 7), then we hve to crete new node in the dt guide, dd the reltion between this new node nd the set s2 into R (line 8) nd dd the correct edge into the dt guide (line 9). Only if clss hs been newly creted we hve to invoke the recursive procedure gin for this new clss (line 10) in order to crete necessry extensions of the dt guide s result of the cretion of the new clss. The lgorithm termintes for cycles, since when rriving second time t the sme clss in the cycle the corresponding clss hs lredy been creted nd we go to the condition of line 6, where no recursive clls will be mde. Observe tht even in the presence of cycles such n lgorithm must terminte, s the number of subsets of nodes in the dt grph is finite. Thus in the worst cse for ech subset of the nodes of the dt grph node in the dt guide is constructed. This gives n ide on principl problem of dt guides, nmely complexity. However, in prctice if we hve regulr dt grphs with lot of redundnt structure this does not ply role. 36

37 Exmple project {root} progrmmer sttisticin {p1,p2,p3,p4,p5, p6,p7,p8,p9} {c1} {c2} {e1,e2,e3,e4} {e1,e2} {e2,e3} {p1,p3,p5, p7,p9} {p2,p4, p6,p8} {p4,p9} root progrmmer c1 sttisticin c2 {p1,p3} {p2,p4} {p1,p3,p5,p7} {p4,p6} {p4} project e1 e2 e3 e4 p1 p2 p3 p4 p5 p6 p7 p8 p9 "exercise" "lecture" "finnce""dminstr.""pr" "undergrd"grd" "postgrd" "web" 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 37 We give the exmple of dt guide. When crefully nlyzing the resulting dt guide one cn observe severl effects. -The nodes typiclly occur in multiple plces in the dt guide -Repeting structures in the dt grph re reduced to single occurrence in the dt guide -Nevertheless the dt guide cn be more complex thn the dt grph itself -The dt guide is deterministic. Tht mens tht ll outgoing edges in the dt guide re different. In ddition, if cycles re present in the dt grph, they led to cycles in the dt guide. 37

38 Properties of the Dt Guide - Observtions - cycles in the dt grph led to cycles in the dt guide - nodes my occur in multiple plces in the dt guide - Repeting structures in the dt grph re reduced to single occurrence in the dt guide - the dt guide cn be more complex thn the dt grph itself - The dt guide is deterministic: ll outgoing edges in the dt guide hve different lbels - It cn be shown tht: The dt guide is the miniml, deterministic schem grph - Every other deterministic schem grph is subsumed by the dt guide 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 38 The property of being the miniml, deterministic schem grph provides concise criterion for defining dt guides. 38

39 Semistructured Dt Indexing Ech node in the dt guide is hsh tble The outgoing lbel is the key The ddress of the dt guide node reched by the lbelled edge is stored in the hsh tble entry The list of nodes in the dt grph rechble vi the lbel is stored in the hsh tble entry Query processing If in pth query lbel is encountered, then the respective lbel is looked up If nother lbel follows, the next node in the dt guide is looked up If it is the lst node of the query the dt objects re returned A problem with dt guides is their potentil complexity Cn be much lrger thn the dtbse Question: is it necessry to require deterministic grph for indexing? 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 39 As dt guide summrizes precisely wht kind of pths occur in grph dtbse nd ssocites the nodes of the grph dtbse with the pths, we cn use the dt guide s n index structure. With respect to evlution of queries we bsiclly evlute the query ginst the dt guide rther thn ginst the dtbse itself, nd obtin the result nodes from the grph dtbse only t the end when the query hs been processed nd n nswer hs been found in the dt guide. To mke dt guide rel index structure we hve lso to think bout the wy it is ctully stored, such tht it cn be efficiently ccessed. To tht end simple pproch is to crete for every node of the dt guide hsh tble with the outgoing lbels s keys nd the references to the nodes reched by the lbeled edge s vlues. 39

40 Query Answering Evluting /progrmmer// {p1,p2,p3,p4,p5, p6,p7,p8,p9} {e1,e2} project {e2,e3} {c1} {root} progrmmer {p1,p3,p5, p7,p9} sttisticin {c2} {p2,p4, p6,p8} {e1,e2,e3,e4} {p4,p9} {p1,p3} {p2,p4} {p1,p3,p5,p7} {p4,p6} {p4} 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

41 Node Equivlence Observtion: If the possible pths leding to two nodes in the dt grph re the sme, the nodes re equivlent from the viewpoint of query processing p1 nd p3 (s well s p5 nd p7) re indistinguishble, whichever query is used to distinguish them project {root} progrmmer sttisticin Formlly: Lnguge Equivlence {p1,p2,p3,p4,p5, p6,p7,p8,p9} {e1,e2} {e2,e3} {c1} {p1,p3,p5, p7,p9} {c2} {p2,p4, p6,p8} {e1,e2,e3,e4} {p4,p9} L x = {p=p 1..p n p is pth from the root to x } x y if L x = L y equivlence clss of x denoted s [x] {p1,p3} {p2,p4} {p1,p3,p5,p7} {p4,p6} {p4} 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 41 A problem with dt guides is their potentil spce complexity. It results minly from the requirement of hving deterministic grph (ech node hs only outgoing edges with different lbels). It is wellknown fct tht deterministic grph (utomton) tht is equivlent to non-deterministic one is in the worst cse exponentil in the size of the non-deterministic grph (utomton). This to the ide tht it might be worthwhile to consider the possibility of using non-deterministic schem grphs in order to construct indexes. The construction of non-deterministic utomton for indexing strts with bsic observtion. If the set of pths tht led to node in the dt grph is the sme for two nodes, they re indistinguishble from the viewpoint of processing pth query. Either they re lwys both in the result or not. Thus we cn declre them s equivlent. This to the notion of lnguge equivlence which cn be formlly defined nd for which we introduce the corresponding nottions. 41

42 Index Construction Bsed on the lnguge equivlence clsses root Nodes of Index I re the node equivlence clsses of the dt grph Edge between equivlence clsses if corresponding edge exists in the dt grph for the members of the equivlence clss (unmbiguously determined) Nondeterministic grph (multiple outgoing edges with sme lbel) progrmmer c1 sttisticin c2 project e1 e2 e3 e4 {p1,p3} p2 p4 p6 {p5,p7} p8 p9 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 42 Hving introduced the notion of lnguge equivlence we re lmost done. We define the index by declring every equivlence clss s one node of the index grph nd connecting the nodes with lbeled edge if such n edge exists between the members of the equivlence clss in the dt grph. This is n unmbiguous definition, since the equivlence clsses cnnot be distinguished from the viewpoint of ccessibility through pths. Thus either ll members of two equivlence clsses re connected by lbeled edge with specific lbel or none (otherwise the clsses would split). This completes the definition of the index grph! Similrly s for dt guides we cn now crete dt orgniztion bsed on hsh tbles for ech node in the grph nd query processing proceeds in the sme wy, with the only difference tht probbly from given node multiple pths need to be explored for the sme lbel. For obvious resons the index cnnot be lrger thn the dtbse itself now! 42

43 Query Answering Evluting /progrmmer// root progrmmer c1 sttisticin c2 project e1 e2 e3 e4 {p1,p3} p2 p4 p6 {p5,p7} p8 p9 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

44 Computing Lnguge Equivlence Drwbck of using lnguge equivlence Checking for lnguge equivlence is PSPACE complete Ide: pproximte lnguge equivlence by computtionlly more fesible reltion Approximtion: reltion is n pproximtion of if for ll x, y x y x y Using pproximte reltion for constructing the index grph Queries will be correctly nswered Query evlution more expensive (more nodes to check) 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 44 Building the index (or better schem used for indexing) bsed on lnguge equivlence suffers from the fundmentl problem tht lnguge equivlence is very hrd problem to solve in generl. Thus for hving prcticlly useble pproch one considers reltions on the nodes tht re esier to determine but mintin the properties required for indexing. This is chieved by pproximting the lnguge equivlence reltion, by refining it. Refining mens tht equivlence clsses under lnguge equivlence my be split in multiple clsses. This does not influence the correctness of query nswering on the schem grph (but it cn increse the necessry mount of work since more nodes need to be inspected) In the following we introduce construction for n pproximtion reltion tht in prctice delivers lmost in ll cses the sme reltion s the lnguge equivlence reltion (in other words it will be hrd to find exmples where the reltions differ), but is more efficient. 44

45 Reverse Bisimultion Reverse Bisimultion reltion Equivlent to the simultion reltion in both directions with the edges of the grphs inverted On ny grph there exists mximl reversed bisimultion reltion which cn be efficiently computed nd for ll x, y x y x y Formlly: Reverse Bisimultion if x» y nd x is the root then y is the root, nd vice vers if x» y nd there exists n edge x fi l x, then there exists n edge y fi l y nd x» y, nd vice vers 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 45 For introducing n pproximtion reltion we come bck to the notion of simultion: we perform simultion in the bckwrds direction (why?) nd the simultion must be done in both directions in order to obtin n equivlence reltion. This produces n pproximtion reltion nd there exist efficient lgorithms to compute this reltion, which we will not be ble to discuss here. 45

46 Lnguge Equivlence vs. Reversed Bisimultion Normlly lnguge equivlence nd reverse bisimultion coincide Counterexmple: x y z, but x y z cnnot be equivlent since not reched by sme lbeled edges 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt - 46 In prctice reversed bisimultion nd lnguge equivlence coincide. We give here n exmple were this is not the cse. 46

47 Summry Wht is dt guide nd how does dt guide compre to other schem grphs? Wht re properties of dt guides with respect to the dt grph they re constructed from? How is dt guide used for indexing? Wht is the pproch to reduce the potentil size of n index derived from dt guide? How is lnguge equivlence used to construct n index? How is the problem tht checking lnguge equivlence is expensive circumvented? 2003/4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt

48 References Course mteril bsed on S. Abiteboul, P. Bunemnn, D. Suciu: Dt on the Web: From Reltions to Semistructured Dt nd XML, Morgn Kufmn, 2000 (in prticulr chpters 4.1, 7.1, 7.3.3, 7.4.1) Relevnt rticles Roy Goldmn, Jennifer Widom: DtGuides: Enbling Query Formultion nd Optimiztion in Semistructured Dtbses. VLDB 1997: Tov Milo, Dn Suciu: Index Structures for Pth Expressions. ICDT 1999: /4, Krl Aberer, EPFL-SSC, Lbortoire de systèmes d'informtions réprtis Semi-structured Dt