Semistructured Data and Mediation

Transcription

1 Semistructured Data and Mediation Prof. Letizia Tanca Dipartimento di Elettronica e Informazione Politecnico di Milano

2 SEMISTRUCTURED DATA FOR THESE DATA THERE IS SOME FORM OF STRUCTURE, BUT IT IS NOT AS PRESCRIPTIVE REGULAR COMPLETE AS IN TRADITIONAL DBMSs EXAMPLES WEB DATA XML DATA BUT ALSO DATA DERIVED FROM THE INTEGRATION OF HETEROGENEOUS DATASOURCES 1

3 AN EXAMPLE OF SEMISTRUCTURED DATA a page produced from a database 2

4 AN EXAMPLE OF SEMISTRUCTURED DATA 3

5 AN EXAMPLE OF SEMISTRUCTURED DATA GRAPH-BASED REPRESENTATION: THE IRREGULAR DATA STRUCTURE APPEARS MORE CLEARLY PROFESSOR PROFESSOR NAME NAME TEACHES TEACHES AGE AGE 37 COURSE YEARS NAME? LAST NAME KING ABSTRACT OBJECTS NAME DATABASES 45 CONCRETE VALUES 4

6 A SIMPLE XML DOCUMENT WITH ITS GRAPH BASED REPRESENTATION <producer> <mn-name>mercury</mn-name> <year>1999</year> <model> <mo-name>sable LT</mo-name> <front-rating>3.84</front-rating> <side-rating>2.14</side-rating> <rank>9</rank> </model> </producer> PRODUCER mn-name year Mercury 1999 Sable LT mo-name front-rating MODEL.. side-rating rank

7 INFORMATION SEARCH IN SEMISTRUCTURED DATABASES WE WOULD LIKE TO: INTEGRATE QUERY COMPARE DATA WITH DIFFERENT STRUCTURES ALSO WITH SEMISTRUCTURED DATA, JUST AS IF THEY WERE ALL STRUCTURED 6

8 DYNAMIC INTEGRATION OF SEMISTRUCTURED DATABASES AN OVERALL DATA REPRESENTATION SHOULD BE PROGRESSIVELY BUILT, AS WE DISCOVER AND EXPLORE NEW INFORMATION SOURCES 7

9 BASED ON TEXT TREES GRAPHS LABELED NODES LABELED ARCS BOTH SEMISTRUCTURED DATA MODELS THEY ARE ALL DIFFERENT AND DO NOT LEND THEMSELVES TO EASY INTEGRATION 8

10 Recall: MEDIATORS The term mediation includes: the processing needed to make the interfaces work the knowledge structures that drive the transformations needed to transform data to information any intermediate storage that is needed (Wiederhold) 9

11 TSIMMIS APPLICATION 1 APPLICATION 2 (LOREL) (LOREL) (OEM) (OEM) MEDIATOR 1 MEDIATOR 2 WRAPPER 1 WRAPPER 2 WRAPPER 3 DATASOURCE 1 (RDBMS) DATASOURCE 2 (LOTUS NOTES) DATASOURCE 3 (WWW) 10

12 Mediator-based approach IN TSIMMIS: UNIQUE, GRAPH- BASED DATA MODEL DATA MODEL MANAGED BY THE MEDIATOR WRAPPERS FOR THE MODEL- TO- MODEL TRANSLATIONS 11

13 OEM (Object Exchange Model) (TSIMMIS) Graph-based Does not represent the schema Directly represents data : self-descriptive <temp-in-farenheit,int,80> 12

14 Object structure <Object-id,label,type,value> Nested structure 13

15 OEM (Object Exchange Model) (TSIMMIS) 14

16 Typical complications when integrating Each mediator is specialized into a certain domain (e.g. weather forecast), thus Each mediator must know domain metadata, which convey the data semanwcs On- line duplicate recogniwon and removal (no designer to solve conflicts at design Wme here) 15

17 Query formulation Find books authored by Aho select library.book.title where library.book.author = Aho from library (if more than one root is available) OK, but if this query must be produced at run-time, how does the user (or the system, if a transformation has to be applied) know that: A node library exists, which contains nodes book, which in turn contain fields author and title TSIMMIS uses the Dataguide: a-posteriori schema, progressively built while exploring the data sources 16

18 TSIMMIS s language is LOREL Lightweight Object REpository Language Object-based Similar to OQL, with some modifications appropriate for semistructured data select library.book.title where library.book.author = Aho from library 17

19 WRAPPERS (translators) Convert queries into queries/commands which are understandable for the specific data source they can extend the query possibilities of a data source Convert query results from the source s format to a format which is understandable for the application 18

20 WRAPPERS BookTitle Author Editor The HTML Sourcebook J. Graham... Computer Networks A. Tannenbaum... Wrapper Database Systems Data on the Web R. Elmasri, S. Navathe S. Abiteboul, P.Buneman, D.Suciu... HTML page database table(s) (or XML docs) 19

21 Example: estraction of information from HTML docs Information extraction Source Format: plain text with HTML tags (no semantics) Target Format: relational table (possibly nested, NF 2 ) or XML (we add structure, i.e. semantics ) Wrapper Software module which performs an extraction step Intuition: use extraction rules which exploit the marking tags 20

22 A complex extraction process 20-30KB IN HTML >10 attributes with nesting 21

23 Problems Web sites change very frequently A layout change may affect the extracwon rules Human- based maintenance of an ad- hoc wrapper is very expensive Be]er: automa&c wrapper genera&on 22

24 Automa'c wrapper genera'on... We can only use them when pages are regular to some extent OK when: Many pages sharing the same structure e.g. pages are dynamically generated from a DB à data intensive web sites 23

25 Online library Name John Smith Paul Jones Authors Title Books Editions Description Id Book Id Name #e1 #b1 Editions Year Price Details Details Year Price $ First #a1 John Smith #e2 #b1 First Edition, 2000 Paperback 30$ 1998 Second 20$ Database Primer This book #a2 Paul Jones #e3 #b $ First Second Edition, Hard Cover $ #e4 #b $ First #e5 #b $ null Books Computer Systems An undergraduate First Edition, Paperback $ #e6 #b $ Second Id Author Title Description XML at Work A comprehensive #e7 #b5 First Edition, 2000 Paperback 50$ 1999 null 30$ #b1 #a1 Database Primer #b2 #a1 Computer Systems null $ HTML and Scripts An useful HTML #b3 #a2 XML at Work Second Edition, Hard Cover $ #b4 #a2 HTML and Script JavaScripts A must in null $ #b5 #a2 Java Script Source Dataset HTML Pages 24

26 The ROAD RUNNER project Page Class It is the collec'on of all pages generated by the same script from a common dataset Schema DerivaWon Given a set of HTML sample pages, belonging to the same class, find the underlying dataset structure (database schema) à SoluWon: Wrapper Generator Underlying dataset structure ExtracWon rules 25

27 B A C D F E G First Edition, 1998 Database This book John Primer Second 2000 Smith Edition, An Computer First Edition, undergraduate 1995 Input HTML Systems Sources A <HTML><BODY><TABLE> First Edition, XML at Work <HTML><BODY><TABLE> comprehensive 1999 <TR> <TR> <TD><FONT>books.com<FONT></TD> <TD><FONT>books.com<FONT></TD> null 1993 <TD><A>John Smith</A></TD> HTML and <TD><A>Paul An useful HTML Jones</A></TD> </TR> Paul Jones Second Scripts </TR> <TR> <TR> Edition, 1999 <TD><FONT>Database Primer<FONT></TD> <TD><FONT>XML at Work<FONT></TD> <TD><B>First Edition, Paperback</B></TD>, JavaScripts <TD><B>First A must in Edition, null Paperback</B></TD>, 2000 Wrapper Output H 20$ 30$ 40$ 30$ 30$ 45$ 50$ Wrapper = Grammar <HTML><BODY><TABLE> <TR> <TD><FONT>books.com<FONT></TD> <TD><A> #PCDATA</A></TD></TR> (<TR> ( <TD><FONT> #PCDATA <FONT></TD> ( <TD><B> #PCDATA </B> )? </TD> ( <TR><TD><FONT> #PCDATA <FONT> <B> #PCDATA </B> </TD></TR> )+ )+ Target Schema SET ( TUPLE (A : #PCDATA; B : SET ( TUPLE ( C : #PCDATA; D : #PCDATA; E : SET ( TUPLE ( F: #PCDATA; G: #PCDATA; H: #PCDATA))) 26

28 Model Management approach (Atzeni, Bernstein, others) Given two different data models (e.g. OO and relational, or XML and OO, etc.) (when datasources are at least semistructured) Define general mappings from one model into another, which allow to Map SQL schema to XML schema Map data source to data warehouse Map OO classes to data source tables, To this end, one possibility is to use a metamodel

29 OBSERVATION: constructs in the various models are similar Only few categories of basic constructs (see Hull & King 1986): Lexical: set of printable values (domain) Abstract (entity, class, ) Aggregation: a construction based on (subsets of) Cartesian products (relationship, table) Function (attribute, property) Hierarchies A metamodel uses these basic constructs to create the constructs of the model we need to represent 28

30 METAMODEL A METAMODEL IS AN ABSTRACT MODEL FOR THE SPECIFICATION OF CONCRETE MODELS TWO TYPES OF METAMODELS: 1. GENERAL ENTITIES WHOSE SPECIALIZATIONS BECOME OBJECTS IN THE TARGET MODEL, E.G. GSMM 2. ENTITIES DESCRIBING THE OBJECTS OF THE TARGET MODEL, E.G.GEOGRAPHIC DATA FILES (GDF) 29

31 Using a metamodel for integrated data representation TRANSLATION OF DIFFERENT MODELS INTO A UNIQUE FORMALISM EASY A-PRIORI COMPARISON BETWEEN THE DIFFERENT MODEL S FEATURES AUTOMATIC TRANSLATION DICTATED BY THE REPRESENTATION RULES OF THE CONCRETE MODEL INTO THE METAMODEL 30

32 Data IntegraWon based on a Meta- model Q (In the metamodel language) Meta- model- based schema Q1 (In DS1 language) Q2 (In DS2 language) Q3 (In DS3 language) Qn (In DSn language) DS1 DS2 DS3 DSn 31

33 GSMM (General Semistructured Meta- Model) GRAPH- BASED DOES NOT REPRESENT THE SCHEMA: SELF- DESCRIPTIVE AS TSIMMIS GSSM REPLACES THE CONCEPT OF SCHEMA WITH THAT OF CONSTRAINT INTRODUCTION OF FLEXIBILITY IN DATA REPRESENTATION 32

34 An XML document PRODUCERS.. <producers> <producer> <mn-name>mercury</mn-name> <year>1999</year> <model> <mo-name>sable LT</mo-name> <front-rating>3.84</frontrating> <side-rating>2.14</side-rating> <rank>9</rank> </model> </producer> </producers> PRODUCER.. mn-name year MODEL Mercury 1999 rank mo-name Sable LT front-rating side-rating

35 Its representation in GSMM <PRODUCERS,root,0, >.. sub-element of <PRODUCER,element,1, > sub-element of <mn-name,element,1, Mercury> sub-element of sub-element of.. <MODEL,element,3, > <year,element,2, 1999> sub-element of sub-element of sub-element of <mo-name,element,1, SableLT> <rank,element,4, 9> <front-rating,element,2, 3.84> sub-element of <side-rating,element,3, 2.14> 34

36 APPLICATION OF THE GSMM METAMODEL THE METAMODEL ALLOWS THE CONSTRUCTION OF A GENERIC GRAPH WHICH REPRESENTS AN INSTANCE OF A CONCRETE MODEL PLUS CONSTRAINTS (also represented by means of graphs): HIGH LEVEL, OR META- CONSTRAINTS these dictate the sintactic rules of the object data model LOW LEVEL CONSTRAINTS as usual, these dictate the application domain semantics IT IS A METAMODEL OF THE FIRST KIND 35

37 ModelGen (Atzeni et al.) A model management operator: given two data models M1 and M2, and a schema S1 of M1 (the source schema and model) generate a schema S2 of M2 (the target schema and model), corresponding to S1 and, for each database D1 over S1, generate an equivalent database D2 over S2 36

38 Geographic Data Files (GDF) GDF is a standard, used for describing roadmaps CITY STREET REGISTER STREET SIGN ARCHIVE TRAFFIC CONTROL SYSTEMS CAR NAVIGATOR SYSTEMS (GPS).. IT IS A METAMODEL OF THE SECOND KIND 37

39 GDF STRUCTURE DATASET OF 82 ASCII CHARACTERS RECORDS ENTITIES ATTRIBUTES RELATIONS 11 INTEREST THEMES ROADS AND FERRIES BRIDGES AND TUNNELS RAILROADS RIVERS PUBLIC TRANSPORTATION ADMINISTRATION AREAS.. 3 DESCRIPTION LEVELS FOR EACH THEME 38

40 GDF LEVEL 0 TOPOLOGY A PLANAR GRAPH: POINT ARC NODE POLYGON EACH ELEMENT IS UNIQUELY IDENTIFIED BY AN ID TOPOLOGICAL RULES GOVERN INTERNAL STRUCTURE AND RELATIONSHIPS AMONG ELEMENTS 39

41 GDF LEVELS LEVEL 1 ELEMENTARY ENTITIES THIS IS THE BASIS FOR THE CITY STREET REGISTER ROAD ELEMENT JUNCTION TRAFFIC AREA LEVEL 2 COMPLEX ENTITIES THIS IS THE BASIS FOR THE GEOGRAPHIC INFORMATION SYSTEMS STREET INTERSECTION 40

42 GDF LEVELS VIALE ABRUZZI CITY MAP LEVEL 2 CL234 CS102 CL203 CL235 CS103 L525 S867 S868 L321 L322 L524 S869 S870 LEVEL 1 LEVEL 0 C622 C621 L818 L811 L812 C623 L817 L819 C624 C625 L813 C627 L820 C628 L814 C629 L815 C630 L816 C631 C626 L821 41

43 SUMMARY Letizia Tanca DB design and integration 42

44 Bibliography on Data IntegraWon 43