Outline A Survey of Approaches to Automatic Schema Matching. Outline. What is Schema Matching? An Example. Another Example

Transcription

1 A Survey of Approaches to Automatic Schema Matching Mihai Virtosu CS7965 Advanced Database Systems Spring 2006 April 10th, What is Schema Matching? A basic problem found in many database application domains It has to do with combining data from different data sources It is of particular interest in data integration, E-business, data warehousing, and semantic query processing 3 4 An Example Another Example 5 6 1

2 Challenges Actual semantics not available we must use clues in the schema and data Clues are often misleading and ambiguous (e.g. Agent-contact is Agent Name or Phone Number?) Matching process can be very costly and very hard to do manually Matching often depends on the application 7 8 Application Domains Schema Integration Schema integration Data warehouses E-commerce Semantic query processing Many others (XML message mapping, etc ) 9 10 Data Warehouses E-commerce

3 Semantic Query Processing Query: All employees that earn more than $40,000 Result Applying Match Mapping DATABASE: Employees(FName, LName, Salary) New query in SQL format: SELECT FName, Lname FROM Employees WHERE Salary>40, The Match Operation A function that takes two schemas S1 and S2 as input and returns a mapping between those two schemas as output The criteria used to match: not very mathematical, but rather based on heuristics Assumptions Schemas to be matched are in a uniform internal representation There is a semantics-preserving importer that translates schemas from their native representation into the internal representation Similarly, there is also an exporter The process of matching schemas is not fully automated (at the time this paper was written) Schema Matching Approaches Schema Matching Approaches Schema-based Linguistic Constraintbased Names Types Descriptions Keys Individual matchers Constraintbased Graph matching Instance-based Hybrid Element-level Structure-level Element-level Taxonomy from [E. Rahm, P. Bernstein, 2001] Combined matchers Linguistic Constraint -based IR (word Value pattern frequencies, and ranges key terms) Composite manual composition automatic composition 17 Instance vs. schema: data contents vs. only schema-level information Element vs. structure matching: individual schema elements (e.g. attributes) vs. combinations of attributes Language vs. constraint: names and textual descriptions vs. keys and relationships Matching cardinality: 1:1, 1:n, n:1, n:m Auxiliary information: dictionaries, global schemas, previous matching decisions, and user input 18 3

4 Schema-level Matchers Granularity of Match Consider schema information instead of instance data: Name, Description, Data Type, Relationship Types, Constraints, Structure Often produces multiple candidates and estimates a degree of similarity for each Element-level matching: operates at the atomic level Structure-level matching: combination of elements that appear together in a structure There is also a coarser grained elementlevel matching, but it ignores the substructure and its components Match Cardinality Linguistic Approaches Local match cardinalities 1:1, element level n:1, elementlevel 1:n, elementlevel n:1, structurelevel (n:m elementlevel) S1 element(s) Price Price, Tax Name B.Title, B.PuNo, P.PuNo, P.Name S2 element(s) Amount Cost FirstName, LastName A.Book, A.Publisher Matching expression Amount = Price Cost = Price * (1 + Tax/100) FirstName, LastName = Extract(Name, ) A.Book, A.Publisher = select B.Title, P.Name from B, P where B.PuNo = P.PuNo Name matching: Equality of names Equality of canonical name representations (e.g. CName -> Customer Name) Equality of synonyms (e.g make brand) Equality of hypernyms (e.g. book is-a publication implies book publication) Similarity of names based on common substrings, edit distance, pronunciation, and soundex (e.g. ShipTo Ship2) User provided name matches (e.g. issue bug) Linguistic Approaches Description matching Comments in natural language Can be as simple as keyword extraction and synonym matching, or as complex as using natural language understanding technology Example: S1: empn //employee name S2: name //name of employee Constraint-based Approaches Schemas often contain constraints to define data types and value ranges, optionality, relationship types, cardinalities, etc. Top-down and bottom-up approaches for hierarchical schemas

5 Reusing Schema and Match Information Instance-level Approaches Look for common schema components E.g., In e-commerce structures often repeat within different message formats (address, customer, employee, purchase, order, invoice, etc) Libraries of partial schema mappings Domain specific (e.g. salary is different for payroll and tax applications) Need to match new schema to partial mappings Always the match problem Useful where schema isn t provided (e.g. semistructured data) but can be partially constructed Can be used as a check against schema-level approach Many approaches like schema-level Linguistic characterization for text elements E.g., keywords based on relative frequency Constraint-based characterization E.g., numeric value ranges, averages, character patterns (phone numbers, addresses, ISBN, dates, etc.) Works well with libraries of past matches Mainly works for element-level matches, not structural Requires similar data in both representations Combining Matchers An example: COMA++ Hybrid matcher Combine several matching approaches into one system to determine ranked candidates Performance usually better than composite matcher as fewer passes through schema and data are required Composite matcher Combines results of several independent matchers Provides more flexibility than a hybrid matcher Trials of machine learning to combine matchers COMA++ Supports higher level strategies to address complex match problems: Fragment-based matching Reuse-oriented matching The MatchCompose operation: performs a joinlike operation on a mapping path consisting of two or more mappings, such as A-B, B-C, and C-D, successively sharing a common schema, to derive a new mapping between A and D

6 SemInt (Northwestern Univ.) Mapping between individual attributes (i.e. match cardinality = 1:n). It uses two approaches: Euclidian distance between signatures Suitable for almost identical attributes. Neural networks This one is doing better at identifying less similar attributes that match. TransScm (Tel Aviv University) Uses schema matching to derive an automatic data translation between schema instances. Schemas are transformed into labeled graphs. Matching is performed node by node (element-level, 1:1) starting at the top. Requires user intervention if no match is found (i.e. to provide a new rule) SKAT (Stanford University) Semantic Knowledge Articulation Tool Follows a rule-based approach to semiautomatically determine matches between two ontologies. User input required: The user must provide application specific match/mismatch relations. The user must approve or reject matches. SKAT matching is used within the ONION architecture for ontology integration. In ONION, an articulation ontology is constructed from the rules. Matching is based on is-a relationships between the articulation ontology and the source ontology. DIKE (Univ. of RC, Univ. of C) Compares pairs of objects by their attributes and the is-a relationships that they are involved in. These pairs are given a match score between 0 and 1. User must specify synonyms, homonyms, and inclusion properties ARTEMIS and MOMIS It operates on a hybrid-oo model. Matches are computed by a weighted sum of name (thesauri) and data type affinity (table of data type compatibility) and structural affinity (similarity of relationships). ARTEMIS is a component of a bigger database mediator called MOMIS. Cupid (Microsoft Research) Hybrid matcher Element and Structural-Level matches. Phase 1: Linguistic Element-Level. categorizes elements based on name, data types, and domains. calculates a linguistic similarity coefficient. Phase 2: transforms the original schema into a tree then perform a bottom-up structure matching. calculates a similarity value. calculates a weighted mean of linguistic and structural similarity of pairs of elements Phase 3: uses the mean from phase 2 to decide on a mapping

7 LSD (Univ. of Washington) LSD (Univ. of Washington) LSD (Univ. of Washington) Learning Source Descriptions Uses machine learning techniques to match a new data source against a previously determined global schema. Uses a name matcher and several instancelevel matchers. System is trained with sample user inputs and it learns patterns and matching rules. Mostly instance-oriented but can use schema information too. Also supports user input domain constraints on the global schema Conclusion and Discussion The paper presents a comprehensive survey of schema matching techniques All the methods presented have a limited degree of automation, e.g. there are no fully automated matchers as suggested by the title References Rahm E., Bernstein P.A. A survey of approaches to automatic schema matching Madhavan J., Bernstein P.A., Rahm E. Generic Schema Matching with Cupid Doan A., Halevy A.Y. Semantic Integration Research in the Database Community: A Brief Survey Kaschek R., Mayr H.C. Schema Integration Oracle 9i Data Warehousing Guide Gregory A.T. - XML Schema Design for business-tobusiness e-commerce Aumueller D., Do H., Massmann S., Rahm E. - Schema and Ontology Matching with COMA

8 The End Thank you. 43 8