Information Quality Measurements in Data Integration Schemas

Transcription

1 Information Quality Measurements in Data Integration Schemas Maria da Conceição Moraes Batista, Ana Carolina Salgado Centro de Informática, Universidade Federal de Pernambuco Av. Professor Luis Freire s/n, Cidade Universitária Recife PE, Brasil VLDB 2007 Presented by: Kristian Torp

2 Motivation Q select name, phone_no from emp Name Phone_No Jens 46 Pete 75 Q R R = R union R R Q Eid Ename Phone 1 Jens Jane Schema 1 Emp_id Name Emp_id PhoneNo 11 Pete Paul Schema 2 Database Specialization Course

3 Overview Integration Quality Criteria Minimality Type consistency Schema completeness X-Entity Model Formal model (ERDs formalized) The overall algorithm Conclusion Critique Database Specialization Course

4 Minimality Informally No extra complete entities No extra complete relationships No extra attributes Reformulated: Avoid redundancy Additional attribute Database Specialization Course

5 Minimality (cont) Redundant relationship More than one path Database Specialization Course

6 Type Consistency Examples: Birthdays should all be dates and not varchar Employee number should all be integers not varchar Based on the notation of attribute equivalence Schema mappings actor m.birthdate m actor 1.birth 1 movie m..birthdate m actor 2.birth 2 movie m..birthdate m actor 3.bd 3 Generally <entity name>.<attribute name> actor m.birthdate m, actor 1.birth 1, actor 2.birth 2, are dates actor 3.bd 3 is varchar There is a type inconsitency Database Specialization Course

7 Schema Completeness The percentage of domain concept from source schemas represented in the integrated schema Database Specialization Course

8 Metrics Schema completeness = 1 (#incomplete item/# total items) Minimality = 1 (#redundant schema element/ # #total schema elements Type consitency = 1 (#inconsistent schema elements/ #total schema elements 1 = perfect 0 = totally messed up Database Specialization Course

9 X-Entity Model Schema S = (E, R) Entity type E({A 1,,A n },{R 1,,R m }) Name: E Attributes: {A 1,,A n } Relationships: {R 1,,R m } Containment relationship: R(E 1, E 2, (min, max)) From entity: E 1 To entity: E 2 Cardinality constraint (min, max) Reference relationship: R(E 1, E 2, {A 11,,A 1n }, {A 21,,A 2n }) From attributes: {A 11,,A 1n }, To attribute: {A 21,,A 2n } Database Specialization Course

10 X-Entity Model: Example Entities novel 2 ({name 2,year 2 }, {novel 2 _chapter 2, novel 2 _publisher 2 }) chapter 2 ({ch_title 2 },{}) publisher 2 ({pub_name 2 },{}) Containment Relationship novel 2 _chapter 2 (novel 2, chapter 2,(1,N)) novel 2 _publisher 2 (novel 2, publisher 2, (1,1)) Database Specialization Course

11 Examples of Redundancies Red(A ki, E k ) = 1, (two attributes equivalent in same entity) If: E k.a kj, j i, A kj in {A k1,,a ka } such as E k.a ki E k.a kj Red(A ki, E k ) = 1, (two attributes equivalent in diff. entity) If E o, 0 k, E o in S m, E k E o E o ({B o1, B o2,..., B oao }) and E o B oj, in {B o1, B o2,..., B oao } and E k.a ki E o.b oj, 1<= i <= a k, 1 <= j <= a o Database Specialization Course

12 Overall Algorithm 1. Calculate minimality score if minimality = 1, then stop 2. Search for fully redundant entities in S m 3. If there are fully redundant entities then eliminate the redundant entities from S m 4. Search for redundant relationships in S m 5. If there are redundant relationships then eliminate the redundant relationships from S m Search for redundant attributes in S m 6. If there are redundant attributes then eliminate the redundant attributes from S m 7. Go to Step 1 Database Specialization Course

13 Conclusion Can check an integrated database schema for Completeness Type consistency Minimality Schema with high score on quality attributes is Simpler to understand Easier/faster to query Uses existing model for computing quality attribute metrics Database Specialization Course

14 Good Highly relevant topic being addressed Could save man-hours => interesting for companies Paper well motivated speed up querying on integrated schema Very good examples concrete, minimal, relevant and many of them Overall flow of paper is good Good focus: completeness, minimality, type consistency Provide metrics for quality attributes Ideas supported by an implementation Good use of references Not too many self-references Database Specialization Course

15 Could be improve Not clear what happens with actual data in source schemas Do a modification what does then happen? Only binary relationships Not clear n-ary handled in real-world example Implementation section weak The results obtained with these experiments have been satisfactory [section 7.4] Why not state that reference relationships are foreign keys? Not clear if useful in practice due to verbose rules Very small font-size in some places Footnote 1 seems misplaced Real-world example missing Or a discussion of a larger example Overall algorithm too high level Database Specialization Course