Efficient Semantically Equal Join on Strings in Practice

Transcription

1 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 Effcent Semantcally Equal Jon on Strngs n Practce Juggapong Natwcha Computer Engneerng Department, Faculty of Engneerng Chang Ma Unversty, Chang Ma, Thaland 5000 juggapong@changma.ac.th Xngzh Sun IBM esearch Laboratory, Bejng, Chna sunxngz@cn.bm.com Mara E. Orlowska School of Informaton Technology and Electrcal Engneerng The Unversty of Queensland, Brsbane, Qld 407, Australa mara@tee.uq.edu.au Abstract In general, data ntegraton begns wth schema ntegraton and must be followed by detaled data nstances resemblance n order to reach data representaton unfcaton. In ths paper, we address the lmts of the data-level reconclaton automaton process n cases where the compared data s semantcally equvalent, but the data representaton of the values of gven attrbutes s dfferent. We assume that a semantc relatonshp between potentally used terms s establshed by a human expert pror to the desgned computatons, and s represented n an auxlary table bult and mantaned for each attrbute. On such a context, we ntroduce a noton of semantcally equal jon (SEJ), whch s the jon operaton based on a pre-defned semantc relatonshp. Our goal s to propose a soluton for SEJ that can be supported by standard SQL. The paper begns wth an llustraton of the approach for a sngle attrbute jon, followed by a generalsaton of SEJ for any number of jon attrbutes. The paper contnues wth performance consderatons for the nvented method. Fnally, ths aspect s supported by extensve expermentaton based on our mplementaton of SEJ executed on synthetc datasets. Keywords: Approxmate Strng Jon, Data Integraton, Semantcally Equal Jon.. Introducton The data ntegraton ssue [] s one of the most challengng problems that computer scence and IT practtoners have faced n the last decade or so. Ths problem orgnally emerged when demand for data access from multple sources was observed n the context of dstrbuted mult-database systems; later, database federaton systems and other versons of data ntegrated constructons ncludng data warehousng and, very recently, e-research applcatons. Frstly, ntegraton problems occur at the schema level [] where the name, meanng and constrant scope of attrbutes must be addressed. Usually, human nvolvement n ths process s necessary [3]. Subsequently, the data-level ntegraton 54

2 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 problems need to be addressed. The problem at ths level exsts due to: potental msmatch of attrbutes domans, adopted strngs to represent attrbutes values, as well as conventons to express the data felds. Addtonally, the data msmatch may be caused by many other reasons, such as, for nstance, typng errors. In ths paper, we consder the data ntegraton where two gven relatons are joned wth each other on some attrbutes. On ths context, we address the synonym msmatch problem,.e., the values of jon attrbutes are semantcally equvalent, but unable to match due to dfferent representaton, e.g. dfferent abbrevaton standards. Naturally, on such an attrbute doman, the semantc equvalence relatonshp between strngs needs to be pre-defned. We call the jon operatons that are based on ths semantc equvalence relatonshp, semantcally equal jon (SEJ). For example, from Fgure a, and are two relatons whch contan the medcal records of two dfferent hosptals respectvely. Assume that one ssues a query to fnd pars of patents who have the same type of dsease. Due to the fact that n medcal doman, strngs Tumours, Kunb, Neoplasms are semantcally equvalent and so are the strngs Mad Cow" and B.S.E." (Bovne Spongform Encephalopathy), the result of the query should be the relaton gven n Fgure b. We can see that ths query result can be obtaned by jonng two relatons and on ther common attrbute Dsease" such that the jon condton s true f and only f the values of the jon attrbute are semantcally equvalent. In the rest of ths paper, the focus s to formally defne the SEJ operaton and to propose an effcent approach to compute SEJ n a standard SQL envronment. Our dscusson begns wth an llustraton of the approach for -attrbute SEJ. Frst, to represent the semantc relatonshp between potentally used terms, we ntroduce the auxlary table whch s an addtonal relaton bult and mantaned for the par of attrbutes on whch the jon s to be performed. The desgn of the auxlary table s mportant because t drectly affects the SEJ performance. To address ths ssue, we desgn the schema of the auxlary table, based on whch, the relatonal algebra for SEJ s proposed. Snce, n general, the way that the SQL statement s wrtten wll affect the performance of queres n DBMS, we optmze the query performance for the group-based approach by adjustng ts SQL formulaton. Partcularly, we propose a subquery-based SQL statement for SEJ, whch s much more effcent than the SQL, that s drectly translated from the relatonal algebra. PID Dsease PID Dsease P Tumours Q Neoplasms P Knub Q B. S. E P 3 Mad Cow Q 3 Brd Flu P 4 Brd Flu a) Example relaton PID Dsease PID Dsease P Tumours Q Neoplasms P Knub Q Neoplasms P 3 Mad Cow Q B. S. E P 4 Brd Flu Q 3 Brd Flu b) SEJ result Fgure : SEJ Example 55

3 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 Learnng from dscusson of - attrbute SEJ, we generalze the SEJ problem to k-attrbutes, where k auxlary tables need to be desgned and mantaned. Based on the sub-query-based SQL statement proposed for -attrbute SEJ, t s straghtforward to get a query for computng k-attrbutes SEJ. Because the jon condton of k-attrbutes SEJ s the conjuncton of k predcates (each of whch corresponds to the condton for a -attrbute SEJ), the order of these k predcates n the SQL statement wll sgnfcantly affect the cost of the query [4]. To optmze the performance for k-attrbutes SEJ, we modfy the order on these k predcates by referencng the sze of each auxlary table, and the jon selectvty of each par of jon attrbutes, such that the total cost of accessng k auxlary tables s mnmum. Fnally, we evaluate the effcency of the proposed approaches by performng ntensve experments on synthetc datasets wth varous characterstcs. The paper organzaton s gven as follows. Secton provdes a revew of related work and how the SEJ s postoned n ths context. Our proposed approach s shown n Secton 3. The experments and results are reported n Secton 4. Fnally, we provde concludng remarks n Secton 5. elated Work and Dscusson The problem of jons wth nexact matchng of correspondng data, approxmate strng jons, has already been addressed by many authors expermentng wth a varety of approaches. Most of them search for solutons that can be effectvely executed by the exstng and wdely used tradtonal relatonal database envronments, DBMS and SQL, wthout any necessty for any specal purpose programmng. The problem was addressed frstly n [5]. The man concept behnd ths approach s to decompose a strng attrbute value nto substrngs called q-gram [6] whch has been used wdely n strng matchng context. The length of substrngs s specfed by q value. For example, f q s set at 3, a strng John" can be decomposed nto the set of substrngs { ##J", #Jo", Joh", ohn", hn$", n$$"}. When matchng two strngs, a dstance functon based on the edt dstance s computed on the two sets of ther substrngs. A number of q-gram related propostons were proposed to reduce the cost of edt dstance computaton. In [7], an approxmate strng jon approach based on cosne smlarty computaton was proposed. In ths work, strng data of each attrbute (whch can be composed of many terms) s represented as a mult-dmensonal vector. For each attrbute, the number of dmensons s the number of all possble terms used n the attrbute (a strng may consst of multple terms). For each dmenson, a magntude s assgned accordng to the sgnfcance of the correspondng term; an nfrequent term wll have a hgher value, whle a common term wll have a lower value. For example, a strng Informaton Technology and Electrcal Engneerng" n an afflaton attrbute wth eght possble terms can be represented as an 8-dmensonal vector (0, 0, 0.5, 0., 0.6, 0.4, 0, 0), where 0.5, 0., 0.6 and 0.4 represent the sgnfcance of the terms Informaton, Technology, Electrcal and Engneerng respectvely. When strngs are to be matched, the cosne smlarty functon s then computed. If the cosne smlarty of two strngs s hgher than a prespecfed threshold, they can be matched. For example, f a specfed threshold s set at 0.4, the above strng and another strng Informaton Engneerng" represented as a vector (0, 0, 0.5, 0, 0, 0.4, 0, 0) can be matched because ther cosne smlarty s 0.4 (0.5x x0.4). Clearly, one can buld many examples where such a matchng wll not delver hgh qualty results, manly due to an evdent lack of dependency measures between the terms. 56

4 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 Also, t s ntrnscally dffcult to allocate the best values for sgnfcance measures, and any allocaton wll be regarded as hghly subjectve. Also, n the same work, a samplng based computaton has been used to reduce the computatonal cost of the functon. Instead of computng a smlarty functon for every par of the jon canddates, whch s an expensve operaton, nsgnfcant canddates wll be sampled out, wth only the promsng pars computed. In the same way as n [5], ths work does not requre any modfcaton to DBMS. Only auxlary tables are requred to be created and they are used to drve the matchng mechansm. In [8], web data ntegraton was proposed as another applcaton for approxmate strng jons. In that work, more extensve theoretcal aspects and expermental results were presented. Furthermore, many combnatons of dstance functons were dscussed, for example, edt dstance, block edt dstance, cosne smlarty wth words as terms, and cosne smlarty wth q-grams as terms. As a result, the last type of combnaton, cosne smlarty wth q-grams as terms, has been suggested as the best dstance functon because t can capture all types of msmatch, ncludng spellng errors, nserton and deleton of short or common words and varatons of word order. An attempt to nclude semantc knowledge nto the approxmate strng jon computaton (rather than purely structural strng comparson as ndcated above), s presented n [9]. To mprove smlartybased jons, two terms whch share the same semantc meanng, but have too much dfference n ther representatons (e.g. MCI" and Worldcom" whch refer to the same company), are formed as a par n an auxlary table by human experts. Subsequently, n the pre-processng phase of a jon, each tuple s extended by assocatng ts value wth the pars n the auxlary table whch have a hgh" smlarty score. For example, consder a tuple wth a value MCI Corp". After the pre-processng phase, the value wll be extended to ( MCI Corp", MCI", Worldcom") by assocatng the value wth a par of ( MCI", Worldcom"). When answerng a matchng query, the attrbute values of the query strng are also extended by usng the auxlary table n the same way as n the pre-processng phase. Suppose that an attrbute value Worldcom Corp" s gven n the query, t wll be extended to ( Worldcom Corp", Worldcom", MCI"). Consequently, t can be matched wth the tuple above, wth the MCI Corp" value, by tradtonal cosne smlarty computatons. Although only brefly dscussed n ths paper, ths approach can remedy the shortcomngs of prevously smlarty-based approaches by applyng semantc knowledge. Smlar to our work, ths work also addresses the synonym msmatch problem by applyng semantc knowledge. However, unlke our work, whch s to effcently compute SEJ, the focus of that approach s on how to use semantc knowledge to pre-process two relatons such that later approxmate jon approaches can acheve more accurate results wthout addtonal operatons. 3. Semantcally Equal Jon Approach 3. Prelmnares Let us frst consder two relatons ( A,... A ) and ( B,... B ) such that m n attrbutes A and B have the same doman Dom, where Dom s a set of strngs. Defnton. Semantc equvalence, denoted as, s a relaton defned on a gven doman Dom such that for every a, b Dom, a b ff a s a synonym of b. Also, we defne that a and b are strct semantcally equvalent, denoted as a b, f a b and a b. 57

5 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 A B USA Unted States GB France The States Unted Kngdom France a) Jon attrbutes {(USA, USA), (Unted States, Unted States), (The States, The States), (GB, GB), (Unted Kngdom, Unted Kngdom), (France, France)} {(USA, Unted States), (Unted States, USA), (USA, The States), (The States, USA), (Unted States, The States), (The States, Unted States), (GB, Unted Kngdom), (Unted Kngdom, GB)} b) Semantc equvalence relaton Fgure : Semantc Equvalence Example. Suppose that Fgure a gves the values of jon attrbutes A and B whch are the results of projectng the relatons and respectvely. Dom n ths example s the strng set {USA, Unted States, The States, GB, Unted Kngdom, France}. Accordng to the standard conventon used for expressng country names, we can naturally defne the semantc equvalence relaton on Dom as Fgure b. Also observe that n ths example, the strct semantc equvalence relatonshp s shown as the rght-hand-sde of the unon operator. Defnton. Gven a relaton of semantc equvalence, semantcally equal jon (SEJ) s a theta jon operator n the relatonal algebra such that the theta comparson operator s.. Further, the semantcally equal jon operator s denoted as To perform the semantcally equal jon between two tables and on attrbutes A and B, the semantc equvalence relaton on doman Dom needs to be predefned. Snce t s straghtforward to observe that a strng can semantcally jon wth tself, we defne a relaton Au, called auxlary table, to only specfy the relatonshp of strct semantcally equvalent,.e., to gve the nformaton of synonyms n Dom. Accordng to Defnton, t s easy to observe that semantc equvalence,, s an equvalence relaton. Formally, gven a relaton of semantc equvalence relaton on Dom, we can partton Dom nto a set of equvalence classes C, = l such that for any C, there does not exst elements a, b holdng the condton of a b. C Note that only the class C wth C provdes the nformaton of synonyms. We defne the set of non-trval equvalence classes, formally, { C C } where = l. Accordng to the above dscusson, Au can be regarded as the representaton of the set. Let us further consder the Example. Gven the relaton, Dom s parttoned nto three equvalence classes: C { USA, Unted States, The States }, C { GB, Unted Kngdom}, and C3 { France}. The As an equvalence relaton, = has the followng property: ) a a for all a Dom, ) a b b a for all a, b Dom 3) a b b c a c for all a, b, c Dom. 58

6 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 set of non-trval equvalence classes s { C, C }. After ntroducng the key concepts, we generalze the problem of semantcally equal jon to k attrbutes and gve the formal problem statement as follows. Problem statement: Let ( A,... A ) and m ( B,... B ) be two relatons such that n attrbutes.a and.b (= k) have the same doman Dom, where Dom s a set of strngs. For each Dom (= k), let be the equvalence relaton defned on Dom and Au be the auxlary table that represents the correspondng strct semantcally equvalent relatonshp. The semantcally equal jon (SEJ) problem s: gven relatons, and auxlary table Au,..., Au k, express k A B,... A k Bk by a sngle SQL statement. In the rest of Secton 3, we wll frst study how to buld an SQL statement to effcently compute SEJ on one attrbute, then dscusson s extended to SEJ on k- attrbutes (k>). 3. Schema of Auxlary Table and Semantcally Equal Jon Frstly, we propose a structure of auxlary tables, called the group-based approach for the schema of the auxlary table. In ths approach, gven the set, the auxlary table Au s defned as Au ( Gd, P) {(, a) C and a C }. In Example, the auxlary table s gven n Fgure 3. Wth ths schema, we can buld the prmary ndex on the attrbute P for effcent access because there can not exst a strng that belongs to two dfferent synonym groups. GID P USA The States Unted States GB Unted Kngdom Fgure 3: The example of group-based auxlary table Gven the schema of Au, the A expresson for the semantcally equal jon for ths schema s: ( (( Au) P A ) B A Au. GID Au '. GID where ( B Au' )) P Au ' s the copy of Au. () 3.3 Query for Implementng SEJ Gven the relatonal algebra operaton n Equaton (), there are multple ways to buld an SQL statement to compute the semantcally equal jon. Although generally, dfferent queres may have the same executon plan n a gven DBMS, n many cases, the way to wrte a query wll affect the executon plan and the query performance. In ths secton, we propose an SQL statement whch can effcently compute semantcally equal jon. ather than consderng query optmzaton technques whch are dependant on the DBMS, we only put our effcency concern at the SQL level SQL Statements Accordng to Equaton (), we can drectly translate the A nto an SQL 59

7 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 statement, whch s shown n Fgure 4a. Snce part of ths SQL statement requres three jon operatons to lnk four tables, we call t three-jon-based SQL statement. (SELECT * FOM, WHEE A =B ) UNION (SELECT * FOM,, Au Au, Au Au WHEE (A =Au.P AND B =Au.P AND Au.d = Au.d)) a) Three-jon-based SELECT * FOM, WHEE A =B O ((SELECT Gd FOM Au WHEE P=A ) = (SELECT Gd FOM Au WHEE P=B )) b) Sub-query-based Fgure 4: SQL statements Intutvely, the cost of jonng four tables s expensve. Therefore, to mprove the effcency, we propose another SQL statement n Fgure 4b, called sub-querybased SQL statement. The semantc meanng behnd ths query s that two tuples can be joned together f ether of the values of the jon attrbutes are the same, or they belong to the same group n the auxlary table. Note that when both values of A and B are not n the auxlary table, the condton after O wll compare two Null values. In the standard SQL, the result of Null=Null s unknown, whch wll not make two tuples jon Cost Analyss We brefly dscuss the I/O cost for computng the SEJ n ths secton. Let us frst consder the three-jonbased SQL. In Fgure 4a, we can see that the query conssts of three operatons: equal jon, operaton of three-jon, and unon. It s not hard to observe that ths query wll generate a sgnfcant amount of ntermedate results from jons and requre the external sort to remove duplcates for the unon operaton, both of whch wll lead to hgh I/O cost. For the sub-query-based approach, the query plan s much smpler: When and are joned, for any par of tuples whch can not equally jon, the auxlary table s accessed by the prmary ndex (on the attrbute P) twce to check the semantc jon condton. Note that n practce, an auxlary table s not very large and often can be cached n the buffer. In ths case, the I/O cost of sub-query-based SQL s close to the cost of an equal jon of and. For the above dscusson, we can see the sub-query based approach requres less dsk I/O when the sze of Au s small. Ths wll be also demonstrated n the experment at secton. 3.4 k-attrbutes Semantcally Equal Jon Now, we consder the task of semantcally jonng two relatons on k attrbutes, as s descrbed n the problem statement n Secton 3.. As the jon condton of k-attrbutes SEJ s the conjuncton of the condtons for the sngle attrbute jons, we can readly extend the prevous SQL statement shown n Fgure 4 nto the statement n Fgure 5. Snce the jon condtons are lnked by AND, f any set of condton s not satsfed, t s not necessary to check the other jon condtons of other attrbutes. Apparently, we can order the jon condtons sets to mnmze the cost of accessng auxlary tables. Intutvely, two factors need to be consdered: ) the sze of auxlary table Au ) the jon/sej selectvty of each par of jon attrbutes. SELECT * FOM, WHEE (A =B O ((SELECT Gd FOM Au WHEE P=A )=(SELECT Gd FOM Au WHEE P=B ))) 60

8 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 AND(A k =B k O ((SELECT Gd FOM Au k WHEE P=A k )=(SELECT Gd FOM Au k WHEE P=B k ))) Fgure 5: SQL for k-semantcally equal jon It s well-known to the database communty that the query performance can be adjusted by orderng jon condtons lnked by AND. However, drven by the specalty of a new type query, we dscuss ths orderng problem on the context of SEJ wth the purpose of mnmzng the cost of auxlary table access. Let p be the number of pages of the auxlary table Au where =,...,k. Gven relaton and, let ndex of the attrbute s,..., l ( j s the j k ) be the semantcally equal jon selectvty of and on l attrbutes. Formally, A B,..., A l B l to: s,..., l s equal. Smlarly, we defne the equal jon selectvty s,, l as: A B,..., A B l Gven an order of jon attrbutes,..., k, under the assumpton that the jon attrbutes are ndependent, the total I/O cost on access k auxlary tables s: [( s ) p s ( s ) p... s,,... l ( s ) p k k k Also due to the assumpton that the jon attrbutes are ndependent, we have ^ s s,... l. So the orderng task j... l can be presented as: j ]. Gven the szes of auxlary tables p,p,,p k, the equal jon selectvty of each par attrbutes s,s, s k, and the semantcally equal jon selectvty of each par attrbutes s,s, s k, are ordered to fnd an order,,, k cost functon such that the [( s ) p s ( s ) p... s,,... k ( s ) p ] k s mnmzed. k To fnd the order wth mnmum cost, theoretcally, we need to compute the costs for k! orders. However, based on the cost functon, we can apply some heurstcs to speed up the search. The equal jon selectvty or semantcally equal jon selectvty s often small. Consderng the cost functon, assume that the values of (s )p for =,, k are not sgnfcantly dfferent, we can expect that the cost of accessng -th auxlary table s less than that of accessng the ( ) -th auxlary table n a manner of magntude. Intutvely, to mnmze the total cost, t could be helpful to frst select ndex wth mnmum (s )p. Based on ths dea, we can gnore the semantcally equal jon selectvty and order the ndex accordngly accordng to the value (s )p. Once the optmum order O of ndex on the jon attrbutes s found, the query n Fgure 5 can be optmzed by re-orderng the jon condtons based on O. 4. Expermental esults In ths secton, to evaluate the effcency of SEJ, we report the result of the performance study on synthetc datasets. 4. Expermental Confguraton System resource: The experments are conducted on an 800 MHz Intel Pentum III 6

9 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 wth 56 megabytes man memory, runnng a commercal DBMS (Oracle 8). All nvented methods are mplemented usng standard SQL. Datasets: In our experments, all the relatons (,, and auxlary table Au) are generated by dfferent random varables, each of whch follows the unform dstrbuton. For brevty, we only show how to generate relatons for -attrbute SEJ. The dscusson can be readly extended to k- attrbutes SEJ. There are two controlled parameters for the auxlary table, the cardnalty Au and the number g of semantcally equvalent groups. Gven Au and g, we generate the group-based auxlary table Au(Gd,P) by randomly assgnng Au dstnct strngs nto g groups. Based on the unform dstrbuton, Au g gves the expected number of strng values n each group, whch s denoted as v. For the two relatons and, ther schema are desgned as ( A, A ) and ( B, B ) wth A and B to be the jon attrbutes. Further, we consder the nstances n each relaton. Agan for smplcty, we set the constrants that the values of attrbute A are exactly the same as the values of attrbutes B Consderng the attrbute A ( B ), ts values consst of three parts: the S part whch has the attrbute values n the auxlary table (.e., can semantcally match the values n the other relaton), E part whch has the values that can only exact match the values n the other relaton, and the N part, whch has the values that cannot match any values n the other relaton. Let S%,E%,N% (S%+E%+N%=) be the percentages of the values n part S, part E, and part N respectvely, and ns,ne,nn be the numbers of dstnct values n these there parts. Gven the above parameters, the jon relatons are generated by followng the equal dstrbuton. For example, n S part, we can expect each dstnct value appears * S% tmes, where s the ns cardnalty of ( ). 4. Expermental esults and Dscusson -Attrbute SEJ esults and Dscusson Effect of jon relaton sze : In ths experment, we ncrease from 00 records to 4,000 records to examne the performance of two types of query wth dfferent szes of jon relaton. The other parameters of the jon relatons are set as: S%=80%, E%=0, N%=0%, ns=0, ne=0, nn=3. For auxlary table, we set Au =0 and g=3. The performance of -attrbute SEJ by dfferent types of queres s presented n Fgure 6a. It can be seen that the three-jonbased approach s very neffcent. Therefore, t cannot be used for a large amount of data. As dscussed n Secton 3, the SQL statement for three-jon-based approach requres that four tables are joned, whch leads to expensve I/O cost. Therefore, for the rest of ths secton, we wll omt the result of the three-jon-based approach and only dscuss the group-based sub-query. Effect of selectvty s ^ : The second consderaton for -attrbute SEJ s about ts performance regardng the semantc jon ^ ^ selectvty s of and where s s composed of two parts, strct semantc jon selectvty s ' (.e. the selectvty for the pars of tuples that are matched based on strct semantcally equvalent relatonshp) and equal jon selectvty s (ntroduced n Secton 3.4). By defnton, we always have ^ s s s'. Note here that we slghtly change the way of generatng synthetc datasets n ths part. The records n S part are allowed to only be joned through an auxlary table (.e., no equal jon n ths part). In ths way, due to the equal 6

10 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 dstrbuton, the strct semantc jon ( S%) selectvty s, where g s the g number of groups n Au. Smlarly, gven E% and ne, the equal jon selectvty can be ( E%) estmated as. In the experments, ne the strct semantc jon selectvty s and equal jon selectvty can be adjusted by changng parameters S%, ns, E%, and ne. Both experments use some common parameters whch are the sze of jon relatons at 0,000 tuples wth ne at 5 and nn at 00. Also, for both experments, we set the sze of auxlary table Au at 00 wth g at 5, whch means that v s set at 4. To evaluate the mpact of s ', we fx s by always settng E%=0% and ne=5. We ncrease the strct semantc jon selectvty by ncreasng S% from 0% to 80%. Note that when S% s ncreased, we also decrease N% to keep the sze of jon relaton the same. For evaluatng the performance of SEJ regardng to equal jon selectvty s, we fx the s ' by always settng S%=0% and ns=00 and ncrease E% from 0% to 80%. a) Effect of jon relaton b) Effect of selectvty Fgure 6: Performance of SEJ We frstly measure the performance of SEJ regardng to s ', whch s shown n Fgure 6b. Obvously, the performance of SEJ s rrelevant to strct semantc jon selectvty. Ths verfes our thoughts that the amount of tme for auxlary table access only depends on the equal jon selectvty s. The reason s that for any two records t and t the auxlary table needs to be accessed unless t and t can be equal joned. Also, n Fgure 6b, we report the performance of SEJ when the selectvty s s ncreased. We can see that due to the same reason, the executon tme decreases when E% s ncreased. k-attrbutes SEJ esults and Dscusson In ths experment, we dscuss the SEJ jon n terms of the order of k attrbutes n an SQL statement. Partcularly, we verfy our cost analyss n Secton 3.4 and demonstrate how the proposed heurstcs help to quckly fnd an optmal order. As dscussed prevously, the followng two factors are consdered for the orderng: ) sze of auxlary table Au ) jon/sej selectvty. We generate jon relatons wth three jon attrbutes (k=3) of sze 00,000 records. The characterstcs and the estmated selectvtes of the relatons are shown n Table. As shown n Table, the frst and the 63

11 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 second attrbutes have the same Au, but they are dfferent n selectvtes. On the other hand, even f the thrd attrbute has the lowest selectvty, t has the largest Au. The performance of all possble orders of jon condtons for the dataset s evaluated. The performance on k-attrbutes SEJ wth all possble jon condton orderngs s reported n Fgure 7. On the x- axs, we put the ascendng order for all theoretcal I/O costs (for dfferent orderng) computed from the cost functon. The result shows that the order of actual cost s accordant wth the order of estmated cost, theoretcally. Ths demonstrates the correctness of our prevous analyss. Compared wth the worst case order 3--, the best order --3 has better performance, by 7.9%. As mentoned n Secton 3.4, we can also compute the best order by computng estmated cost ( s ) p, n whch the result s also the order --3. Another nterestng observaton s that the order --3 has more cost than the order - 3-, whch shows that the SEJ selectvty should also be consdered, not only the sze of auxlary tables. On the other hand, orderng only by SEJ selectvty can not perform very well, as presented by the order 3--. Table : Characterstcs and Selectvtes of Expermental Dataset Attrbute Auxlary table Jon relaton Selectvty Au g v S% E% N% ns ne nn s ' s % 60% 30% % 3.6% 3.8% % 0% 40% % 0.% 5.% % 0% 40% % 0.4% 3.6% ^ s Fgure 7: Performance of k-attrbute SEJ. 5. Concluson In ths paper, we have addressed the problem of synonym msmatch n the context of data ntegraton, where the data from dfferent sources are semantcally equvalent, but the data representaton s dfferent. We have ntroduced a new relatonal algebrac operaton, called semantcally equal jon (SEJ), whch can be generally appled for ntegratng two relatons based on predefned equvalence relaton(s). The orgnalty and contrbutons of our work nclude the followng aspects: 64

12 Thammasat Int. J. Sc. Tech., Vol. 4, No., Aprl-June 009 ) we have ntroduced the semantc equvalent relatonshp to resolve the synonym msmatch problem. In fact, ths study can also complement prevous work of approxmate jon that manly focuses on resolvng typo msmatch. ) We have formally defned the concept of SEJ and proposed an effcent approach to mplement the SEJ operator n the standard DBMS by a sngle SQL statement. 3) By extensve experments we have demonstrated performance of our approach for SEJ. 6. eferences [] Halevy, A.Y., Ashsh, N., Btton, D., Carey, M., Draper, D., Pollock, J., osenthal, A., Skka, V., Enterprse Informaton Integraton: Successes, Challenges and Controverses, In Proceedngs of the 005 ACM SIGMOD Internatonal Conference on Management of Data, New York, NY, USA, ACM Press, pp , 005. [] ahm, E., Bernsten, P.A., A Survey of Approaches to Automatc Schema Matchng, the VLDB Journal,Vol. 0, pp , 00. [3] Colomb,.M., Orlowska, M.E.: Interoperablty n Informaton Systems. Informaton Systems, Journal,Vol.5, pp.37 50,994. [4] Orlowsk, M.W., On Optmsaton of Jons n Dstrbuted Database System, In: Future Databases, pp.06 4,99. [5] Gravano, L., Iperots, P.G., Jagadsh, H.V., Koudas, N., Muthukrshnan, S., Srvastava, D., Approxmate Strng Jons n a Database (Almost) for Free, In Proceedngs of 7th Internatonal Conference on Very Large Data Bases, Morgan Kaufmann, pp , 00. [6] Ullmann, J.., A Bnary n-gram Technque for Automatc Correcton of Substtuton, Deleton, Inserton and eversal Errors n Words. The Computer Journal, Vol. 0, pp.40 47, 976. [7] Gravano, L., Iperots, P.G., Koudas, N., Srvastava, D., Text Jons for Data Cleansng and Integraton n an dbms, In Proceedngs of the 9th Internatonal Conference on Data Engneerng, IEEE Computer Socety, pp.79 73, 003. [8] Gravano, L., Iperots, P.G., Koudas, N., Srvastava, D., Text Jons n an dbms for Web Data Integraton, In Proceedngs of the Twelfth Internatonal World Wde Web Conference, pp.90 0, 003. [9] Koudas, N., Marathe, A., Srvastava, D., Flexble Strng Matchng Aganst Large Databases n Practce, In Proceedngs of the Thrteth Internatonal Conference on Very Large Data Bases, Morgan Kaufmann, pp ,