Mapping Data in Peer-to-Peer Systems: Semantics and Algorithmic Issues

Transcription

1 Mapping Data in Peer-to-Peer Systes: Seantics and Algorithic Issues Anastasios Keentsietsidis Marcelo Arenas Renée J. Miller Departent of Coputer Science University of Toronto ABSTRACT We consider the proble of apping data in peer-topeer data-sharing systes. Such systes often rely on the use of apping tables listing pairs of corresponding values to search for data residing in different peers. In this paper, we address seantic and algorithic issues related to the use of apping tables. We begin by arguing why apping tables are appropriate for data apping in a peer-to-peer environent. We discuss alternative seantics for these tables and we present a language that allows the user to specify apping tables under different seantics. Then, we show that by treating apping tables as constraints (called apping constraints) on the exchange of inforation between peers it is possible to reason about the. We otivate why reasoning capabilities are needed to anage apping tables and show the iportance of inferring new apping tables fro existing ones. We study the coplexity of this proble and we propose an efficient algorith for its solution. Finally, we present an ipleentation along with experiental results that show that apping tables ay be anaged efficiently in practice. 1. INTRODUCTION Traditionally, data integration and exchange between heterogeneous data sources is provided ainly through the use of views, i.e., queries that ap and restructure data between the heterogeneous scheas [13, 20]. Since queries depend on the underlying scheas, to correctly restructure and ap data, the sources ust be willing to share their scheas and cooperate in establishing and anaging the queries. In our work, we consider peer-topeer settings in which such close cooperation is either not desirable (perhaps for privacy reasons) or not feasible (perhaps due to resource liitations or the dynaic nature of the data structures) [11, 17]. Perission to ake digital or hard copies of all or part of this work for personal or classroo use is granted without fee provided that copies are not ade or distributed for profit or coercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific perission and/or a fee. SIGMOD 2003, June 9-12, 2003, San Diego, CA. Copyright 2003 ACM X/03/06...$5.00. To find data when there is no agreeent on the logical design of data (or on how different logical designs correspond), we ust focus on data values and how values correspond. If we can ap values, particularly identifying values (naes or keys), we can still request and exchange specific data of interest. Indeed, this idea provides the basis for data exchange in current peer-topeer systes. In file-sharing systes, like Gnutella [2] and Napster [7], querying is perfored by using siple value searches on file naes [19]. So queries are of the for: Retrieve all files naed X (or containing the phrase X). This siple for of search has proven effective for so any applications, that there has been a flurry of research on aking such searches ore efficient and scalable [24; 26, and others]. This schee works in doains where there is consensus on what the naes should be. So for usic files, where there is a standard, coonly accepted nae for each song or albu, data can be shared because each peer uses the sae (or siilar) values to nae files. However in other doains, where there is no accepted naing standard, different peers ay necessarily have had to develop their own naing conventions. Standards often eerge after any sources have set up their own naing conventions. There ay be any applications that depend on the use of the internal conventions. So, igration to confor to external standards is tie-consuing and expensive [12]. To search data in such environents, people have ade use of apping tables that store the correspondence between values. At their siplest, these tables are binary tables containing pairs of corresponding identifiers fro two different sources. With such tables, we can still use our siple value searches but for a peer to find a file called X it first consults a (shared or local) apping table to find the nae(s) of X in each acquainted peer. In general, we ay need to ap values containing ultiple attributes. For exaple, geographic locations ay be indicated by soe for of federal postal code in one peer and by pairs of area codes and town naes in a second. However, we can still use these apping tables to exchange data related to specific values. The query retrieve all inforation related to postal code X in peer one becoes retrieve all inforation related to the area code, town pair (Y, Z) in peer two. Or the translation ay be to a set of values if the ap-

2 ping is not one-to-one or any-to-one. Note that data exchange in ore structured data anageent settings (as opposed to file sharing settings) is often achieved using self-describing data odels. Since there is no a priori agreeent on the structure of the result, the data, once found, is exchanged with its descriptive schea. Nonetheless, siple value queries, where there is agreeent on how values correspond, can be quite effective for facilitating search and exchange between data anageent systes. Contributions: Mapping tables represent expert knowledge and are typically created by doain specialists. Indeed, currently the creation of apping tables is a tie-consuing and anual process perfored by a set of expert curators. While widely used, especially in the biological doain [15], we are aware of no data anageent tools currently designed to facilitate the creation, aintenance and anageent of these tables. In this work, we discuss alternative seantics for these tables and we present a language that allows the specification of apping tables under different seantics. We illustrate how autoated tools can help anaging apping tables between ultiple sources by inferring new appings (that is, new entries in a apping table). Specifically, we show that by treating apping tables as constraints (called apping constraints) on the exchange of inforation between peers it is possible to reason about the and check their consistency. Note that these constraints are not on the peers theselves (or their content), only on the way in which search values are translated. We study the coplexity of the inference and consistency probles and we propose an algorith for solving these probles. Finally, we present an ipleentation of our algorith along with experiental results which show that, by using our results, apping tables ay be anaged efficiently in practice. The work presented in this paper is part of the Hyperion project [4]. The ain goals of the project are: the definition of a peer-to-peer data anageent architecture; the study of viable data integration, exchange, and apping echaniss; the developent of algoriths for the efficient search, retrieval and exchange of data aong peers. Mapping tables provide the foundation for exchanging inforation between peers. Outline of the paper: In Section 2, we illustrate our techniques with a specific exaple. Section 3 forally introduces the notion of apping tables. Then, in Section 4, we present the benefits of considering apping tables as constraints and we introduce a language for specifying how tables ay be cobined. Section 5 analyzes the coplexity of both the inference and the consistency probles. Section 6 presents an efficient algorith for solving these probles while Section 7 presents its ipleentation along with experiental results. Section 8 describes related work, while Section 9 offers a suary of the conclusions. 2. MOTIVATING EXAMPLE Consider an exaple drawn fro the doain of biological databases. Genoic data can be found in a large nuber of authoritative sources ranging fro wellknown public sources, to ones specific to individual research labs. The exaples in this paper will be drawn fro public sources including GDB [1] (a gene database), SwissProt [8] (a protein database), and MIM [6] (a database about genes and genetic disorders related with these genes). Integration of these sources to provide unifor access for scientists, although extreely desirable, sees unattainable due to a yriad of political, financial and technical reasons [15]. Aong the technical reasons is the inherent heterogeneity of the sources which range fro relational databases to foratted files or spreadsheets. In addition, the scheas and forats of the sources evolve rapidly in response to new biological techniques and requireents. To achieve soe degree of integration, biologists coonly use what we have called apping tables. For exaple, apping tables can be used to relate gene data in one source to the related protein data in another source (where the gene is said to encode for the protein). Note that the apping table is not necessarily a function, there ay be any proteins related to a gene. Even a apping table relating gene identifiers ay be any-to-any. This occurs often in biological sources where there ay be aliases for the sae identifier. As identifiers are updated, old identifiers ay need to be kept. For exaple, they ay refer to the content of static (non-updateable) sources such as journal articles which ay contain antiquated naes and identifiers for entities. In what follows, we discuss soe of the ain characteristics and uses of apping tables. First, we show that apping tables can be used to associate values both within and across doains. Second, we show that apping tables are an appropriate tool to use in peer-to-peer systes since they respect the autonoy of the peers. Finally, we otivate the need for alternative seantics in apping tables and we present soe exaples that otivate why reasoning capabilities are desirable in an environent where apping tables are used. Associations within and Across Doains: Notice that by using apping tables we are able to associate seeingly unconnected databases, soething that has been called ediation across ultiple worlds [21]. In a typical integration scenario, we are often dealing with one world, for exaple, a set of sources all containing inforation about genes. However, there are situations, where sources fro disjoint worlds can be associated since the corresponding worlds are seantically close to each other. As an exaple, the GDB database has a apping table in which it stores associations between its gene identifiers and protein identifiers fro SwissProt. Using the apping table, users of the GDB database are able to retrieve, for each gene, related protein(s) fro SwissProt. An exaple of such a table is shown in Figure 1. While we have siplified (and shortened!) the data in the figure for exposition, the experients we report in Section 7 use the actual GDB to SwissProt apping table containing 8,780 entries. Peer Autonoy: Autonoy is of utost iportance in any peer-to-peer syste and in any types of

3 GDB id GDB: GDB: SwissProt id P21359 O00662 Q9UMK3 P35240 P01138 Figure 1: Mapping Table 1 Open-world Closed-world present Any indicated X-value Y -value Y -values issing Any No X-value Y -value Y -value Table 1: Alternative open/closed world seantics networked applications. Mapping tables respect the autonoy of the sources that they associate. They are inially invasive in that they perit searching across peers, but do not restrict the operation of peers in anyway beyond the agreeent on values expressed in the tables. To see this, notice that the apping table shown in Figure 1 does not express how genes and proteins are related in general, nor how they should be represented or stored in their respective sources. Rather, it only encodes the fact that a doain expert has deterined that certain genes are related to certain proteins. Seantics: Doain specialists have varying levels of expertise. Mapping tables should record not only the associations suggested by the doain specialists but also their confidence on these associations. In what follows, we discuss alternative seantics for apping tables and show how these can be used to express partial or coplete knowledge for a doain of interest. A apping table consists of two disjoint sets of attributes X and Y (we use a double line in figures to distinguish between the two). A tuple (x, y) in the apping table is called a apping and it indicates that the value x is associated with y. We say that an X-value appearing in a apping table follows the open-world seantics if it can be associated with any Y -value. Under this seantics, the table encodes only partial inforation about X-values that appear in it. Alternatively, we say that an X-value appearing in the apping table follows the closed-world seantics if it can only be associated with the indicated Y -values. Under this seantics, the table encodes coplete inforation about the X-values that appear in it. Siilarly, for an X- value not present in a apping table, we say it follows the open-world seantics if it can be associated with any Y -value, while it follows the closed-world seantics if it cannot be associated with any Y -value. Tables 1 suarizes the above discussion. Thus, we are led to a space of four alternative seantics for apping tables. Specifically, under the openopen-world (OO-world) seantics both the X-values present in the table and those issing follow the corresponding open-world seantics. This seantics essentially allows the association of any X-value with any Y -value and is thus of no practical interest. Under the open-closed-world (OC-world) seantics the X-values present in the table follow the open-world seantics, while those issing follow the corresponding closed-world seantics. Given such a seantics, a apping table essentially specifies the set of X-values that can be apped to any Y -value, while the Y -values indicated on the table are not taken into consideration. As such, the seantics is of little practical use to a specialist aiing to record particular appings between specific X and Y -values. Of ore practical use, a apping table under the closed-open-world (CO-world) seantics is capable of representing partial knowledge. This proves useful in situations where the doain specialist is an expert only on a subset of the doain. As such, she is able to record her expertise by constraining the Y -values with which specific X-values can be associated. However, since she is agnostic regarding the reaining X-values, she allows the to be associated with any Y -value. The closed-closed-world (CC-world) seantics is useful for representing coplete knowledge for a doain. Under this seantics, a specialist specifies the coplete set of correct appings. Since the latter two seantics are of ost interest, we focus our attention only on these two. Autoated discovery of appings: Given a seantics for apping tables, we would like to reason about the. To achieve this, we treat apping tables as constraints on the exchange of inforation. The siplest rule for cobining apping tables is to take their conjunction, i.e., to look for all the associations that satisfy all constraints. Consider the apping tables shown in Figure 2. Suppose doain specialists have specified a CO-world seantics for all three tables. The first table indicates that pairs of genes and proteins can together be associated with a genetic disorder. Table 2(b) associates genes with proteins while Table 2(c) associates genes directly with genetic disorders. Users ay use Table 2(c) directly in a query to find all genetic disorders associated with a specific gene. However, a user ay also wish to ake use of the expertise of the doain specialists who created Tables 2(a) and 2(b) to discover if there are additional disorders that ay be associated with this gene. Under a CO-world seantics, the apping (, ) can be derived fro these three tables since we can find a witness tuple that involves all the attributes of these tables and has as GDB id and as MIM id. This tuple is t = (, O00662, ). Notice that t satisfies Table 2(c) since is not entioned there. On the other hand, the apping (, ) is not valid with respect to these apping tables, since there is no witness tuple for these values (no value of SwissProt id satisfies the conditions entioned above). Alternatively, the specialists could have used a CC-world seantics. If one or ore of the apping tables in Figure 2 have a CC-world seantics, the set of appings between GDB and MIM changes. In this paper, we present solutions for inferring new appings under both seantics.

4 GDB id SwissProt id MIM id P O GDB: P GDB id SwissProt id O00662 GDB id MIM id GDB: Mapping table 2(a) Mapping table 2(b) Mapping table 2(c) Figure 2: An initial set of apping tables 3. MAPPING TABLES In what follows, we offer a foral definition of apping tables. We use the relational odel to present our ideas (since apping tables fit conveniently into the relational odel). However, our solutions do not require that any of the peers use this odel. Indeed, our solutions are designed to work with peers that are inforation retrieval systes, DBMS, or file-sharing systes. We use the letters A, B, C, D to denote individual attributes. For an attribute A, do(a) is the doain of A. The doains we consider are the typical doains found in ost relational databases, such as, integers, strings, real nubers, booleans etc. The letters U, X, Y are used to denote sets of attributes. Letter R is used to denote a relation schea. We use the notation R[U] to explicitly show the attributes of a relation schea. Letter r is used to denote a relation instance. The letter t is used to represent tuples. We use t[x] to denote the values of tuple t in the attributes of X. Let X = {A 1, A 2,..., A k } and let do(a i) (i [1, k]) denote the doain of attribute A i. Then, do(x) = do(a 1) do(a 2)... do(a k ). Finally, we use standard relation algebra operators such as projection (π X) and selection (σ X=x). The values appearing in the appings (and apping tables) presented thus far are only constants. However, to represent the different seantics of apping tables (CO-world or CC-world seantics) it is necessary to introduce variables. Specifically, let V be a set of variables where V do(a) =, for every attribute A. We define a apping to be a tuple which ay contain constants or variables (such tuples are usually called free tuples in the literature [10]). More forally: Definition 1. Given a set of attributes U, t is a apping over U if for each A U, t[a] is either a constant in do(a), a variable in V or an expression of the for v S, where v V and S is a finite subset of do(a). To describe a set of appings, we use the ter apping table instead of the ter apping relation. This is consistent with the literature where the ter table has been used to denote relations containing variables [10]. Moreover, we ipose the restriction that each variable appears in at ost one apping. This restriction is consistent with our intuition that two different appings in a apping table are copletely independent. The following definition foralizes the above. Definition 2. Let X and Y be nonepty disjoint sets of attributes. A apping table fro X to Y is a finite set of appings over X Y such that each variable appears in at ost one apping. Variables offer a copact and convenient way of representing coon associations between values. An exaple of such an association is the identity. Exaple 3. Consider a biological database at the University of Toronto and assue that it uses the sae identifiers as the GDB database. We can represent this apping table as a list of appings of the for (id, id), where id is an identifier in the GDB database. Alternatively, we can construct a ore succinct, data independent, apping table containing the single apping, (v, v), where v is a variable. Without variables, users ust anually specify in their queries whether an identity apping should be used. In apping SwissProt data to GDB, the answer is likely no, while as the above exaple shows, other searches should ake use of the identity. Variables also perit a siple representation for alternative apping table seantics. Exaple 4. Consider the apping tables in Figure 3. The apping table on the top of the figure uses the CO-world seantics. Thus, any gene, apart fro the ones explicitly entioned, can be associated with any protein. Now, consider the table on the botto of the sae figure which uses CC-world seantics. The first two appings of this table are identical with the ones appearing in the top table. The last apping states that all GDB ids, except and GDB:120232, ay be apped to any protein. Notice that the top table with CO-world seantics expresses the sae associations as the botto table with the CC-world seantics. We conclude this section by introducing the notion of valuation which will prove useful in the following paragraphs. Definition 5. A valuation ρ over a apping table is a function that aps each constant value in to itself and each variable v of to a value in the intersection of the doains of the attributes where v appears. Furtherore, if v appears in an expression of the for v S, then ρ(v) S. 4. MAPPINGS AS CONSTRAINTS In this section, we view apping tables as constraints (called apping constraints) on the exchange of inforation between the sources. In doing so, we show that

5 GDB id GDB: SwissProt id P21359 P35240 GDB id SwissProt id P21359 GDB: P35240 v - {, GDB:120232} v Figure 3: CO-world vs. CC-world seantics we are able to reason about apping constraints, that is, given a set of apping constraints, we are able to infer new apping constraints and deterine if a set of constraints is inconsistent. In the following paragraphs we consider first how a single apping table can be treated as a constraint. Then, we consider how sets of apping constraints can be cobined. 4.1 Mapping Constraints Consider relations r and r with scheas R[U] and R [U ], respectively, and also consider a apping table fro X to Y, where X U and Y U. Let r be the Cartesian product of relations r and r where every tuple t of r is related to every tuple t of r. Given a apping table fro X to Y, we can use as a condition to filter the above Cartesian product. Specifically, let t be a tuple of the Cartesian product. Tuple t ust be reoved fro relation r if there is no valuation ρ over such that t [X] π X(ρ()) and t [Y ] π Y (σ X=t [X](ρ())). The intuition behind our definition is as follows. Consider the apping table fro X to Y, a valuation ρ over and a value x π X(ρ()). Then, the value x is associated with a certain set of values in the doain of Y, naely, with the set π Y (σ X=x(ρ())). As a result, a tuple t r such that t[x] = x can be apped, with respect to apping table and valuation ρ, only to tuples t r for which t [Y ] π Y (σ X=x(ρ())). guarantees exactly this. The above condition Exaple 6. In Figure 4, the first two relations correspond to relations in the GDB and SwissProt databases, respectively. The third relation is a apping table between GDB and SwissProt which uses the CC-world seantics. If we take the Cartesian product of the first two relations and use the apping table as a condition to filter this product, we get the relation at the botto of the sae figure. For exaple, the first tuple in this relation is valid since there is a valuation ρ such that ρ(v) = and ρ(v ) = P By treating a apping table as a constraint, we are able to identify (in)valid appings between objects that reside in different sources. That is, given a set of apped objects we are in a position to tell whether they satisfy a apping table. We now foralize the above and introduce a new type of constraint, called a apping constraint. Then, we introduce the notion of satisfiability for these new constraints and we discuss a nuber of issues that arise. Let be a apping table fro X to Y. We define Y (x), where x do(x), as follows: Y (x) = {y t and there exists valuation ρ over such that ρ(t[x]) = x and ρ(t[y ]) = y}. Notice that Y (x) contains all the values in do(y ) with which value x do(x) can be apped, under the apping table. Definition 7. Let U = X Y. An expression of the for X Y is a apping constraint over U. A U-tuple t satisfies X Y, denoted as t X Y, if t[y ] Y (t[x]). Furtherore, a relation r satisfies X Y if every t r satisfies X Y. As an exaple, in Figure 4, the relation at the botto of the figure satisfies the constraint GDB id SwissProt id, where is the apping table shown in 4(c). We use the letter µ to denote a apping constraint, and Σ to denote a set of apping constraints. Notice that the previous definition assues a CCworld seantics. We choose this seantics since we can translate (as shown in Exaple 4) any constraint µ under the CO-world seantics into a constraint µ under the CC-world seantics such that r satisfies µ (under the CO-world) if and only if r satisfies µ (under the CC-world). Fro now on, we assue that (by default) every apping constraint is under the CC-world seantics. Every tie that we ention a apping constraint µ under the CO-world seantics, we assue that we are referring to the apping constraint µ entioned above. 4.2 Mapping Constraint Forulas In this section, we introduce a language which allows us to for expressions that cobine apping constraints by using conjunction ( ), disjunction ( ) and negation ( ). Before we forally introduce the language, we otivate the use of such expressions. Exaple 8. Assue that apping constraints µ 1 and µ 2 shown in Figure 5 were constructed by two different curators. How should these apping constraints be cobined? Clearly, this is a decision that only the user can ake. For instance, if the user trusts both curators, then she will probably take the union of the apping constraints (represented by µ 1 µ 2). Alternatively, if the user only wishes to use appings that have been validated by both curators, then she would use the intersection of apping constraints (represented by µ 1 µ 2). In what follows we introduce a language for apping constraint forulas (MCF) that can be used to express situations such as the ones entioned in the exaple. The graar of the language is defined as follows: MCF := (MCF MCF) (MCF MCF) MCF µ where µ is the only terinal sybol representing a apping constraint defined over soe apping table. We have already defined what it eans for a tuple to satisfy a apping constraint µ. We now offer a definition of what it eans for a tuple to satisfy a apping constraint forula defined over a set of apping constraints.

6 GDB id GDB: GDB: Gene Nae NF1 NF2 NGFB SwissProt id P21359 P35240 Protein Nae NF1 MERL GDB id SwissProt id GDB: P35240 v { GDB: } v { P35240 } 4(a) Relation in GDB 4(b) Relation in SwissProt 4(c) Mapping table fro GDB to SwissProt GDB id Gene Nae SwissProt id Protein Nae NF1 P21359 NF1 GDB: NF2 P35240 MERL GDB: NGFB P21359 NF1 Tuples that can be apped fro GDB to SwissProt Figure 4: Using apping tables as constraints GDB id SwissProt id P21359 Q9UMK3 (a) Mapping constraint µ 1 GDB id SwissProt id Q14930 Q9UMK3 (b) Mapping constraint µ 2 Figure 5: Constraints fro GDB to SwissProt Definition 9. Consider apping constraint forula φ over a set of attributes U and a U-tuple t. If φ = µ, then t φ iff t µ. If φ = φ 1, then t φ iff it is not true that t φ 1. If φ = φ 1 φ 2, then t φ iff t φ 1 and t φ 2. If φ = φ 1 φ 2, then t φ iff t φ 1 or t φ 2. Apart fro cobining apping constraints, apping constraint forulas augent our expressiveness, as the following exaple illustrates. Exaple 10. The identity between pairs of attributes A, B and C, D can be specified by eans of the apping constraint µ : AB CD with apping table containing only the apping (u, v, u, v), where u and v are variables. Assue that a user wants to specify a apping constraint where the values in A, B are identical to the values in C, D except for the set of tuples {(a i, b i) i [1, n]} do(a) do(b). How should apping constraint µ be odified to express this? By definition, through apping constraints, we are only able to exclude certain values, of soe colun, fro participating in any apping. Mapping constraint forulas allow us to do the sae thing for whole tuples. Going back to our exaple, for every i [1, n], let µ i : AB i CD be a apping constraint with apping table i containing only the apping (a i, b i, a i, b i). Then, the following apping constraint forula expresses the desired constraint: µ µ 1 µ n. A final note, regarding the introduction of negation. In the next section, we show that it allows us to have a unifor approach to solve the two probles considered in this paper: consistency and inference. 5. CONSISTENCY AND INFERENCE Given that considerable effort is put into creating apping tables we have found that curators and users alike often need the following capabilities. Infer new apping tables: A coon task is to find the set of all apping tables that are valid over a specific set of attributes U. To do this, we ust cobine the knowledge fro the apping tables available in a network of peers. As we show experientally in Section 7, a set of apping tables, viewed as constraints, can often be used to infer additional appings appings that are not explicitly represented in any peer. Deterine consistency of apping constraints: Curators edit, copy, or erge apping tables that coe fro a variety of sources and it can be a cubersoe task to ensure that the apping constraints of one table do not invalidate those expressed by another. As an exaple, note that the conjunction of apping constraints shown in Figure 2 is inconsistent under the CC-world seantics. Even for this sall exaple, inconsistency is not apparent and close inspection of the appings is necessary to discover this. We want to provide autoated techniques to help curators deterine whether a set of apping constraints is consistent. Inferred appings and knowledge of inconsistencies ay be directly useful to a curator. In addition, both of these echaniss play an iportant role in helping a curator understand and correctly specify the seantics of a set of apping constraints. We do not expect a curator to write down a set of coplex constraint forulas unaided. Rather, we expect that autoated inference and consistency checks will help a curator understand whether a default seantics (perhaps the disjunction of a set of constraints under the CC-world seantics) is appropriate for a specific set of doains.

7 5.1 Proble Definition Given a apping constraint forula (MCF) φ over a set of attributes U, we say that φ is consistent if there exists a nonepty relation r of U that satisfies φ. Furtherore, given a set of MCFs Σ {φ} over U, we say that Σ iplies φ, denoted by Σ = φ, if for every relation r of U, if r = Σ then r = φ. The consistency proble is the proble of deterining whether a given MCF is consistent, and the inference proble is the proble of verifying whether a set of MCFs iplies another MCF. The consistency and inference probles for MCFs are equivalent. To check whether a apping constraint forula φ is consistent we verify whether it is not true that φ iplies a apping table with no appings. To check whether a set of apping constraint forulas Σ iplies φ we verify whether φ ϕ Σ ϕ is not consistent. Thus, to analyze the coplexity of these probles, we can focus in the consistency proble. 5.2 Coplexity of Consistency Proble The following result shows that the consistency proble for apping constraint forulas cannot be solved efficiently. Theore 11. The consistency proble for apping constraint forulas is NP-coplete. The size of the input of this proble is the size of the forula to be checked for consistency. This size depends on the nuber of apping constraints in the forula and the nuber of attributes and appings in each apping constraint. Thus, the consistency proble is NP-coplete in the size of these three paraeters. A natural question at this point is what kind of restrictions can be iposed on the consistency proble to reduce its coplexity. A natural restriction is to consider only conjunctions of apping constraints. Given a set of apping constraints Σ = {µ 1,..., µ n}, we say that Σ is consistent if µ 1 µ n is consistent. The following theore shows that the coplexity does not change for conjunctions of apping constraints. Theore 12. The consistency proble for conjunctions of apping constraints is NP-coplete. To deal with this high coplexity we provide a solution that is based on the notion of path. A path θ is a list P 1, P 2,..., P n of peers such that peer P i stores apping tables between its data ites and the data ites of peer P i+1, where i ranges fro 1 to n 1. The idea behind our proposal is to check the consistency for conjunctions of apping constraints associated to paths. The details of this algorith are presented in the next section. In the reainder of this section, we describe what are the paraeters that deterine the coplexity of the consistency proble over paths, and we present results that show how these paraeters influence its coplexity. We use these results to understand under what assuptions the proble can be solved efficiently. A set of apping constraints Σ over a set of attributes U fors a path if there exists a collection U 1,..., U n of nonepty pairwise disjoint subsets of U such that for every X Y Σ, there exists i [1, n 1] such that X U i and Y U i+1. The coplexity of the consistency proble for conjunctions of apping constraints foring paths depends on the nuber of apping constraints in each peer, the nuber of appings in each apping constraint, the length of the paths and the arity of the apping constraints. Mapping constraints can contain thousands of appings and, therefore, it does not see reasonable to ipose restrictions on this nuber. Thus, we investigate only the assuptions that we have to ipose on the other paraeters in order to obtain an efficient algorith to solve the consistency proble. The following theore shows two conditions that ust be taken into account in order to construct an efficient algorith for the consistency proble. First, if the nuber of apping constraints per peer is not fixed, then the consistency proble cannot be solved efficiently. Second, if the length of the path and the arity of the apping constraints are not fixed, then the consistency proble cannot be solved efficiently. Theore 13. For each of the following conditions the consistency proble for conjunctions of apping constraints foring paths is NP-coplete. The length of the paths and the arity of the apping constraints are fixed. The nuber of apping constraints per peer is fixed. Given the above, we ake two assuptions to solve the consistency proble. First, we assue that the nuber of apping constraints per peer is sall. Second, we assue that the length of the paths is also sall. Paths of fixed length arise often in practice. For exaple, it is known that in Gnutella the paths of interest have a axiu size of 7. Under these assuptions, in the next section we present an efficient algorith for checking consistency and doing inference of apping constraints foring paths. 6. THE ALGORITHM Consider a path θ = P 1, P 2,..., P n of peers, and let U i be the set of attributes in peer P i, 1 i n. Let Σ denote the set of apping constraints over path θ. Two ore notions are necessary for our purposes. The first notion is that of an extension of a apping constraint. Specifically, given a apping constraint µ : X Y, we define the extension of µ, denoted as ext(µ), to be: ext(µ) = {ρ(t) t and ρ is a valuation over }. Furtherore, we say that µ is a cover of a set of apping constraints Σ over U if 1. Σ is consistent if and only if there exists t ext(µ) 2. For every apping constraint µ : X Y, Σ = µ if and only if ext(µ) ext(µ ). The algorith presented in the following paragraphs accepts as input a path θ, a set Σ of apping constraints over path θ, and two sets of attributes X U 1, Y U n

8 in peers P 1 and P n, respectively. Then, it coputes a apping constraint µ : X Y that is a cover of the set Σ of constraints. As such, the algorith can be used to solve both the inference and the consistency probles. For the inference proble, given Σ and a apping constraint µ : X Y we want to check whether Σ = µ. To solve this proble, it is sufficient to run the proposed algorith and check whether ext(µ) ext(µ ). The check, due to Condition 2 above, provides an answer to the inference proble. For the consistency proble, we run our algorith as before with the exception of sets X and Y which, in this case, are all the attributes in peers P 1 and P n respectively, i.e., X = U 1 and Y = U n. At the end of the algorith, we can check whether Condition 1 above is satisfied. If this is the case, set Σ is consistent. 6.1 Design Decisions Most algoriths for checking consistency or doing inferencing of constraints are centralized in that they assue that all constraints are locally available. However, in a peer-to-peer environent each peer stores locally only the constraints that involve itself and its iediate acquaintances. Still, for sall networks with a sall nuber of appings per constraint it ay be reasonable to send all the constraints to a single peer and perfor all the necessary coputation there. In a ore realistic scenario, though, there will be tens or even hundreds of peers. Although we do not expect each peer to have a large nuber of apping constraints, we do expect that each constraint ay have a large nuber of appings. In soe situations, the size of constraints ay be proportional to the size of stored relations. In the GDB peer, for exaple, there are approxiately 6.5 illion objects stored in the peer while there are 3.5 illion links to external sources. This latter nuber is the cuulative nuber of appings in the apping constraints of the GDB peer. Given the above, we propose an algorith that takes advantage of the distributed nature of the peer-to-peer architecture and distributes its coputation aong the peers in a given path. The algorith runs on top of a prototype peer-to-peer data anageent syste in which each peer anages a collection of data. Each peer autonoously chooses a logical design and physical organization for the data. Peers counicate using our own ipleentation of a Gnutella-like protocol, custoized to our specific needs. The ain algorith was developed with two ain goals in ind. First, it ust distribute the coputation and, thus, take advantage of the coputational resources of each peer. Second, it should deliver results in a streaing fashion. Streaing has proven valuable in any deployed peer-to-peer systes where results are delivered as soon as they becoe available. To present our algorith, we use a siple running exaple that involves a path θ = P 1, P 2, P 3, P 4 of four peers. The apping constraints Σ in this path are shown in Figure 6. Peer P 1 has attributes A i (i [1, 6]), P 2 has attributes B i (i [1, 6]), P 3 has attributes C i (i [1, 4]) and P 4 has attributes D i (i [3, 4]). Our ai is to copute the cover µ between the attributes of Peer P 1 µ 1 : A 1 1 B1 µ 2 : A 1, A 2 2 B1, B 2 µ 3 : A 3 3 B2, B 3 µ 4 : A 4 4 B4 µ 5 : A 5 5 B5 µ 6 : A 6 6 B6 Peer P 2 µ 7 : B 1, B 7 4 C1 µ 8 : B 8 3 C2 µ 9 : B 9 5 C3 Peer P 3 µ 10 : C 10 3 D 3 µ 11 : C 11 4 D 4 Figure 6: A path θ = P 1, P 2, P 3, P 4 of 4 peers. peer P 1 and those of peer P 4. Consider the following algorith for coputing the cover. First, peer P 1 sends to P 2 all the apping constraints between these two peers. Peer P 2 uses these constraints, along with its own constraints, to create a cover between peers P 1 and P 3. Then, peer P 2 forwards the cover to peer P 3. Peer P 3 repeats the process and creates a cover between peers P 1 and P 4. Peer P 3 sends the coputed cover back to peer P 1 and the coputation concludes. The above algorith, although it does distribute coputation, suffers fro two shortcoings. First, it perfors unnecessary coputation. Notice in Figure 6 that peer P 1 need not send to peer P 2 the actual appings corresponding to constraint µ 6. The reason for this is that the part of the coputation of the cover that involves attribute A 6 can be done locally in peer P 1. Furtherore, this coputation can be done independently of the coputation that involves the reaining constraints. The second shortcoing of the algorith is that it does not work in a streaing fashion since peer P 1 has to wait for the whole coputation to finish in order to retrieve the cover between itself and peer P 4. In the following paragraphs, we present an algorith which addresses these issues. 6.2 Partitions A key concept in our algorith is that of partitions. Consider peers P and P and let Σ P,P be the set of constraints between these two peers. A partition Π is a subset of Σ P,P and it is constructed as follows. Consider a graph G P,P = (V, E), where V contains one vertex for each constraint in Σ P,P and there is an edge between two constraints if their attributes overlap. Partition Π contains all the constraints of Σ P,P whose corresponding vertices belong to the sae connected coponent of G P,P. Applying the above procedure in the constraints between peer P 1 and peer P 2, shown in Figure 6, we get four partitions. The first partition Π 1 consists of the first three constraints, while each of the reaining three constraints constitutes a partition by itself. Figure 7 shows the partitions for the first three peers. The benefit gained fro partitioning the constraints in each peer is two-fold. First, while coputing the cover, we are able to consider the constraints of each partition in isolation. This reduces the coputational cost, with the exception where there is only one partition in Σ P,P, since we consider fewer constraints at

9 Peer P 1 Peer P 2 A 1 1 B1 Π 1 A 1, A 2 2 B1, B 2 A 3 3 B2, B 3 Π 2 A 4 4 B4 Π 3 A 5 5 B5 Π 4 A 6 6 B6 Π 5 B 1, B 7 4 C1 Π 6 B 8 3 C2 Π 7 B 9 5 C3 Peer P 3 Π 8 C 10 3 D 3 Π 9 C 11 4 D 4 A 1 1 Peer P 1 Peer P 2 B1 A 1, A 2 2 B1, B 2 A 3 3 B2, B 3 A 4 4 B4 B 1, B 7 4 C1 B 8 3 C2 A 5 5 B5 B 9 5 C3 A 6 6 B6 Figure 8: Inferred partitions over P 1 and P 2. Figure 7: Peer P 1, P 2 and P 3 partitions. a tie. Second, we can work on different partitions in parallel. This reduces the coputation tie and it also facilitates the delivery of results in a streaing fashion. 6.3 Description of the Algorith The algorith has two phases, naely, the inforation gathering phase and the coputation phase. The objective of the forer phase is to collect enough inforation fro the peers so as to reduce the coputation in the latter phase. At the sae tie, the inforation gathered is used to reduce network traffic and to help deterine how uch of the coputation can be executed in parallel. In what follows, we exaine each of the two phases in ore detail. To explain the algorith, we use the exaple of Figure The Inforation Gathering Phase The phase begins in peer P 1 with the coputation of partitions. Then, peer P 1 sends to peer P 2, for each of his partitions, the set of attributes in the partition. No appings are sent at this phase. Peer P 2 coputes its own partitions and, using the inforation for the partitions of peer P 1, it coputes a new set of inferred partitions. The inferred partitions possibly involve constraints of both peers and they are coputed as follows. We construct a partition graph G Π = (V Π, E Π), where V Π contains one vertex for each partition in the union of partitions of P 1 and P 2. There is an edge between two partitions if their attributes overlap. Each connected coponent of graph G Π is an inferred partition. Figure 8 shows the inferred partitions for the first two peers. We use inferred partitions to discover interdependencies, or lack thereof, between partitions. Then, in the coputation phase, we perfor parallel coputation and streaing of results along different inferred partitions. As an exaple, if the inforation gathering phase terinates here, then coputation of the cover between peers P 1 and P 3 can be perfored independently and in parallel along the three inferred partitions. The inforation gathering phase continues with peer P 2 sending to peer P 3 the sets of attributes corresponding to its inferred partitions. Peer P 2 sends only the inferred partitions involving soe its own constraints, i.e., only the top two in Figure 8. Peer P 3 coputes its own partitions, and using the inforation regarding the propagated inferred partitions fro peer P 2, it coputes a new set of inferred partitions. This concludes the inforation gathering phase The Coputation Phase For each of the inferred partitions in the previous phase, the set of constraints in the partition belongs to a set of peers that for a sub-path θ of path θ. Let Σ θ denote the set of constraints in an inferred partition over path θ. During this phase, we consider each inferred partition and we copute its cover. Specifically, given the path θ and the set Σ θ of constraints, we copute a cover of Σ θ over the attributes of the peers that are the endpoints of θ. In ore detail, assue an inferred partition over the whole path, i.e., θ = θ. The coputation of the cover starts, in general, at the last peer of the path. In the special case where θ = θ, the coputation starts at the penultiate peer. Thus, in our exaple we start at peer P 3. Peer P 3, using the local constraints of the current inferred partition, executes a local algorith that coputes a cover between peers P 3 and P 4. The cover only involves the attributes of the two peers that appear in the inferred partition. The appings belonging to the cover are streaed to peer P 2. Using the inforation fro the inferred partitions, P 2 deterines with which of its own partitions the incoing strea of appings should be associated. Then, it uses this inforation to generate a cover between itself and peer P 4. The appings fro this cover are, in turn, streaed to peer P 1. In the final step, peer P 1 uses the incoing strea of appings to generate a cover between its own attributes and those of peer P 4. Notice that there can be ore than one inferred partitions over the whole path θ. Each such partition produces a cover over non-overlapping subsets of attributes of peers P 1 and P 4. To copute the cover µ between all the attributes of the two peers, we first take the Cartesian product µ of the coputed covers. To finish the coputation of µ we need to take into account the inferred partitions that involve attributes of peer P 1, but are not over the whole path θ (e.g. the partition in Figure 8 involving attribute A 6). Specifically, in the fi-

10 GDB 1 MIM Locus 7 GDB GDB 2 SwissProt Locus 8 Unigene Hugo 3 GDB Locus 9 MIM Hugo 4 Locus Unigene 10 SwissProt Hugo 5 SwissProt SwissProt 11 MIM Hugo 6 MIM Figure 9: Biological apping tables nal step of the coputation of µ we take the Cartesian product of µ with the values of attribute A EXPERIMENTAL RESULTS To evaluate our algorith, we undertook two studies. The first was designed to understand whether (and to what extent) our solutions provide added value for counities that already use and exchange apping tables extensively. For this study, we used real apping tables fro several publicly available biological databases. The second study was designed to evaluate whether the characteristics of our algorith are appropriate and effective in a peer-to-peer environent. We are not aware of any other work designed to anage apping tables, so we were unable to do a coparison study. However, we do present soe of the perforance characteristics of our algoriths on both the biological data and on a B2B exaple. Our ipleentation uses geographically distributed achines with one peer per achine. Each peer consists of two odules. The first odule interacts with the peers storage anager to retrieve appings (fro disk) and perfors the coputation of the cover. Also, the odule is responsible for the creation of inferred partitions and the validation of incoing apping streas. The second odule ipleents the peer-to-peer networking protocol. In ters of eory requireents, we allow each peer to decide how uch cache to use. Peers that use a sall cache are able to store only a liited nuber of appings during the coputation of covers. Such peers generally produce ore network traffic since their available cache ay fill up quickly and thus they have to strea appings ore often. On the other hand, peers with a larger cache generate less traffic and do ore coputation between two consecutive network transissions. Biology Doain For this study we used actual apping tables retrieved fro six biological databases, GDB, MIM, and SwissProt (which have been discussed in our exaples), along with Hugo, Locus, and Unigene [3, 5, 9]. The table sizes range fro seven thousand to twentyeight thousand appings with an average of thirteen thousand appings. These apping tables are relatively siple tables (they are all binary and coprise a single partition). However, this setting is a very coon one where apping tables represent links between identifiers used in different data sources. Throughout the paper, we have stressed the iportance of inferring new appings especially when two Coputed New Tie Path Length Mappings Mappings (in secs) Figure 10: Inferred appings peers first becoe acquainted. This experient shows the benefit of inferring appings even between peers that are already acquainted. Figure 9 lists the apping tables that we found between the 6 biological databases. Consider two such peers, the Hugo database and the MIM database. There exists a apping table with eight thousand appings between the identifiers of these two databases. We assued two sources to be acquainted if one contained a apping table with attributes fro the other. Under this assuption, our peer-to-peer network contained seven different paths, of different sizes, between Hugo and MIM. For exaple, one such path is θ = Hugo, GDB, SwissProt, MIM. In our experients, we visited these paths in turn, in the order shown in the first colun of Figure 10, and coputed the cover between the endpoint peers for each of these paths. In the sae figure, we show both the nuber of appings we coputed between Hugo and MIM in each path, and the nuber of new appings that we coputed which are not in the initial Hugo to MIM apping table (and are not coputed by the previously visited paths). Notice that the length of the path is not correlated with the nuber of coputed appings. Rather, the nuber of coputed appings is ore a reflection of the knowledge ebedded in the different tables used. Overall, after visiting all seven paths we copute approxiately two thousand new appings which is a 25% increase with respect to our initial set of appings. The whole coputation along each path takes on average 19 seconds. Since we strea results along each path, the reported coputation tie here, and in the following paragraphs, is the tie it takes to receive the last coputed apping. The first apping always arrives with no perceptible delay. To understand how well our algorith perfors, we consider the scalability of the algorith for different path sizes and different apping table sizes. To understand the effect of path size in isolation, we use three paths of different lengths that all produced about the sae nuber of coputed appings. Figure 11 shows that the running tie of the algorith scales gracefully for each of the path lengths, as we change the average nuber of appings in each of the apping tables along the path. In all the above experients, we keep constant the size of the cache in each peer. Specifically, we allow each peer to store, on average, 64 appings. Our experients with different cache sizes indicate that