Unsupervised Learning of Link Discovery Configuration

Transcription

1 Unsupervised Learning of Link Discovery Configuration Andriy Nikolov, Mathieu d Aquin, and Enrico Motta Knowledge Media Institute, The Open University, UK {a.nikolov, m.daquin, e.motta}@open.ac.uk

2 Link discovery problem Goal: Which individuals refer to the same real-world entity? Challenges Schema heterogeneity Different value representation formats Incorrect/missing data values Homonymy/synonymy dbpedia:heraklion fbase:heraklion dbpedia:s.s._heraklion geonames:261745/

3 Instance matching Decision rule avg max Jaro Levenshtein Monge-Elkan Jaccard Many parameters to set Expert knowledge often required Jaro-Winkler (personal names) Monge-Elkan (document titles) No straightforward guidance (rdfs:label;dc:title) (foaf:name;dc:title) (dbprop:residence;bnb:birthyear)

4 How to avoid manual configuration? Learn from existing linked data repositories available on the Web [Hu et al., 2011] Only for some datasets/ontologies Utilising Web of data Machine learning Can we do it without labelled training data? Active learning Adapts to the domain by picking appropriate parameters for particular datasets Training data required Reduce the amount of training data [Sarawagi et al., 2002, Ngonga- Ngomo et al., 2011]

5 What is a good decision rule? Minimises defects no spurious mappings Maximises output discovered actual mappings

6 Evaluation measures

7 Reference datasets Some datasets are considered as reference ones: - Reliable - Comprehensive Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch.

8 Assumptions

9 Fitness criterion George W. Bush (1946 George H. W. Bush (1924 Bill Clinton (1946 Bush, G. W. (1946 Bush, G. H. W. (1924 Clinton, W. J. (1946 George W. Bush (1946 George H. W. Bush (1924 Bill Clinton (1946 Bush, G. W. (1946 Bush, G. H. W. (1924 Clinton, W. J. (1946 George W. Bush (1946 George H. W. Bush (1924 Bill Clinton (1946 Bush, G. W. (1946 Bush, G. H. W. (1924 Clinton, W. J. (1946

10 Fitness function: unsupervised case

11 Approach genetic algorithm Initialise a pool of decision rules with random parameters Apply these rules to the datasets Calculate fitness of each decision rule by evaluating its results Use a genetic algorithm to improve the pool of decision rules Elitist selection Crossover Mutation Candidate solution pool Fitness calculation Decision rule

12 Encoding decision rules Gene Genotype

13 Mutation

14 Crossover

15 Implementation - Stop-word filtering - Indexing Determining & applying the optimal decision rule Filtering resulting mappings - Enforcing 1-to-1 rule

16 Fitness function behaviour 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0, Avg. pseudo-f Avg. F1 Best pseudo-f Best F1 In general, N=20 was sufficient for convergence

17 Genetic algorithm parameters 1,2 1 0,8 0,6 0,4 0,2 - Varying population size 0 - Comparing with ideal case (actual F1-measure) F1-fitness (MC) Pseudo-F-fitness (MC) F1-fitness (BA) Pseudo-F-fitness (BA) - Varying crossover and mutation rate didn t result in significant differences in performance

18 OAEI 2010: Person/Restaurant Comparing actual F1-measure (average after 5 runs) Benchmark adapted from the database record linkage 1,2 1 0,8 0,6 0,4 0,2 0 Person1 Person2 Restaurant (OAEI) Restaurant (fixed) KnoFuss+GA ObjectCoref ASMOV CODI LN2R RiMOM FBEM

19 OAEI 2011: New York Times data Benchmark taken from LOD: links between NYT data, DBpedia, Freebase, and Geonames 1,2 1 0,8 0,6 0,4 0,2 0 KnoFuss+GA AgreementMaker SERIMI Zhishi.links

20 Reducing computations Link discovery has to be performed many times: Having population size 100 and 20 generations 2000 decision rule applications over both datasets Solution: Take a random sample of candidate instances (after blocking) Learn a decision rule based on the sample Apply the rule once to the complete datasets

21 Reducing computations: sampling 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0, F1-fitness Unsupervised (sample) Unsupervised (complete)

22 Alternative indirect evidence

23 Fitness function comparison 1 0,8 0,6 0,4 0,2 Pseudo-F-fitness NG-fitness 0 Musicians Book authors OAEI (Geonames) OAEI (Freebase) Neighbourhood growth can also perform well but is less robust

24 Summary Can we do without labelled training data? Yes, we can! well, in certain conditions Different types of indirect evidence can be exploited Genetic algorithms provide a suitable tool to learn a decision rule

25 On-going & future work Improving the performance Is it possible to weaken the assumptions? More complex forms of decision rules Combining different types of evidence Scalability Efficient blocking mechanisms Parallelisation of computations

26 Questions? Thanks for your attention! More info & download link:

27 Experiments: default parameters

28 Issue: what does pseudo-recall mean?

29 Instance matching rdfs:label Jung, C. G. (Carl Gustav) bnb:jungcg(carlgustav) dc:creator bio:event bio:event bnb: bnb: /birth bnb: /death dc:title bio:date bio:date Visions Carl Jung foaf:name dbprop:knownfor Analytical Psychology dbprop:birthdate dbpedia:carl_jung dbprop:residence Switzerland dbpedia:sigmund_freud dbprop:influenced dbprop:placeofbirth Kesswil

30 Experiments: OAEI datasets OAEI 2010 (Person/Restaurant) Dataset KnoFuss+GA ObjectCoref ASMOV CODI LN2R RiMOM FBEM Person N/A Person Restaurant (OAEI) N/A Restaurant (fixed) N/A N/A N/A N/A 0.96 OAEI 2011 (New York Times data) Dataset KnoFuss+GA AgreementMaker SERIMI Zhishi.links DBPedia (locations) DBPedia (organisations) DBPedia (people) Freebase (locations) Freebase (organisations) Freebase (people) Geonames (locations) Average

31 Parameters: population size 1,2 1 0,8 0,6 0,4 0,2 F1-fitness (MC) Pseudo-F-fitness (MC) F1-fitness (BA) Pseudo-F-fitness (BA) Varying crossover and mutation rate didn t result in significant differences in performance

32 Instance matching rdfs:label Jung, C. G. (Carl Gustav) bnb:jungcg(carlgustav) dc:creator bio:event bio:event bnb: bnb: /birth bnb: /death dc:title bio:date bio:date Visions Comparable pairs of attributes Carl Jung foaf:name dbprop:knownfor Analytical Psychology dbprop:birthdate dbpedia:carl_jung dbprop:residence Switzerland dbpedia:sigmund_freud dbprop:influenced dbprop:placeofbirth Kesswil

33 Instance matching bnb:jungcg(carlgustav) Appropriate similarity measures rdfs:label Jung, C. G. (Carl Gustav) token set similarity Carl Jung foaf:name bio:event/bio:date 1875 date inclusion dbprop:birthdate dbpedia:carl_jung

34 Instance matching rdfs:label bnb:jungcg(carlgustav) average() 0.85 Aggregation function, weights, threshold bio:event/bio:date Jung, C. G. (Carl Gustav) token * token set set similarity 0.6 date * date inclusion Carl Jung foaf:name dbprop:birthdate dbpedia:carl_jung

35 Instance matching rdfs:label average() 0.85 bio:event/bio:date Jung, C. G. (Carl Gustav) token * token set set similarity 0.6 substring * date inclusion Carl Jung foaf:name dbprop:birthdate average( 0.4*token-set-sim( rdfs:label : foaf:name); 0.6*substr-sim(bio:event/bio:date : dbprop:birthdate) ) 0.85 Decision rule