Knowledge Harvesting for Business Intelligence

Transcription

1 Knowledge Harvesting for Business Intelligence Nesrine Ben Mustapha and Marie-Aude Aufaure 1

2 2

3 Context Intelligence in Business Intelligence (BI)? Intelligence: application of Information, skills, experiences and reasoning to solve business problems Information acquisition from wide varity of sources Harvesting knowledge for decision making 3

4 Talk Overview Context of BI: need of semantics Correlated dimensions related to semantic BI Web evolution Semantic evolution New trend of search paradigm Progress in ontology engineering Ontology-based knowledge harvesting for Business Intelligence 4

5 Context Data, Information and Knowledge in BI Process Data What data is to be collected and managed and in what context? 5

6 Context Data of BI Related Stock News COMPANIES in Same or Related INDUSTRY Business Data Competition COMPANIES in INDUSTRY with Competing PRODUCTS Technology Products Important to INDUSTRY or COMPANY Industry News Regulations Impacting INDUSTRY or Filed By COMPANY 6

7 Context Data, Information and Knowledge in BI Data What data is to be collected and managed and in what context? Information Knowledge Data warehousing, online analytical processing, data quality, data profiling, business rule analysis, and data mining 7

8 Context Data, Information and Knowledge in BI Data What data is to be collected and managed and in what context? Information Knowledge Business plans Explicit, embedded or tacit Knowledge? Data warehousing, online analytical processing, data quality, data profiling, business rule analysis, and data mining 8

9 Context Business Intelligence: closed or open environment Knowledge Exploiters Competition BI Environment Knowledge learners May be focused on certain areas and not able to integrate different streams of knowledge. 9 Knowledge Explorers Maintain a good balance between internal and external learning. Knowledge Innovators Combine internal and external learning

10 Motivating Use cases Business Case study in Knowledgeweb Network Main Goal: Transfer of ontology-based technologies from the field of academia to industry Service Industry: Recruitment B2C market place for tourism Media and comms Multimedia Content analysis News aggregation service Technology providers Product lifecycle management Health: Data warehousing in healthcare Hospital information systems 10

11 Motivating Use cases Recruitment Current practices Job request: Creation of electronic profiles (contact, qualifications, work experiences, skills, etc) for job seeker Searching in the database of vacancies according to the described profile Application is automatically forwarded to the firm Job offers: State job Center Creation of electronic description of the position Publishing the description to the database of vacancies Requesting a list of suitable candidates (only registrated jobseekers in that system) Automatic sending s to the selected candidates German federal Employment office Monster Jobpilot 11

12 Motivating Use cases Recruitment Challenges and new requirements Facilitate efficiently open job vacancies with qualified suitable candidates Automatic matching between job offers and job seekers Semantic Solution: Semantic support based on expressing relationships between job characteristics and candidate qualifications Representing Searching Sharing on the web Semantic matching Knowledge for decision making 12

13 Motivating Use cases Semantic solution and key business benefit Solution Job offers Semantic querying Machmaker Job seekers profiles Metadata crawler Knowledge Base Enriching Semantics (ontologies) Enriching 13

14 Motivating Use cases Business Case study in Knowledgeweb Network Main Goal: Transfer of ontology-based technologies from the field of academia to industry Service Industry: Recruitment B2C market place for tourism Media and comms Multimedia Content analysis News aggregation service Technology providers Product lifecycle management Health: Data warehousing in healthcare Hospital information systems 14

15 Motivating Use cases B2C market place for tourism Current systems Only information pages and no tourism offers No personalized services 15 19/07/2012

16 Motivating Use cases B2C market place for tourism Challenges and new requirements Offer on-line personalized tourism packages 16

17 Motivating Use cases B2C market place for tourism Challenges and new requirements Offer on-line personalized tourism packages Geo-localization Dynamic exploitation of content, service providers and personalized data Semantic Solution: Semantic data integration Natural language processing Personal data representation and exploitation Semantic web services Business partners: In France, Tourism market was evaluated 32 Billions Tourism content providers Tour operators Regional tourism councils euros which river tourism represents a turnover greater than 250 M euros. 17

18 Motivating Use cases Business Case study in Knowledgeweb Network Main Goal: Transfer of ontology-based technologies from the field of academia to industry Service Industry: Recruitment B2C market place for tourism Media Multimedia Content analysis News aggregation service Technology providers Product lifecycle management Health: Data warehousing in healthcare Hospital information systems 18 19/07/2012

19 Motivating Use cases Multimedia Content analysis Current systems Difficult to develop and maintain large multimedia databases Difficult to organize, find and distribute multimedia content Solutions Multimedia data can be annotated in terms of knowledge extracted from it Machine processable data models supporting Semantic search Navigation Reasoning functionalities 19

20 Motivating Use cases Multimedia Content analysis AceMedia Project 20

21 Motivating Use cases Multimedia Content analysis 21

22 Motivating Use cases Business Case study in Knowledgeweb Network Main Goal: Transfer of ontology-based technologies from the field of academia to industry Service Industry: Recruitment B2C market place for tourism Media Multimedia Content analysis News aggregation service Technology providers Product lifecycle management Health: Data warehousing in healthcare Hospital information systems 22

23 Motivating Use cases News aggregation service Business interest in following news in specific categories including economics, sciences and IT, or on specific companies. Manual fastidious task!! Knowledge Exploiters BI Environment Competition 23

24 Motivating Use cases News aggregation service Traditional systems: Feed services based on RSS or Atom ( 24

25 Motivating Use cases News aggregation service 25

26 Motivating Use cases News aggregation service Traditional systems: Feed services based on RSS or Atom ( very basic Model (e.g. title, author, link to full story) and not suitable for any intelligent searching or organizing Portals such as Google News Large body of information that can be processed 26

27 Motivating Use cases News aggregation service Current Semantic approaches The news aggregation service from neofonie GmbH 27

28 Motivating Use cases News aggregation service 28

29 Motivating Use cases News aggregation service Current Semantic approaches The news aggregation service from neofonie GmbH Manual creation by a source expert of a XSLT template for each news source Automatic processing of that news source through a thematic clustering algorithm (NLP) and classification with category mappings Extraction of semantics from the source documents 29

30 Motivating Use cases Data warehousing in healthcare Main Goal: Transfer of ontology-based technologies from the field of academia to industry Service Industry: Recruitment B2C market place for tourism Media Multimedia Content analysis News aggregation service Technology providers Product lifecycle management Health: Data warehousing in healthcare Hospital information systems 30 19/07/2012

31 Motivating Use cases Data warehousing in healthcare Large health insurance company use a cognos data warehousing solution to administrate its data Business data are stored in various PC and don t share the same data formats Results: Manual search over data sources Need of introducing common terminology for healthcare data Problem of updating data Semantic solution Common terminology of healthcare domain for ontology-based integration 31

32 Motivating Use cases Data warehousing in healthcare Data Mart ontology Builder Knowledge Explorers BI Tools Data Mart Data Mart Core Inference Engine 32

33 Motivating Use cases Data warehousing in healthcare Main Goal: Transfer of ontology-based technologies from the field of academia to industry Service Industry: Recruitment B2C market place for tourism Media Multimedia Content analysis News aggregation service Technology providers Product lifecycle management Health: Data warehousing in healthcare Hospital information systems 33 19/07/2012

34 Motivating Use cases Hospital information systems Hospitals have dispersed data sources: administrative information about patients, diagnoses and treatment history for each division Different type formats of stored data (databases, texts) Need of efficient access Semantic solution: Ontology engineering from unstructured data Data wrapper (from database to Ontology) Query mediation and semantic matching solution Middleware for database integration 34

35 Correlated dimensions related to semantic BI Semantic Information Retrieval Ontology Engineering Semantic Representation Web 35

36 Evolution of Web: Semantic Web Towards Open Linked Data 36

37 Evolution of the web Web 4.0 (2020) Web 1.0 (1990) Semantic Web (Web2.0)(2000) Weblogs Web 3.0 (2010) Weblogs Social networking Desktop PC-ERA (1980) 37 Groupware Databases File servers File Systems Directory portals Web sites Linked Open DATA YAGO-NAGA

38 Evolution of the web Semantic web Web 1.0 Web 2.0 Web Web

39 Evolution of the web From Wikipedia to YAGO, DBPEDIA, OLD 39

40 Evolution of the web Existing large knowledge bases Extraction of structured information (info Box) Dbpedia Wikipedia NLP-Based Extraction YAGO Linked Open Data Wordnet GENONAMe 40 Event, Time, and spatial data extraction YAGO 2

41 Evolution of the web Existing large knowledge bases 6,000,000 Ontologies should not be judged purely by the number of facts! This is just an informational overview. 2,000,000 30,000 60, , , KnowItAll SUMO WordNet OpenCyc Cyc Yago

42 Evolution of the web Existing large knowledge bases 42 18/07/2012

43 Evolution of the web Existing large knowledge bases Open Linked Data 43 18/07/2012

44 Correlated dimensions related to semantic BI Information Retrieval Web Ontology Engineering Semantic Representation 44

45 Evolution of Semantics: From Dictionaries To Ontologies 45

46 Evolution of Semantics Semantic Web (Web2.0)(2000) Web Web 3.0 (2010) Web 1.0 (1990) Linked Open Data (LOD) YAGO-NAGA 46

47 Evolution of Semantics: Levels of Knowledge Representation 47

48 Evolution of Semantics: Levels of Knowledge Representation Menu Object Person Topic Document Student Researcher Semantics Doctoral Student PhD Student F-Logic Ontology A Taxonomy 48

49 Evolution of Semantics: Levels of Knowledge Representation Menu Object Person Topic Document Student Researcher Semantics Doktoral Student PhD Student F-Logic Ontology synonym similar Thesaurus Graph with primitives, 2 fixed relationships (similar, synonym 49

50 Evolution of Semantics: Levels of Knowledge Representation Menu knows Object Person Topic Document writes described_in Student Researcher Semantics Doktoral Student PhD Student F-Logic Ontology Tel Affiliation synonym similar Topic Map 50

51 Evolution of Semantics: Levels of Knowledge Representation is_a Object is_a knows Person Topic Document writes described_in Student is_a Doktoral PhD PhD Student Tel Researcher Affiliation Affiliation PhD Student instance_of York Sure Semantics subtopicof F-Logic F-Logic similar Ontology Ontology Rules T described_in similar D T is_about D P writes D is_about T P knows T AIFB 51

52 Evolution of Semantics: Impact on business domain After 2005 Before Information Systems Engineering (Infological view) Enterprise Engineering (Ontological view on enterprise) Data Systems Engineering (Datalogical view) 52

53 Evolution of Semantics: Ontology definition Origin: A branch of philosophy that investigates and explains the nature and essential properties and relations of all beings, or the principles and causes of being. Computer Science (Artificial Intelligence): An ontology is a formal and explicit specification of a shared conceptualization (Gruber, 93) among a community of people (and agents) of a common area of interest. 53

54 Evolution of Semantics: Ontology: communication principle Concept evokes refers to Form Jaguar Stands for Referent [Odwen, Richards, 1923] 54

55 Evolution of Semantics: Ontology structure Concept : class of objects referred by a set of terms (synonyms) Customer Product Marketing strategy Person Invoice Taxonomic (hyponymy) relations: is-a relation Customer is a person Meronymy relations: part-of relation An invoice includes Marketing Strategies 55

56 Evolution of Semantics: Ontology structure Non taxonomic relations: associative properties between concepts A customer subscribes to products An invoice belongs to a customer An invoice has products Axioms: consist in defining the meaning of concepts; setting restrictions on attribute value verifying the validity of specified knowledge 56

57 Evolution of Semantics: Ontology structure Rules: used to infer additional statements 57 The discount for a customer buying a product is 7.5 percent if the customer is premium and the product is luxury. <Implies> <head> <Atom> <Rel>discount</Rel> <Var>customer</Var> <Var>product</Var> <Ind>7.5 percent</ind> </Atom> </head> <body> <And> <Atom> <Rel>premium</Rel> <Var>customer</Var> </Atom> <Atom> <Rel>luxury</Rel> <Var>product</Var> </Atom> </And> </body> </Implies>

58 Evolution of Semantics: Ontology Types Four typology in the literature: Formalization degree: formal, informal and semi-formal Granularity degree Level of completeness Domain of knowledge Top-level Ontology (generic ontology) Lexical Ontology: WorldNet Domain Ontology Ontology of tasks Application Ontology 58

59 Evolution of Semantics: Ontologies - Some Examples General purpose ontologies: WordNet, EuroWordNet Upper level ontologies: DOLCE Upper-Cyc Ontology, IEEE Standard Upper Ontology, Domain and application-specific ontologies: RDF Site Summary RSS, UMLS, AIFB Web Page Ontology, Web-KB Ontology, Dublin Core, Gene Ontology: Mesh Ontology:

60 Evolution of Semantics: Enterprise Ontology Enterprise Ontology: a formal and explicit specification of a shared conceptualization among a community of people (managers, developers, employees and users) 60

61 Evolution of Semantics: Enterprise Ontology: CoProE Ontology CoProE Ontology: a newsevents ontology [Lösch, 2009] and United Nations Standard Products and Services Code (UNSPSC) 1 classification of products and segments of industries. Towards e leadership: Higher profitability through innovative management and leadership systems project Lösch, U., Nikitina, N.: The newsevents Ontology An Ontology for Describing Business Events. In: 8th International Semantic Web Conference, 1st Workshop on Ontology Design Patterns, Washington DC, USA (2009) 1. < 61

62 Evolution of Semantics: Enterprise Ontology: CoProE Ontology 62

63 Evolution of Semantics: Marketing ontology 63

64 Correlated dimensions related to semantic BI Information Retrieval Web Ontology Engineering 64

65 New trends of search paradigm 65

66 Semantic web search classification in Web 2.0 Ontology search engines Ontology Meta-search : OntoSearch [Y. Zhang et al., 2004] Swangler [T. Fini et al., 2005 ] Semantic search engines Contextual semantic search QuizRDF [J. Davies et al., 2005], Corese [O. Corby], Infofox [B. Sigrist et al., 2003], SHOE [B. Aleman-Meza et al., 2003], DOSE [D. Bonino 2003], SERSE [V. Tamma, 2004] Crawler-based ontology search: Ontokhoj [C. Patel et al., 2003] Swoogle [T. Finin et al., 2005] Evolutive semantic search W3C Semantic Search [R. Guha et al., 2005] Semantic association discovery SemDis [C. Rocha et al., 2004] 66

67 Ontology Search Engines Ontology Meta-search : OntoSearch Swangler Use of conventional search engines Provide specific type of files (RSS, RDF, OWl) Search by name or options like filetype 67

68 Ontology Search Engines OntoSearch 68

69 Ontology Search Engines OntoSearch 69

70 Semantic web search classification in Web 2.0 Ontology search engines Ontology Meta-search : OntoSearch [Y. Zhang et al., 2004] Swangler [T. Fini et al., 2005 ] Semantic search engines Contextual semantic search QuizRDF [J. Davies et al., 2005], Corese [O. Corby], Infofox [B. Sigrist et al., 2003], SHOE [B. Aleman-Meza et al., 2003], DOSE [D. Bonino 2003], SERSE [V. Tamma, 2004] Crawler-based ontology search: Ontokhoj [C. Patel et al., 2003] Swoogle [T. Finin et al., 2005] Evolutive semantic search W3C Semantic Search [R. Guha et al., 2005] Semantic association discovery SemDis [C. Rocha et al., 2004] 70

71 Ontology Search Engines Crawler-based ontology search : Ontokhoj Swoogle Specific crawler for semantic web document 71

72 Ontology Search Engines 72

73 Semantic web search classification in Web 2.0 Ontology search engines Ontology Meta-search : OntoSearch [Y. Zhang et al., 2004] Swangler [T. Fini et al., 2005 ] Semantic search engines Contextual semantic search QuizRDF [J. Davies et al., 2005], Corese [O. Corby], Infofox [B. Sigrist et al., 2003], SHOE [B. Aleman-Meza et al., 2003], DOSE [D. Bonino 2003], SERSE [V. Tamma, 2004] Crawler-based ontology search: Ontokhoj [C. Patel et al., 2003] Swoogle [T. Finin et al., 2005] Evolutive semantic search W3C Semantic Search [R. Guha et al., 2005] Semantic association discovery SemDis [C. Rocha et al., 2004] 73

74 Semantic search Engine 1. Contextual semantic search Annotation service Indexation service Web Crawler Context detector and query annotator Ontologies Document filtering service Concept Matching service 74

75 Semantic search Engine 1. Contextual semantic search 75

76 Semantic search Engine 1. Contextual semantic search 76

77 Semantic web search classification in Web 2.0 Ontology search engines Ontology Meta-search : OntoSearch [Y. Zhang et al., 2004] Swangler [T. Fini et al., 2005 ] Semantic search engines Contextual semantic search QuizRDF [J. Davies et al., 2005], Corese [O. Corby], Infofox [B. Sigrist et al., 2003], SHOE [B. Aleman-Meza et al., 2003], DOSE [D. Bonino 2003], SERSE [V. Tamma, 2004] Crawler-based ontology search: Ontokhoj [C. Patel et al., 2003] Swoogle [T. Finin et al., 2005] Evolutive semantic search W3C Semantic Search [R. Guha et al., 2005] Semantic association discovery SemDis [C. Rocha et al., 2004] 77

78 Semantic search Engine 2. Evolutive semantic search 78

79 Semantic web search classification in Web 2.0 Ontology search engines Ontology Meta-search : OntoSearch [Y. Zhang et al., 2004] Swangler [T. Fini et al., 2005 ] Semantic search engines Contextual semantic search QuizRDF [J. Davies et al., 2005], Corese [O. Corby], Infofox [B. Sigrist et al., 2003], SHOE [B. Aleman-Meza et al., 2003], DOSE [D. Bonino 2003], SERSE [V. Tamma, 2004] Crawler-based ontology search: Ontokhoj [C. Patel et al., 2003] Swoogle [T. Finin et al., 2005] Evolutive semantic search W3C Semantic Search [R. Guha et al., 2005] Semantic association discovery SemDis [C. Rocha et al., 2004] 79

80 Semantic search Engine 3. Association Discovery 80

81 Enterprise Search Intelligent Content = What You Asked for + What you need to know! Content which does contain the words the user asked for Content which does not contain the words the user asked for, but is about what he asked for. Content the user did not think to ask for, but which he needs to know. Extractor Agents + Value-added Metadata + Semantic Associations 81

82 Correlated dimensions related to semantic BI Ontology Engineering Web Semantic Search Syntactic Search Document retrieval Bag of words 82

83 Progress in Ontology Engineering Research 83

84 Ontology engineering vs. learning Ontology engineering Expert-driven, small-scale Knowledge and concept identification Preliminary informal representation Formalization (RDF, OWL, etc) Evaluation and Maintenance Ontology learning Data driven large scale Source selection Data exploration Concept and relation learning Evaluation and Updating 84

85 Ontology Learning 85

86 Ontology Learning History 86

87 Manual Ontology engineering Manual ontology engineering: Ontology building «from scratch» [Fernandez et al., 99] OTK Methodology: Knowledge Meta Process [Y. Sure and R. Studer, 2002] 87

88 Manual Ontology engineering Manual ontology engineering: Ontology building «from scratch» [Fernandez et al., 99] OTK Methodology: Knowledge Meta Process [Y. Sure and R. Studer, 2002] Methontology [Fernández-Lopez et al., 1997] But. Manual engineering of ontologies is a very time consuming task! => Need of automatic way to reduce the burden of engineering! 88

89 Domain Ontology Learning Knowledge Discovery ONTOLOGY LEARNING Ontology Engineering Ontology learning is defined as an approach of ontology building from knowledge sources using a set of machine learning techniques and knowledge acquisition methods. 89

90 DomainOntology Learning Main steps x, y( sufferfrom ( x, y) ill ( x)) cure(dom:doctor,range:disease) is_a(doctor,person) DISEASE:=<Int,Ext,Lex> Axioms & Rules Relations Taxonomy Concepts {disease, illness, Krankheit} disease, illness, hospital (Multilingual) Synonyms Terms Introduced in: Philipp Cimiano, PhD Thesis University of Karlsruhe, forthcoming 90

91 Domain Ontology Learning 91

92 Domain Ontology learning from Texts Domain MDX 92

93 Domain Ontology learning from Texts Linguistic analysis Shallow linguistic parsing Linguistic patterns Consistency grammar Dependency grammar Domain Term extraction MDX MDX language Multidimensional Expressions (MDX) Is based on the position of the words in a sentence and how can be grouped => Lexico-syntactic Patterns Provide binary grammatical links between words in a sentence. => Dependency relationships 93

94 Domain Ontology learning from Texts Term Extraction Text: Multidimensional Expressions (MDX) is a query language for OLAP databases Linguistic analysis Consistency grammar Dependency grammar 94

95 Domain Ontology learning from Texts Linguistic analysis Shallow linguistic parsing & Linguistic patterns Consistency grammar Dependency grammar Domain Term extraction MDX MDX language Multidimensional Expressions (MDX) Statistical analysis Co-occurrence analysis Selection measures (IR) Methods that determine a score representing the relationship between two terms and retain those with scores greater than a given threshold 95

96 Domain Ontology learning from Texts Term Extraction Statistical analysis TF-IDF [Robertson 1976] Entropy [Brini 2005] Relevance to the domain Term_frequency(t,d) * Inverse_term_frequency(t, D) Dn P t log (P(t)) t Di Consensus to the domain Pointwise Mutual Information (PMI) 96

97 Domain Ontology learning from Texts Term Extraction Statistical analysis TF-IDF [Robertson 1976] Entropy [Brini 2005] Relevance to the domain [Verladi, 2001 Consensus to the domain Pointwise Mutual Information (PMI) Term t MDX P(t, domain RD(t, domain i )= i ) i=1..n p(t, domain i ) P(t, domain i )= freq(t in domain i ) i=1..n p(t, domain i ) Economy Business analysis Finance Accounting 97

98 Domain Ontology learning from Texts Term Extraction Statistical analysis TF-IDF [Robertson 1976] Entropy [Brini 2005] Relevance to the domain [Verladi, 2001] Consensus to the domain Pointwise Mutual Information (PMI) Analysis of the distribution of the term t on a set of documents dj belonging to the domain Di. CD(t, D i )= P t, dj log 2( P(t, D i )= freq(t in dj) i=1..n p(t in dj) 1 ) P(t,dj) 98

99 Domain Ontology learning from Texts Term Extraction Statistical analysis TF-IDF [Robertson 1976] Entropy [Brini 2005] Probability is estimated to the co-occurrence of the term t with the domain concepts concept Relevance to the domain [Verladi, 2001] Consensus to the domain Pointwise Mutual Information (PMI) PMI(t, concept)=log( p(t/concept) p(concept) ) 99

100 Domain Ontology learning from Texts Concept Discovery Hybrid methods Lexical resources: Wordnet Domain Term Extraction Concept Learning MDX MDX language, MDX, Multidimensional Expressions (MDX) Query language standard 100

101 Domain Ontology learning from Texts Concept Discovery Hybrid methods Lexical resources: WordNet Domain Term Extraction Concept Learning MDX MDX language, MDX, Multidimensional Expressions (MDX) Query language standard Topic signature construction (Aguire, 2000) Classification of concepts based on contextual properties of words Each The concept meaning is represented of word is strongly by a vector correlated of co-occurring with the contexts words and in which their it frequency appears. 101

102 Domain Ontology learning from Texts Taxonomy Construction Linguistic analysis Domain Term Extraction MDX MDX language, MDX, Multidimensional Expressions (MDX) Concept Learning Query language standard Statistical analysis Taxonomy construction 102

103 Domain Ontology learning from Texts Taxonomy Construction Linguistic analysis Lexico-syntactic patterns (Hearst, 1998) NP such as NP, NP,... and NP a multidimensional database query language such as MDX. Such NP as NP, NP,... or NP Such query language as MDX. NP, NP,... and other NP screen real estate to financial charts, indices and other news graphics. 103 NP, especially NP, NP,... and NP Accounting, especially financial accounting gives mainly past information in that the events are recorded.

104 Domain Ontology learning from Texts Taxonomy Construction Linguistic analysis Lexico-syntactic patterns (Hearst, 1998) Statistical analysis Hierarchical grouping of concepts Grouping based on probability-based measures 104

105 Domain Ontology learning from Texts Domain Term Extraction MDX MDX language, MDX, Multidimensional Expressions (MDX) Concept Learning Query language standard Taxonomy construction Query language Multidimensional query language standard 105

106 Domain Ontology learning from Texts Relation Discovery Domain Term Extraction MDX MDX language, MDX, Multidimensional Expressions (MDX) Concept Learning Taxonomy construction Query language standard Query language Multidimensional query language standard Relation discovery 106

107 Domain Ontology learning from Texts Relation Discovery Linguistic analysis Syntactic frames [Faure and Nedellec, 1998] <To be used > (<subject: MDX>) (<for: OLAP database analysis>) <To be used > (<subject: MDX>) (< for : OLAP database querying >); Conceptual clustering Statistical analysis Multidimensional query language Is used for OLAP Database management 107

108 Domain Ontology learning from Texts Relation Discovery Linguistic analysis Syntactic frames [Faure and Nedellec, 1998] Statistical analysis w1 w2 w3 Wn-1 wn w1 w2 w3 Wn-1 wn Word space DOODLE II[Sugiura et al., 2003] Association rules Co-occurrence analysis Verb arguments analysis 108

109 Domain Ontology learning from Texts Domain MDX Term Extraction MDX language, MDX, Multidimensional Expressions (MDX) Manual, Expert-based Gold standards WordNet E V A L U A T I O N Concept Learning Taxonomy construction Relation discovery Domain Ontology Query language standard Query language Multidimensional query language standard Query language is used for OLAP Database querying 109

110 Domain Ontology Learning The Web as a learning source. 110

111 Domain Ontology Learning From web pages (texts) Collection of web pages related to a specific domain Application of the same techniques as textual approaches Web: Largest electronic textual corpora Shortcomings NL resources, visual-oriented representation Noise ( commercial bias) Lack of semantic structure Unreliable source Advantages Size and heterogeneity Public massive IR tools 111

112 Domain Ontology Learning 112

113 Ontology Learning from web From web page structure Nouns phrases in the headings, tables or lists of a document can be used to deduce relations between concepts and instances Supervised approaches Difficulty of the interpretation of html structure 113

114 Domain Ontology Learning 114

115 Ontology Learning from web Aggregation of on-line ontologies 115

116 Domain Ontology Learning 116

117 Ontology Learning from web Wikipedia mining YAGO Ontology Wu and Weld,

118 Ontology Learning from web Wikipedia mining terminological ontology not scalable Extraction of schema from infoboxes (Wu and Weld, 2008) 118

119 Domain Ontology Learning 119

120 Ontology Learning from web Ontology learning by Googling 120

121 Unsupervised Web-based semantic relatedness for ontology learning The correlation function C k (a, b ) [Turney, 2001] = p p k ab a p b Symmetric Conditional Probability (SCP) [ferreira da Silva and lopes, 1999] 2 p( a, b) SCP (a,b ) = C 2 (a, b )= p( a) p( b) The point wise mutual information (PMI) [church and al., 1991] p( ab) PMI(a,b ) = log 2 p( a) p( b) 121

122 Unsupervised Web-based semantic relatedness for ontology learning k The correlation function C k (a, b ) [Turney, 2001] = p ab p a * p b If the words a and b are independent p(a and b) = p(a) * p(b) Else p(a and b) > p(a) * p(b) For concept selection: c( problem, choice ) p( problem, choice ) p( choice ) (2) Web = scalable Corpora hit problem AND choice P (problem, choice) = Total _ pages _ Web hits ( problem AND choice ) hits ( choice ) c( problem, choice ) (3) 122

123 Ontology Learning from web Ontology learning by Googling Web-based semantic similarity for ontology learning Hyponym learning score (candidate) = hit ( hearst _ pattern ( keyword, candidate )) hit ( condidate ) 123

124 Ontology Learning from web Ontology learning by Googling Non taxonomic relation learning Extraction verb phrases from snippets score (verb_phrase, concept) = Retrieval and selection of related concepts score (candidate, concept)= hit( verb_phrase concept ) hit(verb_phrase) hit (" concept " AND" candidate ") hit ( concept ) 124

125 Unsupervised Web-based semantic relatedness for ontology learning : problems c( concept, candidate) hits( concept AND candidate) hits( candidate) Language ambiguity Polysemy hits (word) = Somme (hits (sens 1 ), hits (sens 2 ),., hits(sens n )) Need to contextualize these measures (Introducing context of words) 125

126 Unsupervised Web-based semantic relatedness for ontology learning : problems The use of statistical assessment is not sufficient to identify that a given concept candidate or relation is enough relevant to add it to the ontology Using linguistic patterns to discover relations between concepts Score pattern (candidate) = hits(" Max( Patterns i 1.. n patterni (" concept","candidate") ) hits("candidate") 126

127 Synthesis 127

128 Ontology Leaning for Business Intelligence 128

129 Ontology Learning for Business Intelligence Engineering of Semantic Business Intelligence Unified Knowledge representation Semantic integration of multiple sources Semantic search over heterogeneous data Semantic Technologies NLP techniques for question-answering Semantic annotations Ontology-based Information Extraction 129

130 Ontology Leaning for Business Intelligence Semantic Search 130

131 Main objectives PARLANCE: Probabilistic Adaptive Real-Time Learning And Natural Conversational Engine Goal: design and build mobile applications that approach human performance in conversational interaction, specifically in terms of the interactional skills needed to do so All of these skills will be learned or adapted using real data 131

132 Main objectives Develop dialogue systems in 3 languages that are: Incremental: dialogue act units Personalized: adapt to different users with different goals in different contexts Dynamic and evolving: possible to incorporate new concepts Interactive hyper-local: take into account the location and surroundings of the user 132

133 Our focus Modular incremental ontology enrichment Using existing knowledge bases Ontology learning from Web Query-driven knowledge base enrichment Dynamic, evolving rich user models Static information Contextual information Social information collaborative filtering 133

134 Linked Data Extraction SW Concept Instance A = V A = V A = V Web Search Web Ontology Construction Ontology schema Basic Population Ontology Incremental Population Entities Discovery Pattern Learning Instance A Instance = V A = A Instance V = V A = A V = A V = V A = A V = V A = V Web Web 134

135 Web Search, Web Corpus Barack Obama + United States 135

136 Ontology Leaning for Business Intelligence Ontology-based Information Extraction 136

137 Ontologies for Collecting and Analyzing Business Intelligence DAVID (Data Analysis and Visualization aid) system Kakkonen, T., Mufti, T.: Developing and Applying a Company, Product and Business Event Ontology for T Mining. Proceedings of the 11th International Conference on Knowledge Management and Knowledge Technologies, Graz, Austria,

138 Ontology Learning for Business Intelligence Ontologies for Collecting and Analyzing Business Intelligence 138

139 Ontology Leaning for Business Intelligence Annotation of Dashboard 139

140 Annotating BI Visualization Dashboards: Needs & Challenges. Micheline Elias and Anastasia Bezerianos ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2012.

141 Ontological Need: Enriching Ontologies From annotations?? 141

142 Conclusion 142

143 Conclusion Bridging the gap between academic research approaches and real business use cases Scalability of knowledge Modular ontologies Modular reasoning Open Linked data and Privacy issues Toward Open Linked Business Processes?! 143

144 Questions?? 144