Distributed recipe mining to combine historic and social media knowledge

Transcription

1 Distributed recipe mining to combine historic and social media knowledge Sigurd Sippel Betreuung: Kai von Luck

2 Motivation develops during the time Social Media redundant adaptions Historic Books What is the main recipe? 2

3 Search on cluster map 3

4 Inner Cluster 4

5 KDD process [FPSS96] Knowledge discovery in database and data mining Defining goal Osama Fayyad SIGKDD KDD Conference Gregory Piatetsky-Shapiro Data selection Data cleaning Data reduction Mining function Mining algorithm Data mining Interpretation Performance 5

6 Defining goal domain and feature selection Defining goal predictive descriptive Trend Analysis Describing pattern Recommen dation Human Readable 6

7 Feature selection author date description Defining goal Data selection title ingredients units tools preperation glassware 7

8 Data cleaning Plausibility check Handling missing data Defining goal Data selection Data cleaning remove noise Implicit information Fuzzy Logic 8

9 Data reduction & projection Feature extraction Defining goal Keyword Normalization Data selection Data cleaning Database Data reduction Keywords Tags Supporting Data Projection {tag1, tag2,..., tagn} 9

10 Distance function Euklid distance [JMF99] Dynamic-Time-Warping [HLWG08] Speech Recogniction Levenshtein-Distance String recognition 10

11 Choosing mining function [FKPT07, S. 512] [LW06, S. 6] Defining goal Data selection Data cleaning Data reduction Mining function Classification Regression Clustering [JMF99, S. 277] 11

12 Chemical Model [MHC06, S. 80] 12

13 (Graph) Representation [MHC06, S. 80] Trees Bayesian network Neural network

14 Choosing mining algorithm Defining goal Bayes Data selection Data cleaning Support Vector Machines Klassifikation Neuronale Netze Data reduction Mining function Mining algorithm Gradient Boosting Machines 14

15 Data mining: Classification Defining goal Trainingset Testset Data selection Data cleaning Classification SVM Featurevektor -> {true, false} Data reduction Mining function Mining algorithm Data mining 15

16 Interpretation Defining goal Remove redudant patterns Data selection Data cleaning Translating for users Data reduction Mining function Mining algorithm Visualization Data mining Interpretation 16

17 Optimizing Training set Defining goal Data selection Data cleaning Gradient descent technique Hill climbing Data reduction Mining function Mining algorithm Data mining Interpretation Feature Complexity Performance 17

18 Large scale Twitter mining of drug related adverse events [BTY12] Socal Media Facebook Twitter Its hot weather, Tamoxifen is a nightmare Mining is drug related? (hot weather, Tamoxifen) => Nightmare 18

19 Classification Training User Training set Potential drug users Corresponding Keywords in Timeline Tweet Training set Medicine Ontology Unified Medicine Language system 19

20 Mining Process to find user [BTY12, S. 27] 20

21 Feature Extraction of Tweets Bag of words Action State of Drag Using Hash Tags Count Pronouns Count Semantic Types Reply Tags URLs Drug names Relevance Semantic Groups Relevance in Unified Medicine Language system 21

22 Support Vector Machine decision function g(x) = sign(f(x)) hyper plane <w,x> + b = 0 [LW06, S. 6] [BTY12, S. 30] 22

23 Recipe recommendation [TLA12] Preparation Dishes removed quantities removed temperature Ingredient combination key ingredients modification options nutritions Gradient Boosting Machine Classification [Fri01] 23

24 Co-occurrence network User Preferences Sweet or savory [TLA12, S. 301] pointwise mutual information 24

25 Substitution network based on modification options [TLA12, S. 302] 25

26 Prediction performance [TLA12, S. 307] 26

27 Roadmap Project 1 Distance Function Testing principles libsvm svmlight r julia Classification 27

28 Sources [BTY12] Bian, Jiang ; Topaloglu, Umit ; Yu, Fan: Towards Large-scale Twitter Mining for Drug- related Adverse Events. In: Proceedings of the 2012 International Workshop on Smart Health and Wellbeing. New York, NY, USA : ACM, 2012 (SHB 12). ISBN , [FKPT07] Fahrmeir, Ludwig ; Künstler, Rita ; Pigeot, Iris ; Tutz, Gerhard: Statistik. Springer- Verlag Berlin Heidelberg, 2007 (Springer-Lehrbuch). id=zinjp103ircc. ISBN [FPSS96] Fayyad, Usama ; Piatetsky-Shapiro, Gregory ; Smyth, Padhraic: The KDD Pro- cess for Extracting Useful Knowledge from Volumes of Data. In: Commun. ACM 39 (1996), November, Nr. 11, dx.doi.org/ / DOI / ISSN [Fri01] Friedman, Jerome H.: Greedy function approximation: a gradient boosting machine. In: Annals of Statistics (2001), S [JMF99] Jain, A. K. ; Murty, M. N. ; Flynn, P. J.: Data Clustering: A Review. In: ACM Com- put. Surv. 31 (1999), September, Nr. 3, DOI / ISSN [LW06] Lovell, Brian C. ; Walder, Christian J.: Support vector machines for business applica- tions. (2006) [MHC06] Maulik, U. ; Holder, L.B. ; Cook, D.J.: Advanced Methods for Knowledge Discovery from Complex Data. Springer, 2006 (Advanced Information and Knowledge Processing). books?id=ooosx1x2-_sc. ISBN [TLA12] Teng, Chun-Yuen ; Lin, Yu-Ru ; Adamic, Lada A.: Recipe Recommendation Using Ingredient Networks. In: Proceedings of the 3rd Annual ACM Web Science Conference. New York, NY, USA : ACM, 2012 (WebSci 12). ISBN ,