Apache Mahout. Scaling Machine Learning. Presented by: Isabel Drost

Transcription

1 Apache Mahout Scaling Machine Learning Presented by: Isabel Drost

2 Agenda Motivation. Machine learning? Introducing Mahout. How can you help?

3 Some motivation.

4 January 3, 2006 by Matt Callow

5 Follow news stories September 10, 2008 by Alex Barth Search through papers. Automatic topic tracker.

6 March 7, 2008 by extranoise

7 Movie recommendation March 22, 2008 by Crystian Cruz IMDB + movie reviews. Aggregate reviews from IMDB, twitter,...

8 Lots and lots of data. Structured and unstructured.

9 Mission Provide scalable data mining algorithms.

10 Machine Learning?

11

12 Archimedes generates model: Density of Object =. Density of Fluid Weight Weight Apparent immersed weight

13 June 25, 2008 by chase-me

14 March 28, 2007 by dullhunk

15 Machine learning generates model

16 Machine learning pipeline Gather data. (and meta data). Identify characteristics. Chose right algorithm. Keep model in sync when nature changes. Train on the gathered data. Tune parameters of your algorithm.

17 January 8, 2008 by Pink Sherbet Photography

18 Machine learning pipeline Gather data. (and meta data). Identify characteristics. Chose right algorithm. Keep model in sync when nature changes. Train on the gathered data. Tune parameters of your algorithm.

19 E-Bay Auction status? Phishing Spam? Different topic Requested password? password

20 Apache One of your mails:... Hadoop London Lucene London

21 Machine learning pipeline Gather data. (and meta data). Identify characteristics. Chose right algorithm. Keep model in sync when nature changes. Train on the gathered data. Tune parameters of your algorithm.

22

23 Machine learning pipeline Gather data. (and meta data). Identify characteristics. Chose right algorithm. Keep model in sync when nature changes. Train on the gathered data. Tune parameters of your algorithm.

24 Parameter tuning Penalty for mistakes. Kernel type for data transformation. Tune kernel parameters.

25 Machine learning pipeline Gather data. (and meta data). Identify characteristics. Chose right algorithm. Keep model in sync when nature changes. Train on the gathered data. Tune parameters of your algorithm.

26 Training Build model from data.

27 Machine learning pipeline Gather data. (and meta data). Identify characteristics. Chose right algorithm. Keep model in sync when nature changes. Train on the gathered data. Tune parameters of your algorithm.

28 Nature changes? Spammers adapt to spam filters. Users write mails in different styles. Expand to new languages....

29 Machine learning pipeline Gather data. (and meta data). Identify characteristics. Chose right algorithm. Keep model in sync when nature changes. Train on the gathered data. Tune parameters of your algorithm.

30 Introducing Mahout

31 Classification Categorize data. Examples: Identify spam mails. Classify movies as Action, Comedy...

32 Classification Naive bayes. Complementary naive bayes. Winnow/Perceptron Others upcoming.

33 Discovering groups of data Group data by similarity. Examples: News articles by topic. Developers by favorite modules.

34 Discovering groups of data Canopy. PLSI. K-Means. Others upcoming. Dirichlet based.

35 Recommendation mining Recommend items. Examples: Find books a user my like. Identify movies a user likes.

36 Upcoming More algorithms. More examples.

37 What Mahout can do for you Why should I participate?

38 Jumpstart your project with proven code. January 8, 2008 by dreizehn28

39 Discuss with researchers and engineers. November 16, 2005 [phil h]

40 Become a community member.

41 s Thank you to all those making this possible. October 22, 2008 by e_calamar

42 July 9, 2006 by trackrecord We need You: Enthusiasm. Mathematical knowledge. Proficiency in Hadoop. Interest in understanding data.

43 Some advertising Berlin - June* at 5p.m. newthinking store Berlin Tucholskystr. 48 Hadoop** User/Developer Meeting Germany * Exact date is set by speaker that is you! ** Lucene, Tika, Solr, UIMA, Mahout, katta,... people welcome.