Open Source development for students.

Transcription

1 By Inaz Open Source development for students. Why should I work on free software?

2 Isabel Drost Nighttime: Co-Founder Apache Mahout. Organizer of Berlin Hadoop Get Together. Member ComDev PMC. Daytime: Software developer

3 Hello... HPI students.

4 Agenda The Apache Software Foundation. Apache Mahout. Reasons and ways to get started. Invitation.

5 What? Apache Software Foundation

6 Community over code.

7 Meritocracy.

8 Open communication.

9 NOT: Github, Google Code, sourceforge.

10 How? Behind the scenes.

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26 Community development GsoC Mentoring University relations

27

28

29

30 How? Open source collaboration tools are good for you.

31

32

33

34

35

36 Mahout A sub-project of Lucene

37

38

39

40 January 3, 2006 by Matt Callow

41 News aggregation September 10, 2008 by Alex Barth Today: Read news papers, Blogs, Twitter, RSS feed. Wish: Aggregate sources and track emerging topics.

42

43 Go to cinema March 22, 2008 by Crystian Cruz Today: IMDB, zitty, movie review pages, twitter, blogs, ask friends. Wish: Reviews, sentiment detection, recommendations.

44 Machine learning what's that?

45 Image by John Leech, from: The Comic History of Rome by Gilbert Abbott A Beckett. Bradbury, Evans & Co, London, 1850s Archimedes taking a Warm Bath

46 Archimedes model of nature

47 June 25, 2008 by chase-me

48

49 An SVM's model of nature

50 The challenge

51 Large amounts of data. Structured and unstructured data. Diverse tasks.

52 Mission Provide scalable data mining algorithms.

53 Commercially friendly license. Scalable to large amounts of data. Well documented. Healthy community. Targeted to developers.

54 What does Mahout have to offer.

55 Discover groups of items Group items by similarity. Examples: Group news articles by topic. Find developers with similar interests.

56

57

58 Discover groups of similar items Canopy. Dirichlet based. k-means. Others upcoming. Fuzzy k-means.

59 Discover groups of similar items

60 Identify dominant topics Given a dataset of texts, identify main topics. Algorithms: Parallel LDA Examples: Dominant topics in set of mails. Identify news message categories.

61 Assign items to defined categories. Given pre-defined categories, assign items to it. Examples: Spam mail classification. Discovery of images depicting humans.

62 By freezelight,

63

64

65 Assign items to defined categories. Naïve Bayes. Random forests. Complementary naïve bayes. Others upcoming.

66 Assign items to defined categories Examples based on standard datasets: 20 Newsgroups Wikipedia

67 Recommendation mining. Recommend items to users. Examples: Find books related to the book I am buying. Find movies I might like.

68 Recommending places

69 Recommending people

70 Recommendation mining. Integrated Taste. Mature Java library. Java-based, web service / HTTP bindings. Batch mode based on EC2 and Hadoop.

71 Frequent pattern mining Given groups of items, find commonly co-occurring items. Examples: In shopping carts find items bought together. In query logs find queries issued in one session.

72 By crypto,

73 By crypto, By libraryman,

74 By quinnanya, By crypto, By libraryman,

75 Upcoming More algorithms. Optimization of existing implementations. More examples. Release 0.3

76 Jumpstart your project with proven code. January 8, 2008 by dreizeh

77 Discuss ideas and problems online. November 16, 2005 [p

78 Become part of the community.

79 Interest in solving hard problems. Being part of lively community. Engineering best practices. Bug reports, patches, features. Documentation, code, examples. Image by: Patrick McEvoy

80 Isabel Drost Jan Lehnardt newthinking store Simon Willnauer June 7/8th: Berlin Buzzwords 2010 Store, Search, Scale Hadoop Solr HBase Lucene Sphinx Distributed computi CouchDB Business Intelligence Cloud Computing NoSQL Scalability MongoDB

81 Mar., 10th 2010: Hadoop* Get Together in Berlin Bob Schulze (ecircle/ Munich): Database and Table Design Tips with HBase Dragan Milosevic (zanox/ Berlin): Product Search and Reporting powered by Hadoop Chris Male (JTeam/ Amsterdam): Spatial Search * UIMA, Hbase, Lucene, Solr, katta, Mahout, CouchDB, pig, Hive, Cassandra, Cascading, JAQL,... talks welcome as well.

82 Interest in solving hard problems. Being part of lively community. Engineering best practices. Bug reports, patches, features. Documentation, code, examples. Image by: Patrick McEvoy

83 Why? Why should I waste my time with doing stuff for free?

84 Work on what you want... when you want.

85 Share and discuss with peers. Discuss ideas and problems online. November 16, 2005 [ph

86 Learn from the best.

87 Soft Skills.

88 Make work visible and re-usable.

89 Get started Turn users into developers.

90 GSoC

91 ComDev

92

93

94 Interest in solving hard problems. Being part of lively community. Engineering best practices. Bug reports, patches, features. Documentation, code, examples. Image by: Patrick McEvoy