MOA: {M}assive {O}nline {A}nalysis.

Transcription

1 MOA: {M}assive {O}nline {A}nalysis. Albert Bifet Hamilton, New Zealand August 2010, Eindhoven

2 PhD Thesis Adaptive Learning and Mining for Data Streams and Frequent Patterns Coadvisors: Ricard Gavaldà and José L. Balcázar LARCA Laboratory for Relational Algorithmics, Complexity and Learning Universitat Politècnica de Catalunya 2 / 25

3 Adaptive Stream Mining Mining Evolving Streams 1 Framework 2 ADWIN 3 Hoeffding Adaptive Tree 4 ADWIN Bagging 5 Adaptive-Size Hoeffding Tree Bagging Mining XML Tree Streams 6 Closure Operator on Trees 7 Incremental Method 8 Sliding Window Method 9 Adaptive Method 10 XML Classification 3 / 25

4 Adaptive Stream Mining Mining Evolving Streams 1 Framework 2 ADWIN 3 Hoeffding Adaptive Tree 4 ADWIN Bagging 5 Adaptive-Size Hoeffding Tree Bagging Mining XML Tree Streams 6 Closure Operator on Trees 7 Incremental Method 8 Sliding Window Method 9 Adaptive Method 10 XML Classification 3 / 25

5 Mining Massive Data 2007 Digital Universe: 281 exabytes (billion gigabytes) The amount of information created exceeded available storage for the first time Web million registered users 600 million search queries per day 3 billion requests a day via its API. 4 / 25

6 Green Computing Green Computing Study and practice of using computing resources efficiently. Algorithmic Efficiency A main approach of Green Computing Data Streams Fast methods without storing all dataset in memory 5 / 25

7 What is MOA? {M}assive {O}nline {A}nalysis is a framework for online learning from data streams. It is closely related to WEKA It includes a collection of offline and online methods as well as tools for evaluation: boosting and bagging Hoeffding Trees with and without Naïve Bayes classifiers at the leaves. 6 / 25

8 What is MOA? Easy to extend Easy to design and run experiments Philipp Kranen, Hardy Kremer, Timm Jansen, Thomas Seidl, Albert Bifet, Geoff Holmes, Bernhard Pfahringer RWTH Aachen University, University of Waikato Benchmarking Stream Clustering Algorithms within the MOA Framework KDD 2010 Demo 6 / 25

9 WEKA Waikato Environment for Knowledge Analysis Collection of state-of-the-art machine learning algorithms and data processing tools implemented in Java Released under the GPL Support for the whole process of experimental data mining Preparation of input data Statistical evaluation of learning schemes Visualization of input data and the result of learning Used for education, research and applications Complements Data Mining by Witten & Frank 7 / 25

10 WEKA: the bird 8 / 25

11 MOA: the bird The Moa (another native NZ bird) is not only flightless, like the Weka, but also extinct. 9 / 25

12 MOA: the bird The Moa (another native NZ bird) is not only flightless, like the Weka, but also extinct. 9 / 25

13 MOA: the bird The Moa (another native NZ bird) is not only flightless, like the Weka, but also extinct. 9 / 25

14 Data stream classification cycle 1 Process an example at a time, and inspect it only once (at most) 2 Use a limited amount of memory 3 Work in a limited amount of time 4 Be ready to predict at any point 10 / 25

15 Experimental setting Evaluation procedures for Data Streams Holdout Interleaved Test-Then-Train or Prequential Environments Sensor Network: 100Kb Handheld Computer: 32 Mb Server: 400 Mb 11 / 25

16 Experimental setting Data Sources Random Tree Generator Random RBF Generator LED Generator Waveform Generator Function Generator 11 / 25

17 Experimental setting Classifiers Naive Bayes Decision stumps Hoeffding Tree Hoeffding Option Tree Bagging and Boosting Prediction strategies Majority class Naive Bayes Leaves Adaptive Hybrid 11 / 25

18 Hoeffding Option Tree Hoeffding Option Trees Regular Hoeffding tree containing additional option nodes that allow several tests to be applied, leading to multiple Hoeffding trees as separate paths. 12 / 25

19 Extension to Evolving Data Streams New Evolving Data Stream Extensions New Stream Generators New UNION of Streams New Classifiers 13 / 25

20 Extension to Evolving Data Streams New Evolving Data Stream Generators Random RBF with Drift LED with Drift Waveform with Drift Hyperplane SEA Generator STAGGER Generator 13 / 25

21 Concept Drift Framework f (t) 1 f (t) 0.5 α α t t 0 W Definition Given two data streams a, b, we define c = a W t 0 b as the data stream built joining the two data streams a and b Pr[c(t) = b(t)] = 1/(1 + e 4(t t0)/w ). Pr[c(t) = a(t)] = 1 Pr[c(t) = b(t)] 14 / 25

22 Concept Drift Framework f (t) 1 f (t) 0.5 α α t t 0 W Example (((a W 0 t 0 b) W 1 t 1 c) W 2 t 2 d)... (((SEA 9 W t 0 SEA 8 ) W 2t 0 SEA 7 ) W 3t 0 SEA 9.5 ) CovPokElec = (CoverType 5, ,012 Poker) 5,000 1,000,000 ELEC2 14 / 25

23 Extension to Evolving Data Streams New Evolving Data Stream Classifiers Adaptive Hoeffding Option Tree DDM Hoeffding Tree EDDM Hoeffding Tree OCBoost FLBoost 15 / 25

24 Ensemble Methods New ensemble methods: Adaptive-Size Hoeffding Tree bagging: each tree has a maximum size after one node splits, it deletes some nodes to reduce its size if the size of the tree is higher than the maximum value ADWIN bagging: When a change is detected, the worst classifier is removed and a new classifier is added. 16 / 25

25 ADWIN Bagging ADWIN An adaptive sliding window whose size is recomputed online according to the rate of change observed. ADWIN has rigorous guarantees (theorems) On ratio of false positives and negatives On the relation of the size of the current window and change rates ADWIN Bagging When a change is detected, the worst classifier is removed and a new classifier is added. 17 / 25

26 Web abifet/moa/ 18 / 25

27 GUI java -cp.:moa.jar:weka.jar -javaagent:sizeofag.jar moa.gui.gui 19 / 25

28 Command Line EvaluatePeriodicHeldOutTest java -cp.:moa.jar:weka.jar -javaagent:sizeofag.jar moa.dotask "EvaluatePeriodicHeldOutTest -l DecisionStump -s generators.waveformgenerator -n i f " > dsresult.csv This command creates a comma separated values file: training the DecisionStump classifier on the WaveformGenerator data, using the first 100 thousand examples for testing, training on a total of 100 million examples, and testing every one million examples: 20 / 25

29 Easy Design of a MOA classifier void resetlearningimpl () void trainoninstanceimpl (Instance inst) double[] getvotesforinstance (Instance i) void getmodeldescription (StringBuilder out, int indent) 21 / 25

30 Example: Sentiment Analysis on Twitter Sentiment analysis Classification problem into two categories depending on whether they convey positive or negative feelings Emoticons are visual cues associated with emotional states Positive Emoticons Negative Emoticons :) :( :-) :-( : ) : ( :D =) Table: List of positive and negative emoticons. 22 / 25

31 Twitter Empirical evaluation Sliding Window Prequential Accuracy Sliding Window Kappa Statistic Accuracy % ,01 0,08 0,15 0,22 0,29 0,36 0,43 Millions of Instances 0,5 0,57 0,64 0,71 0,78 0,85 0,92 0,99 1,06 1,13 1,2 1,27 1,34 1,41 1,48 1,55 Kappa Statistic ,01 0,08 0,15 0,22 0,29 0,36 0,43 0,50 0,57 0,64 0,71 0,78 0,85 0,92 0,99 1,06 1,13 1,20 1,27 1,34 1,41 1,48 1,55 Millions of Instances NB Multinomial SGD Hoeffding Tree Class Distribution NB Multinomial SGD Hoeffding Tree Class Distribution Figure: Accuracy and Kappa Statistic on twittersentiment corpus 23 / 25

32 Twitter Empirical evaluation Sliding Window Prequential Accuracy Sliding Window Kappa Statistic Accuracy % ,01 0,1 0,19 0,28 0,37 0,46 0,55 0,64 0,73 0,82 0,91 1 1,09 1,18 1,27 1,36 1,45 1,54 1,63 1,72 1,81 1,9 1,99 2,08 Millions of Instances ,01 1 Kappa Statistic 0,1 0,19 0,28 0,37 0,46 0,55 0,64 0,73 0,82 0,91 1,09 1,18 1,27 1,36 Millions of Instances 1,45 1,54 1,63 1,72 1,81 1,9 1,99 2,08 NB Multinomial SGD Hoeffding Tree Class Distribution NB Multinomial SGD Hoeffding Tree Class Distribution Figure: Accuracy and Kappa Statistic on Edinburgh corpus 24 / 25

33 Summary {M}assive {O}nline {A}nalysis is a framework for online learning from data streams. abifet/moa/ It is closely related to WEKA It includes a collection of offline and online as well as tools for evaluation: boosting and bagging Hoeffding Trees MOA deals with evolving data streams MOA is easy to use and extend 25 / 25