Machine Learning Part 1

Transcription

1 Data Science Weekend Machine Learning Part 1 KMK Online Analytic Team Fajri Koto Data Scientist fajri.koto@kmklabs.com

2 Machine Learning Part 1 Outline 1. Machine Learning at glance 2. Vector Representation Evaluation Measurement in Machine Learning Clustering vs Classification K-Means Algorithm Linear Regression

3 Machine Learning Part 1 Outline 1. Machine Learning at glance 2. Vector Representation Evaluation Measurement in Machine Learning Clustering vs Classification K-Means Algorithm Linear Regression

4 1. Machine Learning at Glance Data and the Methods DATA CHOOSE FEATURES DATA PREPROCESSING (data distribution, outliers, manual labelling :(, data cleansing, data splitting, etc) VECTOR REPRESENTATION (Tf, Idf, Tf-Idf, ) OPTIMIZATION (Feature selection, Matrix factorization, PCA, etc) METHODS / ALGORITHM CHOOSE TYPE (classification, clustering, regression, rank, association ) CHOOSE ALGORITHM (linear regression, svm, decision tree, etc.) EVALUATION (accuracy, precision, recall, f1 score, ctr, etc)

5 1. Machine Learning at Glance Overview of Training stage

6 1. Machine Learning at Glance Overview of Testing stage

7 Machine Learning Part 1 Outline 1. Machine Learning at glance 2. Vector Representation Evaluation Measurement in Machine Learning Clustering vs Classification K-Means Algorithm Linear Regression

8 2. Vector Representation TF Vector Space Model The main point is to form every problem into set of vectors. Example:

9 2. Vector Representation TF in Sklearn Python TF Vector Space Model in python, is implemented as CountVectorizer DEMO count_vectorizer.py

10 2. Vector Representation TFIDF Vector Space Model Another example of feature extraction result:

11 Machine Learning Part 1 Outline 1. Machine Learning at glance 2. Vector Representation 3. Evaluation Measurement in Machine Learning 4. Clustering vs Classification 5. K-Means Algorithm 6. Linear Regression

12 3. Evaluation measurement Machine Learning Overview

13 3. Evaluation measurement Evaluation in General

14 3. Evaluation measurement K-Fold cross validation

15 3. Evaluation measurement K-Fold cross validation

16 3. Evaluation measurement K-Fold cross validation (in Rapid Miner)

17 3. Evaluation measurement K-Fold cross validation - Sampling 1. Linear Sampling The Linear sampling simply divides the ExampleSet into partitions without changing the order of the examples i.e. subsets with consecutive examples are created. 2. Shuffle Sampling The Shuffled sampling builds random subsets of the ExampleSet. Examples are chosen randomly for making subsets. 3. Stratified Sampling The Stratified sampling builds random subsets and ensures that the class distribution in the subsets is the same as in the whole ExampleSet.

18 3. Evaluation measurement K-Fold cross validation

19 3. Evaluation measurement Confusion Matrix Pred: Act : Predicted Actual

20 3. Evaluation measurement Precision, Recall, Accuracy

21 3. Evaluation measurement Precision, Recall, Accuracy Practice Given prediction and actual label of binary classification in a list, Please measure: Confusion Metrics Accuracy Precision Recall Please use sklearn python library FILE: - data1.csv - evaluate.py

22 Machine Learning Part 1 Outline 1. Machine Learning at glance 2. Vector Representation 3. Evaluation Measurement in Machine Learning 4. Clustering vs Classification 5. K-Means Algorithm 6. Linear Regression

23 4. Clustering vs Classification Classification: Mengelompokkan data berdasarkan class/label yang diberikan Clustering: Mengelompokkan data berdasarkan data itu sendiri, tanpa memperhatikan class/label Clustering algorithms divide a data set into natural groups (clusters). Instances in the same cluster are similar to each other, they share certain properties

24 4. Clustering vs Classification

25 4. Clustering vs Classification Example of Clustering

26 4. Clustering vs Classification

27 4. Clustering vs Classification Supervised vs Unsupervised Classification Supervised learning: classification requires supervised learning, i.e., the training data has to specify what we are trying to learn (the classes). Unsupervised learning: clustering is an unsupervised task, i.e., the training data doesn t specify what we are trying to learn (the clusters).

28 4. Clustering vs Classification Supervised vs Unsupervised Learning Unsupervised / Clustering K-Means ( ) HCS clustering Canopy DBSCAN Fuzzy Clustering K-SVD Pitman More than 100 approaches Supervised / Classification Naive Bayes Linear Regression ( ) Decision Tree Random Forest Support Vector Machine Neural Network Deep Neural Network

29 Machine Learning Part 1 Outline 1. Machine Learning at glance 2. Vector Representation 3. Evaluation Measurement in Machine Learning 4. Clustering vs Classification 5. K-Means Algorithm 6. Linear Regression

30 5. K-Means Algorithm Why Clustering

31 5. K-Means Algorithm When Clustering

32 5. K-Means Algorithm The K-Means Algorithm Centroid-based clustering

33 5. K-Means Algorithm The K-Means Algorithm Complexity?

34 5. K-Means Algorithm The K-Means Algorithm

35 5. K-Means Algorithm The K-Means Algorithm

36 5. K-Means Algorithm The K-Means Algorithm: Example DATA Step2 ITERATION Step1: Pick the centroid randomly

37 Machine Learning Part 1 Outline 1. Machine Learning at glance 2. Vector Representation 3. Evaluation Measurement in Machine Learning 4. Clustering vs Classification 5. K-Means Algorithm 6. Linear Regression

38 6. Linear Regreession Regression VS Classification - Regression is used to predict continuous values. - Classification is used to predict which class a data point is part of (discrete value)

39 5. K-Means Algorithm Regression

40 3. Regression

41

42

43 6. Linear Regression Model Representation

44

45 6. Linear Regression How to Determine β0 and β1? Simple Matrix Multiplication β = X-1.y

46 6. Linear Regression How to Determine β0 and β1? Stochastic Gradien Descent βt = βt-1 αe

47

48 6. Linear Regression Linear Regression On Sklearn

49 6. Linear Regression Linear Regression On Sklearn >>> data = [[1,2,3],[3,2,1],[2,3,1]] >>> label = [50, 65, 70] >>> from sklearn.linear_model import LinearRegression >>> lr = LinearRegression() >>> lr.fit(data, label) <<< LinearRegression(copy_X=True, fit_intercept=true, n_jobs=1, normalize=false) >>> lr.coef_ <<< array([ , >>> lr.intercept_ <<< , ])

50 6. Linear Regression Evaluating Linear Regression R2 (Coefficient of Determination)

51 6. Linear Regression Evaluating Linear Regression R2 (Coefficient of Determination) >>> data_test = [[1,2,1],[1,1,3],[2,2,1]] >>> label_test = [45, 60, 68] >>> lr.predict(data_test) <<< array([ 63.33, ]) >>> lr.score(data_test, label_test) <<<

52 Machine Learning Part 1 Thank You Questions? KMK Online Analytic Team Fajri Koto Data Scientist fajri.koto@kmklabs.com