Doing the Data Science Dance

Transcription

1 Doing the Data Science Dance Dean Abbott Abbott Analytics, SmarterHQ KNIME Fall Summit

2 Data Science vs. Other Labels 2

3 Google Trends 3 Abbott Analytics,

4 Google Trends 4 Abbott Analytics,

5 What do Predictive Modelers do? The CRISP-DM Process Model CRoss-Industry Standard Process Model for Data Mining Business Understanding Data Understanding Describes Components of Complete Data Mining Cycle from the Project Manager s Perspective Deployment Data Data Data Data Preparation Modeling Shows Iterative Nature of Data Mining Evaluation 5

6 What we Want to Do! 6

7 How The Citizen Data Scientist Will Democratize Big Data Published on April 6,

8 How The Citizen Data Scientist Will Democratize Big Data Published on April 6, Retailer Sears, for example, recently empowered 400 staff from its business intelligence (BI) operations to carry out advanced, Big Data driven customer segmentation work which would previously have been carried out by specialist Big Data analysts, probably with PhDs.

9 Is it a Recipe? 9

10 Is it a Recipe? Can we apply a recipe to machine learning and data science modeling processes? 10

11 Good Set of Data Prep Steps! 11 Abbott Analytics,

12 Data Preparation Dependencies Fill missing values Explode categorical variables *Outliers and scale very influential Sometimes automatic in software; beware of how! 12 Categoricals are fine Numeric data must be binned (except some decision trees) Outliers don t matter Missing values a category Abbott Analytics,

13 Why Are Outliers a Problem? Squares Linear Regression: K-Means Clustering Mean Squared Error quared_error dean_distance 13 Abbott Analytics,

14 Effect of Outliers on Correlations (and Regression) 4,843 records 14

15 Effect of Outliers on Correlations (and Regression) 4,843 records 15

16 Effect of Outliers on Correlations (and Regression) 4,843 records Corresponds to R^2 increase from 0.42 to

17 Decision Trees Can Handle it 17

18 Effect of Distance on Clusters 18 Abbott Analytics,

19 Effect of Distance on Clusters 19 Abbott Analytics,

20 Effect of Distance on Clusters 20 Abbott Analytics,

21 Effect of Distance on Clusters 21 Abbott Analytics,

22 22

23 Log transform the heavily skewed fields 23

24 Dummy Vars Note: stdev are Typically

25 Try K-Means with Different Normalization Approaches 25

26 Measurements are F Statistic K Means Clustering: Magnitude and Dummy Bias 26

27 PCA: Natural Units 27

28 PCA: Scaled Units 28

29 PCA: Scaled and Dummy Scaling 29

30 PCA: Scaled and Dummy Scaling 30

31 Missing Value Imputation Delete the record (row), or delete the field (column) Replace with a constant Replace missing value with mean, median, or distribution Replace missing with random self-substitution Surrogate Splits (CART) Make missing a category Simple for rule-based algorithms; Turn continuous into categorical for numeric algorithms Replace with the missing value with an estimate Select value from another field having high correlation with variable containing missing values Build a model with variable containing missing values as output, and other variables without missing values as an input 31 Abbott Analytics,

32 CHAID Trees: Missing Values are Just Another Category 32

33 Summary Linear K-Means Neural Decision Data Preparation Step Regression K-NN Clustering PCA Networks Trees Fill Missing Values Y Y Y Y Y * Correlation Filtering Y Y Y De-Skew (log, box-cox) Y Y Y Y Mitigate Outliers Y Y Y Y * * Remove Magnitude Bias (Scale) Y Y Y Y * Remove Categorical "Dummy" Bias Y Y Y Y Mitigate Categorical Cardinality Bias Y 33

34 Stratify or Not to Stratify That is the Question!? 5.1% TARGET_B = 1: unbalanced data 34

35 Comparing Logistic Regression with and without Equal Size Sampling No Stratified Sampling Equal Sampling 35

36 Don t Need to Stratify With Many Algorithms 36

37 Know the Algorithm when Developing Sampling Strategy Stratified Natural (orig) Variable Coeff. Std. Err. P> z Coeff._natural Std. Err._natural P> z _natural coeff diff coeff compare RFA_2F within SE D_RFA_2A within SE F_RFA_2A within SE G_RFA_2A within SE DOMAIN within SE DOMAIN within SE NGIFTALL_log 10 LASTGIFT_log within SE within SE Constant outside SE 37

38 Input Variable Interactions Algorithms are mixed on interactions in theory Linear Regression, Logistic Regression, knn, kmeans clustering, PCA. are main effect models Decision trees are greedy searchers Built to find interactions But, only if they can be found in sequence (one at a time, stepwise) Neural Networks find interactions well (XOR) Naïve Bayes find intersections, not interactions Algorithms don t always identify interactions well or well-enough in practice 38 Abbott Analytics,

39 Simple Interaction Function Two uniform variables: x and y 2,564 records if ( x*y > 0 ) return ("1"); else return("0"); 39 Abbott Analytics,

40 Four Classifiers Decision Tree, min Leaf node 50 records Naïve Bayes aaa Logistic Regression 40 Rprop Neural Net, 300 epochs Abbott Analytics,

41 Errors True correct False incorrect False correct True incorrect Decision Tree, min Leaf node 50 records Naïve Bayes Logistic Regression 41 Rprop Neural Net, 300 epochs Abbott Analytics,

42 Don t Build Interactions Manually* Too many too many So what do you do? * Except for those you know about 42 Abbott Analytics,

43 Automatic Interaction Detection Trees: build 2-level trees Pros: works with continuous and categoricals Cons: greedy, only finds one solution at a time (Battery) Association rules: build 2-antecedent rules Pros: exhaustive Cons: only works with categoricals Use the linear/logistic regression algorithm itself, loop over all 2-way interactions Pros: context is the model you may want to use, easy to do in R, Matlab, Python, SAS (coding) Cons: slow, have to code, what to do with dummies 43 Abbott Analytics,

44 Summing up what we ve covered Is this a Recipe? 44

45 Is it a Recipe?...YES! Can we apply a recipe to machine learning and data science modeling processes? 45

46 Conclusions Know what the algorithms can do (and not do!) before deciding on data preparation When are data shapes and data ranges important? It s not hard.just requires some thought Once you know what to do, you have your recipe! 46