Digital Analytics Meetup #6. Θεσσαλονίκη, 21 Μαρτίου 2018

Transcription

1 Digital Analytics Meetup #6 Θεσσαλονίκη, 21 Μαρτίου 2018

2 Hello! Τάσος Βεντούρης Data Scientist and Game Hattrick Ltd You can find me Tasos Ventouris 2

3 More About Me! (2012) BSc Math Auth (2013) MSc Web Math Auth (2013) Open Knowledge Greece (2014) Open Knowledge Inter. (2014) Data Hattrick (2016) Found stackprime 3

4 The Data Science Handbook 4

5 Η ιστορία Πως ξεκίνησαν όλα; 5

6 Timeline Computer Science = Data Science από Peter Naur Πρώτη φορά σε δημοσίευση από Peter Naur Συνέδριο με τίτλο Data Science, classification, and related methods Ομιλία του Jeff Wu με τίτλο Statistics = Data Science? William S. Cleveland χρησιμοποίησε τη Data Science ως ανεξάρτητο όρο σε άρθρο της International Statistical Review Committee on Data for Science & Technology. Νέο περιοδικό με τίτλο Data Science Journal The Journal of Data Science από Columbia University DJ Patil & Jeff Hammerbacher χρησιμοποίησαν τον τίτλο Data Scientist Άρθρο από Harvard Business Review με τίτλο Data Scientist: The Sexiest Job of the 21st Century 6

7 Data Science Bubble or not?

8 8

9 9

10 The creation of data products *Data product = Ένα εργαλείο που δηµιουργήθηκε µε τη χρήση δεδοµένων και βοηθάει στη λήψη αποφάσεων. 10

11 11

12 Data Scientist Ποια είναι τα χαρακτηριστικά του;

13 13

14 A Data Scientist is a statistician who lives in San Francisco 14

15 A Data Scientist is a person who is able to 15

16 A Data Scientist is a person who is able to run a regression 15

17 A Data Scientist is a person who is able to run a regression write a sql query 15

18 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site 15

19 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment 15

20 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices 15

21 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame 15

22 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning 15

23 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery 15

24 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery 15

25 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python 15

26 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce 15

27 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce build a dashboard 15

28 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce build a dashboard clean up messy data 15

29 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce build a dashboard clean up messy data test a hypothesis 15

30 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce build a dashboard clean up messy data test a hypothesis script a shell 15

31 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce build a dashboard clean up messy data test a hypothesis script a shell hack a p-value 15

32 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce build a dashboard clean up messy data test a hypothesis script a shell hack a p-value machine-learn a model 15

33 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce build a dashboard clean up messy data test a hypothesis script a shell hack a p-value machine-learn a model talk to a business person 15

34 A Data Scientist is a person who is able to run a regression write a sql query scrape a web site design an experiment factor matrices use a data frame pretend to understand deep learning steal from the d3 gallery argue r versus python use mapreduce build a dashboard clean up messy data test a hypothesis script a shell hack a p-value machine-learn a model talk to a business person 15

35 Το μονοπάτι ενός Data Scientist RPG character? 16

36 Το μονοπάτι του Data Scientist 17

37 Use your brain to take decisions Don t use it to store info 18

38 Data Science Process Ποια είναι τα βήματα; 19

39 Το

40 Το

41 Reality??? 22

42 Machine Learning Ή αλλιώς μηχανές μάθησης 23

43 Machine Learning family 24

44 Azure ML

45 What is Azure Machine Learning? One of the best tools to learn ML Drag and drop R, Python, SQL, Jupyter integration No code (if you don t want it) Web Service (RESTful API) Part of Azure Cloud 26

46 Azure ML Modules 27

47 User interaction 28

48 Dataset Preview 29

49 Dataset Preview 29

50 Dataset Preview 29

51 Dataset Preview 30

52 Dataset Preview 31

53 Dataset Preview 31

54 Data Transformation 32

55 Data Transformation Filters (like median and moving average) Manipulation (add/remove/edit/join data) Sample/Partition/Split Scale and Reduce (Normalize, PCA) 32

56 Missing Data 33

57 Missing Data 33

58 Missing Data 33

59 Who said anything about PCA? 34

60 Split Data 35

61 Train Model 36

62 Visualise the results 37

63 Evaluate Model 38

64 Feature Importance 39

65 Overfitting? 40

66 Overfitting? 40

67 Overfitting? Min: -3s+x =

68 Overfitting? Min: -3s+x = 0.73 Max: 3s+x =

69 I like to get my hands dirty 41

70 Play time!

71 Thanks! Any questions? 43