Play with Python: An intro to Data Science

Transcription

1 Play with Python: An intro to Data Science Ignacio Larrú Instituto de Empresa

2 Who am I? Passionate about Technology From Iphone apps to algorithmic programming I love innovative technology Former Entrepreneur: Founded several companies from online memorabilia e-commerce to structural civil engineering calculations. Investment Banker: I advise Spanish companies in M&A and IPO processes Venture Capital & Bootcamp: CFO, investment director at K Fund + Academic Director of IE Data Science Bootcamp 2

3 Big Data and Data Science Big Data technologies Data Science

4 Why is Data Science so difficult?

5 Overview of the Data Science process Validation!!!! Framing the Problem Solving the Problem Action!!!

6 Problem recognition Business comes first, think on what moves the needle Focus specific on decisions that will be made as a result of the analysis Helps everyone realize the reason for the analysis Makes identifying key stakeholders easier No decision. No analytics? Plan your objective for your problem: Investigation Exploration A/B Testing Survey Prediction Past performance ( reporting) Scope of the problem should be expansive but by the end of the problem framing you should have a clear statement of the problem

7 Exploratory Data Analysis Use descriptive statistics (median, mode, variance, frequency tables, correlations lines, etc ) to understand the important characteristics of a dataset Identify trends and outliers

8 Overview of Data processing algorithms (i) 1. Classification -> for each individual in a population, which of a set of clasess this individual belongs to. Among all the customers of ACME, which are likely to respond to a given offer? 2. Regression -> Estimate or predict, for each individual, the numerical value of some variable for that individual How much will a customer use the service?

9 Overview of Data processing algorithms (ii) 3. Similarity matching -> identify similar indivduals based on data known about them Other customers also bought 4. Clustering -> Group individuals in a population together by their similarity but not driven by nay specific purpose Do our customers form natural groups or segments? 5. Co occurence ->Find associations between entitites based on transactions involving them What items are commonly purchased together?

10 Can data visualizations hurt your analysis?

11 Can data visualizations hurt your analysis?

12 Can data visualizations hurt your analysis?

13 Can data visualizations hurt your analysis?

14 Lying with graphs Source: Hbr.com

15 Lying with graphs Source: Hbr.com

16 Python Data Science Stack

17 Hello World! print( hello World ) 17

18 Python is interpreted 18

19 Programming Python 19

20 Comments in Python # for a single line comment for a multiple line comment 20

21 Variables in Python 21

22 Variables in Python Don t need to have a pre-defined type, they get the type from the value they are pointing at Four main types: String holds text based values name = Ignacio int Integer numbers name = 10 float floating decimal numbers name = 10.4 Boolean (True or False) More variable types (lists, tuples, dictionaries to be reviewed during the course) 22

23 Variables in Python A variable has a name (identifier) a type a scope and a value A valid identifier is a non-empty sequence of characters with: The start character can be the underscore "_" or a capital or lower case letter The letters following the start character can be anything which is permitted as a start character plus the digits Identifiers are case-sensitive! Python keywords are not allowed as identifier names! 23

24 Python vs. other languages Python Statically typed languages Variable type determined at runtime Variable bound to one object and the object has only one type Varible can change type by changing the type of the object bound to the variable Bound to a type at compile type Bound to an object at runtime Need to declare the variable before using it 24

25 Python is a dynamically typed language 25

26 Python is a strongly typed language 26

27 With every great power Guidelines Use descriptive names (x vs. sales_amount) Be consistent (user_name or username?) Follow the traditions of the language Usually in Python variable names start with a lowercase letter and avoid starting with an underscore Keep the length in check no user_total_sales_month_report 27

28 Mathematical operators Source: 28

29 Mathematical operators 29

30 Converting values float(x) - returns a floating-point value by converting x int(x) str(x) - returns an integer value by converting x - returns a string value by converting x bool(x) returns a boolean value 30

31 If else - elif 31

32 If else - elif 32

33 Logical operators 33

34 While Loop 34

35 While and if 35

36 For Loops 36

37 Range() Function range([start], stop[, step]) 37

38 Break, Continue and pass with else Break -> End loop Continue -> End operation Pass -> Null statement used as placeholder Else at the end of loops: For -> ended normally the loop ( no break) While -> The loop condition is false 38

39 Break, Continue and pass with else 39

40 Python simple data structures 40

41 Sequences 41

42 Strings are sequences 42

43 Using len() and in len() function will return the length of a sequence The in operator checks if an element is a member of a sequence If the element is a member the condition is true else it is false 43

44 Using len() and in 44

45 Programming exercise 45

46 Slicing Sequences 46

47 Programming exercise 47

48 Lists Mutables sequences 48

49 Lists Adding new items append adds at the end of the list Insert(index,value) allows you to insert at a given index 49

50 Lists Remove remove(value) del function 50

51 Lists Remove items 51

52 Sort() vs. Sorted() 52

53 Tuples Inmutable secuences that can contain elements of different types that can be mutable If the contents not need to change used tuples vs. lists Faster than lists 53

54 Tuples are inmutable but not its elements 54

55 Sets Non duplicative unordered collections 55

56 Sets Operations 56

57 Sets Math Operations 57

58 Sets Math Operations 58

59 Dictionaries 59

60 Dictionaries 60

61 Dictionaries - operations update(d) to join dictionaries (or {**x,**y}) copy() to creatr a shallow copy get( key ) returns None if element doesn t exist 61

62 Dictionaries - operations 62

63 Crossfit coding 63

64 Session Wrap-up 64