CS 111: Program Design I Lecture 25: Social networks, nothingness. Robert H. Sloan & Richard Warner University of Illinois at Chicago April 24, 2018

Transcription

1 CS 111: Program Desig I Lecture 25: Social etworks, othigess Robert H. Sloa & Richard Warer Uiversity of Illiois at Chicago April 24, 2018

2 SOCIAL MEDIA AND PRIVACY (CONT.)

3 The DHS Example DHS collects social media hadles, aliases, associated idetifiable iformatio, ad search results o immigrats, icludig aturalized citizes ad permaet residets, ad it stores public-source data (icludig iformatio from social media) ad gathers iformatio from commercial data providers ad public sources such as social media, ews media outlets, ad the Iteret.

4 What Relatioships? There are early 40,000 rows

5 This Makes It Easy To See Computatioal Legal jouralist First 200 rows for edge-data [edge_data ['b'] == 'Computatioal Legal']

6 Who Is Chelsea Rider? Predictive Aalytics Computatioal Legal 7096 Chelsea Rider 2607 Legal Tracker 1847 James D. Williams Army Vet Aimal Advocate Sigle Mom JD. Iterests: #laweforcemet/#lesm/#social media, #hockey, #books, #weightliftig, & most importatly, #dogs/#padas

7 Cell Phoe Calls

8 Cell Phoe Calls The structure ca reveal importat iformatio. Moey lauderig Drug distributio Jouralist/source Protesters

9 Network Aalysis Fraud Detectio Desity measures are extremely useful i determiig potetial fraud hotspots i retail bakig from a maze of accout trasactios ad applied cotrol measures. Credit card trasactio moitorig ad moey-lauderig are potetially two areas where desity metrics could trigger the ecessity for deeper ivestigatios. desity = actual coectios (-1)/2 Other useful measures: Degree, closeess, betweeess (extet of a ode s placemet o the shortest paths betwee odes) Degree for the cell phoe etwork

10 The Privacy Problem Privacy of social etwork data is a growig cocer which threates to limit access to this valuable data source. Aalysis of the graph structure of social etworks ca provide valuable iformatio for reveue geeratio ad social sciece research, but ufortuately, esurig this aalysis does ot violate idividual privacy is difficult. Simply aoymizig graphs or eve releasig oly aggregate results of aalysis may ot provide sufficiet protectio. Who is this? Teaches at UIC ad Chicago-Ket College of Law. Has a Ph.D. i Philosophy ad a JD.

11 Relatioal Privacy Privacy is relatioal whe your cotrol over the collectio ad use of iformatio cosists i your reasoable reliace o others volutarily refraiig from collectig ad usig that iformatio. 11

12 Aoucemet: Do the course evaluatio We'll make it worth your while

13 Aoucemet: Witer Fial Exam is Comig Most of you are doig great A small hadful of you are i trouble with reuiremet: You must pass the exam part of the course to pass the exam Meaig average of 2 midterms ad fial exam (weighted eual to both midterms take together) For all: Will use >= 70 fial exam to replace ay lower midterm exams But two midterms i the 30s or 40s ad fial i 50s will fail this course eve with good labs ad projects

14 PANDAS FILE READING ISSUE SOME STUDENTS ENCOUNTERED

15 Are padas reluctat readers? padas.read_csv ca take as its first argumet either file referece (obtaied from ope()) or file ame i uotes Ad optioal sep argumet you kow about sep=',' # Default ad therefore ca be omitted sep=' ' # Space separated sep='\t' # Tab separated CS111 had you usig ope() because that's default i rest of Pytho ad wated you to practice; padas people typically use text file ame ad skip ope()

16 Headers ad commets up top Some graph files start with commets startig with # up top Ad also ca have colum headers Say first 4 lies start with #. Ca tell padas either 1. Start readig at Pytho Lie 3 (0..3) as header header=3 2. or commet='#', header=noe file ame strig type argumet okay with either; fileref oly with 1!

17 padas read_csv Suggestio: use file ame versio of pd.read_csv header: Gives lie umber to treat as lie cotaiig headers, coutig lies Pythoically as 0, 1, 2, 3, Reads headers from that lie; skips earlier lies; reads data from ext lie commet: character for commet to ed of lie; all are igored Next lie after commet always take as header If it's data must specify header=noe

18 NESTED LISTS

19 B = [[1,2,3], [5,10,20]] prit(b[1]) This will prit Clicker A 2 B [1,2,3] C [5,10,20] D E This will cause a error I do t kow

20 How cofidet are you of your aswer? A. Very Highly cofidet: I've got this B. Very cofidet C. Somewhat cofidet D. Not so cofidet: educated guess E. Not cofidet at all: radom guess ad/or bullied ito by the rest of my small group

21 Matrix Famous 1999 Fatasy/Actio movie about Neo ad the elusive Morpheus Way some studets believe that they ca lear Computer Sciece: By pluggig themselves ito it

22 Matrix Famous 1999 Fatasy/Actio movie about Neo ad the elusive Morpheus Way some studets believe that they ca lear Computer Sciece: By pluggig themselves ito it Rectagular array of (usually) umbers, e.g.,

23 Matrices i Pytho Two commo ways to represet: For us: For m-by- matrix, list of m lists, where each ier ested list is of same legth () ad represets oe row (Ca also use umpy module)

24 Creatig ested list Literal otatio: matrix = [ [5, 10, 15, 20, 25], [30, 35, 40, 45, 50], [55, 60, 65, 70, 75], [80, 85, 90, 95, 100], [105, 110, 115, 120, 125] ]

25 Buildig up ested list Create distict list of desired row or row of 0s to chage later for each row apped i: matrix = [ ] for row i rage(umber_rows): ew_row = [ ] for col i rage(umber_cols): ew_row.apped(0) #if startig all-0 matrix.apped(ew_row)

26 Useful fuctio def make_0array(rows, cols): '''returs ew rows x cols 2-d list/array of all 0s''' array = [ ] # Build up array of umbers here for j i rage(rows): ew_blak_row = [ ] # Make a NEW row for i i rage(cols): ew_blak_row.apped(0) array.apped(ew_blak_row) retur array

27 prit fuctio: stayig o oe lie (review) prit() fuctio by default always eds with ewlie. Not ice to prit 2-D m x array 1 umber/lie usig m* lies; wat whole row per lie prit() has optioal argumet ed= that ca give alterate character to put at ed istead of ewlie; e.g., a space: prit (somethig, ed=' ')

28 def ice_prit(a): for i i rage(le(a)): for j i rage(le(a[i])): prit(a[i][j], ed=" ") prit() A = [[2,5,10],[1,17,0]] ice_prit(a) This will prit Clicker A B C D E This will cause a error I do t kow

29 How cofidet are you of your aswer? A. Very Highly cofidet: I've got this B. Very cofidet C. Somewhat cofidet D. Not so cofidet: educated guess E. Not cofidet at all: radom guess ad/or bullied ito by the rest of my small group

30 def col_prit(a): for i i rage(le(a)): for j i rage(le(a[i])): prit(a[j][i], ed=" ") prit() A = [[2,5,10],[1,17,0]] col_prit(a) This will prit Clicker A B C D E This will cause a error I do t kow

31 How cofidet are you of your aswer? A. Very Highly cofidet: I've got this B. Very cofidet C. Somewhat cofidet D. Not so cofidet: educated guess E. Not cofidet at all: radom guess ad/or bullied ito by the rest of my small group

32 Drawig graphs MATPLOTLIB MODULE HIGHLIGHTS

33 A Picture is Worth 1000 Excel cells Year,Aual aomaly,lower 95% cofidece iterval,upper 95% cofidece iterval 1880, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Ad so o

34 matplotlib: A module for drawig graphs import matplotlab.pyplot as plt matplotlib is super commoly used module for 2-D graphics i Pytho. There are others, but matplotlib is most widely used Style take from MATLAB

35 Fuctios i plt that might be ice to use: plot plt.plot(ls) with oe list iput parameter: Makes a lie graph assumig x's are rage(le(ls)), i.e., 0, 1,, le(ls) -1 With 2: x vs. y Already see: Ca set x ad/or y-axis label, ad the title Importat after CS 111! Exact details of whe ad where plot appears deped heavily o whether you are usig Spyder, cosole, Jupyter, or other, ad o your settigs

36 Basic demo code import matplotlib.pyplot as plt import radom # simple plottig demo of plai lie graph x = [1, 2, 3, 4, 5] ylie =[] # y values will go here for i i rage(le(x)): ylie.apped(radom.radom()) plt.ylabel('some 0 to 1 radom umbers') plt.xlabel('x is 1 to 5') plt.title('lie graph of radom umbers') plt.plot(x, ylie)

37 You probably wat plots i their ow widow Aybody havig plots show up iside the lower-right cosole widow ad havig trouble savig them out?

38 To always get graphs i their ow widow Spyder prefereces (uder Pytho meu o Mac, o Widows maybe uder Tools?) The: Ipytho Cosole à Graphics à Graphics Backed à Backed: "automatic" Reuires you to restart Spyder (oce) to start workig I theory there's also commad ca give at ipytho prompt for this, but Prof. Sloa could't figure it out, ad it's defiitely ot what's stated i official Spyder tutorial (%matplotlib t), which gives error Ad plt.figure(1), plt.figure(2), etc. starts ew plots Util the, plt.whatever() keeps addig to curret plot

39 More specific stylig thigs Described i Zybooks 15.2 ad 15.3, like addig legeds to describe what differet lies o multi-lie lie graph are, makig lies differet colors ad styles (dotted, dashed, solid) that you choose istead of matplotlib choosig automagically Nobody i his or her right mid memorizes this stuff uless you are workig o graphs as full-time job; we would't ask exam uestios about it But you do eed to kow where to fid it if a lab asks you to plot somethig with a gree dashed lie

40 A few examples If last argumet to plot is strig, that's the format 'b-' Solid blue lie; matplotlib default 'r--' Red dashes 'bs' Blue suares 'g^' Gree triagles

41 HEAT MAPS

42 Heat map (or Heatmap) Heat map: Graphical represetatio of data where idividual values i matrix represeted as colors (Word Heatmap was coied ad trademarked i 1991)

43

44 import matplotlib.pyplot as plt x = [1, 2, 3, 4, 5, 6] y = [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0] itesity = [ [5, 10, 15, 20, 25], [30, 35, 40, 45, 50], [55, 60, 65, 70, 75], [80, 85, 90, 95, 100], [105, 110, 115, 120, 125], [130, 135, 140, 145, 150] ] plt.pcolor(x, y, itesity) # Creates heatmap, origi lower left corer plt.summer() # Color scheme: Gree (low) to Yellow (hi) plt.colorbar() # Adds colorbar so we kow what values mea plt.title("simple Heat Map Example")

45 Heat maps i Pytho Everythig we eed is i matplotlib.pyplot, so import matplotlib.plot as plt plt.pcolor(c) creates heat map from array C of color values Row 0 displayed at bottom of figure Each row displayed left to right So C[0][0] i lower left corer plt.pcolor(x, Y, C) has lists of umbers for X- ad Y-axes

46 Fecepost issue If you wat to specify borders of a ru of rectagles placed side-by-side, eed +1 specificatios left ed of every rectagle, ad right ed of last plt.pcolor(x, Y, C) should have le(x) = 1 + umber of colums of C le(y) = 1 + umber of rows of C I plt.pcolor(c) form labels are supplied startig at 0 for both X ad Y