Network Analysis. Dr. Scott A. Hale Oxford Internet Institute 16 March 2016

Transcription

1 Network Analysis Dr. Scott A. Hale Oxford Internet Institute 16 March 2016

2 Outline for today 1 Basic network concepts 2 Network data 3 Software for networks 4 Layout algorithms 5 Statistical measures 6 Creating simple interactive visualizations 7 Python and NetworkX 8 Further resources

3 What are networks? Networks (graphs) are set of nodes (verticies) connected by edges (links, ties, arcs)

4 What are networks? Networks (graphs) are set of nodes (verticies) connected by edges (links, ties, arcs) Additional details Whole vs. ego: whole networks have all nodes within a natural boundary (platform, organization, etc.). An ego network has one node and all of its immediate neighbors. Edges can be directed or undirected and weighted or unweighted Additionally, networks may be multilayer and/or multimodal.

5 Why? Characterize network structure How far apart / well-connected are nodes? Are some nodes at more important positions? Is the network composed of communities? How does network structure affect processes? Information diffusion Coordination/cooperation Resilience to failure/attack

6 A network First questions when approaching a network What are edges? What are nodes? What kind of network? Inclusion/exclusion criteria

7 Sources of network data Precollected data Default.aspx Collecting your own Basic point-and-click options in NodeXL (more) and Gephi Twitter data with a small amount of programming A good general resource: Russell (2011), Mining the Social Web, O Reilly Media.

8 Network data formats Data format The simplest form of network is simply two spreadsheets or comma-separated values (csv) files. One sheet for the nodes of the network and one sheet for the edges of the network. Example Nodes id name profession n1 Alan student n2 Betty teacher n3 Charlotte student Edges id source target weight e1 n1 n2 1 e2 n1 n3 100

9 Gephi Open-source, cross-platform GUI interface Primary strength is to visualize networks Basic statistical properties are also available Alternatives include NodeXL, Pajek, GUESS, NetDraw, Tulip, and more

10 Other software NodeXL ( Add-in for Microsoft Excel (Windows only) Good data collection options (Twitter, Facebook, YouTube,... ) Basic visualization Pajek Standalone, Windows (or Linux with wine) Good interactive environment for metrics and basic visualization igraph install.packages( igraph ) in R (r-project.org) (or python) Cross-platform Text driven: powerful for analysis NetworkX Cross-platform python library Examples/introduction in this slide deck

11 Demo Gephi Open Filter (e.g., only giant component) Visualize Find clusters statistics: Modularity Export image Export data (gexf)

12 Additional termonology Force-directed layout: Minimize crossing edges, often based on physics analogies (e.g., edges as springs, nodes as charged particles) Components: disconnected parts of a graph Clusters: Dense subsets of a graph with more links within each subset than between the subsets (often based on maximizing modularity). Homophily: Similar nodes group together ( Birds of a feather flock together )

13 Layout algorithms Fruchterman Reingold (1991): Standard; edges as springs, nodes as charged particles goal to minimize energy in the system. Uses edge weight in Gephi Force Atlas 2: Designed for small-world/scale free networks Faster than FR, uses edge weight Harel-Koren Fast Multiscale: GEM (Generalized Expectation-Maximization) Minimize edge overlap; very sensitive to initial layout

14 Network measures With many nodes visualizations are often difficult/impossible to interpret. Statistical measures can be very revealing, however. Node-level Degree (in, out): How many incoming/outgoing edges does a node have? Centrality (next slide) Constraint Network-level Components: Number of disconnected subsets of nodes observed edges Density: maximum number of edges possible closed triplets connected triples Clustering coefficient Path length distribution Distributions of node-level measures

15 Centrality measures Degree Closeness: Measures the average geodesic distance to ALL other nodes. Informally, an indication of the ability of a node to diffuse a property efficiently. Betweenness: Number of shortest paths the node lies on. Informally, the betweenness is high if a node bridges clusters. Eigenvector: A weighted degree centrality (inbound links from highly central nodes count more). PageRank: Not strictly a centrality measure, but similar to eigenvector but modeled as a random walk with a teleportation parameter

16 Interactive Visualizations Install Sigmajs Exporter plugin in Gephi (Tools Plugins) Once installed, open a network and visualize as you like. Then export the network via File Export Sigma.js template Edit the description of the network as needed. This produces a folder of files. Open in Firefox and/or upload to any webserver. Chrome prevents viewing locally for security reasons. It will work fine if uploaded to a webserver.

17 Python resources tweepy: Package for Twitter stream and search APIs search and stream API example code along with code to create mentions/retweet network at Python two versions: 2.7.x many packages, issues with non-english scripts 3.x less packages, but excellent handling of international scripts (unicode)

18 NetworkX Package to represent networks as python objects Convenient functions to add, delete, iterate nodes/edges Functions to calculate network statistics (degree, clustering, etc.) Easily generate comparison graphs based on statistical models Visualization Alternatives include igraph (available for Python and R)

19 NetworkX: Nodes import networkx as nx g=nx.graph() #A new (empty) undirected graph g.add_node("alan") #Add one new node g.add_nodes_from(["bob","carol","denise"])#add three new nodes #Nodes can have attributes g.node["alan"]["gender"]="m" g.node["bob"]["gender"]="m" g.node["carol"]["gender"]="f" g.node["denise"]["gender"]="f" for n in g: print("{0} has gender {1}".format(n,g.node[n]["gender"]))

20 NetworkX: Edges #Interesting graphs have edges g.add_edge("alan","bob") #Add one new edge #Add two new edges g.add_edges_from([["carol","denise"],["carol","bob"]]) #Edge attributes g.edge["alan"]["bob"]["relationship"]="friends" g.edge["carol"]["denise"]["relationship"]="friends" g.edge["carol"]["bob"]["relationship"]="married" #New edge with an attribute g.add_edges_from([["carol","alan", {"relationship":"friends"}]])

21 NetworkX: Edges for e in g.edges_iter(): n1=e[0] n2=e[1] print("{0} and {1} are {2}".format(n1,n2, g.edge[n1][n2]["relationship"]))

22 NetworkX: Measures g.number_of_nodes() g.nodes(data=true) g.number_of_edges() g.edges(data=true) nx.info(g) nx.density(g) nx.number_connected_components(g) nx.degree_histogram(g) nx.betweenness_centrality(g) nx.clustering(g) nx.clustering(g, nodes=["bob"])

23 NetworkX: Visualize or save #Save g to the file my_graph.graphml in graphml format #prettyprint will make it nice for a human to read nx.write_graphml(g,"my_graph.graphml",prettyprint=true) #Layout g with the Fruchterman-Reingold force-directed #algorithm and save the result to my_graph.png #with_labels will label each node with its id import matplotlib.pyplot as plt nx.draw_spring(g,with_labels=true) plt.savefig("my_graph.png") plt.clf() #Clear plot

24 NetworkX: Odds and ends #Read a graph from the file my_graph.graphml in graphml format g=nx.read_graphml("my_graph.graphml") #Create a (empty) directed graph g=nx.digraph() See index.html for many more commands. Note that some commands are only available on directed or undirected graphs.

25 Resources NetworkX documentation ( documentation/latest/reference/) Newman, M.E.J., Networks: An Introduction Kadushin, C., Understanding Social Networks: Theories, Concepts, and Findings De Nooy, W., et al., Exploratory Social Network Analysis with Pajek Shneiderman B., and Smith, M., Analyzing Social Media Networks with NodeXL

26 Network Analysis Dr. Scott A. Hale Oxford Internet Institute 16 March 2016