Electronic Supplementary Material for: Analytical reasoning task reveals limits of social learning in networks Iyad Rahwan, Dmytro Krasnoshtan, Azim Shariff, Jean-François Bonnefon A Experimental Interface A.1 Tutorial Screenshots 1
2
3
4
5
A.2 Quiz Questions Following is the set of quiz questions that participants have to correctly answer before they can proceed to the experiment. The correct answers are highlighted in bold. How many times will you see each question? 1. Only once 2. 5 times Does your reward depends on the responses of other players? 1. Yes 2. No How do you make money in this game? 1. Every correct response gets me money 2. Not every correct response gets me money, only the last trial counts for each question 3. Not every correct response gets me money, multiple correct responses to the same question only count as one 6
B Questions Below is the list of all questions. The first three questions corresponded to the Cognitive Reflection Test (CRT). These three questions generate an incorrect intuitive answer, which must be inhibited in order to produce the correct analytic answer [1]. 1. [CRT1] In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake? (Answer: 47) 2. [CRT2] If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets? Write the answer in minutes. (Answer: 5) 3. [CRT3] A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? (Answer: 0.05) After the three CRT questions, subjects moved on to another series of four questions from the Berlin Numeracy Test (BNT), which we do not discuss in this article [2]. Being either too easy or too hard, these questions produced little variance between participants in our networks, and thus did not allow us to test our hypotheses (see Figure 1 below for a visualization of the responses in the BNT questions). As these question came after participants had completed the three CRT questions, there is no concern that they could have contaminated the CRT data that we analyze in this article. 1. [BNT1] Imagine we are throwing a five-sided die 50 times. On average, out of these 50 throws how many times would this five-sided die show an odd number (1, 3 or 5)? out of 50 throws. (Answer: 30) 2. [BNT2] Out of 1,000 people in a small town 500 are members of a choir. Out of these 500 members in the choir 100 are men. Out of the 500 inhabitants that are not in the choir 300 are men. What is the probability that a randomly drawn man (not a person) is a member of the choir? (please indicate the probability in percents) (Answer: 25) 3. [BNT3] Imagine we are throwing a loaded die (6 sides). The probability that the die shows a 6 is twice as high as the probability of each of the other numbers. On average, out of these 70 throws, how many times would the die show the number 6? out of 70 throws. (Answer: 20) 4. [BNT4] In a forest 20% of mushrooms are red, 50% brown and 30% white. A red mushroom is poisonous with a probability of 20%. A mushroom that is not red is poisonous with a probability of 5%. What is the probability that a poisonous mushroom in the forest is red? (Answer: 50) 7
TOPOLOGY Full Erdos Renyi Barabasi Albert Clustered Baseline Proportion of correct responses 1.00 0.75 0.50 0.25 0.00 First Question Second Question Third Question Fourth Qestion 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Trial Figure 1: BNT questions are either too easy or too hard, reducing variance observed in CRT questions. 8
C Network Structures & Their Properties C.1 The Networks A network (or graph) consists of a set V vertices (a.k.a. nodes, individuals), and a set E of edges (a.k.a. connections or links) between them. Elements of E can be denoted by a pair E ij = (i, j) where i, j V. Each of our experimental sessions ran on one of the four graphs: (1) Barabasi-Albert model; (2) Clustering graph; (3) Erdos-Renyi model; (4) Full graph. The different graph structures are visually depicted below. These graphs were chosen due to significant discrepancy in their measures on the macro (network) and micro (node) level, as shown below. Barabasi Albert model Clustering graph Full graph Erdos Renyi model Figure 2: List of graphs 9
C.2 Micro Measures And on the micro level (for each node): Degree: The degree k i of a vertex i is simply the number of edges incident to this vertex. In a directed graph, we can distinguish between the out-degree (outgoing edges) and in-degree (incoming edges). In the present paper, edges are considered undirected. The different graph structures we used have significantly varying distributions of node degrees, as shown below. The degree distribution of the Erdo-Renyi graph follows a Poission distribution, thus most nodes have a more or less equal number of neighbors (no one is disproportionately popular). In contrasted, in the Barabasi-Albert graph, the distribution is long-tailed, with a few very highly connected nodes. In the full graph, everyone has equal degree, since everyone is connected to everyone else. Finally, in the clustering graph, degrees are more or less identical. However, as we will see below, some nodes are a more privilaged position in the middle of the graph. 20 Barabasi model 20 Erdos Renyi model 15 15 10 10 5 5 0 0 5 10 15 20 0 0 5 10 15 20 20 Full graph 20 Clustering graph 15 15 10 10 5 5 0 0 5 10 15 20 0 0 5 10 15 20 Figure 3: Degree distribution Local clustering coefficient: The local clustering coefficient captures the following intuition: out of all pairs of friends that i is connected to, how many of those friends are also friends with one another. In other words: C i = number of triangles connected to node i number of triples centered around node i where a triple centred around node i is a set of two edges connected to node i (if the degree of node i is 0 or 1, we which gives us C i = 0/0, we can set C i = 0). High local clustering coefficient for node i indicates that i belongs to a tightly knit group. More formally, the local clustering coefficient c i is defined as follows: C i = {E jk} k i (k i 1) : v j, v k N i, E jk E where k i is the out-degree of vertex i, and N i = {v j : E ij E} is the set of out-neighbours of vertex i. For 10
undirected graphs the value of C i is normalized as C i = 2C i. If to rephrase this in the simple words, the local clustering coefficient of a vertex in a graph shows how close its neighbors are to being a full graph. The figure below highlights how the distribution of local clustering coefficients varies significantly across the different network structures. In particular, nodes in the Erdos-Renyi and Barabasi-Albert graphs have much lower clustering compared to the Clustering graph. Note that in the full graph, every node has a local clustering coefficient of 1, since everyone is connected to everyone else. Betweenness centrality: The betweenness centrality of a node is equal to the number of shortest paths (among all other vertices) that pass through that node. The higher the number, the more important is the node, in the sense that there is a small number of hops between that node and the majority of the network. Mathematically it can be defined as g(v) = s v t σ st (v) σ st where σ st is the total number of shortest paths from node s to node t and σ st (v) is the number of those paths that pass through v. The figure below shows that the betweenness centrality of nodes in the Clustering graph vary significantly (contrast this with the fact that the node degrees in this graph are almost identical to one another). 1 Clustering coefficient 0.5 0 Barabasi model Erdos Renyi model Full graph Clustering graph 60 40 20 0 20 Betweenness centrality Barabasi model Erdos Renyi model Full graph Clustering graph Figure 4: Clustering coefficient, betweenness centrality C.3 Macro Measures Graph density: In graph theory, graph density is defined as V ( V 1). Density represents the ratio of the number of edges to the maximum number of possible edges. Density will therefore have a value in the interval [0, 1]. Clustering coefficient of a graph: The clustering coefficient of an undirected graph is a measure of the number of triangles in a graph. The clustering coefficient for the whole graph is the average of the local clustering coefficients C i : C = 1 n C i n 11 i=1 2 E
where n is the number of nodes in the network. By definition 0 C i 1 and 0 C 1. Diameter: Diameter of the graph is the lenght of the longest shortest path between any two vertices of the graph. Macro level parameters for the four classes of networks are summarized in the table below. Note how the density and diameter of all graphs is almost identical, with the exception of the full graph, which has maximum density. graph type Density Clustering Diameter Number of edges Barabassi 0.195 0.208 4 37 Erdos-Renyi 0.211 0.158 4 40 Full graph 1 1 1 190 Clustering graph 0.179 0.714 5 34 12
D Evolution of Network States The figures below show samples of the detailed evolution of correct (blue) and incorrect answers (red) in a selection of network/question combinations. Figure 5: Evolution of the game (Barabasi-Albert, question 1) 13
Figure 6: Evolution of the game (Barabasi-Albert, question 3) 14
Figure 7: Evolution of the game (Full, question 1) 15
Figure 8: Evolution of the game (Full, question 2) 16
Figure 9: Evolution of the game (Erdos-Renyi, question 1) 17
References [1] Frederick S. Cognitive reflection and decision making. The Journal of Economic Perspectives. 2005;19(4):25 42. [2] Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia-Retamero R. Measuring risk literacy: The Berlin numeracy test. Judgment and Decision Making. 2012;7(1):25 47. 18