Recommendation Subgraphs for Web Discovery

Transcription

1 Reommation Subgraphs for Web Disovery Arda Antikaioglu Department of Mathematis Carnegie Mellon University R. Ravi Tepper Shool of Business Carnegie Mellon University Srinath Sridhar BloomReah In. ABSTRACT Reommations are entral to the utility of many popular e-ommere websites. Suh sites typially ontain a set of reommations on every produt page that enables visitors to easily navigate the website. These reommations are essentially universally present on all e-ommere websites. Choosing an appropriate set of reommations at eah page is a ritial task performed by dediated bak software systems. At BloomReah, an engine onsisting of several indepent omponents analyzes and optimizes its lients e-ommere website. This paper fouses on the struture optimizer omponent whih improves the website navigation experiene that enables the disovery of previously undisovered ontent by exposing them from popular pages. We formalize the onept of reommations used for disovery as a natural graph optimization problem on a bipartite graph: the left partition represent highly visited pages while the right represents rarely visited ones, and the edges are andidate reommation links that an be used. The goal is to pik at most a fixed number of out-links from eah left node to maximize the number of right nodes that are in-linked redundantly. The allowed out-degree of left nodes and minimum redundany of overage of right nodes are parameters defining the problem. In the speial ase when the in-link requirement is one, the problem redues to maximum bipartite mathing whih already requires superlinear time and is not salable. Also, implementing simple algorithms is ritial in pratie beause they are signifiantly easier to maintain in a prodution software pakage. This motivated us to analyze three methods for solving the problem in inreasing order of sophistiation: a loal random sampling algorithm, a greedy algorithm and a more involved partitioning based algorithm. We first theoretially analyze the performane of these three methods on random graph models and haraterize when eah method will yield a solution of suffiient quality and the parameter ranges when more sophistiation is needed. We omplement this by providing an empirial analysis of these algorithms on simulated and real-world prodution data from a retail website. Our results onfirm that it is not always neessary to implement ompliated algorithms in the real-world, and demonstrate that very good pratial This work was supported in part by an internship at Bloom- Reah In. Supported in part by NSF CCF Supported in part by NSF CCF results an be obtained by using simple heuristis that are baked by the onfidene of onrete theoretial guarantees. 1. INTRODUCTION 1.1 Web Relevane Engines The digital disovery divide [14] refers to the problem of ompanies not being able to present users with what they seek in the short time they sp looking for this information. The problem is prevalent not only in e-ommere websites but also in soial networks and miro-blogging sites where surfaing relevant ontent quikly is important for user engagement. BloomReah is a big-data marketing ompany that uses the lient s ontent as well as web-wide data to optimize both ustomer aquisition and satisfation for e-retailers. Bloom- Reah s lients inlude top internet retail ompanies from around the ountry. In this paper, we desribe the struture optimizer omponent of BloomReah s Web Relevane Engine. This omponent works on top of the reommation engine so as to arefully add a set of links aross pages that ensures that users an effiiently navigate the entire website. 1.2 Struture Optimization of Websites An important onern of retail website owners is whether a signifiant fration of the site is not reommed at all or hardly reommed) from other more popular pages within their site. One way to address this problem is to try to ensure that every page will obtain at least a baseline number of links from popular pages so that great ontent does not remain undisovered, and thus bridge the disovery divide mentioned above. If the website remains onneted, this also ensures a simple ondutane for the underlying link graph. We use this riterion of disoverability as the objetive for the hoie of the links to reomm. We start with a small set of already disovered or popular nodes available at a site, and want to use this set to make as many new nodes disoverable as possible. This objetive leads to a new strutural formulation of the reommation seletion problem. In partiular, we think of ommonly visited pages in a site as the already disovered pages, from whih there are a large number of possible reommations available using more traditional information retrieval methods) to related but less visited peripheral pages. The problem of hoosing a limited number of pages to reomm at eah disovered page an

2 be ast with the objetive of maximizing the number of peripheral non-visited pages that are redundantly linked. We formulate this as a reommation subgraph problem, and study pratial algorithms for solving these problems at sale with real-life data. 1.3 Reommation Systems as a Subgraph Seletion Problem Formally, we divide all pages in a site into two groups: the disovered pages and the undisovered ones. Furthermore, we assume that traditional reommation systems [1, 22, 23] provide us with a large set of related andidate undisovered page reommations for eah disovered page using relevane metris. In this work, we assume d suh related andidates are available per page reating a andidate reommation bipartite graph with degree d at eah disovered page node). Our goal is to analyze how to prune this set to < d reommations suh that globally we ensure that the number of undisovered pages that have at least a 1 reommations to them in the hosen subgraph. This gives the, a)-reommation subgraph introdued in Setion 3.1. Even though the ase of a = 1 redues to a polynomially solvable version of a mathing problem, the more usual ases of a > 1 are most likely NP-hard prohibiting exat solution methods at sale. Even the simple versions that redue to mathing are too omputational expensive on memory and proessing to run on real-life instanes 1.4 Our Contributions We introdue three simple heuristi methods that an be implemented in linear or near-linear time and thoroughly investigate their theoretial performane. In partiular, we delineate when eah method will work effetively on popular random graph models, and when a pratitioner will need to employ a more sophistiated algorithm. We then evaluate how these simple methods perform on simulated data, both in terms of solution quality and running time. Finally, we show the deployment of these methods on BloomReah s real-world lient link graph and measure their atual performane in terms of running-times, memory usage and auray. It is worthwhile to note that the simplest of the three methods that we propose sampling) an be easily adapted to the inremental dynami setting when the set of pages and andidate reommations is hanging rapidly. To summarize, our ontributions are as follows. 1. The development of a new strutural model for reommation systems as a subgraph seletion problem for maximizing disoverability Setion 3). 2. The proposal of three methods sampling, greedy and partition) with inreasing sophistiation to solve the problem at sale along with assoiated theoretial performane guarantee analyses Setion 4). In partiular, we show very strong theoretial bounds on the size of the disoverable set for the sampling algorithm in the fixed degree random graph model Theorem 1); in the Erdös-Renyi model for the greedy algorithm Theorem 7) and for a partition-based algorithm Theorem 10). 3. An empirial validation of our onlusions with simulated and real-life data Setion 5). Our simulations show that sampling is the least resoure intensive and performs satisfatorily, while partition is the most resoure intensive but performs better for small values of disoverability threshold a; Greedy is the overall best-performer using a single pass over the data and produing good results over a variety of parameters. In the tests with real retailer data, we see these trs broadly refleted in the results: Greedy performs well when gets moderately large giving almost optimal starting from a = 2. The partition method is promising when the targeted a value is low. Sampling is typially worse than greedy, but unlike the partition algorithm, its performane improves dramatially as beomes larger, and does not worsen as quikly when a gets larger. 2. RELATED WORK Reommation systems have been studied extensively in the literature, broadly separated into two different streams: ollaborative filtering systems and ontent-based reommer systems [2]. Muh attention has been foused on the former approah, where either users are lustered by onsidering the items they have onsumed or items are lustered by onsidering the users that have bought them. Both itemto-item and user-to-user reommation systems based on ollaborative filtering have been adopted by many industry giants suh as Twitter [11], Amazon [19] and Google [5]. Content based systems instead look at eah item and its intrinsi properties. For example, Pandora has ategorial information suh as Artist, Genre, Year, Singer, Tempo et. on eah song it indexes. Similarly, Netflix has a lot of ategorial data on movies and TV suh as Cast, Diretor, Produers, Release Date, Budget, et. This ategorial data an then be used to reomm new songs that are similar to the songs that a user has liked before. Deping on user feedbak, a reommer system an learn whih of the ategories are more or less important to a user and adjust its reommations. A drawbak of the first type of system is that is that they require multiple visits by many users so that a taste profile for eah user, or a user profile for eah item an be built. Similarly, ontent-based systems also require signifiant user partiipation to train the underlying system. These onditions are possible to meet for large ommere or entertainment hubs, but not very likely for most online retailers that speialize in a just a few areas, but have a long-tail [3] of produt offerings. Beause of this onstraint, in this paper we fous on a reommer system that typially uses many different algorithms that extrat ategorial data from item desriptions and uses this data to establish weak links between items andidate reommations). In the absene of other data that would enable us to hoose among these many links, we onsider every potential reommation to be of equal value and fous on the objetive of disovery, whih has not been studied before. In this way, our work differs from all the previous work on reommation systems that emphasize on finding reommations of high relevane and quality rather than on strutural navigability of the realized link struture. However, while it s not inluded in this paper for brevity, some of our approahes an be exted to the more general ase where different reommations have different weights See

3 Theorem 5). On the graph algorithms side, our problem is related to the bipartite mathing and more generally, the maximum b-mathing problems. There has been onsiderable work done in this area. In partiular, both the weighted mathing and b-mathing problems have exat polynomial time solutions [10]. Furthermore the mathing problem admits a near linear time 1 ɛ)-approximation algorithm [8], while the weighted b-mathing problem admits a 1/2-approximation algorithm [17]. However, all suh algorithms are based on ombinatorial properties of mathings and b-mathings, and do not arry over to the more important version of our problem when a > 1. Finally, our problem bears resemblane to some overing problems. For example, the maximum overage problem asks for the maximum number of elements that an be overed by a fixed number of sets and has a greedy 1 1/e)- approximation [21]. However, as mentioned earlier, our formulation requires multiple overage of elements. Furthermore note that the olletion of sets that an be used in the redundant overage are all possible subsets of out of the d andidate links, and is expressed impliitly in our problem. The urrently known theoretial methods for maximum overage heavily rely on the submodularity of the objetive funtion, whih our objetive doesn t satisfy. Hene the line of reent work on approximation algorithms for submodular maximization does not apply to our problems. 3. OUR MODEL We model the struture optimization of reommations by using a bipartite digraph, where one partition L represents the set of disovered i.e., often visited) items for whih we are required to suggest reommations and the other partition R representing the set of undisovered not visited) items that an be potentially reommed. If needed, the same item an be represented in both L and R. 3.1 The Reommation Subgraph Problem We introdue and study this as the the, a)- reommation subgraph problem in this paper: The input to the problem is the graph where eah L-vertex has d reommations. Given the spae restritions to display reommations, the output is a subgraph where eah vertex in L has < d reommations. The goal is to maximize the number of verties in R that have in-degree at least a target integer a. Note that if a = = 1 this is simply the maximum bipartite mathing problem [20]. If a = 1 and > 1, we obtain a b-mathing problem, that an be onverted to a bipartite mathing problem [10]. The typial and interesting ases when a > 1 is most likely NP-hard, ruling out the possibility of efiient exat algorithms. We now desribe typial web graph harateristis by disussing the sizes of L, R, and a in pratie. As noted before, in most websites, a small number of head pages ontribute to a signifiant amount of the traffi while a long tail of the remaining pages ontribute to the rest [7, 13, 18]. This is supported by our own experiene with the 80/20 rule, i.e. 80% of a site s traffi is aptured by 20% of the pages. Therefore, the ratio k = L / R is typially between 1/3 to 1/5, but may be even lower. From our own work at BloomReah and by observing reommations of Quora, Amazon, and YouTube), typial values for range from 3 to 20 reommations per page. Values of a are harder to nail down but it typially ranges from 1 to Pratial Requirements There are two key requirements in making graph algorithms pratial. The first is that the method used must be very simple to implement, debug, deploy and most importantly maintain long-term. The seond is that the method must sale graefully with larger sizes. Graph mathing algorithms require linear memory and super-linear run-time whih does not sale well. For example, an e-ommere website of a lient of BloomReah with 1M produt pages and 100 reommation andidates per produt would require easily over 160GB in main memory to store the graph and run exat mathing algorithms; this an be redued by using graph ompression tehniques but that adds more tehnial diffiulties in development and maintenane. Algorithms that are time intensive an sometimes be sped-up by using distributed omputing tehniques suh as map-redue [6]. However, effiient map-redue algorithms for graph problems are notoriously diffiult. Finally, all of these methods apply only to the speial ase of our problem when a = 1, leaving open the question of solving the more interesting and typial ases of redundant overage when a > Simple Approximation Algorithms To satisfy these pratial requirements, we propose the study of three simple approximate solutions strategies that not only an be shown to sale well in pratie but also have good theoretial properties that we demonstrate using approximation ratios. Sampling: The first solution is a simple random sampling solution that selets a random subset of links out of the available d from every page. Note that this solution requires no memory overhead to store these results a-priori and the reommations an be generated using a random number generator on the fly. While this might seem trivial at first, for suffiient and often real-world) values of and a we show that this an be optimal. Also, this method is very easy to adapt to the ase when the underlying graph is dynami with both nodes and edges hanging over time. Furthermore, our approah an be exted to the ase where the reommation edges have weights representing varying strengths of assoiation as is typially provided by the traditional methods that generate andidate reommation links 1. Greedy: The seond solution we propose is a greedy algorithm that hooses the reommation links so as to maximize the number of nodes in R that an aumulate a in-links. In partiular, we keep trak of the number of in-links required for eah node in R to reah the target of a and hoose the links from eah node in L giving preferene to adding links to nodes in R that are loser to the target in-degree a. 1 We omit a full desription of this result for brevity.

4 This method bears lose resemblane in strategy with greedy methods used for maximum overage and its more general submodular maximization variants. Partition: The third solution is inspired by a theoretially rigorous method to find optimal subgraphs in suffiiently dense graphs: it partitions the edges into a subsets by random sub-sampling, suh that there is a good hane of finding a perfet mathing from L to R in eah of the subsets. The union of the mathings so found will thus result in most nodes in R ahieving the target degree a. We require the number of edges in the underlying graph to be signifiantly large for this method to work very well; moreover, we need to run a near-)perfet mathing algorithm in eah of the edge-subsets whih is also a omputationally expensive subroutine. Hene, even though this method works very well in dense graphs, its resoure requirements may not sale well in terms of running time and spae. As a summary, the table below shows the time and spae omplexity of our different algorithms. Sampling Greedy Partition Time O E ) O E ) O E V ) Working Spae O1) OV ) O E ) Figure 1: Complexities of the different algorithms assuming onstant a and ) In the next setion, we elaborate on these methods, their running times, implementation details, and theoretial performane guarantees. In the setion after that, we present our omprehensive empirial evaluations of all three methods, first the results on simulated data and then the results on real data from some lients of BloomReah. 4. ALGORITHMS FOR RECOMMENDA- TION SUBGRAPHS 4.1 The Sampling Algorithm We present the sampling algorithm for the, a)- reommation subgraph formally below. Data: A bipartite graph G = L, R, E) Result: A, a)-reommation subgraph H for u in L do S a random sample of verties without replaement in Nu); for v in S do H H {u, v)}; return H; Algorithm 1: The sampling algorithm Given a bipartite graph G, the algorithm has runtime omplexity of O E ) sine every edge is onsidered at most one. The spae omplexity an be taken to be O1), sine the adjaeny representation of G an be assumed to be pre-sorted by the point of eah edge in L. We next introdue a simple random graph model for the supergraph from whih we are allowed to hoose reommations and present a bound on its expeted performane when the underlying supergraph G = L, R, E) is hosen probabilistially aording to this model. Fixed Degree Model: In this model for generating the andidate reommation graph, eah vertex u L uniformly and indepently samples d neighbors from R with replaement. While this allows eah vertex in L to have the same vertex as a neighbor multiple times, in reality r d is so edge repetition is very unlikely. This model is similar to, but is distint from the more ommonly known Erdös-Renyi model of random graphs [15]. In partiular, while the degree of eah vertex in L is fixed under our model, onentration bounds an show that the degrees of the verties in L would have similarly been onentrated around d for p = d/r in the Erdös-Renyi model. We prove the following theorem about the performane of the Sampling Algorithm. We denote the ratio of the size of L and R by k, i.e., we define k = l r. Theorem 1. Let S be the random variable denoting the number of verties v R suh that deg H v) a in the fixed-degree model. Then k+ E[S] r 1 e r ) k) a 1 k 1 To get a quik sense of the very good performane bounds refleted in this theorem, please see Figure 2 that plots the approximation ratio as a funtion of k for the, 1)- reommation subgraph problem, as well as Figure 3 that shows how large needs to be in terms of k) for the solution to be 95% optimal for different values of a, both in the fixed degree model. Proof. We will analyze the sampling algorithm as if it piks the neighbors of eah u L with replaement, the same way the fixed-degree model generates G. This variant would obviously waste some edges, and perform worse than the variant whih samples neighbors without replaement. This means that any performane guarantee we prove for this variant holds for our original statement of the algorithm as well. To prove the laim let X v be the random variable that represents the degree of the vertex v R in our hosen subgraph H. Beause our algorithm uniformly subsamples a uniformly random seletion of edges, we an assume that H was generated the same way as G but sampled instead of d edges for eah vertex u L. Sine there are l edges in H that an be inident on v, and eah of these edges has a 1/r probability of being inident on a given vertex in L, we an now alulate that ) l Pr[X v = i] = i r )l i r l) i 1 1 r )l i 1 r Using a union bound, we an ombine these inequalities to upper bound the probability that deg H v) < a. ) i ) i

5 ) l Pr[X v < a] = 1 1 ) l i ) i 1 i r r ) i l 1 1 ) l i r r 1 1 ) l ) k) i r 1 1 ) l ) k) a 1 r k 1 k+ k) a 1 e r k 1 Letting Y v = [X v a], we now see that E[S] = E [ v R ] k+ Y v r 1 e r ) k) a 1 k 1 We an ombine this lower bound with a trivial upper bound to obtain an approximation ratio that holds in expetation. Theorem 2. The above sampling algorithm gives a 1 1 e ) - fator approximation to the, 1)-graph reommation problem in expetation. Proof. The size of the optimal solution is bounded above by both the number of edges in the graph and the number of verties in R. The former of these is l = kr and the latter is r, whih shows that the optimal solution size OP T r mink, 1). Therefore, by simple ase analysis the approximation ratio in expetation is at least 1 exp k))/ mink, 1) 1 1 e For the, 1)-reommation subgraph problem the approximation obtained by this sampling approah an be muh better for ertain values of k. In partiular, if k > 1, then the approximation ratio is 1 exp k), whih approahes 1 as k. When k = 3, then the solution will be at least 95% as good as the optimal solution even with our trivial bounds. Similarly, when k < 1, the approximation ratio is 1 exp k))/k whih also approahes 1 as k 0. In partiular, if k = 0.1 then the solution will be at 95% as good as the optimal solution. The ase when k = 1 represents the worst ase outome for this model where we only guarantee 63% optimality. Figure 2 shows the approximation ratio as a funtion of k for the, 1)-reommation subgraph problem in the fixed degree model. Figure 2: Approx ratio as a funtion of k For the general, a)-reommation subgraph problem, if k > a, then the problem is easy on average. This is in omparison to the trivial estimate of l. For a fixed a, a random solution gets better as k inreases beause the derease in e k more than ompensates for the polynomial in k next to it. However, in the more realisti ase, the undisovered pages in R too numerous to be all overed even if we used the full set of budgeted links allowed out of L, i.e. l < ra or rearranging, k < a; in this ase, we need to use the trivial estimate of kr/a, and the analysis for a = 1 does not ext here. For pratial purposes, the table in Figure 3 shows how large needs to be in terms of k) for the solution to be 95% optimal for different values of a, again in the fixed degree model. a k k k k k 1 Figure 3: The required k to obtain 95% optimality for, a)- reommation subgraph We lose out this setion by showing that the main result that holds in expetation also hold with high probability for a = 1, using the following variant of Chernoff bounds. Theorem 3. [4] Let X 1,..., X n be non-positively orrelated variables. If X = n Xi, then for any δ 0 ) e δ E[X] Pr[X 1 + δ)e[x]] 1 + δ) 1+δ Theorem 4. Let S be the random variable denoting the number of verties v R suh that deg H v) 1. Then S r1 2 exp k)) with probability at most e/4) r1 exp k)). Proof. We an write S as v R 1 Xv) where Xv is the indiator variable that denotes that X v is mathed. Note that the variables 1 X v for eah v R are non-positively orrelated. In partiular, if Nv) and Nv ) are disjoint, then 1 X v and 1 X v are indepent. Otherwise, v not laiming any edges an only inrease the probability that v has an edge from any vertex u Nv) Nv ). Also note that the expeted size of S is r1 exp k)) by Theorem 1. Therefore, we an apply Theorem 3 with δ = 1 to obtain the result. For realisti senarios where r is very large, the above theorem gives very tight bounds on the size of the solution, also explaining the effetiveness of the simple sampling algorithm in suh instanes. The results presented in this setion an be naturally exted to weighted models as shown by the theorem below. The proof is left out due to spae onstraints. Theorem 5. Let G = K l,r be a omplete bipartite graph where the edges have i.i.d. weights and ome from a distribution with mean µ that is supported on [0, b]; Assume that kµ 1 + ɛ for some ɛ > 0. If the algorithm from Setion 4.1 is used to sample a subgraph H from G and S is the set of verties in R of inident weight at least one, then E[S] = )) E[X v] = r 1 exp 2lɛ2 v R Further appliations of the sampling method to different random graph models an be found in the Appix so as to not break the flow of this paper. b 2

6 4.2 The Greedy Algorithm We next analyze the natural greedy algorithm for onstruting a, a)-reommation subgraph H iteratively. In the following algorithm, we use Nu) to refer to the neighbors of a vertex u. Data: A bipartite graph G = L, R, E) Result: A, a)-reommation subgraph H for u in L do d[u] 0 for v in R do F {u Nv) d[u] < }; if F a then restrit F to a elements; for u in F do H H {u, v)}; d[u] d[u] + 1; return H; Algorithm 2: The greedy Algorithm The algorithm loops through eah vertex in R, and onsiders eah edge one. Therefore, the runtime is Θ E ). Furthermore, the only data struture we use is an array whih keeps trak of deg H u) for eah u L, so the memory onsumption is Θ L ). Finally, we prove the following tight approximation property of this algorithm. Theorem 6. The greedy algorithm gives a 1/a + 1)- approximation to the, a)-graph reommation problem. Proof. Let R GREEDY, R OP T R be the set of verties that have degree a in the greedy and optimal solutions respetively. Note that any v R OP T along with neighbors {u 1,... u a} forms a set of andidate edges that an be used by the greedy algorithm. Eah seletion of the greedy algorithm might result in some andidates beoming infeasible, but it an ontinue as long as the andidate pool is not depleted. Eah time the greedy algorithm selets some vertex v R with edges to {u 1,..., u a}, we remove v from the andidate pool. Furthermore eah u i ould have degree in the optimal solution and used eah of its edges to make a neighbor attain degree a. The greedy hoie of an edge to u i requires us to remove suh an edge to an arbitrary vertex v i R adjaent to u i in the optimal solution, and thus remove v i from further onsideration in the andidate pool. Therefore, at eah step of the greedy algorithm, we may remove at most a + 1 verties from the andidate pool as illustrated in Figure 4. Sine our andidate pool has size OP T, the greedy algorithm an not stop before it has added OP T/a + 1) verties to the solution. Figure 4: One step of the greedy algorithm. When v selets edges to u 1,..., u a, it an remove v 1,..., v a from the pool of andidates that are available. The potentially invalidated edges are shown in red. This approximation guarantee is as good as we an expet, sine for a = 1 we reover the familiar 1/2-approximation of the greedy algorithm for mathings. Furthermore, even in the ase of mathings a = 1), randomizing the order in whih the verties are proessed is still known to leave a onstant fator gap in the quality of the solution [16]. Despite this result, the greedy algorithm fares muh better when we analyze its expeted performane. Swithing to the Erdös- Renyi model [9] instead of the fixed degree model used in the previous setion, we now prove the near optimality of the greedy algorithm for the, a)-reommation subgraph problem. Reall that in this model sometimes referred to as G n,p), eah possible edge is inserted with probability p indepent of other edges. In our version G l,r,p, we only add edges from L to R eah with probability p indepent of other edges in this omplete bipartite andidate graph. For tehnial reasons, we need to assume that lp 1 in the following theorem. However, this is a very weak assumption sine lp is simply the expeted degree of a vertex v R. Typial values for p for our appliations will be Ωlogl)/l) making the expeted degree lp = Ωlog l). Theorem 7. Let G = L, R, E) be a graph drawn from the G l,r,p where lp 1. If S is the size of the, a)- reommation subgraph produed by the greedy algorithm, then: E[S] r alp) r 1 1 p) l ia 1 p) a When the underlying random graph is suffiiently dense, Theorem 8 shows that the above guarantee is asymptotially optimal. Proof. Note that if edges are generated uniformly, we an onsider the graph as being revealed to us one vertex at a time as the greedy algorithm runs. In partiular, onsider the event X i+1 that the greedy algorithm mathes the i + 1) st vertex it inspets. While, X i+1 is depent on X 1,..., X i, the worst ondition for X i+1 is when all the previous i verties were from the same verties in L, whih are now not available for mathing the i + 1) st vertex. The maximum number of suh invalidated verties is at most ia/. Therefore, the bad event is that we have fewer than a of the at least l ia/ available verties having an edge to this vertex. The probability of this bad event is at most Pr[Y Binl ia, p) : Y < a], the probability that a Binomial random variable with l ia trials of probability p of

7 suess for eah trial has less than a suesses. We an bound this probability by using a union bound and upper-bounding Pr[Y Binl ia, p) : Y = t] for eah 0 t a 1. By using the trivial estimate that n i) n i for all n and i, we obtain: l Pr[Y Binl ia, p) : Y = t] = ia 1 p) l ia t p t t l ia ) t l ia 1 p) t p t ) lp) t 1 p) l ia t Notie that the largest exponent lp an take within the bounds of our sum is a 1. Similarly, the smallest exponent 1 p) an take within the bounds of our sum is l ia a+1. Now applying the union bound gives: Pr[Y Binl ia, p) : Y < a] Pr[Y Binl ia, p) : Y = t] t=0 lp) t 1 p) l ia t t=0 = alp) 1 p) l ia a+1 Finally, summing over all the X i using the linearity of expetation and this upper bound, we obtain r 1 E[S] r E[ X i] r 1 r Pr[Y Binl ia r 1 r alp) 1 p) l ia a+1, p) : Y < a] Asymptotially, this result explains why the greedy algorithm does muh better in expetation than 1/a + 1) guarantee we an prove in the worst ase. In partiular, for a reasonable setting of the right parameters, we an prove that the error term of our greedy approximation will be sublinear. Theorem 8. Let G = L, R, E) be a graph drawn from the G l,r,p where p = γ log l for some γ 1. Suppose that, a l and ɛ > 0 are suh that l = 1 + ɛ)ra and that l and r go to infinity while satisfying this relation. If S is the size of the, a)-reommation subgraph produed by the greedy algorithm, then E[S] r or) Proof. We will prove this laim by applying Theorem 7. Note that it suffies to prove that lp) r 1 ia 1 p)l = or) sine the other terms are just onstants. We first bound the elements of this summation. Using the fats that p = γ log l, l/a = 1 + ɛ)r and that i < r throughout the l summation, we get the following bound on eah term: 1 p) l ia 1 γ log l l exp γ log l l = exp log l) ) l ia l ia )) γ ia )) l γ+ ia γ+ = l l = l i 1+ɛ)r l ɛ = l ɛ 1+ɛ Finally, we an evaluate the whole sum: lp) r 1 1 p) l ia log l ) r 1 log l ) rl l 1+ɛ ɛ 1+ɛ ɛ = log l ) 1 + ɛ)a l1 ɛ 1+ɛ = ol) However, sine r is a onstant times l, any funtion that is ol) is also or) and this proves the laim. 4.3 The Partition Algorithm To motivate the partition algorithm, we first define optimal solutions for the reommation subgraph problem. Perfet Reommation Subgraphs: We define a perfet, a)-reommation subgraph on G to be a subgraph H suh that deg Hu) for all u L and deg Hv) = a for minr, l/a ) of the verties in R. The reason we define perfet, a)-reommation subgraphs is that when one exists, it s possible to reover it in polynomial time using a min-ost b-mathing algorithm mathings with a speified degree b on eah vertex) for any setting of a and. However, implementations of b-mathing algorithms often inur signifiant overheads even over regular bipartite mathings. This motivates a solution that uses regular bipartite mathing algorithms to find an approximately optimal solution given that a perfet one exists. We do this by proving a suffiient ondition for perfet, a)- reommation subgraphs to exist with high probability in a bipartite graph G under the Erdös-Renyi model [9] where edges are sampled uniformly and indepently with probability p. This argument then guides our formulation of a heuristi that overlays mathings arefully to obtain, a)-reommation subgraphs. Theorem 9. [15] Let G be a bipartite graph drawn from log n log log n G n,n,p. If p, then as n, the probability n that G has a perfet mathing approahes 1. We will prove that a perfet, a)-reommation subgraph exists in random graphs with high probability by building it up from a mathings eah of whih must exist with high probability if p is suffiiently high. To find these mathings, we identify subsets of size l in R that we an perfetly math to L. These subsets overlap, and we hoose them so that eah vertex in R is in a subsets.

8 Theorem 10. Let G be a random graph drawn from G l,r,p log l log log l with p a then the probability that G has a perfet, a)-reommation subgraph ts to 1 as l, r l. This theorem guarantees the existene of an optimal reommation subgraph in suffiiently dense subgraphs, and provides a onstrutive proof of this fat that is also the basis of our partition algorithm. Proof. We start by either padding or restriting R to a set of l before we start our analysis. If r l, then a a we restrit R to an arbitrary subset R of size l. Sine indued subgraphs of Erdös-Renyi graphs are also Erdös-Renyi a graphs, we an instead apply our analysis to the indued subgraph. Sine the optimal solution has size bounded above by l a perfet, a)-reommation subgraph in G[L, a R ] will imply a perfet reommation subgraph in G[L, R]. On the other hand, if r l, then we an pad R with a l r dummy verties and adding an edge from eah suh a vertex to eah vertex in L with probability p. We all the resulting right side of the graph R. Note that G[L, R ] is still generated by the Erdös-Renyi proess. Further, sine the original graph G[L, R] is a subgraph of this new graph, if we prove the existene of a perfet, a)-reommation subgraph in this new graph, it will imply the existene of a perfet reommation subgraph in G[L, R]. Having piked an R satisfying R = l, we pik an enumeration of the verties in R = {v 0,..., v l/ } and add a eah of these verties into a subsets as follows. Define R i = {v i 1)l/a,..., v i 1)l/a+l 1 } for eah 1 i where the arithmeti in the indies is done modulo l/a. Note both L and all of the R i s have size l. Using these new sets we define the graphs G i on the bipartitions L, R i). Sine the sets R i are interseting, we annot define the graphs G i to be indued subgraphs. However, note that eah vertex v R falls into exatly a of these subsets. Therefore, we an uniformly randomly assign eah edge in G to one of a graphs among {G 1,..., G } it an fall into, and make eah of those graphs a random graph. In fat, while the different G i are oupled, taken in isolation we an onsider any single G i to be drawn from the distribution G l,l,p/a sine G was drawn from G l,r,p. Sine p/a log l log log l)/l by assumption, we onlude by Theorem 9, the probability that a partiular G i has no perfet mathing is o1). If we fix, we an onlude by a union bound that exept for a o1) probability, eah one of the G i s has a perfet mathing. By superimposing all of these perfet mathings, we an see that every vertex in R has degree a. Sine eah vertex in L is in exatly mathings, eah vertex in L has degree. It follows that exept for a o1) probability there exists a, a)-reommation subgraph in G. Approximation Algorithm Using Perfet Mathings: The above result now enables us to design a near linear time algorithm with a 1 ɛ) approximation guarantee to the, a)-reommation subgraph problem by leveraging ombinatorial properties of mathings. In partiular, we use the fat a mathing that does not have augmenting paths of length > 2α is a 1 1/α approximation to the maximum mathing problem. We all this method the Partition Algorithm, and we outline it below. Data: A bipartite graph G = L, R, E) Result: A,a)-reommation subgraph H R a random sample of L /a verties from R; Choose G[L, R 1],..., G[L, R ] as in Theorem 10; for i in [1..n] do M i A mathing of G[L, R i] with no augmenting path of length 2/ɛ; H M 1... M; return H; Algorithm 3: The partition algorithm Theorem 11. Let G be drawn from G l,r,p where p log l log log l a. Then Algorithm 3 finds a 1 ɛ)- l approximation in O E ) time with probability 1 o1). ɛ Proof. Using the previous theorem, we know that eah of the graphs G i has a perfet mathing with high probability. These perfet mathings an be approximated to a 1 ɛ/ fator by finding mathings that do not have augmenting paths of length 2/ɛ [20]. This an be done for eah G i in O E /ɛ) time. Furthermore, the union of unmathed verties makes up an at most ɛ/) fration of R, whih proves the laim. Notie that if we were to run the augmenting paths algorithm to ompleteness for eah mathing M i, then this algorithm would take O E L ) time. We ould redue this further to O E L) by using Hoproft-Karp. [12] Assuming a sparse graph where E = Θ L log L ), the time omplexity of this algorithm is Θ L 3/2 log L ). The spae omplexity is only Θ E ) = Θ L log L ), but a large onstant is hidden by the big-oh notation that makes this algorithm impratial in real test ases. 5. EXPERIMENTAL RESULTS 5.1 Simulated Data We simulated performane of our algorithms on random graphs generated by the graph models we outlined. In the following figures, eah data point is obtained by averaging the measurements over 100 random graphs. We first present the time and spae usage of these algorithms when solving a 10, 3)-reommation subgraph problem in different sized graphs. In all our harts, error bars are present, but too small to be notieable. Note that varying the value of a and would only hange spae and time usage by a onstant, so these two graphs are indiative of time and spae usage over all ranges of parameters. The ode used ondut these experiments an be found at https: //github.om/srinathsridhar/graph-mathing-soure Reall that the partition algorithm split the graph into multiple graphs and found mathings using an implementation of Hoproft-Karp [12]) in these smaller graphs whih were then ombined into a reommation subgraph. For this reason, a run of the partition algorithm takes muh longer to solve a problem instane than either the sampling or greedy algorithms. It also takes signifiantly more memory as an be seen in Figures 5 and 6. Compare this to greedy and sampling whih both require a single pass over the graph, and no advaned data strutures. In fat, if the edges of G

9 Figure 5: Time needed to solve a 10,3)-reommation problem in random graphs where R / L = 4 Notie the log-log sale.) Figure 7: Solution quality for the, 1)-reommation subgraph problem in graphs with L = 25k, R = 100k, d = 20 Figure 6: Spae needed to solve a 10,3)-reommation problem in random graphs where R / L = 4 Notie the log-log sale.) is pre-sorted by the edge s point in L, then the sampling algorithm an be implemented as an online algorithm with onstant spae and in onstant time per link seletion. Similarly, if the edges of G is pre-sorted by the edge s point in R, then the greedy algorithm an be implemented so that the entire graph does not have to be kept in memory. In this event, greedy uses only O L ) memory. Next, we analyze the relative qualities of the solutions eah method produes. Figures 7 and 8 plot the average performane ratio of the three methods ompared to the trivial upper bounds as the value of, the number of reommations allowed is varied, while keeping a = 1. They olletively show that the lower bound we alulated for the expeted performane of the sampling algorithm aurately aptures its behavior when a = 1. Indeed, the inequality we used is an aurate approximation of the expetation, up to lower order terms, as is demonstrated in these simulated runs. The random sampling algorithm does well, both when is low and high, but falters when k = 1. The greedy algorithm outperforms the sampling algorithm in all ases, but its advantage vanishes as gets larger. Note that the dip in the graphs when l = ar, at = 4 in Figure 7 and = 2 in Figure 8 is expeted and was previously demonstrated in Figure 2. The partition algorithm is immune to this drop that affets both the greedy and the sampling algorithms, but omes with the ost of higher time and spae utilization. In ontrast to the ase when a = 1, the sampling algorithm performs worse when a > 1 but performs inreasingly better with as demonstrated by Figures 9 and 10. The greedy algorithm ontinues to produe solutions that are nearly optimal, regardless of the settings of and a, even beating the partition algorithm with inreasing values of a. Our simulations suggest that in most ases, one an simply use our sampling method for solving the, a)-reommation sub- Figure 8: Solution quality for the, 1)-reommation subgraph problem in graphs with L = 50k, R = 100k, d = 20 graph problem. In ases where the sampling is not suitable as flagged by our analysis, we still find that the greedy performs adequately and is also simple to implement. These two algorithms thus onfirm to our requirements we initially laid out for deployment in large-sale real systems in pratie. To summarize, our syntheti experiments show the following strengths of eah algorithm: Sampling Algorithm: Sampling uses little to no memory and an be implemented as an online algorithm. If keeping the underlying graph in memory is an issue, then hanes are this algorithm will do well while only needing a fration of the resoures the other two algorithms would need. Partition Algorithm: This algorithm does well, but only when a is small. In partiular, when a = 1 or 2, partition seems to be the best algorithm, but the quality of the solutions degrade quikly after that point. However this performane omes at expense of signifiant runtime and spae. Sine greedy performs almost as well without requiring large amounts of spae or time, partition is best suited for instanes where a is low the quality of the solution is more important than anything else. Greedy Algorithm: This algorithm is the all-round best performing algorithm we tested. It only requires a single pass over the data thus very quikly, and uses relatively little amounts of spae enabling it run ompletely in memory for graphs with as many as tens of millions of edges. It is not as fast as sampling or aurate as partition when a is small, but it has very good performane over all parameter ranges. 5.2 Real Data We now present the results of running our algorithms on several real datasets. In the graphs that we use, eah node orresponds to a single produt in the atalog of a merhant and the edges onnet similar produts. For eah produt up to 50 most similar produts were seleted by a proprietary algorithm of BloomReah that uses text-based features suh as keywords, olor, brand, ger where appliable) as well

10 Figure 9: Solution quality for the, 2)-reommation subgraph problem in graphs with L = 50k, R = 100k, d = 20 Figure 11: Solution quality for the, 1)-reommation subgraph problem in retailer data Figure 10: Solution quality for the, 4)-reommation subgraph problem in graphs with L = 50k, R = 100k, d = 20 as user browsing patterns to determine the similarity between pairs of produts. Suh algorithms are ommonly used in e-ommere websites suh as Amazon, Overstok, ebay et to display the most related produts to the user when they are browsing a speifi produt. Two of the lient merhants of BloomReah presented here had moderate-sized relation graphs with about 10 5 verties and 10 6 input edges andidate reommations); the remaining merhants 3, 4 and 5) have on the order of 10 6 verties and 10 7 input edges between them. We estimated an upper bound on the optimum solution by taking the minimum of L /a and the number of verties in R of degree at least a. Figures 11, 12 and 13 plot the average of the optimality perentage of the sampling, greedy and partition algorithms aross all the merhants respetively. Note that we ould only run the partition algorithm for the first two merhants due to memory onstraints. From these results, we an see that that greedy performs exeptionally well when gets even moderately large. For the realisti value of = 6, the greedy algorithm produed a solution that was 85% optimal for all the merhants we tested. For several of the merhants, its results were almost optimal starting from a = 2. The partition method is also promising, espeially when the a value that is targeted is low. Indeed, when a = 1 or a = 2, its performane is omparable or better than greedy, though the differene is not as pronouned as it is in the simulations. However, for larger values of a the partition algorithm performs worse. The sampling algorithm performs mostly well on real data, espeially when is large. It is typially worse than greedy, but unlike the partition algorithm, its performane improves dramatially as beomes larger, and its performane does not worsen as quikly when a gets larger. Therefore, for large sampling beomes a viable alternative to greedy mainly in ases where the linear memory ost of the greedy algorithm is too prohibitive. Figure 12: Solution quality for the, 2)-reommation subgraph problem in retailer data Figure 13: Solution quality for the, 3)-reommation subgraph problem in retailer data 6. GENERALIZED MODELS OF RECOM- MENDATION GRAPHS In this Setion, we throw some light into the theoretial haraterization of some generalizations of the uniform model whih is likely to have greater pratial appliations. Even though we studied the fixed-degree uniform model in detail, reommation systems based on relevane in pratie will not have edges that are spread uniformly at random. Items that are about speifi topis are muh more likely to interlink within themselves than to those outside that topi, leading to lusters of reommations. To understand this lustering in underlying graphs in pratie, we ompiled results from several e-ommere retailers that have been aggregated and anonymized in the table shown below. For eah retailer, we ompiled the produt ontology present within the site that plaes a produt in this tree-like ategorization. E.g., a juier alled Breville Juie Fountain Plus is in the tree path: Home Juiers High Speed Juiers Breville Juie Fountain Plus. We then examined the reommations from produts at different depths of the hierarhy. In the table in Figure 14 we show

11 the edges adjaent to produts at depth 4 or greater. We alulated the perentage of edges onneting to produts that had different least ommon anestors LCA) with the urrent produt. We then randomized the edges so that we an ompare how the graph would have looked if there was no lusters and re-alulated the distribution of the edges and the LCA levels. We notied that the uniform distribution had edges that had very shallow LCA indiating that most edges did not follow the produt hierarhy while in reality, the points of edges reommed had muh deeper LCA meaning reommation edges were lustered based on the produt hierarhy. This led us to formalize this new model of input graphs that we study in Subsetion 6.1 as the hierarhial tree model. LCALevel Uniform Hierarhial Figure 14: Perent edges for depth-4 produts by LCA of points in reality from hierarhial data) and simulated uniform distribution of edges. In a seond analysis, we simply trunated the produt hierarhy at depth 3 and olleted the resulting disjoint lusters in the hierarhy. We then examined all the reommations and partitioned them into those going between eah pair of these lusters. In a uniform distribution, we would expet theedges to be equally likely to span aross eah pair of lusters if lusters are equal sized). But what we observed was that different pairs of lusters had different edge-densities. For instane, an Espresso Mahine might point more to other Coffee Mahines or Coffee Beans note that Coffee Beans and Espresso Mahine might share no LCA apart from the root) than to other lusters. These results are shown in Figure 15 whih learly demonstrates that the pairs of lusters responsible for the most number of reommation edges produe many more edges than the uniform model would predit. This motivated us to define and study the artesian produt model in Subsetion 6.2 whih is orthogonal to the uniform and hierarhial tree models. The way we performed the analyses for these new models are similar to those that we arried out for the uniform model. While we only present results for the approximation of, 1)-reommation subgraphs for brevity, these results an be exted to the more general problem of finding, a)-reommation subgraphs as done in Setion Hierarhial Tree Model In this model, the vertex sets L and R are the leaf sets of two trees T L and T R of depth D where there is a 1-to-1 orrespondene between the subtrees of these two trees. We also assume that eah branhing in both T L and T R splits the nodes evenly into the two subtrees. As in the previous setions, we set L / R = k, and require that this ratio is still k if we divide the size of any subtree on the left and that of its orresponding subtree on the right. For simpliity of notation, we will use a subtree and its leaf set interhangeably. We assume that the trees are fixed in advane but the bipartite reommation graph G = L, R, E) is generated probabilistially aording to the following proedure. Let u L and TL, 0... T D 1 L be the subtrees it belongs at depths 0,..., D 1. Also, let TR, 0..., T D 1 R be the subtrees on the right that orrespond to these trees on the left. We Figure 15: Histogram of perent edges between pairs of lusters. Eah point on x-axis is a pair of lusters. The x-axis has no inherent order but they have been sorted by number of edges for easier visualization. The tail is omitted. let u make a reomming edge to d D 1 of the verties in T D 1 R, d D 2 edges to the verties in T D 2 R \T D 1 R and so on. The d i edges out of u are hosen uniformly from TR i \ T i+1 R. Let d = d d D 1. Figure 16: This diagram shows the notation we use for this model and the 1-to-1 orrespondene of subtrees. For the a = 1 ase, our goal now is to find a b-mathing [10] in this graph that is lose to optimal in expetation. That is, our degree upper and lower bounds on verties in L and R are and 1 respetively. Let = D 1 be similar to how we defined d. To ombine the analysis of the randomness of the algorithm and the randomness of the graph, the algorithm will pik i edges uniformly from among the d i edges going to eah level of the subtree to form a, 1)- reommation subgraph H. This enables us to think of H as being generated by the same proess that generated G but with fewer neighbors seleted. With this model and parameters in plae, we an have the following analog of our main theorem for a = 1 for the hierarhial model. Theorem 12. Let S be the subset of edges v R suh that deg H v) 1 in the hierarhial tree model. Then E[S] r1 exp k)) Proof. Let v R and let T D 1 L, T D 2 L \T D 1 L,..., TL\T 0 L 1 be the sets it an take edges from. Sine T L and T R split perfetly evenly at eah node the verties in these sets will be hosen from r D 1, r D 1, r D 2,..., r 1 verties in R as neighbors, where r i is the size of subtree of the right tree rooted at depth i. Furthermore, eah of these sets desribed above have size l D 1, l D 1, l D 2,..., l 1 respetively, where l i is the size of a subtree of T L rooted at depth i. It follows that the probability that v does not reeive any edges at all is at most