Your Neighbors Affect Your Ratings: On Geographical Neighborhood Influence to Rating Prediction

Transcription

1 Your Neghbors Affect Your Ratngs: On Geographcal Neghborhood Influence to Ratng Predcton Longke Hu Axn Sun Yong Lu School of Computer Engneerng, Nanyang Technologcal Unversty, Sngapore ABSTRACT Ratng predcton s to predct the preference ratng of a user to an tem that she has not rated before. Usng the busness revew data from Yelp, n ths paper, we study busness ratng predcton. A busness here can be a restaurant, a shoppng mall or other knd of busnesses. Dfferent from most other types of tems that have been studed n varous recommender systems (e.g., move, song, book), a busness physcally exsts at a geographcal locaton, and most busnesses have geographcal neghbors wthn walkng dstance. When a user vsts a busness, there s a good chance that she walks by ts neghbors. Through data analyss, we observe that there exsts weak postve correlaton between a busness s ratngs and ts neghbors ratngs, regardless of the categores of busnesses. Based on ths observaton, we assume that a user s ratng to a busness s determned by both the ntrnsc characterstcs of the busness and the extrnsc characterstcs of ts geographcal neghbors. Usng the wdely adopted latent factor model for ratng predcton, n our proposed soluton, we use two knds of latent factors to model a busness: one for ts ntrnsc characterstcs and the other for ts extrnsc characterstcs. The latter encodes the neghborhood nfluence of ths busness to ts geographcal neghbors. In our experments, we show that by ncorporatng geographcal neghborhood nfluences, much lower predcton error s acheved than the state-of-the-art models ncludng Based MF, SVD++, and Socal MF. The predcton error s further reduced by ncorporatng nfluences from busness category and revew content. Categores and Subject Descrptors H.3.3 [Informaton Systems]: Informaton Search and Retreval Informaton Flterng Keywords Recommendaton; Ratng predcton; Matrx factorzaton; Yelp 1. INTRODUCTION Recommender systems have attracted sgnfcant attenton from both academa and ndustry snce the last decade. Varous recom- Permsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. Copyrghts for components of ths work owned by others than the author(s) must be honored. Abstractng wth credt s permtted. To copy otherwse, or republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. Request permssons from permssons@acm.org. SIGIR 14, July 6 11, 014, Gold Coast, Queensland, Australa. Copyrght s held by the owner/author(s). Publcaton rghts lcensed to ACM. ACM /14/07...$ mender systems have been developed to facltate the matchng between consumers (.e., users) wth approprate products or servces (.e., tems). Example tems nclude songs, moves from content provders as well as books from E-commerce webstes. Recently, recommender systems have also been appled to socal network platforms (e.g., Facebook, Twtter, Lnkedn) for people recommendaton, e.g., recommendng frends to a user or recommendng who to follow. The prevalence of GPS-enabled devces (e.g., smart phones) n the past few years further extends the landscape of recommender systems n locaton-based socal networks (LBSN), exemplfed by Foursquare and Gowalla. Dependng on the applcaton, dfferent recommendaton problems have been defned and studed. The top-n tem recommendaton and ratng predcton are two most wdely studed categores of recommendaton problems. On the one hand, top-n tem recommendaton tasks am to recommend a user a lst of tems that she may be nterested n. For example, n LBSN, POI recommendaton ams to recommend unvsted POIs to users. Here POI stands for pont-of-nterest, referrng to a focused geographc entty such as dstrct and street, or a specfc pont locaton such as landmarks and restaurants [0]. Ratng predcton, on the other hand, s to predct the preference ratng of a user to a product or servce that she has not rated before. The products or servces wth hgh predcted ratngs are recommended to users. In ths study, we are nterested n the busness ratng predcton problem wth busness revew data from Yelp. Yelp s a busness revew ste and has attracted 47 mllon revews to local busnesses snce 004. Example busnesses n Yelp nclude restaurants, shoppng malls, beauty & spas, etc.. The webste reports that t had an average of approxmately 117 mllon monthly unque vstors n the thrd quarter of A Yelp user or Yelper can share her experence wth a busness by postng a revew of the busness and also a ratng from 1 to 5 stars. Our task s therefore to predct how many stars a user would gve to a busness that she has yet revewed. At frst glance, ratng predcton of busness s the same as predcton of user s ratng to any other knd of tems (e.g., a song or a move), wth the only dfference that the tem here refers to a busness. The key dfference, between a busness and other knd of tems that have been studed n lterature, s that a busness physcally exsts at a specfc geo-locaton wth lattude/longtude coordnates. More mportantly, most busnesses (e.g., restaurants and shops) are not geographcally solated from others. That s, when a user vsts a busness, there s a good chance that she walks by ts neghbors f they are located wthn walkng dstance. The overall envronment of that regon mght affect a user s vew about a busness and subsequently affect user s revew and ratng to the 1

2 busness. For example, the hygene standard of a regon mght affect user s ratng to many restaurants located n that regon. On the other hand, a regon dstngushes tself from other regons and becomes attractve to vstors often because there are a few good busnesses co-located n the regon. For busness ratng predcton, an nterestng queston here s: Is there any correlaton between a busness s ratng and the ratng of ts geographcal neghbors? To answer the queston above, we conducted data analyss on Yelp s busness ratng data. We observe that there does exst weak postve correlaton between the ratng of a busness and the ratng of ts neghbors, regardless of the category of the busness. Based on ths observaton, we ncorporate geographcal neghborhood nfluence nto our busness ratng predcton model whch s based on the wdely adopted latent factor model realzed by Matrx Factorzaton (MF). Together wth nfluences from other factors ncludng user revews, busness category, and busness popularty, we show that the proposed model outperforms state-of-the-art baselnes ncludng Based MF, SVD++ and Socal MF [1, 17], measured by both Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). To the best of our knowledge, ths s the frst study explotng geographcal neghborhood nfluence n busness ratng predcton. We note that geographcal nfluence has been consdered n POI recommendaton and POI predcton. However, we argue that both POI recommendaton and POI predcton are dfferent research problems from busness ratng predcton (see Secton for more detals). To summarze, the man contrbutons arse from ths study are as follows: We conduct data analyss and observe that there exsts weak postve correlaton between a busness s average ratng and the average ratng of ts neghbors, regardless of the categores of busnesses. Ths s an mportant observaton that could be useful for not only busness ratng predcton, but also related studes, e.g., POI recommendaton and sentment analyss of busness revews. We drectly model the nfluence of geographcal neghborhood nto busness ratng predcton usng matrx factorzaton. Specfcally, for each busness, we use two dfferent latent factors to represent ts ntrnsc and extrnsc characterstcs respectvely. The nfluences from geographcal neghbors of a busness are then modeled as the lnear combnaton of the latent factors for the extrnsc characterstcs of ts neghbors. In our recommender system, we have also consdered other factors ncludng busness s revew, category, and popularty. To the best of our knowledge, ths s the frst study that models geographcal neghborhood nfluence nto busness ratng predcton. We conducted extensve experments to evaluate the effectveness of ncorporatng nfluence of geographcal neghborhood and other factors n busness ratng predcton and compared the predcton accuracy of the proposed model wth an array of strong baselne models. We further show that the geographcal dstance between a user and a busness adversely affects busness ratng predcton, although ths s an mportant and effectve factor n POI recommendaton problems. The rest of ths paper s organzed as follows. We revew the related work on ratng predcton, POI recommendaton and POI predcton n Secton. The data analyss s reported n Secton 3. The detals of the proposed busness ratng predcton model s presented n Secton 4, followed by expermental evaluaton n Secton 5. Fnally, Secton 6 concludes ths paper.. RELATED WORK In ths secton, we brefly revew the recent advances n recommender systems, manly focusng on the applcatons of collaboratve flterng on ratng predcton. As one of the major contrbutons of ths study s the nfluence of geographcal neghborhood, we also revew the related work employng geographcal nfluence for POI recommendaton and POI predcton. We then hghlght the major dfferences between our research problem and these two problems..1 Collaboratve Flterng Collaboratve flterng (CF) s the most wdely used technque n recommender systems [11, 1, 18, 19]. The underlyng assumpton of CF s that smlar users would rate tems smlarly or smlar tems would receve smlar ratngs from users. Ratngs here ndcate users preferences to tems. There are manly two knds of approaches, namely, memory-based CF and model-based CF. The basc dea of memory-based CF s to fnd smlar users or tems by usng smlarty measures,.e., user-based CF [1, 10] and tem-based CF [8, 1]. The commonly used smlarty measures are Pearson s correlaton and cosne smlarty, computed based on user-tem ratngs as well as other user-/tem-specfc features dependng on the applcaton. The smlar users or tems are also known as neghbors and therefore memory-based CF s also known as neghborhood-based CF. However neghbor n ths context s defned by the chosen smlarty measure. Dfferent smlarty measures lead to dfferent sets of neghbors for a gven user or tem. In ths paper, the neghborhood of a specfc busness refer to the busnesses that are located physcally close to t,.e., geographcal neghborhood. Model-based CF usually employs machne learnng technques to buld models from the observed user-tem ratngs, and then to predct the unobserved ratngs. Latent factor model s one of the most successful CF models, n whch users and tems are jontly mapped nto a shared latent space of much lower dmensonalty. As the most successful realzaton of latent factor model, matrx factorzaton (MF) [1,14,17] has been successfully appled to varous recommender systems ncludng musc ratng predcton for Yahoo! Musc [11] and Last.fm [4], move ratng predcton for Netflx [1, 13] and Douban [17, 4], and personalzed tweet recommendaton []. Among varous MF models proposed, SVD++ [1] s probably one of the most successful models. Ths model ntegrates the mplct feedback nformaton from a user to tems (e.g., based on user s purchase hstory or browsng hstory). More specfcally, the user vector of latent factors n ths model s complemented by the latent factors of the tems to whch the user has provded mplct feedback. Recently, Ma [17] proposed a socal regularzaton MF method, named Socal MF, to employ the smlar and dssmlar relatonshps between users and/or tems to mprove recommendaton accuracy. The smlarty between tems s measured based on ther ratngs usng Pearson s correlaton or cosne smlarty. In our experments, the proposed models acheve better ratng predcton accuracy than Socal MF ndcatng that the nfluence from geographcal neghborhood s more effectve than the nfluence from neghbors chosen by ratng-based smlarty measures. Based on MF, nfluence from other aspects of users or tems besdes the ratngs can be flexbly and easly modeled. For example, Koengsten et al. [11] ncorporated rch tem bas nto MF model to capture the taxonomy nformaton of musc. Each musc has multple types of nformaton such as track, album, artst and genre. In ther proposed model, MF was extended by addng shared bas parameters for tems lnked by common taxonomy. Moreover, some other work has also shown that popularty s helpful n mprovng

3 the recommendaton accuracy [7, ]. To capture users nterests over tweet content, Chen et al. [] proposed to use topc-level latent factors. Instead of drectly askng whether a user s nterested n a tweet, the model captures user s preference over the words n the tweet. In our problem settng, we consder the revew from a user further elaborates about the ratng. We model the words n revew as latent factors and ncorporate the revew nto ratng predcton.. POI Recommendaton and Predcton POI recommendaton has attracted sgnfcant attenton recently wth the popularty of LBSN [3, 6, 5, 6]. Because geographcal neghborhood s the man focus n our study, there s a need to revew recent advances n POI recommendaton. The man research ssue n POI recommendaton s to accurately recommend unvsted POIs to users. Geographcal nfluence and temporal nfluence are major consderatons n POI recommendaton [5, 6]. Geographcal nfluence s based on the observaton that users tend to vst nearby POIs of ther home or offce locatons, as well as nearby locatons of the POIs n ther favor. Temporal nfluence s reflected by the observaton that users check-n dfferent types of POIs at dfferent tme slots of a day (e.g., restaurant n lunch hour and bar n nght). Moreover, POI recommendaton has also consdered socal nfluence among frends [5]. POI predcton ams to predct whch POI a user would vst next gven her current locaton and tme [4, 6]. Dfferent from POI recommendaton, the POI predcted for a user to be vsted next may have been vsted by the user before. Smlarly, both geographcal nfluence and temporal nfluence have been consdered n the modelng. Lu et al. [16] proposed a category-aware POI predcton model by explotng users preference transton over locaton categores. In ths model, MF s utlzed to predct a user s preference transtons over categores and then the locatons n the correspondng categores by consderng geographcal nfluences. The model was evaluated on Gowalla check-n data wth three-level category herarchy. In our model, we also consder category nformaton of the busness. However, we drectly ncorporate category latent factors nto our factorzaton model to reflect the affnty of busnesses lnked by the categores they belong to. Note that, categores n the Yelp dataset are not organzed n herarchy. In ths paper, we study the mpact of geographcal neghborhood nfluence to busness ratng predcton. We argue that the geographcal neghborhood nfluence n our research problem s essentally dfferent from the geographcal nfluence n POI recommendaton and/or predcton. For POI recommendaton/predcton, the geographcal nfluence s more related to the cost of travel (e.g., tme cost or monetary cost) from a user s pont of vew. For ratng predcton, on the other hand, we predct a user s degree of preference to a yet revewed busness. The cost of travel here s expected to be a less mportant factor. The contextual factors (e.g., the current locaton, the tme of vstng) whch have demonstrated effectve n POI recommendaton/predcton are also less relevant to our research problem. In our research problem, geographcal neghborhood nfluence s more related to the busness envronment created by the surroundng busnesses to one busness at a specfc locaton. Next, we conduct data analyss of the Yelp data. 3. DATA ANALYSIS 3.1 Dataset Our study s based on the recently released Yelp Dataset Challenge. The data s sampled by Yelp from the greater Phoenx, AZ Table 1: Number of revews/ratngs per user/busness Statstcs User Busness Mnmum number of revews/ratngs 1 3 Maxmum number of revews/ratngs Average number of revews/ratngs Percentage neghbor 3 neghbors 6 neghbors 10 neghbors Dstance threshold (meter) Fgure 1: Percentage of busnesses havng at least 1, 3, 6, 10 neghbors wthn a dstance threshold from 0 to 000 meters metropoltan area from March 005 to January 013. It contans 11,537 busnesses, 9,907 revews by 43,873 users, and 8,8 check-n sets. Ths dataset was used as tranng data n ACM Rec- Sys Challenge A busness has a unque d, name, address, lattude longtude, ts categores and some other attrbutes lke cty, state, and neghborhoods. However, we observe that the neghborhoods attrbute s empty n the dataset. A revew contans busness d, user d, ratng from 1 to 5 stars, date, revew text, and votng. In ths study, we do not use the votng feature for ts less relevance to our research problem. Table 1 reports the mnmum, maxmum, and average number of ratngs per user and per busness respectvely. A check-n set for a busness contans the aggregated number of check-ns n every hour from Monday to Sunday. The largest number of check-ns to one busness observed from the data s,977. Note that, the dataset does not provde detaled check-n records of whch user checkn a busness at what tme, and there are only about 7% of busnesses have check-n sets. The percentage becomes 85.13% among all busnesses havng at least 5 revews. In other words, a busness may have a number of revews but no check-n records. The dataset does not provde much detals of a user. 3. Observatons Tobler s Frst Law of Geography states Everythng s related to everythng else, but near thngs are more related than dstant thngs [3]. Next, we present 3 observatons made from the data wth respect to geographcal neghborhood. Observaton 1. Most busnesses have neghbors wthn a short geographcal dstance from ther locatons. More than 44% of busnesses have one neghbor next to t wthn 0 meters and 95% of busnesses have one neghbor wthn 500 meters. Based on the busness lattudes and longtudes, we calculate the geographcal dstance between two busnesses usng Haversne formula.4 Fgure 1 plots the percentage of busnesses havng at least 1, 3, 6, and 10 neghbors respectvely, wthn a geographcal dstance threshold rangng from 0 to 000 meters. It shows that most busnesses are not solated geographcally from others. Specfcally,

4 Correlaton coeffcent NN 3NN 6NN 10NN Random Dstance threshold (meter) Num. of busnesses (logscale) Category d by number of busnesses n descendng order Fgure : Pearson s correlaton between a busness s ratng and the average ratng of ts {1, 3, 6, 10} nearest neghbors (NN), wthn a dstance threshold. 44.3% of busnesses have one neghbor next to t wthn 0 meters. The percentage rses to 95.6% f the dstance threshold s set to 500 meters, a dstance for a 6-mnute walk.5 Wthn ths walkng dstance, about 80% of busnesses have at least 6 neghbors, and 66% have at least 10 neghbors. Observaton. The average ratng of a busness s weakly postvely correlated wth the average ratng of ts neghbors. Before explorng a new place, we often receve advce on whch regon s famous for good food (e.g., a few famous restaurants collocatng n that regon), or advce on whch regon to avod for shoppng because the shops there are nfamous for cheatng. Ths phenomenon of thngs of one knd come together s also reflected from the postve correlaton between the ratngs of busnesses and ther geographcal neghbors. We calculate the average ratng of a busness from all ts user revews. Hereafter, for easy presentaton, we refer ths average ratng smply as the ratng of a busness or a busness s ratng when the context s clear. Fgure plots the Pearson s correlaton coeffcent between a busness s ratng and the average ratng of ts 1, 3, 6, and 10 nearest neghbors, at dfferent dstance thresholds from 0 to 000 meters. For a gven dstance threshold, f a busness has no neghbors wthn the dstance, then ths busness s excluded from the computaton. If a busness has fewer than the number of nearest neghbors specfed (e.g., 10NN) wthn the dstance threshold, then the ratngs of all ts nearest neghbors wthn the dstance threshold are averaged. For reference, for each busness partcpated n the computaton wthn a dstance threshold, we also randomly sample a busness from the dataset (regardless of the dstance between the two) and then compute the Pearson s correlaton between the two sets of ratngs, labeled Random n the plot. From Fgure, we observe that the ratng of a busness s weakly postvely correlated wth the average ratng of ts nearest neghbor(s). Pearson s correlaton coeffcent s n the range of to The correlaton s relatvely stronger wthn a smaller dstance (e.g., 0 or 50 meters) and becomes stable for dstance threshold of 500 meters or larger. In comparson, correlaton coeffcent between the ratngs of the same set of busnesses and ther randomly selected counterparts s n the range of to 0.00,.e., no correlaton, as expected. We have two nterpretatons of the weak postve correlaton between ratngs of busnesses and ther neghbors. Frst, the ratng of a busness should manly depend on the ntrnsc characterstcs 5 Human preferred walkng speed s about 1.4 m/s or 5km/h. Fgure 3: Busness category dstrbuton of the busness (e.g., qualty of products/servces), not ts neghbors. Second, a busness s not geographcally ndependent from ts neghbors. When a user vsts one busness, she has at least glanced over ts neghbors f not vsted them. These neghbors gve her the sense of the surroundng envronment of the busness (e.g., hygene standard). Such extrnsc characterstcs may affect her vew of the busness, leadng to the weak postve correlaton. Based on the weak postve correlaton, we expect more accurate busness ratng predcton can be acheved by consderng the nfluence of the geographcal neghbors of a busness. Observaton 3. The weak postve correlaton n ratngs s ndependent of the categores of the busnesses and/or ther neghbors. A busness n Yelp s assgned one or more category labels (e.g., restaurant, shoppng). Plotted n Fgure 3, the number of busnesses n categores demonstrate a power-law lke dstrbuton, wth a few categores each contanng a large number of busnesses and many small categores each has only one or two busnesses. Restaurants, the largest category, covers nearly 40% of the 11,537 busnesses n the dataset. Next, we study the 5 largest categores as representatve examples. They are: Restaurants (4, 503), Shoppng (1, 681), Food (1,616), Beauty & Spas (764), and Nghtlfe (640), where the numbers n parentheses ndcate the number of busnesses n each category. For each of the 5 categores, we calculate the ratng correlaton between the busnesses n the category and ther neghbors, regardless of the categores of ther neghbors because the neghbors here are defned geographcally. The category-wse ratng correlatons wth 1NN and 6NN are plotted n Fgures 4(a) and 4(b) respectvely. The lnes labeled random, plotted for reference purpose, report the correlaton between the ratngs of busnesses n any of the 5 categores and randomly selected busnesses from the dataset (regardless of ther categores and geographcal dstance). Shown n Fgure 4(a), busnesses n Beauty & Spas demonstrate slghtly stronger ratng correlaton wth ther 1NN compared wth busnesses n Nghtlfe and Shoppng. When the number of nearest neghbors s enlarged to 6, ths pattern becomes less apparent, partcularly when the dstance threshold reaches 500 meters, plotted n Fgure 4(b). In both Fgures 4(a) and 4(b), weak postve correlaton s observed for ratngs of busnesses n all 5 categores across all dstance thresholds.6 Fgures 4(c) and 4(d) show the percentage of the nearest neghbors that are also n the same category. The two fgures show very smlar patterns: () Restaurants are more lkely collocated wth restaurants wth about 50% chance that the neghbors of a restaurant are also restaurants; () Beauty & Spas on the other 6 Smlar observaton also holds for 3NN and 10NN. The fgures are not plotted due to page lmt.

5 Correlaton coeffcent Restaurants Shoppng Food Beauty & Spas Nghtlfe Random Correlaton coeffcent Restaurants Shoppng Food Beauty & Spas Nghtlfe Random Dstance threshold of the 1NN (meter) (a) Ratng correlaton of 1NN Dstance threshold of 6NN (meter) (b) Ratng correlaton of 6NN Percentage n same category Restaurants Shoppng Food Beauty & Spas Nghtlfe 0 Dstance threshold of 1NN (meter) (c) Percentage of 1NN beng n the same category Percentage n same category Restaurants Shoppng Food Beauty & Spas Nghtlfe 0 Dstance threshold of 6NN (meter) (d) Percentage of 6NN beng n the same category Fgure 4: Category-wse ratng correlaton wth nearest neghbors (1NN, 6NN), and the percentage of neghbors n the same category hand s much more dstnctve from ther neghbors. Below 0% of ther neghbors are also n the same category. Wth respect to Fgures 4(a) and 4(b), we argue that weak postve ratng correlaton between busnesses and ther neghbors s ndependent of whether the neghbors are n the same category. In summary, the three observatons made from the data suggest that geographcal neghborhood has nfluences on the ratng of a busness. Next, we ncorporate geographcal neghborhood nfluence nto busness ratng predcton. 4. BUSINESS RATING PREDICTION The problem of ratng predcton has been well formulated n lterature. We use r u to denote the revew ratng that user u gves to tem (.e., a busness). r u s n the range of 1 to 5 stars wth more stars ndcatng hgher preference. Gven the exstng ratngs made by m users to n tems, the task s to predct the unknown ratng ˆr u, f user u has not rated tem before. In the followng, we frst brefly ntroduce matrx factorzaton and then present our proposed model by ncorporatng varous nfluences nto the predcton. Table lsts the notatons used n ths paper. 4.1 Matrx Factorzaton Our proposed method s based on the latent factor model realzed by matrx factorzaton. Through matrx factorzaton, each user and each tem s assocated wth a f -dmensonal vector, where f mn(m,n). The nner product of a user vector p u R f 1 and an tem vector q R f 1 s used to approxmate the user s preference to the tem (see [13] for a detaled ntroducton of matrx factorzaton). Accordngly, the predcted ratng of user u to tem s computed usng ˆr u = p u q, where p u and q can be learned from the user-tem ratng matrx wth known ratngs. However, users may have certan degree of b- Table : Notatons and semantcs N Set of geographcal neghbors of tem C Set of categores tem belongs to R Set of words n tem s revew K Set of (u,) pars wth known r u ratngs r u, ˆr u Observed and predcted ratngs of user u to tem µ Mean of all known r u ratngs b u, b Bas parameters for user u and tem, respectvely p u Latent factors of user u q Latent factors of tem for ts ntrnsc characterstcs v Latent factors of tem for ts extrnsc characterstcs d c Latent factors of category c q w Latent factors of revew word w ρ Normalzed popularty of tem β Popularty weghtng parameter for tem τ u, Normalzed geo-dstance between user u and tem β u Geo-dstance weghtng parameter for user u ases: some users are more lenent and some are very strct about ratngs. Smlarly, tems may also have some degree of bases because of locaton or brandng for example. To acheve more accurate ratng predcton, Based MF extends the basc matrx factorzaton by consderng the bases, ˆr u = µ + b u + b + p u q, (1) where µ s the average ratng of all known ratngs; b u and b are the user bas and tem bas, respectvely. Learnng the unknown parameters p u, q, b u, and b s an optmzaton problem to mnmze the regularzed squared error on the set of known ratngs K. mn (r u ˆr u ) ( + λ 1 pu + q ) ( + λ b u + b ) p,q,b (u,) K

6 In ths equaton, λ 1 and λ are regularzaton parameters used to avod overfttng. Both stochastc gradent descent (SGD) and alternatng least squares (ALS) algorthms can be used to solve the optmzaton functon and learn the parameters [1, 13]. In ths paper, we adopt SGD to learn the parameters followng the algorthm presented n [1]. 4. Incorporatng Neghborhood Influence Based on our observatons n Secton 3., most busnesses have neghbors wthn a short geographcal dstance, and more mportantly, the ratng of a busness s weakly postvely correlated wth the ratng of ts neghbors. These observatons suggest that consderng the geographcal neghborhood nfluence may mprove the accuracy of busness ratng predcton. In ths paper, to model users ratng behavor on busnesses, we frst assume that a user s ratng to a gven busness s determned by ts ntrnsc characterstcs and the extrnsc characterstcs of ts geographcal neghbors. For a busness, we use q and v to model ts ntrnsc and extrnsc characterstcs, respectvely. More specfcally, q models the ntrnsc characterstcs of a busness (e.g., taste of food and qualty of servce) observable by users who have nteracted wth the busness. v models the extrnsc characterstcs of a busness (e.g., hygene standard) n nfluencng ts geographcal neghbors observable by the pass-by vstors. Let N be the set of geographcal neghbors of a busness, satsfyng certan selecton crtera (e.g., the top-6 nearest neghbors). Let n N be a neghbor of busness. Incorporated wth nfluence from geographcal neghbors, the predcted ratng s now computed wth both q and v n s, shown n Equaton. ˆr u = µ + b u + b + p u q + α 1 N n N v n In above equaton, parameter α 1 [0,1] controls the mportance of geographcal neghborhood nfluence n busness ratng predcton, denotes the cardnalty of a set. Accordngly, the objectve functon s updated wth regularzaton components for v n, shown n Equaton 6 n Table 3, where λ 3 s the newly ntroduced regularzaton parameter, smlar to λ 1 and λ. Note that, by consderng v n s n the predcton, Equaton partally addresses the cold-start problem for newly establshed busnesses that do not appear n tranng data. Although q of a new busness s empty, the v n s of ts geographcal neghbors are not empty f they appear n tranng data. Therefore Equaton can be appled to make ratng predcton of an exstng user to a newly establshed busness by usng p u, b u and v n s. () 4.3 Incorporatng Category Influence Analyzed n Secton 3., a busness n Yelp may belong to one or more categores. The category of a busness usually reflects the characterstcs of a busness, e.g., product/servce offered by the busness or the way the busness s conducted. Intutvely, users may use dfferent crtera to evaluate busnesses n dfferent categores. For example, the crtera commonly used for revewng restaurant (e.g., taste of food) cannot be used to revew busnesses n beauty & spas category. Moreover, a recent study has also shown that POI recommendaton acheves better accuracy by consderng the categores of the POIs [16]. In our model, we ntroduce category latent factors to explot busness categores for more accurate busness ratng predcton. For a busness category c, t s assocated wth a latent vector d c R f 1. Let C be the set of categores a busness belongs to. By ncorporatng the category nfluence, the predcted ratng ˆr u s now defned n Equaton 3, where α [0,1] s a parameter that controls the mportance of category nfluence n ratng predcton. The objectve functon s updated accordngly, see Equaton 7 n Table 3. ˆr u = µ + b u + b + p u q + α 1 N v n + α C n N c C d c 4.4 Incorporatng Revew Content In Yelp, when gvng a ratng to a busness from 1 to 5 stars, the user usually wrtes a revew. Typcally, the revew elaborates the reason behnd the ratng and partally reflects the characterstcs of the busness. Collectvely, words n all revews to a busness provde a much better descrpton about the busness than the learned vector of latent factors. However, n order to make use of the revew words n the predcton model, the revew words have to be mapped to the same f -dmensonal vector space. Smlar to the topc level decomposton of tweets for tweet recommendaton n [], we decompose the latent factors of a busness q nto a combnaton of latent factors of revew words. Let R be the set of words that appear n busness s revew, the decomposton s shown as follows, where q w denotes the latent factors of word w. q 1 R (3) w R q w (4) Wth the decomposton, the predcted ratng s now shown n Equaton 5. The objectve functon s shown n Equaton 8 n Table 3. ˆr u = µ + b u + b + p 1 u R q w + α 1 N w R v n + α C n N c C d c 4.5 Popularty and Geo-dstance Influences Next, we dscuss two features that have been studed n POI recommendaton, namely popularty and geographcal dstance. In Secton 1, we argue that the key dfference between a busness wth other knd of tems that have been studed n ratng predcton s that a busness physcally exsts at a geographcal locaton. Although we argue that busness ratng predcton s dfferent from POI recommendaton, t remans nterestng to nvestgate whether the features explored n POI recommendaton are useful here. Regonal popularty s mentoned as one of the unque characterstcs n LBSNs whch dstngush POI recommendaton from other recommendaton tasks [15]. For example, busnesses located n downtown area are lkely to receve more vsts than those n suburban. In the Yelp dataset, a busness has check-n sets and revews, both are ndcators of busness popularty. Note that, a user may wrte one revew to a busness but may check-n at the busness multple tmes. However, the Yelp dataset only provdes the aggregated check-n numbers on hourly bass from Monday to Sunday but not user-specfc check-ns. Some of the busnesses n the dataset do not have check-n sets. We therefore smply sum up the number of revews and the number of check-ns of a busness to be ts popularty. In POI recommendaton, geographcal dstance s a major consderaton because users tend to vst nearby POIs [5, 6]. Wth users revew data, we estmate a user s home locaton by usng the recursve grd search algorthm proposed n [5]. Then the geographcal dstance between a user and an unrated busness can be easly calculated based on the estmated user locaton and the locaton of the busness. (5)

7 Table 3: Objectve functons for ncorporatng neghborhood nfluence, category nfluence, revew words and other factors mn (r u ˆr u ) ( + λ 1 pu + q ) ( + λ b u + b ) p,q,b,v + λ3 v n (6) (u,) K n N mn (r u ˆr u ) ( + λ 1 pu + q ) ( + λ b u + b ) p,q,b,v,d + λ3 v n + d c (u,) K n N c C mn (r u ˆr u ) + λ 1 u p,q,b,v,d (u,) K p + q w ( + λ b u + b ) + λ3 v n + d c w R n N c C mn (r u ˆr u ) + λ 1 u p,q,b,v,d, β (u,) K p + q w ( + λ b u + b + β + u) β + λ3 v n + d c w R n N c C (7) (8) (9) For smplcty, we model both popularty and geographcal dstance as a ratng bas z. z = β ρ + β u τ u, (10) In above equaton, ρ s the normalzed popularty of tem by takng the common logarthm of ts raw popularty value; τ u, s the normalzed dstance between user u and busness by takng the common logarthm of the geographcal dstance. The two parameters β and β u are the weghtng parameters for busness and user u respectvely; both are learned from the tranng data. Wth ratng bas z, the predcted ratng s shown n Equaton 11. ˆr u = µ + b u + b + z + p 1 u R q w + α 1 N w R v n + α C n N c C d c (11) Note that, we do not replace the user bas b u and tem bas b by ratng bas z. The reason s that the ratng bas β ρ + β u τ u, captures bases specfc to popularty and geographcal dstance respectvely, whle b u and b are used to capture bas from all unknown factors. The objectve functon consderng both popularty and geographcal dstance s shown n Equaton 9 n Table EXPERIMENTS We now conduct experments on the Yelp dataset to evaluate the proposed models and compare the proposed models wth state-ofthe-art baselnes. 5.1 Expermental Settng Dataset. We use the Yelp dataset that has been studed n Secton 3.1 n our experments. The preprocessng of the dataset n our experments ncludes removal of busnesses and users havng fewer than 10 revews, stop words removal and stemmng n revews. After the preprocessng, we have 113,514 ratngs by 3,965 users to 3, 760 busnesses. For each user, we sort her ratngs n chronologcal order. The frst 70% of ratngs are used for tranng, and the remanng 30% for testng. As the result, we have 79,309 ratngs to buld the matrx factorzaton model for the predcton of the remanng 34,05 ratngs. The data sparsty s 99.47%. Evaluaton Metrc. We adopt two popular evaluaton metrcs, namely, Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). The smaller MAE or RMSE value means better ratng predcton accuracy. In the followng equatons, T s the set of user-tem ratng pars (u,) used n testng. 1 M AE = r u ˆr u T (u,) T 1 RMSE = (r u ˆr u ) T (u,) T Baselne Methods. We compare the proposed models wth the followng 8 baselne methods. All these baselne methods are mplemented by usng the MyMedaLte lbrary [9]. 1. Global Mean: ths method predcts an unknown ratng to be the average of all known ratngs,.e., ˆr u = µ.. User Mean: ths method utlzes the mean ratng of each user to predct the mssng values for the correspondng user. 3. Item Mean: ths method uses the mean ratng of each tem to predct the mssng values for the correspondng tem. 4. User KNN: ths method s the user-based collaboratve flterng. The unknown ratngs are predcted by consderng the ratngs gven by smlar users. User smlarty s computed by cosne smlarty of ratngs and k = 150 n our experments. We have also evaluated Pearson s correlaton as user smlarty whch gves poorer results compared wth cosne smlarty. 5. Item KNN: ths method s the tem-based collaboratve flterng. The unknown ratngs are predcted by consderng the ratngs receved by smlar tems. Item smlarty s computed by cosne smlarty of ratngs and k = Based MF: ths s the MF model wth user and tem bases brefly descrbed n Secton 4.1. Based MF s wdely used as a baselne n recommender systems. 7. SVD++: ths model consders mplct feedback from users for ratng predcton [1]. SVD++ usually offers a superor accuracy and s consdered as a state-of-the-art matrx factorzaton algorthm for recommendaton. 8. Socal MF: ths model consders mplct socal nformaton between tems and/or users. The mplct socal nformaton can be derved from most smlar and dssmlar users/tems usng Pearson s correlatons or cosne smlarty of ratngs. As our model consders nfluences from nearest geographcal neghbors, for a far comparson, we nclude the mplct

8 socal nformaton from most smlar tems n Socal MF.7 Followng [17], the most smlar tems are selected by usng Pearson s correlaton of ratngs greater than Proposed Methods. We extended Based MF to ncorporate nfluences from multple factors: geographcal neghborhood (N), busness category (C), revew content (R), busness popularty (P), and geographcal dstance (D). The proposed methods are denoted usng the letters n parentheses to ndcate the nfluences consdered n each method. 9. N-MF: ths method ncorporates geographcal neghborhood nfluence (Secton 4., Equaton ). 10. NC-MF: ths method ncorporates both geographcal neghborhood and category nfluences (Secton 4.3, Equaton 3). 11. NCR-MF: ths model ncorporates neghborhood, category and revew content nfluences (Secton 4.4, Equaton 5). 1. NCRP-MF: ths model ncorporates neghborhood, category, revew content and popularty nfluences, by settng z = β ρ n Equaton 10 (Secton 4.5). 13. NCRPD-MF: ths model ncorporates all factors: neghborhood, category, revew content, popularty and geographcal dstance, z = β ρ + β u τ u, (Secton 4.5). We also evaluate another two methods: CRP-MF and CRPD-MF. These two methods do not ncorporate geographcal neghborhood nfluence but ncorporate nfluences from other factors (.e., C, R, P, and D) ndcated by the method names. Parameter Settng. We performed 5-fold cross-valdaton on the tranng set to emprcally set the hyperparameters. The number of latent factors f = 0; the relatve mportance of neghborhood and category nfluences are set to α 1 = 0.8, α = 0.6; The regularzaton parameters: λ 1 = 0.8, λ = 0.4, λ 3 = 0.6. The latent factors are learned by SGD wth ntal learnng rate γ = 0.008, whch decreases by a factor of 0.9 after each teraton (see Appendx A). The same parameters are used n all methods for far comparson for all our proposed methods and the baselne methods whenever applcable. For example, the number of latent factors s also set to 0 n baselne methods Based MF, SVD++ and Socal MF. For geographcal neghborhood nfluence, by default, the proposed methods use the 6 nearest neghbors for each busness. For all the methods based on matrx factorzaton, the reported results are averaged over 5 runs to avod the mpact of ntalzaton n parameter learnng. 5. Expermental Results We frst compare the proposed methods wth baselne methods and then evaluate the two schemes for defnng the set of geographcal neghbors for a busness. Lastly, we evaluate the proposed methods wth cold-start settng Method Comparson The predcton errors measured by MAE and RMSE of all methods are reported n Table 4 wth best results hghlghted n boldface. We make four observatons from the results. Frst, ncorporatng geographcal neghborhood nfluence nto busness ratng predcton greatly reduces predcton errors measured 7 Note that, our model can also be extended to nclude the other knds of mplct socal nformaton as n Socal MF. However, the man purpose of our expermental evaluaton s to evaluate the nfluence from geographcal neghbors n comparson to the nfluence from smlar tems measured by ratng-based smlarty. Table 4: MAE and RMSE of all methods, the lower the better. ID Method MAE RMSE 1 Global Mean (µ) Item Mean User Mean Item KNN User KNN Based MF SVD Socal MF N-MF NC-MF NCR-MF NCRP-MF NCRPD-MF CRP-MF CRPD-MF by both MAE and RMSE. All the proposed methods wth geographcal neghborhood nfluence (.e., methods 9-13) outperform all baselne methods (methods 1-8). The best predcton accuracy s acheved by NCRP-MF whch consders geographcal neghborhood (N), busness category (C), revew content (R), and busness popularty (P). Wth geographcal neghborhood nfluence alone, N-MF outperforms all baselnes ncludng state-of-the-art methods SVD++ and Socal MF. Ths result suggests that geographcal neghbors are more effectve than the neghbors derved from ratng-based smlarty as n Socal MF. Second, compared wth geographcal neghborhood, further consderng factors lke busness category (C), revew content (R), and busness popularty (P) leads to relatvely small addtonal reducton n predcton errors. The geographcal dstance factor (D) adversely affects busness ratng predcton accuracy. Method NCRPD- MF performs the worst among all methods wth geographcal neghborhood nfluence measured by both MAE and RMSE. Ths result suggests that the geographcal dstance factor ntroduces nose n the predcton model. Such result supports our earler dscusson that geographcal dstance between users and tems s less relevant to the problem of busness ratng predcton. Thrd, wthout ncorporatng geographcal nfluence, CRP-MF performs poorer than most methods wth geographcal neghborhood nfluence ncludng N-MF, NC-MF, NCR-MF, and NCRP- MF. The poorer performance of CRP-MF aganst N-MF suggests that the geographcal neghborhood nfluence s more effectve than the combnaton of the three factors (C, R, and P) n busness ratng predcton. The much poorer performance of CRPD-MF n comparson wth CRP-MF agan suggests the adverse effect of geographcal dstance factor (D). On the other hand, the effectveness of geographcal neghborhood nfluence s also reflected from the better performance of NCRPD-MF compared wth CRPD-MF. Last, among the 8 baselne methods, SVD++ and Socal MF, the two state-of-the-art methods, perform the best evaluated by RMSE. By MAE measure, User KNN gves surprsngly low error rate and the two methods SVD++ and Socal MF reman among the three best methods. 5.. Impact of Neghborhood Sze Gven a busness, ts geographcal neghbors can be defned based on ether a geographcal dstance threshold or the number of nearest neghbors. In ths set of experments, we evaluate the two schemes on ther mpact to the predcton error of the N-MF method. The

9 MAE MAE Dstance threshold (meter) (a) Neghbors by dstance (MAE) Nearest neghbor (b) Neghbors by neghborhood sze (MAE) RMSE RMSE Dstance threshold (meter) (c) Neghbors by dstance (RMSE) Nearest neghbor (d) Neghbors by neghborhood sze (RMSE) Fgure 5: Predcton error of N-MF wth two schemes for determnng geographcal neghbors of a busness N-MF method s selected as the method for evaluaton because t only consders the geographcal neghborhood factor. Fgures 5(a) and 5(c) respectvely plot MAE and RMSE of the N-MF method by defnng geographcal neghbors wth a dstance threshold rangng from 0 to 000 meters. Fgures 5(b) and 5(d) respectvely plot the MAE and RMSE of the method by takng the number of nearest neghbors rangng from 1 to 10. The y-axes of the two sets of fgures are plotted n the same scale for easy comparson. From the two sets of fgures, ether the dstance threshold of 1000 meters or the top-6 nearest neghbors gve the best predcton accuracy by consderng both MAE and RMSE. Plotted n Fgure 1, wth a dstance threshold of 1000 meters, more than 90% of busnesses have at least 6 neghbors and about 84% of the busnesses have at least 10 neghbors. Because some busnesses may have a large number of neghbors wthn 1000 meters, we choose to use 6 nearest neghbors manly for mnmzng computatonal cost. We have also evaluated the method by usng at most 6 nearest neghbors wthn a 1000-meter dstance threshold. However, poorer predcton accuracy was obtaned compared wth smply usng 6 nearest neghbors wthout a dstance threshold Cold-Start Busness Ratng Predcton Cold-start s a challengng ssue n any recommender systems when there s no enough nformaton about ether users or tems. We now evaluate the performance of the proposed methods n predctng ratngs by exstng users to newly establshed busnesses. Recall that n our data preprocessng (see Secton 5.1), busnesses and users wth fewer than 10 revews were removed. In ths set of experments, we try to predct the ratngs by exstng users n the tranng data to the busnesses that were removed n data preprocessng. That s, from exstng tranng data, we have the user vector of latent factors p u, but not the tem vector q because the newly establshed busnesses were not the tranng data. In total, there are 0,395 ratngs made by 3,319 exstng users to 6,939 new busnesses. Ths collecton of ratngs wll be used as test set. Among all baselne methods, Global Mean and User Mean can be easly appled. Item Mean, User KNN and Item KNN cannot be appled as we assume that the newly establshed busnesses have no Table 5: MAE and RMSE n cold-start settng Method MAE RMSE Global Mean User Mean Based MF N-MF NC-MF revews yet. Both Based MF and SVD++ are reduced to µ + b u because b and q are both unknown (see Equaton 1). Socal MF s not applcable here because t reles on smlar tems by some smlarty functons. Among the proposed methods, both the latent factors for the extrnsc characterstcs of geographcal neghbors (.e., v n s of exstng busnesses) and busness category (.e., d c ) can be utlzed n predcton. Here, we assume the category of the newly establshed busness s known (e.g., restaurant or bookstore). The predcton s made by Equaton 3 after removng b and q from the equaton. Reported n Table 5, the proposed method NC-MF acheves the best predcton accuracy by consderng both geographcal neghborhood and busness category. Utlzng geographcal nfluence alone, N-MF s the second best performng methods by both MAE and RMSE. Both User Mean and Based MF are better than Global Mean wth the former acheves better MAE and the latter acheves better RMSE. In summary, the proposed methods predct more accurate ratngs for exstng users to newly establshed busnesses n cold-start settng. 6. CONCLUSION To the best of our knowledge, ths s the frst study on geographcal neghborhood nfluence to users busness ratng behavor. We beleve the observaton that a busness s ratng s weakly postvely correlated wth ts geographcal neghbors ratng s an mportant one. Based on ths observaton, we model a busness wth two vectors of latent factors one for ts ntrnsc characterstcs and the other for ts extrnsc characterstcs (or ts nfluence to ts geographcal neghbors). In our experments, we show that by ncorporatng ge-

10 ographcal neghborhood nfluence, the proposed methods outperform state-of-the-art methods. Other factors lke busness category, popularty, and revew content can further mprove the ratng predcton accuracy. Nevertheless, the geographcal dstance between a user and a busness adversely affects the predcton accuracy, although t s an mportant factor n POI recommendaton and POI predcton. The ncorporaton of geographcal neghborhood nfluence also partally enables our methods to better handle coldstart stuaton, for ratng predcton of newly establshed busnesses based on both ther geographcal neghbors and busness categores. As a part of our future work, we are nterested n nvestgatng the nfluence of geographcal neghborhood n POI recommendaton and sentment analyss of busness revews. 7. REFERENCES [1] J. S. Breese, D. Heckerman, and C. Kade. Emprcal analyss of predctve algorthms for collaboratve flterng. In UAI, pages Morgan Kaufmann Publshers Inc., [] K. Chen, T. Chen, G. Zheng, O. Jn, E. Yao, and Y. Yu. Collaboratve personalzed tweet recommendaton. In SIGIR, pages ACM, 01. [3] C. Cheng, H. Yang, I. Kng, and M. R. Lyu. Fused matrx factorzaton wth geographcal and socal nfluence n locaton-based socal networks. In AAAI, 01. [4] C. Cheng, H. Yang, M. R. Lyu, and I. Kng. Where you lke to go next: Successve pont-of-nterest recommendaton. In IJCAI, pages AAAI Press, 013. [5] Z. Cheng, J. Caverlee, K. Lee, and D. Z. Su. Explorng mllons of footprnts n locaton sharng servces. In ICWSM, pages The AAAI Press, 011. [6] E. Cho, S. A. Myers, and J. Leskovec. Frendshp and moblty: User movement n locaton-based socal networks. In KDD, pages ACM, 011. [7] P. Cremones, Y. Koren, and R. Turrn. Performance of recommender algorthms on top-n recommendaton tasks. In RecSys, pages ACM, 010. [8] M. Deshpande and G. Karyps. Item-based top-n recommendaton algorthms. ACM Trans. Inf. Syst., (1): , 004. [9] Z. Gantner, S. Rendle, C. Freudenthaler, and L. Schmdt-Theme. MyMedaLte: A free recommender system lbrary. In RecSys, pages ACM, 011. [10] R. Jn, J. Y. Cha, and L. S. An automatc weghtng scheme for collaboratve flterng. In SIGIR, pages , 004. [11] N. Koengsten, G. Dror, and Y. Koren. Yahoo! musc recommendatons: Modelng musc ratngs wth temporal dynamcs and tem taxonomy. In RecSys, pages ACM, 011. [1] Y. Koren. Factorzaton meets the neghborhood: A multfaceted collaboratve flterng model. In KDD, pages ACM, 008. [13] Y. Koren. Collaboratve flterng wth temporal dynamcs. In KDD, pages ACM, 009. [14] Y. Koren, R. Bell, and C. Volnsky. Matrx factorzaton technques for recommender systems. Computer, 4(8):30 37, 009. [15] B. Lu, Y. Fu, Z. Yao, and H. Xong. Learnng geographcal preferences for pont-of-nterest recommendaton. In KDD, pages ACM, 013. [16] X. Lu, Y. Lu, K. Aberer, and C. Mao. Personalzed pont-of-nterest recommendaton by mnng users preference transton. In CIKM, pages ACM, 013. [17] H. Ma. An expermental study on mplct socal recommendaton. In SIGIR, pages ACM, 013. [18] S. Moghaddam, M. Jamal, and M. Ester. ETF: Extended tensor factorzaton model for personalzng predcton of revew helpfulness. In WSDM, pages ACM, 01. [19] N. Natarajan, D. Shn, and I. S. Dhllon. Whch app wll you use next?: Collaboratve flterng wth nteractonal context. In RecSys, pages ACM, 013. [0] A. Rae, V. Murdock, A. Popescu, and H. Bouchard. Mnng the web for ponts of nterest. In SIGIR, pages , 01. [1] B. Sarwar, G. Karyps, J. Konstan, and J. Redl. Item-based collaboratve flterng recommendaton algorthms. In WWW, pages ACM, 001. [] H. Steck. Item popularty and recommendaton accuracy. In RecSys, pages ACM, 011. [3] W. R. Tobler. A computer move smulatng urban growth n the detrot regon. Economc geography, 46:34 40, [4] D. Yang, T. Chen, W. Zhang, Q. Lu, and Y. Yu. Local mplct feedback mnng for musc recommendaton. In RecSys, pages ACM, 01. [5] M. Ye, P. Yn, W.-C. Lee, and D.-L. Lee. Explotng geographcal nfluence for collaboratve pont-of-nterest recommendaton. In SIGIR, pages ACM, 011. [6] Q. Yuan, G. Cong, Z. Ma, A. Sun, and N. M. Thalmann. Tme-aware pont-of-nterest recommendaton. In SIGIR, pages ACM, 013. APPENDIX A. PARAMETER ESTIMATION All the objectve functons (e.g., Equatons 6, 7, 8, and 9) n the proposed models share the same form. Next, we detal the parameter estmaton for Equaton 9 (where z = β ρ + β u τ u, ) as an example usng Stochastc Gradent Descent (SGD) algorthm [1]. Let e u be the error assocated wth the predcton e u = r u ˆr u. The parameters are learned by movng n the opposte drecton of the gradent wth a learnng rate γ n an teratve manner. In our experments, learnng rate γ s set to n the frst teraton and s decreased by a factor of 0.9 after each subsequent teraton. b u b u + γ (e u λ b u ) b b + γ (e u λ b ) β β + γ (e ) u ρ λ β β u β u + γ (e u τ u, λ β u ) p u p u + γ e u 1 R + α d c λ C 1 p u c C ( w R : q w q w + γ e u n N : v n v n + γ c C : d c d c + γ ( e u ( e u w R q w + α 1 N n N v n ) 1 p u λ R 1 q w ) α 1 p u λ N 3 v n ) α p u λ C 3 d c