LOCALIZING USERS AND ITEMS FROM PAIRED COMPARISONS. Matthew R. O Shaughnessy and Mark A. Davenport

Size: px

Start display at page:

Download "LOCALIZING USERS AND ITEMS FROM PAIRED COMPARISONS. Matthew R. O Shaughnessy and Mark A. Davenport"

Brendan Sims
5 years ago
Views:

2016 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT.

targeted advertsement, and psychologcal studes.

although tem localzaton produces a set of non-convex constrants, we can make the problem convex by

Fnally, we show that by teratvely applyng ths method to every user and tem, we can recover an accurate

Index Terms localzaton, pared comparsons, nonmetrc multdmensonal scalng, deal pont models,

INTRODUCTION In ths paper we consder several problems related to learnng an embeddng of ponts from

nvolve recommendaton systems, targeted advertsement, and psychologcal studes where x represents an deal

In ths model, whch dates back at least to [1], tems whch are close to x are those most preferred by the

preference are generally much more dffcult to assgn than comparatve M.

Davenport was supported by grants NRL N00173-14-2-C001, AFOSR FA9550-14-1-0342, NSF CCF- 1409406,

Shadng represents the feasble regon resultng from the set of comparsons. judgments [2].

opton to act on two (or more) alternatves; for nstance, they may choose to watch a partcular move or

In such contexts, the true dstances n the deal pont model are generally naccessble n any drect way, but

Some of the smplest questons n ths model arse n the settng where we are gven an exstng embeddng of

1 2016 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT , 2016, SALERNO, ITALY LOCALIZING USERS AND ITEMS FROM PAIRED COMPARISONS Matthew R. O Shaughnessy and Mark A. Davenport Georga Insttute of Technology School of Electrcal and Computer Engneerng {moshaughnessy6, mdav}@gatech.edu ABSTRACT Suppose that we wsh to determne an embeddng of ponts gven only pared comparsons of the form user x prefers tem q to tem q j. Such observatons arse n a varety of contexts, ncludng applcatons such as recommendaton systems, targeted advertsement, and psychologcal studes. In ths paper we frst present an optmzaton-based framework for localzng new users and tems when an exstng embeddng s known. We demonstrate that user localzaton can be formulated as a smple constraned quadratc program, and that although tem localzaton produces a set of non-convex constrants, we can make the problem convex by strategcally combnng comparsons to produce a set of convex lnear constrants. Fnally, we show that by teratvely applyng ths method to every user and tem, we can recover an accurate embeddng, allowng us to teratvely mprove a gven embeddng or even generate an embeddng from scratch. Index Terms localzaton, pared comparsons, nonmetrc multdmensonal scalng, deal pont models, recommendaton systems 1. INTRODUCTION In ths paper we consder several problems related to learnng an embeddng of ponts from pared comparsons of the form x s closer to q than q j, where x, q, q j R n correspond to ponts whose locatons we would lke to estmate. Observatons of ths form arse n a varety of contexts, but a partcularly mportant class of applcatons nvolve recommendaton systems, targeted advertsement, and psychologcal studes where x represents an deal pont that models a partcular user s preferences and q and q j represent tems that the user compares. In ths model, whch dates back at least to [1], tems whch are close to x are those most preferred by the user. Pared comparsons arse naturally n ths context snce precse numercal scores quantfyng a user s preference are generally much more dffcult to assgn than comparatve M. O Shaughnessy was supported by the Georga Tech Presdent s Undergraduate Research Award program. M. Davenport was supported by grants NRL N C001, AFOSR FA , NSF CCF , CCF , and CMMI Fg. 1: Set of lnear constrants nduced by the bnary pared comparsons. Shadng represents the feasble regon resultng from the set of comparsons. judgments [2]. Moreover, data consstng of pared comparsons s often generated mplctly n contexts where the user has the opton to act on two (or more) alternatves; for nstance, they may choose to watch a partcular move or clck a partcular advertsement out of those dsplayed to them [3]. In such contexts, the true dstances n the deal pont model are generally naccessble n any drect way, but t s nevertheless stll possble to learn useful nformaton from such data. Some of the smplest questons n ths model arse n the settng where we are gven an exstng embeddng of users/tems (as n a mature recommender system) and consder the on-lne problem of localzng a sngle new user or tem based on pared comparsons. (We wll return later to the problem of how ths ntal embeddng mght be generated.) In ths context, there are two man problems of nterest: localzng users and tems. In the case of localzng a user, [4] proposes a relatvely smple approach (whch we revew n greater detal n Secton 2) whch bulds on the observaton that each comparson provded by a user wth deal pont x dvdes the space R n n half and tells us whch sde of a hyperplane the pont x les n. As we wll see n Secton 3, however, localzng a new tem from the same knd of measurements s a bt more complcated n partcular, the comparsons now gve rse to a set of non-convex constrants. However, we wll demonstrate how to relax ths problem to a /16/$31.00 c 2016 IEEE

2 convex one that s extremely smlar to the approach we take for localzng users, and whch we demonstrate n Secton 4 to be hghly effectve n practce. In Secton 5 we return to the problem of generatng an embeddng of users/tems n the frst place. One approach would be to generate an tem embeddng ndependently usng a varety of methods, such as multdmensonal scalng appled to a set of tem features, and then localze each user usng the approach descrbed above. A more robust approach that uses only pared comparsons s descrbed n [5], whch proposes a general purpose algorthm for nonmetrc multdmensonal scalng based on semdefnte programmng. Unfortunately, ths algorthm scales poorly wth the number of total users/tems. In ths paper we consder an alternatve and hghly scalable scheme based on the teratve applcaton of the algorthms descrbed n Sectons 2 and 3, whch we demonstrate to be surprsngly effectve. 2. LOCALIZING A USER We frst revew the method of localzng a new user (gven an exstng embeddng of tems) presented n [4]. Each bnary pared comparson of the form user x prefers tem q (k) to tem q (k) j, whch we denote x q (k) 2 2 x q (k) j 2 2, s represented n the optmzaton problem as a constrant. Here, q refers to the tem that s preferred (over q j ), and the superscrpt (k) ndcates that the two tems are represented n the k th comparson. In the objectve functon of our optmzaton problem, we mnmze the total magntude of comparson volatons (expressed n the vector of slack varables ξ) plus a regularzaton term: mnmze x, ξ 1 2 x x k subject to x q (k) ξ k 0 c k ξ k 2 2 x q (k) j ξ k The values {c k } represent how heavly volatons of each comparson are penalzed relatve to each other and the regularzaton term; adjustng {c k } s a natural way to represent the relatve confdence we have n each the accuracy of each comparson. In the objectve functon, x 0 represents an ntal estmate of the deal pont for the user to be localzed (n the case where we do not have a prevous estmate of x, we ntalze x 0 to the orgn or the center of the embeddng of tems). The regularzaton term x x serves three purposes: () t constrans x when the set of comparsons does not result n a completely bounded feasble regon; () t regularzes the soluton to avod senstvty to outlers; and () t allows us to bas the soluton towards a pre-exstng estmate, whch can be very helpful when such an estmate exsts and our comparsons are nosy or small n number. Whle perhaps not ntally obvous, the optmzaton problem n (1) s a standard quadratc program wth lnear (1) constrants. To see ths, one needs to smplfy the constrant, elmnatng the x 2 2 term from both sdes and rearrangng to obtan the lnear constrant: 2 ( q (k) j q (k) ) T x + q (k) 2 2 q (k) j 2 2 ξ k 0 (2) From ths form, we can see that each lnear constrant resultng from a bnary pared comparson defnes a half-space n R n where the localzed user x lves. Fgure 1 shows a geometrc nterpretaton of the feasble regon created by several comparsons. See [4] for further detals. From here we can smplfy the optmzaton problem by assgnng the vector 2(q (k) j q (k) ) T from each comparson to the rows of matrx 2 2 q (k) j 2 2 as entres of the A and the scalar entres q (k) column vector b. Wth ths notaton, the optmzaton problem takes ts fnal form as: mnmze x, ξ 1 2 x x c T ξ subject to Ax + b ξ 0 ξ 0 where the nequalty sgn n both constrants s appled element-wse. Ths s a standard quadratc program, and s solvable usng a convex optmzaton software package such as CVX [6, 7]. 3. LOCALIZING AN ITEM To localze a new tem gven an exstng embeddng of users/tems, we begn by constructng an optmzaton problem smlar to (1). However, when localzng a new tem, t s useful to dvde the constrants nto two groups: () those dervng from comparsons n whch the tem to be localzed, q, s the tem preferred to some alternatve q (k) j, and () those for whch q represents the tem preferred less than some q (k). We wll let T + and T ndex these two sets of constrants n our optmzaton problem as follows: mnmze q, ξ 1 2 q q k (3) c k ξ k (4) subject to x (k) q 2 2 x (k) q (k) j ξ k k T + x (k) q (k) 2 2 x (k) q ξ k k T ξ k 0 As llustrated geometrcally n Fgure 2, ths set of constrants makes the problem nonconvex. On the one hand, each constrant n T + (where q s the preferred tem) defnes a hypersphere wthn whch the localzed tem must le. Constrants of ths type are convex (but quadratc, n contrast to the lnear constrants that arse when localzng a user). On the other hand, each constrant n T (where q s the less preferred tem) defnes a hypersphere wthn whch the localzed tem cannot le. The ncluson of the constrants n T + make the optmzaton problem n (4) nonconvex.

(a) Constrant resultng n convex feasble regon (shaded) (b) Constrant resultng n

2: Feasble regon n R 2 for (a) comparsons where the tem to be localzed s the preferred

To make the optmzaton problem convex, we consder one possble convex relaxaton; we combne

lnear constrants, leads to a computatonally effcent algorthm, and produces good results

To descrbe our procedure, we wll suppose that q denotes the tem that we wsh to update.

of the form x (k) q 2 2 x (k) q (k) j 2 2 x (l) q (l) 2 2 x (l) q 2 2 and x (k) x (l).

terms nvolvng q cancel when addng, and x (k) x (l) s necessary to avod makng the lnear

For each nequalty, the sdes not contanng the tem that we wsh to localze (q ) are

We then combne the nequaltes as x (k) q 2 2 c {}}{ x (k) q (k) j 2 2 + x (l) q (l) 2 2 x

d Expandng the norm n each nequalty, we note that combnng the comparsons results n the

quadratc constrants nto a sngle lnear one.

3: Lnear constrant generated by the combnaton of two Gray shadng represents the feasble

the feasble regon defned by the hyperplane resultng from the combnaton of two comparsons.

We can substtute these constrants nto (4) to obtan a lnearly-constraned quadratc problem

x (l) ) T assgned to a row of A and each c d + x (l) 2 2 x (k) 2 2 assgned to an entry

of the two It s mportant to note that whle the lnear relaxaton obtaned by ths procedure

of many such pars of constrants may stll be an effectve smplfcaton of the orgnal set of

Note that f we have a large number of comparsons, the number of vald combnatons T + T

In ths case, we smply choose a subset of m combnatons at random, although more ntellgent

3 (a) Constrant resultng n convex feasble regon (shaded) (b) Constrant resultng n non-convex feasble regon (shaded) Fg. 2: Feasble regon n R 2 for (a) comparsons where the tem to be localzed s the preferred tem, and (b) comparsons where the tem to be localzed s the less preferred tem. To make the optmzaton problem convex, we consder one possble convex relaxaton; we combne pars of comparsons by smply addng them, creatng a sngle lnear constrant from two quadratc constrants. Whle other convex relaxatons are possble, we wll see that ths smple method results n lnear constrants, leads to a computatonally effcent algorthm, and produces good results n practce. To descrbe our procedure, we wll suppose that q denotes the tem that we wsh to update. We can combne constrants by addng them to produce useful lnear constrants when the par s of the form x (k) q 2 2 x (k) q (k) j 2 2 x (l) q (l) 2 2 x (l) q 2 2 and x (k) x (l). The requrement that q appear on opposte sdes of the two nequaltes ensures the quadratc terms nvolvng q cancel when addng, and x (k) x (l) s necessary to avod makng the lnear term n the resultng constrant zero and the constrant vacuous. For each nequalty, the sdes not contanng the tem that we wsh to localze (q ) are constants, whch we denote c and d. We then combne the nequaltes as x (k) q 2 2 c {}}{ x (k) q (k) j x (l) q (l) 2 2 x (l) q 2 }{{} 2. d Expandng the norm n each nequalty, we note that combnng the comparsons results n the cancellaton of the quadratc term q 2 2, smplfyng the par of more computatonally expensve quadratc constrants nto a sngle lnear one. Addng and smplfyng yelds ( 2 x (l) x (k)) T q c d + x (l) 2 2 x (k) 2 2. (7) (5) (6) Fg. 3: Lnear constrant generated by the combnaton of two quadratc constrants. Gray shadng represents the feasble regons resultng from each of the two pared comparsons; blue hatched shadng represents the feasble regon defned by the hyperplane resultng from the combnaton of two comparsons. Note that the lnear relaxaton results n a much larger feasble regon. We can substtute these constrants nto (4) to obtan a lnearly-constraned quadratc problem that has the same form as the user localzaton optmzaton problem n (3), wth each 2(x (k) x (l) ) T assgned to a row of A and each c d + x (l) 2 2 x (k) 2 2 assgned to an entry of b. Fgure 3 shows a geometrc nterpretaton of the lnear constrant nduced by a combnaton of the two quadratc constrants. It s mportant to note that whle the lnear relaxaton obtaned by ths procedure results n a much larger feasble regon (n partcular, one that s unbounded), the combnaton of many such pars of constrants may stll be an effectve smplfcaton of the orgnal set of nonconvex constrants. Note that f we have a large number of comparsons, the number of vald combnatons T + T could be extremely large. In ths case, we smply choose a subset of m combnatons at random, although more ntellgent choces are lkely possble (partcularly n real-world applcatons, where the dstrbuton of users and tems present n comparsons wll be hghly uneven). It s also mportant to note that only pars of comparsons that produce a hyperplane lke the one shown n Fgure 3, where the hyperplane splts x (k) and x (l), are combned n ths way. To only select combnatons wth ths behavor, we also requre that ether a T x (k) < b < a T x (l) or a T x (l) < b < a T x (k) for each combnaton. 4. SIMULATIONS In ths secton, we demonstrate the performance of these approaches to localzng a new user or tem. For now we assume a pror knowledge of an exstng embeddng of users/tems, and use our lst of pared comparsons to localze a new user or tem n the embeddng. In each smulaton, a ground truth embeddng of many users and tems s generated unformly on the unt square n R n. Comparsons are generated by randomly selectng a user x (k) and two tems q 1 and q 2, q 1 q 2. For each comparson, q 1 and q 2 are assgned to q (k) and q (k) j to make them

of the total number of comparsons avalable to the system, whch contans 100 users

(b) consstent wth x (k) q (k) 2 x (k) q (k) j 2.

tems, and performance s measured by computng the l 2 dstance between the estmated

We repeat each smulaton 30 tmes and report the mean and standard devaton.

user based on the number of comparsons we have avalable.

results n a large error, but ths error drops off rapdly wth an ncreasng number of

Note that n ths paper we consder comparsons chosen randomly; smlar performance

compare were adaptvely selected (see, for example, the approaches n [8, 9, 10]).

Ths s expected; on the unt square n R n, the maxmum possble error grows as n.

tem gven an exstng embeddng of users/tems.

from the (possbly large) set of vald combnatons; here, we pck m = 20000 combnatons.

localze a new tem wth accuracy comparable to that of the user localzaton problem.

constrants seems to dscard a sgnfcant amount of nformaton. 5.

can localze a new user or tem nto an embeddng of users and tems gven only a lst of

embeddng usng a lst of pared When presented wth an exstng embeddng, we can terate

Because the accuracy wth whch we can localze a new pont depends on the accuracy of

4 (a) Fg. 4: Localzaton error measured n terms of l 2 dstance from ground truth as a functon of the total number of comparsons avalable to the system, whch contans 100 users and 100 tems. (a) User localzaton error. (b) Item localzaton error. (b) consstent wth x (k) q (k) 2 x (k) q (k) j 2. In each of the followng smulatons, our ground truth embeddng conssts of 100 users and 100 tems. Comparsons are generated unformly at random from the complete set of users and tems, and performance s measured by computng the l 2 dstance between the estmated pont and the ground truth. We repeat each smulaton 30 tmes and report the mean and standard devaton. In the frst smulaton, we demonstrate the accuracy wth whch we can localze a new user based on the number of comparsons we have avalable. As shown n Fgure 4(a), a very small number of comparsons nvolvng the localzed user results n a large error, but ths error drops off rapdly wth an ncreasng number of comparsons. Note that n ths paper we consder comparsons chosen randomly; smlar performance could be obtaned wth a much smaller number of comparsons f the user and tems to compare were adaptvely selected (see, for example, the approaches n [8, 9, 10]). Fgure 4(a) also shows the large mpact of dmensonalty on the l 2 recovery error. Ths s expected; on the unt square n R n, the maxmum possble error grows as n. These results are consstent wth recent theoretcal results for ths problem establshed n [10, 11]. Next, we apply the tem localzaton procedure descrbed n Secton 3 to localze a new tem gven an exstng embeddng of users/tems. As noted n the prevous secton, we select a subset of comparson combnatons at random from the (possbly large) set of vald combnatons; here, we pck m = combnatons. In Fgure 4(b), we show that our convex relaxaton of addng comparsons allows us to localze a new tem wth accuracy comparable to that of the user localzaton problem. Ths may be somewhat surprsng snce our procedure for generatng a convex set of constrants seems to dscard a sgnfcant amount of nformaton. 5. GENERATING A COMPLETE EMBEDDING Usng the procedures descrbed n Sectons 2 and 3, we can localze a new user or tem nto an embeddng of users and tems gven only a lst of pared comparsons. However, our localzaton procedures can also be used to mprove an exstng embeddng usng a lst of pared comparsons. When presented wth an exstng embeddng, we can terate through each pont and use our localzaton procedure to mprove ts accuracy. Because the accuracy wth whch we can localze a new pont depends on the accuracy of every other pont n the embeddng, each update can mprove the current pont as well as the results of future teratons. The teratve localzaton procedure s as follows: () Create an ntal embeddng (f no ntal embeddng s avalable, ntalze each pont at random); () Iterate through each user and tem n the embeddng, updatng the user/tem s locaton at each step by solvng the optmzaton problem from Sectons 2 or 3 as approprate. Repeat ths teraton untl convergence. Usng ths procedure, we can use a lst of pared comparsons to mprove the accuracy of a nosy embeddng of users and tems. Further, even f we do not have an ntal estmate of the embeddng, we can use ths procedure to generate one from scratch wth surprsng accuracy. Improvng an exstng embeddng. To evaluate the potental of ths approach, we begn by generatng comparsons between users and tems as n the experments n Secton 4, but after generatng the comparsons we perturb every user and tem n the embeddng wth Gaussan nose. Fgure 5 shows the reducton of error our procedure can acheve for three dfferent perturbaton nose levels (σ emb ). We show performance both n terms of the l 2 dstance to the ground truth (across all ponts n the embeddng, after applyng an affne transformaton, also known as the Procrustes dstance) but also n

Fg. 5: Reducton n nose acheved for a nosy embeddng n R 2, measured n terms of percent of all comparsons volated and mean l 2 recovery error calculated usng

Gven a sutable number of teratons, we observe that we can reduce the error n the embeddng to approxmately the same level, regardless of the perturbaton nose

Our teratve procedure s surprsngly effectve at denosng an embeddng, even when t s corrupted wth enough nose to volate up to 35% of the gven comparsons.

In ths smulaton, we agan create a ground truth embeddng of users and tems dstrbuted unformly on the unt square n R n.

the lst of comparsons to recreate the embeddng.

Fgure 6 shows that even wth a random ntal embeddng (approxmately 50% of comparsons volated), we can generate an embeddng wth only a modest level of error n

Whle the mean l 2 recovery error n Fgure 6 shows error ncreasng wth dmensonalty, the error does not grow as quckly as the scalng of dstances wth dmenson

DISCUSSION In ths work, we have extended the optmzaton-based method n [4] for localzng a new user nto an embeddng of users/tems usng a set of bnary pared

Next, we develop a method for teratvely mprovng a nosy embeddng or generatng an embeddng from scratch usng only a set of pared comparsons.

generate an embeddng wth good accuracy, even n hgh dmensons.

generatng an approxmaton of the entre embeddng only requres a few teratons through each user/tem n the embeddng (whch can be performed n parallel).

Ths algorthm scales far better n terms of both computaton and memory requrements than exstng procedures based on semdefnte programmng such as generalzed

5 Fg. 5: Reducton n nose acheved for a nosy embeddng n R 2, measured n terms of percent of all comparsons volated and mean l 2 recovery error calculated usng the Procrustes dstance. terms of the percentage of observed comparsons whch are volated by the learned embeddng. Gven a sutable number of teratons, we observe that we can reduce the error n the embeddng to approxmately the same level, regardless of the perturbaton nose level. Generatng an embeddng from scratch. Our teratve procedure s surprsngly effectve at denosng an embeddng, even when t s corrupted wth enough nose to volate up to 35% of the gven comparsons. Ths suggest that t may be possble to obtan smlar results when we have no a pror knowledge of the embeddng at all, generatng an embeddng from scratch. In ths smulaton, we agan create a ground truth embeddng of users and tems dstrbuted unformly on the unt square n R n. We use ths ground truth embeddng to generate a lst of bnary pared comparsons, then ntalze the reconstructed embeddng to a random set of ponts and use only the lst of comparsons to recreate the embeddng. As n the prevous experments, we measure the accuracy of our reconstructon usng the Procrustes dstance between the reconstructon and orgnal embeddng. Fgure 6 shows that even wth a random ntal embeddng (approxmately 50% of comparsons volated), we can generate an embeddng wth only a modest level of error n only a few passes over the entre dataset. Whle the mean l 2 recovery error n Fgure 6 shows error ncreasng wth dmensonalty, the error does not grow as quckly as the scalng of dstances wth dmenson (whch ncreases as n). 6. DISCUSSION In ths work, we have extended the optmzaton-based method n [4] for localzng a new user nto an embeddng of users/tems usng a set of bnary pared comparsons. Ths paper has two man contrbutons: frst, we derve a convex relaxaton whch allows us to use a smlar procedure for localzng tems nto an embeddng. Next, we develop a method for teratvely mprovng a nosy embeddng or generatng an embeddng from scratch usng only a set of pared comparsons. Smulatons on synthetcally generated data show that gven a sutable number of comparsons, we can localze new users and tems extremely accurately and can generate an embeddng wth good accuracy, even n hgh dmensons. The algorthm s computatonally effcent and easly parallelzable: localzng a sngle pont only requres solvng a smple lnearly-constraned quadratc program, and generatng an approxmaton of the entre embeddng only requres a few teratons through each user/tem n the embeddng (whch can be performed n parallel). Ths effcency s key, because n real-world recommendaton systems the number of users and tems may be extremely large. Ths algorthm scales far better n terms of both computaton and memory requrements than exstng procedures based on semdefnte programmng such as generalzed non-metrc multdmensonal scalng [5]. However, the performance of our algorthm mght be sgnfcantly mproved by an accurate knowledge (or estmate) of several landmark ponts, whch could be generated by such a procedure. Fnally, we brefly note that the deal pont model descrbed n ths paper, whle closely related, s dstnct from the low-rank model used n matrx completon approaches whch have recently ganed much attenton, e.g., [12, 13]. Although both models suppose preferences are guded by a small number of factors, the deal pont model leads to preferences that are non-monotonc functons of those attrbutes. There s emprcal evdence that the deal pont model captures user behavor more accurately than factorzaton based approaches do [14]. Nevertheless, the results n [15, 16, 17, 18], whch consder bnary observatons, pared comparsons, and other more general ordnal measurements n the low-rank context, share a smlar nspraton as ths work.

Fg. 6: Recovery error for a generated embeddng measured n terms of percent of all comparsons volated and mean l 2 recovery error calculated usng the Procrustes dstance. 7. REFERENCES [1] C.

[3] F. Radlnsk and T. Joachms, Actve exploraton for learnng rankngs from clckthrough data, n Proc. ACM SIGKDD Int. Conf. on Knowledge, Dscovery, and Data Mnng (KDD), San Jose, CA, August 2007, ACM.

[5] S. Agarwal, J. Wlls, L. Cayton, G. Lanckret, D. Kregman, and S. Belonge, Generalzed non-metrc multdmensonal scalng, n Proc. Int. Conf. Art. Intell. Stat. (AISTATS), San Juan, Puerto Rco, Mar.

[7] Mchael Grant and Stephen Boyd, Graph mplementatons for nonsmooth convex programs, n Recent Advances n Learnng and Control, V. Blondel, S. Boyd, and H. Kmura, Eds.

Nowak, Actve rankng usng parwse comparsons, n Proc. Adv. n Neural Processng Systems (NIPS), Granada, Span, Dec. 2011. [9] K. Jameson and R.

6 Fg. 6: Recovery error for a generated embeddng measured n terms of percent of all comparsons volated and mean l 2 recovery error calculated usng the Procrustes dstance. 7. REFERENCES [1] C. Coombs, Psychologcal scalng wthout a unt of measurement, Psych. Rev., vol. 57, no. 3, pp , [2] H. Davd, The method of pared comparsons, London, UK: Charles Grffn & Company Lmted, [3] F. Radlnsk and T. Joachms, Actve exploraton for learnng rankngs from clckthrough data, n Proc. ACM SIGKDD Int. Conf. on Knowledge, Dscovery, and Data Mnng (KDD), San Jose, CA, August 2007, ACM. [4] M. Davenport, Lost wthout a compass: Nonmetrc trangulaton and landmark multdmensonal scalng, n Proc. IEEE Int. Work. Comput. Adv. Mult-Sensor Adaptve Processng (CAMSAP), Sant Martn, Dec [5] S. Agarwal, J. Wlls, L. Cayton, G. Lanckret, D. Kregman, and S. Belonge, Generalzed non-metrc multdmensonal scalng, n Proc. Int. Conf. Art. Intell. Stat. (AISTATS), San Juan, Puerto Rco, Mar [6] Mchael Grant and Stephen Boyd, CVX: Matlab software for dscplned convex programmng, verson 2.1, Mar [7] Mchael Grant and Stephen Boyd, Graph mplementatons for nonsmooth convex programs, n Recent Advances n Learnng and Control, V. Blondel, S. Boyd, and H. Kmura, Eds., Lecture Notes n Control and Informaton Scences, pp Sprnger-Verlag Lmted, 2008, boyd/ graph_dcp.html. [8] K. Jameson and R. Nowak, Actve rankng usng parwse comparsons, n Proc. Adv. n Neural Processng Systems (NIPS), Granada, Span, Dec [9] K. Jameson and R. Nowak, Low-dmensonal embeddng usng adaptvely selected ordnal data, n Proc. Allerton Conf. Communcaton, Control, and Computng, Montcello, IL, [10] A. Massmno and M. Davenport, As you lke t: Localzaton va pared comparsons, Preprnt, [11] A. Massmno and M. Davenport, Bnary stable embeddng va pared comparsons, n Proc. IEEE Work. Stat. Sgnal Processng, Palma de Mallorca, Span, June [12] J. Renne and N. Srebro, Fast maxmum margn matrx factorzaton for collaboratve predcton, n Proc. Int. Conf. Machne Learnng, Bonn, Germany, Aug [13] E. Candès and B. Recht, Exact matrx completon va convex optmzaton, Found. Comput. Math., vol. 9, no. 6, pp , [14] A. Maydeu-Olvares and U. Böckenholt, Modelng preference data, The SAGE handbook of quanttatve methods n psychology, pp , [15] M. Davenport, Y. Plan, E. van den Berg, and M. Wootters, 1-bt matrx completon, Inf. Inference, vol. 3, no. 3, pp , [16] Y. Lu and S. Negahban, Indvdualzed rank aggregraton usng nuclear norm regularzaton, arxv: , [17] D. Park, J. Neeman, J. Zhang, S. Sanghav, and I. Dhllon, Preference completon: Large-scale collaboratve rankng form parwse comparsons, n Proc. Int. Conf. Machne Learnng, Llle, France, July [18] S. Oh, K. Thekumparampl, and J. Xu, Collaboratvely learnng preferences from ordnal data, n Proc. Adv. n Neural Processng Systems (NIPS), Montréal, Québec, Dec

Parallelism for Nested Loops with Non-uniform and Flow Dependences

Parallelism for Nested Loops with Non-uniform and Flow Dependences Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr