Learning to Rank. from heuristics to theoretic approaches. Hongning Wang

Transcription

1 Learning to Rank from heuristics to theoretic approaches Hongning Wang

2 Congratulations Job Offer from Bing Core Ranking team Design the ranking module for Bing.com CS 6501: Information Retrieval 2

3 How should I rank documents? Answer: Rank by relevance! CS 6501: Information Retrieval 3

4 Relevance?! CS 6501: Information Retrieval 4

5 The Notion of Relevance (Rep(q), Rep(d)) Similarity Relevance P(r=1 q,d) r {0,1} Probability of Relevance Relevance constraints [Fang et al. 04] Div. from Randomness (Amati & Rijsbergen 02) P(d q) or P(q d) Probabilistic inference Different rep & similarity Vector space model (Salton et al., 75) Prob. distr. model (Wong & Yao, 89) Regression Model (Fuhr 89) Learn. To Rank (Joachims 02, Berges et al. 05) Doc generation Generative Model Classical prob. Model (Robertson & Sparck Jones, 76) Query generation LM approach (Ponte & Croft, 98) (Lafferty & Zhai, 01a) Different inference system Prob. concept space model (Wong & Yao, 95) Inference network model (Turtle & Croft, 91) CS 6501: Information Retrieval 5

6 Relevance Estimation Query matching Language model BM25 Vector space cosine similarity Document importance PageRank HITS CS 6501: Information Retrieval 6

7 Did I do a good job of ranking PageRank BM25 documents? CS 6501: Information Retrieval 7

8 Did I do a good job of ranking documents? IR evaluations metrics Precision@K MAP NDCG CS 6501: Information Retrieval 8

9 Take advantage of different relevance estimator? Ensemble the cues Linear? aa 1 BBBBBB + αα 2 LLLL + αα 3 PPPPPPPPPPPPPPPP + αα 4 HHHHHHHH Non-linear? {αα 1 = 0.4, αα 2 = 0.2, αα 3 = 0.1, αα 4 = 0.1} {MMMMMM = 0.20, NNNNNNNN = 0.6} {αα 1 = 0.2, αα 2 = 0.2, αα 3 = 0.3, αα 4 = 0.1} {MMMMMM = 0.12, NNNNNNNN = 0.5 } Decision tree-like {αα 1 = 0.1, αα 2 = 0.1, αα 3 = 0.5, αα 4 = 0.2} {MMMMMM = 0.18, NNNNNNNN = 0.7} BM25>0.5 True False True LM > 0.1 False True PageRank > 0.3 False r= 1.0 r= 0.7 r= 0.4 r= 0.1 CS 6501: Information Retrieval 9

10 What if we have thousands of features? Is there any way I can do better? Optimizing the metrics automatically! Where to find those tree structures? How to determine those ααs? CS 6501: Information Retrieval 10

11 Rethink the task Given: (query, document) pairs represented by a set of relevance estimators, a.k.a., features DocID BM25 LM PageRank Label Needed: a way of combining the estimators DD ff qq, dd ii=1 DD ordered dd ii=1 Criterion: optimize IR metrics P@k, MAP, NDCG, etc. Key! CS 6501: Information Retrieval 11

12 Machine Learning Input: { XX 1, YY 1, XX 2, YY 2,, (XX nn, YY nn )}, where XX ii RR NN, YY ii RR MM Objective function : OO(YY, YY) Output: ff XX YY, such that ff = argmax ff FF OO(ff (XX), YY) NOTE: We will only talk about supervised learning. Classification OO YY, YY = δδ(yy = YY) istical_classification Regression OO YY, YY = YY YY ion_analysis CS 6501: Information Retrieval 12

13 Learning to Rank General solution in optimization framework Input: (qq ii, dd 1 ), yy 1, (qq ii, dd 2 ), yy 2,, (qq ii, dd nn ), yy nn, where d n RR NN, yy ii {0,, LL} Objective: O = {P@k, MAP, NDCG} Output: ff qq, dd YY, s.t., ff = argmax ff FF OO(ff (qq, dd), YY) DocID BM25 LM PageRank Label CS 6501: Information Retrieval 13

14 Challenge: how to optimize? Evaluation metric recap Average Precision DCG Order is essential! ff order metric Not continuous with respect to f(x)! CS 6501: Information Retrieval 14

15 Approximating the objective function! Pointwise Fit the relevance labels individually Pairwise Fit the relative orders Listwise Fit the whole order CS 6501: Information Retrieval 15

16 Pointwise Learning to Rank Ideally perfect relevance prediction leads to perfect ranking ff score order metric Reducing ranking problem to Regression OO ff QQ, DD, YY = ii ff qq ii, dd ii yy ii Subset Ranking using Regression, D.Cossock and T.Zhang, COLT 2006 (multi-)classification OO ff QQ, DD, YY = ii δδ(ff qq ii, dd ii = yy ii ) Ranking with Large Margin Principles, A. Shashua and A. Levin, NIPS 2002 CS 6501: Information Retrieval 16

17 Subset Ranking using Regression D.Cossock and T.Zhang, COLT 2006 Fit relevance labels via regression Emphasize more on relevant documents Weights on each document Most positive document CS 6501: Information Retrieval 17

18 Ranking with Large Margin Principles A. Shashua and A. Levin, NIPS 2002 Goal: correctly placing the documents in the corresponding category and maximize the margin Y=1 Y=0 Y=2 Reduce the violations αααα CS 6501: Information Retrieval 18

19 Ranking with Large Margin Principles A. Shashua and A. Levin, NIPS 2002 Ranking lost is consistently decreasing with more training data CS 6501: Information Retrieval 19

20 What did we learn Machine learning helps! Derive something optimizable More efficient and guided CS 6501: Information Retrieval 20

21 Deficiency Cannot directly optimize IR metrics (0 1, 2 0) worse than (0-2, 2 4) Position of documents are ignored Penalty on documents 1 at higher positions should dd 0 1 be larger -2 Favor the queries with more documents dd 1-1 dd 1 dd 2 2 CS 6501: Information Retrieval 21

22 Pairwise Learning to Rank Ideally perfect partial order leads to perfect ranking ff partial order order metric Ordinal regression OO ff QQ, DD, YY = ii jj δδ(yy ii > yy jj )δδ(ff qq ii, dd ii < ff(qq ii, dd ii )) Relative ordering between different documents is significant E.g., (0-2, 2 4) is better than (0 1, 2 0) Large body of research dd 1 1 dd 2 2 dd 1 CS 6501: Information Retrieval dd 1 22

23 Optimizing Search Engines using Clickthrough Data Thorsten Joachims, KDD 02 Minimizing the number of mis-ordered pairs yy 1 > yy 2, yy 2 > yy 3, yy 1 > yy linear combination of features yy 1 > yy 2 ff qq, dd = ww TT XX qq,dd Keep the relative orders RankSVM CS 6501: Information Retrieval 23

24 Optimizing Search Engines using Clickthrough Data Thorsten Joachims, KDD 02 How to use it? ff scccccccc order yy 1 > yy 2 > yy 3 > yy 4 CS 6501: Information Retrieval 24

25 Optimizing Search Engines using Clickthrough Data Thorsten Joachims, KDD 02 What did it learn from the data? Linear correlations Positive correlated features Negative correlated features CS 6501: Information Retrieval 25

26 Optimizing Search Engines using Clickthrough Data Thorsten Joachims, KDD 02 How good is it? Test on real system CS 6501: Information Retrieval 26

27 An Efficient Boosting Algorithm for Combining Preferences Y. Freund, R. Iyer, et al. JMLR 2003 Smooth the loss on mis-ordered pairs yyii >yy jj PPPP dd ii, dd jj eeeeee[ff qq, dd jj ff qq, dd ii ] square loss hinge loss (RankSVM) 0/1 loss exponential loss CS 6501: Information Retrieval from Pattern Recognition 27 and Machine Learning, P337

28 An Efficient Boosting Algorithm for Combining Preferences Y. Freund, R. Iyer, et al. JMLR 2003 RankBoost: optimize via boosting Vote by a committee BM25 PageRank Cosine Updating Pr(dd ii, dd jj ) Credibility of each committee member (ranking feature) from Pattern Recognition and Machine Learning, P658 CS 6501: Information Retrieval 28

29 An Efficient Boosting Algorithm for Combining Preferences Y. Freund, R. Iyer, et al. JMLR 2003 How good is it? CS 6501: Information Retrieval 29

30 A Regression Framework for Learning Ranking Functions Using Relative Relevance Judgments Zheng et al. SIRIG 07 Non-linear ensemble of features Object: yyii >yy jj max 0, ff qq, dd jj ff qq, dd ii 2 Gradient descent boosting tree Boosting tree Using regression tree to minimize the residuals rr tt (qq, dd, yy) = OO tt (qq, dd, yy) ff tt 1 (qq, dd, yy) BM25>0.5 True False PageRank LM > 0.1 > 0.3 True False True False r= 1.0 r= 0.7 CS 6501: Information Retrieval 30 r= 0.4 r= 0.1

31 A Regression Framework for Learning Ranking Functions Using Relative Relevance Judgments Zheng et al. SIRIG 07 Non-linear v.s. linear Comparing with RankSVM CS 6501: Information Retrieval 31

32 Where do we get the relative orders Human annotations Small scale, expensive to acquire Clickthroughs Large amount, easy to acquire CS 6501: Information Retrieval 32

33 What did we learn Predicting relative order Getting closer to the nature of ranking Promising performance in practice Pairwise preferences from click-throughs CS 6501: Information Retrieval 33

34 Listwise Learning to Rank Can we directly optimize the ranking? ff order metric Tackle the challenge Optimization without gradient CS 6501: Information Retrieval 34

35 From RankNet to LambdaRank to LambdaMART: An Overview Christopher J.C. Burges, 2010 Minimizing mis-ordered pair => maximizing IR metrics? Mis-ordered pairs: 6 Mis-ordered pairs: 4 AP: 5 8 AP: 5 12 DCG: DCG: Position is crucial! CS 6501: Information Retrieval 35

36 From RankNet to LambdaRank to Weight the mis-ordered pairs? LambdaMART: An Overview Christopher J.C. Burges, 2010 Some pairs are more important to be placed in the right order Inject into object function yyii >yy jj Ω dd ii, dd jj Inject into gradient λλ iiii = OO aaaaaaaaaa ΔOO Depend on the ranking of document i, j in the whole list exp wwδφ qq nn, dd ii, dd jj Change in original object, e.g., NDCG, if we switch the documents i and j, leaving the other documents unchanged Gradient with respect to approximated objective, i.e., exponential loss on mis-ordered pairs CS 6501: Information Retrieval 36

37 Lambda functions Gradient? Yes, it meets the sufficient and necessary condition of being partial derivative Lead to optimal solution of original problem? Empirically From RankNet to LambdaRank to LambdaMART: An Overview Christopher J.C. Burges, 2010 CS 6501: Information Retrieval 37

38 Evolution From RankNet to LambdaRank to LambdaMART: An Overview Christopher J.C. Burges, 2010 RankNet LambdaRank LambdaMART Objective Cross entropy over the pairs Unknown Unknown Gradient (λλ function) Gradient of cross entropy Gradient of cross entropy times pairwise change in target metric Gradient of cross entropy times pairwise change in target metric Optimization method neural network stochastic gradient descent Multiple Additive Regression Trees (MART) As we discussed in RankBoost Optimize solely by gradient Non-linear combination CS 6501: Information Retrieval 38

39 A Lambda tree From RankNet to LambdaRank to LambdaMART: An Overview Christopher J.C. Burges, 2010 splitting Combination of features CS 6501: Information Retrieval 39

40 A Support Vector Machine for Optimizing Average Precision Yisong Yue, et al., SIGIR 07 RankSVM Minimizing the pairwise loss SVM-MAP Minimizing the structural loss MAP difference Loss defined on the number of mis-ordered document pairs Loss defined on the quality of the whole list of ordered documents CS 6501: Information Retrieval 40

41 A Support Vector Machine for Optimizing Max margin principle Average Precision Yisong Yue, et al., SIGIR 07 Push the ground-truth far away from any mistakes one might make Finding the most likely violated constraints CS 6501: Information Retrieval 41

42 A Support Vector Machine for Optimizing Average Precision Yisong Yue, et al., SIGIR 07 Finding the most violated constraints MAP is invariant to permutation of (ir)relevant documents Maximize MAP over a series of swaps between relevant and irrelevant documents Right-hand side of constraints Start from the reverse order of ideal ranking Greedy solution CS 6501: Information Retrieval 42

43 A Support Vector Machine for Optimizing Experiment results Average Precision Yisong Yue, et al., SIGIR 07 CS 6501: Information Retrieval 43

44 Other listwise solutions Soften the metrics to make them differentiable Michael Taylor et al., SoftRank: optimizing nonsmooth rank metrics, WSDM'08 Minimize a loss function defined on permutations Zhe Cao et al., Learning to rank: from pairwise approach to listwise approach, ICML'07 CS 6501: Information Retrieval 44

45 What did we learn Taking a list of documents as a whole Positions are visible for the learning algorithm Directly optimizing the target metric Limitation The search space is huge! CS 6501: Information Retrieval 45

46 Summary Learning to rank Automatic combination of ranking features for optimizing IR evaluation metrics Approaches Pointwise Fit the relevance labels individually Pairwise Fit the relative orders Listwise Fit the whole order CS 6501: Information Retrieval 46

47 Experimental Comparisons My experiments 1.2k queries, 45.5K documents with 1890 features 800 queries for training, 400 queries for testing MAP ERR MRR ListNET LambdaMART RankNET RankBoost RankSVM AdaRank plogistic Logistic CS 6501: Information Retrieval 47

48 Connection with Traditional IR People have foreseen this topic long time ago Recall: probabilistic ranking principle CS 6501: Information Retrieval 48

49 Conditional models for P(R=1 Q,D) Basic idea: relevance depends on how well a query matches a document P(R=1 Q,D)=g(Rep(Q,D) θ) Rep(Q,D): feature representation of query-doc pair E.g., #matched terms, highest IDF of a matched term, doclen Using training data (with known relevance judgments) to estimate parameter θ Apply the model to rank new documents Special case: logistic regression a functional form CS 6501: Information Retrieval 49

50 Broader Notion of Relevance Social network Language Model Query Cosine BM25 Query relation Language Model Query Cosine BM25 Query relation Language Model Query Cosine BM25 Documents Documents Documents Click/View Linkage structure Visual Structure Linkage structure likes CS 6501: Information Retrieval 50

51 Future Tighter bounds Faster solution Larger scale Wider application scenario CS 6501: Information Retrieval 51

52 Resources Books Liu, Tie-Yan. Learning to rank for information retrieval. Vol. 13. Springer, Li, Hang. "Learning to rank for information retrieval and natural language processing." Synthesis Lectures on Human Language Technologies 4.1 (2011): Helpful pages Packages RankingSVM: RankLib: Data sets LETOR Yahoo! Learning to rank challenge CS 6501: Information Retrieval 52

53 References Liu, Tie-Yan. "Learning to rank for information retrieval." Foundations and Trends in Information Retrieval 3.3 (2009): Cossock, David, and Tong Zhang. "Subset ranking using regression." Learning theory (2006): Shashua, Amnon, and Anat Levin. "Ranking with large margin principle: Two approaches." Advances in neural information processing systems 15 (2003): Joachims, Thorsten. "Optimizing search engines using clickthrough data." Proceedings of the eighth ACM SIGKDD. ACM, Freund, Yoav, et al. "An efficient boosting algorithm for combining preferences." The Journal of Machine Learning Research 4 (2003): Zheng, Zhaohui, et al. "A regression framework for learning ranking functions using relative relevance judgments." Proceedings of the 30th annual international ACM SIGIR. ACM, CS 6501: Information Retrieval 53

54 References Joachims, Thorsten, et al. "Accurately interpreting clickthrough data as implicit feedback." Proceedings of the 28th annual international ACM SIGIR. ACM, Burges, C. "From ranknet to lambdarank to lambdamart: An overview." Learning 11 (2010): Xu, Jun, and Hang Li. "AdaRank: a boosting algorithm for information retrieval." Proceedings of the 30th annual international ACM SIGIR. ACM, Yue, Yisong, et al. "A support vector method for optimizing average precision." Proceedings of the 30th annual international ACM SIGIR. ACM, Taylor, Michael, et al. "Softrank: optimizing non-smooth rank metrics." Proceedings of the international conference WSDM. ACM, Cao, Zhe, et al. "Learning to rank: from pairwise approach to listwise approach." Proceedings of the 24th ICML. ACM, CS 6501: Information Retrieval 54

55 Ranking with Large Margin Principles A. Shashua and A. Levin, NIPS 2002 Maximizing the sum of margins Y=0 Y=1 Y=2 αααα CS 6501: Information Retrieval 55

56 AdaRank: a boosting algorithm for Loss defined by IR metrics qq QQ PPPP qq eeeeee[ OO(qq)] Optimizing by boosting information retrieval Jun Xu & Hang Li, SIGIR 07 BM25 PageRank Cosine Target metrics: MAP, NDCG, MRR Updating Pr(qq) Credibility of each committee member (ranking feature) from Pattern Recognition and Machine Learning, P658 CS 6501: Information Retrieval 56

57 Analysis of the Approaches What are they really optimizing? Relation with IR metrics g a p CS 6501: Information Retrieval 57

58 Pointwise Approaches Regression based Discount coefficients in DCG Classification based Regression loss Discount coefficients in DCG Classification loss CS 6501: Information Retrieval 58

59 From CS 6501: Information Retrieval 59

60 Discount coefficients in DCG From CS 6501: Information Retrieval 60

61 Listwise Approaches No general analysis Method dependent Directness and consistency CS 6501: Information Retrieval 61