Advanced Click Models & their Applications to IR

Transcription

1 Advanced Click Models & their Applications to IR (Afternoon block 1) Aleksandr Chuklin, Ilya Markov Maarten de Rijke University of Amsterdam Google Switzerland SIGIR 2015 Tutorial AC IM MdR Advanced Click Models & their Applications to IR 1

2 Afternoon block 1 Outline 1 Introduction 2 Advanced Click Models 3 Demo 4 Recap AC IM MdR Advanced Click Models & their Applications to IR 2

3 Introduction Advanced Click Models Demo Recap Who are we? Aleksandr Software engineer at Google AC IM MdR Advanced Click Models & their Applications to IR 3

4 Introduction Advanced Click Models Demo Recap Who are we? Aleksandr Ilya Software engineer at Google Postdoc at U. Amsterdam AC IM MdR Advanced Click Models & their Applications to IR 3

5 Introduction Advanced Click Models Demo Recap Who are we? Aleksandr Ilya Maarten Software engineer at Google Postdoc at U. Amsterdam Professor at U. Amsterdam AC IM MdR Advanced Click Models & their Applications to IR 3

6 Aims Describe existing click models in a unified way, so that different models can easily be related to each other AC IM MdR Advanced Click Models & their Applications to IR 4

7 Aims Describe existing click models in a unified way, so that different models can easily be related to each other Compare commonly used click models AC IM MdR Advanced Click Models & their Applications to IR 4

8 Aims Describe existing click models in a unified way, so that different models can easily be related to each other Compare commonly used click models Provide ready-to-use formulas and implementations of existing click models and detail parameter estimation procedures to facilitate the development of new ones AC IM MdR Advanced Click Models & their Applications to IR 4

9 Aims Describe existing click models in a unified way, so that different models can easily be related to each other Compare commonly used click models Provide ready-to-use formulas and implementations of existing click models and detail parameter estimation procedures to facilitate the development of new ones Summarize current efforts on click model evaluation evaluation approaches, datasets and software packages AC IM MdR Advanced Click Models & their Applications to IR 4

10 Aims Describe existing click models in a unified way, so that different models can easily be related to each other Compare commonly used click models Provide ready-to-use formulas and implementations of existing click models and detail parameter estimation procedures to facilitate the development of new ones Summarize current efforts on click model evaluation evaluation approaches, datasets and software packages Provide an overview of click model applications and directions for future development of click models AC IM MdR Advanced Click Models & their Applications to IR 4

11 Structure of the tutorial Two parts, with two blocks each An Introduction to Click Models for Web Search Advanced Click Models and their applications to IR AC IM MdR Advanced Click Models & their Applications to IR 5

12 Structure of the tutorial Two parts, with two blocks each An Introduction to Click Models for Web Search Advanced Click Models and their applications to IR Part II (this afternoon) Introduction Advanced Click Models Break Demo Guest presentation by Yiqun Liu (Tsinghua University): Building a Click Model: From Idea to Implementation Applications Future Directions AC IM MdR Advanced Click Models & their Applications to IR 5

13 Materials Complete draft of the book on which the tutorial is based: Aleksandr Chuklin, Ilya Markov, Maarten de Rijke. Click Models for Web Search. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool, July, 2015 AC IM MdR Advanced Click Models & their Applications to IR 6

14 Materials Complete draft of the book on which the tutorial is based: Aleksandr Chuklin, Ilya Markov, Maarten de Rijke. Click Models for Web Search. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool, July, 2015 Copy of the slides Parts I and II AC IM MdR Advanced Click Models & their Applications to IR 6

15 Materials Complete draft of the book on which the tutorial is based: Aleksandr Chuklin, Ilya Markov, Maarten de Rijke. Click Models for Web Search. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool, July, 2015 Copy of the slides Parts I and II Code and data samples to follow live demos AC IM MdR Advanced Click Models & their Applications to IR 6

16 Materials Complete draft of the book on which the tutorial is based: Aleksandr Chuklin, Ilya Markov, Maarten de Rijke. Click Models for Web Search. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool, July, 2015 Copy of the slides Parts I and II Code and data samples to follow live demos See for updates and additional materials AC IM MdR Advanced Click Models & their Applications to IR 6

17 What is a click model? AC IM MdR Advanced Click Models & their Applications to IR 7

18 Why click models? Understand users Simulate users Evaluate search Improve search AC IM MdR Advanced Click Models & their Applications to IR 8

19 Click models... AC IM MdR Advanced Click Models & their Applications to IR 9

20 Click models... A set of events/random variables AC IM MdR Advanced Click Models & their Applications to IR 9

21 Click models... A set of events/random variables A directed graph with nodes corresponding to events and edges corresponding to dependencies between events AC IM MdR Advanced Click Models & their Applications to IR 9

22 Click models... A set of events/random variables A directed graph with nodes corresponding to events and edges corresponding to dependencies between events Conditional probabilities in the nodes of the dependency graph expressed by parameters AC IM MdR Advanced Click Models & their Applications to IR 9

23 Click models... A set of events/random variables A directed graph with nodes corresponding to events and edges corresponding to dependencies between events Conditional probabilities in the nodes of the dependency graph expressed by parameters A correspondence between the model s parameters and features of a SERP and user s query AC IM MdR Advanced Click Models & their Applications to IR 9

24 Position-based model ru uq E u A u C u document u AC IM MdR Advanced Click Models & their Applications to IR 10

25 Position-based model P(C u = 1) = P(E u = 1) P(A u = 1) P(A u = 1) = α uq P(E u = 1) = γ ru AC IM MdR Advanced Click Models & their Applications to IR 11

26 Click models for web search Random click model CTR models Position-based model Cascade model Dependent click model Dynamic Bayesian network model User browsing model AC IM MdR Advanced Click Models & their Applications to IR 12

27 Parameter estimation Maximum likelihood estimation Expectation-maximization and other AC IM MdR Advanced Click Models & their Applications to IR 13

28 Evaluation Log-likelihood Perplexity and other AC IM MdR Advanced Click Models & their Applications to IR 14

29 Afternoon block 1 Outline 1 Introduction 2 Advanced Click Models 3 Demo 4 Recap AC IM MdR Advanced Click Models & their Applications to IR 15

30 Advanced click models Aggregated search Beyond a single SERP User diversity Beyond clicks Non-linear examination Using features AC IM MdR Advanced Click Models & their Applications to IR 16

31 Aggregated search AC IM MdR Advanced Click Models & their Applications to IR 17

32 Federated click model (FCM) P(A = 1) = χ c AC IM MdR Advanced Click Models & their Applications to IR 18

33 Federated click model (FCM) P(A = 1) = χ c P(E r = 1 A = 0) = ɛ r AC IM MdR Advanced Click Models & their Applications to IR 18

34 Federated click model (FCM) P(A = 1) = χ c P(E r = 1 A = 0) = ɛ r P(E r = 1 A = 1) = ɛ r + (1 ɛ r )β dist AC IM MdR Advanced Click Models & their Applications to IR 18

35 Federated click model (FCM) P(A = 1) = χ c P(E r = 1 A = 0) = ɛ r P(E r = 1 A = 1) = ɛ r + (1 ɛ r )β dist χ c can depend on vertical rank: χ rvert vertical content: χ vert AC IM MdR Advanced Click Models & their Applications to IR 18

36 Federated click model (FCM) P(A = 1) = χ c P(E r = 1 A = 0) = ɛ r P(E r = 1 A = 1) = ɛ r + (1 ɛ r )β dist χ c can depend on vertical rank: χ rvert vertical content: χ vert dist = r u r vert can be negative AC IM MdR Advanced Click Models & their Applications to IR 18

37 Vertical-aware click model (VCM) 1 Attraction bias 2 Global bias 3 First place bias 4 Sequence bias AC IM MdR Advanced Click Models & their Applications to IR 19

38 Intent-aware click models P(C 1,..., C n ) = P(I = i) P(C 1,..., C n I = i) i AC IM MdR Advanced Click Models & their Applications to IR 20

39 Beyond a single SERP AC IM MdR Advanced Click Models & their Applications to IR 21

40 Pagination Pagination is clicked 5 10% of times AC IM MdR Advanced Click Models & their Applications to IR 22

41 Pagination Pagination is clicked 5 10% of times Should be considered as a separate clickable object AC IM MdR Advanced Click Models & their Applications to IR 22

42 Task-centric click model (TCM) P(M = 1) = µ SIGIR 2015 Santiago Chile Santiago hotels AC IM MdR Advanced Click Models & their Applications to IR 23

43 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν SIGIR 2015 Santiago Chile Santiago hotels AC IM MdR Advanced Click Models & their Applications to IR 23

44 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν H r = 1 previous examination of u r SIGIR 2015 Santiago Chile Santiago hotels AC IM MdR Advanced Click Models & their Applications to IR 23

45 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν H r = 1 previous examination of u r P(F r = 1 H r = 0) = 1 P(F r = 1 H r = 1) = φ SIGIR 2015 Santiago Chile Santiago hotels AC IM MdR Advanced Click Models & their Applications to IR 23

46 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν H r = 1 previous examination of u r P(F r = 1 H r = 0) = 1 P(F r = 1 H r = 1) = φ P(E r = 1 E <r, S <r, C <r ) = ɛ r SIGIR 2015 Santiago Chile Santiago hotels P(A r = 1) = α ur q C r = 1 M = 1, F r = 1, E r = 1, A r = 1 AC IM MdR Advanced Click Models & their Applications to IR 23

47 User diversity Picture taken from XXX. AC IM MdR Advanced Click Models & their Applications to IR 24

48 Matrix factorization click model (MFCM) Documents Documents (U) Queries α uqv Users Tensor Users (V) Queries (Q) P(A u = 1) = α uqv α uqv N ( U u Q q V v, σ 2) AC IM MdR Advanced Click Models & their Applications to IR 25

49 Other models with users UBM-user P(A u = 1) = α uq ɛ v AC IM MdR Advanced Click Models & their Applications to IR 26

50 Other models with users UBM-user P(A u = 1) = α uq ɛ v LOG-rd-user (logistic model) P(E r = 1 E r 1 = 1, C <r ) = σ (γ rr + ε v ) P(A r = 1) = σ (α ur q + ɛ v ) 1 σ(x) = 1 + e x AC IM MdR Advanced Click Models & their Applications to IR 26

51 Other models with users UBM-user P(A u = 1) = α uq ɛ v LOG-rd-user (logistic model) P(E r = 1 E r 1 = 1, C <r ) = σ (γ rr + ε v ) DIL-user (dilution model) P(A r = 1) = σ (α ur q + ɛ v ) 1 σ(x) = 1 + e x P(E r = 1 E r 1 = 1, C <r ) = (β v ) r 1 (λ v ) k (µ v ) r r P(A r = 1) = α ur qɛ v AC IM MdR Advanced Click Models & their Applications to IR 26

52 Beyond clicks AC IM MdR Advanced Click Models & their Applications to IR 27

53 Mousing and scrolling P(E r = 1 h H : r r h ) = 1 H set of hovered documents AC IM MdR Advanced Click Models & their Applications to IR 28

54 Mousing and scrolling P(E r = 1 h H : r r h ) = 1 P(E r = 1 r V ) = 1 H set of hovered documents V set of shown documents AC IM MdR Advanced Click Models & their Applications to IR 28

55 2) which is usually remembered by users, they carefully read and comprehend the results selected from Stage 1 and based on the reading, decide whether to click on it or not. In Figure 1, the user's examination on the third result might come into Stage 2, and the user remembers that it has been read. Introduction Advanced Click Models Demo Recap From skimming to reading Figure 1. A user's eye fixation sequence and the corresponding explicit feedback on result reading for top results in a search session of query 学雷锋作文 " (Essays on learning from Lei Feng in Chinese). Radius of circle means fixation length. Our proposed two-stage examination model and the choice of two stages are inspired by the attention selection mechanism [33] which is widely accepted in cognitive psychology studies. It says that human attention consists of two functionally independent, hierarchical stages: An early, pre-attentive stage (similar to Stage 1) that operates without capacity limitation and in parallel across the entire visual field, followed by a later, attentive limited-capacity stage (similar to Stage 2) that can deal with only one item (or at most a few items) at a time. Attention selection is one of the basic Picture taken from Y. Liu, C. Wang, K. Zhou, J. Nie, M. Zhang, and S. Ma. From skimming to reading: A two-stage examination model for web search. In CIKM, ACM Press. some concluding 2. RELAT Two lines of re current endeavo from the user s further in this lin we propose a focuses on the and exploits mou We follow this li for relevance est 2.1 Eye-tra The application a considerable industry. Eye-tr real-time eye understand how Granka et al. [1 use eye-tracking and sequence p found that the d closely related to into how user's intents. Wang e result appearanc behavior for Navalpakkam et AC IM MdR Advanced Click Models & their Applications to IR 29

56 2) which is usually remembered by users, they carefully read and comprehend the results selected from Stage 1 and based on the reading, decide whether to click on it or not. In Figure 1, the user's examination on the third result might come into Stage 2, and the user remembers that it has been read. Introduction Advanced Click Models Demo Recap From skimming to reading Figure 1. A user's eye fixation sequence and the corresponding explicit feedback on result reading for top results in a search session of query 学雷锋作文 " (Essays on learning from Lei Feng in Chinese). Radius of circle means fixation length. P(C u = 1) = P(C u = 1 R u = 1) P(R u = 1 F u = 1) P(F u = 1) Our proposed two-stage examination model and the choice of two stages are inspired by the attention selection mechanism [33] which is widely accepted in cognitive psychology studies. It says that human attention consists of two functionally independent, hierarchical stages: An early, pre-attentive stage (similar to Stage 1) that operates without capacity limitation and in parallel across the entire visual field, followed by a later, attentive limited-capacity stage (similar to Stage 2) that can deal with only one item (or at most a few items) at a time. Attention selection is one of the basic R u reading (examining) a document F u fixing eyes on a document Picture taken from Y. Liu, C. Wang, K. Zhou, J. Nie, M. Zhang, and S. Ma. From skimming to reading: A two-stage examination model for web search. In CIKM, ACM Press. some concluding 2. RELAT Two lines of re current endeavo from the user s further in this lin we propose a focuses on the and exploits mou We follow this li for relevance est 2.1 Eye-tra The application a considerable industry. Eye-tr real-time eye understand how Granka et al. [1 use eye-tracking and sequence p found that the d closely related to into how user's intents. Wang e result appearanc behavior for Navalpakkam et AC IM MdR Advanced Click Models & their Applications to IR 29

57 position Predicting mouse movements ed to examinao layouts. Figure 5: Probability of starting at di erent rank positions based on mouse-tracking data. inear scan user f partially obnon-sequential ly very close to OM model asto novel page uru present a but limit anal- [28]. a fundamental e design. Logploited to imticated models ion include asattractiveness existing SERP formation, our odels to incorodels of visual visual salience sely related to n s r h a n s r h a (a) Linear Scan n s r h a n s r h a (b) Actual Mousing Figure 6: Hinton diagrams representing the probability of transitioning between pairs page modules, including the ten algorithmic results as well as navigational modules (n), query suggestion (s), related searches (r), search history (h), and advertisements (a). Each row shows the conditional distribution of the second position (column id) given the first position (row id). Figure 6(a) reflects the transition probabilities assumed under the linear scan assumption. Figure 6(b) displaying the empirical transition probabilities from Picture taken from F. Diaz, R.W. White, G. Buscher, and D. Liebling. Robust models of mouse movement on dynamic web search results pages. In CIKM, ACM Press AC IM MdR Advanced Click Models & their Applications to IR 30

58 position Predicting mouse movements ed to examinao layouts. Figure 5: Probability of starting at di erent rank positions based on mouse-tracking data. inear scan user f partially obnon-sequential ly very close to OM model asto novel page uru present a but limit anal- [28]. a fundamental e design. Logploited to imticated models ion include asattractiveness existing SERP formation, our odels to incorodels of visual visual salience sely related to n s r h a n s r h a (a) Linear Scan n s r h a n s r h a (b) Actual Mousing Figure 6: Hinton diagrams representing the probability of transitioning between pairs page modules, including the ten algorithmic results as well as navigational modules (n), query suggestion (s), related searches (r), search history (h), and advertisements (a). Each row shows the conditional distribution of the second position (column id) given the first position (row id). Figure 6(a) reflects the transition probabilities assumed under the linear scan assumption. Figure 6(b) displaying the empirical transition probabilities from Estimate the probability of mousing element j after element i Picture taken from F. Diaz, R.W. White, G. Buscher, and D. Liebling. Robust models of mouse movement on dynamic web search results pages. In CIKM, ACM Press AC IM MdR Advanced Click Models & their Applications to IR 30

59 position Predicting mouse movements ed to examinao layouts. Figure 5: Probability of starting at di erent rank positions based on mouse-tracking data. inear scan user f partially obnon-sequential ly very close to OM model asto novel page uru present a but limit anal- [28]. a fundamental e design. Logploited to imticated models ion include asattractiveness existing SERP formation, our odels to incorodels of visual visual salience sely related to n s r h a n s r h a (a) Linear Scan n s r h a n s r h a (b) Actual Mousing Figure 6: Hinton diagrams representing the probability of transitioning between pairs page modules, including the ten algorithmic results as well as navigational modules (n), query suggestion (s), related searches (r), search history (h), and advertisements (a). Each row shows the conditional distribution of the second position (column id) given the first position (row id). Figure 6(a) reflects the transition probabilities assumed under the linear scan assumption. Figure 6(b) displaying the empirical transition probabilities from Estimate the probability of mousing element j after element i Use Maximum Likelihood Estimation and Farley-Ring Model Picture taken from F. Diaz, R.W. White, G. Buscher, and D. Liebling. Robust models of mouse movement on dynamic web search results pages. In CIKM, ACM Press AC IM MdR Advanced Click Models & their Applications to IR 30

60 Noise-aware click model (NCM) P(C r = 1 E r = 1, R r = 1, N r = 0) = α ur q P(C r = 1 E r = 1, R r = 0, N r = 0) = 0 R relevance AC IM MdR Advanced Click Models & their Applications to IR 31

61 Noise-aware click model (NCM) P(C r = 1 E r = 1, R r = 1, N r = 0) = α ur q P(C r = 1 E r = 1, R r = 0, N r = 0) = 0 P(C r = 1 E r = 1, N r = 1) = β q R relevance N noisy environment AC IM MdR Advanced Click Models & their Applications to IR 31

62 Non-linear examination t (c) Mouse Data Picture taken from F. Diaz, R.W. White, G. Buscher, and D. Liebling. Robust models of mouse movement on dynamic web search results pages. In CIKM, ACM Press AC IM MdR Advanced Click Models & their Applications to IR 32

63 Click models for aggregated search AC IM MdR Advanced Click Models & their Applications to IR 33

64 Temporal click models POM-based click models User actions as a Markov chain Temporal information in search logs AC IM MdR Advanced Click Models & their Applications to IR 34

65 Temporal click models POM-based click models User actions as a Markov chain Temporal information in search logs Temporal Hidden Click Model (THCM) P(E r+1 = 1 E r = 1) = α P(E r 1 = 1 E r = 1) = γ AC IM MdR Advanced Click Models & their Applications to IR 34

66 Temporal click models POM-based click models User actions as a Markov chain Temporal information in search logs Temporal Hidden Click Model (THCM) P(E r+1 = 1 E r = 1) = α P(E r 1 = 1 E r = 1) = γ Partially Sequential Click Model (PSCM) Between adjacent clicks users examine results in a single direction Users can skip results without examining them AC IM MdR Advanced Click Models & their Applications to IR 34

67 Whole-page click model (WPC) Search 1 1 encyclopedia ram that prints -dates the age ages... ren of all ages any language cement to any orld World for free Hello World Software Buy hello world Transitions between blocks 5 (macro-model) are modeled as a Markov 6 chain e idea of Hello ing English in as that Search 9 9 tion (b) Arrangement (c) Mouse Data representation of mouse-tracking data. The session sequence for this data wou Picture taken from F. Diaz, R.W. White, G. Buscher, and D. Liebling. Robust models of mouse movement on dynamic web search results pages. In CIKM, ACM Press m 0 m 0 AC IM MdR Advanced Click Models & their Applications to IR m 35 m 1

68 Whole-page click model (WPC) Search 1 1 encyclopedia ram that prints -dates the age ages... ren of all ages any language cement to any orld World for free e idea of Hello ing English in as that... Hello World Software Buy hello world Transitions between blocks 5 (macro-model) are modeled as a Markov 6 chain Behavior within a block 7 (micro-model) is modeled using standard 8 click models Search 9 9 tion (b) Arrangement (c) Mouse Data representation of mouse-tracking data. The session sequence for this data wou Picture taken from F. Diaz, R.W. White, G. Buscher, and D. Liebling. Robust models of mouse movement on dynamic web search results pages. In CIKM, ACM Press m 0 m 0 AC IM MdR Advanced Click Models & their Applications to IR m 35 m 1

69 Using features AC IM MdR Advanced Click Models & their Applications to IR 36

70 Using features 290 Benchmarking Learning-to-Rank Algorithms For the Gov corpus, 64 features were extracted for each query document pair, as shown in Table 6.2. For the OHSUMED corpus, 40 features were extracted in total, as shown in Table 6.3. Table 6.2 Learning features of TREC. ID Feature description 1 Term frequency (TF) of body 2 TF of anchor 3 TF of title 4 TF of URL 5 TF of whole document 6 Inverse document frequency (IDF) of body 7 IDF of anchor 8 IDF of title 9 IDF of URL 10 IDF of whole document 11 TF*IDF of body 12 TF*IDF of anchor 13 TF*IDF of title 14 TF*IDF of URL 15 TF*IDF of whole document 16 Document length (DL) of body 17 DL of anchor 18 DL of title 19 DL of URL 20 DL of whole document 21 BM25 of body 22 BM25 of anchor 23 BM25 of title 24 BM25 of URL 25 BM25 of whole document 26 LMIR.ABS of body 27 LMIR.ABS of anchor 28 LMIR.ABS of title 29 LMIR.ABS of URL 30 LMIR.ABS of whole document 31 LMIR.DIR of body AC IM MdR Advanced 32 Click LMIR.DIR Models of anchor & their Applications to IR 36

71 Regression-based click models P(C u = 1) = e Z Z = w i f i AC IM MdR Advanced Click Models & their Applications to IR 37

72 Regression-based click models P(C u = 1) = e Z Z = w i f i Factorization machines AC IM MdR Advanced Click Models & their Applications to IR 37

73 Regression-based click models P(C u = 1) = e Z Z = w i f i Factorization machines Neural networks (for click prediction in sponsored search) AC IM MdR Advanced Click Models & their Applications to IR 37

74 General click model (GCM) Click model parameters can depend on other attributes (e.g., time of the day, user browser, length of a URL, etc.) P(C r = 1 E r = 1) = P i θ f user i + j θ f url j + ɛ > 0 AC IM MdR Advanced Click Models & their Applications to IR 38

75 General click model (GCM) Click model parameters can depend on other attributes (e.g., time of the day, user browser, length of a URL, etc.) P(C r = 1 E r = 1) = P i θ f user i + j θ f url j + ɛ > 0 CM, DCM, CCM and DBN can be considered as special cases of GCM AC IM MdR Advanced Click Models & their Applications to IR 38

76 Afternoon block 1 Outline 1 Introduction 2 Advanced Click Models 3 Demo 4 Recap AC IM MdR Advanced Click Models & their Applications to IR 39

77 Live demo: Building an advanced click model Demo AC IM MdR Advanced Click Models & their Applications to IR 40

78 Setup AC IM MdR Advanced Click Models & their Applications to IR 41

79 Data and tools Publicly-available click log from Yandex (Relevance Prediction Challenge 2011) AC IM MdR Advanced Click Models & their Applications to IR 42

80 Data and tools Publicly-available click log from Yandex (Relevance Prediction Challenge 2011) PyClick an open-source Python library of click models for web search AC IM MdR Advanced Click Models & their Applications to IR 42

81 Task-centric click model (TCM) P(M = 1) = µ SIGIR 2015 Santiago Chile Santiago hotels AC IM MdR Advanced Click Models & their Applications to IR 43

82 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν SIGIR 2015 Santiago Chile Santiago hotels AC IM MdR Advanced Click Models & their Applications to IR 43

83 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν H r = 1 previous examination of u r SIGIR 2015 Santiago Chile Santiago hotels AC IM MdR Advanced Click Models & their Applications to IR 43

84 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν H r = 1 previous examination of u r P(F r = 1 H r = 0) = 1 P(F r = 1 H r = 1) = φ SIGIR 2015 Santiago Chile Santiago hotels AC IM MdR Advanced Click Models & their Applications to IR 43

85 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν H r = 1 previous examination of u r P(F r = 1 H r = 0) = 1 P(F r = 1 H r = 1) = φ P(E r = 1 E <r, S <r, C <r ) = ɛ r SIGIR 2015 Santiago Chile Santiago hotels P(A r = 1) = α ur q C r = 1 M = 1, F r = 1, E r = 1, A r = 1 AC IM MdR Advanced Click Models & their Applications to IR 43

86 Task-centric click model (TCM) P(M = 1) = µ P(N = 1 M = 0) = 1 P(N = 1 M = 1) = ν H r = 1 previous occurence of u r P(F r = 1 H r = 0) = 1 P(F r = 1 H r = 1) = φ P(E r = 1 E <r, S <r, C <r ) = ɛ r SIGIR 2015 Santiago Chile Santiago hotels P(A r = 1) = α ur q C r = 1 M = 1, F r = 1, E r = 1, A r = 1 AC IM MdR Advanced Click Models & their Applications to IR 43

87 Demo starting point TCM class copied from PBM Basic experiment Load the train and test data Train TCM Test TCM Additional classes explained during the demo AC IM MdR Advanced Click Models & their Applications to IR 44

88 Preliminaries AC IM MdR Advanced Click Models & their Applications to IR 45

89 Preliminaries AC IM MdR Advanced Click Models & their Applications to IR 45

90 Preliminaries 1 Initial run (a) Load data and train the model (b) Get the same output as for PBM, save the output 2 Switch from single query sessions to search tasks (a) Extend SearchSession into TaskCentricSearchSession (b) Create SearchTask (c) Extend the data loading procedure, re-load data (d) Output the number of search tasks 3 Train on search tasks (a) Pass search tasks to TCM.train() (b) Get an error 4 Implement new inference (a) Create TaskCentricInference as a subclass of EMInference (b) Pass a search task into Param.update() (c) Use the new inference in TCM, re-run training (d) Get the same output as for PBM (compare with the previously saved output) AC IM MdR Advanced Click Models & their Applications to IR 46

91 Afternoon block 1 Outline 1 Introduction 2 Advanced Click Models 3 Demo 4 Recap AC IM MdR Advanced Click Models & their Applications to IR 47

92 Afternoon block 1 Recap A very quick introduction to click models A landscape of advanced click models Demo AC IM MdR Advanced Click Models & their Applications to IR 48

93 Afternoon block 1 Recap A very quick introduction to click models A landscape of advanced click models Demo After the break Demo (cont d) Invited talk Applications Future directions AC IM MdR Advanced Click Models & their Applications to IR 48

94 Acknowledgments All content represents the opinion of the authors which is not necessarily shared or endorsed by their respective employers and/or sponsors. AC IM MdR Advanced Click Models & their Applications to IR 49