Understanding User s Search Behavior towards Spiky Events

Transcription

1 Understanding User s Search Behavior towards Spiky Events Behrooz Mansouri Mohammad Zahedi Ricardo Campos Mojgan Farhoodi Maseud Rahgozar Ricardo Campos TempWeb WWW Lyon, France, Apr 23, 2018 Iran Telecommunication Research Center Instituto Politécnico de Tomar LIAAD INESC TEC

2 Agenda Introduction 1 Related Work 2 Contributions 3 Experimental Setting 4 Experimental Results 5 Conclusions 6

3 Internet Stats

4 cinema Social occasions attracting society s attention Haiti earthquake When these events occur, multiple spikes can be observed in the query logs triggered by a change in the user s behavior and an increase in the frequency of the user s query, which depending of the query, may be manifested in several different ways Halloween

5 Length of Queries During the US presidential election week, queries related to this event were 2.5 longer than on the week before Use of Temporal Expressions 12.5% of the queries related to this event contained temporal expressions (e.g., 2016) Clicked Pages Most of the clicked pages during the election week concern pages which are constantly being updated, while one year before refer to static pages such as Wikipedia

6 It turns out evident that understanding the user s intent and their dynamics is an essential step to develop more effective web search engines. This is not a new problem and has been extensively studied over the last few years: [Subasic & Castillo. The effect of query bursts on web search. In WI-IAT 10] [Zhang et al. Learning Recurrent Event Queries for Web Search. In EMNLP 10] [Shokouhi. Detecting Seasonal Queries by Time-Series Analysis. In SIGIR 11] [Kulkarni et al. Understanding Temporal Query Dynamics. In WSDM 11] [Radinsky et al. Behavioral Dynamics on the Web: Learning, Modeling and Prediction. In TOIS 13] [Gupta & Berberich. Temporal Query Classification at Different Granularities. In SPIRE 15] [Karmaker. Modeling the Influence of Popular Trending Events on User Search Behavior. In WWW 17]

7 Spiky Events We go a step ahead, by analyzing user s behavior toward spiky events under a new taxonomy, which we believe better describes this kind of happenings. Spiky Events Periodic APeriodic Ongoing Commemorative Special Days Predictable Unpredictable

8 Contributions Our aim is to study the user s behavior of each one of these categories according to several different features:

9 Window We aim to do this under several different time frames First, we begin by defining two main time frames: the event window and the normal window Normal Window Event Window Normal Window

10 Event Window Within the Event Window we then define between three different time periods Event Frame Pre-Event Frame Post-Event Frame Pre-Event Frame A week before the event takes place Post-Event Frame A week after the event has occurred April 23-27, 2018 April 16-22, 2018 April 28 - May 4, 2018 Event Frame Event Occurrence

11

12 Event Name Ballon d'or Valentine September 11th Keywords Ballon d'or, FIFA world player of the year, Valentine, Valentino, The day of Love September 11, 9/11, al-qaeda attacks to US

13 APeriodic Periodic Query Frequency We aim to understand how the frequency of queries related to spiky events change before, during and after the event Average percentage of queries submitted to the search engine regarding each class of spiky events in each time frame. Event Type NW EW preef EF EF post NW EW NW P OnGoing 7.56% 32.51% 23.16% 63.23% 36.77% P SpecialDays 38.14% 26.77% 5.31% 70.22% 29.78% P Commemorative 30.64% 17.47% 10.11% 58.23% 41.77% A Predictable 8.64% 15.32% 33.94% 57.90% 42.10% A UnPredictable 1.29% 13.90% 45.58% 60.77% 39.23% 62.07% of the queries (on average) have been submitted to the search engine during the event window

14 APeriodic Periodic Query Frequency We aim to understand how the frequency of queries related to spiky events change before, during and after the event Average percentage of queries submitted to the search engine regarding each class of spiky events in each time frame. Event Type NW EW preef EF EF post NW EW NW P OnGoing 7.56% 32.51% 23.16% 63.23% 36.77% P SpecialDays 38.14% 26.77% 5.31% 70.22% 29.78% P Commemorative 30.64% 17.47% 10.11% 58.23% 41.77% A Predictable 8.64% 15.32% 33.94% 57.90% 42.10% A UnPredictable 1.29% 13.90% 45.58% 60.77% 39.23% Another interesting thing is that we can observe a significant number of POnGoingrelated queries, which confirms the fact that people try to look for information even after the event is finished.

15 APeriodic Periodic Query Frequency We aim to understand how the frequency of queries related to spiky events change before, during and after the event Average percentage of queries submitted to the search engine regarding each class of spiky events in each time frame. Event Type NW EW preef EF EF post NW EW NW P OnGoing 7.56% 32.51% 23.16% 63.23% 36.77% P SpecialDays 38.14% 26.77% 5.31% 70.22% 29.78% P Commemorative 30.64% 17.47% 10.11% 58.23% 41.77% A Predictable 8.64% 15.32% 33.94% 57.90% 42.10% A UnPredictable 1.29% 13.90% 45.58% 60.77% 39.23% In contrast PspecialDays and Pcommemorative are mostly submitted during the Pre and the Event Frame, which given the characteristic of these queries is understandable as people try to anticipate their needs

16 APeriodic Periodic Query Frequency We aim to understand how the frequency of queries related to spiky events change before, during and after the event Average percentage of queries submitted to the search engine regarding each class of spiky events in each time frame. Event Type NW EW preef EF EF post NW EW NW P OnGoing 7.56% 32.51% 23.16% 63.23% 36.77% P SpecialDays 38.14% 26.77% 5.31% 70.22% 29.78% P Commemorative 30.64% 17.47% 10.11% 58.23% 41.77% A Predictable 8.64% 15.32% 33.94% 57.90% 42.10% A UnPredictable 1.29% 13.90% 45.58% 60.77% 39.23% For the Aperiodic, their frequency is significantly higher in the Post Event Frame as people tend to look for information in the coming days (as is the case of the Aunpredictable Paris Terrorist Attack )

17 Query Length We aim to study the distribution of query length for each type of spiky event as several experiments have proven that existing retrieval methods perform, in general, worse for long queries than for short ones Boxplot of query length distribution for each of spiky event category. Query length distribution is nearly the same for all categories ranging from 4 to 7 terms (5.7 terms on average; 2.2 more terms than other types of Persian Queries)

18 Query Length We aim to study the distribution of query length for each type of spiky event as several experiments have proven that existing retrieval methods perform worse, in general, for long queries than for short ones Boxplot of query length distribution for each of spiky event category. Interestingly, queries of POnGoing events were longer than the other query categories with 6.1 terms per query on average.

19 Query Length We aim to study the distribution of query length for each type of spiky event as several experiments have proven that existing retrieval methods perform worse, in general, for long queries than for short ones Boxplot of query length distribution for each of spiky event category. We can also observe that the frequency of queries with more than 7 terms drops considerable

20 Query Length We aim to study the distribution of query length for each type of spiky event as several experiments have proven that existing retrieval methods perform worse, in general, for long queries than for short ones Boxplot of query length distribution for each of spiky event category. We can also observe that the frequency of queries with more than 7 terms drops considerable Queries with more than 13 terms are a rarity

21 Use of Temporal Expressions We aim to understand how temporality explicitly affects the different types of spiky events (e.g., Haiti earthquake 2011) To this purpose we used our recently developed Persian time tagger tool ParsTime [Behrooz et al. In ECIR 18] We found that 23.7% of the total spiky queries contain temporal expressions which is higher than the 1.5% stated by Nunes et al. for generic queries This confirms that queries related to spiky events are temporal dependent

22 Use of Temporal Expressions We aim to understand how temporality explicitly affects the different types of spiky events (e.g., Haiti earthquake 2011) To this purpose we used our recently developed Persian time tagger tool ParsTime [Behrooz et al. In ECIR 18] 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Temporal queries Pspecial days Pcommemorative Pongoing Apredictable Aunpredictable Overall, we found that queries regarding periodic events contain more temporal expressions when compared to Aperiodic, which is understandable given the different characteristics of both types of events

23 Use of Temporal Expressions We aim to understand how temporality explicitly affects the different types of spiky events (e.g., Haiti earthquake 2011) To this purpose we used our recently developed Persian time tagger tool ParsTime [Behrooz et al. In ECIR 18] 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Temporal queries Pspecial days Pcommemorative Pongoing Apredictable Aunpredictable POnGoing contains more temporal expressions on average, as different episodes of an event usually appear associated to an year (for instance, for the Fifa World Cup usually people issue Fifa World Cup 2014 or Fifa World Cup 2018 )

24 APeriodic Periodic Use of Temporal Expressions We aim to understand how temporality explicitly affects the different types of spiky events (e.g., Haiti earthquake 2011) Distribution of queries with temporal expressions according to the defined time frames Event Type NW EW preef EF EF post NW EW NW P OnGoing 7.5% 10.4% 21.7% 39.6% 60.4% To this purpose we used our recently developed Persian time tagger tool ParsTime [Behrooz et al. In ECIR 18] P SpecialDays 35.9% 23.4% 10.7% 70.0% 30.0% P Commemorative 5.4% 4.8% 9.8% 20.0% 80.0% A Predictable 13.2% 2.6% 30.0% 45.8% 54.2% A UnPredictable 0.0% 0.0% 24.5% 24.5% 75.5% Another thing that stands out here is that 70% of the queries with temporal expressions for PSpecialDays were posed to the search engine during the event window mostly concerning current events and how to celebrate them (e.g. valentine )

25 Diversity of Clicked Web Pages We aim to study the variability of clicked web pages to understand if there is any different effect due to the diverse topics of spiky queries 8,0 7,0 6,0 5,0 4,0 3,0 Event Window Normal Window Low click entropy indicates that users prefer a few unique URLs regarding the event High click entropy indicates that a wide range of URLs were target by the users 2,0 1,0 0,0 Pspecial days Pcommemorative Pongoing Apredictable Aunpredictable The average click entropy is 6.21 The highest click entropy belongs to PSpecialDays The lowest click entropy belongs to AUnpredictable

26 Diversity of Clicked Web Pages We aim to study the variability of clicked web pages to understand is there is any different effect due to the diverse topics of spiky queries 8,0 7,0 6,0 5,0 4,0 3,0 Event Window Normal Window Low click entropy indicates that users prefer a few unique URLs regarding the event High click entropy indicates that a wide range of URLs were target by the users 2,0 1,0 0,0 Pspecial days Pcommemorative Pongoing Apredictable Aunpredictable By looking at the figure we can conclude that users tend to click, during the Event Window, on more pages on periodic events than on Aperiodic

27 Content of Clicked Web Pages We aim to analyze the content of clicked web pages For each one of the 100 events we considered the top-200 pages that were more frequently clicked (totalizing 20K pages) We then asked three annotators to look at the content of each web page and to manually classify them with regards to: Recency Web pages providing information about the most recent episode of the event Oldness Old episodes of the event Wikipedia-like page A type of navigational page which gives general information about the event

28 APeriodic Periodic Content of Clicked Web Pages We aim to analyze the content of clicked web pages For each one of the 100 events we considered the top-200 pages that were more frequently clicked (totalizing 20K pages) Event Type Average percentage of clicked pages in each content category, considering type of spiky event during Event and normal Recent Page Wikipedia-Like pages Old Pages EW NW EW NW EW NW P OnGoing 94.3% 4.9% 4.1% 91.3% 1.6% 3.8% P SpecialDays 54.7% 7.3% 44.9% 90.6% 0.4% 2.1% P Commemorative 92.1% 51.4% 2.5% 7.1% 5.4% 41.5% A Predictable 83.8% 14.9% 12.6% 77.3% 3.6% 7.8% A UnPredictable 94.2% 34.8% 4.2% 35.4% 1.6% 29.8% We then asked three annotators to look at the content of each web page and to manually classify them with regards to: When talking about Event Window, the preferred pages by users were recent pages While during Normal Window the preferred pages are Wikipedia-like pages

29 APeriodic Periodic Content of Clicked Web Pages We aim to analyze the content of clicked web pages For each one of the 100 events we considered the top-200 pages that were more frequently clicked (totalizing 20K pages) Event Type Average percentage of clicked pages in each content category, considering type of spiky event during Event and normal Recent Page Wikipedia-Like pages Old Pages EW NW EW NW EW NW P OnGoing 94.3% 4.9% 4.1% 91.3% 1.6% 3.8% P SpecialDays 54.7% 7.3% 44.9% 90.6% 0.4% 2.1% P Commemorative 92.1% 51.4% 2.5% 7.1% 5.4% 41.5% A Predictable 83.8% 14.9% 12.6% 77.3% 3.6% 7.8% A UnPredictable 94.2% 34.8% 4.2% 35.4% 1.6% 29.8% We then asked three annotators to look at the content of each web page and to manually classify them with regards to: Interestingly, one can note that user s issuing Pcommemorative and AUnPredictable queries use to resort to old pages during the Normal Window

30 We studied queries related to 100 spiky events (20 per each of the 5 categories proposed): ongoing events (e.g., Olympics ) historical events (e.g., September 11th attacks ) special days events (e.g., Valentine ) aperiodic expected events (e.g., Lunar eclipse ) unanticipated events (e.g., Earthquake )

31 To understand the changes in user s behavior we defined 2 time-frames: Normal Window Event Window Pre Event Frame Event Frame Pos Event Frame

32 Investigated query frequency, query length and use of temporal expressions Also investigated click entropy and content of clicked pages For each category of spiky events, users behavior is different And thus different services such as query suggestion, query auto-completion and result ranking can be provided to the user.

33