Query Heartbeat: A Strange Property of Keyword Queries on the Web

Transcription

1 Query Heartbeat: A Strange Property of Keyword Queries on the Web Karthik.B.R Aditya Ramana Rachakonda Dr. Srinath Srinivasa International Institute of Information Technology, Bangalore. December 16, 2008 A Strange Property of Keyword Queries on the Web 1 / 27

2 Outline of Topics 1 Motivation 2 Introduction 3 Dataset 4 Approach 5 Query HeartBeat Properties 6 Results 7 Conclusion A Strange Property of Keyword Queries on the Web 2 / 27

3 Motivation 1 A Strange Property of Keyword Queries on the Web 3 / 27

4 Motivation Consider an example query on Google Trends A Strange Property of Keyword Queries on the Web 3 / 27

5 Motivation Consider an example query on Google Trends 1. Queries: aircondition, watersport. 1 A Strange Property of Keyword Queries on the Web 3 / 27

6 Motivation Consider an example query on Google Trends 1. Queries: aircondition, watersport. We see that the queries have a similar temporal behavior. 1 A Strange Property of Keyword Queries on the Web 3 / 27

7 Motivation Consider an example query on Google Trends 1. Queries: aircondition, watersport. We see that the queries have a similar temporal behavior. Also, the shape of the temporal patterns are same, even though the volumes differ. 1 A Strange Property of Keyword Queries on the Web 3 / 27

8 Introduction Objective was to observe the temporal property of keyword queries and cluster keywords based on their temporal properties. 2 A Strange Property of Keyword Queries on the Web 4 / 27

9 Introduction Objective was to observe the temporal property of keyword queries and cluster keywords based on their temporal properties. However, we found that generic keyword queries that have large enough volumes tend to have the same temporal shape. We call this shape the attractor distribution. 2 A Strange Property of Keyword Queries on the Web 4 / 27

10 Introduction Objective was to observe the temporal property of keyword queries and cluster keywords based on their temporal properties. However, we found that generic keyword queries that have large enough volumes tend to have the same temporal shape. We call this shape the attractor distribution. Visually similar phenomenon were also apparent when keywords were searched on Google Trends A Strange Property of Keyword Queries on the Web 4 / 27

11 Example Consider an example for queries yahoo, access. A Strange Property of Keyword Queries on the Web 5 / 27

12 Example Consider an example for queries yahoo, access. When their volumes are normalized A Strange Property of Keyword Queries on the Web 5 / 27

13 Dataset We used the AOL query log dataset A Strange Property of Keyword Queries on the Web 6 / 27

14 Dataset We used the AOL query log dataset 3. The collection consists of around 20 million web queries. 3 A Strange Property of Keyword Queries on the Web 6 / 27

15 Dataset We used the AOL query log dataset 3. The collection consists of around 20 million web queries. The collection is done over 650,000 users. 3 A Strange Property of Keyword Queries on the Web 6 / 27

16 Dataset We used the AOL query log dataset 3. The collection consists of around 20 million web queries. The collection is done over 650,000 users. The data is collected over a period of 3 months from 1st March, 2006 to 31st May, A Strange Property of Keyword Queries on the Web 6 / 27

17 Approach Data extraction from query logs. A Strange Property of Keyword Queries on the Web 7 / 27

18 Approach Data extraction from query logs. Remove scale factor from shape. A Strange Property of Keyword Queries on the Web 7 / 27

19 Approach Data extraction from query logs. Remove scale factor from shape. Encoding the temporal shape of the queries. A Strange Property of Keyword Queries on the Web 7 / 27

20 Approach Data extraction from query logs. Remove scale factor from shape. Encoding the temporal shape of the queries. Computing pair-wise similarities. A Strange Property of Keyword Queries on the Web 7 / 27

21 Encoding Temporal Shape of Queries Consider the quantum of change in query volumes from one day to next. A Strange Property of Keyword Queries on the Web 8 / 27

22 Encoding Temporal Shape of Queries Consider the quantum of change in query volumes from one day to next. The quantum change is projected onto a set of seven primitives A through G. A Strange Property of Keyword Queries on the Web 8 / 27

23 Encoding Temporal Shape of Queries Consider the quantum of change in query volumes from one day to next. The quantum change is projected onto a set of seven primitives A through G. The primitives are assigned by mapping the slope of the query graph in one time interval onto a radical region. A Strange Property of Keyword Queries on the Web 8 / 27

24 Least Biased Distribution The slope of the line depicting change is determined as: q = q t+1 q t t (1) A Strange Property of Keyword Queries on the Web 9 / 27

25 Least Biased Distribution The slope of the line depicting change is determined as: q = q t+1 q t t To minimize biases in the calibration, we need to set t to a value such that it maximizes the entropy of points. (1) A Strange Property of Keyword Queries on the Web 9 / 27

26 Least Biased Distribution The slope of the line depicting change is determined as: q = q t+1 q t t (1) To minimize biases in the calibration, we need to set t to a value such that it maximizes the entropy of points. The entropy in the distribution of the primitives is given as H( q) = q [A...G] p( q)log 2 (p( q)) (2) A Strange Property of Keyword Queries on the Web 9 / 27

27 Normalization Consider an example for queries yahoo, access. A Strange Property of Keyword Queries on the Web 10 / 27

28 Normalization Consider an example for queries yahoo, access. When their volumes are normalized A Strange Property of Keyword Queries on the Web 10 / 27

29 Normalization Consider an example for queries yahoo, access. When their volumes are normalized The data is normalized as follows: Val = X µ σ A Strange Property of Keyword Queries on the Web 10 / 27 (3)

30 Measure Similarity between Temporal Sequences We use Dynamic Time Warping (DTW) to compute the distance measure between the respective time series. A Strange Property of Keyword Queries on the Web 11 / 27

31 Measure Similarity between Temporal Sequences We use Dynamic Time Warping (DTW) to compute the distance measure between the respective time series. The algorithm finds the least weighted path in a matrix where the rows represent one temporal signature and the columns represent another temporal signature. A Strange Property of Keyword Queries on the Web 11 / 27

32 Query HeartBeat Properties We noticed that queries which cross a certain volume tend to follow the same temporal pattern. A Strange Property of Keyword Queries on the Web 12 / 27

33 Query HeartBeat Properties We noticed that queries which cross a certain volume tend to follow the same temporal pattern. The queries that tend to follow this attractor distribution are generic queries. Their volume is not influenced by any external events (say, an event associated with the query). A Strange Property of Keyword Queries on the Web 12 / 27

34 Query HeartBeat Properties We noticed that queries which cross a certain volume tend to follow the same temporal pattern. The queries that tend to follow this attractor distribution are generic queries. Their volume is not influenced by any external events (say, an event associated with the query). Similar behavior of queries was observed on Google Trends. A Strange Property of Keyword Queries on the Web 12 / 27

35 Attractor Distribution on Google Trends A Strange Property of Keyword Queries on the Web 13 / 27

36 Attractor Distribution on Google Trends 4 Queries: book, calculator, address, mobile. 4 A Strange Property of Keyword Queries on the Web 13 / 27

37 Attractor Distribution on Google Trends 4 Queries: book, calculator, address, mobile. Queries: book, california, women, hair, access. 4 A Strange Property of Keyword Queries on the Web 13 / 27

38 Attractor Distribution for AOL Dataset The query google is the most queried word on the AOL dataset. A Strange Property of Keyword Queries on the Web 14 / 27

39 Attractor Distribution for AOL Dataset The query google is the most queried word on the AOL dataset. Queries: google, yahoo, ebay, sale, lyric. A Strange Property of Keyword Queries on the Web 14 / 27

40 Attractor Distribution for AOL Dataset The query google is the most queried word on the AOL dataset. Queries: google, yahoo, ebay, sale, lyric. Query volume: , , , , A Strange Property of Keyword Queries on the Web 14 / 27

41 Attractor Distribution for AOL Dataset The query google is the most queried word on the AOL dataset. Queries: google, yahoo, ebay, sale, lyric. Query volume: , , , , DTW distances with respect to google: yahoo , ebay , sale , lyric A Strange Property of Keyword Queries on the Web 14 / 27

42 Normalized Results for AOL Dataset Queries: google, yahoo, ebay, sale, lyric. A Strange Property of Keyword Queries on the Web 15 / 27

43 Attractor Distribution for AOL Dataset Queries: google, book, car, picture. A Strange Property of Keyword Queries on the Web 16 / 27

44 Attractor Distribution for AOL Dataset Queries: google, book, car, picture. Query volume: , 93649, , A Strange Property of Keyword Queries on the Web 16 / 27

45 Attractor Distribution for AOL Dataset Queries: google, book, car, picture. Query volume: , 93649, , DTW distances with respect to google: book- 6, car , picture A Strange Property of Keyword Queries on the Web 16 / 27

46 Normalized Results for AOL Dataset Queries: google, book, car, picture. A Strange Property of Keyword Queries on the Web 17 / 27

47 Attractor Distribution for AOL Dataset Queries: google, house, school, mexico. A Strange Property of Keyword Queries on the Web 18 / 27

48 Attractor Distribution for AOL Dataset Queries: google, house, school, mexico. A Strange Property of Keyword Queries on the Web 18 / 27

49 Attractor Distribution for AOL Dataset Queries: google, house, school, mexico. Query volume: , , , DTW distances with respect to google: house , school , mexico A Strange Property of Keyword Queries on the Web 18 / 27

50 Normalized Results for AOL Dataset Queries: google, house, school, mexico. A Strange Property of Keyword Queries on the Web 19 / 27

51 Normalized Results for AOL Dataset We plotted the normalized graph for volume of all queries put together and google. A Strange Property of Keyword Queries on the Web 20 / 27

52 Normalized Results for AOL Dataset We plotted the normalized graph for volume of all queries put together and google. The aggregated heartbeat of all queries follows the query heartbeat even though their volumes vastly differ. A Strange Property of Keyword Queries on the Web 20 / 27

53 AOL Attractor Distribution We plotted the DTW distance of queries to google on a log-log graph of query volume Vs. queries in decreasing order of volume. A Strange Property of Keyword Queries on the Web 21 / 27

54 AOL Attractor Distribution The variation of the DTW distance Vs. the log of query volume. A Strange Property of Keyword Queries on the Web 22 / 27

55 AOL Attractor Distribution The variation of the DTW distance Vs. the log of query volume. The correlation between DTW distance and log of query volume was indicating a high negative correlation. A Strange Property of Keyword Queries on the Web 22 / 27

56 Idea of major events distorting the attractor distribution in queries Consider an example of queries google and myspace on AOL dataset. Query volume: , DTW distance: A Strange Property of Keyword Queries on the Web 23 / 27

57 Idea of major events distorting the attractor distribution in queries On March 31st 2006, the market share of visits to Myspace Video increased by 1242%. A Strange Property of Keyword Queries on the Web 24 / 27

58 Idea of major events distorting the attractor distribution in queries On March 31st 2006, the market share of visits to Myspace Video increased by 1242%. On May 2nd 2006, Myspace denied access to a user who had created a profile in the name of Barack Obama, a Chicago Senator. A Strange Property of Keyword Queries on the Web 24 / 27

59 Idea of major events distorting the attractor distribution in queries On March 31st 2006, the market share of visits to Myspace Video increased by 1242%. On May 2nd 2006, Myspace denied access to a user who had created a profile in the name of Barack Obama, a Chicago Senator. On May 22nd 2006, two teenagers were charged with illegal computer access into Myspace and attempted extortion worth $150,000. A Strange Property of Keyword Queries on the Web 24 / 27

60 Characteristics of attractor distribution The primitives can be divided into three broad categories: Rise, Fall and Constant trend in Query volume. A Strange Property of Keyword Queries on the Web 25 / 27

61 Characteristics of attractor distribution The primitives can be divided into three broad categories: Rise, Fall and Constant trend in Query volume. We calculated the transition probabilities of queries in these three categories: P(fall rise) 0.68 Rise P(rise rise) 0.27 P(constant rise) 0.05 P(rise fall) 0.60 Fall P(fall fall) 0.28 P(constant fall) 0.12 A Strange Property of Keyword Queries on the Web 25 / 27

62 Conclusion and Future Work Conclusion We showed that disparate keyword queries on the Web exhibit strange central limit properties. A Strange Property of Keyword Queries on the Web 26 / 27

63 Conclusion and Future Work Conclusion We showed that disparate keyword queries on the Web exhibit strange central limit properties. The heartbeat seems to be characteristic of generic search terms that have large enough volumes and are not affected by periodicity or external events. A Strange Property of Keyword Queries on the Web 26 / 27

64 Conclusion and Future Work Conclusion We showed that disparate keyword queries on the Web exhibit strange central limit properties. The heartbeat seems to be characteristic of generic search terms that have large enough volumes and are not affected by periodicity or external events. The volume of the query determines the closeness of the query distribution with the attractor distribution. A Strange Property of Keyword Queries on the Web 26 / 27

65 Conclusion and Future Work Conclusion We showed that disparate keyword queries on the Web exhibit strange central limit properties. The heartbeat seems to be characteristic of generic search terms that have large enough volumes and are not affected by periodicity or external events. The volume of the query determines the closeness of the query distribution with the attractor distribution. Future Work A Strange Property of Keyword Queries on the Web 26 / 27

66 Conclusion and Future Work Conclusion We showed that disparate keyword queries on the Web exhibit strange central limit properties. The heartbeat seems to be characteristic of generic search terms that have large enough volumes and are not affected by periodicity or external events. The volume of the query determines the closeness of the query distribution with the attractor distribution. Future Work It would be interesting to discern interesting characteristics of the attractor distribution. A Strange Property of Keyword Queries on the Web 26 / 27

67 Conclusion and Future Work Conclusion We showed that disparate keyword queries on the Web exhibit strange central limit properties. The heartbeat seems to be characteristic of generic search terms that have large enough volumes and are not affected by periodicity or external events. The volume of the query determines the closeness of the query distribution with the attractor distribution. Future Work It would be interesting to discern interesting characteristics of the attractor distribution. Why this attractor distribution? A Strange Property of Keyword Queries on the Web 26 / 27

68 Conclusion and Future Work Conclusion We showed that disparate keyword queries on the Web exhibit strange central limit properties. The heartbeat seems to be characteristic of generic search terms that have large enough volumes and are not affected by periodicity or external events. The volume of the query determines the closeness of the query distribution with the attractor distribution. Future Work It would be interesting to discern interesting characteristics of the attractor distribution. Why this attractor distribution? What is the generative model that gives rise to this distribution? A Strange Property of Keyword Queries on the Web 26 / 27

69 Thank you A Strange Property of Keyword Queries on the Web 27 / 27