An Introduction to Search Engines and Web Navigation

Transcription

1 An Introduction to Search Engines and Web Navigation MARK LEVENE ADDISON-WESLEY Ал imprint of Pearson Education Harlow, England London New York Boston San Francisco Toronto Sydney Tokyo Singapore Hong Kong Seoul Taipei New Delhi Cape Town Madrid Mexico City Amsterdam Munich Paris Milan

2 Figures Tables Preface Publisher's Acknowledgements xv xix xxi xxiv 1 Introduction Brief Summary of Chapters Brief History of Hypertext and the Web Brief History of Search Engines 6 2 The Web and the Problem of Search g 2.1 Some Statistics Web size statistics Web usage statistics Tabular Data versus Web Data The Structure of the Web Bow-tie structure of the web Small-world structure of the web Information Seeking on the Web Direct navigation Navigation within a directory Navigation using a search engine Problems with web information seeking Informational, Navigational and Transactional Queries Comparing Web Search to Traditional Information Retrieval Recall and precision Local Site Search versus Global Web Search The Difference between Search and Navigation 29 Chapter Summary 30 Exercises 31 3 The Problem of Web Navigation зз 3.1 Getting Lost in Hyperspace and the Navigation Problem How Can the Machine Assist in User Search and Navigation? The potential use of machine learning algorithms The naive Bayes classifier for categorizing web pages 38

3 3.3 Trails Should Be First-Class Objects Enter Markov Chains and Two Interpretations of Their Probabilities Markov chains and the Markov property Markov chains and the probabilities of following links Markov chains and the relevance of links Conflict between Web Site Owner and Visitor Conflict between Semantics of Web Site and the Business Model 49 Chapter Summary 51 Exercises 52 Searching the Web The Mechanics of a Typical Search Search Engines as Information Gatekeepers of the Web Search Engine Wars: Is the Dust Settling? Competitor number 1: Google Competitor number 2: Yahoo! Competitor number 3: MSN Search Other competitors Statistics from Studies of Search Engine Query Logs Search engine query logs Search engine query syntax The most popular search keywords Architecture of a Search Engine The search index The query engine The search interface Crawling the Web Crawling algorithms Refreshing web pages The Robots Exclusion Protocol Spider traps What Does It Take to Deliver a Global Search Service? 73 Chapter Summary 74 Exercises 75 How Does a Search Engine Work? Content Relevance Processing web pages Interpreting the query Term frequency 81

4 5.1.4 IDF Computing keyword TF-IDF values Caching queries Phrase matching Synonyms Link text URL analysis Date last updated HTML structure weighting Spell checking Home page detection Link-Based Metrics Referential and informational links Combining link analysis with content relevance Are links the currency of the web? PageRank explained Online computation of PageRank Hyperlink-Induced Topic Search Counting incoming links Weblogs'influence on PageRank Link spam Citation analysis The wide-ranging interest in PageRank Popularity-Based Metrics Direct Hit's popularity metric Document space modification Using query log data to improve search Evaluating Search Engines Search engine awards Evaluation metrics Performance measures Test collections Inferring ranking algorithms 110 Chapter Summary 111 Exercises 113 Different Types of Search Engines Directories and Categorization of Web Content Paid Inclusion and Paid Placement Paid inclusion Paid placement and pay-per-click 118

5 6.3 Metasearch Fusion algorithms Operational metasearch engines Clustering search results Personalization Personalized results tool Privacy and scalability Relevance feedback Personalized PageRank Outride's personalized search Question Answering (Q&A) on the Web Natural language annotations Open domain Q&A Semantic headers Image Search Text-based image search Content-based image search Image search for finding location-based information Special-Purpose Search Engines 147 Chapter Summary 151 Exercises Navigating the Web Frustration in Web Browsing and Navigation HTML and web site design Hyperlinks and surfing Web site design and usability Navigation Tools The basic browser tools The back and forward buttons Search engine toolbars The bookmarks tool The history list Identifying web pages Breadcrumb navigation Hypertext orientation tools HyperCard programming environment Navigational Metrics The potential gain Structural analysis of a web site Measuring the usability of web sites 173

6 7.4 Web Data Mining Three perspectives on data mining Measuring the success of a web site Web analytics E-metrics Weblog file analyzers Identifying the surfer Sessionizing Supplementary analyses Markov chain model of web site navigation Applications of web usage mining The Best Trail Algorithm Effective view navigation Web usage mining for personalization Developing a trail engine Visualization that Aids Navigation Visualization of navigation patterns Overview diagrams Web site maps Fisheye views Visualizing trails within a web site Visual search engines Mapping cyberspace Navigation in Virtual and Physical Spaces Real-world web usage mining The museum experience recorder Navigating in the real world 206 Chapter Summary 206 Exercises The Mobile Web The Paradigm of Mobile Computing Wireless Markup Language The i-mode service Mobile Web Services M-commerce Delivery of personalized news Delivery of learning resources Mobile Device Interfaces Mobile web browsers Information seeking on mobile devices 222

7 8.3.3 Text entry on mobile devices Presenting information on a mobile device The Navigation Problem in Mobile Portals Click-distance Adaptive mobile portals Adaptive web navigation Mobile Search Mobile search interfaces Search engine support for mobile devices Focused mobile search Laid-back mobile search Mobile query log analysis Personalization of mobile search Location-aware mobile search 242 Chapter Summary 243 Exercises Social Networks What Is a Social Network? Milgram's small-world experiment Collaboration graphs Instant messaging social network The social web Social network startups Social Network Analysis Social network terminology The strength of weak ties Centrality Web communities Pajek - large-network analysis software Peer-to-Peer Networks Centralized P2P networks Decentralized P2P networks Hybrid P2P networks BitTorrent file distribution JXTA P2P search Collaborative Filtering Amazon.com Collaborative filtering explained User-based collaborative filtering

8 9.4.4 Item-based collaborative filtering Model-based collaborative filtering Content-based recommendation systems Evaluation of collaborative filtering systems Scalability of collaborative filtering systems A case study of Amazon.co.uk Some other collaborative filtering systems Weblogs (Blogs) Blogrolling Blogspace Blogs for testing machine learning algorithms Spreading ideas via blogs Power-Law Distributions in the Web Detecting power-law distributions Power-law distributions in the internet A law of surfing The evolution of the web via preferential attachment The evolution of the web as a multiplicative process The evolution of the web via HOT Small-world networks The robustness and vulnerability of a scale-free network Searching in Social Networks Social navigation Social search engines Navigation within social networks Navigation within small-world networks Testing navigation strategies in social networks 305 Chapter Summary 305 Exercises The Future of Web Search and Navigation 311 Notes 315 Bibliography 327 Index 351