Searching 1 Outline Goals and Objectives Topic Headlines Introduction Directories Open Directory Project Search Engines Metasearch Engines Search techniques Intelligent Agents Invisible Web Summary 2 1
Goals and Objectives Goals Understand searching find relevant information fast know what / how / where to search Objectives 3 Objectives Subject Directories Open Directory project Search and metasearch engines Search techniques Intelligent agents The visible web The invisible web Search techniques for the invisible web 4 2
Topic Headlines Introduction directory / search engines Directories subject tree manually / use engine Open Directory Project 5 Search Engines ranking search results How do search engines do their job Metasearch Engines multiple search engines at once Search internet more effectively Search Techniques Intelligent Agents Invisible Web 6 3
Introduction what / how / where to search Search results web pages main tools Directory subject guide organized by major topics and subtopics Search Engines crawler / bot Each database Directory compiled by humans Engine s generated automatically 7 Directories human-powered search engines organize information in hierarchical tree by subjects general specific two ways to search directory Manual browse subjects hierarchically Search engine enter search terms Example Yahoo Directory 8 4
Directories 9 Open Directory Project Search results ranked human editors to rank web pages As number of pages for topic increase more time-consuming and cost-bearing to rank Open Directory Project ranking system to users Users become editors & evaluate web sites in area of expertise lot more content http://dmoz.org 10 5
Search Engines Examples Google google.com Yahoo yahoo.com Ask : ask.com MSN Search search.msn.com AOL Search search.aol.com Answers answers.com Tips on using search engine and much more http://www.searchenginewatch.com 11 Search Engines based three important parts Spider / crawler / robot : follow links in databases Indexer: identify web page content + store in database Searcher: sift through engine s index to find matches query + rank matches Relevance ranking algorithm crucial Different engines different results 12 6
Metasearch Engines multi-engine search skip engine that is down no own database Examples : http://www.dogpile.com http://www.metacrawler.com http://www.profusion.com 13 Search Techniques Searching guidelines: Change query to improve results Search string = key words not exact phrase Advanced Searching techniques: Words and exact phrase Boolean search AND, OR, NOT Title search web page title Site search limit search to particular site URL search, Link search Wildcard (fuzzy) search * Features search special features of engines 14 7
Search Techniques 15 Intelligent Agents Three retrieval paradigms: Statistical correlations of word counts in documents Semantic natural language processing and artificial intelligence Contextual use thesaurus and encoded relationships intelligent agent: gather information / perform tasks based on human input E.g., Spider part of search engine 16 8
Intelligent Agents Advantages: More intelligent search Create and update own knowledge database Perform tasks quicker Communicate & co-operate w/ other agents Customizable Continuously scan internet for information Free user from mundane tasks 17 Invisible Web hidden web content Database contents Dynamically generated pages estimated to be larger than visible web search invisible web: Directories (Invisible Web Catalogue) Databases 18 9
Summary Search engines Directories Open Directory Project most popular search engines Metasearch engines Intelligent agents Invisible web 19 10