A Study of Two Search Engines. A search engine at work

Transcription

1 A Study of Two Search Engines. A search engine at work Sounds farfetched, but these search engine people put spiders to work. These crawl around your website to determine the content. Well, OK, they may not quite look like this, but everyone involved in web design should know about them! Search engines are tools designed to help people find information on the Internet. Many Internet users search with just a few engines, although over 3600 different ones exist (Search Engine Guide, 2001). Most search engines use a combination of methods to identify potentially relevant material: a crawler component (a computer generated index of Web sites) as well as a directory component (a manually edited index of Web sites) (Search Engine Watch, 2001). After ident ifying material, search engines employ different methods to rank the sites they have found. Examples of ranking methods are discussed within: (The Search Engine Guide.) For part one of this assignment I shall concentrate on the following two search engines: and with emphasis on: Appearance and efficiency. Functionality and ease of use.

2 Introducing Search engine Yahoo URL Address: www. Yahoo.com Type of engine: Human engine & portal website figure 1-1 figure 1-2 The site opens to an ever changing, busy, web portal and search engine. Content is an obvious principal on this site. and Messenger services are available to the registered. A frenzy of data covers a whole range of topics, from Autos & shopping, to World stock markets & education or simply just surfing, it is all here. figure 1-3 Online Games populate the portal with the opportunity of tournament or league gaming. Sign up to a Yahoo account and have the added benefit of being able to customise aspects of the site.

3 Introducing Search engine Google URL Address: www. Google.com Type of engine: Crawler Speed & Simplicity are key issues on this site. The site opens to a clear concise search box in a matter of seconds. No fuss, no adverts, simply a let s search approach. figure 2-1 The Two search button facility could be conceived as a gimmick, only it works, Google Search performs a usual search engine motion and results a typically recognizable table of data. Alternatively the I m Feeling Lucky search option throws convention to favour the more simplistic approach of opening the topmost search result, based on the search criteria you had entered. (figure 2-1) figure 2-2 It also has options for searching, IMAGES, GROUPS, DIRECTORY or traditionally the WEB. Finally they have an advanced search page where Boolean operators are graphically represented. Additionally you can tweak the result language, file format date, occurrences, domain, and safe search filtering. A new beta feature of searching headline news from the last seven days is also a feature. (figure 2-2)

4 Search Engine Advantages: Speed and Simplicity. One-stop shopping. Boolean search Operations. (AND, OR, NOT, NEAR, FOLLOWED BY.) No Adverts. Search option: I m Feeling Lucky. Search topic shortcuts to: Web Groups Images Directory Searches textual content of a site not just the meta-tag value. Crawler engine, with extensive up to date content. Layout more suitable for visually impaired surfers. (Simplicity of the portal page) Should be easier for a webmaster to get a ranking site. Free site Submission. (More sites will be submitted to a free engine.) For a small download Google can customize your browser with a Google bar. (A Search engine as part of your browsers standard buttons.). Popularity. Google is popular. Google does not stick with one algorithm like other search engines. Instead, Google switches between a few different algorithms from month to month. (The web Guide) Boolean search Operations. (AND, OR, NOT, NEAR, FOLLOWED BY.) The portal offers plenty to keep you occupied and coming back for more. (shopping, games.) Yahoo mail and messenger are long established modes of communication. Clear and concise result formatting, with good search result content. (Less reading is good for the visually impaired.) Human engine, vetted / filtered, content. A viable resource site for accurate data retrieval. Ranked #1 by Media Matrix in June Charged submissions. (Though, only for commercial or adult content, reduces some business and porn sites.)

5 Search engine Disadvantages: The home page is un-eventful and dull, there is very little to see. It doesn t offer enough content to survive as a homepage. Defaults to international searching. Produces too many sites on searches. (vague / ambiguous searches can result in thousands of listings in the report.) Free submissions.( This facilitates mass submissions as there is little criteria for site rejection. Systems can be devised to submit sites to promote higher ranking.) With a changing algorithm, sites that were top last month may be this month s history. Multimedia content doesn t come free. Advertisements fund your way and they are never far from view. (A personal dislike) At times, with all that s going on, it s difficult remembering you re on a search engine. Effectiveness of eye-candy can soon dwindle (Media content needs constant work to be of interest.) Reading all the data on the portal home page can become tiresome and not yielding to visually impaired surfers. The vetting can be counter productive, due to the human nature involved in the site submissions process Sites may be vetted out as being untrustworthy, inaccurate or unsuitable, though this is documentation from their site. We want to be as inclusive as possible, but sometimes we have to make tough decisions about what to include. We do not include every site that is submitted to us, and we reserve the right to choose which sites we include. This engine recommends taking advantage of the Sponsor Matches program to gain further listings.

6 Report Format: The formats to which both reports are drawn differ greatly. Google represents the ranked site with data taken from the page title tag, <title> data from here </title>, and from the text of the, <body> data from here </Body> of a crawled site, and displays the format (Figure 4-1), Title, text, the resulted URL, file size, a cached option and finally a link to similar sites. Figure 4-1. This examples a search result entry Preston & College. (Google) The cached link is a powerful tool. Example: This is Goggles cache of Goggles cache is the snapshot that we took of the page as we crawled the web. The cached link goes on to highlight the matched searched values, again matched on page text content not just html meta-tag values. Yahoo s reports are far more concise, offering far less information than that of Google. Title, a link to the sorted site, the URL and the option to further your search into similar sites matched on geographical values is the results page format, no fancy toys to play with here. (Figure 4-2) Figure 4-2 Representation of search report entry Preston & College. (Yahoo) Google and Yahoo both offered numerous matches based on the search criteria entered, in the example Yahoo matched 7 sites, Google claimed to have matched About 286,000 and boasts a search time of 0.07 seconds obviously I have no viable method for verifying these claims.

7 Findings: The Google approach of being a crawler type engine reflects a corporate mission: To provide efficiency and speed to the service of fast data retrieval. This goal they achieve by use of many strategies, one of which is use of its crawler. The Google crawler algorithm takes many things into account when ranking web pages. Some of the variables for ranking in Google are: (The Search Engine Guide) Page Title Link text PageRank Heading Tags ALT Tags Domain names Filenames Directory names Link popularity Keyword density Page Relevance. Proximity of search criteria. Text Matching. Ranking by the location and frequency of key words within a Web site. Popularity. Ranking by the number of times a site is linked to by other Web sites. Ranking by payment from Web sites. (Sponsors Links) Ranking by human editing. (directories only) Text Matching Considers Data Information From: Meta-Tags: A meta-tag is a head element of a html document. This tag holds fundamental information about the current page, such as the character encoding, author, copyright, and keywords. ( These meta-tag can also be used to give information to the server, such as the expiration date, refresh interval, and PICS rating for the page. (PICS, the Platform for Internet Content Selection, provide a method for assigning ratings, such as movie ratings, to Web pages.) Keyword: Many search-engines read the contents of the Keywords meta-tag and use the information to index your pages in their databases. Some search engines limit the number of keywords or characters they index, or ignore all keywords beyond a limit. Description: Search-engine crawlers read the contents of the Description meta-tag. Crawlers use the information to index pages in their databases, and may also display the information on the search results page (instead of displaying the first few lines of your document). Some search engines limit the number of characters they index. Author: This is another less familiar or utilized meta-tag. Search engines like HotBot use this meta-tag when searching. Comment: Again and, like the author tag, the comments tag is considered when being searched by the likes of HotBot and Inktomi. Title: The title meta-tag appears in the title bar of the browser when you view the page in most browsers. Crawlers read this text not only for its title but also to match search criteria. Hidden Field: Comments meta-tag holds text that a webmaster enters in the HTML code to explain the code or provide other information. The comment appears only in the view source code view and will not display in web page or the browser. (This can be read by a crawler.) Text Content: A lot of the textual content within a given URL is searched for matches of your search criteria. This is the actual data that the site displays to the viewer, usually the content, the purpose to why they bothered to view the site in the first place. Image & Frame, Names: Not exactly a meta-tag, but read all the same. Crawlers may reference the names given to images, frames, frame sets and alt - tags. This is increasingly more important to viewers who have visual disabilities. Pc-resident programmes that assist a visually impaired surfer will also read this textual content The Body: The main body of your web page may also be scanned.

8 Comparisons: With the vast recourse of the entire internet available to its crawler algorithm Google is better suited for a more experienced surfer, who can make full use of the vast number of sites, Google finds as it reports a search. Sites are crawled from anywhere in the world and there is no vetting for accuracy or quality of content. Knowing this a good webmaster should be able to facilitate a high ranking site, regardless of its quality or content. Yahoo uses a completely different approach. Yahoo is a directory, this means there is no such crawlerrobot to do any reconnaissance, and instead, the sites are searched and vetted by humans (after first being searched for on Google. Apparently!). People employed to review sites and assess their data content. When a site is uploaded to the internet the webmaster needs to submit the site to search engines. At Yahoo, this process of site submission gives the directory (the humans), an inclination there is a new site for them to review. The reviewing process is manual. It is often suggested the ranking is biased; the site is searched twice, once by the search-engine, as you yourself use it to perform a search on the Yahoo directory, but moreover it has already been searched, assessed, ranked and vetted by people. This is at the directory review process, when it is entered into the directory. It is suggested personal taste may vary greatly between yourself (the surfer) and the vetting agent. Yahoo state, We do not include every site that is submitted to us. A more positive opinion on this human involvement Vetting stage, is the results from a directory type search engine vetted down to a far lesser results page, whereas a crawler can report every instance it can find. This vetting can prove beneficial; remember back to when I said Google claimed to have matched About 286,000 sites, there is no way I would ever bother to view every site. Yahoo in contrast reported 7 sites, certainly a number I would consider viable to view all. Users would be more inclined to review all sites listed in the search result table if there was only a few and not thousands. This could as be conceived as beneficial to a lesser experienced surfer, a novice maybe or where only a general report is required and not an in-depth study requiring the cross-reference of many different opinions. Finally it would be my opinion that Yahoo could be difficult to a visually disabled surfer, on the basis of, all the content on their portal. This information would have to read-out, audibly (Use of third party software), and the extent of the content could loose the surfer s attention, in contrast their concise reports produced would be an asset to visually impaired viewers as the report data is concise. This I interpret as anomalies in the company, mission and strategies, perhaps even room for improvement.

9 Searching: Yahoo and Google are two of today s more popular web search engines. As far as a simple search is concerned there is no obvious advantage in using either; however that s as far as it goes. These two engines couldn t possibly be any more different. Google is a crawler driven searchengine whereas Yahoo is a human-engine directory, moreover Yahoo is swamped in multimedia on the web portal site in which it resides (figure 1-1). Google on the other hand is on a page of its own (figure 2-1). Both sites offer all the benefits of modern day searching, Boolean operators and advanced search functionality. (AND, OR, NOT, NEAR, FOLLOWED BY, quotation marks, and an asterisk as for wildcard matching) (figures 1-2 & 2-2) The Search: Preston & College When I performed the search Preston & College (note: the quote marks were not in the search) Google places the college site first. Also Yahoo, below clear indicated Sponsor Matches ranks Preston College in pole-position. Interestingly this instance of the search, Yahoo goes on to list other local colleges Myerscough, Runshaw, Burnley and Preston Uni (figure 1-3). Google reports several of Preston Colleges many pages, unfortunately its reports are of Preston College Carolina, although in defence of this, I had not requested a UK only search. Google also reported this line of text "&" is a very common word and was not included in your search. At this point I opted to continue with this interesting, but seemingly, flawed Boolean search. Yahoo may well have found more local sites but in contrast, Google did rank closer to the searched criteria and continued to rank sites with topics concerned with both Preston and College in their content, while Yahoo was satisfied to match either criteria. The Search: Preston AND College A corrected search criteria Preston AND College, AND being the Boolean operator, instructing the search engines to report only sites which conform to both search key words. My findings confirm both engines result exactly the same with either & or AND operator. Google also reported yet another message: The "AND" operator is unnecessary -- we include all search terms by default. Repeat The Search: one week later: One week later and the same search criteria, Google has changed by lowered the raking of Preston Colleges official site to now rank in eighth position. Other sites that were listed have also changed rank. Sites that were not listed last week have jumped into the top ten this week, and so on.

10 Accuracy: Is The Information Reported Generally Accurate? Most sites provide content that is correct and accurate. Indeed, there appeared to be a much greater chance that; the more important topics covered will be addressed most accurately. Lesser important topics were not. However, I did find more than one instance of conflicting information. Consumers who carefully read most or all of the information on a site might well be confused when they encounter such conflicting information. From this I deemed Yahoo to be the most consistent in offering accurate information. This would be more a general pattern than an accepted fact for all searches. Information on the Internet Is Commercialized A substantial proportion of the information that Internet users are likely to find on Web sites is promotional i.e., it sells products or services but is not clearly labeled as an advertisement. The commercial nature of this material is not always clear, so it may be difficult for consumers to weigh whether or not commercial interests potentially bias the information they encounter. Dirtworld.com may well be a commercial site. Conclusion There is not a lot more than personal taste to sway your preference of either these two professional web search engines. Both engine(s) are more than adequately poised to relay any data requests. However when you assimilate all of the information contained within this report you can easily choose which site meets your partiality, and for what task. SEARCH: What is a search engine

11 Bibliography: Data within this report has been collated for the following sources: 1) The Search Engine Guide (Search Engine Guide, 2001). (Search Engine Watch, 2001). 2) Leech Design, Graphic Design and Illustration Grange over Sands, Cumbria, UK, ) Google 4) Yahoo 5) 6) Inktomi 7) WAB: World Wide Web Access for Blind and Visually Impaired Computer Users 8) Personal Web Sites of the Blind and Visually Impaired 9) Web Site Design for Visually-Impaired Users 10) Media Matrix 11) Sites Studied