Ranking Search Results by Web Quality Dimensions

Transcription

1 Rankng Search Results by Web Qualty Dmensons Joshua C. C. Pun Department of Computer Scence HKUST Clear Water Bay, Kowloon Hong Kong Frederck H. Lochovsky Department of Computer Scence HKUST Clear Water Bay, Kowloon Hong Kong ABSTRACT Currently, search engnes rank search results usng manly lnkbased metrcs. Whle usually most of the search results are relevant to a user's query, users often are stll not totally satsfed wth them. Usng a proposed framework of web data qualty, t s found that current search engnes usually only consder one facet of web data qualty. In ths paper, one of the newly dentfed qualty dmensons, approprateness, whch s based on the lngustc complexty of a web page, s studed. Expermental results show that t helps a search engne rank web pages accordng to whether they are (lngustcally) approprate for a user. 1. INTRODUCTION Currently, prmarly lnk-based metrcs are used by most search engnes to rank the search results. The results returned are usually relevant to the query. However, relevance alone s sometmes not enough to satsfy a user s need. For example, f a user submts the query "computer games" to a search engne, most, f not all, of the pages n the returned results would be relevant to the query. However, the qualty of the search results may stll not meet the user s expectatons (for example, whether the results are comprehensble to the user). In ths paper, we examne a mssng lnk between the relevance and the qualty of the search results to the users. Relevance does not necessarly mply good qualty. It s possble to have relevant search results but poor qualty. Current search engnes (e.g. Google) only take one facet (.e., hyperlnk-based metrcs) of web data qualty dmensons nto consderaton. Search engnes seldom employ the remanng dmensons. We propose a framework for web data qualty dmensons that ncorporates the essental results from other studes on data qualty and examnes the applcablty of each data qualty dmenson to web data. In addton, the framework also ncorporates some specal dmensons, whch are unque to web data. For example, one possble unmatched user expectaton of "good qualty" search results arses from the ssue of whether the results returned by a search engne are approprate for the user. In vew of ths, a new qualty dmenson "approprateness" s added to the framework. In ths paper, we examne how ths new qualty dmenson can be used to mprove the rankng of web search results. The approprateness of a web page can depend on many factors. It may be a matter of the expected level of detal or of the use of words n the page. For example, a layman and a specalst on a gven topc often have dfferent requrements for ther Copyrght s held by the author/owner(s). WWW 2004, May 17 22, 2004, New York, New York, USA. ACM X/04/0005. nformaton need. A layman may be very satsfed wth a general artcle on a newspaper web page. However, f the reader of the page s a specalst on the topc, he/she mght prefer to read a web page from a professonal organzaton or a paper from a famous and prestgous journal. To determne the approprateness of a web page, the ntended audence of the web page needs to be known. To determne the ntended audence, we analyse a web page s lngustc complexty and classfy t as one of three man types: scholarly, news/general nterest or popular. For each type of web page, there are some language-based attrbutes that can be used to characterze t. Our approach s to measure these attrbutes automatcally and to estmate the lkelhood that a web page s for a partcular ntended audence type (.e., s one of the three types of web pages). Then, when users submt ther queres to the search engne wth ther preferred type of page, the search engne can provde them wth both relevant and approprate pages. Ths rest of ths paper s organzed as follows. The next secton brefly descrbes related work on data qualty. Secton 3 presents a framework of web data qualty and some new dmensons of web data qualty. Secton 4 explores one of these new dmensons, approprateness, n further detal by dscussng a methodology for measurng t n terms of three new web metrcs. Secton 5 dscusses the ssues nvolved n the mplementaton of the three new web metrcs related to the approprateness dmenson. In Secton 6 we present our expermental results on the effectveness of the web metrcs n fndng approprate web pages for users. Fnally, Secton 7 concludes the paper. 2. OVERVIEW AND RELATED WORK To devse a framework for web data qualty, prevous studes on the dmensonalty of data qualty can be examned, as web data s also a knd of data. Fox et al. [6] lay a foundaton for the study of data qualty. They dscuss fve dfferent approaches to defne data and propose data qualty dmensons where the most mportant dmensons are: accuracy, completeness, consstency and currentness. Yang et al [15] contnued the Fox et al. study and classfed data qualty wth dfferent dmensons: accessblty, nterpretable, useful and belevable. Each of these dmensons can be further classfed. For example, accessblty gves avalable. Interpretable gves syntax and semantcs. Usefulness gves relevant and (current and non-volatle) tmely. Lastly, belevable gves complete, consstent, credble and accurate. Strong et al. [12] defne hgh-qualty data as data that s ft for use by data consumers. Ths means that usefulness and usablty are mportant aspects of qualty. Usng ths defnton, the characterstcs of hgh qualty data consst of four data qualty (DQ) categores: ntrnsc, accessblty, contextual and representatonal. Each category has dfferent data qualty dmensons. For example, Intrnsc DQ has the dmensons accuracy, objectvty, belevablty and reputaton. Accessblty

2 web data qualty data qualty accessblty nterpretablty usefulness belevable cohesveness vsual appearance mnmalty avalable syntax relevant completeness popularty approprateness navgaton access securty semantcs tmelness consstency credble currency accuracy non-volatle objectvty Fgure 1. Framework of web data qualty. DQ has accessblty and access securty dmensons. Contextual DQ has dmensons relevance, value added, tmelness, completeness and amount of data. Fnally, Representatonal DQ has dmensons nterpretablty, ease of understandng, concse representaton and consstent representaton. All these gve a broader conceptualzaton of DQ than the conventonal ntrnsc vew, whch only focuses on ntrnsc aspects of DQ and fals to address the broader DQ concerns of data consumers. Not all dmensons of data qualty are necessarly applcable to or meanngful for web data. Consder the dmenson of avalablty. For web data, t can be nterpreted as whether web pages can be accessed free of charge (.e., publcly avalable). However, some pages are restrcted to regstered users only (e.g., Intranet pages whch requre a logn by ther users). Web accessblty, on the other hand, can refer to the unversalty of free access of web pages by everyone regardless of ther (ds)ablty. Interpretablty relates to whether web pages are easy to understand. By usng more tables, pont form, headngs and properly emphaszed text and color, a web page may mprove ts nterpretablty. Usefulness of web data relates to the tradtonal nformaton retreval area where a determnaton s made of how relevant search results are to the user query. For web data, belevable (and credble) relate to the reputaton and authorty of web pages. They are commonly used n search engnes followng the ntroducton of the lnk-based metrcs PageRank [3] and Hubs/Authortes [9]. These metrcs have also been nterpreted as a measure of page popularty [4] and page qualty [1, 2]. The Appendx summarzes the varous data qualty dmensons prevously proposed. 3. WEB DATA QUALITY 3.1 Framework of Web Data Qualty Wth reference to the gudelnes recommended by lbrary scence researchers [13] and nformaton systems [16], as well as the understandng of the ntrnsc dfference between tradtonal data and web data, we have dentfed sx addtonal qualty dmensons for web data, namely approprateness, popularty, cohesveness, mnmalty, navgaton and vsual appearance. Together wth the dmensonaltes of data qualty from prevous work, these form a framework for web data qualty as shown n Fgure Approprateness Approprateness s a measure of how well the content of a page matches the ablty of the user to understand t 1. Ths, n turn, may depend on the educatonal background of the user. Therefore, to fnd content that s hghly suted to the user, a good understandng of the user s needed. For example, whch readng level s approprate for the user? Whch content s approprate for the age or knowledge level of the user? The ratonale for ths dmenson s that when authors wrte somethng n whatever format (e.g., books, artcles or web pages), they normally have ts ntended audence n mnd. The ntended audence reflects the purpose of ther wrtng, for example, nformaton dssemnaton, educaton and tranng, commerce and advertsng, or entertanment and communcatons to name a few. If the type of reader s a perfect match wth ther ntended audence type, then readers usually enjoy ther wrtng; else, the msmatch may result n an unhappy readng experence. 3.3 Popularty The popularty dmenson consders how popular a web page s to all users n the Internet. It can be nterpreted as the number of hts or vsts per page. Ths fgure however s only avalable to the page owner tself. It could be calculated by examnng the log fle of a web ste, but ths s an unrelable ndcator of page popularty. In [7], a refned metrc s proposed whch takes nto account structural nformaton. Another possble way to measure ths dmenson s to measure t ndrectly by countng how many other web pages have cted ths page. 3.4 Mnmalty The mnmalty dmenson s defned as the proporton of useful nformaton that s contaned n a web page. A web page can contan a lot of text and mages. Often, not all of ths nformaton s useful to users. For example, navgaton hyperlnks that appear on every page are not useful nformaton from an nformaton content perspectve. As much as possble, everythng on a web page should be useful (.e., we want to mnmze the 1 Approprateness can be vewed from two perspectves: queres and users. Query approprateness (usually called relevance) consders how well the retreved web pages match wth the query. Approprateness, on the other hand, as defned here, consders how well the retreved web pages match wth the user.

3 Table 1. Relatonshp of measurable web metrcs to the approprateness dmenson. Approprateness Metrcs Alternate Metrcs News /General Scholarly [0 numerc] [0 1] Interest Popular Colour count Useful mages count Fog ndex Words count Hard words count Rato of hard words Font face count Font sze count Short forms count Rato of short forms Numerc words count Rato of numerc words Pont forms count Rato of pont forms used readng of unnecessary and junk nformaton). The value of ths dmenson for a page wll, therefore, gve an ndcaton of how much "real" nformaton content there s n the page. If there are two web pages that have smlar real nformaton content, then one would probably prefer to read the web page that s shorter, snce t wll take less tme to dgest the materal. 3.5 Cohesveness The cohesveness dmenson gauges how closely the concepts are related to each other n a web page. It consders all concepts that are contaned n each web document. Each concept s descrbed by a set of features. It s a self-contaned concept and does not relate to other factors (e.g., the query). As such, t s possble to have a hghly cohesve page returned that s totally unrelated to the query! Therefore, what we want s a page that s hghly cohesve as well as relevant to the query. In software engneerng, a good programmng practce s to wrte a cohesve software module. Usng the same ratonale, when wrtng a document or a web page, a good wrtng practce s to wrte a cohesve page. If the concepts n a web page are hghly dversfed and unrelated, the cohesveness of the page s weak. If not, the cohesveness s consdered to be strong. To be of good qualty, a web page should be cohesve. If two web pages are equally relevant to a query, a user would defntely prefer to read a more cohesve web page snce t would be more focused on topc. 3.6 Navgaton The navgaton dmenson s related to the features on movng around the web ste. A good web ste and page should let the user feel comfortable to navgate the web ste. Ths s a measure of the extent to whch the web page s easy to navgate and to apply to dfferent tasks. To enable ease of navgaton, t can be acheved by provdng ndcatons of the user's locaton wthn a web ste, navgaton ads, and drectons for navgatng a web ste, etc. 3.7 Vsual Appearance The vsual appearance dmenson s related to features havng to do wth the aesthetcs of a web page. A web page wth a good vsual appearance s more lkely to draw and keep the users attenton. Vsual appearance relates to features as smple as consstency n the page layout or to more complex features (e.g., attractve screen layout, background and pattern, overall use of colour, sharp dsplays, adequate brghtness of pages, presence of eye-catchng mages or ttle on homepage). Ths dmenson s more complcated and subjectve to measure than the accessblty dmenson, whch s based prmarly on the recommendatons by the Web Accessblty Intatve (WAI) on the use of colour. 4. CLASSIFYING THE APPROPRIATENESS OF WEB PAGES In ths secton, the approprateness dmenson wll be further explored n detal. For our purpose n determnng the approprateness of a web page, the emphass s on dentfyng a broad category of the ntended audence based on the readablty (or lngustc complexty) of pages. Ther precse functonalty wll generally not be consdered. The man focus of the classfcaton proposed by Cornell Unversty Lbrary [5] s therefore adopted. However, the categorzaton s generalzed nto only three types: scholarly, news/general nterest and popular. To measure approprateness, we need to determne the lngustc complexty of a web page. Understandng a page s lngustc complexty allows us to estmate the ntended audence of the page (.e., scholarly, news/general nterest or popular). The measurable web metrcs and ther relatonshp wth the approprateness dmenson are summarzed n Table 2 usng the followng notaton: : the greater the number of up arrows (.e., the larger the value of the web metrc), the greater the lkelhood that the web page falls nto the ndcated classfcaton. : the fewer the number of down arrows (.e., the smaller the value of the web metrc), the less the lkelhood that the web page falls nto the ndcated classfcaton. 4.1 Methodology for Formula Dervaton To derve the formulas for classfyng web pages as one of scholarly, news/general nterest or popular, the general methodology s as follows: 1. A sample data set s bult and analysed to understand the characterstcs (or dstrbuton) of the web metrcs for that type of web page. Based on ther characterstcs, the values of the web metrcs are normalzed. 2. For each type of web page, some web metrcs (e.g., those n Table 4 for scholarly pages) are selected to correlate wth ths partcular type of web page. 3. Wth reference to Table 2, the web metrcs are classfed nto three levels of sgnfcance: hgh, medum and low. In addton, the relatonshp between a web metrc and ts use n classfyng a web page type s dentfed (.e., whether t s drectly or nversely related).

4 Fog ndex Mnmalty Numerc words count Rato of numerc words Short forms count Rato of short form Font face count Font sze count Colour count Pont forms count Rato of pont form Top x% of pages Fgure 2. Dstrbuton of the values of dfferent web metrcs for scholarly pages. 4. Dfferent weghts are assgned to dfferent web metrcs and the metrc formula s then defned. Some of these steps are explaned n more detal n the rest of ths secton. 4.2 Characterstcs of Web Metrcs To understand the characterstcs of the web metrcs for classfyng a type of web page, a sample data set was analysed. From ths data set, varous aspects of the web metrcs were measured to obtan the followng values: denotes web metrc for a web page (e.g., Fog ndex s one of the web metrcs). denotes the mean of web metrc for all web pages denotes that for web metrc of a type of web page, the top 20% of web pages (ranked n descendng order of web metrc ) have a value hgher than. denotes that for web metrc of a type of web page, the top 50% of web pages (ranked n descendng order of web metrc ) have a value hgher than. denotes that or web metrc of a type of web page, the top 80% of web pages (ranked n descendng order of web metrc ) have a value hgher than. The values of, and are determned as follows. For each web metrc, web pages n the data set are ranked n descendng order and the dstrbuton of ther values s obtaned. Whle the actual top three x percentages for the three turnngpont (or cut-off) values for each web metrc may not be the same, determnng the cut-off percentages for each cut-off value of the dfferent web metrcs s too complcated and s over-specfed. Therefore, for the sake of smplcty, all web metrcs use the same cut-off percentages at the top 20%, top 50% and top 80%. Hence, the values of, and are determned accordngly. 4.3 Sgnfcance and Relatonshps of Web Metrcs Wth reference to Table 3, n determnng the sgnfcance of the measured web metrcs n characterzng each web page type, three levels of sgnfcance have been dentfed: hgh, medum and low. The correspondng sets of web metrcs for each level of sgnfcance are A H, A M and A L, respectvely. In addton, the relatonshp between a web metrc and ts use n determnng the approprateness dmenson for a type of web page s dentfed, that s, whether t s drectly or nversely related. If the hgher the value of a web metrc the more lkely the web page s of a partcular type, then t s drectly related. If, on the other hand, the lower the value of a web metrc the more lkely the web page s of a partcular type, then t s nversely related. 4.4 Metrc for Each Type of Web Page In formulatng the lkelhood value that a web page s a partcular type of web page, the value of each web metrc s normalzed. Ths normalzaton ncludes classfyng ther values nto four categores by means of the correspondng cut-off values and then dscretzng these categores nto ther normalzed values. For a partcular web metrc, ts three cut-off values for the four categores are A, A and respectvely. Wth these cut-off values, the normalzed value of each web metrc (A norm_ ) s defned as follows: 1 f A A 2 f A A 3 Anorm _ = 1 f A A 3 0 otherwse Equaton 1: Drectly related metrc normalzaton. A norm _ = f f f A A A A A A otherwse Equaton 2: Inversely related metrc normalzaton. The hgher the sgnfcance of a web metrc, the hgher the weght assgned to that web metrc. The metrc of each type of web page s then defned as the summaton of the three levels of sgnfcance wth the summaton of the normalzed values of each web metrc multpled by ther weght factors as follows: Metrcs WebPageType = w h A norm_ + w m A norm_ + w l A norm_ A H A M A L Equaton 3: Web page classfcaton.

5 4.5 Weght Determnaton for Dfferent Web Metrcs The formula for determnng whether a page s scholarly, news/general nterest or popular s the same wth the only dfference n the weghts assgned. The weghts to assgn are determned as follows: 1. Hgher weghts are assgned to those web metrcs wth hgher sgnfcance, as they are more nfluental n determnng the web page type. 2. Intal weghts are arbtrarly assgned based on the 80:20 rule. That s, the value of all web metrcs wth a hgh level of sgnfcance wll contrbute 80% of the fnal metrc value for that web page type. The rest of the 20% wll agan follow ths 80:20 rule to be dstrbuted among the web metrcs wth medum and low level of sgnfcance. 3. The weghts (w ) for each set of web metrcs n dfferent levels of sgnfcance are adjusted expermentally such that a good classfcaton s acheved from the sample data set. 4. The value of max(metrcs WebPageType ) = WEB METRICS IMPLEMENTATION As the mplementaton of the three new web metrcs s very smlar, only the scholarly metrc wll be dscussed n ths secton. The detals of the mplementaton of the other two metrcs, namely news/general nterest metrc and popular metrc, can be found n [11]. To understand the characterstcs of the web metrcs for classfyng scholarly pages, a sample data set of more than 125 web pages from dfferent conferences (such as the WWW and META conferences) was analysed. From ths data set, varous aspects of the web metrcs were measured. The results are shown n Table 2 and the dstrbuton of the values of the varous web metrcs s shown n Fgure 2. Table 2. Web metrc characterstcs of scholarly pages. Web metrc ( ) Averag e ( ) Top 20% ( ) Top 50% ( ) Top 80% ( ) Useful mages count Rato of useful mages Fog Index Mnmalty Numerc words count Rato of numerc words Short forms count Rato of short forms Words DSE Font face count Font sze count Colour count Pont forms count Rato of pont forms No. of hard words Rato of hard words Gven the precedng analyss, Table 3 shows the sgnfcance and relatonshp of the measured web metrcs n characterzng a scholarly web page. Wth reference to Equaton 3, the emprcally determned values of the weghts w h, w m and w l for the Scholarly metrc are 0.125, 0.05 and , respectvely. Table 3. Sgnfcance and relatonshp of web metrcs for scholarly pages. Web metrcs Sgnfcance Relatonshp Fog Index Hgh + Mnmalty Hgh + Numerc words count Low + Rato of numerc words Low + Short forms count Hgh + Rato of short forms Hgh + Words DSE Medum + Font face count Medum - Font sze count Medum - Colour count Medum - Pont forms count Low + Rato of pont forms Low + Hard words count Hgh + Rato of hard words Hgh + (+) means drectly related, and (-) means nversely related 6. EXPERIMENTAL RESULTS In our experments, the data set was generated from the rankng results of the Google search engne usng the frst 100 pages returned for the query data qualty. The contents of these pages were analysed wth respect to dfferent web page attrbutes. A total of 41 attrbutes were measured. The web pages n the data set were categorzed manually to determne ther type (.e., scholarly, news/general nterest or popular). Wthn the data set, "nosy" web pages were frst removed, such as pages that were naccessble (as reflected by ther HTTP status code). Fnally, 94 web pages were left n the data set. To show the effectveness of the three proposed metrcs n classfyng web pages, they were appled to the pages of the data set. The frst x pages ranked by each of the proposed metrcs were used as a reference. Accordng to a study and analyss of user's queres on the web [9], about 80% of users wll not vew more than two query result pages. Usually, each page contans ten query results and hence x was set to 20 n our experments. For a partcular web page metrc, all the pages n the data set were ranked accordng to the metrc. The resultng rankng for the frst x pages was then compared wth the orgnal rankng by Google as well as wth the expected value for the type of web page under consderaton. The expected value s the number of web pages of the type under consderaton that should appear n the frst x pages gven the dstrbuton of that type of web page n the entre populaton 2. It s calculated by multplyng the rato of the partcular type of page (e.g., scholarly) n the data set wth the number x. Hence, for a partcular type of page, f n the frst x pages, the number of those types of pages ranked by ts correspondng metrc s larger than that ranked by Google or gven by the expected value, then ths shows that the web page type metrc can effectvely dentfy those types of pages. In the data set, 28 (~30%), 46 (~49%) and 10 (~10%) of the 94 pages were scholarly pages, news/general nterest pages and 2 We assume a random dstrbuton of the type of web page under consderaton n the populaton.

6 popular pages, respectvely (see Table 4 and Table 5). When ranked usng the scholarly metrc, the frst 20 pages contaned 14 scholarly web pages (.e., 70% of the frst 20 pages were scholarly pages and 50% of the scholarly pages were ranked n the frst 20 pages) (see Table 4 and Table 5). The orgnal rankng from Google contaned only 7 scholarly pages, whle the expected value of scholarly pages n the frst 20 pages s 5.96 (.e., ). When ranked usng the news/general nterest metrc, the 94 frst 20 pages contaned 14 news/general nterest pages. The orgnal rankng from Google contaned 12 such pages, whle the expected number of news/general nterest pages n the frst 20 pages s 9.79 (.e., ). Fnally, when ranked usng the popular metrc, the frst 20 pages contaned 6 popular web pages. The orgnal rankng from Google dd not contan any popular pages n the frst 20 pages, whle the expected number of popular 10 pages n the frst 20 pages s (.e., 20 ). 94 Table 4. Results of the three web metrcs. Type of web page Scholarly News / Popular General Interest In the frst 20 pages: Ranked by ts metrc Ranked by Google Expected value Total no. of pages: Table 5. Percentages of varous types of web pages ranked usng ther correspondng metrcs. Type of web page (X) % of the frst 20 pages that were type X of web pages % of the type X of web pages that were ranked n the frst 20 pages No. of pages Scholarly Scholarl y News / General Interest Popular 70% 70% 30% 50% ~30% 60% News / General Interest Popular Ranked by Web Page Type Metrc Ranked by Google Expected Value Fgure 3. Effectveness of the three metrcs. Fgure 3 clearly show that the three web page type metrcs are effectve at rankng a web page of ther own type. However, the effectveness of the news/general nterest metrc s not as hgh as the other metrcs. Ths may be due to the fact that the dfferentaton among dfferent web page types cannot be totally clear-cut. A web page sometmes can be a combnaton of several dfferent types (e.g., a combnaton of popular page and news/general nterest page). In analysng the few non news/general nterest pages n the top 20 pages, most of them are n fact a combnaton of web page types. In summary, the expermental results show that the three web page type metrcs (.e., scholarly, news/general nterest, and popular) can model and help dentfy the correspondng type of web pages correctly. 7. CONCLUSIONS Currently, prmarly lnk-based metrcs are used n most search engnes to rank the search results. Whle most of the results returned are relevant to the query, sometmes query relevance s nsuffcent to satsfy a user s needs. In some cases, t may also be mportant to return pages that are approprate for the user. To understand ths dfference, a framework for web data qualty s proposed and sx new addtonal dmensonaltes, approprateness, popularty, cohesveness, mnmalty, navgaton and vsual appearance related to web data qualty are ntroduced. Wth reference to ths framework, t was found that lnk-based metrcs are focused on only one aspect of the web data qualty dmensons (.e., belevable). In order to have a more complete pcture of the factors requred to produce a good qualty rankng of search results for users, search engnes should consder more aspects of web data qualty dmensons. The new web data qualty dmenson, approprateness, was further explored wth respect to ts measurement and use n rankng search results. To model the approprateness of a page for a query, web pages are classfed nto three man types accordng to ther lngustc complexty: scholarly, news/general nterest or popular. To measure the lkelhood that a page s one of the above three types, varous attrbutes of a web page were measured quanttatvely. Ths new dmenson of web data qualty, approprateness, s then computed by three new web metrcs. From the experments, the effectveness of the metrcs has been shown. Users enter a query wth ther expected type of pages to the search engne. The metrcs can then help the search engne to rank web pages not only by relevance to the user's query but also by ts approprateness to ther lngustc need. Automatc trackng of user preference for the type of pages they would lke returned would defntely be useful. Wth such trackng, t would be possble to develop a user preference profle. The preferred type of web page for a partcular query could be kept n ths profle allowng the search engne to automatcally know the preferred type of web page for a user and return the most approprate pages. Consderng the approprateness dmenson to rank search results s the frst step to mprove the pages returned for a query. The measurement and use of the other new web qualty dmensons s currently under nvestgaton. 8. REFERENCES [1] B. Amento, L. Terveen and W. Hll. Does Authorty mean Qualty? Predctng Expert Qualty Ratngs of Web Documents. Proc. of the 23rd ACM SIGIR Conf., , [2] Rcardo Baeza-Yates, Felpe Sant-Jean, and Carlos Castllo. Web Structure, Dynamcs and Page Qualty, In Proceedngs of SPIRE 2002, LNCS, Sprnger, Lsbon, Portugal, 2002.

7 [3] S. Brn, and L. Page. The Anatomy of a Large-scale Hypertextual Web Search Engne, Proceedngs of the 7th World Wde Web Conference, [4] Junghoo Cho and Sourashs Roy. Impact of Web Search Engnes on Page Popularty. In Proceedngs of the World- Wde Web Conference (WWW), May [5] Cornell Unversty Lbrary. Dstngushng Scholarly Journals from Other Perodcals. [6] C. Fox, A. Levtn, and T. Redman. The Noton of Data and ts Qualty Dmensons. Informaton Processng and Management, 30(1): 9-19, [7] J. D. Graofalaks, P. Kappos and D. Mourloukos. Web Ste Optmzaton usng Page Popularty. IEEE Internet Computng, 3(4): 22-29, [8] B. J. Jansen, A. Spnk and T. Saracevc. Real lfe, Real users, and Real needs: A Study and Analyss of User Queres on the Web, Informaton Processng and Management, 36(2), , [9] Jon Klenberg. Authoratve Sources n a Hyperlnked Envronment. Proc. 9th Symposum on Dscrete Algorthms, [10] Leo L. Ppno, Yang W. Lee, and Rchard Y. Yang. Data Qualty Assessment. Communcaton of the ACM, 45(4): , Aprl [11] Joshua C. C. Pun and Frederck H. Lochovsky. Fndng an Approprate Web Page. Techncal Report, HKUST-CS-xx-xx, [12] D. M. Strong, Y. W. Lee, and R. Y. Wang. Data Qualty n Context. Communcatons of the ACM, 40(5): , [13] UC Berkeley Lbrary. Evaluatng Web Pages: Technques to Apply & Questons to Ask. aluate.html [14] Cyntha D. Waddell. Applyng the ADA to the Internet: A Web Accessblty Standard. [15] Rchard Y. Yang, M. P. Reddy and Henry B. Kon. Toward Qualty Data: An Attrbuted-based Approach. Decson Support Systems, 13(3): , [16] Png Zhang and Gsela M. von Dran. User Expectatons and Rankngs of Qualty Factors n Dfferent Web Ste Domans. Internatonal Jounral of Electronc Commerce. 6(2): 9-33, Wnter

8 Appendx: Dmensonalty of Data Qualty -- How Applcable to Web Data? DQ Dmensons DQ Category [12] Descrptons [10, 12, 15] Applcable to Web? Intrnsc Contextual Accessblty Representatonal Accessblty X User must be able to get to the data and user has the means to get the data. Avalablty Exsts n some form that can be accessed, percentage of tme an nformaton source s "up". Access securty X Relates to the confdental nature of data whch requre specal access permsson (e.g. sgn on, enter password). Interpretablty (or Ease of Understandng) Syntax Semantcs Usefulness X User understands the syntax and semantcs of the data, user can nterpret values correctly; degree to whch the nformaton conforms to techncal ablty of the consumer; the extent to whch data s n approprate languages, symbols and unts and the defntons are clear. Data can be used as an nput to the user's decson-makng process. Relevant X Fts requrement for makng the decson, degree to whch nformaton satsfy users need, the extent to whch data s applcable and helpful for the task at hand, nformaton that drects to the pont, havng to do wth the matter at hand. Tmelness (or Freshness) Currency Non-volatle X Whether the recorded data value s not out of date and avalablty of output on tme; the extent to whch the data s suffcently up-to-date for the task at hand; the tme dfference between when the process s supposed to have created a value and when t actually has. Degree to whch data n queston s up-to-date; when the data tem was stored n the database. How long an tem remans vald. Belevable X User can use the data as a decson nput; the extent to whch data s regarded as true and credble. Completeness X The extent to whch data s not mssng, s of suffcent breadth and depth for the task at hand and values are present n a data collecton; all values for a certan varable are recorded. Consstency X It can refer to several aspects of data: values of data, representaton of data and physcal representaton of data. Relatng consstency to the values of data, a data value s expected to be the same n all cases. Credtable (or Reputaton or Authorty) Accuracy (or Correctness) X X Degree to whch the nformaton or ts source s n hgh standng, authortatve and hgh reputaton. The extent to whch data s hghly regarded n terms of ts source or content, nformaton dependable. The recorded value s n conformty wth actual value; rato of no. of correct values n a source to the overall no. of values n a source; nformaton has no error, correct, exact, precse, rght and true. Objectvty X The extent to whch data s unbased, unprejudced, and mpartal. Web accessblty relates to the nformaton on a web page that can be accessed by the broadest range of users of computers and communcatons equpment, regardless of age or dsablty [14]. Web pages can be publcly accessble. It has also been nterpreted as the number of broken lnks contaned n the web pages. Intranet pages can be accessed by regstered users only. Free form of web pages makes t dffcult to evaluate the nterpretablty. Use of sem-structure (e.g. table, pont form) and specal HTML tags (e.g. ttles, headngs, emphaszed words) can help user nterpret the content n a web page. Not easy to nterpret unless the web page has been expressed n XML format. Relates to the measurement of relevancy between queres and search results. Some web pages are rather statc and ther contents do not change frequently. Frequently updated web pages can also be found n the fnancal and news type of web stes. Dates are not always delberately stated n a web page. Usually, the only date nformaton can be obtaned from the date of the web page (or fle). However, whether t means date frst created, last updated, or placed on the web s not clear. Dffcult to defne all values as the scope of web s unlmted. As web page s free format, consstency of web page (and the ts content) s dffcult to guarantee. It relates to the reputaton of an organzaton that owns the web page. It can be estmated by the hub/authorty rank, by any external recognton of the web ste (e.g. awards, no. of vsted tmes). Almost everyone can publsh on the web and hence the content of web pages may not be verfed by edtors. Dffcult to guarantee the accuracy of content. Dffcult to judge as the goals or ams of persons presentng the materal often are not clearly stated.