presented on the World Wide Web. We describe research related to the evaluation of

Size: px

Start display at page:

Download "presented on the World Wide Web. We describe research related to the evaluation of"

Stephanie Bishop
5 years ago
Views:

1 The Perceived Accuracy of World Wide Web Information Dale L. Lunsford, Ph.D. The University of Southern Mississippi, Hattiesburg, MS Abstract This paper examines undergraduate students' perceptions of the accuracy of infonnation presented on the World Wide Web. We describe research related to the evaluation of information product quality, the selection of information sources, and problems associated with information available on the World Wide Web. We describe an initial empirically study that explores the infonnation-accuracy perceptions of undergraduate students. In this study, we asked our subjects to rank various information sources including the World Wide Web, business magazines, scholarly journals, general-interest magazines, and books in terms of accuracy. Our findings indicate that undergraduate students do not perceive the World Wide Web to be any less accurate as an infonnation source than other common infonnation sources. In fact, we find that our subjects perceive the World Wide Web to be more accurate than several other information sources. We discuss implications of these findings including the need to educate World Wide Web users on the systematic differences in the World Wide Web vis-a-vis other information sources. Finally, we discuss extensions to this research project. Introduction The World Wide Web has rapidly become a popular source of information in the business and academic communities. McKeown and Watson (1996,3) indicate that a significant reason for the rapid growth in popularity of the World Wide Web is that with the World Wide Web people can find and access infoffi1ation quickly. Although the World Wide Web may be a fast, useful source of infonnation, problems may arise when using the World Wide Web as an infonnation 161

2 source. Because of the open nature of the World Wide Web, the infonnation available on the World Wide Web may not always be as accurate as information provided by other common infomlation sources. Fitzgerald (1995) points out that "in cyberspace, everyone can hear you scream -and distort, defame, dissemble, and lie." Unfortunately, WorldWide Web users may, 1!', not be cognizant of the differences between the World Wide Web and other information that may impact the accuracy of the infomlation presented (Pask and Snow; 1995). sources In an academic setting, faculty need to be aware of students' perceptions about the accuracy of infonnation presented on the World Wide Web. For example, students may not be aware of the systematic differences in the publication process for presenting information on the World Wide Web vis-a-vis other infonnation sources. As a result, students may consider infonnation on the World Wide Web to be as accurate as information presented by other information sources. When information presented on the World Wide Web is not as accurate as information presented using other infonnation sources, students may not be able to make this detennination, and may use the infoffilation as if it were accurate. This paper presents an initial empirical investigation of students' perceptions regarding the accuracy of information available on the World Wide Web compared with information presented using other common infonnation sources. This paper is organized as follows The next section provides a review ofwork related to infonnation product quality and the role of the World Wide Web in research. The third section discusses the research method employed in this study. The fourth section presents the findings from this study. The fifth section discusses the findings and 162

3 the implications of these findings. The final section provides concluding remarks and discusses extensions to this work. Literature Review Klobas (1995) describes an Infonnation Use Model that depicts the influence that infonnation product quality and accessibility have on infonnation use. Infonnation product quality is the quality of the infomlation provided by an infomlation source. Klobas divides infomlation product quality into three elements: information quality, technical quality, and relevance According to Klobas, attributes of information quality include accuracy, currency, authority, and novelty. Figure 1 graphically depicts the influences on infonnation product quality. According to Klobas, infonnation providers may measure quality independent of the use of the infonnation ~ r Accuracy 1 [~~J--t ~ Figure 1: Detenninants of Infonnation Product Quality Stair and Reynolds (1997, 7) provide a definition for accuracy; infonnation is accurate if it is free from en-ors. According to Tate and Alexander (1996), information is current ifit is up to date, Klobas (1995) defines authority as "the extent to which the infom1ation resource contains 163 L

4 infonnation from authoritative sources." Klobas also defines novelty as "the extent to which the resource contains infoffilation new to the user." Gerstenberger and Allen (1968) define technical quality as the reliability of the infomlation obtained from a source. Infomlation is relevant if the infomlation is important to a decision maker (Stair and Reynolds, 1997, 7). Gerstberger and Allen (1968) examined factors that influence an engineer'.s selection of vendors, customers, external sources, etc. Gerstberger and Allen asked engineers to rate these experience. In a field study, Gerstberger and Allen then compared the engineers' actual use of solution paths by selecting the path with the "least average rate ofprobable work." Since the the academic community must examine how people perceive the WorldWide Web, and provide infon1lation about potential weaknesses of the World Wide Web as an infon1lation source Pask and Snow (1995) identify several problems with information on the World Wide Web unorganized, there is no fomlal review process for ensuring quality, infomlation on the World 164

5 Wide Web is subject to continual update and revision (the infomlation evolves dynamically in public), and anyone with access may publish information. Brandt (1996) describes three ways in which information may be filtered to ensure quality: an authoritative source may issue infomlation, editors or peer reviewers may review infomlation, or a panel of experts may select infonnation sources for inclusion in a collection. Brandt expresses concern that anyone, at any time, may publish information on the World Wide Web. Brandt also points out that World Wide Web search engines do not necessarily provide a mechanism for filtering out bad information Search engines compile infonnation references using one of two techniques: the infonnation provider registers their site, or the search engine provider registers the site. Ifthe search engine provider registers the site, the provider may do this using one of two techniques: the provider may automatically query each World Wide Web server to locate web pages, or the provider may manually examine World Wide Web servers to determine the information available and the appropriateness of the infonnation for inclusion in the search engine's database. Clearly, only one of these techniques provides any measure of information product quality, the others either rely on the infonnation provider or automated processes to build a search engine's database Tate and Alexander (1996) describe several risk factors associated with the World Wide Web First, the World Wide Web is not limited to academic or industry research; businesses often use the World Wide Web as a marketing tool by including marketing-oriented web pages. Second, World Wide Web sites blend infomlation, entertainment, and advertising in ways that may make it difficult for the user to distinguish between the three, Third, hypertext links connect World Wide Web sites in a manner that sometimes makes it difficult for the user to determine when they have moved from information provided by one organization to information provided by 165 L

6 another organization. Finally, the content of the World Wide Web and World Wide Web sites is subject to frequent revision so infonnation available one day may not be available the next, or the infoffilation may be dramatically altered Research Method This research project empirically explores students' perceptions of the accuracy of information presented on the World Wide Web versus other information sources The research question addressed is, do students perceive the World Wide Web as being more accurate, less accurate, or equal in accuracy to other information sources. We compared the following information sources advertisements, company annual reports, books, business magazines, general-interest magazines, scholarly journals, newspapers, television news, World Wide Web, and Internet newsgroups Based on predictions by Pask and Snow (1995) and Brandt (1996), we expect to find that our subjects do not perceive the World Wide Web as being any less accurate than other iilfonnation source. The subjects were 110 undergraduate business students enrolled in various sections of an introductory Management Infoffi1ation System course who volunteered to participate in this study. In addition to the subject's ranking of information sources, we collected descriptive infomiation about each subject. 46% of the participants in this study were female The average age was 22.3 years old with a range from 19 to 42 years old; the median age was 21. Most of the participants in this study were business majors (98%) with 24% majoring in Accounting, 20% majoring in Marketing, 12% majoring in Banking and Finance, 12% majoring in Business Administration, and the remainder majoring in other business disciplines; only 5% had a 166

7 declared major of Management Information Systems. 51% of the participants were juniors and 45% were seniors. We also collected information about the subject's perceptions and use of the World Wide Web. At the time of the study, 98% of the subjects had used the World Wide Web. Of the subjects who had used the World Wide Web, subjects on average had used the World Wide Web times in the past thirty days with a range from zero to 100 times; the median was 6 times. Of the subjects who have used the World Wide Web, 21% had never used the World Wide Web to study research reports, 8% had used the World Wide Web to study research reports before but would not do so again, 65% had used the World Wide Web approximately three or fewer times per week to study research reports, and 6% had used the World Wide Web four or more times per week to study research reports. Only 4.5% had a personal web page ofwhich all had designed their web pages and four had constructed their own page. Finally, 14.5% of the subjects had completed a research methods course, We conducted this study early in the semester before discussing data communications or the Internet, and before the students completed any outside assignments that required the use of the Internet. We asked the subjects to rank various common business infonnation sources using the following instructions: 1 Assign a rank of' l' to the source that you feel is most accurate. Assign a rank of '2' the source that you feel is second most accurate, and so on. 2. If you feel that two sources are equally accurate, assign the same rank to both 167 I

8 3. Please do not skip numbers when ranking sources. If you have no perceptions about a specific information source, do not assign a rank and, instead, check the "No Opinion " column. Findings According to Ker1inger (1986, ) and Davis and Cosenza (1985, 169), in most cases, it is best to treat ordinal measurements as interval measurements. Therefore, even though our measurement is ordinal, we used a one-way analysis ofvariance (one-way ANOV A) with the accuracy ranking as the dependent variable and the information source as the independent variable. Our one-way ANOV A reported a test statistic F value equal to (p-value = 0.000) which is significant. As a result, we conclude that the information source does influence the accuracy ranking assigned by our subjects. We also generated descriptive statistics including a confidence interval for each of the infornlation sources. Table 1 provides infolmation about the accuracy ranking for each of the infolmation sources. The Responses column lists the number of subjects who provided a ranking for each information source. The Mean column lists the average mean response for the ranking of each information source. Since subjects ranked the infonnation sources starting with one for the most accurate source, a lower mean indicates a higher level of perceived accuracy. The StDev column lists the standard deviation for the ranking of each information source. The Lower and Upper columns list the upper and lower boundaries for the 95% confidence interval for each information source. 168

$8171 901,- 102 4.0490 2.9996 1021 -, 104 3.8269 2.2958,~~- -~. -107 3.7009 2.4349 -~~ ~ ~ 102 3.2647 2.4532 102 \ 94 3.2553 2.3506 -! 2.7130 1.$ 7404 ~ 108 94 Table 1: 95% Confidence Intervals for Accuracy Rankings As can be seen from Table 1, the infonnation source ranked most accurate was business magazines (mean =

7404 ~ 108 94 Table 1: 95% Confidence Intervals for Accuracy Rankings As can be seen from Table 1, the infonnation source ranked most accurate was business magazines (mean =

However, based on the confidence intervals (see Figure 2), there is no significant difference in the perceived accuracy of these three information sources.

9 I Information Source Responses I Mean StDev Advertisements General-interest Magazines T.., ~ ' , , ,~~- -~ ~~ ~ ~ \ ! ~ Table 1: 95% Confidence Intervals for Accuracy Rankings As can be seen from Table 1, the infonnation source ranked most accurate was business magazines (mean = ), followed by scholarly journals (3.2533), and then the World Wide Web (3.2647). However, based on the confidence intervals (see Figure 2), there is no significant difference in the perceived accuracy of these three information sources. Additionally,subjects did not perceive the World Wide Web as being significantly different from newspapers, books, or company annual reports. Subjects did perceive the World Wide Web as being significantly more accurate than Internet newsgroups, the television news, general-interest magazines, and advertisements

Accuracy Ranking (95% Confidence Intervals) Figure 2: Confidence Intervals for Accuracy Rankings We used Duncan's multiple range test, Tukey's honestly significant difference test, and the hold the

10 Accuracy Ranking (95% Confidence Intervals) Figure 2: Confidence Intervals for Accuracy Rankings We used Duncan's multiple range test, Tukey's honestly significant difference test, and the hold the overall a equal to Our results were confirmed using these tests Discussion Consistent with Pask and Snow (1995) and Brandt's (1996) predictions, this study found that undergraduate business students do not perceive the World Wide Web to be any less accurate than other infonnation sources often used in business. Although the World Wide Web was ranked as the third most accurate information source in terms of the mean accuracy ranking, this 170

11 study did not fmd a statistica11y significant difference in the accuracy ranking for the World Wide Web versus business magazines, scholarly journals, newspapers, books, and company annual reports. The World Wide Web was ranked as more accurate than Internet newsgroups, the television news, general-interest magazines, and advertisements. Since the publication process for the World Wide Web is clearly different than the publication process for other inforn1ation sources, this is an important finding Faculty often ask students to use the World Wide Web to obtain infonnation about various topics. We conducted an infoffi1al survey of students in the introductory Management Information Systems course and found that approximately one half of the students had been specifically directed to use the World Wide Web on at least one course assignment prior to participating in this study. At the same time, no student in this course had ever been presented with information comparing the World Wide Web with other information sources in terms of accuracy or other infonnation product quality attributes, Suggestions for reducing the risk of using low quality infornlation found on the World Wide Web center around educating World Wide Web users about the risks associated with World Wide Web-provided infonnation and techniques for evaluating infonnation quality. Brandt (1996) suggests introducing students to the World Wide Web information product quality by comparing the World Wide Web publication process with the publication process for other infonnation sources. Brandt also suggests that World Wide Web users evaluate the validity, reliability, and authenticity of inforn1ation provided, in addition to the appropriateness of the information for the intended use. Tate and Alexander (1996) suggest that World Wide Web l 171

12 evaluation skills be integrated with instruction related to the use of the World Wide Web. Tate and Alexander present a three part lesson plan for providing these skills. The first lesson focuses on evaluating infonnation sources, Tate and Alexander describe five criteria for evaluating inforn1ation sources: accuracy, authority, objectivity, currency, and coverage. Tate and Alexander relate these five criteria to the World Wide Web. According to Tate and Alexander, World Wide Web infonnation is often not reviewed, World Wide Web pages often do not include infomlation about the author's qualifications, World Wide Web pages often do not include information about the currency of the information presented, World Wide Web pages often do not include information about the information-provider's motivations or goals from presenting infonnation, and differences between printed infonnation and World Wide Web infonnation are not always clarified. Tate and Alexander's second lesson focuses on providing students with checklists for evaluating different types ofworld Wide Web pages including advocacy pages, business/marketing pages, infonnational pages, news pages, and personal pages. The third lesson provides students with the opportunity to evaluate real World Wide Web pages using the checklists provided Conclusion This paper provides an initial, exploratory look at how people rank various information sources in tenns of accuracy. This paper confirms the predictions of researchers and academics regarding people's perceptions of the accuracy ofinfonnation on the World Wide Web. Although the World Wide Web does not include many of the systematic controls associated with other infoffi1ation sources, our subjects ranked the World Wide Web as being of equivalent accuracy to other infonnation sources. This finding is cause for concern since the World Wide 172

13 Web is rapidly becoming the default first information source for students and business people As a result, this study indicates a need to provide World Wide Web users with training to sensitize them to the risks associated with using World Wide Web-provided information and skills to assess the accuracy of this information, Since this is a preliminary study, much research still remains in this area. One extension involves comparing the perceptions of accuracy of other groups including graduate business students, university faculty, library and information sciences students, and business people. A second extension involves examining the factors that lead people to use the World Wide Web as an infonnation source; this will provide a basis for defining ways that other, potentially more accurate information sources can be made more accessible. A third extension involves providing subjects with a pre-instruction questionnaire and a post-instruction questionnaire to determine if subjects change their evaluation of the accuracy of the information sources. A fourth extension involves exploring how perceptions vary with the type ofinfonnation reported (e.g., product announcements versus business forecasts). A final extension involves assessing people's perceptions of the currency, authority, novelty, and relevance of World Wide Web infoffilation We plan to explore these extensions during the next year. Bibliography Brandt, D. S., Techman's TechPage- Evaluating Infomlation Libraries, 16,5 (1996), on the Internet, Computers in Davis, D. and Cosenza, R. M., Business Research for Decision Making, 3rd edition, Wadsworth Publishing Company, California, Fitzgerald, M., Interactive Communications -First Publisher, 128,9 (1995), Job in Cyberspace: Establish Trust, Editor & 173 l

14 Gerstberger, P. G. and Allen, T. J., Criteria Used by Research and Development Engineers in the Selection of an Information Source, Journal of Applied Psychology, 52,4 (1968), Kerlinger, F. N., Foundations of Behavioral Research, 3rd edition, Holt, Rinehart, and Winston, Inc., New York, Klobas, J. E., Beyond Infonnation Quality: Fitness for Purpose and Electronic Infonnation Resource Use, Journal oflnformation Science, 21,2 (1995), '.. }1 McKeown, P. G. and Watson, R. T., Metamorphosis: A Guide to the World Wide Web & Electronic Commerce, John Wiley & Sons, Inc., New York, Stair, R. M. and Reynolds, G. W., Principles of Information Systems: A Managerial Approach, 3rd edition, Course Technology, Massachusetts, Tate, M. and Alexander, J., Teaching Critical Evaluation Skills for World Wide Web Resources, Computers in Libraries, 16, 10(1996),49-52;

University of Mumbai

AC 7/4/2014. Item 4.36 University of Mumbai Revised Scheme of Evaluation for Continuous Assessments and Semester End Examinations for Under-graduate Programmes of Faculty of Arts, Commerce and Science