Using HCI techniques to evaluate Electronic Commerce sites

Transcription

1 Using HCI techniques to evaluate Electronic Commerce sites Prof Carina de Villiers Address: Department of Informatics, University of Pretoria, Pretoria, address: Telephone number: Fax number: Abstract According to Dix et al [1998] human-computer interaction (HCI) involves the design, implementation and of interactive systems in the context of the user's task and work. This paper will focus on the HCI techniques used in the design and implementation of interactive systems with the emphasis on electronic commerce sites. Electronic commerce is according to Turban et al [1999] the process of buying and selling or exchanging of products, services and information via computer networks, which includes the Internet. Books on electronic commerce seldom, if ever focus on the of web sites [Turban et al, 1999; Kalakota & Whinston, 1996; Dahl & Lesnick, 1996]. The fundamental question that needs answering is whether we can apply the existing HCI techniques to evaluate electronic commerce sit or do we need more than that or totally different techniques. Existing techniques that will be discussed with a view of applying them to evaluate electronic commerce sites are cognitiv walkthroughs, heuristic, review-based, model-based, observational techniques and usability testing. In order to answer the question the paper will appl some of these HCI techniques to a number of South African electronic commerce sites Proposals for the extension of these techniques to make them more applicable for electronic commerce, as well as the possibility of new techniques, will be discussed. Keywords: Human-computer interaction, electronic commerce, techniques [ Introduction ] [ Defining Electronic Commerce ] [ HCI techniques ] [ Applications ] [ Conclusion ] [ References ] Introduction The importance of in HCI design is mentioned by several authors of HCI books [Dix e al, 1998; Preece, 1994; Schneidermann, 1992]. Preece [1994:601] says that "Without doing some form of it is impossible to know whether or not the design or system fulfils the needs o the users and how well it fits the physical, social and organizational context in which it will be used." In the electronic commerce field, especially in business-to-consumer commerce, it is essential to evaluate the success of the business on the web. If one, however, consult books writte on electronic commerce there is a lack of discussion or mostly no discussion at all on the of electronic commerce sites. [Turban et al, 1999; Whiteley, 2000; Kalakota & Whinston, 1996; Dahl & Lesnick, 1996] This paper discusses the use of HCI techniques in an electronic commerce postgradua

2 class as a tool to teach the students how to evaluate business-to-consumer sites. The application o the various students of these methods to different South African web sites will be described. All sites were developed by leading South African web development companies as listed in Intelligence, a monthly computer magazine published in South Africa. The problems that came forward using HCI techniques will be highlighted and possible solutions and further research will be mentioned. The first part of the paper sets the scene by describing what is understood by electronic commerce and HCI. Defining Electronic Commerce Turban et al [1999:4] describes electronic commerce as "an emerging concept that describes the process of buying and selling or exchanging of products, services and information via computer networks including the Internet." Some people use the concept of electronic business and then refers to a broader definition of electronic commerce, which includes servicing customers, collaborating with business partners and conducting electronic transactions within an organisation Electronic commerce will be used in this paper in its broadest sense. We normally classify electronic commerce by the nature of the transactions which can be business-to-business, busines to-consumer and intra-business or organisational electronic commerce. The focus of the s done for this study is on business-to-consumer electronic commerce, which involves retailing transactions with individual shoppers and was done on the final web site that was published by the particular company and not during the design phase. HCI techniques Defining Preece[1994:602] defines as follows: "Evaluation is concerned with gathering data about the usability of a design or product by a specified group of users for a particular activity within a specified environment or work context." Why do we want to do? The answer to this question is normally to find out what the users want and what problems they experience. Without the system that reaches the consumer will be untried and only reflect the ideas of the design team with no relationship betwe design and use. [Preece, 1994] During it is important to consider the characteristics of the users of the system, the types of activities the users will do and the nature of the product (prototype or fully developed system) being evaluated. We normally distinguish between formative and summative s In this study a summative of a fully developed product was done. According to Rubin[1988], there are three general objectives which can be identified in the of any user interface, irrespective of the type of interface, the hardware or software, th stage in the design, the use or not of humans in the process, the method used and the type of data gathered. These objectives are: The assessment of the capabilities of the design against the requirements of the users. The assessment of the impacts of design decisions on the user and the user's interactions with the system.

3 The diagnosis of problems with the design in an attempt to identify and clarify the scope o specific problems. Review of HCI techniques Several authors described different techniques. [Preece, 1994; Dix et al, 1998, Schneiderman, 1992] These include observing and monitoring usage, collecting user's opinions a surveys, experiments and benchmarking, interpretive, predictive, usability laboratories, field studies, cognitive walkhrough, heuristic, review-based, model-based, empirical methods, et cetera. The following methods, based on the work of Dix et al[1998] and Preece [1994] were discussed with the students in the Electronic Commerce class: Cognitive walkthrough During the cognitive walkthrough the evaluator step through the actions required by the user interface to check for possible usability problems. The walkthrough requires four things: a description of the system, a description of the task that the user must perform, a list of the actions needed to complete the task and a description of who the users are. Heuristic This method was developed by Jakob Nielsen and Rolf Molich and concerns the use of a set of guidelines or general principles to critique a decision that has already been made. The list of heuristics includes: visibility of system status, match between system and real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibilit and efficiency of use, aesthetic and minimalist design, help functions, and help users recognise, diagnose and recover from errors. Review-based In review-based, a study of literature for evidence to support or refute the different aspects of the design is done. According to Dix et al [1998] "the review should take into account similarities and differences between the experimental context and the design under consideration." Model-based Certain cognitive and design models provide a means to evaluate, for example, the GOMS model keystroke model (at a low level) and the design methodology - design rationale - which provides framework in which design options can be evaluated. Empirical methods Although the use of an experiment is a powerful tool for evaluating design, only the basic principles were discussed with the students due to the lack of statistical background of some of th students. In this method the evaluator chooses a hypothesis to test, using certain subjects and variables. Observational techniques

4 Think aloud, protocol analysis and post-task walkthroughs were discussed. Query techniques Two main types of query techniques were mentioned, namely interviews and questionnaires Usability testing The usability principles from HCI, namely learnability, flexibility and robustness [Preece, 1994, D et al, 1998] were discussed with the students. Considering usability from a web site perspective: According to Keith Instone ( ) you need to follow five steps and the watch people carefully as they try to use your site. The steps are: Know your purpose with the sit find ordinary users, watch and learn, collect the data and go back to the drawing board. During the lecture on techniques Jakob Nielsen's top ten mistakes in web design ( ) was also mentioned as well as the principles of good digital store design as described by Dahl & Lesnick [1996]. Applications of HCI techniques to a number of South African electronic Students in the fourth year (honours) Informatics course on electronic commerce were given the assignment to evaluate different web sites where business-to-consumer transactions take place. They had to choose at least one of the methods discussed in class. In total, seven different sites were given to them, divided equally amongst the students. All of these sites were outsourced to South African web development companies and not developed in-house in the company. Only three of the sites evaluated will be described in this paper, namely the site of Fort King Hotels, AA Auto finance and Cellular Shop. The students background on HCI techniques were information given to them in a two-hour lecture and study material handed out in class. They have had extensive exposure to interface design, both in the undergraduate studies an in the design of web sites during the electronic commerce course. Fortes King Hotels (

5 Fortes King is a South African company that manages 11 hotels in the Western Cape Province. O the site, the user can browse through a list of hotels, see what the hotels look like and make holiday or conference reservations. The main objective of the site is to advertise and sell their services. The target users are people who want to go on holiday or seek a conference location. Evaluation of this site was done using an score sheet as described in the conclusion, cognitive walkthroughs, heuristic, think-aloud observational techniques, usability testi and post-test interviews. The summarised impression of the 9 student evaluators is given in table Evaluator Methods used Impression 1 Evaluation score sheet Cognitive walkthroughs Think-aloud observation Post-test interviews 2 Heuristic Usability testing 3 Heuristic 4 Review-based An average site with no serious errors, but rather dull. No entertainment, specials or new features that would prompt the customer to return to the site. Design is simple and clear and looks professional but rather clinical and boring. No entertainment and is purely an information provider. Inconsistent in some places and there is room for improvement The site is easy to use, understand and navigate. Includes the correct amount of information. The use of images is done very effectively The site has a professional appearance, easy to navigate and use includes all the necessary information. 5 Usability testing The site can do with a lot of improvement: It does not add any information that could not be included in a brochure or given over the phone. It does not give Fortes-King a competitive edge 6 Cognitive walkthroughs Heuristic 7 Review-based 8 Heuristic The web site provided enough information to satisfy customer needs at a high level, screens effectively displays the company profile, but more advertising and less boring screens could be used. Site is properly designed, maintained and developed. Content and context offered are very effective in providing the necessary information and means to enable customers to interac More effort can be made to entertain visitors and rewarding of loyal customers. A very good web site.

6 Page 6 of 11 9 Heuristic Content of the site is sufficient but site navigation is difficult, much more can be provided on the site to expand their business Use of Netmechanic Table 1: Fortes-King Hotels AA Auto Finance ( ) The AA Auto finance web site is aimed at the prospective vehicle buyer and gives a customer the opportunity to apply for vehicle financing online. It also aims to inform potential customers of services offered and to offer the initial stage of those services online. Eleven students evaluated this site and all of them chose to do a heuristic of the site. Their impressions of the site are summarised in table 2. Evaluator Impressions 1 The site is relatively well-designed, but includes some of the top ten design mistake [Jakob Nielsen]. At first glance the site seems cluttered. 2 The site is a pleasing and informative web site with all the basic information that is needed. Some minor design errors were encountered. 3 The site is aesthetically pleasing, but lack on the content side. 4 The site is technologically intensive and has an easy to use interface. The testing shows very positive results with easy access to information. Downloading time may be a problem. 5 The overall impression is very good, except for slow response times. Layout was done very well. 6 Use of colours on the site is not working, eg white on yellow. A number of errors were picked up, eg links that are not working, orphan pages. Overall impression is good and the use of graphics effective. 7 It is a good site in terms of simplicity, provision of feedback and short loading time There are erroneous links, empty pages and misleading heading links.

7 Page 7 of 11 8 This student used a rating sheet for the different heuristics and came up with 62% f content and 73% for design. His overall impression was good although some errors were reported. 9 This student also used a point system and awarded the site with a mark of 20 out of possible 40 marks. His final comment: "It takes more than a pretty face to create successful web business". 10 Too many mistakes and oversights make this a fairly average to poor web site, due an overall lack of content and interactivity. 11 Positive: no long scrolling pages and overly long download times. Negative: links a not working and use of colour. Table 2: AA Auto Finance Cellular Shop ( ) The Cellular Shop is the only independent specialist retailer authorised to offer all network products from the two official cellular service providers Vodacom and MTN. They specialise in providing unbiased advice about communication requirements and the selling of cellular phone, satellite phones, PDAs and cell phone accessories online with free delivery of you purchase to yo doorstep. The site has both a normal and a flash version. Ten students evaluated this web site and their impressions are summarised in table 3. Evaluator Methods used Impression 1 Usability testing Heuristic 2 Heuristic Review-based It has an overall good design with a few problems. Pages are easy to access, navigation is well-designed, no broken links o orphan pages. Users are encouraged to return with news and links pages. Site is quite good but people will probably use it only to colle information before going to a traditional store. Added service like choosing your phone and contract on the web is necessar

8 Page 8 of 11 3 Heuristic The site performs satisfactory with respect to consistency and aesthetic and minimalist design. Some information is outdated and there are some orphan pages 4 Heuristic No proper design was carried out before the implementation. The site is not impressive at all and there are lots of common mistakes that web designers make. 5 Heuristic Usability testing Overall impression was that the site is aesthetically pleasing a functional. The content is relevant and useful, it is easy to lea and is entertaining and interesting. An excellent site. 6 Usability testing The student used a scale to rate the site and came up with a average value of 4.2 on a scale of 1 (not acceptable) to 5 (excellent). 7 Heuristic There are interesting links which make this more than just a s selling things. It is a good site which can be awarded if they clear up some minor mistakes. 8 Cognitive walkthrough Heuristic Review-based 9 Heuristic Usability testing The site is functional and easy to use. The student rated the si as 7 / 10. The site reaches the goals of the company, which is providing professional image. The site is successful, scoring high on bas design principles and provides a functional and effective digit storefront 10 Heuristic On the whole a good web site with lots of information, but there are minor problems that should be solved to improve the company's image. Table 3: Cellular Shop Concluding remarks on the Judging from the different responses from the students, who are not experienced evaluators but knowledgable in electronic commerce and have at the stage when they did this assignment alread completed the development of their own electronic commerce site, the process was no entirely successful. Most students chose to use the heuristic, which was probably the easy way out. They did not, however, use the same set of guidelines. Many of the students used the top ten mistakes mentioned by Jakob Nielsen on his site as a set of guidelines. Others used the set of heuristics mentioned by Dix et al [1998], while there were also students who used the design principles of a digital storefront described by Dahl & Lesnick[1996 and any guidelines they could get on the web describing good web design in general. The use of these different sets of guidelines probably resulted in the difference of opinion about the three sit evaluated. As could be expected, students who used more than one method came up with far better results and a more complete than those who used only one method.

9 Software exists on the web that allows you to test your web site. Most of those packages and guidelines are general and not developed specifically for electronic commerce business-toconsumer sites. One such package is on the web site garage site ( ). National Chamberweb also allows you to form an idea of the quality of your site ( ) and concentrate more on electronic commerce type of applications. This tool considers the following sections: Findability, where the site is rated on whether the right actions are taken to build traffic. One such action will be have the URL on all printed material and promote the site in advertisements, events and conversations. The site must also be registered with the main search engines such as Yahoo, LookSmart, Snap! and others. External links to your site must exist Relevance and features, which evaluates relevant content, member directory, two-way communication (eg electronic mail) and special features such as calendars, chat rooms, et cetera. Usability, covers a number of criteria including fast loading, text tags on graphics, visible navigation choices, compatibility with common browsers and navigation aids. The company's objectives with the site are also evaluated and can include an increase in revenue, decrease in expenses and an enhancement of the existing features of the company The following are features that need, but they are still relevant to all types of web sites and not necessarily electronic commerce sites. Broken Links A number of software packages are available to check the existence of broken links and construction pages such as InfoLink Link Checker 1.9 ( Testing of contact information and reply time What is the turnaround time for queries and the quality of the answers? Updates and maintenance Last date updated should be given and information should be recent and relevant. Competition's sites A look at the main competitors may give you an idea of the standard of your site. Srivihok [2000] explains in his article that the of electronic commerce is critical for th success of future systems. He developed a tool based on the task-technology fit model, which focuses on user of information systems. Srivihok proposes that more systematic techniques are needed to measure user of electronic commerce web sites. In his methodology he uses he used confirmatory methods to develop a hypothesised model to randoml collect data from electronic commerce users surfing the Internet. In the second step he uses exploratory methods to empirically evaluate the data and thirdly confirmatory methods will be us

10 to test the hypothesised model against new data collected. This is, however, still research in progress and is not necessarily that different from the use of empirical methods in HCI From traditional literature, conference proceedings and publications on the web, it seems as if the focus stays on what is good web site design and what constitutes a usable web site. The evaluatio of electronic commerce web sites seems to be considered of less importance. In a paper by Tilson et al where they evaluated four e-commerce sites, they discovered that there are a number of new issues, but that these issues can still be understood within the context of traditional HCI principle More research needs to be done on how we can use the HCI techniques, enhance them to make them usable for the of electronic commerce web sites. At the moment it seem as if electronic commerce developers think that if they teach good web design, no is necessary! References Dahl, A; Lesnick,L Internet Commerce, New Riders, Indianapolis. Dix, A; Finlay, J; Abowd, G; Beale, R Human-computer interaction (2 nd edition), Prentice Hall, London. Instone, Keith ( ) Kalakota, R; Whinston, AB Frontiers of Electronic Commerce, Addison Wesley, Reading. Preece, J Human-computer interaction, Addison Wesley, Wokingham. Rubin, T User interface design for computer systems, John Wiley and Sons, New York. Srivihok, A An Assessment Tool for Electronic Commerce: End user of web commerce sites. Available from the author. Tilson, R; Dong, J; Martin, S; Kieke, E Factors and principles affecting the usability of four e-commerce sites, Turban, E; Lee, J; King, D; Chung HM Electronic Commerce A managerial perspective, Prentice Hall, New Jersey. Whiteley, D E-Commerce: strategy, technologies and applications, McGraw-Hill, London.

11