INTRODUCTION USER POPULATION

Transcription

1 iscreen Usability Cognitive Walkthrough Report Christine Wania George Abraham INTRODUCTION Context and motivation The College of IST recently installed an interactive kiosk called iscreen, designed to serve as an information resource for student/visitors to the College of IST. The content is designed to answer some of the common questions students/visitors might have, for example: faculty office locations, building hours etc. The hardware comprises of a touch-screen capable 42 Plasma monitor, featuring a walk-up-and-use interface. The interface needs to support novice as well as repeat users in terms of content and accessibility. A user-based evaluation and an expertbased heuristic evaluation (HE) of the interface revealed issues that could hinder usability of the iscreen. In this report, we use the cognitive walkthrough (CW) technique (Wharton et al.) to investigate usability issues. Each of the methods mentioned address different aspects of usability. While userbased evaluations reveal usability issues as viewed by the user and helps in identifying usability issues of a more global nature (task-level), cognitive walkthrough enables one to investigate usability issues at a more granular level (action-level). In other words, the CW would compliment and at times help validate usability issues identified in the previous studies (refer Portfolio Pt1, Pt2). Using multiple approaches for studying usability would enable us to identify problems at various levels. Goal and findings Our goal is to study the interface using the CW technique to find usability problems that would generate recommendations/design implications for the (re)design of the information kiosk, the iscreen. The findings reveal that there are interface elements that do not communicate the associated actions successfully to the user. Although the kiosk menu is reasonably shallow (two effective levels), the progression through the levels is not devoid of problems. The majority of the problems encountered relate more to interface elements like icons and feedback. We also find the choice of information presentation/layout may lead to some confusion for the user in finding/processing the information requested. USER POPULATION Target users The current content is targeted at common queries students and visitors at the College of IST might have, like for e.g. Directory assistance, faculty office location, classroom location etc. The interface and content needs to support both novice and repeat users. It has been designed as a walk-up-and-use kind of interface and should require minimal or no training. User group tested and profile The undergraduate user group was selected for the purposes of the CW as they represent a large user-group within the school. The undergraduates are enrolled in information systems or library science program, and have reasonable exposure to computer based technology in the classroom. By this we mean, they are familiar with the appearance and behavior of buttons and the concept of hyperlinks. 1

2 iscreen Usability Cognitive Walkthrough Report Christine Wania George Abraham TASK SELECTION Tasks simulated and correct steps for each task are presented in following table 1.. Task Correct path selected 1. Find Mike Atwood s phone number Directory Faculty 2. Find the phone number for the Teaching Assistants Office 3. Find the hours of operation for the Computing Resource Center. Building Info Phone Numbers Building Info Hours of operation 4. Find when the fall quarter exams begin IST calendar-academic calendar 5. Find advising walk in hours Academic FAQ Undergrad FAQ 6. Find the location, day, and time INFO IS Mgmt is offered. Course schedule-fall Table 1: Selected tasks for the CW exercise and the correct paths. These tasks were selected based on information traditionally requested by the undergraduate student community. Our previous ethnographic study (Portfolio Pt.1) and interviews (Portfolio Pt.2) with the undergraduate students contributed greatly to the choice of tasks. It is important that the tasks simulated are aligned with the motivations/goals of undergraduate students if the CW approach has to succeed. KEY FINDING (PROBLEMS) In this section we present our findings based on the cognitive walkthroughs of the tasks listed in table 1. We have chosen to report findings that show a mismatch between how the system has been designed and how a simulated user would interpret the design, in other words problematic situations. We believe that reporting successful actions would not add much value to the objective of finding and recommending fixes for the usability problems. Each of the tasks was broken down to constituent actions/steps that would help in attaining the task goal. For each of the steps we ask three questions to target the specific issues that impeded the usability of the interface. These issues relate to visibility of actions (Q1), mapping of interface elements to actions (Q2), and feedback (Q3) provided to the user (right of wrong choice). For a detailed description of task related findings consult the CW data forms provided in appendix. Task 1. Find Mike Atwood s phone number Step1. Select Directory Feedback about right/wrong choice by the user is not available. The user still needs to make another decision at this point. The users are not presented with any information to help them make a decision between choosing faculty or staff. If the user does not know whether Mike Atwood is faculty or staff this may present a problem. 2

3 iscreen Usability Cognitive Walkthrough Report Christine Wania George Abraham Step2. Select Faulty The visibility of this action is incumbent on the user knowing if Mike Atwood is faculty or staff. The existing interaction aids do not help user make this choice. The extension numbers provided in the extension number format may cause problems in users judging the quality of choice. Task 2. Find the phone number for the Teaching Assistants Office Step1. Select building info The correct action is not evident based on the interface. The logical choice for users would be to look under directory and not building info. That being said, the users would correctly select directory based on what is evident to them. But, this would be the wrong selection for the purposes of the task. The appearance of the action users are trying to achieve would be appropriate, but would lead to the wrong information. Step2. Select telephone numbers The feedback regarding right or wrong choice for this step is not very evident. There is a good chance the user may not find the information sought because the word teaching assistant has been presented as an acronym T.A. on the interface. This may lead to confusions especially if some one is explicitly scanning for Teaching Assistants. The extension numbers provided in the extension number format may cause problems in users judging the quality of choice. Task 5. Find advising walk in hours Step1. Select Academic FAQ The correct action is not evident on the interface. There would be some confusion for the user regarding a choice, but may decide on FAQ by elimination. The appearance of the action does not provide any aid to the users regarding the choice. The information resulting from the action would lead to a set of links that refine the FAQ into graduate and under-graduate. So this neither informs the user regarding the appropriateness of the choice, nor presents additional information regarding what the user might find. Step2. Select undergraduate FAQ Although the information sought is available on the screen, it is a bit difficult to find. There is a lot of information presented as full sentences on the screen lending it a crowded appearance. This may make it difficult for the users to evaluate if they made a right or wrong choice. Task 6. Find the location, day, and time INFO IS Mgmt is offered. Step1. Select course schedule The interface does not help the users decide if they made a right/wrong choice. On selection users are presented with a blank screen with two buttons named fall courses and main. The task does not explicitly state the term for the course details. In this particular case there is no other option other than fall courses. 3

4 iscreen Usability Cognitive Walkthrough Report Christine Wania George Abraham Step2. Select fall Correct action and the fact if the users have made a right/wrong choice is not evident. The resulting page does not show anywhere on the screen if it is the fall course schedule. The users would be expected to remember their previous choice to confirm the resulting information. Step3. Select page If the course is not found on the page being currently viewed, the user has to use a non-standard link for progressing to the next page. n standard links may force the users to miss it. This highlights the problem that the action is not visible on the interface, and also, the user would fail to connect the action with the appearance. IMPLICATIONS FOR DESIGN 1. Improve the phone directory interface to facilitate browsing capabilities a. Facilitate last name based searching b. List important phone numbers as default page of the phone directory c. Provide instructions for dialing the extension and finding the office d. If necessary provide phone numbers both under building info and directory. e. Avoid use of acronyms and abbreviations as much as possible when presenting information. 2. Improve visibility and layout of F.A.Q section Instead of presenting the users with a choice of links, present both graduate and undergraduate FAQ in a tabular format. List only the questions, so that the users are able to choose what they need before being presented with the information. Ensure that the buttons and links communicate to the user the action it represents. 3. Enhance course schedule menu a. Either display the fall schedule by default b. Or, provide additional buttons for all the terms in the academic year. Our findings reveal that users do not realize when they are on the fall schedule courses. We suggest adding a title: Fall Schedule on the page so that users realize what they are viewing. This would also help them confirm the quality of their choice. In addition to this, the courses can be visually separated as 400 level, 500 level courses etc. The next page links should be standardized throughout the interface, and should communicate information that could be accessed by clicking on it. Like for e.g. Links could be shown as 400 Level courses 500 Level 600 level etc. instead of pg1 pg2 etc. APPENDIX Cognitive Walkthrough data forms for tasks listed in Table 1, Pg.2 4

5 Task 1 name _Find Mike Atwood s phone number Step number 1 Step name Select Directory Directory is a standard name for a place you would find phone numbers. If the user has touched the screen they know to touch the buttons to select things. The user still needs to make another decision at this point. The user is not presented with any information of help and they need to make a decision between choosing faculty or staff. If the user does not know whether Mike Atwood is faculty or staff this may present a problem. Step number 2 Step name Select Faculty If the user does not know whether Mike Atwood is faculty or staff this may present a problem. If the user does not know whether Mike Atwood is faculty or staff this may present a problem. The user will know they have made a wrong choice if they do not see Mike Atwood s name in the list or the user will know they made the right choice if they see Mike Atwood s name in the list. Also, it is hard to find names in the list.

6 Task 2 _Find the phone number for the Teaching Assistants Office Step number 1 Step name Select Building Info The logical choice for the user would be to look for this information in the directory. As mentioned in the previous question, users would correctly select directory based on what is evident to them. But, this would be the wrong selection for the purposes of the task. Users would become aware when they do not find the requisite information on the selected page, in other words, a wrong choice. Step number 2 Step name Select Phone Numbers If user successfully reaches this level on the menu, and step the page clearly displays a link for telephone numbers. Based on previous experience in reaching this menu level, the users would have become aware of the button appearance and its functionality. There is a good chance the user may not find the information sought because the word teaching assistant has been presented as an acronym T.A. on the interface. This may lead to confusions especially if some one is explicitly scanning for Teaching Assistants.

7 Task 3 _Find the hours of operation for the Computing Resource Center Step number 1 Step name Select Building Info Hours of operation are traditionally found under building info. So the user would not have any problem in choosing this option. Building information category is available explicitly as a button on the main page. The users would have not confusion regarding the functionality. The page refreshes to show additional links. One of the links on the new page is explicitly labeled Hours of operation. The label can confirm the users choice. Step number 2 Step name Select Hours of Operation A button labeled Hours of operation is available. Explorative learning would lead to knowledge regarding the functionality of the labeled buttons. The successive page displays the CRC operating hours as one of the three options, with the hours displayed in conventional format (HH:MM am-hh:mm pm)

8 Task 4_Find when the fall quarter exams begin Step number 1 Step name Select IST Calendar The IST calendar category is available on the main screen, and such kind of information is conventionally found under calendars. The label on the button reads IST calendar. On selecting the button, the next screen shows IST events and Academic calendar.. Step number 2 Step name Select Academic Calendar Only three options available, IST events, Academic calendar and main. Academic calendar would be the logical choice. Previous experience with the system would factor into the action, and the users would successfully click the appropriate Academic calendar button. The information requested is shown on the results screen.

9 Task 5 Find advising walk in hours Step number 1 Step name Select Academic FAQ There would be some confusion for the user regarding a choice, but may decide on FAQ by elimination. There is no aid to inform the user regarding a good choice. The information resulting from the action would lead to a set of links that refine the FAQ into graduate and under-graduate. So this neither informs the user regarding the appropriateness of the choice, nor presents additional information regarding what the user might find. Step number 2 Step name Select undergrad FAQ Based on the target user profile, the user would choose button appropriately labeled undergrad FAQs. The button functionality would have become evident to the user by now, and the presence of an appropriate label would support the users perception. Although the information sought is available on the screen, it is a bit difficult to find. There is a lot of information presented as full sentences on the screen lending it a crowded appearance.

10 Task 6 Find the location, day, and time INFO IS Mgmt is offered Step number 1 Step name Select Course Schedule The main screen has a button labeled course schedule, and the task motivation would lead the user to notice this. The labeled button is visibly located on the interface. The button like appearance would inform the user what to do. The target user for this walkthrough is an undergraduate user who would be familiar with the behavior of buttons. They are presented with a blank screen with two buttons named Fall courses and main. The task does not explicitly state the term for the course details. In this particular case there is no other option other than fall courses. Step number 2 Step name Select Fall Courses The task description is not school-term specific, and since there is only one relevant labeled button, the user would notice it. Only one option available and the button is labeled Fall courses. The resulting page does not show anywhere on the screen if it is the fall course schedule. The users would be expected to remember their previous choice to confirm the resulting information. Step number 3 Step name Select page until course is found If the course is not found on the page being currently viewed, the user has to use a nonstandard link for progressing to the next page. This could make the users miss the link.

11 n standard link may force the users to miss it. Q3: Will the users know if they have made the right or wrong choice? But, only after finding the course the users are looking for.