Cengage Learning Opposing Viewpoints In Context Usability Test Hao Zeng, Mikko Tuomela, Josh Keyes, Travis LaFleur SI 622 Group 3 April 14, 2011
Cengage Learning Publisher of print and digital information services for academic, professional and library markets Delivers customized learning solutions for colleges, universities, professors, students, and libraries
Opposing Viewpoints In Context (OVIC) OVIC is an online resource covering today s hottest social issues It helps students research, analyze, and organize broad variety of data Used for conducting research, writing assignments, debates and more
Test Recruiting Sent email to SI Informatics email list Gave general information about study along with how much time it would take Promised them a $25 Amazon gift card after their participation We aimed for 5-7 subjects Scheduled 7, 2 canceled so we tested 5 subjects
Usability Test Tasks 4 Tasks 15 minutes Client: focus on search and multimedia content + Try to simulate normal operations that a general user will usually do in the system
Usability Test Tasks 1. Find two articles about nuclear power, one in favor and one opposed. Translate the second article into French and send it to your French friend (hao.hci@gmail.com). 2. Find 2 radio broadcasts about the 2011 earthquake and tsunami in Japan, save them to your personal favorites in the system, and then start playing one of them. 3. Find a magazine article about unemployment and download the article s APA citation. 4. Find an article related to stem cell research from a peer reviewed journal that was published in 2005.
Test Logging Logging sheet for each task o From 2 to 4 subtasks Completed? Y / N o General questions Surprised? Y / N Stuck? Y / N Confused? Y / N o Space for comments and notes To be filled either by pen or by computer (Word document)
Moderation & Debriefing Introduction script Sign consent form Pre-test questionnaire o Background, demographics, experience Begin tasks Encourage talk aloud Post-test questionnaire o Reflect on experience o Likes, dislikes, improvements Ask for questions Thank and dismiss
Recording
Location and Setup 1150B/1150C at North Quad 3-4 researchers present Audio recording by computer
Usability Test Findings and Recommendations
Usability Test Findings: #1 The Listen appears on top of the content of every document for people with visual impairment; But it causes significant confusion for general users.
Usability Test Findings: #1
Usability Test Findings: #1
Usability Test Recommendations: #1 Recommendation: Rename to "Read the Screen" and move to Tools section.
Usability Test Findings: #2 Pro and Con Indicators Are Needed In Search Results System is designed to give users arguments from "both sides" of an issue Sides of articles aren't readily identifiable in search results
Usability Testing Recommendation #2 Implement short summary to let user know if articles is "Pro" or "Con" for the subject that they are searching about "Pro" or "Con" summary could be anywhere from one sentence up to a short paragraph. Summary text could be placed to left of search results, just make column with results in it a bit skinnier (which would be ok)
Usability Testing Recommendation #2
Usability Test Findings: #3 Basic Search is used often, but is not fully satisfactory.
Usability Test Findings: #3 Does not handle Google-style queries "stem cell research 2005" "Japan tsunami audio" Recommendation: improve search algorithm to accept more parameters (not just keyword)
Usability Test Findings: #3 Recommendation: add filtering options from advanced search into results page for basic search
Usability Test Findings: #4 Finding 4: Options in Advanced Search Are Ambiguous Users were confused about what "Content Level" means Because they don't know what content level means, they can't make sense of the options that they can select It frustrated users when they weren't able to tailor the advanced search options clearly
Usability Test Recommendations: #4 Cengage should name "Content Level" something that is more descriptive of what the feature actually filters Cengage could add a help section that describes all of the features that are offered in the Advanced Search
Thank you! Questions?