How do you know what your users want?

Transcription

1 How do you know what your users want? An overview of usability studies Kristin Antelman NCSU Libraries January 23, 2009

2 this is a large topic, so I will focus on usability principles and methods as it relates to today s topic silos with a few examples from what I know and a few personal opinions

3 why usability testing library websites is hard the context is artificial 90%+ of our users start with Google they bring constraints (e.g., scholarly ) their information need may not even be known to them many don t understand library research

4 the importance of mental models what is a mental model? A mental model is a kind of internal symbol or representation of external reality, hypothesized to play a major role in cognition and decision-making. (wikipedia) or... the way that the user perceives that the system works

5 the importance of mental models, 2 why are they important? Users now have precise expectations for the behavior of search. Designs that invoke this mental model but work differently are confusing. (Jacob Nielsen) why are they important for us? our own mental models are different we have silos

6 how to learn about our users mental models? task analysis surveys, focus groups contextual inquiry participatory design usability testing

7 user research tools: task analysis: tasks vs. scenarios task find the book x by author y scenario find one book and one article on global warming why these work well and are popular

8 Search TRLN scenarios

9 a caution on task-based testing Task Difficulty: Old Interface Task Difficulty: New Interface Failed 23% Hard 22% Easy 43% Hard 7% Failed 22% Medium 11% Easy 60% Medium 12%

10 useful, easy and low tech usability methods wireframing: a tool for iterative development log analysis simulating the user experience

11

12 log analysis: an overview of user behavior

13 simple visualizations Subject: Topic LC Classification Format Library New Subject: Genre Author Subject: Region Language Subject: Era Availability Facet Use: July 06 - Feb 07 In order of use 0 20,000 40,000 60,000 80, , , , ,000 Requests

14 more complex visualizations

15 simulating the user experience appropriate for certain questions, e.g., relevancy how do we know if b is better than a? a quick and easy test: select topical queries from your search logs search them in each system subjectively judge relevancy of top n hits (using a ratio for result sets < n)

16 scoping, pt. 1: your work build on what s already known use common sense buy skills or capacity you don t have

17 build on what s already known

18 use common sense!

19 buy skills or capacity you don t have libraries without a usability program can still do usability testing benefits to contracting out user research

20 scoping, pt 2: your question content vs. tools: what are you testing? what is important in a complex, siloed information environment? recoverability engagement/flow/scent context and integrated help emotional response

21 framing what we are trying to do with our silos our approach will be different if we are: trying to get rid of silos denying them embracing them at least do logically seamed silos well... from that point, assess your options

22 is there an emerging best practice for silos?

23 we can t forget we re in a rapidly shifting landscape GoogleScholar / metasearch? WorldCat Local / the local catalog? GoogleBooks / metadata? etc.!

24 an exercise strategic goal: develop an understanding of how students use your current site (their "mental models") that will help you improve your site and also prepare for conversations with vendors about new search tools usability test objective: for the discovery "block" on the website, identify tabs and their labels

25 an exercise context: you are convening focus groups of undergrads your current website is the one on the handout working from one (or more) of the scenarios listed, develop questions and discussion topics for the focus group that will help with the test objective