Electronic Gateway Functional Team Website Usability Working Group Usability Test September 2005 Queen s University Library website http://library.queensu.ca/ SUMMARY There were 11 testers, drawn from among Queen s Learning Commons Student Assistants. They ranged from under-grad to graduate student, from a variety of disciplines. Because respondents recorded their own observations, the comments were candid. There were some common themes: eliminate acronyms, avoid library jargon, improve access to journal article indexes, simplify pathways. At the same time, there were conflicting responses: for every respondent who said a particular task was difficult, another respondent said it was easy with the exception of finding articles about Canadian Tire and placing a campus book delivery request, both of which everyone found difficult. Based on the test results, we recommend the following changes be considered: 1. Change the QCAT link (on the Research Tools menu) to: Find books, etc. (in QCAT catalogue) 2. Change the wording of the link on the Research Tools menu to: Journal Article Indexes & Databases. Even though these words aren t strictly true (most databases index much more than just journal articles!), it seems to be important to get the word journal in the link so that students don t zero in on the Electronic Journals & Newspapers link farther down the menu, when they really should be selecting the Indexes & Databases link. 3. Alternatively, if the QCAT were changed to Find books, the Article Indexes & Databases link should be changed to Find articles. 4. Consider removing the Electronic Journals & Newspapers link altogether from the Research Tools menu. Now that QCAT comprehensively includes this material, is the E- Journals link doing more harm than good? 5. Have the database descriptions (in the DB of DBs) been thoroughly reviewed and revised? Most are still very long. We are mindful of the frequent survey comment: a lot of text on the screen, hard to read / zero in on critical information. 6. Change the Indexes & Databases main page to default to Search by Subject rather than Search by Title. Queen s Library Website Usability Study (Fall 2005) / 1
7. Add an entry on the Research Tools drop-down menu for style guides. Recommended terminology for link: Bibliography Style Guides. 8. Also change the text of the existing RefWorks link to RefWorks Bibliography Creator. This name matches the terminology used by the vendor on the RefWorks login screen and ties in with the previous recommendation. 9. Add a link on the QCAT Help menu to the RefWorks QCAT Export/Import Guide. DONE. 10. Implement, as soon as it is ready, the one-click export/import process for QCAT (Voyager) being developed by Guelph. 11. On the library website, change the link in the Library Services drop-down menu from Document Delivery/ILL to Document Delivery & Interlibrary Loan. DONE 12. Even better, change the link text to: Interlibrary Loan (assumption: no campus libraries are using the phrase Document Delivery on their signage, brochures, etc.?). In QCAT, change the two navigation bar buttons as follows: Place a Request Request Item in QCAT Document Delivery/ILL Request Item not in QCAT 13. Suggest to RACER folks that a Forgot your password? link be added to the RACER login screen, as is common on many commercial websites. The link would behave the same as the existing First time users link. 14. Add a separate menu item (under LIBRARY SERVICES) for: Campus Book Retrieval. 15. Ask the ILL Working Group to investigate the feasibility of allowing campus book retrieval requests to be accepted through RACER, in order to simplify the requesting service options currently presented to patrons. 16. Recommend to Access Services Team that the feasibility of offering online payment of library fines be investigated once again. 17. Add a link for Ask a general question to the Ask a Librarian page (http://library.queensu.ca/librequest/ref_index.htm). Queen s Library Website Usability Study (Fall 2005) / 2
TEST RESULTS Task 1: Find a Book Does Queen s University Library have the novel A complicated kindness by Miriam Toews? Which library is it at? Is it available or signed out? Easy (& less than 2 clicks) 2 Easy (but more than 2 clicks) 9 Fairly difficult 0 Unsuccessful 0 Most respondents said that this task was very easy. Interesting comments included: - the QCAT acronym is not necessarily a familiar or understandable one (particularly to a new student) - suggestion that link be called Find a book - unfortunate that system software can t cope with initial articles in title search string; could there not be a pop-up warning on the screen when a patron types in an initial article? 1. Consider changing the QCAT link (on the Research Tools menu) to: Find books, etc. in QCAT catalogue Task 2: Find a Database You need to look for journal articles in the subject area of business. How would you search for them? Easy (& less than 2 clicks) 2 Easy (but more than 2 clicks) 7 Fairly difficult 2 Unsuccessful 0 Interesting comments included: - would be more helpful if the word journal was included in the heading on the Research Tools menu (i.e. journal article indexes & databases instead of simply article indexes & databases ) - at least three respondents attempted to answer this question by selecting Electronic Journals & Newspapers instead of Article Indexes and Databases and, as result, found the question difficult to answer. Queen s Library Website Usability Study (Fall 2005) / 3
1. Change the wording of the link on the RESEARCH TOOLS menu to: Journal Article Indexes & Databases. Even though these words aren t strictly true (most databases index much more than just journal articles!), it seems to be important to get the word journal in the link so that students don t zero in on the Electronic Journals & Newspapers link when they really should be selecting the Indexes & Databases link. 2. Alternatively, if the QCAT were changed to Find books, the Article Indexes & Databases link should be changed to Find articles. 3. Consider removing the Electronic Journals & Newspapers link altogether from the RESEARCH TOOLS menu. Now that QCAT comprehensively includes this material, is the E-Journals link doing more harm than good? Task 3: Use an Abstracting and Indexing Database Still in the subject area of business, can you find information about the company Canadian Tire? Easy (& less than 2 clicks) 0 Easy (but more than 2 clicks) 1 Fairly difficult 5 Unsuccessful 5 We have a lot of work to do! Most respondents had great difficulty answering this question because they had no idea where to start. Several got wildly off-track by selecting QCAT or the subjects Corporate Financial & Stock Information ( the word information is ambiguous ) or Company Information. As in the previous question, some respondents tried to answer the question by starting from the Electronic Journals & Newspapers link, rather than the Article Indexes & Databases or Resources by Subject links. Additional comments: - links at left are often ignored, perhaps could stand out more - information is all stored in the left column bar; this is not obvious - database descriptions are confusing and unhelpful - should list by subject area first (rather than by title) as this is how users approach looking for articles 1. See items listed under Question # 2. Change the Indexes & Databases main page to default to Search by Subject rather than Search by Title. Queen s Library Website Usability Study (Fall 2005) / 4
3. Have the database descriptions (in the DB of DBs) been thoroughly reviewed and revised? Many are still very long. Task 4: Looking up a Style Guide You re writing a paper, and need to cite your references using MLA Style. Find some instructions and examples. Easy (& less than 2 clicks) 1 Easy (but more than 2 clicks) 8 Fairly difficult 1 Unsuccessful 1 Interestingly, the majority of respondents found the style guide by selecting Research Tools Quick Facts Online Style Guides (as opposed to choosing: Help Find How to Cite sources correctly). Either path leads to the same result but the route overwhelmingly chosen was to look under the Research Tools button for something appropriate. Additional comments: - Found nothing on the homepage; went to the Writing Centre. - Quick Facts Online is very useful but many students don t know it exists. 1. Add an entry on the RESEARCH TOOLS drop-down menu for style guides. Recommended terminology for link: Bibliography Style Guides. 2. Also change the text of the existing RefWorks link to RefWorks Bibliography Creator which matches the terminology used by the vendor on the RefWorks login screen and ties in with the previous recommendation. Task 5: Exporting Citations using RefWorks You want to know how to save book citations from QCAT into your RefWorks database. Can you find out how? Easy (& less than 2 clicks) 1 Easy (but more than 2 clicks) 3 Fairly difficult 6 Unsuccessful 1 Queen s Library Website Usability Study (Fall 2005) / 5
Fortunately, this process will be greatly streamlined in future, thanks to programming being done by the Web/digital initiatives librarian at Guelph. Additional comments: - We found what we were looking for in the end but it was confusing because you think that you are going to find the information you need under the What? Who / Where? links on the RefWorks guide page, but the information was not there. - RefWorks is a great tool and easy to use but it should be promoted more and its function more clearly explained. - Put all tutorials together under one link on the Quick Links menu. - I looked for information about how to do this under the Help button on QCAT but couldn t find anything. You should add some instructions there. 1. Add link to QCAT Export/Import Guide on the QCAT Help menu. DONE. 2. Implement, as soon as it is ready, the one-click export/import process being developed by Guelph. Task 6: Document Delivery You need to order a journal article because the library doesn t have the journal you need. Find the form to order the article. Easy (& less than 2 clicks) 1 Easy (but more than 2 clicks) 2 Fairly difficult 8 Unsuccessful 0 Two respondents comments say it all: Easy once acquainted with the service but otherwise difficult, and, Who is going to know what ILL means if they are a new undergraduate student? Acronyms are not helpful. Only three respondents selected the Request an Interlibrary Loan link from the Quick Links section on the library homepage. Instead, several chose Request Forms (from the Library Services menu) and then drilled their way down to interlibrary loan requesting. Additional comments: - Add pop-up menu to RACER to tell person what their login ID and Password are. Queen s Library Website Usability Study (Fall 2005) / 6
1. On the library website, change the link on the LIBRARY SERVICES drop-down menu from Document Delivery/ILL to Document Delivery & Interlibrary Loan. DONE 2. Even better, change the link text to simply: Interlibrary Loan (are all campus libraries using this terminology on their signage, brochures, etc.?). 3. In QCAT, change the two buttons as follows: Place a Request Request Item in QCAT Document Delivery/ILL Request Item not in QCAT 4. Suggest to RACER folks that a Forgot your password? link be added to the RACER login screen, as is common on many commercial websites. The link would behave the same as the existing First time users link. Task 7: Remote Fine Payment You are a part-time student, living more than 2 hours from the campus. You have overdue fines and want to pay them without going in to the library but don t know how to do this (by credit card? by cheque? and if so made out to whom and mailed to what address?) Easy (& less than 2 clicks) 1 Easy (but more than 2 clicks) 7 Fairly difficult 2 Unsuccessful 1 Wide diversity of comments, ranging from everything on this site is great, instructions are clear and easy to follow, to very straight-forward, to had to use Site Search to locate, to impossible to find. The fact that you have to print and fax a Visa/Master Card form online is extremely antiquated. Not many students have access to a fax machine. You should be able to deal with these things online. 1. Recommend to Access Services Team that the feasibility of offering online payment of library fines be investigated once again. Task 8: Book Retrieval Service How do you order a book to be delivered from the Education Library to the Stauffer Library? Easy (& less than 2 clicks) 0 Easy (but more than 2 clicks) 6 Fairly difficult 4 Unsuccessful 1 Queen s Library Website Usability Study (Fall 2005) / 7
All respondents found this task difficult and in fact, even though some respondents thought they had found the correct path (by connecting to RACER), ultimately this would not have resulted in a book being delivered from Education to Stauffer as RACER is configured to deny these on-campus requests! Very difficult to figure out where to start. We thought interlibrary loans would be the obvious place to start but it wasn t @ all. [This respondent ended up selecting All Requests ] I could find how to do requests in between schools but not requests between libraries within the same school. This task was not too hard as long as you know what ILL is. 1. Add a separate menu item (under LIBRARY SERVICES) for: Campus Book Retrieval. 2. Ask the ILL Working Group to investigate the feasibility of allowing campus book retrieval requests to be accepted through RACER, in order to simplify the requesting service options currently presented to patrons. Task 9: Web Proxy Help You re having trouble accessing library resources from home. There are instructions on the website but you can t get them to work. You need help and are looking for a technical expert to contact for help both name and phone number, or help form. Easy (& less than 2 clicks) 1 Easy (but more than 2 clicks) 7 Fairly difficult 2 Unsuccessful 1 Most respondents did not see the Connecting from Off-Campus link in the Quick Links section of the library homepage but, instead, went to the HELP button. Other comments: For someone who is not so great with computers, this is wonderful. The instructions are clear and the link is easy to find on the homepage. Don t use acronym IT. Many students won t know what it means. Queen s Library Website Usability Study (Fall 2005) / 8
Task 10: E-Mail a Librarian How would you write and ask for help with a specific research question you are struggling with? Easy (& less than 2 clicks) 6 Easy (but more than 2 clicks) 5 Fairly difficult 0 Unsuccessful 0 Definitely the easiest thing to do on this site although a Contact us link would be even better. One respondent got to the list of subject librarian specialists to contact but commented that at this point, a student may not know which library to pick. Another respondent described the selected path as: Help Ask a Librarian Contact library staff Paul Wiens but then commented I found his email address but there was nothing to indicate whether he was available to answer questions. 1. Add a link for Ask a general question to the Ask a Librarian page. What did you like about this site? From the many comments supplied: - There were a lot of resources organized under specific sections that made it easy to navigate. - If you read everything, most enquiries were answerable without much frustration. - If you know how to search for what you re looking for, then this website contains pretty much everything to answer your questions. There is tons of information. - I really like having access to the volume of resources available through the library site. I like the Quick Link section but would like to see Ask a Librarian added to this section. - Quick Facts are very useful. Tabs are generally straight-forward and helpful. - Good stuff on the site if only I knew how to use it better. Queen s Library Website Usability Study (Fall 2005) / 9
- Quick Facts Online is very useful. - Unfortunately, very very little. What did you not like about this site? - Too many acronyms. - Very busy a lot of text and small font and Way too much text on this site. - Confusing routes for some tasks. - RACER is hard to use if you re not acquainted with it. - Most students don t have any idea that all this information is accessible to them on this website. Advertise the accessibility of ALL this info. Put out flyers, bookmarks For more difficult items, rephrase them as a question, e.g. for interlibrary loans say Need to request an interlibrary loan? click here. - Journal articles online and offline are very difficult to find. Difficult to locate all journals on subject [that are] not covered in subject index. Very difficult to log in to resources such as economist.com. Difficult to find/differentiate journal databases such as Factiva and JSTOR. Very difficult to find specific journal articles, i.e. Gulf War. - Back-tracking is rather frustrating. - Too difficult to find things, easy to get lost, difficult to repeat your steps. Often the titles of links are confusing and do not accurately reflect the content of the link. If you had three things to tell the site manager, what would they be? - Simplify, simplify, simplify. - Updates on homepage are distracting. - More/better clear-cut instructions and descriptions. It should be the library for dummies - More tutorials for first-time users. Queen s Library Website Usability Study (Fall 2005) / 10
- It s too complicated for even me and I ve taken 6 workshops on how to use the website properly. [graduate student] - Eliminate or simplify the proxy set-up. - Consolidate/organize/explain journals, especially e-journals. - Make it easier to find/connect to newspaper & magazine databases. - Make signed out/signed in more prominent on searching for titles [in QCAT]. - Promote RefWorks as it is unknown to many students. - Clean up homepage so there isn t as much stuff bombarding you when you log on. Add more Quick Links. Have more headings, i.e. borrowing a book from another library -- having more headings means less searching. - Move the Quick Links over to the left side of the page and make the most popular links bolded. Queen s Library Website Usability Study (Fall 2005) / 11