Overview In this phase of the project you test and revise your client site using three different testing methods: quality assurance testing (done individually), user testing, and heuristic evaluation. Each method will isolate a unique set of design considerations. The project submissions for this phase include an evaluation of the test results and a set of site revisions suggested by each method. The final test-bed site is uploaded to the server and the final site design documentation is delivered to the client. Quality Assurance (QA) Testing Compile together the individual QA Testing Checklists for your team site. User testing observes actual users interacting with the web site. Users are asked to perform specific tasks while usability experts observe and take note of their actions. The objective of this testing is to discover the mistakes users make when using the interface and to observe their reactions during the session. Special attention should be given to results that repeat between multiple test participants. Create Task List Develop a user task list containing at least four (4) distinct tasks. Tasks are based on the site objectives and user goals identified for your site. Below is one process for developing your tasks into testable scenarios: 1. Select a user goal. This can be determined from the original site objectives and user profiles (see site specification). 2. Write a clear, task description, base on the user s goals, not on the website navigation labels. 3. Formulate a method for accomplishing that goal via the web site. (You may need to add content to the site it doesn t have to be real, just workable) 4. Perform a trial walk-through to establish a target completion time based on reasonable expectations for user activity. 5. Record the number of clicks and time necessary to complete the task. Each class member is encouraged to invite three (3) test participants to attend the testing session at its appointed time. A testing environment will be set up in the classroom. Participants are greeted as they enter and directed to a computer test station: the participant seated at a computer with two chairs placed on either side. A moderator and data logger from the design team will sit on either side of the participant. Test team will describe the testing procedure to the participant via an orientation script. The moderator then reads from the team s task list, one task at a time. The participant attempts to complete each task and should be asked to think out-loud as s/he works. The data logger will track the results, including the actual time to complete the task, errors, number of clicks, comments by the participant, and any observations made of the participant s behavior. The data logger and moderator should refrain from answering questions posed by the participant in order to get a more accurate assessment of the user behavior outside the test environment. The test is repeated with 5 to 8 participants.
Each participant is then asked to complete a post-test questionnaire to gain their personal impressions regarding the usability and aesthetics of the site. Finally, participants are thanked for their efforts at the completion of the test. Create a Usability Test Results report The team should collect and calculate the average time to complete each task, the average number of clicks, the percentage of participants who finished each task successfully, and the user preference data summary. The team should also make note of recurrent issues arising from the test observation or the participant questionnaire. Compile the results into the Usability Test Results report (see sample document.) Heuristic Critiques For this test, members from a different team (the test team) will act as usability experts to analyze your site according to the website design heuristic developed earlier in the class. 1) The test team will use the effective critique methodology in order to analyze the site for clarity, consistency and navigation in relationship to the site objectives. Each item is ranked numerically and comments are strongly encouraged. Evaluators should explain why they feel something is not working while referencing the heuristic in question. Evaluators should also include rationale for positive aspects of the design. 2) The test team reviews their results as a group and selects one representative to present their results to the design team. The test team representative presents and discusses the heuristic critiques from the test team in a debriefing session with the design team. 3) The design team then records the feedback in a suggested revisions summary included in the final site documentation. Suggested Revision Summary The results for each of the testing modes (QA,, and Heuristic Critiques) are collected, documented, and shared as part of the client presentation. Only minor revisions need to be made to the site following the user testing. Most issues are just noted for revision in the Suggested Revision Summary document, rather than corrected. Site Design Documentation Packet The Site Design Documentation Packet is compiled as the final deliverable to the client of the work that your team has accomplished this quarter. The documentation packet should be a clear, cohesive, professional-quality report of the development phase results: The revised scope documents: Strategic Brief, Comparables, User Profiles, and Navigational Flowchart (see Phase 1.) The design composites (as layered Photoshop document) and the revised Style Guide (see Phase 2.) The Site hierarchy chart showing the file organization of the site and the folder containing the HTML templates (see Phase 3.) The usability test documents from this phase.
One copy of the Site Documentation packet should be posted in a folder called documentation on your team website and another copy prepared to give to the client at your final presentation. The client s copy should include a cover letter summarizing the project results and thanking the client for the opportunity to work on the project and a table of contents outlining the documentation contents. All documents should be delivered in electronic format and summary documents as hardcopy. When formatting the documentation, pay attention to making the packet useful for the client, intern, or developer who will continue work on the site this person may be you! Final Submissions 1) Website Testing Report that includes all user test material and summarizes the results of each of the three different test forms: Q/A,, and Heuristic Critiques. a) All the results from the Q/A Checklist (completed individually) b) Complete documentation of the in-class user testing with a Usability Test Results Document that includes the task list, user profiles, test methodology, and the Performance and Preference Data Summary (see Test Result Sample document) c) Heuristic critique sheets from the test team s evaluation d) A summary of the results from each of the three different test formats: Q/A, Heuristic, and. Present these results with suggested revisions as a Summary of Suggested Revisions Table (see below) 2) The final, validated test-bed HTML/CSS posted to your team portfolio with a link called Final Site 3) The (due at final presentation to client) including: a) Cover letter summarizing the project results b) Table of contents outlining the documentation contents c) Electronic format (CD) of all project documents d) Hardcopy of summary documents: Strategic Brief, Navigation Model, Design Mockup, HTML/CSS Test-bed, Style Guide, Test Results Summary, and Suggested Revisions
Sample Summary of Suggested Revisions Error Description On the Programs & Degrees page the language at the AA in Integrated Studies link isn t right. On Related Services page OPL should be OLC. In older IE browser, some of the CSS was ignored, but the site worked and was readable. The pages show properly at standard resolution. Changing font size to the largest possible causes position shifting Many users had a difficult time reading the font for the navigation buttons Most users could not locate the client contact form. Testing Phase where Error Discovered Revision Recommendati on Revised No Revision needed Background image in the sidebar make the text difficult to read. Heuristic Colors for the links and visited links were too similar. Heuristic Revised Team Comments Changed OPC to OLC. Changed a few typos on some of the pages. Changed E-Porfolio to eportfolio. Changed SOAR Orientation to Orientation (SOAR). This browser is being phased out shortly. Update position model. Change would make the site more usable but not critical. Increase the navigation font to 14px. Revisions strongly recommended before the site goes live. Add a Contact Now link in the sub navigation Reduce the opacity of the image or change the color of the sidebar text. Consider using a solid color for background. Tried using #37A217 for the visited link color
BIT 113: User Interface Design Phase 4: Testing - Assessment Sheet Team: Score: 16/16 Score Effort (1 point) Developing (2 points) Adequate (3 points) Well Developed (4 points) Report Complete 4 report is incomplete, missing task list, performance, or preference data. report contains some tasks and partially summarized results report shows complete tasks and data, some results summarized. report provides complete tasks and data; all results summarized clearly. Suggested Revisions Summary complete 4 Revision summary includes only a few types of data and minor revisions Summary includes most test data (Q/A, User, Heuristic) but suggests few or poorly worded revisions Revisions summary includes all test results (Q/A, User, Heuristic) and suggests a variety of revision. Suggested Revisions Summary fully complete, and clearly written. Data used to suggest effective changes. HTML/CSS Test-bed Site Complete and Valid 4 HTML/CSS Test-bed site incomplete, invalid, or many errors. HTML/CSS Test-bed site validates and is mostly complete Valid HTML/CSS Test-bed site complete; site goals incorporated into the design Valid HTML/CSS Test-bed site complete; site goals and design reads clearly incorporated into the design Documentation Packet appears professionalquality 4 Site Documentation Packet incomplete or lacks cover letter mostly complete; missing electronic or hardcopy format mostly complete; contents could more clearly support future work on the site fully complete and effectively presents material for client and future developers. Project Lead: Casey Cascadia Score: 12/12 4 Many elements are Some elements are Most elements are complete All elements are complete Completeness incomplete incomplete Consistency 4 Many elements are inconsistent Some elements are consistent Most elements are consistent with sample submissions. All elements are consistent with sample submissions. Timeliness 4 Many elements submitted late Some elements submitted late Most elements submitted ontime All elements submitted ontime