Heuristic Evaluation Project. Course Agent

Size: px

Start display at page:

Download "Heuristic Evaluation Project. Course Agent"

Blaise Clark
5 years ago
Views:

1 Heuristic Evaluation Project Course Agent ( Jeremy Dennis Ken Garlitz Ed Hayden

2 Heuristic Evalution Summary No. HE1 HE2 HE3 HE4 Message instructing users new to the system what to do for login Visualize button produces errors Career goals are listed for users Sub menu navigation is marked X Speaks the user's language Prevent errors X Minimize user memory load Simple and natural Dialog Be Consistent Ease of access Feedback Good Feature X X X X X X X Problem HE5 Can't plan into the future X X HE6 Lists are out of date X X HE7 Color coding does not work X X HE8 Missing course information X X HE9 Inconsistent with other SIS lists X X X HE10 Missing courses in catalog/inconsistent catalog X X X HE11 Workload indicator X X X HE12 Registration page does not bring up future courses X X HE13 User profile contains incoreect information X X X HE14 Job Recommender link is broken X X HE15 Help button goes directly to the appropriate section of the documentation X X X HE16 Differing contact information X X HE17 Control Panel X X HE18 Profile X X HE19 Courses taken and evaluation X X X HE20 Career interests X X HE21 Study plan X X HE22 Advisor X X X

3 HE1 - Good Feature Message instructing users new to the system what do for login. Heuristic: Speak the user s language On the login page there is a message that clearly tells you what to do if you do not have a login. A link is provided to assist the user in contacting the required parties. It is a good practice to help the user get into the system. Benefit The user is concisely directed what to do in order to obtain a login. Trade-offs A system admin must the requesting user. When I tested this, the admin never ed me back. HE16: Differing contact information. Ed Hayden

HE2 - Problem Visualize button produces errors Heuristic: Prevent errors Pressing the Visualize button on any screen creates a window that hangs for a minute or more then errors out.

4 HE2 - Problem Visualize button produces errors Heuristic: Prevent errors Pressing the Visualize button on any screen creates a window that hangs for a minute or more then errors out. It is not even apparent what the visualize button is for. The button needs to produce something or immediately return an error. Users are waiting and nothing happens. Users quickly get frustrated with the system. It does not appear to affect the rest of (or any known) functionality. By itself, the error is severe (nothing happens) for the system as a whole, it appears minor. Possible Solution Remove the button or reduce the time of the time out. Anything that prevents the user from waiting. Ed Hayden

HE3 Good Feature Career goals are listed for users Heuristic: Speak the user s language, Minimize user memory load A fairly comprehensive list of Career Goals is available to the user.

5 HE3 Good Feature Career goals are listed for users Heuristic: Speak the user s language, Minimize user memory load A fairly comprehensive list of Career Goals is available to the user. It is easy enough to search through the goals and add them as needed. Benefit The user does not have to remember a list of goals. There is no risk of error associated with entering a goal the system may not recognize because the list is controlled. Trade-offs There may be more specific goals that are not available to the user that the user would like to enter. Additionally, an overwhelming number of goals may confuse the user as to what they should select. HE20: Career Interests Ed Hayden

HE4 Good Feature Sub menu navigation is marked Heuristic: Speak the user s language, Minimize user memory load When in the Control Panel, the sub-menu items that represent what current page the user

6 HE4 Good Feature Sub menu navigation is marked Heuristic: Speak the user s language, Minimize user memory load When in the Control Panel, the sub-menu items that represent what current page the user is on becomes a larger font. If the user were to select Profile, the user would be brought to that page and the sub menu Profile would appear as being a larger font than the other sub menu items. Benefit The user knows what page they are currently using. Trade-offs The font isn t quite big enough to be easily noticed. This keeps the overall feel consistent but perhaps making the sub menu bold or making the other sub menus that represent where the user could go Ed Hayden

7 HE5 Problem Can t plan into the future Heuristic: Simple and natural dialog The user is led to believe that they can plan into the future. That is one of the primary reasons the system exists; to let users plan their courses. There are courses in the list however. This not a programming issue or design but a system that is touted to help plan the courses should have at least tentative scheduling available. This is severe to someone trying to plan courses. If all the system can do is present current and prior classes, then it is doing nothing more than replicating systems that already accomplish this. Possible Solution This is an error that requires more of a philosophical change. Outward semesters should be populated but requires planning at a level that may not have been previously or regularly done by the institution. If the lists are not to be populated, remove the empty semesters from the list of options the users can choose from. HE6: Lists are out of date HE12: Registration page does not bring up future courses Ed Hayden

8 HE6 Problem Lists Are Out Of Date Heuristic: Simple and natural dialog As can be seen in this example, course listings are out of date. Summer classes, which have been tentatively posted on the SIS web page for some time, do not appear in course agent. This is severe. It does not have the information for students near term planning. Possible Solutions Populate the lists or remove empty semesters from the users options. HE5: Can t Plan Into The Future HE12: Registration page does not bring up future courses Ed Hayden

9 HE7 Problem Color coding does not work Heuristic: Simple and natural dialog The system leads one to believe that the user will be shown classes they have taken. The taken classes will be marked with color coding. In the example above, the tester has taken course 2510 and is currently taking None of this is indicated via the interface in the manner suggested at the top of the web page. Moderate severity. Most users will know what classes they have taken. As long as the information is available in the system for planning functions, the error is more of an inconvenience. If, however, the system has no record of a user s current and previous course work and this lack of data is detrimental to the planning functions, then this error becomes more severe. Possible Solutions Remove the dialog telling the user about how classes will appear if they have already been taken or fix the functionality. HE8: Missing course information HE9: Inconsistent with other SIS lists HE10: Missing courses in catalog/inconsistent catalog HE11: Workload indicator Ed Hayden

10 HE8 Problem Missing course information Heuristic: Simple and natural dialog It appears that information pertaining to the course has not been entered. Damages credibility of the system. Missing titles may not be severe but user perception of the system is damaged. Do I trust a course planning system that has basic incomplete data? Possible Solution Audit fields or refresh them periodically to ensure that information is up to date. HE7: Color coding does not work HE9: Inconsistent with other SIS lists HE10: Missing courses in catalog/inconsistent catalog HE11: Workload indicator Ed Hayden

11 HE9 Problem Inconsistent with other SIS lists Heuristic: Simple and natural dialog, Be consistent The SIS website is a primary source for most students in regards to class information. The information on that site should be mirrored by course agent. The lack of consistency damages the credibility of the system. Not very severe. Most students are cognizant about what they need to take. Possible Solutions Keep the data up to date and make sure it is consistent with other published data. HE7: Color coding does not work HE8: Missing course information HE10: Missing courses in catalog/inconsistent catalog HE11: Workload indicator Ed Hayden

12 HE10 Problem Missing courses in catalog/inconsistent catalog Heuristic: Simple and natural dialog, Be consistent There are courses missing entirely from the list of available courses. The Data Structures course could not be found anywhere on this interface. Severe. Can plan a course if it is not available Possible Solution Keeping the lists up to date. There needs to be a reconciliation between other published sources and Course Agent HE7: Color coding does not work HE8: Missing course information HE9: Inconsistent with other SIS lists HE11: Workload indicator Ed Hayden

HE11 Good Feature Workload indicator Heuristic: Simple and natural dialog, Speaks the user s language This provides a quick look at what the user can expect from the course.

13 HE11 Good Feature Workload indicator Heuristic: Simple and natural dialog, Speaks the user s language This provides a quick look at what the user can expect from the course. Benefit Users can plan their courses by workload, i.e. taking something intense and offsetting with something that is not as demanding. Trade-offs An apparent scale could be beneficial. Is this two out 3, 5, etc. HE7: Color coding does not work HE8: Missing course information HE9: Inconsistent with other SIS lists HE10: Missing courses in catalog/inconsistent catalog Ed Hayden

HE12 - Problem Registration page does not bring up future courses Heuristic: Simple and natural dialog The Register page does not have course information past the current semester. Major severity.

14 HE12 - Problem Registration page does not bring up future courses Heuristic: Simple and natural dialog The Register page does not have course information past the current semester. Major severity. The system is designed for planning future courses Possible Solution the courses for future terms should be entered as soon as they re available. An error message should be displayed for terms with no course information. HE5: Can t Plan Into The Future HE6: Lists are out of date Jeremy Dennis

15 HE13 - Problem Username differs from PITTNet ID Heuristic: Prevent Errors, Minimize user memory load, Speak the user s language The username for the system is different than the PITTNet ID, which means the user must remember an additional login. User login information should be tied to a central directory server. Moderate severity, though storing user passwords on this system can be a security problem. Possible Solution Using a directory server will allow the user to use existing PITTNet username and password, reducing the number of passwords user has to remember. Authenticating against another database will also remove liability of Course Agent system holding user passwords. Additional information could also be imported via the directory including previous courses. HE18: Profile Jeremy Dennis

16 HE14 - Problem Job Recommender link is broken Heuristic: Prevent Errors Clicking Job Recommender takes you to which doesn t load. Moderate. The link appears to take the user to a different system, so it is unlikely that the Course Agent admins have any control over the external system. Possible Solution Remove the link to the Job Recommender. Jeremy Dennis

HE15 - Good Feature Help button goes directly to the appropriate section of the documentation Heuristic: Prevent Errors, Speak the user s language The user was in the Career Scope page and the help

17 HE15 - Good Feature Help button goes directly to the appropriate section of the documentation Heuristic: Prevent Errors, Speak the user s language The user was in the Career Scope page and the help loaded the Career Scope section of the documentation. Loads the documentation in a new window at the section for the current page the user was on. Benefit Loading the documentation to the current section prevents the user from having to search the for the answers Trade-offs The help section opens a new page, taking the focus away from the main system. A smaller pop-up window would allow the help to display while still showing the main system on the screen. Jeremy Dennis

HE16 - Problem Differing contact information. Heuristic: Be consistent Login screen Help page The link on the login page links to cagent@mail.sis.pitt.

18 HE16 - Problem Differing contact information. Heuristic: Be consistent Login screen Help page The link on the login page links to cagent@mail.sis.pitt.edu and the on the welcome page and in the help links to rosta@cs.pitt.edu. Moderate severity. Though multiple s from our team to the cagent address have gone unanswered. Possible Solution Make sure all contacts are the same address. HE1: Message instructing users new to the system what do for login. Jeremy Dennis

19 HE17 - Good Feature Control Panel Easy reference point for user to find basic information Heuristic: Ease of access The Control Panel provides an easy way for the user to access their Profile, Taken Courses and Evaluation, Career Interests, Study Plan and Advisor. Users will often refer to features such as their system ID, password, courses taken while in the system. Benefit The control panel provides an easy way to access this information from anywhere the user might be within the system. Trade-offs None apparent,as the functionality is all in the sub-categories HE18: Profile HE19: Taken courses and evaluation HE20: Career Interests HE21: Study Plan HE22: Advisor Ken Garlitz

20 HE18 - Good feature - Profile Profile allows access to various categories which the user would need to see at different times Heuristic: Ease of access The PROFILE function allows the user to see their system ID, change their password, see who their advisor is, all in one screen. It even allows them to see their address,the program and degree they are enrolled in, the term they started in, and provides a way to link to a photo of themselves. Benefit Users may easily see their system ID and change their password. Also, can update their address and photo link. Trade-offs This feature will not help the user recover their password, only change it. HE13: Username differs from PITTNet ID HE17: This is a sub-category of the Control Panel. HE19: Taken courses and evaluation HE20: Career Interests HE21: Study Plan HE22: Advisor Ken Garlitz

HE19 - Good feature Courses Taken and Evaluation Courses Taken and Evaluation allows the user to add courses which they have previously taken and

Heuristic: Feedback, speak the user s language Courses Taken and Evaluation allows the user to add courses which they have previously taken and

21 HE19 - Good feature Courses Taken and Evaluation Courses Taken and Evaluation allows the user to add courses which they have previously taken and then give a personal rating to them. Heuristic: Feedback, speak the user s language Courses Taken and Evaluation allows the user to add courses which they have previously taken and then give a personal rating to them. This provides feedback to the user by color coding the courses which they HAVE provided ratings for and no coloring for ones which they have not. It even shows courses which have only partial evaluations done on them..

22 Benefit Users may easily see their courses taken and what ones they have evaluated and what ones still need to be evaluated. Trade-offs This feature does not automatically create the course list, it is entirely dependent on the user creating the list and providing the feedback. HE17: This is a sub-category of the Control Panel. HE18: Profile HE20: Career Interests HE21: Study Plan HE22: Advisor Ken Garlitz

23 HE20 - Problem Career Interests Career Interests should allow user to select career goals and then be provided feedback on related jobs which would fall under their selections. Heuristic: Feedback The career interests section SHOULD allow user to add careers they have in mind to the right side of the page and then see related jobs when they press the submit button below. However, this functionality does not work on this page. Large. This is an entire page of the site which does not function properly. Possible solution Either remove this page altogether or provide functionality as it was intended to some other server which could return results based in the criteria the user selected. HE3: Career goals are listed for user HE17: This is a sub-category of the Control Panel. HE18: Profile HE19: Taken courses and evaluation HE21: Study Plan HE22: Advisor Ken Garlitz

HE21 - Problem Study Plan Career Interests should allow user to select career goals and then be provided feedback on related jobs which would fall under their selections.

workload and career interests relevant. The page, however, does not return any results at all. Large. This is an entire page of the site which does not function properly.

24 HE21 - Problem Study Plan Career Interests should allow user to select career goals and then be provided feedback on related jobs which would fall under their selections. Heuristic: Feedback The study plan section should allow users to enter courses which they plan to take and then see a page returned to them which shows those courses overall ratings, number of votes, workload and career interests relevant. The page, however, does not return any results at all. Large. This is an entire page of the site which does not function properly. Possible solution Either remove this page altogether or provide functionality as it was intended to some other server which could return results based in the criteria the user selected. HE17: This is a sub-category of the Control Panel. HE18: Profile HE19: Taken courses and evaluation HE20: Career Interests HE22: Advisor Ken Garlitz

25 HE22 - Good feature Advisor Advisor provides information on the student s advisor. Heuristic: Feedback This section provides a simple to access and easy to locate page which provided links to the student s advisor s and webpage. Benefit Very useful to have a single place where one can find advisor and website information. Trade offs None noted. HE17: This is a sub-category of the Control Panel. HE18: Profile HE19: Taken courses and evaluation HE20: Career Interests HE21: Study Plan Ken Garlitz

Heuristic Evaluation Project

INFSCI 2470: Interactive System Design Heuristic Evaluation Project Evaluated System: Course Agent http://wwwsispittedu/~cagent Group Members Abdul Raqeeb Abdul Azeez Arash Farsi Sriranjani Mandayam Denis