Heuristic Evaluation Collections on Beta on Android device
Overall Impressions no glaring and few overall issues that prevent users from using/navigating collections there are 3 bugs that need to be fixed asap, though however, collections pages, interactions, and workflows are rife with opportunities for users to experience the following: contextual confusion lack of consistency layout, icons used, or other interaction elements that may not follow users mental models performing inadvertent actions
Guide Scoring for Severity Flows tested high = Show stopper. User can not accomplish task because it takes too long to figure out, or is non-functioning. No work-around. logged out user creates a collection logged in user adds a collection logged in user adds to existing collection from article logged in user finds collection they have created logged in user adds to existing collection from the collection page logged in user adds description to collection logged in user shares collection logged out user follows shared link and finds collection (stricken flows not available) medium = Takes a while, or is hard to find or learn how to do, but can still accomplish the task with a work around. Some users may give up. low = Irritating or annoying though learnable. none = no usability concerns. other = Observations that don t impact usability, but call out consistency, language, and other considerations.
High Severity - Bugs No high severity issues with workflows, yay!
High Severity - Bug Watchlist landing page (Collections) if watchlist list is empty, user is prompted to add by clicking edit there is no edit button, to add pages (or to edit existing pages, if watchlist is populated)
High Severity - Recommendation Add an edit button.
High Severity - Bug Watchlist and non-watchlist landing pages (Collections) users are supposed to be able to share collections via a shareable link a. there is no method to share on this page
High Severity - Recommendation Add a share button/functionality.
High Severity - Bug Non-watchlist edit page Delete button doesn t work
High Severity - Recommendation Make sure the delete button does what it s supposed to (surface warning dialog and delete collection when requested).
Medium Severity - Issues
Medium Severity - Issue Add to existing collection action When performing more than one interaction with star/collections menu, the pop-up from the previous action briefly blocks the ability to interact with text lower on the window. This can potentially lead to accidental unintended actions and/or cause confusion.
Medium Severity - Recommendation Consider an (x) for the pop-up so that they can be manually removed when desired.
Medium Severity - Issue Add to existing collection action pages are now starred (in a green color) only when they ve been added to watchlist users may not understand this and add pages to new or existing collections, but go back to the page and think they hadn t because they didn t add it to the watchlist collection specifically and the star is not green-activated
Medium Severity - Recommendation Uncertain what would be the best overall solution at this point, but the green star, green checkbox, + icons etc. all in my opinion should be re-aligned and made more consistent. Re: this particular issue, I recommend starring an article when it has been added to any collection a user owns, not just specifically the watchlist collection, as an indicator that it has indeed been activated in terms of collections.
Low Severity - Issues
Low Severity - Issue Normal navigation to pages on beta pop-up about star/collections will continue to occur on all pages visited Tap elsewhere on page dismisses pop-up on that page navigating to another page shows pop-up again
Low Severity - Recommendation Is it accepted/understood that if a user taps off of a pop-up, that it continues to appear upon further navigation? It seems that a dismissive action like tapping off the pop-up (in addition to tapping no thanks, which is more directly dismissive) should both dismiss pop-ups during further navigation.
Low Severity - Issue Star/Collections Menu if user scrolls down the star/collections menu, when scrolling back up in the menu (dragging downward), user may inadvertently reload the page
Low Severity - Recommendation If there is a way to separate the star menu from the typical browser actions to minimize this potential inadvertent action, that solution would be ideal.
Low Severity - Issue Add to new public collection no contextual understanding of what a new public collection is. is there a private collection? am i creating something? where exactly am i adding this? am i somehow doing both? what happened to my watchlist? is it still somewhere? is it mine, but publicly viewable, or will it be a collection anyone can view and add to?
Low Severity - Issue Enter a new collection and Create button (Add to new public collection) keyboard can hide the field on some devices box for text entry doesn t always load fully, and it looks like a [] next to the create button
Low Severity - Recommendation Minimizing the white space on the top section of the star menu may help with this issue. If this is not possible, it would be helpful to jump the user to that field, where the field is then positioned immediately above the keyboard fold. Make sure the box for text entry always loads correctly/fully.
Low Severity - Issue After creating a new collection (Add to new public collection) tapping back or save from edit collection page goes back to the wikipedia page. however, the star is not activated on the page and it doesn t look like anything was done, even though it is listed in the collections with a green checkbox also, both actions (back/save) do the same thing. this is potentially confusing.
Low Severity - Recommendation I believe the star- activation has since been changed to the following condition, that it is only activated-looking if the article is added to watchlist collection. Again, would recommend that article is green-starred when it has been added to any collection a user owns.
Low Severity - Issue After creating a new collection (Add to new public collection) even though users are being directed to the new collection (blocked by volume in screenshot), it does not necessarily eliminate contextual confusion + icon for collections and star for watchlist in the sidebar, star on article page, watchlist within collections
Low Severity - Issue Add to existing collection no contextual understanding of what an existing collection is. why is watchlist under existing collection? if there are existing collections, why is watchlist the only one? are these existing collections in wikipedia or are they just mine?
Low Severity - Issue Add to existing collection watchlist entry is below the fold on some devices (Abbey s iphone exhibits this issue) could be larger typeface setting by user Test collection I made is under the fold. Users may not even know that their newly created collection(s) are below the fold and can be added to. Perhaps contributes to the numerous one-item collections.
Low Severity - Recommendation Adding a scroll bar would make the UI inconsistent, but there needs to be a more apparent way for users to realize that there is content below the fold.
Low Severity - Issue Add to existing collection when users add to existing collections, a checkmark is green-activated. This is not a problem in and of itself, but potentially creates more general contextual confusion due to the star on the page, the star in the menu, the checkmark in adding to existing collections, then the plus in the hamburger menu
Low Severity - Recommendation Suggest rethinking of the gray and the green activated checkbox as an indicator of status and action: added vs not added or added vs removed. Generally, the icons involved with watchlists and collections need to be streamlined.
Low Severity - Issue Watchlist since the star/collections menu will now be the first interaction from article pages for adding to lists/collections, the following confusion may result: on the watchlist page itself, there is no indication of it being a collection there is no indication that it is public or private
Low Severity - Recommendation On beta, I think it would make more experimental sense to do a full absorption of the watchlist by the new collections, and collect feedback from that. Having both is confusing, especially when the star watchlist page looks totally different than its corresponding page within collections, and it is not marked public/private as it is in collections, among other inconsistencies.
Low Severity - Issue Collections main landing page if watchlist is the only collection listed and it is labeled private, this may cause confusion back on the article pages (with the star/collections menu) when users are prompted to add to new public collection
Low Severity - Recommendation Make collections all public or all private by default, and users can edit that setting after the fact.
Low Severity - Issue Collections main landing page cannot tap on the gray rectangle area to access the collection; must click directly on the name area within the box unless the photographic representation from a page in the collection shows up, in which case the photo part can also be tapped to access
Low Severity - Recommendation Entire area should be tap-able to access that particular collection (entire gray box and photo area, if applicable).
Low Severity - Issue Non-watchlist edit page when the list is empty or populated, the buttons edit title and description (which is also too long for translations) and delete are present potentially confusing what exactly is being edited and deleted
Low Severity - Recommendation The buttons could just be Edit and Delete, if it is clearly associated to the collection itself and not the pages listed within it. However, these buttons (and adding/deleting pages to a collection) could all be functionalities surfaced more directly from the collection main landing page and the collection landing page.
Low Severity - Issue Non-watchlist edit page, Edit title and description why is the public/private collection toggle buried here? after unchecking public collection, going back to the landing page requires a refresh to see the private designation up top
Low Severity - Recommendation This is a bit of commentary on both existing flow and potential for additional features. Again, public/private tag on collections pages can be tap-able toggles. Certain pages require distinct confirmation actions, like the save button on this page. Others, like adding to existing collection on star menu, do not. Should we be consistent about these?
Low Severity - Issue Non-watchlist edit page, Delete many users may hesitate just to click on this for fear of what this will delete is it deleting one page, if there s one page in the collections? is it deleting all the pages, if there s more than one page? is it deleting the collection? only clicking it surfaces the warning dialog (and this button is sometimes non-functional)
Low Severity - Recommendation The button does not need to be such a bright red, which is potentially very deterring. Secondly, it and the edit buttons should both be (as previously mentioned) more closely associated to the collection itself, and not the pages listed within it.
Low Severity - Issue Non-watchlist edit page, search bar tapping brings user to new page. existing pages in collections is there, but searching covers those pages. user needs to exit out of search results by clicking back or the NO icon, which is a confusing icon if user taps back again, user will realize that user needs to save even though the page(s) ( Saucony, in screenshot) already appears added
Low Severity - Recommendation It is confusing when on an article page, tapping on an existing collection green activates it and requires no extra confirmation step, but an extra confirmation step is required here. Making that consistent could be helpful for users. The no icon is very confusing; is there any reason this is used instead of a standard x for cancel search/clear search bar?
Other Issues (in list form) Observations that don t impact usability, but call out consistency, language, and other considerations.
Other Issues Observations that don t impact usability, but call out consistency, language, and other considerations. Star/Collections Menu there is a lot of horizontal white space Star/Collections Menu Add to new public collection long; translation may become an issue down the line Star/Collections Menu Add to existing collection long; translation may become an issue down the line Watchlist for first-time users, there may be confusion around the tabs list and modified the tabs under modified (all, pages, talk, other) may add to the confusion other than the star in the hamburger menu, there is no star (or + icon) on the watchlist page Collections Main landing page does not reinforce the plus icon re/branding, icon not present no photographic representation of a page and/or pages from the collections sometimes populated collections don t have a photo background, other times there are. buggy, or is there a rule around this determination?
Other Issues Observations that don t impact usability, but call out consistency, language, and other considerations. Collections Watchlist landing page Watchlist from hamburger menu and watchlist collections landing page look totally different more contextual confusion still no reinforcement of collections icon here headings are awkward, spacing between lines is awkward, font sizes look random Collections Non-watchlist landing page still no reinforcement of collections icon here though there was an indication on watchlist landing page that it is private, there is no parallel indication on non-watchlist landing pages that these collections are public headings are awkward, spacing between lines is awkward, font sizes look random photos from pages are very large. potentially a lot of scrolling will be involved for collections with 2+ pages suggest a photo view and a listing view, similar to etsy storefronts, and a text listing view. photo view is more of an organized pinterest view, listing view offers an avatar photo view but a full page name, and text listing offers a quick list of just the names of the pages
Other Issues Observations that don t impact usability, but call out consistency, language, and other considerations. Collections Non-watchlist edit page menu bar (where the collections title is located) looks too much like a search bar why is the edit title and description button blue? why is the edit capability for toggling between public and private collection setting buried here why is there a tag for private, but not public? why is this edit capability not surfaced on the actual tag on the collection landing page why is the delete button red? why is there red, blue, and green on this page? Collections Non-watchlist edit page, search cta in the search box needs shortening
Issues/Future feature considerations Lack of current functionality that potentially affects usability; and recs that can help steer future product iterations.
Medium Severity - Issue Add to existing collection action users can only add to/remove from one existing collection at a time; inconvenient if trying to add to/remove from more than one existing collection
Medium Severity - Recommendation Consider whether this menu should be a single-action and minimized menu, or if more than one action can be taken.
Medium Severity - Issue Collections main landing page no option to add a new collection bucket from this page user can only work with existing collections here and create new buckets from wiki pages no centralized way to create new collections
Medium Severity - Recommendation Add functionality to surface actions like adding a new collection bucket (and possibly deleting current buckets, if not too destructive) from the main collections landing page, and directing the add to a modified version of the current collections edit page. Making users go to an article page to start a new bucket every time seems excessive.
Medium Severity - Issue Watchlist and non-watchlist landing pages (Collections) when users do not access this page from the main collections landing page, they will be unable to go back or jump there and see all collections a. two taps from here back to the main collections landing page *android devices have a return button, but again that is only usable if user accesses the page from main landing
Medium Severity - Recommendation From watchlist and non-watchlist landing pages in collections, allow users a quick navigation option to return to the main collections landing page, regardless of their click path getting there.
Low Severity - Issue Collections main landing page cannot access edit collection directly from main landing page
Low Severity - Recommendation From the collections main page, users should be able to quickly access add collection, delete collection, and edit collection options directly instead of having to click twice (or more).
Low Severity - Issue Watchlist landing page cannot tap to change setting to public (or vice versa)
Low Severity - Recommendation The ability to change public/private collections setting should be surfaced where it appears (and there should be a tag for public as well as private), not after 4 taps (nav to edit, nav to edit title/description, toggle public on/off, save).
Low Severity - Issue Non-watchlist landing page there is no quick way to remove pages on this landing page when populated
Low Severity - Recommendation When on a collections page, user should have the ability to quickly access delete, add and edit lists (as mentioned previously).
Low Severity - Issue Non-watchlist landing page if a user is brought to this page after adding to an existing collection from an article page, user may not immediately realize that the pages are organized according to descending alphabetical order, and may have to scroll a lot to confirm the page is added
Low Severity - Recommendation When a collection has more than one article, there should perhaps be a sort by feature (alphabetical, date added, date modified, etc).
Low Severity - Issue General users have no ability to add to watchlist/existing collections and to a newly created one simultaneously
Low Severity - Recommendation If users can perform multiple actions like this within the star menu, then it could be made more consistent with the issue on the last two pages; within the star menu, there can be a confirmation button after one or more actions are made. This potentially can make one more page interaction confirmation-based, as in requiring a user to tap a save button after one or more action.