FOREVER 21. Usability Evaluation Report. HCI 460 July 27, Group 6 Gelayol Moradzadeh Vicky Moreira Mauli Shukla Frank Sweis

Transcription

1 FOREVER 21 Usability Evaluation Report HCI 460 July 27, 2015 Group 6 Gelayol Moradzadeh Vicky Moreira Mauli Shukla Frank Sweis

2 Table of Contents I. Executive Summary 2-3 II. Introduction 3 III. Methodology 4 A. Participants 4 B. Procedure 4 IV. Findings and Recommendations A. Positive Findings 5-6 B. Negative Findings 7-27 V. Team Contributions 28 VI. Appendix

3 I. Executive Summary This report describes the usability issues that users experience while using the Forever 21 Android application to perform common shopping tasks such as browsing and purchasing. Forever 21 is a popular brand with stores located around the world due to its affordably priced and trendy clothing. Their mobile application is designed to help customers purchase their prefered apparel and accessories online, as well as locate stores around them instantly on their mobile devices. To explore the usability issues our team prepared a test plan that defined our objectives and also framed our research questions in a way that would help us identify the strengths and weaknesses of the application. We also defined our target participant, evaluation measures, the tasks for users to perform, and possible questions that could be asked before and after the usability test session. In order to conduct the test we first created a moderator s guide, which included a consent form, introduction, and a series of tasks to keep all team members on track during the test sessions. We conducted a total of 6 tests with both male and female participants that were between the ages of 25 and 45. These participants assisted us in forming both positive and negative findings. We found three major positive findings in the application and they are regarding the following: As soon as the user gets to the shopping page, adding to cart is an easy process Icons used in the application are very intuitive Important elements are not hidden behind multiple screens Some issues were also found during the usability evaluations. The major issues are: Application has a slow speed. Application does not have proper scaling in some pages of the app, which makes it difficult for the participant to obtain information with ease. Users do not notice the shop button easily which causes to miss some of the products available in the application. There is no quick way on home page to access shopping and searching features, which slows down the user in reaching the wanted page in the application. While scanning a barcode, there is no feature to help focus barcodes properly. No feedback or message is provided to the user to inform the users when they have successfully shared and posted items on their social media successfully. Two map buttons are available on the application for locating other Forever 21 stores. Location results are in random order causing users to scroll in order to find the closest store. By conducting the usability evaluation sessions, we found out that all these issues triggers frustration and dissatisfaction with using the application in the user. The most common problems that were observed were unresponsiveness and slowness in the application the difficulty of finding the closest Forever 21 store. In addition to methodology and procedure, this report describes the most common problems with the application including the recommendation for resolving the issues. 2

4 II. Introduction The focus of our study was the Forever 21 Android application. With more than 1,000,000 downloads, the brand has shown to be a popular and trendy store worldwide. This store primarily targets a younger demographic with affordable clothing prices, which now includes male clothing and accessories. Due to the store s popularity, Forever 21 is a recognizable brand to the everyday consumer. In recent years, the company has adopted new marketing tactics to increase their client base. Such tactics include introducing a men s line to their collection and releasing a mobile application in 2012 for quick shopping experiences and become part of today s mobile-driven world. We evaluated Forever 21 s application based on real world scenarios that it s users face. The scope of our research was the user s experience browsing, searching, sharing an item of clothing, and finding the closest Forever 21 store near by. In particular, we wanted to answer the following questions: 1. Can users navigate the application easily to search and buy (add to bag) clothing? 2. How many taps and/or screens does it take for users to perform the outlined tasks (such as buying clothing)? 3. What are the major usability flaws that: a. Impede users from purchasing an item? b. Impede users from finding a Forever 21 store? c. Impede users from sharing an item on Pinterest? 4. Are users satisfied with their interactions with the application when searching and buying clothing, exploring styles, finding stores, and sharing on social media? (data will be gathered through the wrap-up questions) We created specific tasks for our participants to address these questions. Our main goal for these tasks was also to identify the application s strengths and assess how the application could be further improved to better meet the needs and expectations of its users. As a group, we identified some areas areas of improvement for the application, prioritized these areas based on their impact on each task, and identified actionable recommendations which were all incorporated into this report. III. Methodology 3

5 Participants Each team member conducted at least one sit-by evaluation with participants located around Chicago and identified their own method for recruiting and recording their interview; there were six interviews in total. All participants were over 18 years old and familiar with online shopping and mobile applications. We selected participants that either shopped for themselves online or were shopping for their kids. Our team obtained consent from participants by using a consent form. Procedure Each member of our group individually conducted sit-by evaluations that consisted of predefined tasks that we had identified in a Moderator s Guide and below: Browse for an item on the app Prepare item for purchase Scan barcode using scan feature Share an item on social media Find information of store closest to user Within each task, we decided to note quantitative measures such as the number of taps or screens taken to accomplish a given task. We also noted the number of errors or challenges users encountered while completing the tasks. We predefined errors to be when users navigated to the wrong screen, mistapped an element on the application, asked for help, or were unable to accomplish a task. We utilized questions on a Likert scale to understand the user s level of satisfaction and the level of difficulty they encountered. Our group also evaluated the tasks based on qualitative measures such as user comments, feedback, and through wrap up questions that specifically asked how they felt at certain points through the tasks and overall what they liked and disliked. IV. Findings and Recommendations We found a total of 3 positive findings and 11 negative findings. The negative findings are grouped by tasks and then each finding is ordered based on their appropriate severity rating (from highest to lowest) within each task. Our severity levels are based on how much each finding impacts the user s interaction with the website, if it impedes the user from reaching their goal, and the feasibility of improving the finding. We defined each severity level as: High: This usability problem will slow down some users when completing a common task leading them to either find another way of completing it or possibly making them unwilling or unable to complete the task. High severity level denotes that the specified usability problem should be first priority to fix. Medium: This usability problem will make some users feel frustrated or irritated, but will not affect task completion. Medium severity denotes the specified usability problem should be second priority to fix. 4

6 Low: This usability problem does not affect the user s performance as much. This is a quality/aesthetic issue. Even though this is a minor issue, too many low level problems can create a larger usability issue. Low severity level denotes that the specified usability problem should be last priority to fix. All specific tasks include an explanation of each task, a summary with quotes of the participants experience, and an explanation of the results/data gathered. Each finding include images as visual aids to help with the implementations of our recommendations. For a complete lists of our quantitative findings, see appendix. Positive Findings 1. Shopping experience (Adding to bag) Forever 21 uses its mobile website for its application s shopping experience. Their website uses responsive web design, which means that it is optimized to visit over a range of devices from desktop to smartphone. The screenshots below showcase how the website responds to screen size (smartphone vs. desktop) and input method (touch vs. cursor). The responsive design allows for the Shopping cart button to populate in the same area a customer would expect to see it on the website. The icon sizing and placement are intuitive to the user thereby making it a easy and quick task to complete on the mobile application. 2. Icons are intuitive 5

7 Forever 21 uses a mixture of real world and common interface metaphors for it s navigating toolkit. This allows users to recognize the functionality of each icon, and quickly become familiar with the application. 3. Elements are not hidden beneath multiple screens. Forever 21 is designed so that all features of the application are accessible by one or two screens. Along with the real world metaphors, this allows the user to quickly scan the application and get to what they want. However, the application falters when the user begins to use the app. This is due to poor performance and lack of optimization. Negative Findings 1. General Findings consistent throughout all tasks 6

8 Tasks Explanation: Some issues were recurring and consistent throughout multiple tasks. Therefore, below are the issues that impacted all our tasks with the negative feedback and comments from our participants. As such, most of these findings should be treated as highly severe. We found that the type of device (e.g. older device) increases these problems tremendously. Finding 1 (Objective: Major usability flaws, number of taps required) There are overall issues with the application responsiveness and speed. Currently: Takes about 5 to 10 seconds for pages and features to load. Participant Experience Justification As our participants went through all of our tasks, they demonstrated frustration with the speed of the application and an unresponsiveness of elements in the app. One participant exclaimed, Ahh, I can t go back...it s so slow!. The slow speed of the application causes high frustration and dissatisfaction to the users. The application suppose to provide the users a quicker and simpler experience to the user than actually going to the store or website so users expect the application to run faster and make purchases quickly. They do not expect or want to have to wait for a single command to take more than 3 seconds. Recommendation Make the speed of the application faster by optimizing it for all hardware. 7

9 ( This is an example of how fast we recommend the responsiveness of the Forever 21 Android mobile application should be at all times. Severity Level High Finding 2 (Objective: Major usability flaws, number of taps required) User inputs are unresponsive. Currently: This is the blank page that sometimes appears when the app is processing a request to go to another page. 8

10 Participant Experience Justification Recommendation While going through our task, users had difficulty with the application registering their inputs. This caused multiple re-taps and general unresponsiveness. While the problem surfaced throughout the app, it was most apparent when navigating back a page (as the finding above mentions) or using the barcode scan tool. Once a button is clicked it does not immediately respond and complete the expected action. For example, once the scan button is clicked in the main menu, the application freezes. Sometimes the button successfully links to the scan tool while other times it doesn t and once the application freezes, the user may click other buttons to get out of the frozen area of the app. Upon hitting other buttons, the application still not directed to the expected page. It is critical to identify the cause of the button irresponsiveness. If the application is slow because the buttons are not created/linked properly, it would be important to either properly link those buttons. If the placement of hot spot on the tabs or the size of the tabs is an issue that should also be further studied and rectified. This expected pages should appear instantly after choosing (tapping) the appropriate buttons and tabs. Severity Level High 9

11 Finding 3 (Objective: Navigating to find Forever 21 store) Elements within the application do not scale properly or make best use of high resolution displays. Currently: 1440p Display 720p Display 640p Display Participant Experience Justification Participants using the application were unimpressed with the look of Forever 21. One participant commented that Forever 21, Didn t look slick. Font sizes were weird and it looked like a rough draft. Tiny icons and additional white space made the tasks more difficult for the participants. The lack of proper scaling makes elements such as the bottom and upper toolkit harder to navigate. This creates small text which is hard to read and creates lots blank space that can be used for improved usability, most notably when searching for a store. This is also aesthetically unpleasant. New smartphones can offset some of the usability performance problems of the application, but with their higher resolution displays, this create a new usability problem. 10

12 Recommendation The application must detect the resolution of the smartphone and elements (toolkits, font size and font type) accordingly. 1440p Display 720p Display 640p Display Severity Level Medium 2. Task: Preparing an item for purchasing 11

13 Task Explanation: For this task, we asked participants to prepare the item that they d already found for purchasing. In this task users we expected to add the item to their cart successfully. Finding 4 (Objective: Navigating to shop and buy, number of screens required) The placement of the menu is not clear or intuitive to the user and the Shop button in the menu is not very accessible to the user through the application. Participant Experience Justification While this task wasn t difficult for most participants, they had issues locating the shop button for the application. In particular male participants struggled to find their department. How do I shop for men in this app!? asked one participant, as they failed to notice the shop button at the bottom. The user went on, There should be a really easy way. Just clicking around because the website moves slow and there is no options for men here. This is annoying." 3 out of 6 participants completed the task with 5 taps. One participant completed the task with 3 taps and two other participants completed the task with 7 and 9 taps. On average, participants completed their tasks with 3 screens. All participants but one rated the purchasing task as being very easy. one participant rated moderate for this task. The participant thought the Continue shopping tab took a long time to work which made the user frustrated. Participants expected the application to automatically open up to the shop section of the application since that is the primary function of the mobile application. The unintuitive placement of the menu causes users to click on the wrong buttons until they finally locate the shopping button. The amount of 12

14 time spent on attempting to find what they are looking for wastes the user s time and requires them to to spend more time inspecting the app for what they re looking for by either having to scroll or tap through the application more than needed. Recommendation Make the Home page, Shop page and remove the bottom menu bar. Put a searching space at the top of the home page. Severity Level Medium Finding 5 (Objective: Navigating to shop and buy) 13

15 Home page of application offers limited space for shopping and searching. Currently Participant Experience Justification "Why there is a shop page? oh... this makes more sense than home page! I should have came here at the first step," remarked one of our participants, who questioned why the Forever 21 application did not open to the store page. This especially impacted our male users, since the homepage of the application favored female customers. One participant even recommended an alternative UI element for the application. There should be a hamburger dropdown menu on the homepage, they said, where you can select whether you want to shop for women or men s clothing. The hamburger menu that opens up once you go to the shop section would be better placed on the homepage. Forever 21 is a clothing store, and most users will access the application to shop. However, the user must go through various screens to access the main feature of the application to like search for products and shop. Recommendation Our recommendation supported by user feedback is to introduce a hamburger menu that links directly to different areas of the shop section on the homepage. 14

16 Severity Level Medium 3. Task: Scan Barcode Findings Task Explanation: For this task, we asked our participants to scan a Forever 21 price tag with the built-in scan feature of the application. Scanning the price tag should allow the user to find it on the online store, read reviews, and share via social media. Finding 6 (Objective: Impedes users from purchasing/searching for an item) 15

17 There is no system feedback when scanning a barcode. Participant Experience Justification Recommendation While conducting the tasks, our participants were not sure if the application was scanning, if the tag was framed properly, or if the application was working at all. Participant said, Come on! Focus! when the application was slow to respond. Participant 6 refused to complete the task and said, I would never use this. Why would I scan an item I found in the store already to check to check the online inventory? 4 out of 6 participants took less than 3 taps and 2 screens during this task. Half of the participants were able to complete the task successfully. Two of the participants that were unsuccessfully in completing the task took 5 and 9 taps respectively. The remaining unsuccessful participant refused to complete the task, because she did not believe it was something she would ever use. Aside from the user that refused to complete the task, all of the participants rated the difficult level for this task as difficult or very difficult. While scanning a barcode, the user has no feedback on whether the position of the barcode is correct or that the application is scanning. The user wastes time holding and fiddling around with the camera to see if they are doing it correctly. The user can also manually enter the barcode, and may decide to do that rather than attempt to scan the barcode. Have the borders of the camera frame light up green when the barcode is 16

18 properly positioned and scanned. Severity Level High 4. Task: Sharing an item on social media Task Explanation: We asked our participants to share the item that they like in the Forever 21 app on their social media account. Finding 7 (Objective: Usability flaws impede users from sharing item, number of screens and taps required) There is no feedback when user taps on sharing icon and there are lots of steps to connect to social media websites. 17

19 Participant Experience Justification The application did not link to social media websites successfully. It took multiple taps for most of the participants to share an item because they were exited from the app and had to sign in. A few participants struggled with remembering their password. Participants were not sure if they posted on their social media successfully without any feedback. They had to go back and forth between their social media account and the app several times. One participant said, "There is no visible way to know on [this] page that you've posted." Another one exclaimed, "It's freaky twitter opens inside this app... ahh I can't go back...it's so slow!" No participants completed this task successfully Only one participant thought this task was very easy. Five participants believed the task was moderately difficult. The task was considered successful if the users could see the shared item on their social media. While some of the participants believed they have completed the task successfully, they all realized that the item had not been shared after checking their social media page. Therefore, the task was considered as incomplete for all the participants. The process of getting to the share screen and logging in was easy; it was a system error that caused the post to not register on her social media site. Sharing on social media should not take the user a lot of time or steps to complete this task. The application s lack of system status creates uncertainty for the user when they attempt to share. As a result, the user may turn to alternative methods to share. 18

20 Recommendation Fix the functional problems related to connecting to social media websites. Provide feedback when user has posted on their page successfully. Severity Level Low 5. Task: Locating a Forever 21 Store Task Explanation: We asked our participants to use the Forever 21 application to find a store location near them, and to give us the phone number. Finding 8 (Objective: Application Navigation) There are two map or location services buttons on the application. 19

21 Participant Experience Justification Participants were able to find the store locator tool easily, but depending on the page they were on the application displayed two icons--one on the application, and the other on the store webpage. This resulted in one user error observed during this task. Having two buttons for location services on the application creates confusion and difficulty in learnability since there are two paths to do the same kind of task. Also, the two separate location services buttons have different features from each other as they do not link to the same page. Recommendation For the time being, since the Forever 21 app simply opens up the Forever 21 website inside the application, which causes there to be two location services options on the app, make the application s location services button link to the same page as the location services on the website. 20

22 Severity Level Medium Finding 9 (Objective: Usability flaws impede users from finding Forever 21 store) The location services button to find the nearest Forever 21 store does not generate results in order of proximity. There is no logic to the order of results. Currently: The locations are listed at random. Participant Experience Once the stores were listed, users were also confused about the display order of the stores. One participant expected a specific location to be displayed first, but was surprised when the store wasn t list. Why doesn t it put [the locations] in order, they asked, "Why isn't Michigan Ave listed? 21

23 It's the second closest... it's not listed by distance... hmm I wonder how it s sorted?" Only one participant was able to complete the task successfully and as expected with just one tap and one screen with the application displaying the top result as the store in closest proximity to her. Justification Recommendation Since the location services button automatically populates results based on your current location, users expect to see a list of store locations by distance (closest to furthest) to their current location. By not having the store locations list out in nearest distance order, it could potentially make users believe that there aren t Forever 21 locations in their vicinity. This could in turn make locations lose in person traffic because shoppers are not aware of the locations nearest to them. Our recommendation would be to populate the results in order of proximity regardless if the application bases it off current GPS location or zip code. Recommendation: The Forever 21 locations include the distance from the user s location and are ordered from the closest to furthest distances. Severity Level Medium Finding 10 (Objective: Usability flaws impede users from finding Forever 21 store) 22

24 Store Locator: The store icons under the map view do not contain useful contact information. Currently: Icons only displays only the store name. Participant Experience When users we re asked for the phone number of the closest Forever 21 store, users became frustrated due to the lack of information in the map view of the store locator. This resulted in multiple taps and screens to locate the store closest to them from the map, then find the store s phone number from the list view. Two of the six participants took over 20 screen taps and over 6 separate screens to finally find the closest location of a Forever 21 to them. Justification The app currently only shows the store name when viewing the map. This causes the user to have to move to a separate screen for store information (hours, address, phone number). If a user is comparing the hours of a few stores, they must switch back and forth between the two screens, and leads to wasted time and effort. The listed view should be an alternative, rather than a requirement to find store information. 23

25 Recommendation Populate the store pins on the map with store information. Add Forever 21 location and options to call or get step by step directions. Severity Level Medium Finding 11 (Objective: Usability flaws impede users from finding Forever 21 store) The Store Locator tool does not open to the map view. 24

26 Currently: Defaults to this page. Participant Experience Justification As with the previous finding, the application s store locator defaults to the list view. This view is problematic as stores are not listed by proximity, and requires the user to search the map view for the closest store. Participants consistently attempted to tap on the Forever 21 icons to display store information, a behavior learned through other mapping applications. Since the list of stores the application populates is not sorted by distance, the map view is more helpful for the user to determine the closest store. Other mapping applications and competitors show the map view first. This gives the user a visual and spatial reference to find their store Recommendation Have the store locator default to the map view. 25

27 Default to this page automatically when map icon is tapped. Severity Level Low 6. Overall Experience & Satisfaction Explanation: We asked our participants rank their satisfaction with the Forever 21 application on a Likert scale from Very Unsatisfied to Very Satisfied. Table: My level of satisfaction navigating the Forever 21 application was: Data Description: Out of six participants only one participant was satisfied with their overall experience navigating the Forever 21 application. The participant was happy with the shopping experience on the application, and was able to quickly add the dress to her bag. It should be noted that the same participant also struggled the most trying to find the shop page of the application. V. Team Member Contributions 26

28 Team Member Address Specific Contributions Mauli Shukla Attended all meetings, contributed in brainstorming and writing the User Test Plan, Moderator s Guide, and writing/ finalizing negative findings. Conducted 1 usability evaluation. Wrote the procedure and participant sections on Evaluation Report. Vicky Moreira vmorei1@hotmail.com Attended all meetings, contributed in brainstorming and writing the User Test Plan, Moderator s Guide, and putting together a long list of negative findings. Conducted 2 usability evaluations. Created the spreadsheet to compile all team member s participant's data. Wrote the introduction, made visuals for findings, and set up the layout and format of the Evaluation Report. Frank Sweis frsweis@gmail.com Attended all meetings, contributed in brainstorming and writing the User Test Plan and Moderator s Guide. Conducted 2 usability evaluations. Wrote and finalized positive and negative findings as well as created visuals for finding. Gelayol Moradzadeh gelayol.moradzadeh@gmail.com Attended all meetings, contributed in brainstorming, writing the User Test Plan and Moderator s Guide, wrote and finalized negative findings, positive findings and Executive Summary on the Usability Report. Conducted 1 usability evaluation. VI. Appendix Below are tables containing the data gathered from our usability evaluation wrap-up session. 27

29 See attachments for Test Plan, Mod Guide, template of signed Consent Forms, and a spreadsheet of all the data gathered from the participants (includes two sheets- 1 with participants demographic information and 1 with observations notes and their feedback). Finding a piece of clothing: Participant Number of taps Number of screens Number of errors Complete the task? P Y P Y P Y P Y P Y P Y Purchasing a clothing: Participant Number of taps Number of screens Number of errors Complete the task? P Y P Y P Y P Y P Y P Y Sharing on Social media Participant Number of taps Number of screens Number of errors Complete the task? 28

30 P N P N P N P N P N P N Finding Store: Participant Number of taps Number of screens Number of errors Complete the task? P N P N P Y P Y P Y P Y Scan Barcode: Participant Number of taps Number of screens Number Challenges P Y Complete the task? 29

31 P Y P Y P N P N P N Level of difficulty for Purchasing (1 being Very Easy and 5 being Very Difficult): Participant Rating (1-5) P1 3 P2 1 P3 1 P4 1 P5 1 P6 1 Level of difficulty for scan barcode (1 being Very Easy and 5 being Very Difficult): Participant Rating (1-5) P1 5 P2 4 P3 4 P4 4 P5 5 P6 1 Level of difficulty for Social media (1 being Very Easy and 5 being Very Difficult): Participant Rating (1-5) P1 5 P2 3 30

32 P3 3 P4 3 P5 3 P6 1 Level of difficulty for locating a Forever 21 Store (1 being Very Easy and 5 being Very Difficult): Participant Rating (1-5) P1 5 P2 4 P3 4 P4 4 P5 3 P6 1 My level of satisfaction navigating the Forever 21 application was: Participant P1 P2 P3 P4 P5 P6 Very satisfied- very unsatisfied Unsatisfied Unsatisfied Unsatisfied Unsatisfied Satisfied Neither satisfied or unsatisfied 31