Usability Report for Love and Robots.com Fiona Brady, Aoibhίn Hassett, Louise Grant & Hannah O Sheehan N00147351, N00144119, N00144786 & N00147667 Usability C.A. Applied Psychology
Table of Contents Introduction...1 Aims & Uses of the Website...2 Participants...3 Method...3 Findings and Recommendations...5 Ease of Use & Ease of Learning...10 Conclusion...12 Appendices...13
Introduction This study of Loveandrobots.com was conducted to test the sites usability and enhance its user interface. The interface was tested by two select users. These users were given tasks to carry out and were observed by the tester. The tasks given were similar those an average visitor to Loveandrobots.com would wish to carry out. For each of these tasks qualitative or quantitative data were measured. This report contains observations and recommendations interpreted from the user s feedback, both physical and verbal. These observations and recommendations are broken down and laid out according to each of the tasks given to the users. Participants were also asked for any suggestions they had which would create a smoother experience of the site. 1
Aim and Uses of the Website https://www.loveandrobots.com/ Love and Robots is a design platform where you can tweak, personalise and re-design fashion accessories through a method of 3D printing. It provides a range of different accessories for both men and women. The aim of this website is to offer a unique way of customising your own accessories at the user s ease. Accessories include earrings and necklaces for women, and for men it includes cufflinks and bowties. The use of the website is to register and login, to create and customise accessories, to add these to your cart and purchase them. They have also opened a pop-up shop which you can visit and customise accessories in person, on-the-spot and they will be custom-made for you within the hour. You can also contact this website through e-mail and phone. The website is also advertised on a multitude of social media such as Instagram, Facebook, Twitter and Pinterest. 2
Participants Two participants took part in the study. Both participants have previous internet experience. They were separately tested in an IADT computer lab. This environment was one they are comfortable in as they usually attend classes and study in it. They were asked to complete some tasks and answer some questions about the website and their impressions. They each gave informed consent to have their actions recorded using a screen capturing software. The sessions were observed and notes were taken by the tester as the user completed tasks, encouraged to think out loud. Method 1. Participants were first given a consent form to sign and welcomed to the usability test. The tester gave the user an introduction and then the testing began. 2. They were asked a few basic questions before beginning to gain some insight about their level of computer and internet skills. 3. In signing the consent for the users agreed to their actions being recorded using screen capture software and microphone. They were encouraged to think out loud while completing the given tasks. 4. Each participant was shown the homepage for five seconds then asked to give their first impression of the site and its contents. 5. Four scenarios were read aloud to the user for them to complete. They were also given a typed task sheet to which they could refer. Each scenario was revealed after the preceding one was completed. 6. After the tasks had been completed, the users were asked exit questions to gain their overall impression of the site and any issues they encountered. 7. The ease of completion, path taken by the user, time taken to complete the task, number of hints needed by the user to complete the task as well as any errors the user makes while executing the task and how well they can recover from these were all recorded. 3
The four scenarios: Before beginning, the users were asked some basic questions of their experience with the internet. Task 1: First impressions Look at the site homepage of LoveandRobots.com for 5 seconds. Have you seen this site before? Just from looking briefly at this site, what do you think this website is about? Please be specific. Scenario 1: Create an account and login You are looking to buy a present for a friend who is moving away. You have heard about this website, and think it would make a nice present for that friend. Firstly, you want to sign up. We will provide the details such as email and password. Scenario 2: Customise map cufflinks For your friend, you have heard about a certain product on this site. Find the map cufflinks and customise them to show his hometown (Dublin) and where he is going (Boston). Engrave the back of the cufflinks with his name (Conor). Scenario 3: Add this item to your cart and proceed to checkout Add the cufflinks you have customised to your cart and purchase them (as far as asking for credit card details) Scenario 4: Find the returns policy In case the cufflinks don t arrive in the post on time, check the returns policy. Exit questions were asked once the tasks above were completed. 4
First Impressions Feedback When they first heard the name of the website they were intrigued as to what it would be about. The name doesn t easily suggest what is on sale on the website. On first glance, participants believed that Loveandrobots.com was a clothing website. They liked the overall look of the website but were misled as to what was being advertised. They felt the layout of the homepage was clear and extremely accessible. Recommendations On the Loveandrobots.com homepage the carousel of images looks modern and inviting but rearranging the order may help give a better first impression. A visitor to the website s first view is of information about the Loveandrobots.com pop-up shop but at this point, many would still be unaware of the product advertised. The use of the 3D printer to create the custom jewellery should be advertised more openly as is makes this idea very unique. 5
Scenario 1: Sign Up and Login Observations This task was very easily executed by both users. The link was clearly visible on the top toolbar of the site s homepage. In order to sign up, an email and a password is all that is required avoiding any concerns some web users have about submitting personal details. Love and Robots also allows it s users to login via their social media accounts such as Facebook and Gmail. This makes for even easier ways of setting up an account with the site. Recommendations Confusion may occur if a returning customer with an existing account wants to login as it is not very clear that the Sign Up and Login links are separate from each other. Another possible point of confusion could occur when Login is clicked and the title of the pop-up box is Sign In. Users may think they are signing up rather than logging in. 6
Scenario 2: Customise Map Cufflinks Observations Both users tested clicked on the red Map text believing it would lead them to more information about the product. The contrast of colour against the images makes the true links less obvious that the eye-catching red text. Recommendations A search bar would allow users to find products more easily. Though the selection is quite small and finding certain products does not take long, a search bar would make the task even more straightforward. A link on the word Map would benefit users as the colour is eye catching and attracts people to click on it. Rewording the links of 'View Bow Ties' and 'View Cufflinks' could be more inviting for visitors of the website. 7
Scenario 3: Add Item to Basket and Proceed to Purchase Observations This task was very straight forward and easily achieved by the users. The users proceeded in completing this quickly and efficiently. The choice to continue shopping or proceed to checkout allows the user to do as they wish with ease. Recommendations Having the returns policy available to the customer at this point would be very helpful. Having delivery details (i.e. delivery period) at this point would be very helpful. 8
Scenario 4: Find the Returns Policy When given the task to find the returns policy users found it very difficult to complete. The users mentioned that they wouldn t usually look in FAQ for returns policies. They were used to having a separate link for delivery and returns information on different websites. Once they clicked on FAQ it was still unclear whether they would find the information they were looking for here. The links of the side bar of the FAQ page don t stand out well. They clicked on multiple tabs, scrolling through a lot of information before locating the returns policy. Recommendations Creating a separate tab for the returns policy on the footer of the site would enable the visitor to locate the information with more ease. It would be very useful for the user to view the returns policy easily when at the checkout, but before processing the order. Highlighting the links on the FAQ page (General, Shopping & Product Care) would catch the visitor s attention more easily. This would give the user indication of their location on the site and allow them to see other areas of the FAQ section available. 9
Ease of Use and Ease of Learning While testing the website, ease of use and ease of learning of the website will be explored. To do so, tasks were devised so that feedback and results, based on these two pivotal aspects of the website s experience, could be examined and presented. Ease of use of the website is defined by how quickly and efficiently a user can perform a given task; ease of learning refers to how easy it is to get to use the website s interface and layout, and how much can be learned whilst executing a task. The first task is for the user to view the website for the first time for a duration of 5 seconds and to report what their first impression is of the website; what they believe it is about, who the target market is, what it is trying to sell, do they think it is visually appealing. In this case the website has a slideshow of alternating images in three separate places on the home screen displaying three sections of the website; men s, women s and the pop-up shop. But as you hover over the chosen section/ graphic or as you spend a certain amount of time on the page, the graphic changes to reveal images of the products the site sells. This brief task allows the tester to examine what the user sees upon first glance and to evaluate if the website is what it seems. The second task is to sign up/ login. This task relies on clear placement of the sign up/ login link and that the link they click on performs the intended purpose. Ease of use was evident due to a short amount of time spent completing the task. They found the sign up/log in tab swiftly and ease of learning was established when they managed to fill in their details showing clear instruction, which aids ease of learning. The third task entails customizing the map cufflinks. In this case the user would first have to either click on the men s tab at the top of the page or else click on the shop-men s feature on the slideshow of graphics in the centre screen (both equally clear). After clicking on either of these you are directed to a men s section, whereby the map range of accessories are featured amongst three other styles, which you cannot click into the Map collection as a whole you have to click individually on view cufflinks. The user then has to click on the customize option, the user was given directions based on which place and name to engrave 10
on the front and back of cufflinks, so they could either drag the map to that location in the centre of the screen or write in the information in the boxes to the right. These subtle cues definitely aid the ease of learning on the website. They then had to add their item to the cart. Each step in the process of completing this task appears to have several ways of doing it. This definitely adds to ease of use as it takes into consideration that one way might not be initially obvious to a user, cutting down on time spent being confused or lost on the website. Lastly the user had to find the returns policy. In order to find the returns policy the user would have to scroll to the bottom of the website and under support the user would have to click on FAQ. The fact that what you are directly looking for isn t immediately advertised can hinder the ease of use on the website. The user then has to click on the shopping option on the left of the screen and then scroll down the page to find the Refunds and Returns heading. The fact that the user would have to search quite far into the website could leave them doubting themselves and could influence their decision on whether or not they would buy it. 11
Conclusions The tasks were all completed by the users. Some required more work than others but a general response would suggest that the website is easily accessible. This indicates that users would rarely encounter difficulties while using the site. Out of all tasks, the requirement to find the returns policy was the only task that proved difficult. This is a problem that can be easily resolved. Many of the recommendations suggested in the above sections are for making information more easily accessible for visitors of the site. They also suggest adding clarity of certain aspects of the site for the user s needs, avoiding frustration and confusion. A key element of the site s originality is their use of 3D printing their products is not well presented. It is this aspect of the site that would draw people in and encourage them to customise and buy products. Promoting this more would possibly increase visitors and boost sales. 12
Appendices Tester s Introduction Script.....14 Consent Form....15 Pre Test Questionnaire. 16 Scenarios...17 Exit Questions. 18 Results Spreadsheets. 19 Pre Test Questionnaire (Completed)..23 Exit Questions (Completed)... 25 13
Tester s Introduction Script: Welcome and Purpose Thank you for agreeing to participate in this web site evaluation. Today we are asking you to serve as an evaluator of the site Love & Robots and to complete a set of tasks. Our goal is to see how easy or difficult you find the site to use. We will record your reactions and opinions; I may ask you to clarify statements that you make from time to time. Test Facilitator s Role I m here to record your reactions and comments of the Web site you ll view. During this session I will not be able to offer any suggestions or hints. There may be times, however, when I ll ask you to explain why you said or did something. Test Participant s Role I will ask you to search for information on this site to learn if it works well for you. We ll do this by giving you tasks to complete on the site. You are also asked to think out loud as you complete these tasks, to verbalise what you are thinking. Things to Keep in Mind This is not a test of you; you re testing the site. So don t worry about making mistakes. There is no right or wrong answer. We just want to know if the site is well designed for you. If you ever feel that you are lost or cannot complete a task with the information you have been given, please let me know. I ll ask you what you might do in a real-world setting and then either put you on the right track or move you on to the next task. We will be video recording this session for further study if needed. Your name will not be associated or reported with data or findings from this evaluation. Finally, as you use the site, please do so as you would at home or your office. I do ask that when looking for information, you do so as quickly and as accurately as you can. Do you have any questions before we begin? 14
Participant Consent Form: The purpose of this usability study is to evaluate the design of a website. We are interested in determining if people can accomplish common tasks and easily find information using the site. The session will not test' you or your ability, rather the site in order to provide information on areas that could be improved. Please be advised that there are no risks associated with participation in this session. During this session, you will be asked to complete some tasks using the website and answer some questions. As you complete the tasks will be observed and notes will be taken. In addition, the session will be captured on video for future review. The session should last no longer than fifteen minutes. If for any reason you are uncomfortable during the session and do not want to complete a task, you may say so and we will move on to the next task. In addition, if you do not want to continue, you may end the session and leave at any time. Two people will participate in this study. Results from all sessions will be included in a usability report to be presented to Love & Robots. Your name will not be included in the report nor will your name be associated with any data collected throughout this session. I,, have read and fully understand the extent of the study and any risks involved. All of my questions, if any, have been answered to my satisfaction. My signature below acknowledges my understanding of the information provided in this form and indicates my willingness to participate in this user testing session. I have been given a blank copy of this consent form for my records. Signature: Date: 15
Pre-Test Questionnaire How would you rate your proficiency using the Internet, on a 1-10 scale? For what purposes do you normally use the Internet? If you were to book a course, what would be your preferred method of doing so? In person On the phone Using the Internet Other (please specify) Upon first impression, does the website look like it would be easy to use? What is your reason for the answer above? Do you have any comments based on your first impression of the website? 16
Scenarios: Task 1: First impressions Look at the site homepage of Loveandrobots.com for 5 seconds. Have you seen this site before? Just from looking briefly at this site, what do you think this website is about? Please be specific. Scenario 1: Create an account and login You are looking to buy a present for a friend who is moving away. You have heard about this website, and think it would make a nice present for that friend. Firstly, you want to sign up. We will provide the details such as email and password. Scenario 2: Customise map cufflinks You have heard about a certain product on this site that you feel your friend would like. Find the map cufflinks and customise them to show his hometown (Dublin) and where he is going (Boston). Engrave the back of the cufflinks with his name (Conor). Scenario 3: Add this item to your cart Add the cufflinks you have customised to your cart and purchase them (as far as asking for credit card details). Scenario 4: Find the returns policy In case the cufflinks don t arrive in the post on time, check the returns policy. 17
Exit Questions After using the website loveandrobots.com, does your first impression still stand, and if not, what is your impression of the website now? Did you face any obstacles/issues when performing the tasks, if so, what were they? Do you have any suggestions for how any of the issues mentioned above could be improved? What do you think about the search capability? Is there anything that you feel is missing on this site? How would you rate the website, on a 1 10 scale? Do you have any other final comments or questions? 18
Result Spreadsheets: Participant Number Task Number Date Time 1 1 05/11/15 12:00 Task completed successfully? Yes Time taken to complete task 47 seconds No. of hints needed 0 Type of hints needed n/a No. of steps needed to perform task 3 No. of steps taken by user 3 Path taken by user Clicked sign up/ log in > typed in details > clicked create account No. of errors made by user 0 Could they recover from errors? n/a Problems/obstacles faced by user None Ease of performing task (1 [easy] -5 1 [difficult]) Participant Number Task Number Date 1 2 05/11/15 Task completed successfully? Yes Time taken to complete task 70 seconds No. of hints needed 0 Type of hints needed n/a No. of steps needed to perform task 4 No. of steps taken by user 5 Path taken by user Clicked Men > Clicked View Cufflinks > Clicked Map (no response) > Clicked Customise > Entered locations & engravings No. of errors made by user 1 Could they recover from errors? Yes Problems/obstacles faced by user Clicking "Map" text before seeing "View Cufflinks" Ease of performing task (1 [easy] -5 [difficult]) 2 19
Participant Number Task Number Date 1 3 05/11/15 Task completed successfully? Yes Time taken to complete task 20 seconds No. of hints needed 0 Type of hints needed n/a No. of steps needed to perform task 3 No. of steps taken by user 4 Path taken by user Clicked Add to Cart > clicked out of pop up > clicked Cart > clicked Proceed to Checkout No. of errors made by user 1 Could they recover from errors? Yes Problems/obstacles faced by user none Ease of performing task (1 [easy] -5 1 [difficult]) Participant Number Task Number Date 1 4 05/11/15 Task completed successfully? Yes Time taken to complete task 140 seconds No. of hints needed 1 Type of hints needed To be pointed in the right direction No. of steps needed to perform task 2 No. of steps taken by user 7 Path taken by user Clicked Our Story (wrong step) > clicked back to homepage > clicked FAQ > clicked Factory (wrong step) > clicked Terms of Sale (wrong step) > clicked FAQ > clicked Shopping No. of errors made by user 3 Could they recover from errors? With help Problems/obstacles faced by user Difficulty finding returns policy Ease of performing task (1 [easy] -5 5 [difficult]) 20
Participant Number Task Number Date Time 2 1 05/11/15 13:00 Task completed successfully? Yes Time taken to complete task 42 seconds No. of hints needed 0 Type of hints needed n/a No. of steps needed to perform task 3 No. of steps taken by user 3 Path taken by user Clicked sign up/ log in > typed in details > clicked create account No. of errors made by user 0 Could they recover from errors? n/a Problems/obstacles faced by user none Ease of performing task (1 [easy] -5 1 [difficult]) Participant Number Task Number Date 2 2 05/11/15 Task completed successfully? Yes Time taken to complete task 70 seconds No. of hints needed 0 Type of hints needed n/a No. of steps needed to perform task 4 No. of steps taken by user 5 Path taken by user Clicked Men > clicked Cufflinks > clicked View Cufflinks > entered details No. of errors made by user 0 Could they recover from errors? n/a Problems/obstacles faced by user None Ease of performing task (1 [easy] -5 1 [difficult]) 21
Participant Number Task Number Date 2 3 05/11/15 Task completed successfully? Yes Time taken to complete task 12 seconds No. of hints needed 0 Type of hints needed n/a No. of steps needed to perform task 3 No. of steps taken by user 3 Path taken by user Clicked Add to Cart > clicked View Cart > clicked Proceed to Checkout No. of errors made by user 0 Could they recover from errors? n/a Problems/obstacles faced by user None Ease of performing task (1 [easy] -5 1 [difficult]) Participant Number Task Number Date 1 4 05/11/15 Task completed successfully? Yes Time taken to complete task 55 seconds No. of hints needed 0 Type of hints needed n/a No. of steps needed to perform task 2 No. of steps taken by user 2 Path taken by user Clicked FAQ > clicked Shopping No. of errors made by user 0 Could they recover from errors? n/a Problems/obstacles faced by user A lot of time spent looking for something that should be easily accessed Ease of performing task (1 [easy] -5 3 [difficult]) 22
Pre-Test Questions: USER 1 How would you rate your proficiency using the Internet, on a 1-10 scale? 7. For what purposes do you normally use the Internet? Social media and college work. If you were to book a course, what would be your preferred method of doing so? In person On the phone Using the Internet Other (please specify) Upon first impression, does the website look like it would be easy to use? Yes, it does. What is your reason for the answer above? It s well laid out. Do you have any comments based on your first impression of the website? After using the site, I thought it was a bit confusing. 23
Pre-Test Questions: USER 2 How would you rate your proficiency using the Internet, on a 1-10 scale? 8. For what purposes do you normally use the Internet? Facebook and games. If you were to book a course, what would be your preferred method of doing so? In person On the phone Using the Internet Other (please specify) Upon first impression, does the website look like it would be easy to use? Yes. What is your reason for the answer above? Not too much text and it looks easy to navigate. Do you have any comments based on your first impression of the website? Nothing other than what I have already said. 24
Exit Questions/User Impressions: USER 1 After using the website loveandrobots.com, does your first impression still stand, and if not, what is your impression of the website now? My opinion changed, I originally thought it was a clothes website but after, I realised it was a website for accessories. Did you face any obstacles/issues when performing the tasks, if so, what were they? The returns policy was difficult to find. Do you have any suggestions for how any of the issues mentioned above could be improved? When you add the item into the cart, the returns policy should be presented to you clearly and visibly. What do you think about the search capability? There was no search bar. Is there anything that you feel is missing on this site? No. How would you rate the website, on a 1 10 scale? 5. Do you have any other final comments or questions? What they were selling wasn t obvious when I initially looked at the website. 25
Exit Questions/User Impressions: USER 2 After using the website loveandrobots.com, does your first impression still stand, and if not, what is your impression of the website now? My opinion stayed the same. Did you face any obstacles/issues when performing the tasks, if so, what were they? The returns policy was a bit difficult to find, but wasn t impossible. Do you have any suggestions for how any of the issues mentioned above could be improved? When you add the item into the cart, the returns policy should be presented to you clearly and visibly. What do you think about the search capability? There were no problems. Is there anything that you feel is missing on this site? No. How would you rate the website, on a 1 10 scale? 8. Do you have any other final comments or questions? No. 26