Three.co.uk: Usability Research

Size: px

Start display at page:

Download "Three.co.uk: Usability Research"

Eleanor Melton
5 years ago
Views:

1 Three.co.uk: Usability Research Usability24/7 example report Findings & recommendations Date Accredited Practitioner name

2 Introduction Usability24/7 conducted a usability evaluation of the Three website to demonstrate our reporting and analysis approach. We created objectives that: assess users ability to find relevant information on the three.co.uk website elicit comments on the overall design and presentation of information detect any specific usability issues, and any barriers to successful transactions on the website. Usability24/ of 41

3 Methodology The following approach was adopted for the research: Usability24/7 contacted and recruited 6 participants for the research via a study recruitment agency. Each research session lasted around 45 minutes. During the session, the moderator explained the purpose of the session and introduced the tasks which participants were asked to complete. The tasks are listed at Annex A. Participants were asked for their reactions to the information presented and to justify their opinions. 3 participants were shown the relevant screens on a laptop; 2 on an ipad; and 1 on an iphone. Usability24/ of 41

4 Test configuration The following configuration was used for the research: Test Configuration Platform Desktop PC iphone 5 ipad 2 Operating System Windows 8 ios 8.1 ios 8.1 Browser Chrome Safari Safari Screen Size 15.7 monitor Screen Resolution 1366 x x x1536 Usability24/ of 41

Ratings Findings in this report have been classified in terms of importance, with

Often leading to abandonment of the site.

It is imperative that this is rectified immediately.

Should be rectified as soon as time and budget allow.

5 Ratings Findings in this report have been classified in terms of importance, with the following rating system: Seriously impacts the user s experience. Often leading to abandonment of the site. Hinder conversion, as they are difficult to overcome if encountered. It is imperative that this is rectified immediately. Often linked to the experience of the site. Easier to overcome than major issues. Should be rectified as soon as time and budget allow. Often linked to the look and feel or minor functionality. Should be fixed if time and budget allow. No action is required. Often these types of issues are positive findings of interest. Usability24/ of 41

6 Executive summary 1 In general, participants found the website straightforward to use, and they were able to find the information that they were looking for. Participants did not experience any significant navigation issues on any device. However, participants found it difficult to make direct comparisons between different phone/tablet models (except for variations of the same model, such as the ipad). Although the product information met participants expected needs prior to a purchase decision, some thought that it could be laid out more helpfully and that there was little to motivate a purchase decision. Usability24/ of 41

7 Executive summary 2 The checkout process was considered usable and broadly helpful. Search worked effectively and mostly returned useful results. Participants were mostly able to find support information that met their needs. Most participants did not have strong perceptions of Three in advance, although there were some doubts about the network s price-competitiveness. Testing the website tended to improve perceptions of the network. Usability24/ of 41

8 Summary of issues The following table lists all of the issues identified from the usability evaluation: Issues Severity Slide Home page design 12 Want to Shop? panel 13 Comparing phones in Explore our products 14 Store overview 15 Filters 16 Viewing on ipad 17 Compare function 18 Product information layout 19/20/21 Link to network benefits 22 Network benefits pop-over 23 Understanding pricing plans 24 Usability24/ of 37

9 Summary of issues The following table lists all of the issues identified from the usability evaluation: Issues Severity Slide Your Basket layout 26 About you form layout 27 Address details 28/29/30 Choose delivery 31 Use of Search 33 Use of Search 34 Finding support information 35 Finding support information 36 Device support pages 37 Usability24/ of 37

10 Summary of issues The following table lists all of the issues identified from the usability evaluation: Issues Severity Slide Overall impressions of Three website 38 Views on design 39 iphone display 40 Usability24/ of 37

11 Choosing a mobile package Usability24/ of 41

12 Home page Topic: Home page design There were few specific comments on the design of the Home page: it mainly functions as a jumping-off point. Participants tended to navigate from the centre or bottom of the page rather than from the navigation bar at the top. Some thought that offers and devices should be displayed more prominently: the promotional panel at the top of the page takes up a large amount of space and may be distracting to the user. Severity: Moderate Recommendation 1. Give greater prominence to devices/ promotions above the fold. Usability24/ of 41

13 Home page Topic: Want to Shop? panel The Want to Shop? panel links to Explore our products (within Discover), rather than to Store, as the user would logically expect. This makes it difficult for the user to access relevant product information quickly (see below). Severity: Moderate Recommendation 1. Link to Store rather than Explore our products from the panel on the Home page. Usability24/ of 41

Explore our products Topic: Comparing phones in Explore our products Explore our products does not enable users to filter/sort by price (unlike Store).

14 Explore our products Topic: Comparing phones in Explore our products Explore our products does not enable users to filter/sort by price (unlike Store). Price and other device features are not displayed, to enable users to make choices/ comparisons users are only able to filter by manufacturer/model. Users therefore have to select each model and click through to the product details in order to assess its suitability. Severity: Moderate Recommendation 1. Display filters on Explore our products similar to those in Store. Usability24/ of 41

15 Store Topic: Store overview In general, participants found Store helpful as a way to focus on devices of interest to them. Most had a pre-existing device preference (such as the iphone or Samsung Galaxy) and so were mostly interested in comparing product variants and pricing plans relating to the device. A surprising number of devices were shown as being out of stock. Recommendation 1. Do not display out of stock items, or 2. Provide some indication to the user as to when the item will come back into stock (and enable the user to be notified when the item is in stock). Usability24/ of 41

16 Store Topic: Filters The left-hand filters and sort dropdowns were helpful to enable participants to focus on products of interest. Key filters are: Pay Monthly price and manufacturer. Camera specification is an important aspect of users choice of device: a filter for camera specifications would assist users. Severity: Minor Recommendation 1. Add filters for camera specifications. Usability24/ of 41

17 Store Topic: Viewing on ipad Viewing the Store page on the ipad in portrait view is difficult: it is not possible to view both the filter column and the results without significantly reducing the overall page size. By contrast, there are no difficulties viewing this page in landscape view. Severity: Moderate Recommendation 1. Automatically re-scale page for ipad in portrait view, or show filters at top of page. Usability24/ of 41

18 Store Topic: Compare function Participants tended to miss the Compare function (which enables the user to select different devices and compare their features), and where tested, were unable to make it work. The function requires the user to select more than one device, then click on the right arrow that appears against the second device to generate the comparison page. This is a potentially very useful function, but its current design appears to be too obscure. Severity: Moderate Recommendation 1. Re-design the Compare function so that it is (a) more prominent, (b) gives clearer visual guidance to users on how to generate the comparison page. Usability24/ of 41

Product information Topic: Product information layout Participants mostly found the product information that they thought they would need to make a purchase decision.

19 Product information Topic: Product information layout Participants mostly found the product information that they thought they would need to make a purchase decision. But they had reservations about current information layout, and the relative prominence of different information elements. Information on pricing plans is usually more important to users than technical specifications/ feature descriptions: in most cases they are already familiar with the device (either from real-world experience or from researching devices elsewhere online) and are looking for the best deal. Usability24/ of 41

20 Product information Topic: Product information layout Participants therefore wanted pricing plans to be visible above the fold of the page. Since participants are highly influenced by the opinions of others, links to user reviews would be helpful, particularly from trusted third-party sources. Learn more takes the user to lengthy brochure-type information lower down the page: most participants preferred more concise information. Usability24/ of 41

Product information Topic: Product information layout Severity: Moderate Recommendations 1. Re-structure product information pages, so that pricing plans and other pricing information (e.g. special offers) are immediately visible above the fold of the page.

21 Product information Topic: Product information layout Severity: Moderate Recommendations 1. Re-structure product information pages, so that pricing plans and other pricing information (e.g. special offers) are immediately visible above the fold of the page. 2. Link to user reviews on the product information page, from a reputable third-party source. 3. Summarise key product information in bullet points/ short chunks. Usability24/ of 41

22 Why choose Three? Topic: Link to network benefits Participants tended to miss the link to the pop-over describing the benefits of the Three network, as See more reasons does not appear to be a link. They were more likely to click on the coloured tick symbols, with no result. Severity: Moderate Recommendation 1. Display a clearer link to Why choose Three? benefits (e.g. a standard HTML link or brightly-coloured button) to encourage users to view the pop-over. Usability24/ of 41

Why choose Three? Topic: Network benefits pop-over Those who viewed the pop-over were impressed by the text, and found the reasons given persuasive.

23 Why choose Three? Topic: Network benefits pop-over Those who viewed the pop-over were impressed by the text, and found the reasons given persuasive. They were particularly attracted to: Use your phone abroad at no extra cost No nasty Pay Monthly price hikes Your Three phone, unlocked Free 0800 calls Some commented that this text should be more prominent. Recommendation 1. Highlight these benefits throughout the website. Usability24/ of 41

24 Pricing plans Topic: Understanding pricing plans Most found the pricing plans were easy to understand and to compare. Some used the Sort by dropdown, distrusting the default Three Recommends selection. Users tend to be uncertain about how much data they need for different types of activities e.g. to download music, videos, s, or to consult mapping applications. Severity: Minor Recommendation 1. Provide a brief guide to indicate how much data is typically used for common applications. Usability24/ of 41

25 Checkout process Usability24/ of 41

26 Your Basket Topic: Your Basket layout Participants reacted favourably to the layout and transparency of information in Your Basket: very clear and concise it s good, they don t hide anything and your can edit the details Several participants also noted the warning about credit checks: showing this prior to checkout demonstrates openness. Usability24/ of 41

27 About you Topic: About you form layout Participants had few real difficulties completing their details on the About you page, on any device. They commented positively about ease of use on the ipad and iphone (e.g. preselection of number-pad to fill in phone number). They also welcomed: confirmation of completed fields by tick symbols help text which appears on the right as the user completes the form. Usability24/ of 41

28 About you Topic: Address details There was some criticism of the requirement to enter previous addresses (within the last 3 years): why is this relevant? Users who have lived abroad recently will be unable to complete the form: it does not allow an international address. Usability24/ of 41

29 About you Topic: Address details Most had no difficulty with finding their address by postcode recognition. But this causes difficulties for some users, where their current address is not recognised or correctly identified. Information on coverage is welcome and recognised as accurate. But this information is unlikely to make any difference to users willingness to continue with the form. Usability24/ of 41

30 About you Topic: Address details Severity: Moderate Recommendations 1. Enable users to enter address details manually as an alternative to postcode recognition: create a link which opens up address fields for manual entry. 2. Enable users to enter a non-uk address (or alternatively, simply to tick a box confirming that they lived outside the UK prior to their current address). Usability24/ of 41

31 Delivery Topic: Choose delivery There was a positive reaction to choice of delivery address. Most participants thought that they would select delivery in-store rather than at home, both for convenience during the working day, and for security. Usability24/ of 41

32 Finding support information Usability24/ of 41

33 Search Topic: Use of Search Search is easy and effective in retrieving relevant information Participants tended to use Search rather than navigate when looking for Support information (e.g. information on roaming charges). In most cases, the first result that participants selected took them to a page which was directly relevant to their information search. Usability24/ of 41

34 Search Topic: Use of Search Participants did not use the tabs (filtering results into the different areas of the website) to refine their search. The display of Search results was perceived as functional and Google-like. But the lack of imagery when displaying results for products was criticised. Severity: Minor Recommendation 1. Consider displaying product images where search results link to product information. Usability24/ of 41

35 Support information Topic: Finding support information Participants had little difficulty finding relevant support information on topics including: International roaming charges How to deal with a faulty device Coverage How to unlock your phone How to port your phone number They navigated to support information in a variety of ways, including: search; using the sitemap; through the links at the bottom of the home page; through Support). Usability24/ of 41

Support information Topic: Finding support information Support information was considered to be mostly clear and relevant, with some exceptions: e.g. information on dealing with a faulty ipad only covers updating software, and not hardware issues.

36 Support information Topic: Finding support information Support information was considered to be mostly clear and relevant, with some exceptions: e.g. information on dealing with a faulty ipad only covers updating software, and not hardware issues. Several participants commented that it would be useful to be able to access a FAQs section, as a quick way to reach relevant support information. Severity: Minor Recommendations 1. Develop a FAQs section, to link to support content; structure FAQs to give prominence to the issues which most commonly create customer queries. 2. Ensure that support information relating to faults deals with all aspects of faults, not only software issues. Usability24/ of 41

37 Device support Topic: Device support pages The device support pages displayed within the Support section were well received. Participants located relevant devicespecific support information rapidly: It s great that it s all on one page rather than spread around as on a lot of sites normally you have to go hunting around Usability24/ of 41

Overall impressions Topic: overall impressions of Three website Most were pleasantly surprised by the website: it seemed easy to use/ to find relevant information, it offered a good range of phones

38 Overall impressions Topic: overall impressions of Three website Most were pleasantly surprised by the website: it seemed easy to use/ to find relevant information, it offered a good range of phones (albeit with many out of stock) and extensive support information and features (e.g. LiveChat, Call-back). Network benefits were also attractive. For many, perceptions of Three improved as a result of using the website. But some were surprised by the lack of offers and other incentives to transact, in comparison with other websites with which they were familiar. Usability24/ of 41

39 Overall impressions Topic: Views on design There were mixed views about the site design. Some participants liked its simplicity and clarity, and contrasted the design approach favourably with more flashy mobile sites such as Virgin Mobile. Others thought that the design lacked warmth and engagement: aesthetically it s lacking something. Usability24/ of 41

40 Viewing the site on iphone Topic: iphone display The website was only tested by one participant on the iphone: it is not possible to provide comprehensive feedback on the site s mobile performance. However, this participant navigated the Store section and checkout process without difficulty. The key navigation elements matched her expectations, and she commented favourably on several design elements e.g. the simplicity of the About You page, and the tabbed layout of supporting information (e.g. Specs) on the product information page. Usability24/ of 41

41 Annex A User Tasks Usability24/ of 41

42 User tasks Task 1 Select a mobile phone package (10-15 mins) Description You are looking for new mobile phone/phone package to replace your current phone. Browse the selection of phones in the 3 shop, and select a mobile phone package. Completion criteria The participant has selected a mobile phone package for purchase. Purpose - to evaluate how participants browse and select mobile phone packages: what are their main selection criteria? How do they organise the process? What are the key information elements? - to identify any usability issues in the current Shop interface - to assess whether any additional information elements are required to enable users to take a confident purchase decision - to elicit comments on barriers to purchase. Usability24/ of 41

43 User tasks Task 2 Registration/payment (10-15 mins) Description Completion criteria Now pay for the mobile phone package which you have selected. The participant registers their details and completes the purchase of the mobile phone package (up to but not including payment). Purpose - to identify any usability issues in the checkout and registration processes - to evaluate whether there are ways to encourage users to complete the process, and to identify any barriers to completion - to elicit participant comments on any aspect of the purchase or registration processes. Usability24/ of 41

44 User tasks Task 3 Search (5-10 mins) Description You have decided to purchase an ipad Air 2, with a mobile data package. Using Search, find an appropriate package for your needs. Completion criteria The participant selects a mobile package for the ipad Air 2. Purpose - to evaluate the Search process and whether there are any usability issues which impede the user in selecting from Search results - to elicit participant reactions to the information presented about different mobile packages: do they have a clear understanding of their data needs? Usability24/ of 41

45 User tasks Task 4 Support (5-10 mins) Description Completion criteria You need to find information about international roaming charges/ how to unlock your phone. The participant finds relevant information in the Support section of the website. Purpose - to assess ease of navigation/ clarity of labelling across the site - to assess whether participants understand where to look for specific support information - to elicit participant comments on the layout and comprehensibility of support information. Usability24/ of 41

46 Annex B Pre-evaluation questionnaire Usability24/ of 41

47 Q1: Please indicate your age range: Answered: 6 Skipped: 0 47

48 Q2: Please indicate your gender: Answered: 6 Skipped: 0 48

49 Q3: Please describe your occupation: Answered: 6 Skipped: 0 Information consultant Au Pair Administrator Author Office Manager Personal Advisor 49

50 Q4: What activities do you typically conduct on the Internet in a typical month? [please select all that apply] Answered: 6 Skipped: 0 50

51 Q5: How comfortable do you feel about making transactions online? (for example: shopping, banking, paying bills) Answered: 6 Skipped: 0 51

52 Q6: Factors that influence your level of comfort regarding online payment: [please select all that apply] Answered: 6 Skipped: 0 52

53 Annex C Post-evaluation questionnaire Usability24/ of 41

54 Q1: On the following scale please indicate how easy or difficult you felt Three.co.uk was to use: Answered: 6 Skipped: 0 54

55 Q2: On the following scale please indicate how likely you would be to use the Three.co.uk website in the future: Answered: 6 Skipped: 0 55

56 Q3: Please give examples of where you thought the Three.co.uk website was difficult to use: only problems came when trying to read the footnotes on screen. very small print and greyed out as if you should not be seeing them. Finding the information regarding damaged phones took a bit of searching but wasn't impossible. Some information you had to fill out seemed a bit unnecessary.. would feel more comfortable if I had reasons why they needed them available. No FAQ's I got a bit stuck on the comparison part of the website - trying to compare two different manufacturers of phones for example to Nokia Lumia and an Applie iphone. Comparing the iphones was easy to find but not any other brands benefits page was EXCELLENT but in my opinion should be further towards the top of the page/main body copy Search function was quick and gave accurate results N/A 56

57 Q4: What did you most like about the Three.co.uk website?: background colour and print font size I liked the format especially the support, store and discover tabs. Lots of options regarding products and I loved the comparison part. It was easy to navigate round and I learnt about how good they are compared to other Networks see above response. answered questions which are a concern - plus the check out/ details pages were very clear and efficient Navigation was fairly simple The information on each phone i searched for was available on the 1st page without renavigating or clicking into other tabs etc. The layout was pleasant on the eye, not a busy site with too much info bundled together. Colours complimented website not too bright/blinding but gentle almost pastel colours added to site looking nice. Liked that I could also chat live if needed. 57

58 Q5: What did you like least about the Three.co.uk website?: small print in footnotes Using the forms you had to fill out to purchase an item on their mobile device was long and took some effort ie no international option for past addresses. Colour/Design. I thought it was dull and not very eye catching to how it could be too much in the way of graphics Didn't give me any special offers to make me sign up The HTML looking pages via search engine. 58

59 Q6: What additional features would you like to see on the Three.co.uk website?: nothing FAQ's, maybe more information tabs on the first page so you can see a number of things to click on- ie maybe some popular or latest products, where to go if you have a damaged item. Enticing offers to get new customers would be good. I need a reason to be tempted from my existing network that nobody else can offer me nothing really I just prefer 'facts' to glossary Handset reviews n/a 59

60 Q7: Please add any other comments that might be useful in improving the Three.co.uk website?: nothing None really, found it very easy to use for me and I assume most of my generation. Just the look really and the comparison feature as described above have probably covered above More incentives to sign up and more 12 month & 18 month packages n/a 60

61 Q8: Overall, how likely would you be to recommend the Three.co.uk website to a friend or colleague? Answered: 6 Skipped: 0 61

PNC.com, Weather.com & SouthWest.com. Usability Analysis. Tyler A. Steinke May 8, 2014 IMS 413

PNC.com, Weather.com & SouthWest.com Usability Analysis Tyler A. Steinke May 8, 2014 IMS 413 2 P a g e S t e i n k e Table of Contents Introduction 3 Executive Summary 3 Methodology 4 Results 4 Recommendations