Shutterfly Usability Evaluation: Usability Testing Proposal Alexandra De Lory, Bayan Nada, Timothy Visich, Vishwas Sastry ISE 215

Transcription

1 Shutterfly Usability Evaluation: Usability Testing Proposal Alexandra De Lory, Bayan Nada, Timothy Visich, Vishwas Sastry ISE Page

2 TABLE OF CONTENTS TABLE OF CONTENTS CHAPTER PAGE Executive Summary Introduction 3 Study Design 3-4 Assessment Sessions 3 Post use surveys 3 Participants 4 Context 4 Findings 4-7 Novice Users 4-5 Experienced Users 5-7 What s to come? 7 Introduction... 8 Why Usability Testing... 8 Study Purpose and Objectives... 8 Study Design Study/Test Type... 9 Novice User Protocol Experienced User Protocol Post Use Surveys...19 Session Time/Participant Comp Context...22 Study Findings Findings by Task...24 Novice Users Tas#1 Task #2 Task #3 Experienced Users Task #1 2 Page

3 Task #2 Task #3 Comparative Trends Findings by Interface Element Homepage Sign in Photo Book Creation Checkout Process Help Menu Global Usability Issues Local UsabilityIssues Appendices Appendix 1: Screening Application for Non-Users... Appendix 2: Screening Application for Experienced Users... Appendix 3: Testing Recruitment Advertisement... Appendix 4: Testing Schedule... Appendix 5: Research Participant Non-Disclosure Agreement... Appendix 6: Session Introduction... Appendix 7: Experimenter Checklist... Appendix 8: Metrics Sheet Appendix 9: Task Sheets Appendix 10: Post Task Questions Appendix 11: Moderator Script/Protocol for Novice Users Appendix 12: Moderator Script/Protocol for Experienced Users Page

4 Executive Summary Introduction Shutterfly Inc. is one of the leading online services for product personalization and customization. The user interface itself directly relates to the success and profit of the company. The intent of this evaluation is to identify key behavioral patterns that users display when accomplishing a goal on the interface, or navigating the website by exploration. The study consists of two main parts: an assessment component and a comparative component. Study Design Study/Test Type Researchers will carry out a study with two different parts: an assessment portion and a comparative portion. An assessment consists of a formal process and predetermined tasks. Researchers defined the tasks based off of Shutterfly.com s company goals and objectives. The comparative component evaluates the usability of Shutterfly.com by having users complete similar tasks on both Shutterfly.com and a top competitor s website. Objectively comparing Shutterfly.com to a top competitor enables researchers to identify usability problems with Shutterfly.com that may not otherwise be identified. Assessment Sessions Different protocols will be used for users and non-users. Each session will consist of three parts: Task 1, Task 2, and Task 3. After the completion of each task, participants will be asked various questions to evaluate their perceptions and knowledge of Shutterfly.com. Task 1 aims to evaluate the login process, site navigation, the photo book creation process, the design studio feature, and user satisfaction. Task 2 aims to evaluate secondary features of Shutterfly.com. The secondary features evaluated include the use of social media to retrieve pictures, and the My Shutterfly photo library feature. For novice users, Task 3 evaluates the customer support/help link and navigation on Shutterfly.com. For experienced users, task 3 is a comparative assessment in which the usability of Shutterfly.com will be compared to that of a major competitor (snapfish.com). Snapfish.com offers many of the same services that Shutterfly.com offers, and their website includes many of the same features. The comparative study will allow researchers to draw contrasts and similarities between the two websites. Post Use Surveys After completing each individual task (1, 2, 3, and the Comparative Assessment), participants will be given a post-use survey. The post use survey will evaluate user knowledge and perception of Shutterfly.com and user satisfaction. 4 Page

5 Participants For the entirety of the evaluation, a total of 16 participants will be recruited. All 16 participants will be placed in the assessment session and the comparative session. Participants will be divided into subgroups on the basis of experience level. Context All sessions will be carried out in a usability laboratory. The laboratory will consist of a well-lit room with a desk, chair, and computer. Two adjacent rooms (Room A & Room B) are connected on either side of the test laboratory, with a two-way mirror on each side of the lab providing observers in Room A and study moderators in Room B a view into the study site. Cameras in the lab will be pointed at both the user and their screen, allowing observers in both adjacent rooms full insight into the participant s actions via the two-way mirror and monitors provided. Findings Novice Users Task #1 Users were asked to create a Photo Book on Shutterfly. The users, being novice to Shutterfly, used a generic login provided by the researchers. A preliminary set of photos were sent from the participant, to the researcher ahead of time. The participant was instructed to use the photos to create a customized Photo Book. The users were analyzed first in the submenu of the website. It was found that users only used one portion of the submenu, while failing to use other important features. It is recommended that the interface use better visibility for important features in the main menus. Users eventually had to commit to a specific type of template to customize. In order to find a template they found of interest, users had to browse through a screen, which was heavily populated with pictures and text. Users showed to struggle with focusing on templates of interest among these groups of images. A better layout for sample template browsing is needed to promote easier user decision making. Shutterfly offers a tool used for sampling a template before committing to the specific layout. This sampling tool provides users with photos and page examples for the template they clicked on for more information. Users, in general, showed poor adherence to this function. When users began the design process, specific metrics were found based on the menus and functions that Shutterfly provides for customization. Users showed fairly poor performance of many of the functions in the menus and icons. This was due to poor transparency of functions in the menu tabs. The participants also seemed lost in regards to what they should accomplish first, and need more order or automaticity and intelligence within the workflow. Task#2 Participants were asked to pursue a fairly open ended task in this section. Users were told to create any product that interests them, however they were constrained to using social media to retrieve photos. It was found that functions designed to help users create products were neglected in use to reference for preliminary product creation. It was very clear that a 1,2,3 step process should be adopted on 5 Page

6 the homepage when users are beginning product creation. Sources, such as the Design Studio could naturally be found in such a workflow. The product customization via the editing tab was found to be a challenge for users. The products in general were not found to be unsatisfactory, however, the process of editing the products did show hardships. Inconsistencies between product type editing included different tools and functions. This instigated challenges to users. Consistency is a very important aspect to an interface that provides artistic effects, and needs improvement throughout the interface editing tools. Task#3 Users were asked to find the help section from the main menu. Finding information for assistance on the website is an extremely important aspect of customer support. When asked to find support information, few users described this task as challenging. However, some user took more time in finding the desired costumer support information in comparison to others. With this being said, it is suggested that the customer support information is changed to become more prominent. To do this, the help information should be made available within the main navigation areas of the website. Experienced Users Task #1 The most commonly made product on Shutterfly.com is the Photo Book. Photo Book design features allow users to customize products based on photos, layout, templates and embellishments. Experienced users were asked to create a Photo Book using social media for photo retrieval. This meant the participants were required to login to their existing Shutterfly accounts. In this task, it was found that users, after signing in, were immediately directed to the homepage of their My Shutterfly account page. By automatically attending the their account page, the users were less likely to see advertisements and special pricing announcements. Also, the users were more constrained to the projects they had already started. This overall decreases the chance of users purchasing a product. After selecting a specific template to use for Photo Book creation, the interface changes to a photo-editing screen. The main menus in which users can use to change dramatic functions of photo editing procedures were largely unnoticed. This was due to inefficient placement of important menus, as well as using vertical positions of icons and text, which are designed to describe what important editing functions do. A post-task survey was implemented at the end of Task #1. Overall satisfaction was measured using subjective questions pertaining to their experience on the interface. It is suggested that user 6 Page

7 dissatisfaction was not attributed to a lack of features, however the users felt their lack of knowledge of the features decreased their success. Task #2 Experienced users were asked to locate a project that they had previously created or started. The users were asked to use a social media account to retrieve pictures and make adjustments to an original design. During this task, the design studio feature was analyzed. The design studio tool allows users to view examples of created designs, which are meant to inspire customers to design products. The design studio was found to be bypassed by users very frequently. Shutterfly should take a different approach to inspiring users to view sample images and products. The path users took to complete a product creation was also evaluated. Shutterfly is an openended interface, and allows many pathways to be taken to accomplish similar goals. During the analysis of product creation, many of the product design processes were found to be inconsistent between types of products being customized. In order for users to be efficient in creating multiple types of products, a similar design processes should be offered between all products on Shutterfly. A large aspect to the Shutterfly branding includes the possible integration with a wide variety of social media. Users are capable of retrieving photos from social media accounts to customize products. During testing, it was found that users didn t quickly understand Shutterfly s potential abilities to connect to social media. Shutterfly needs to reinforce the full capabilities to users in a clear way. This reinforcement could be done in a multitude of ways, and in many locations around the interface. Task #3 In this task, experienced users were asked to replicate Task #1 (Photo Book creation), however on a Shutterfly competitor website Snapfish.com. This task was meant to find how Shutterfly compared to Snapfish in regards to user experience. After creating a Photo Book on Snapfish, users were asked a series of questions in regards to photo editing. Overall, experienced Shutterfly users were found to prefer the photo editing options on Snapfish. Also, more users preferred the product result they created on Snapfish. These findings suggest that the editing features on Shutterfly should be designed to be simpler. Shutterfly users responded they felt less overwhelmed on Snapfish by menus and product options. Editing photos and Photo Book customization process were analyzed for comparing Shutterfly and Snapfish. The menus used in the design screens showed grouping, naming and formatting 7 Page

8 issues on Shutterfly, and were not as prevalent on Snapfish. We suggest changing some of the current Shutterfly menu icons suggest to more informative titles. For instance, the Storyboard, Arrange, and Book icons showed negative user behavior traits during testing, and are predicted to be traced to the menu labels. The editing tools in the design screens have grouping and real estate issues. The menus are very spread out throughout the screen, which makes it challenging for users to connect to related functions. These tools should be centralized, as designed on Snapfish, and connected based on the tool functions. What s to Come The following report implements the theories and methods of Human Factors Usability research to evaluate Shutterfly.com. In the report, we describe in extensive detail our experimental procedures, our findings, and the reasoning behind our design recommendations. 8 Page

9 Introduction Why Usability Testing? Website usability is defined as the degree to which a user is able to effectively and efficiently navigate through a website. An unusable program or website may be costly to a company, because more resources are required for conflict resolution. These resources include customer support costs, additional development costs, and customer loss. Usability testing is a technique used to identify usability issues within a website. This is done by testing a product or website with representative users, observing user interaction with the website or product, and collecting user feedback. Based on this information researchers are then able to infer what aspects of the site are lacking in usability, and are able to make recommendations geared towards giving the website optimum usability. With any online service or website, usability is what makes a website stand out when compared to competing websites or services. Study Purpose and Objectives Shutterfly Inc. is the world s most successful online service for customization and personalization of items. Shutterfly enables users to customize items for purchase, and to share photos and ideas between communal users. It is understood that the interface itself directly relates to the profit of the company. The target of this study, and the goal of the researchers, is to identify key behavioral patterns that users display when accomplishing a goal on the interface, or navigating the website by exploration. By exposing user experience patterns, the intention of the study is to develop a professional report that clearly allows Shutterfly to modify the interface for better customer interaction by, 1) the advancement of interface value proposition, promoting first time users, and experienced users with the confidence that they are using the best online tool possible for accomplishing their goals, and 2) allowing customers the direction of making online purchases with optimal user experience. By undergoing this research investment, Shutterfly will continue to stay ahead of the market by understanding the needs and behaviors of potential, and current users. 9 Page

10 Study Design Study/Test Type Researchers will carry out two different protocols, one for novice users and one for experienced users. All sessions will consist of assessment studies. Assessment studies consist of a formal process and predetermined tasks. Researchers defined the tasks based off of Shutterfly.com s company goals and objectives. One of the tasks performed by experienced users will be a comparative task. A comparative task evaluates the usability of Shutterfly.com by having users complete similar tasks on both Shutterfly.com and a top competitor s website. Objectively comparing Shutterfly.com to a top competitor enables researchers to identify usability problems with Shutterfly.com that may not otherwise be identified. Novice User s Protocol The novice user protocol will consist of three parts: Task 1, Task 2, and Task 3. After the completion of each task, the participants will be asked various questions to evaluate their perceptions and knowledge of Shutterfly.com. Task 1. The most commonly purchased items on Shutterfly.com are Photo Books. The interface and process through which these Photo Books are made are highly influential on user satisfaction and loyalty. Task 1 aims to evaluate the login process, site navigation, the photo book creation process, the design studio feature, and user satisfaction. The specific metrics that will be accounted for in Task 1 are shown below. Prior to coming in to participate in the study, participants will be asked to researchers a set of 10 personal pictures. Upon entering the lab study session, participants will be provided with a preset username and password for Shutterfly.com. Participants will then be asked to create a personal Photo Book using the ed pictures, which will already be uploaded to the PC by the researchers. When the user is satisfied with the designed product, they will be asked to add the product to the shopping cart, as if they were to go through the purchasing process. Task time will be measured from the point at which the participant navigates to Shutterfly.com up until they place the item in their shopping cart. Participants will also be evaluated on the number of clicks, as well as the path they chose to accomplish the task. The path chosen to accomplish the task refers to: when users are confronted with creating a Photo 10 Page

11 Book, they have two main choices in workflow for a Photo Book design; 1) They can select My Shutterfly and Photo Books (which is the menu shown below). Figure 1: My Shutterfly Server Or 2), they can choose to browse the menu and submenu for a more customized Photo Book explanation of choices (shown below). Figure 2: Shutterfly Photobook Options 11 Page

12 Figure 3: Shutterfly Photo Book Design Server How the user choses to navigate to make the Photo Book is one of the major metrics shown in Appendix 4. After completing Task 1, users will be asked a set of questions in order to evaluate their satisfaction with the site and Photo Book creation process. These questions can also be found in the Appendix 10. Task 1 Sub-Task Metric Reasoning Path Chosen to get to Photo Book creation If using submenu, time taken to browse choices Selection of 'all book styles' When browsing 'Choose a Style', user will be interrupted with a verbal question My Shutterfly / Photo Books (in submenu) Time Selected / Not Selected Reading labels / Looking at design pics? Pathway chosen may show most obvious workflow to users. Is this submenu serving the purpose of bypassing other decisions in the future? Is there a reason to offer the 5 book styles in the submenu, or will users only want to customize? Should the pictures of designs be the main attribute of the template selection process? When selecting template, does Yes / No Sample books should be a way of 12 Page

13 user browse through 'sample books'? Hesitancy to 'select this Photo Book'? Selecting 'My Computer' when choosing Photos for Photo book When finished selecting photos, time to notice how to add them to the Photo Book "Where do you want the photos" automatic window Interrupt the user with question, "what is the function of the Photo Strip?" Transfer of photos vs. pages menu (clicked on both) Use of template design layout menu Table 1: Task 1 Metrics Qualitative Reaction Time/Qualitative Reaction Time Hesitation Time Verbal Answer Yes / No Yes / No introducing how their Photo Book will look. Is 'select this Photo Book' good wording to start the design process? Menu at the top gives option of where to bring photos from. Message on right side of screen, 'Click on photos you want to add to your Photo Book,' may not be obvious initially. This message could be confusing for first time users. The Photo Strip is an important area in which the user can save, then drag photos out of to add to the book. Pages' menu is located adjacent to photo strip, and needs to be clicked to create more pages. Add notes as to how the user used the layout menu features. Task 2. Task 2 aims to evaluate secondary features of Shutterfly.com. The secondary features evaluated include the use of social media to retrieve pictures, and the My Shutterfly photo library feature. Researchers will also be evaluating the login process, site navigation, the design studio feature, and user satisfaction. For Task 2, participants will be provided with a preset username and password for Shutterfly.com, and be asked to use their own personal Facebook account. Participants will be asked to upload a photo to the website through the use of Facebook.com, create any personalized item of their choice, and initiate the purchasing process (add the created product to their shopping cart). It s important to note, the moderator will mention that the user should create something that they would actually consider purchasing. This part of the moderator s protocol will allow the participant to feel naturally involved in the product creation. Task times will be 13 Page

14 measured from the point at which the participant navigates to Shutterfly.com up until they place the product in their shopping cart. Lastly, the participant will be asked to navigate back to the original Photo Book they created. This will allow us to see if they understand the My Shutterfly menu as the important hub of all the projects they have created. Various metrics shown in the table below will also be measured during the entire experimental process. Task 2 Sub-Task Metric Reasoning Product Design Used Design Studio before design process of product began. Hovering of sub-menu Signs of quickly knowing what user wanted to create (product) Use of advertised product design (displayed on middle of screen Upload pics first, or product choice first Struggle to upload from Facebook. Used /Not Used Time spent exploring submenu options. Lack of time spent exploring product options. Effects of advertised product Order Time to find Facebook sign in Is the inspiration tool an important/noticeable aspect to users design process? Are the submenus for design products being used efficiently? Already have an idea of what they want? Does the advertised product attract users to create? You can only upload pics from PC when choosing 'upload' before choosing product. Must use in specific order. When selecting blank product, first reaction to blank template. Qualitative Reaction Templates can look demanding (calendar) When selecting product, time to find personalize. Hesitation to use Facebook at sign in Understanding of photos choices on menu (my photos, friends photos, Time Time/Qualitative Reaction Qualitative Reaction The Personalize icon allows the user to start the design process. Measures trust in the site (huge importance). Understanding of where to select photos from, action required to find the correct 14 Page

15 etc) Editing Photos (color, style, crop, etc) Time/Qualitative Reaction photo. Editing photos is an important feature Did photos look as they meant to edit? Times repeating edit Success rate of completion Text editing or manipulation Qualitative Reaction Important customization process Did text look as they meant to design it? Times repeating edit Success rate of the editing features Ability to change item orientation (above editing area, top, bottom,etc) Qualitative Reaction Item parts have icons that allow for switching, and may not be obvious. Verbal Question, "would you purchase the product you designed? Add to Cart Hesitation measured in time to find 'add to cart' Table 2: Task 2 Metrics Verbal Answer Time Measures basic success rate of the created design. Task 3. Customer service and support is of great importance to any online service or website. Good customer service and support is positively correlated with user satisfaction and loyalty. Task 3 evaluates the customer support/help link and navigation on Shutterfly.com. Task 3 is a continuation of Task 2. For Task 3, at the completion of purchasing the product, the Participants will be asked to find the customer support/help link. Finding the support link from the checkout location is meant to replicate when the participant may have a question about their order, immediately before they actually place the order. Task time will be measured from the point at which the participant navigates to Shutterfly.com until they explicitly state that they have found the customer support/help link. This link can be found on various points within the website, so the location in which the customer finds support information will also be noted. The metrics are listed in the table below. 15 Page

16 Task 3 Sub-Task Metric Reasoning Customer Support Find Customer Support location Time Users should be able to find support fairly quickly. Reaction to help button redirection Ask user, "do you feel confident in the info given?" Table 3: Task 3 Metrics Time, Qualitative Reaction Q/A Users often have questions during the checkout process, the answers to these questions should be readily available to them. Users should feel confident they are getting appropriate info for their questions. Figure 4: Shutterfly Customer Support Experienced User s Protocol Task 1 It s very useful to see how experienced users and novice users compare when interacting with the same website. Task I for experienced users is designed to be similar to the Task I performed by novice users. The experienced users will be asked to first login to their personal Shutterfly account (video recording will not begin until the login process is complete). Next, the users will asked to create a Photo Book design. Experienced users will not be 16 Page

17 constrained to use the pictures sent to the researchers prior to the experiment. They will, however, be told to create the Photo Book using picture source they feel inclined. Sources of picture retrieval could include, but is not limited to the pictures they sent to the researchers before testing (on hard drive), Facebook, Instagram, Google Images, etc. Task 1 Sub-Task Metric Reasoning Path Chosen to get to Photo Book creation If using submenu, time taken to browse choices Selection of 'all book styles' When browsing 'Choose a Style', user will be interrupted with a verbal question When selecting template, does user browse through 'sample books'? Hesitancy to 'select this Photo Book'? Selecting 'My Computer' when choosing Photos for Photo book When finished selecting photos, time to notice how to add them to the Photo Book "Where do you want the photos" automatic window Interrupt the user with question, "what is the function of the Photo Strip?" Transfer of photos vs. pages menu (clicked on both) 17 Page My Shutterfly / Photo Books (in submenu) Time Selected / Not Selected Reading labels / Looking at design pics? Yes / No Qualitative Reaction Time/Qualitative Reaction Time Hesitation Time Verbal Answer Yes / No Pathway chosen may show most obvious workflow to users. Is this submenu serving the purpose of bypassing other decisions in the future? Is there a reason to offer the 5 book styles in the submenu, or will users only want to customize? Should the pictures of designs be the main attribute of the template selection process? Sample books should be a way of introducing how their Photo Book will look. Is 'select this Photo Book' good wording to start the design process? Menu at the top gives option of where to bring photos from. Message on right side of screen, 'Click on photos you want to add to your Photo Book,' may not be obvious initially. This message could be confusing for first time users. The Photo Strip is an important area in which the user can save, then drag photos out of to add to the book. Pages' menu is located adjacent to photo strip, and needs to be clicked to

18 Use of template design layout menu Yes / No create more pages. Add notes as to how the user used the layout menu features. From what source did the user retrieve the pictures? Verbal Answer Table 4: Task 1 Metrics (Experienced Users) Add notes regarding what source was used. Task 2 Users who have previously created a product on Shutterfly are capable of adding, or editing past projects. It s important for Shutterfly to provide the option of stopping a design in process, then starting the process again to finish, then purchasing the project. In order to analyze this, experienced users will be asked to find a product they ve already created, then simply edit, or add to the project. Participants will be constrained to retrieving at least one picture from two personal social media account of their choice. Task 2 Sub-Task Metric Reasoning Product Design Path Chosen Tome Scanned in My Projects Signs of quickly knowing what user wanted to edit (product) Use of advertised product design (displayed on middle of screen Upload pics first, or product choice first Struggle to upload from Social Media Sources. Hesitation to use Social Media at sign in Used /Not Used Time spent exploring submenu options. Lack of time spent exploring product options. Effects of advertised product Order Time to find Social Media sign in Time/Qualitative Reaction What path the user took (My Shutterfly, My Pictures or Shopping Cart) Are the submenus for design products being used efficiently? Already have an idea of what they want? Does the advertised product attract users to create? You can only upload pics from PC when choosing 'upload' before choosing product. Must use in specific order. Measures trust in the site (huge importance). 18 Page

19 Understanding of photos choices on menu (my photos, friends photos, etc) Qualitative Reaction Understanding of where to select photos from, action required to find the correct photo. Hesitancy to find Edit Icon Qualitative Reaction Moderator Comments Editing Photos (color, style, crop, etc) Time/Qualitative Reaction Editing photos is an important feature Did photos look as they meant to edit? Times repeating edit Success rate of completion Text editing or manipulation Qualitative Reaction Important customization process Did text look as they meant to design it? Times repeating edit Success rate of the editing features Ability to change item orientation (above editing area, top, bottom,etc) Qualitative Reaction Item parts have icons that allow for switching, and may not be obvious. Was the product already in their cart, if so, did this cause confusion? Qualitative Reaction Moderator Comments Signs of confidence that the product was saved Qualitative Reaction Moderator Comments Verbal Question, "would you purchase the product you designed? Add to Cart Hesitation measured in time to find 'add to cart' Table 5: Task 2 Metrics Verbal Answer Time Measures basic success rate of the created design. Task 3 The usability of Shutterfly.com will be compared to that of a major competitor (snapfish.com). Snapfish.com offers many of the same services that Shutterfly.com offers, and their website includes many of the same features. The comparative study will allow researchers to draw contrasts and similarities between the two websites. Identifying contrasts and similarities identifies what may be more efficient, or pleasing to experienced Shutterfly users. For the comparative study, the same protocol for Task 2 will be used. Participants will be asked to 19 Page

20 create a product similar to the produce that they edited in Task 2 (on the Shutterfly website). Researchers will compare the outcomes between the different website workflows. The metrics from this section are in the Appendix 4. Figure 5:Snapfish Home Menu Figure 6: Snapfish Design Track Menu Post Use Survey After completing each individual task (1, 2, 3, and the Comparative Study), participants will be given a post-use survey. The post use survey will evaluate user knowledge and perception of Shutterfly.com and user satisfaction. For the comparative study, questions pertaining to how 20 Page

21 Shutterfly compared to the major competitor will be asked. The post-use surveys are shown for reference in Appendix 9. Session Time and Participant Compensation The time duration of each session is expected to be 90 minutes. Participants will be compensated for their participation based on this time estimate and provided with a check for one-hundred ($160) dollars for their involvement at the conclusion of their study session. Usability Metrics Metrics in this study were designed to target how different types of users behave according to their internet experience background, and experience with Shutterfly. In order for Shutterfly to remain the most competitive internet customization tool on the market, all types of user demographics should be accounted for. The metrics will be compared between the different types of users in the results in order for us to target how each metric effects the different demographics specifically. Refer to tables 1-3. Participants Participant Grouping For the entirety of the evaluation, a total of 16 participants will be recruited. All 16 participants will be placed in the assessment study and the comparative study. Participants will be divided into subgroups on the basis of experience level (refer to table #1). Participants are classified as experienced Shutterfly users, Non-Shutterfly users, Advanced Computer/Internet users, and Beginner Computer/Internet users. Experienced Shutterfly users are those who have used Shutterfly to create at least one product and have a preexisting account. Non-Shutterfly users are those who have never used Shutterfly to create a product. Advanced computer/internet users are those who use the internet on a daily basis and are competent in their typing ability. Beginner computer/internet users are those who use the internet on occasion and are incompetent in their typing ability. 21 Page

22 Advanced Experienced Non-Shutterfly Computer/Internet Beginner Computer/Internet Participant # Shutterfly User User User User 1 Yes No Yes No 2 Yes No Yes No 3 Yes No Yes No 4 Yes No Yes No 5 No Yes No Yes 6 No Yes No Yes 7 No Yes No Yes 8 No Yes No Yes 9 Yes No Yes No 10 Yes No Yes No 11 Yes No Yes No 12 Yes No Yes No 13 No Yes No Yes 14 No Yes No Yes 15 No Yes No Yes 16 No Yes No Yes Table 4: Participant Grouping Recruitment/Screening Process The test participants will be of two types: Experienced Users and Non-Experienced Users. Experienced Users will only be allowed to participate if they have used, and created a product on Shutterfly.com. Any type of product creation is valid for participation. The Non- Experienced Users will be required to not have used Shutterfly before the start of the study, however they need to have a basic interest in some kind of arts and crafts or Photo Book creation/sharing. Every participant be relatively experienced using internet websites, and must have a Facebook account. Participants will be required to login to their Facebook account in order to use a personal picture for product creation. Using the specific Facebook login will be 22 Page

23 used as a way to control the process of participants retrieving personal photos. Participants of this demographic will be contacted through Craigslist, Facebook and other forms of social media. When potential participants respond to the research advertisement, they will be ed a recruitment application form. The responses of this form are to be analyzed by the researchers to decide whether or not a responsive application meets the demographic criteria. The participant application form is shown in Appendices 1 and 2. Existing participants will also be asked if they know of anyone who would fit the demographic of this study. Context Testing Environment/Facilities All sessions will be carried out in a usability laboratory. The laboratory will consist of a well-lit room with a desk, chair, and computer. Two adjacent rooms (Room A & Room B) are connected on either side of the test laboratory, with a two-way mirror on each side of the lab providing observers in Room A and study moderators in Room B a view into the study site. Cameras in the lab will be pointed at both the user and their screen, allowing observers in both adjacent full insight into the participant s actions via the two-way mirror and monitors provided. Additionally, audio information is also recorded and is audible to the adjacent rooms Figure 7: Figure of laboratory layout 23 Page

24 Apparatus & Measuring Techniques As this study relies on a multitude of different qualitative and quantitative measures, we will employ various tools to help record and collect our data. For participants to able to interact with Shutterfly s interface, we will have a portable computer setup with screen capturing software (WebEx) to record the user s actions. Screen capturing software allows us to revisit the entirety of user actions as they occurred in real-time on-site, providing us with valuable timebased metrics to use in our analysis that might have gone unrecorded by the moderator and their stopwatch. Likewise, we will be operating video/audio recording equipment (cameras and microphones) to not only record the session, but also perhaps highlight any behavioral nuances that might be missed during the session. The video equipment will consist of two ceiling video cameras, and a computer webcam which will record the user s face and body movements from the front. Lastly, we will be making use of surveys between-tasks, post-tasks, and post-session for each participant in the study. These surveys allow us to gain insight into the user s personal perceptions regarding their experience of the Shutterfly interface, and help to indicate possible areas of study focus. 24 Page

25 Findings by Task Study Findings NOVICE USERS Novice Users Task #1 Overview Shutterfly s most profitable and popular item available to customers is the Photo Book. Customers are capable of customizing a Photo Book of many holidays and occasions. In order to test the efficiency of the Photo Book creation experience, novice users were recruited and tested. The outline of this section is described based on the metrics, or measurements of the individual tasks. There were multiple measurements taken based on the order of the task. Metric groups were analyzed together based on the relationship of the metrics. Purpose This task was meant to find how a new user could begin using Shutterfly for Photo Book creation. In order to test this, users identified predetermined photos that they would be interested using for product design. The photos were sent to researchers ahead of time, and the photos were placed on the testing PC for easy access during testing. This process alleviated the need of the users to access photos from a social media account, and only focused on the interface itself (See experienced users testing later in the report for social media integration). By testing first time users, specific metrics were analyzed to determine if novice users are capable of interacting efficiently on Shutterfly, which is extremely important for website revisits and customer loyalty over time. Metric Group 1 Users were observed using the Shutterfly.com homepage to find where they could begin the process of designing a Photo Book. When asked to begin Photo Book creation, it was found 25 Page

26 that 4/4 users found Photo Books which is located in the bottom menu bar on the homepage (shown in figure below). The users only found one section related to Photo Books on the homepage. The location of the Photo Book tab was shown to be found easily. This submenu was also capable of being dropped down to more specific types of Photo Books offered by Shutterfly. The table below shows that an average of 11.5 seconds was spent by users browsing through this submenu, then choosing a specific type of Photo Book that was offered. Within the submenu was a choice of all book styles. This link would provide a user with every type of Photo Book that Shutterfly offers. It was found that 0/4 participants selected this link. It was found in the post-task metrics questions that users found the menus to be fairly difficult to navigate (Average Response = 4.5 of 10). This could indicate potential navigation problems within the menus. Post-Task Metrics Unit Measured Novice Users Responses How easy was it to navigate through the menus? Scale 1 to 10: Average = 4.5 High = 8 Low = 2 26 Page

27 Issues The purpose of testing access to Photo Books relates to the importance of Photo Book sales to Shutterfly. With this type of product being so successful, users should be able to access Photo Book creation via multiple locations on the homepage. Based on the results of these In-Task Metrics Unit Measured Novice Users Responses Path chosen to get to Photo My Shutterfly / Photo Books 4/4 used Photo Books Book (in submenu) If using submenu, time taken Time Average = 11.5 sec to browse choices High = 23 sec Low = 5 sec Select of all book styles Selected / Not Selected 4/4 Not Selected metrics, users only used one link on the homepage; therefore there was a lack of crosspollination on the interface for an important feature. Also, the generic choice of all book styles was not selected, which brings up the potential problem of a lack of visibility of important icons within the submenus. An example image of this link is shown below: Impact If a user was not able to locate the Photo Book link on the homepage, Shutterfly would have missed out on a potential sale. With this lack of cross pollination, there is a good chance some users may not find this link, and hence, effects the success of Shutterfly Photo Book sales. 27 Page

28 Recommendations It is recommended that various locations of obvious Photo Book access should be provided to users. There is currently advertisements in the main body of the homepage that contain Photo Book access points, however the advertisements change automatically. There should be a main advertisement for Photo Books on the homepage that is static, and doesn t change. This would make for increased cross pollination of the very popular Photo Book section. It is also recommended to make important submenu items more visible, as the bottom submenu features were shown to lack effectiveness. Metric Group 2 When users eventually found the Photo Book link, they needed to select a specific type of Photo Book to personalize. A menu (shown below) is used on Shutterfly to display the kind s distinctive properties of available Photo Book templates. It is important for Shutterfly to display as many choices to users as possible, as every customer may have different interests of styles. 28 Page

29 The metrics table below shows that ¾ users read both the labels in the menu (shown above), and looked at the pictures to help choose a specific type of template, while 1 user said they only used the design picture to make a decision. The post-task questions found that 2/4 participants found the template menu to be distracting, and not helpful. Also, Shutterfly offers a menu to labeled sample books, which allows the user to see more images of what the template looks like with sample images (shown below). This useful menu was only used by 2/4 participants. It s also important to note that based on the time taken on this menu, 2/4 participants hesitated to make a final decision to commit to the specific template. In order to move on with the designing process, a user must press select this book (icon shown below). In-Task Metrics Unit Measured Novice Users Responses Verbal Question While Reading labels / Looking at Both = ¾ 29 Page

30 choosing a style, Reading design pics Design Pics = ¼ labels / Looking at pics. When selecting template, use Yes / No Yes = 2/4 of sample books. Select this Photo Book Hesitant / Not Hesitant Hesitant = 2/4 Post-Task Metrics Unit Measured Novice Users Responses Did you find the different book styles helpful or distracting? Helpful / Distracting Helpful = 2/4 Issues The results show that the menu layouts were not effective for novice users to navigate. One user responded in regards to the template menus that they wanted to try too many options. The menus with both pictures and text may not efficiently display the options a user has. The sample books tool is a great tool for users to learn more about what the template looks like with images. This tool was not used, as the function of the Photo Book (page swipe) did not afford meaning to the user. It s possible that the sample book tool may have made the user less hesitant (which users showed to be) when it came time to commit to a design. Impact Since users may not have been able to see/find the best template for them, they may not have connected with a Photo Book for design. With that being said, Shutterfly may be missing out on Photo Book sales, due to users not finding the right template. The function of sampling the templates turned out to be troubling, which is also connected to making a decision of going further with the process of designing a product. Recommendations 30 Page

31 The menus which shows examples (via pictures) of Photo Books need to allow users to focus on one template at a time. Users showed to be overwhelmed with the options, so a smaller filter needs to be provided. The function of sampling a template needs to afford the user better. Users need to be able to see that the sample tool is useful. Metric Group 3 As discussed earlier, this phase of testing required users to select predetermined photos for testing. The photos that were used to populate the Photo Book design was store on the PC hard drive, and only those pictures were to be used. After selecting the template of the participant s choice, the user had to figure out how to locate the photos (which they were told were on the desktop of the PC). In order to do this, the user had experience the workflow shown below. 31 Page

32 In the metrics results, it was found that an average of 15.5 seconds persisted to complete this segment of clicks. However, one person was found to take 40 seconds to complete this sub-task. In order to actually add photos to the template, the user was confronted with a very clear menu (shown below). This menu allowed users add the photos of interest to the product in an average of seconds. In the post-task questions, it was found that every user rated the difficulty of adding photos to the template extremely easy (1 out of 10). 32 Page

33 After a photo(s) was added to the template, the user was observed on whether they used a specific menu to manipulate the photo and pages that were previously inserted. The menu highlighted below contains 2 options of drastically different functions. It was found that 0/4 participants used both the Photos and Pages menu while editing the template. During this task, it was also asked of the participant to explain, what the function of the Photo Strip is? (image shown below). The Photo Strip is tool, or photo bank provided to the user which allows easy access for drag and drop accessibility when editing the template. Two of the respondents answered I don t know. 33 Page

34 In-Task Metrics Unit Measured Novice Users Responses Selecting My Computer when choosing photos Time Average = 15.5 High = 40 sec Low = 2 sec Adding photos Time Average = sec High = 45 sec Low = 6 sec Photos menu vs. Pages Menu Used both? 0/4 Used Both Post-Task Metrics Unit Measured Novice Users Responses How difficult was it to add photos to your Photo Book? Scale 1 to 10: Average = 1 High = 1 Low =1 Issues Once a user selects a template to use, there is not a step by step process that tells what the user needs to do next in order to complete, or start working on a Photo Book. There is a lack of automaticity within the workflow of adding pictures. The interface should recognize that it is a new project, and the user is going to want to add photos. Instead, the user must find an icon (which is not salient), and create their own workflow. Also, the menus in the edit screen are not clear to the user of function. It was noticed that the users needed to play around with the interface in order to know what the menus were going to do. Impact An interface that requires users to create their own workflow is will account for a lot of human err. If customers can t successfully create an item, or if they have to remember, or spend a lot of time learning a process, customers will stop using the service. 34 Page

35 Recommendations There needs to be more of an automatic workflow for new users to learn the process of creating a new product, or when a product for returning customers is started. The function of the menus and tool in the Photo Book edit screens need to be more transparent. This could be helped by adding text or better iconography. Task #1 Conclusion It was found that important features, such as the Photo Book, should have more access points throughout the interface. Other important features around the interface, including on the menus, should be more visible. Template browsing is a very important aspect to choosing a specific product. Currently, choosing templates is relatively hard due to the overwhelming format the list is currently in. Users showed to have a hard time focusing on the templates that were available. Usability testing has shown that there needs to be a more efficient workflow, which guides users throughout the Shutterfly experience. This would include using more automaticity relating to if users are new, or if they are just starting a project. Novice Users Task #2 Overview Shutterfly.com allows users to connect to their social media accounts to directly pull images from for custom product creation. Social media integration enables users to quickly sign in to an external account from the Shutterfly interface. In order to do this, a user must be a signed in member of Shutterfly. In this task, we used Novice users who were given a generic Shutterfly login. While the user was to completing the task, they were observed throughout the process. Purpose A service that provides customization of products, such as Shutterfly, must have an efficient way of integrating with social media accounts. Most users rely on the internet to store 35 Page

36 important images. These images are stored on social media websites, such as Facebook and Instagram. The purpose of this task was to analyze how efficient the integration process was between Shutterfly, and external social media websites. The participants were also analyzed during the process of exploring the interesting products of Shutterfly, then took time to personalize anything that they wanted. This task offered very little constraints, and hence, replicated a real website visit. Below shows the results that were recorded during Task #2. Metric Group 1 The instructions for this task were fairly open ended for the user to figure out. We wanted to see how the users responded to use your Facebook account to retrieve photos for the personalization of any item. One participant asked the moderator before starting the task, So, I need to go to my Facebook account first, and use photos from there? If a participant asked us a question like this, or if they immediately navigated to Facebook, then we intervened. They were told that they could use the Shutterfly interface to login to Facebook. Shutterfly offers a Design Studio area of the website which allows users to get design inspiration. This is a great feature for users who are not experienced with designing products. It was found that 0/4 participants clicked on the link for the Design Studio. A picture of where to find the Design Studio is shown below, and can be found on the upper, main toolbar. Users were again analyzed while using the menus and submenu for exploration. An average of 21 seconds was spent browsing through the menus. Shutterfly lists a lot of product options in the submenus, and the participants showed to recognize this, and took their time exploring in the menus. One participant, when asked how difficult it was navigating the menus during this task, responded with a 7 out of Page

37 A large area of the home page includes an advertised product section. This section is very visual, and is meant to capture a user s focus to begin a product design. It was found that 0/4 participants used the advertised section to start creating a product. An image of the advertised section is shown below. Users do have a free flowing workflow in Shutterfly, and the order of how they create products is very much up to them. An example of this is the order in which they choose to start personalizing products. Users are able to initially add their photos to My Shutterfly, which is the hub a logged in users experience. My Shutterfly holds all of the photos, past project, etc. in one place for users to connect to while creating products. The My Shutterfly screen is shown below. 37 Page

38 It was found that 4/4 users first selected the product they wanted, then uploaded photos to the product. However, 1 user was only focused on connecting to Facebook from the start of the task, and could not find a link for Facebook. When users eventually found a product to start customizing, they were timed to see how long it took to connect to their Facebook account. At this point, we asked users why they chose they product that they did. 2/4 participants did not have a reason for their product choice. One participant stated, for sentimental reasons, and other stated I wanted to find something useful. After answering this questions, it was found that one user took 2 minutes to locate the Facebook link. The recorded metrics are shown below. One user (in the post task questions) said finding the Facebook login was fairly difficult, recording a 6 out of 10. It s also important to note most people felt pretty secure while logging into their Facebook account on Shutterfly. 38 Page

39 In-Task Metric Unit Measured Novice Users Responses Use of Design Studio For Number of users that used the 0/4 Inspiration design studio Hovering Over Sub Menu Time Average: 21 seconds High: 30 seconds Low: 12 seconds Selection of Advertised Product Number of users that selected 0/4 the advertised product Choosing a product before Number of users that selected a 4/4 uploading photos product before uploading photos Time to find Facebook login menu Time Average: 66 seconds High: 240 seconds Low: 2 seconds Post-Task Metric Unit Measured Novice Users Responses Difficulty connecting to Facebook via Shutterfly.com On a scale of 1 to 10 where 1 is easy and 10 is hard. Average: 3 High: 6 Low: 1 Difficulty finding photos once connecting to Facebook On a scale of 1 to 10 where 1 is easy and 10 is hard. Average: 2 High: 3 Low: 1 Difficulty Navigating through the menus On a scale of 1 to 10 where 1 is easy and 10 is hard. Average: 4 High: 7 Low: 1 39 Page

40 Issues The Design Studio is a feature of Shutterfly that is meant to inspire users to make great looking products. However, none of the participants found this feature, which is located in the upper main menu of the homepage. Participants also responded negatively when asked about menu navigation during this process. The Design Studio may fall victim to the global issues related to menu structure. Also, a couple users were observed scanning through the advertised product area of the homepage, however none of the users clicked on the ads to start designing specific products. The advertised products are located in a central area for the popularity of the positioned products, however the placement was ineffective. There was also issue related to the sign in process of Facebook. Users were not clear if they should initially sign in to social media upon website arrival, or if the process of logging in came later. This is an interface flaw related to lack of interface direction. Impact If users are not able to find an area related to motivation of product design, there is a chance they will become uninterested, and will leave the website. Again, it is found that the menus of Shutterfly lack an order for customers to follow, and disrupts the entire process on the website, no matter what screen they are on. Global issues related to navigation will plague a user s experience on a large website, such as Shutterfly. Recommendations Users need to see a process upon landing on the website. If they are new to Shutterfly, there needs to be options shown immediately. This could include a 1,2,3 step process to login to a social media account, then create products. A user could then naturally find the Design Studio section, as this would become part of the workflow. There is also a need for a very simple, easy to find statement that tells a user social media is capable of being integrated on Shutterfly. This is currently not clear upon arriving on the site. 40 Page

41 Metric Group 2 Shutterfly is very keen about offering the very best customization tool for personalized items. Shutterfly does have a lot of capabilities for customizing creative products, however the results show an issue between the capabilities provided, and how to use the editing tools. "These are good items, I would buy them. But I had trouble customizing it, editing the photo." A quote, such as the one above, is an example of the editing feature flaws provided to customers. It was found that ¾ users did use photo editing features after uploading pictures to their templates. An average of 8 minutes was spent creating the product of their choice, as the features did show to interest the customers. Similar to photo editing, users are also able to add and edit text within the design. It was found that only 2/4 people actually used the text features in the editing process. An example of how to add text to a product is shown below. 41 Page

42 An additional aspect of the edit features includes choosing the position of the product that the user wants to edit. It was found that 0/4 users actually used this feature to edit the product. This is an extremely important menu (shown below), that become important to the success of a user when there is multiple positions of the product that needs to be edited. When the users completed the editing process, they were asked if they would consider purchasing the product. ¼ participants responded that they would purchase the item. In other words, ¾ participants were not happy enough with the design that they would actually spend money on it. When asked how difficult they thought the formatting/editing process was in regards to photos, the average response was 3 out of 10 (Highest response = 5). However, the users did show to be fairly happy with the templates offered by Shutterfly. 42 Page

43 In-Task Metric Unit Measured Novice Users Responses Users who used photo editing Number of users who used the 3/4 features photo editing features Time editing pictures Time Average: 8 minutes High: 10 minutes Low: 5 minutes Using the text features Number of users who used the 2/4 text features. Parts of card menu (above editing Number of users who used this 0/4 area, top, bottom, etc.) feature Purchasing the Item designed Number of users who would 1/4 purchase the item they designed Post-Task Metric Unit Measured Novice Users Responses Ease of photo formatting/placement into the user s selected product On a scale of 1 to 10 where 1 is easy and 10 is hard. Average: 3 High: 5 Low: 1 Satisfaction with template layout. Yes/No 2/4 yes Issues The functions behind icons involved in editing and manipulating designs are not obvious to the user. Overall, users were not successful in creating a product that they like. Although the templates were found to be satisfactory, the actual designed products were not. It was found that the users did spend a decent amount of time designing the photos, so there is definitely inefficiencies when using the design functions. There is also slight differences in the interface between different products. These inconsistencies could definitely confuse users when creating different products over time. 43 Page

44 Impact Shutterfly advertises product design and customizations for people who don t have design experience. If the functions behind product design are not easy to use, then customers will simply not try using them, and may find an easier website for customization solutions. Recommendations There needs to be one interface, or very similar interfaces for all product customizations. There needs to be more text, which would explain the functions behind what the product editing does. There should be more support for novice users for better efficiency while using the product editing tools. Task #2 Conclusion The integration with social media needs to be a part of this workflow redesign. There is currently not a clear description of how Shutterfly integrates with social media websites. The tools for editing and manipulating product designs needs to be more obvious in terms of functions. Better support is needed for users to learn how to use the Shutterfly interface for product design. Novice Users Task #3 Overview As an online retailer that specializes in customizable products with a highly personal component, Shutterfly.com has to be particularly cognizant of how customers are cared for not only during the shopping experience, but post-product purchase as well. Consider the following scenario: a customer spends an entire afternoon carefully building out a Photo Book as a gift for their parents 50 th anniversary including baby pictures of all the kids, special effects fonts and particularly artistic layouts to really showcase their parents love. However, when the Photo Book finally arrived in the mail days later, they were horrified they had received a completely botched Photo Book in the mail (mistakes happen!), and now they re desperately trying to locate 44 Page

45 support information on the Shutterfly website and/or speak with a representative and rectify the issue a fairly common task for a frequent online shopper. Purpose A great online customer service experience should provide many, if not more, of the same features as an in-person shopping experience ideally, customer service should be: Easy to access Easy to use Detailed and thorough Personal Though these elements are just as essential online, the last point of maintaining a service that is personal is especially critical given the sentimental nature of the Shutterfly products, Shutterfly customers are likely to require an added level of emotional understanding and support. For these reasons, in this task we asked participants to locate relevant customer support information, navigate to a particularly relevant issue that they personally have oftentimes experienced during similar online shopping experiences, and describe how useful and delighted they were during Shutterfly s support process. Metrics Navigation & Satisfaction Users began the task of locating customer service information from the homepage. Shutterfly s homepage provides users access to the service section of their website from three separate locations (shown below): Help hyperlink in the header The search bar located in the header A help sub-menu broken down by elements Header: 45 Page

46 Oh wow! It s so tiny. I thought for sure it would be located in one of the main menu options. Footer: (laughs) No I didn t even notice it had that. Though three of these locations exist, when asked to locate the support section, 4/4 users clicked the Help hyperlink in the upper-right hand section of the header but not without difficulty. All four users appeared hesitant and demonstrated excessive scanning behaviors through each of the navigational bar items before coming across the Help hyperlink tucked away in the corner (and on average taking over 20 seconds to do so). Similarly, while unutilized by the participants, it is important to note the lackluster search bar functionality when locating support services. While some terms redirect to the help page, ironically, the search term help redirects to a product page (shown below). It is only when the user realizes to switch sub-tabs from Products to Help that they are redirected to a more relevant sub-page. Using the Search bar: 46 Page

47 Upon arrival to the help section of the website, users were asked to consider their most common In-Task Metrics Unit Measured Novice Users Responses Path chosen to get to customer support Help link/search bar/footer 4/4 used Help in upper-right hand corner If Help link clicked, Time taken to arrive at Help page Time Average = 21 sec High = 42 sec Low = 6 sec starting points and most frequent help sub-section they ve visited during their past online shopping experiences. It was noted that 4/4 users immediately sought out the Contact Us section located under the Most Popular Help Topics banner (shown below). Shutterfly.com Help Section [I] saw it [Contact Us] because it was under the Track my order button, otherwise probably would have missed it. 47 Page

48 Sub-page selected based on most-relevant personal page Focus area selected 4/4 selected Contact Us Issues The purpose of testing ease-of-use and visibility of the support section relates to the overall customer experience. Shutterfly users should be able to adequately locate relevant help if they encounter problems, or are seeking out solutions to specific product related queries. Though Shutterfly allows multiple access points to the service portal directly from the homepage, the collected qualitative and time-based metrics shed light on the difficulty of locating relevant Post-Task Metrics Unit Measured Novice Users Responses How easy was it to navigate Scale 1 to 10 (10 hard): Average = 5 to the relevant support page High = 8 Low = 2 Why did you choose Response Most common: Direct particular support sub-page? interaction for resolution How would you redesign the Response Most common: Make it process of finding support easier to find information? support information quickly. Users had difficulty noticing the Help hyperlink in the header due to its separation from the main navigational elements, and due to its rather muted font and size (poor visibility and substantial clutter). Likewise, should users use the search functionality, they would be left unguided due to the limited capabilities of the search bar. Also, upon final arrival to the support landing page, users typically sought out the Contact Us sub-menu, revealing that this feature should be a more prominent section of the support section. I found it. I just accidentally scanned all the main ones [navigational menus] first. Impact 48 Page

49 Though users were all ultimately successful in locating the support section of the website, and were able to navigate to their most relevant sub-page, observing the users highlighted their difficulty during their process. One of the major tenets of a support section should be its ease of access in a normal setting, users will be trying to locate help quickly. Hang-ups during this process will lead to an increased level of frustration and lack of faith in the overall website, business, and products. While it may be annoying to have an issue, this irritation will only be magnified if the process of resolution is unsatisfactory. Especially for a novice/first-time user, this dissatisfaction is all it takes to prevent them from ever returning to the business for future purchases. Recommendations While it is important to have multiple entry points to the support section of the website, Shutterfly fails to effectively communicate these access venues from the homepage. It is much more useful to have one solid entry point to the help content directly from the homepage, than have multiple points which are hard to locate and only serve to add to the user s frustration. Since the users natural tendency is to seek out a Help navigational element in the main menu, it is recommended to include Help or Support as a main element, allowing for it to be easily accessible. Furthermore, while it goes without saying that the search functionality of Shutterfly could be vastly improved, it is perhaps more necessary to have the Contact Us sub-menu of the support section be prominently displayed upon arrival and therefore much more accessible. Task #3 Conclusion Shutterfly includes a variety of fully customizable products that continue to delight its customers. However, the customer support page on Shutterfly.com is lacking. When users have questions regarding the products, site capabilities, and/or issues post-purchase, they have difficulty finding an answer/resolution to these questions. Ultimately, users can be successful in locating relevant information, however, the time it takes and reactions noted showcase that this 49 Page

50 process can be vastly improved. Even with minimal changes, increasing visibility of the support section and prominently displaying most relevant help topics could promote user efficiency in use. Ultimately, providing easier access to support allows users to feel more confident in the website, thereby increasing the chance that first-time visitors will become return customers. EXPERIENCED USERS Experienced Users Task #1 Overview The most commonly made product on Shutterfly.com is the Photo Book. Users are given the option to create a Photo Book using a custom route or a simple route. Users also are provided with a variety of customization options including template design and layout, photo editing, adding of embellishments, and creating cover pages. Experienced users were asked to create a Photo Book using Shutterfly.com. The outline of this section is described based on the metrics, or measurements of the individual tasks. There were multiple measurements taken based on the order of the task. Metric groups were analyzed together based on the relationship of the metrics. Purpose This task was meant to evaluate how an experienced user creates a Photo Book using Shutterfly.com. The task also evaluated user satisfaction with the Photo Book creation process, and overall satisfaction with the product created. Participants were asked to send the researchers photos prior to coming in to do the experiment. Participants were made aware that these photos were available to them, however, they could retrieve photos from wherever (ex. Social Media) they so pleased. By testing experienced users we obtain valuable data about how website features are/aren t being used Metric Group 1 Issues 50 Page

51 Users were observed using Shutterfly.com to observe which of the many paths they chose to begin the process of creating their Photo Books. Upon signing in Shutterfly.com redirects users to the My Shutterfly tab. While the My Shutterfly tab is helpful in many other tasks it is not optimal for allowing users to create and customize a Photo Book. From the My Shutterfly menu users deprived of the ability to choose the path by which they create their Photo Book (the custom path or the simple path). In-Task Metric Unit Measured Experienced Users Responses Path chosen to create a Photo Book Path selected by users 1/5 Selected the advertised product 4/5 Used the My Shutterfly Tab Impact 51 Page

52 Depending on the user s needs, they may or may not want to allocate a great amount of time to creating a Photo Book. Many users may be frustrated with a longer process and choose to create a Photo Book elsewhere if they find the creation process too lengthy or complicated. Providing users the option to create a Photo Book using either a Simple Path or Custom Path in creating a Photo Book is among many of the great features of Shutterfly.com and should be readily accessible to users. Recommendations After signing in, users should be redirected to the homepage rather than the My Shutterfly tab. Having users return to the homepage, is advantageous in many ways. Upon returning to the homepage users are able to see the special offers and promotions Shutterfly.com has to offer. Returning to the homepage after login will also allow users to take full advantage of all the customization and process features Shutterfly.com has to offer. Metric Group 2 Issues Users were observed using Shutterfly.com to observe which of the many features they chose to use throughout the process of creating their Photo Books. The neglect or failure to use many features available to users was observed throughout the Photo Book creation process. Upon selecting a Photo Book design/template, Shutterfly.com redirects users to a page that consists of a Storyboard tab, an Edit tab, an Arrange tab, and a Book tab. The Storybook tab allows users to distribute photos among the pages of the Photo Book prior to editing and formatting them. None of the users observed used this feature (the Storyboard tab). 52 Page

53 Researchers also observed that the majority of experienced participants (4 out of 5 participants) did not use the left side bar under the Edit tab. This side bar allows users to select a variety of layouts, backgrounds, embellishments, and idea pages. The features on this side bar optimize the customization process, and set Shuterfly.com apart when compared to other photo gift creation website. 53 Page

54 The photo editing features on Shutterfly.com, while useful, are not prominent enough. These photo editing features are also scattered throughout the Edit tab. In observing users edit photos, researchers found that some (2 out of 5 participants) don t use the editing tools available to them upon clicking on a picture. Researchers also found that most (4 out of 5 participants) did not use the editing tools available to them on the sub bar of the editing page. 54 Page

55 In-Task Metric Unit Measured Experienced Users Responses Browsing through the sample Number of users who browsed 5/5 books through the sample books Use of the Story Book Tab Yes/No 5/5 No Source used to retrieve photos. Where users retrieved photos Instagram 1/5 My Computer 4/5 Use of Template Design Menu Yes/No 4/5 No (Side bar under the Edit Tab) Use of photo editing features Yes/No 2/5 No upon clicking on a photo Use of additional editing Yes/No 4/5 No features located on the Edit page sub bar Impact One of the many factors that differentiate Shutterfly.com from its counterparts is the great number and variety of customization options available to users. It is alarming that experienced users fail or neglect to use many of these features. Making these features more salient and grouping them correctly would make them easier to find and use. Recommendations Several changes in the grouping, naming, and formatting of several features on Shutterfly.com would provide users with the optimal experience. Of these changes may be renaming the tabs on the page users are redirected to upon choosing a Photo Book template. Specifically, the tabs Storyboard, Arrange, and Book should be changed to more descriptive titles. In regards to the editing tools, rather than having them spread across the page, they should be made available to users in one central location. In terms of the left side bar, placing text vertically makes it hard for users to read. The side bar should be redesigned so that it is salient and easy to read. 55 Page

56 Metric Group 3 Issues Users were asked a set of post-task questions after completing task 1. These questions evaluated user s perceptions of the ease of use of Shutterfly.com, as well as their overall satisfaction with the site/their product. Most users found the process of uploading photos and the template layout to be satisfactory. However, in asking users to evaluate the ease of menu navigation and ease of photo formatting and placement we found that most users found these tasks moderately to extremely difficult. One finding that is of great concern is that when users were asked whether they would use Shutterfly.com again to create a customized photo product, most users answered no. Impact Most participants stated that they would not return to Shutterfly.com to create a custom photo product in the future. Return costumers and customer loyalty are of great importance to any online service/business. User dissatisfaction with the site implies that they may go elsewhere for their custom photo product needs. Recommendations User dissatisfaction may not necessarily be attributed to the lack of features, but rather the lack of knowledge of these features. Implementing the changes listed in the sections above would make features more salient and usable. In terms of menu design, the use of submenus can be confusing to users. Rather than having two or more submenus per page, Shutterfly.com may want to reconsider how tabs are labeled and grouped. Post-Task Metric Unit Measured Experienced Users Responses Ease in uploading photos. On a scale of 1 to 10 where 1 Average: 3 is easy and 10 is hard High: 6 Low: 1 Ease of menu navigation On a scale of 1 to 10 where 1 Average: 6 is easy and 10 is hard High: 7 56 Page

57 Ease of formatting/placing a photo into a product Did the user like the template layout? Would the user use Shutterfly.com again to create a customized photo product in the future? Low: 3 On a scale of 1 to 10 where 1 Average: 7 is easy and 10 is hard High: 10 Low: 4 Yes/No Yes: 5/5 Yes/No Yes 2/5 Experienced Users Task #2 Overview One feature Shutterfly.com offers return users or members is ability to save projects and edit them at a later time. Users are able to use the My Shutterfly tab to retrieve previously used photos and projects. Shutterfly.com also allows users to login into social media web sites and retrieve photos from those websites. For task 2, experienced users experienced users were asked to find a product they ve already created, then simply edit, or add to the project. Participants will be constrained to retrieving at least one picture from a social media account of their choice. The outline of this section is described based on the metrics, or measurements of the individual tasks. There were multiple measurements taken based on the order of the task. Metric groups were analyzed together based on the relationship of the metrics. Purpose Task 2 aims to evaluate secondary features of Shutterfly.com. The secondary features evaluated include the use of social media to retrieve pictures, and the My Shutterfly photo library feature. Researchers also evaluated the login process, site navigation, the design studio feature, and user satisfaction. Metric Group 1 57 Page

58 Issues Users were observed using Shutterfly.com to observe which of the many features they chose to use throughout the process of creating their Photo product. The neglect or failure to use many features available to users was observed throughout the Photo product creation process. The design studio is a feature on Shutterfly.com that allows users to browse ideas for inspiration. While this is a great feature to have, it seems that users are either unaware of the function of this tab or unaware of the location of this tab. Researchers found that 0 out of 5 users took advantage of the Design Studio feature. Impact One of the many factors that differentiate Shutterfly.com from its counterparts is the great number and variety of customization options available to users. It is alarming that experienced users fail or neglect to use many of these features. Making these features more salient and grouping them correctly would make them easier to find and use. Recommendations In order to make the design studio tab more prevalent, Shutterfly.com may want to reconsider the naming of the tab. Another option would be rather than having a completely separate Design Studio, provide samples of each product within the product page/tab. In-Task Metrics Unit Measured Novice Users Responses Use of Design Studio Use / Not Used 0/5 Used Metric Group 2 Issues Users were observed using Shutterfly.com to observe which path they chose to use throughout the process of creating their Photo product. The number and type of features on Shutterfly.com differ based on the navigation method to the page. The path through which users 58 Page

59 encounter the fewest customization options/path is through using My Shutterfly. We found that all users navigated to their photo product using My Shutterfly. The photo gift creation process lacks consistency. Consistency is the degree to which a website s element look/function the same across the entire site. Consistency is important in terms of company branding and building user trust. The Photo gift creation process should be the same regardless of what part of the site users navigate from. Impact One of the many factors that differentiate Shutterfly.com from its counterparts is the great number and variety of customization options available to users. It is alarming that experienced users fail or neglect to use many of these features. Making these features more salient and grouping them correctly would make them easier to find and use. Recommendations The photo gift creation process lacks consistency. Consistency is the degree to which a website s element look/function the same across the entire site. Consistency is important in terms of company branding and building user trust. Shutterfly should implement a photo gift creation 59 Page

60 process that is the same across all products and doesn t change based on the user s navigation path. In-Task Metrics Unit Measured Novice Users Responses Path chosen to product design My Shutterfly / Other My Shutterfly = 5/5 Hovering Sub-menu Direct Click: Yes Direct Clicks = 5/5 Clicked on advertised product Yes / No No = 5/5 (for design) Chosen order Upload pictures or Product Product choice = 5/5 choice Metric Group 3 Issues Users were observed using Shutterfly.com to observe how they used the social media photo retrieval feature. While most users sound it easy to connect to social media and upload a photo, all participants had difficulty finding the social media link. Users aren t given the option to login to social media prior to creating a product, and the site does not indicate in any way that this is even an option. It is not until users press on the Add Photos button that they are allowed to login to social media accounts to retrieve pictures. 60 Page

61 Impact One of the many factors that differentiate Shutterfly.com from its counterparts is the great number and variety of customization options available to users. It is alarming that experienced 61 Page

62 users have difficulty accessing these features. Making these features more salient and grouping them correctly would make them easier to find and use. As seen in the table below, upon finding the social media link, users were efficient in photo selection and upload. It is unfortunate that such a usable and helpful feature is inaccessible. Recommendations The social media feature of Shutterfly.com lacks visibility. Visibility is the degree to which the features and functions of a site are visible to users. Users should be aware of all the features and options available to them. Adding a link so social media on the My Shutterfly page and the homepage would allow users to know this feature is available to them. Under the edit tab a separate social media button should be placed next to the add pictures button to reinforce that using social media to upload photos is an option available to users. In-Task Metrics Unit Measured Novice Users Responses Struggle to upload from social Time 5/5 < 15 sec media Hesitation to sign into social Yes / No No = 5/5 media Post-Task Metrics Unit Measured Novice Users Responses How difficult was it to find the social media upload link Scale of 1 to 10: Average: 8 High: 9 Low: 6 How difficult was it to connect to your social media account? Scale 1 to 10: Average = 1.5 High = 2 Low = 1 How difficult was it to find your desired photo(s)? Scale 1 to 10: Average = 2 High = 3 Low = 1 How easy was it to upload a Scale 1 to 10: Average = 1 62 Page

63 photo? High = 1 Low = 1 Metric Group 4 Users were asked a set of post-task questions after completing task 2. These questions evaluated user s perceptions of the ease of use of Shutterfly.com, as well as their overall satisfaction with the site/their product. Most users found the process of uploading photos through social media to be easy. However, in asking users to evaluate the ease of menu navigation and ease of photo formatting and placement we found that most users found these tasks moderately to extremely difficult. One finding that is of great concern is that when users were asked whether they would use Shutterfly.com again to create a customized photo product, most users answered no. Impact Most participants stated that they would not return to Shutterfly.com to create a custom photo product in the future. Return costumers and customer loyalty are of great importance to any online service/business. User dissatisfaction with the site implies that they may go elsewhere for their custom photo product needs. Recommendations User dissatisfaction may not necessarily be attributed to the lack of features, but rather the lack of knowledge of these features. Implementing the changes listed in the sections above would make features more salient and usable. In terms of menu design, the use of submenus can be confusing to users. Rather than having two or more submenus per page, Shutterfly.com may want to reconsider how tabs are labeled and grouped. Post-Task Metrics Unit Measured Novice Users Responses 63 Page

64 How difficult was it to navigate through the menus? How difficult was it to add photos to your product? How difficult was it to format your selected photo? Did you like the template designs layouts? Would you use Shutterfly to create another customized design? Scale 1 to 10: Average = 4.6 High = 7 Low = 1 Scale 1 to 10: Average = 6.2 High = 8 Low = 3 Scale 1 to 10: Average = 5.8 High = 8 Low = 3 Yes / No Yes = 5/5 Yes / No No = 3/5 Experienced Task #3 Overview The usability of Shutterfly.com was compared to that of a major competitor (snapfish.com). Snapfish.com offers many of the same services that Shutterfly.com offers, and their website includes many of the same features. Participants were asked to create a Photo Book using Snapfish.com. The outline of this section is described based on the metrics, or measurements of the individual tasks. There were multiple measurements taken based on the order of the task. Metric groups were analyzed together based on the relationship of the metrics. Purpose Task 3 is considered a comparative study. Experienced users are familiar with custom photo gift websites, and provide a valuable insight into user preferences. The comparative study will allow researchers to draw contrasts and similarities between the two websites. Identifying 64 Page

65 contrasts and similarities identifies what may be more efficient, or pleasing to experienced Shutterfly users. Metric Group 1 Issues Shutterfly.com s competitor Snapfish.com allows users to create many of the same products and offers several similar features. Of these features is the ability to upload photos using social media. Users were more efficient in locating the social media link on Snapfish.com then Shutterfly.com. Impact If users perceive that features are easier to find on Snapfish.com or other photo gift creation sites they may divert their business to these sights. Shutterfly.com does not lack any features, the problem lies in the visibility and accessibility of these features. 65 Page

66 Recommendations The social media feature of Shutterfly.com lacks visibility. Visibility is the degree to which the features and functions of a site are visible to users. Users should be aware of all the features and options available to them. Adding a link so social media on the My Shutterfly page and the homepage would allow users to know this feature is available to them. Under the edit tab a separate social media button should be placed next to the add pictures button to reinforce that using social media to upload photos is an option available to users. In-Task Metrics Unit Measured Novice Users Responses Finding social media login Efficiency 5/5 = Direct click menu Social media login location Home page / create tab Homepage = 3/5 Metric Group 2 Issues Shutterfly.com s competitor Snapfish.com allows users to create many of the same products and offers several similar features. Of these features is the photo editing. When asked we found that users were more satisfied with the photo edits they made on Snapfish.com when compared to the photo edits they made on Shutterfly.com. Another alarming finding is that when asked if they would purchase the item they created, more users stated that they would purchase the item they created on Snapfish.com then Shutterfly.com. Impact One of the key features of Shutterfly.com is photo editing. If users are more satisfied with the photo editing capabilities/options on other sites, they may use these other sites to create photo products instead of using Shutterfly.com. Recommendations 66 Page

67 The editing feature of Shutterfly.com is very crowded and scattered. Users should be aware of all the features and options available to them, but they should not feel overwhelmed. Several changes in the grouping, naming, and formatting of several features on Shutterfly.com would provide users with the optimal experience. Of these changes may be renaming the tabs on the page users are redirected to upon choosing a Photo Book template. Specifically, the tabs Storyboard, Arrange, and Book should be changed to more descriptive titles. In regards to the editing tools, rather than having them spread across the page, they should be made available to users in one central location. In terms of the left side bar, placing text vertically makes it hard for users to read. The side bar should be redesigned so that it is salient and easy to read. 67 Page

68 The photo editing features on Shutterfly.com, while useful, are not prominent enough. These photo editing features are also scattered throughout the Edit tab. In-Task Metrics Unit Measured Novice Users Responses Did photos look as they meant Yes / No Yes= 5/5 to edit? Are you happy/would you Yes / No Yes = 3/5 purchase this product? Post-Task Metrics Unit Measured Novice Users Responses On snapfish, how difficult was it to add photos to your product? Scale 1 to 10: Average = 6.6 High = 10 Low = 3 On snapfish, how easy or difficult was it to format/place pics? Scale 1 to 10: Average = 6 High = 9 Low = 2 Metric Group 3 68 Page

69 Issues Users were observed using Shutterfly.com and Snapfish.com. After using both sites users were asked about which site they preferred in terms of photo upload, navigation, template layouts, and customization menus. Users preferred Snapfish.com over Shutterfly.com for photo upload, navigation, template layouts, and customization menus. When users were asked which site they prefer overall 0 out of 5 users said they preferred Shutterfly.com. Post-Task Metrics Unit Measured Novice Users Responses Did you find photo upload Shutterfly / Snapfish Snapfish = 4/5 easier on Shutterfly or Snapfish? What site was easier to Shutterfly / Snapfish Snapfish = 5/5 navigate Did you like Snapfish s Yes / No Yes = 4/5 template design layouts? Did you prefer the layout Shutterfly / Snapfish Snapfish = 3/5 customization menu on Shutterfly or Snapfish? What do you prefer overall, Shutterfly / Snapfish No Preference = 2/5 Shutterfly or Snapfish? Snapfish = 3/5 Shutterfly = 0/5 Task 3 Conclusion The usability of Shutterfly.com was compared to that of a major competitor (snapfish.com). Snapfish.com offers many of the same services that Shutterfly.com offers, and their website includes many of the same features. Overall, participants seemed to prefer Snapfish.com. This preference does not result from a lack of features on Shutterfly.com, but rather the usability and accessibility of these features. 69 Page

70 Comparative Trends between User Types Overview In this section, we will discuss the differences between different types of users performing a very similar task. Shutterfly.com is a very popular website with a large customer base around the world. Therefore, Shutterfly has a lot of returning customers to the website that will have a familiar sense about the workflow and functions on the interface. On the other hand, Shutterfly obviously wants to continue growing, which means novice users on the website will need to have success on the interface. Purpose This section aims to find differences between Experienced and Novice users when interacting with Shutterfly. Realistically, Shutterfly needs to cater to all different types of users, as this section is meant to find significant differences comparing the performance of different types of users. Task #1 was very similar between both Experienced and Novice users. In Task #1, users were asked to complete a custom Photo Book while using photos from an external source (away from Shutterfly). Metrics, which showed significant findings between these two groups throughout Task #1, are important to discuss. Below is a table that compares the significant results between the different types of users. 70 Page

71 User Comparative Unit Novice Experienced Metrics Measured Responses Responses Pa When selecting Yes, user 2/4 5/5 template, do they browsed. browse through sample books. If using submenu, Time Average=11.5 Average = 3 time taken to browse sec sec choices Transfer of photos vs Participants 0/4 3/4 pages menu (clicked who used on both) both Metric #1) When selecting a template, do the users browse through sample books? After selecting Photo Books, users were offered a very busy screen, which showed the different template designs possible to customize. It was found that only 2/4 Novice users actually spent time browsing through this screen. In comparison, every (5/5) Experienced user spent time browsing this screen. This screen, at first glance, is very cluttered and disorganized. This type of screen shows to be inefficient for Novice users, as they were probably overwhelmed with the amount of choices they had. Metric #2) If using submenu, time taken to browse choices When on the homepage of the website, users will always refer to the main menu of an interface as a first step. The main menu gives the user a good idea of what is offered on the website, and how user may go about completing a task. In general, first time users were fairly overwhelmed with Shutterfly. The amount of products Shutterfly offers is vast, so its important for the website to be organized in displaying the choices. When viewing the main submenu of the website, Novice users were found to spend an average of 11.5 seconds hovering the options. Experienced users were found to spend about 3 seconds (direct click) in the submenu. This result suggests that the submenu does not support first time users as it does a user who is familiar with the submenu, hence the direct clicks that were shown by all experienced users. 71 Page

72 Metric #3) Transfer of photos vs. pages menu When editing a Photo Book in the design screen, users were offered a menu with two functions, however shared one area. The first function was a photo strip, which contained all the photos that were added from an external source. These photos, when in the photo strip, were capable of being dragged and dropped into a location. The second function was a Photo Book position viewer. This function allowed users to quickly navigate toward a certain page of the Photo Book, as they could see a sample image of each page in a small area. It was found that none (0/4) Novice Users actually used both of these tabs, whereas 3/4 Experienced Users did click between the two tabs. This example shows that the transparency of this menu s function was not clear to Novice Users, however with experience, the menu was very useful. Conclusion of comparison between user types Based on these results comparing between Novice and Experienced Users, areas of concern were found in regards to first time users not being familiar with the website. Like stated earlier, its important for the interface to be functional for first time users. Different menus on Shutterfly were found not be explored by Novice Users due to the potential of overwhelming clutter, disorganization, etc. There is an expected learning curve with the website, as Novice Users were found to spend more time exploring the main menus. This learning curve is important to be as small as possible, as accurate and automatic reactions are important for a good interface. Familiarity among functions of the website were also found to be different in regards to different user types. Functions need to be transparent to first time visitors, and in Shutterfly s case, functions were not as transparent for Novices. 72 Page

73 Home Page Overview Findings by Interface Elements The homepage is the single most important section of a website. It is the first impression users have after entering the URL, and communicates the website s value proposition. Good homepage structures provide clear information that help the user understand major options within the website and guide the user on where to start. Purpose Shutterfly.com s homepage must fulfill certain criteria to provide value to the user: Effectively communicate the business (products, value, and market differentiation) Guide the user with relevant cues on where to begin interaction Provide navigational elements that succinctly categorize the site s major components Entice the user to purchase products and highlight the first step of the buying process Be attractive, clutter-free, and delightful Most significantly, the homepage should inherently justify why users benefit from using the site, why consumers should shop at Shutterfly.com, and why users should not seek out a competitor s services. Issues While Shutterfly.com does an effective job of providing an attractive landing page, there are several noteworthy issues that will ultimately cause downstream frustrations to the user: 1. Navigation issues - cluttered dual navigational bars 2. Lack of visibility - a rather hidden Help support page 3. Lack of guidance no clear direction upon arrival 4. Lack of value proposition poor product differentiation against competitors 5. Lack of value proposition poor value indication of registering an account 6. Clutter overwhelming sub-menu options 7. Poor feedforward sub-menu navigation items redirect to different user actions 73 Page

74 Impact As an e-commerce website, Shutterfly.com s main goal is to generate revenue by acquiring customers and enticing previous customers to return. In order to accomplish this goal effectively, it is imperative that the site provide an overall positive user experience through quality products, user guidance, and relevant support information should users experience any difficulty. However, the issues noted above cause major impedances to the user flow: poor navigation makes it much more difficult for users to find products and additionally makes it much more difficult for the users to purchase products, convoluted navigation elements reduce the user s confidence that the website is able to meet their needs potentially provoking them to exit the site, without proper value proposition communication users are likely to leave and seek out competitors where the user has previously had a positive experience. Recommendations By modifying certain page elements, significant gains can be made to the overall user experience. Since the homepage is the most critical page in the sitemap, it is imperative to 74 Page

75 provide a seamless user experience from arrival to shopping cart. In regards to the navigation, navigational content can be divided into distinct menus one being horizontal, and the other vertical. This would alleviate the confusion of the dual navigation bar and provide users a clear context of how to interact with the site. The site s value proposition can be improved by providing a tagline in the header section Shutterfly is most widely known for its Photo Book product and this should be highlighted for potential customers. Finally, in regards to customer care, the Help section should be moved into the major navigational elements of the page and given greater prominence so users can immediately resolve any issues they encounter. Ultimately, mitigating these usability issues will likely increase conversions, provide a positive user experience, and keep customers coming back. Photo Book Creation Overview The Photo Book is the most popular item purchased on Shutterfly, and is a large part of the success in which Shutterfly has had. For most products (such as the Photo Book) Shutterfly.com walks the user through five steps: size selection, template selection, photo adding and editing, photo arrangement, and material selection. Because the Photo Book is the most commonly purchased product on Shutterfly.com, the following section is a walkthrough through the interface a user encounters while making a Photo Book. Purpose This section will show how an online customer will experience Shutterfly s Photo Book creation experience. The purpose of this section is to analyze the Photo Book section of Shutterfly using usability testing results, as well as engineering psychology heuristics and principles. This section of the report will include screenshots of the Photo Book design process while showing additional diagrams for interface analysis illustrations. Step 1: Homepage and Initial Choice 75 Page

76 In the figure below, a screen shot was taken showing the Photo Book tab opened. The submenu below shows different Photo Books that are capable of being selected for specific occasions. Notice on the bottom of the menu, a tab labeled all photo book styles is present. Results from usability testing showed that 1/8 people selected all book styles. The rest of the participants selected one of the standard albums in the submenu. Users were also able to select a Photo Book template based on the design sizes offered. This is also a very important aspect of what is being purchased for many people. Again, 0/8 people used this feature when selecting a Photo Book template. The Photo Book size menu icons are shown below. 76 Page

77 The user was faced with different options when in the submenu of Photo Books. During usability testing, none (0/8) participants selected the Photo Book icon on the left, or learn more. These icons were illustrated below, along with the next screen which is also shown. The second screen allowed users to choose a simple path, or a custom path for creating a Photo Book. This is a really nice feature for users who were not interested in spending a lot of time on the design process. In saying that, however, none of the users during testing were able to navigate to this screen. Issues The navigation menus showed to lack visibility on icons within the menu. This is an issue, because users are definitely not aware of all the possibilities that Shutterfly offers. A large value of using Shutterfly [as opposed to competitor sites] is the vast options it provides. The entire value proposition of the website is minimized when the amount of options that a user sees is limited. By giving a user different paths depending on their experiences (as shown in Custom 77 Page

78 Path vs. Simple Path), the website fails to cater to specific types of personas using the website. This again, decreases the value proposition of the service. Impact The impact of decreasing the value proposition of the website is very obvious. If users don t feel like they are being taken care of, customer loyalty will not occur. Additionally, the pricing of Shutterfly products are fairly expensive in comparison to other companies, so the value proposition is quite important if customers are willing to pay more. A notable Photo Book expert was found stating if price is the top concern, Snapfish's prices are normally lower than Shutterfly's. photobookgirl.com. Recommendations Shutterfly menus need to be responsive to what users are most interested in. Due to the fact that Shutterly is a huge interface with many products, it s important for users to be shown all of the options the service offers. Also, the feature which allows users to choose between a custom path and a simple path needs to be more centralized, and easier to find. This is surely a feature in which more participants in the usability study would have been interested in if they had been able to navigate to it. Step #2 Template Choices 78 Page

79 Regardless of what type of template type the user selected, a screen, such as the figure below, allowed participants to select the exact template they were interested in seeing. After Task #2 for both Novice and Experienced Users, the participants were asked if they felt this screen was helpful, or distracting. The results showed that 4/8 participants found this screen to be distracting. One participant felt not enough information was given regarding the individual Photo Books, like the amount of pages. Also, none of the participants used the Book Ideas menu on the left. This menu allowed users to change from the highlighted template in which they were currently in, to another template type. Upon selecting a specific template, users were offered more information regarding the template design on the screen shown below. One specific piece of important information included the product size, which could be changed, as well as the material in which it could be made of. It was found that none (0/8) participants manipulated these options in any way. The participants quickly noticed Select this photo book and moved on with the design process. Other functions 79 Page

80 on this page were bypassed, such as the sample book function, which would have allowed users to see different sample pages regarding the template design they were interested in. Issues In order to choose a template, the user had to select a specific layout on a much cluttered screen. The templates were very close together, and offered minimal separation between different layouts. This made it very hard for participants to focus on one template at a time, making the choice harder than it should have been. The Book Ideas on the left of the template choices were not noticed by users, as they provide a fast way to navigate directly towards another template type. Important menus on the Sample Book screen were also unnoticed. None of the functions afforded the user to click and change options on this page (other than the select this book link). Impact Users will were not able to focus on a specific template, and thus, probably weren t able to select the best option. One of Shutterfly s goals is to provide customers with an efficient design interface, however, customers should still enjoy browsing the options in which is offered. The 80 Page

81 template choice screen could be designed to be more enjoyable to view. A lack of affordances regarding menus and tabs in this section create missed opportunities for product creation. Problems such as this could end up turning into product returns after orders, as well as unhappiness regarding orders and experiences. Recommendations Template designs need to be more spaced out, or organized differently for users to choose. The templates should also have more immediate information for users to quickly see. By decreasing the bright colors around important functions and links, easier visibility could be directed towards important features that are currently being bypassed. Step #3 Design Screen The design screen contains many functions, and will be explained in sections by order in which users experienced. First, shown below, users naturally wanted to figure out how to add photos to the design screen. Regardless of what they were using to retrieve photos (hard drive, social media, etc.), they had to locate the highlighted button shown below, which reads Add photos. Every user that was tested was successful in finding this icon. Additionally, the user s average rating of the difficulty in uploading photos was scored at 2.3 out of 10 overall (10 being hardest). 81 Page

82 A very simple screen was next shown to users and allowed them to retrieve photos from specific locations (screen not shown). After uploading photos (which showed to be easy), photos were automatically added to the photo strip (see figure below). The photo strip allowed users to drag and drop photos directly into designated spaces in the Photo Book. At this point during testing, every user was asked what is the function of the photo strip? 8/8 users answered the question with either I don t know (didn t use it), or answered with a lack of accuracy. A couple example quotes are shown below. Notice, the second quote shows that there is confusion between the function of the Photos and Pages menus. (More about this is discussed in the next paragraph) To see all of your pics together? "This? It lets you see the layout of the page, lets you see the book as a whole." In addition to the above figure (shows photo strip), the highlighted portion shows a menu which gave users the ability to switch between two functions of the Photo Book. When Pages was selected (alternative to Photos), the user could scroll to see where they were in relation to the entire Photo Book. Interestingly, none (0/4) of the novice users during testing actually used both the Pages and Photos functions, as ¾ experienced users did use both functions. The Pages tool would provide users with good feedback regarding their location on the interface if the function was more transparent. 82 Page

83 Checkout Process Overview As sales of its products are a key source of Shutterfly.com s profits and is in fact the site s primary mission the checkout process is an integral part and perhaps the unapparent backbone of the website as a whole. This is truly a case of an item that is last but not least. A user typically enters the Photo Book creation process with the intent of creating a finished project that they can turn into reality a physical photo book in their hands. Therefore, the user will want to take action when they have finished their Photobook, unless he or she decides to use the save feature in order to finish or edit the book later. Either way, eventually the user will need to order his or her creation and go through the payment process. For these above reasons, shown below are the screens that a user will go through when ordering an item. The checkout process is made up of a few straightforward steps that will be detailed later. Although there are relatively few steps, this area has the opportunity to leave a lasting impression, both because this function is of utmost importance to the user, and because its position as last in interactions with the website makes it very memorable. Also, not just the use of the interface is a factor; the content of those screens, namely the communication of the item price, affects users willingness to pay and their desire to return to the site another time. For the purpose of feasibility, users were asked to complete just the first step in the ordering process by clicking only the initial Order button in the check out process, without going through with the purchase. However, in the real world users would complete the purchase process, and so the entire purchase process, from first Order button to order confirmation, was evaluated by the team of human factors professionals. Purpose This portion will show the users experience through the screens encountered when checking out. Through evaluating users behavioral and verbal reactions, and putting one s self in the users shoes, this section will highlight areas of improvement. 83 Page

84 The user mouses up to the far right corner of the window to click the Order button after feeling satisfied with a product design. Issues While possibly convenient because of its proximity, this placement could potentially delay users slightly in ordering. While the delay was minimal, the button could be placed in a more conventional position to avoid potential frustration or just extra effort. Users are used to clicking an order button located to the near right of the picture of an online product, and this is the convention in a majority of websites. The users instinct to want to order close to the product is also related to theories of perception; according to one of Gestalt s laws of perceptual organization - the law of proximity items that are near each other tend to be grouped together by the viewer. Impact While seemingly minute, the small delay of mousing to this far location could affect users perception of that page s usability. This is because it may cause momentary confusion, and a small, but nevertheless existent extra effort. Recommendations In line with the Gestalt law of proximity described above, placing the Order button closer to the product would better cater to the users instincts. 84 Page

85 After clicking the first ordering button, the user is brought through a series of pop-ups, asking if they would like to upgrade various features of the product, most notable in Photo Book ordering. Although features such as hard cover vs. soft cover was chosen by the user early in the product creation process, they are asked again if they want to upgrade to hard cover. Issues This surprised one user, and can be perceived as somewhat jarring because the user was expecting to go straight to the ordering page. Impact This can negatively affect the users perception of and satisfaction with Shutterfly.com. Users are savvy, and may perceive even helpful attempts to improve their product as trying to have them buy more than they want to. When they face potentially unwanted attempts to upsell their purchase, they may perceive the site as being focused on profits. This could decrease trust in Shutterfly.com, and this takes away the focus from the high quality of the product and service itself. Recommendations While the final offers to upgrade their purchase do not necessarily have to be removed, they could be designed in a way that is more natural and feels like less of a barrier to ordering. 85 Page

86 This is the initial ordering screen that the user sees after totally completing their choice of purchase. This shows the item that the person has just added to their online shopping cart. In equally large size, the products are displayed below it, and take up a relatively sizable amount of the screen as much as the item being ordered. Issues Putting ourselves in the users shoes, the usability team felt momentary confusion about where to look for the information we needed. The user s goal would be to order the item they decided to order, but other products that are popular in relation to the prior choice take up a large part of the screen below, and compete for the user s attention. At first reaction, the user may subconsciously perceive the different items to be equally important to them, because the size and font with which the items are displayed are identical. While the person may be interested in the other products, this can also momentarily confuse the person. Impact Users are steered by how much various elements of a page stand out, and can only take in so much information at a time. Because the item being ordered the items one could order take up the same amount of real estate and have no other distinguishing factors, this could be confusing to the user who entered the purchase process with a set item or items in mind and only wants to purchase that item or items as quickly as possible. Recommendations 86 Page

87 In order to lessen the cognitive load on users, and make the checkout process faster, the suggested products at the bottom should be displayed in a smaller size, and optionally, in less quantity. This goal can also be accomplished through various other graphical methods. This is the next screen that the user is brought to. Here, they see their complete shopping cart this includes any other items they have previously made, and the items that they have ordered during their current use session. To the right is a summary of the price, broken down into shipping and estimated tax, as well as savings, and the total amount. Below is a small Shutterfly informational ad for 4x6 prints. 87 Page

88 Issues Two users from very different backgrounds expressed that they thought the product was not worth the price, and that this is the reason that would stop them from using Shutterfly.com in the future. Impact While Shutterfly.com s products have appropriate pricing, the website needs to ensure that this message comes through to users. In an age of free services and applications, the high quality of Shutterfly.com and the value of its services may be lost on the user, if the price is simply presented to the user without a form of explanation as it is above, or if the value proposition is not even indirectly expressed to the user. As a result, the group of users that are annoyed by the price may be driven to go to another site, or to do it with a lesser respected, cheaper service. Recommendations The value proposition of Shutterfly s product, as well as their service, needs to be communicated in some way, whether graphical, verbal, or a combination of both. Shutterfly users are paying for the convenience and the value of a hassle-free, what-you-see-is-exactly-what-you-get customized product, as well as the instantaneous order of a product that will almost always be exactly what they wanted, while not having to do any physical work. It is important that they understand this in terms of price. Of all the checkout recommendations suggested, this is the most important to Shutterfly.com s profits. 88 Page

89 After finishing the order, the user sees a thank you message with their order confirmation with the order number. To the right, an ad is displayed. Issue An ad is displayed to the right. While not a problem on its own, a large volume of ads on one website can have a collective negative effect. One user passionately expressed that he thought that there were too many ads on Shutterfly, and that he felt the goal of the ads were were to interest him into buying more things. This is the opposite of how a website wants its customers to feel; the customer should have freedom of choice, and feel that they own the process and are doing things on the site because they want to. Shutterfly, in particular, is a website in which users should enjoy the process of creating personalized, memorable, and sentimental products. Impact Users may not feel that they want to return to the site, because they do not perceive it positively, and/or feel pressured by the ads. Recommendation Decreasing the number of ads, or placing them strategically and selectively, according to the user s activities, can lighten the ad load on the user, while creating an emotionally positive experience. Checkout Process Conclusion 89 Page

90 The checkout process presents an opportunity to make the user feel respected and intelligent, as well as improve their view of Shutterfly. These goals can be accomplished by making the process as quick as possible, without compromising safeguards to accidental ordering or leaving out important information. Also, the user should feel as though they are in control of the process, and that the price they are paying is worth the excellent product. Help Menu Overview Customer support services are detrimental to the success of any online business or service. Good customer service reaps many benefits including customer loyalty, customer satisfaction, and website branding. Purpose This section will show how an online customer will experience Shutterfly s customer support page. The purpose of this section is to analyze the Photo Book section of Shutterfly using usability testing results, as well as engineering psychology heuristics and principles. This section of the report will include screenshots of the customer support page as well as an analysis of these pages. Step 1: Homepage and Help Link Location In the figure below, a screen shot was taken of the homepage showing where a costumer would go for costumer support. For customer support users would press on the Help hyperlink. All users were able to locate the Help hyperlink in 20 seconds or less. 90 Page

91 After pressing on the Help hyperlink users were redirected to the page pictured below. Issues While all users were able to locate the Help hyperlink, the time it takes them to do so could be significantly reduced if the hyperlink was more visible. In the current formatting, the hyperlink text is small and difficult to read. The hyperlink also is very similar to those of My Account, and Sign In. Within the page, most (3 out of 4) users were able to find the support they needed; however, they described the help page as: 91 Page

92 Cluttered and Messy Impact The impact of decreasing the value proposition of the website is very obvious. If users don t feel like they are being taken care of, customer loyalty will not occur. The formatting and organization of the customer support page is unappealing, and while most of the information users need is available it is difficult to find. Recommendations Shutterfly menus need to reformat the Help hyperlink to make it more salient and easier to find. Shutterfly should also reorganize the customer support page. One suggestion would be to group similar items under the side bar to the left of the page to reduce the appearance of cluttering and messiness. 92 Page

93 Global (Principle-Based) Usability Issues A Global Usability issues are defined as reoccurring trends on a website that violate key human factors, website architecture, and heuristic principles. Identifying global issues allows researchers to identify systematic problems with a website or interface. Feedback Feedback is defined as the confirmation to a user regarding whether or not an action has/hasn t occurred. A website with good feedback keeps users in the loop in that it constantly gives the user a status of the system or actions carried out by the system. A website with bad feedback provides the user with little to no information about the status of a system or action. Providing users with accurate feedback is detrimental in insuring that a user is confident in their ability to use a website. Violation 1 One example of poor feedback on Shutterfly.com relates to the login process. Upon logging into an account on Shutterfly.com, the user is redirected to the My Shutterfly tab. After clicking the sign-in button on the interface the user is not given a confirmation of the action. The only way for a user to know if they have successfully signed into their account is through referencing the hyperlinks on the upper right corner. Redirection to the My Shutterfly tab is not only a poor transition, but can be confusing to users. Sign In 93 Page

94 94 Page

95 Feed Forward Feed forward is defined as the indication or implications to users about an action prior to committing that action. Interface elements such as buttons should somehow indicate to a user the implication of their action before that action is committed. A system with good feed forward communicates a process, communicates a work flow, and communicates a function or definition. Violation 1 One example of poor feed forward on Shutterfly.com relates to the photo upload process from social media sites. Not only is it difficult to find the screen through which a user can login to social media sites via Shutterfly.com, but upon entering that screen the user is unsure how or where they will be directed. Upon pressing on one of the social media icons, a new pop-up window comes up. This pop-up window requires that users enter their login information for the social media site chosen. Upon logging entering login information users are not redirected to the social media site selected, rather they are only able to view/select photos they have on those sites. In using social media to retrieve pictures on Shutterfly.com, users are unsure where they are going to be navigated until the action actually takes place. 95 Page

96 Clutter Clutter is defined as the degree to which the items on a website are organized or grouped in an efficient and visually appealing manner.cluttered websites can be frustrating, confusing, and distracting for users. Clutter can apply to menu items, photos, and general text. A noncluttered website is well organized, easy to navigate, and more user friendly. Violation 1 Customer support on Shutterfly.com can be accessed through the help tab. While the help tab has a variety of resources and features, the clutter on the page makes it difficult to navigate. Users are forced to spend an extended amount of time scanning the page before finding the information or help they are seeking. The clutter of the Help page on Shutterfly.com can be frustrating, confusing, and distracting for users. 96 Page

97 97 Page

98 Global (Principle-Based) Usability Issues A Global Usability issues are defined as reoccurring trends on a website that violate key human factors, website architecture, and heuristic principles. Identifying global issues allows researchers to identify systematic problems with a website or interface. Feedback Feedback is defined as the confirmation to a user regarding whether or not an action has/hasn t occurred. A website with good feedback keeps users in the loop in that it constantly gives the user a status of the system or actions carried out by the system. A website with bad feedback provides the user with little to no information about the status of a system or action. Providing users with accurate feedback is detrimental in insuring that a user is confident in their ability to use a website. Violation 1 One example of poor feedback on Shutterfly.com relates to the login process. Upon logging into an account on Shutterfly.com, the user is redirected to the My Shutterfly tab. After clicking the sign-in button on the interface the user is not given a confirmation of the action. The only way for a user to know if they have successfully signed into their account is through referencing the hyperlinks on the upper right corner. Redirection to the My Shutterfly tab is not only a poor transition, but can be confusing to users. Sign In 98 Page

99 99 Page

100 Feed Forward Feed forward is defined as the indication or implications to users about an action prior to committing that action. Interface elements such as buttons should somehow indicate to a user the implication of their action before that action is committed. A system with good feed forward communicates a process, communicates a work flow, and communicates a function or definition. Violation 1 One example of poor feed forward on Shutterfly.com relates to the photo upload process from social media sites. Not only is it difficult to find the screen through which a user can login to social media sites via Shutterfly.com, but upon entering that screen the user is unsure how or where they will be directed. Upon pressing on one of the social media icons, a new pop-up window comes up. This pop-up window requires that users enter their login information for the social media site chosen. Upon logging entering login information users are not redirected to the social media site selected, rather they are only able to view/select photos they have on those sites. In using social media to retrieve pictures on Shutterfly.com, users are unsure where they are going to be navigated until the action actually takes place. 100 Page

101 Clutter Clutter is defined as the degree to which the items on a website are organized or grouped in an efficient and visually appealing manner.cluttered websites can be frustrating, confusing, and distracting for users. Clutter can apply to menu items, photos, and general text. A noncluttered website is well organized, easy to navigate, and more user friendly. Violation 1 Customer support on Shutterfly.com can be accessed through the help tab. While the help tab has a variety of resources and features, the clutter on the page makes it difficult to navigate. Users are forced to spend an extended amount of time scanning the page before finding the information or help they are seeking. The clutter of the Help page on Shutterfly.com can be frustrating, confusing, and distracting for users. 101 Page

102 Cross Pollination Cross Pollination is an engineering psychology principle, which relates to the amount of access points available to users. While interacting on a website, users should have multiple links and buttons that navigate to the same page or function. Cross-Pollination is a positive feature of an interface, because it makes for easier access to areas of the website, as well as less of a chance to navigate in error. Violation 1 As mentioned many times in the report, Photo Books are the most popular product purchased on Shutterfly. With that being said, it s very important that key elements of the website should be accessible in many different ways. In the example shown above, 102 Page

103 Transparency Transparency on an interface refers to the level of clarity in which a user can connect, or understand with a shown function. For example, when a user sees a simple search box on a website, there is usually a very transparent meaning behind what the search box does (it searches for text). This example relates to every function that Shutterfly offers. If the website illustrates good transparency among icons and links, the user experience will be benefited. Violation 1 The below image shows the Pages and Photos menu on the design screen. Notice the two menu bars highlighted below, which share the same common area for photo and page displays. It was found that users had a hard time distinguishing the difference between functions of these two tabs. This type of transparency flaw was found on multiple areas of the website. Shutterfly is very useful for people that don t have experience with designing. With this in mind, the transparency of the functions on the website need to be really clear so users efficiently design products. Law of proximity The law of proximity is a perceptual psychology concept that is substantiated upon years of psychological research with humans. Taken from the Gestalt laws of grouping, it helps to explain why humans naturally perceive objects as organized patterns and objects this is how we literally make sense of the things we see. The law of proximity is a general rule that applies how people perceive visual stimuli, and the meanings from what they see, to how objects are displayed in relation to each other. As previously mentioned, the law of proximity states that things that are near each other seem to be grouped together, if they are not separated in any way. On other words, grouping things together according to distance is a way to organize them. To translate this to the context of the Shutterfly.com website, placing visual objects near each other implies to the user that those things are grouped together, and therefore related to each other in the purpose they serve. For example, if a check box is placed next to an answer on screen, one can assume that that check box corresponds with that answer, like so: 103 Page Yes

104 In a poorly-designed webpage, words may be placed on either side of a check box, with each answer an equal amount of close distance from the checkbox, like so: Yes No The user may be unclear whether they are checking Yes or No, because they could both seem related to the checkbox. This could create different reactions for people; some may interpret it correctly and how the web programmer meant for it to be interpreted, and some people may not. This can create a problem if people are not sure of what they re selecting, or get an unwanted outcome because of the misunderstanding created by this. In another case, it could cause the page to simply look confusing and cluttered because things are not organized well and lack distinction between elements, such as with space. Violation 1 In Shutterfly.com, many of the pages feel somewhat cluttered with images, text, ads, and other sections. As a result of the high volume of items on the pages, space is limited, and many objects are placed close together. In addition, these various items that are placed adjacent to each other are sometimes not related. This presents a problem; the user may assume that some items or sections are related to each other, due to their proximity, when in reality they are not. While there may be a reasonable intent to place the most important items in highly visible areas, such as the upper center portion of the screen, this may not have the desired effect as users are overwhelmed by the lack of organization, and users are not able to fully take in the many great features of Shutterfly.com. The objects tend to cancel each other out because there are so many, that the user sees none, or few, of them. In the screenshot below, a saved projects link, free shipping offer on any kind of product, a discount on photo books, a link to make a Yearbook, and a link to make photo books are all located in the same rectangular area, or outlining the edge. It is somewhat confusing to have these various clickable items placed together, especially because they are links to another page, rather than just a notice or announcement. The user may ask, where should I look to find all the information applicable to only Year Books? 104 Page

105 Violation 2 Violations of the proximity law are not just limited to objects placed close together that are not related. As mentioned previously, buttons can also be placed inappropriately far from others, making it hard for users to realize that it corresponds with a certain object. When users from the evaluation went through the check out process, they had to click an order button, which was somewhat far from the product. Although the users didn t seem to have a problem with figuring out how to order, this same issue in other parts of the website may lead to momentary confusion and delay. Consistency Consistency is keeping a website s interface elements and their layout consistent and predictable. There are a few different types of website design consistency. First, a website should stay consistent with other similar sites or applications. Second, a website should stay consistent with its own design. Violation 1 In terms of being consistent with the standards for websites in general, Shutterfly.com scored fairly well. However, Shutterfly lacks consistency within its website in some areas. First, it lacks consistency in the paths the user can take to make a Photobook. Also, the photo upload 105 Page

106 window from one path looks completely different in style and layout from the photo upload window from another path Visibility Visibility is a self-explanatory design term; it is the measure of the degree to which needed options and materials for a given task are visible. Good visibility means that the user can easily translate their goals into actions, through easily locating and utilizing the needed tools. Violation 1 Social media photo uploading for a Shutterfly product is a feature on Shutterfly.com that has very low visibility; one does not know from browsing Shutterfly.com that it integrates with Facebook and Instagram. There are simply no words or symbols on the site that indicate in any way that the sites integrate with social media, except for one symbol within the photo product creation process when one goes to add photos to the product. One novice user in the study was not able to find how he could access his Facebook photos; he eventually worked around it by selecting a product first, and then unintentionally saw the Facebook feature when the window popped up to allow him to upload photos from various sources. However, the other users figured out how to get their Facebook photos without any problems. Still, if one did not go through the photo product creation process, one might never know that Shutterfly offered the convenient feature of integrating with Facebook in order to pull photos to use in products. 106 Page

107 Local Usability Issues Local Usability Issues This section identifies the purpose of each key page in the sitemap, and then discusses every local usability issue of the page, the impact of these violations, and possible suggestions for improvement. Like global usability issues, local usability flaws hinder the natural flow of the website. These design flaws occur in the context of the page, and focus is given to features, elements, and widgets that produce page-specific usability issues. Rectifying these issues is of critical importance, since they affect the user s actions and desired goals on key pages in the Bath & Body Work s sitemap. Issue Impact Recommendations 1. It is recommended that various locations of obvious Photo Book access should be provided to users. There is currently advertisements in the main body of the MEDIUM There should be a main advertisement for Photo Books on the homepage that is static, and doesn t change. This would make for increased cross pollination of the very popular Photo Book section. It is also recommended to make important submenu items more visible, as the bottom submenu features were shown to lack effectiveness. homepage that contain Photo Book access points, however the advertisements change automatically. 2. The menus which shows examples (via pictures) of Photo Books need to allow users to focus on one template at a time. Users showed to be overwhelmed MEDIUM The function of sampling a template needs to afford the user better. Users need to be able to see that the sample tool is useful. Users should be able to view the Photo Books one at a time so that they may identify the one that suits their needs best. 107 Page

108 with the options, so a smaller filter needs to be provided. 3. Once a user selects a template to use, there is not a step by step process that tells what the user needs to do next in order to complete, or start working on a Photo Book. There is a lack of automaticity within the workflow of adding pictures. LOW There needs to be more of an automatic workflow for new users to learn the process of creating a new product, or when a product for returning customers is started. The function of the menus and tool in the Photo Book edit screens need to be more transparent. This could be helped by adding text or better iconography. 4. None of the participants found the Design Studio feature, which is located in the upper main menu of the homepage. Participants also responded negatively when asked about menu navigation during this process. The Design Studio may fall victim to the global issues related to menu structure. 5. The functions behind icons involved in editing and manipulating designs are not obvious to the user. High High Users need to see a process upon landing on the website. If they are new to Shutterfly, there needs to be options shown immediately. This could include a 1,2,3 step process to login to a social media account, then create products. A user could then naturally find the Design Studio section, as this would become part of the workflow. There needs to be one interface, or very similar interfaces for all product customizations. There needs to be more text, which would explain the functions behind what the product editing does. There should be more support for novice users for better efficiency while using the product editing tools. 6. Upon signing in 108 Page Shutterfly.com redirects Medium After signing in, users should be redirected to the homepage rather than the My Shutterfly tab. Having users return to the homepage, is

109 109 Page users to the My Shutterfly tab. From the My Shutterfly menu users deprived of the ability to choose the path by which they create their Photo Book (the custom path or the simple path). 7. The photo editing features on Shutterfly.com, while useful, are not prominent enough. These photo editing features are also scattered throughout the Edit tab. 8. The number and type of features on Shutterfly.com differ based on the navigation method to the page. The path through which users encounter the fewest customization options/path is through using My Shutterfly. We found that all users navigated to their photo product using My Shutterfly. The photo gift creation process lacks consistency. 9. While most users found it easy to connect to social media and upload a photo, High Medium Medium advantageous in many ways. Upon returning to the homepage users are able to see the special offers and promotions Shutterfly.com has to offer. Returning to the homepage after login will also allow users to take full advantage of all the customization and process features Shutterfly.com has to offer. Several changes in the grouping, naming, and formatting of several features on Shutterfly.com would provide users with the optimal experience. Of these changes may be renaming the tabs on the page users are redirected to upon choosing a Photo Book template. Specifically, the tabs Storyboard, Arrange, and Book should be changed to more descriptive titles. In regards to the editing tools, rather than having them spread across the page, they should be made available to users in one central location. In terms of the left side bar, placing text vertically makes it hard for users to read. The side bar should be redesigned so that it is salient and easy to read.. Consistency is important in terms of company branding and building user trust. Shutterfly should implement a photo gift creation process that is the same across all products and doesn t change based on the user s navigation path. Users should be aware of all the features and options available to them. Adding a link so

110 all participants had difficulty finding the social media link. Users aren t given the option to login to social media prior to creating a product, and the site does not indicate in any way that this is even an option. It is not until users press on the Add Photos button that they are allowed to login to social media accounts to retrieve pictures. social media on the My Shutterfly page and the homepage would allow users to know this feature is available to them. Under the edit tab a separate social media button should be placed next to the add pictures button to reinforce that using social media to upload photos is an option available to users. 110 Page

111 Appendices Appendix 1: Screening Application For Non-Shutterfly Users Screening Application for Shutterfly Research Project Non-Shutterfly.com Users First Name: Last Name: Phone: DOB: Please circle, or write in answers. 1) I have basic knowledge and experience using websites: Yes / No 2) I have a Facebook account, and would be willing to login to this account on our Lab PC: Yes / No (All data will be deleted regarding your login information) 3) I have an interest in Arts and Crafts/Photo Books/ Photography and Sharing of Pictures Yes / No 4) I have not used Shutterfly.com before: True / False 5) Have you heard of Shutterfly.com? If possible, please explain what you think Shutterfly is used for 111 Page

112 Appendix 2: Screening Application For Experienced Shutterfly Users Screening Application for Shutterfly Research Project Experienced Shutterfly.com Users First Name: Last Name: Phone: DOB: Please circle, or write in answers. 1) I have basic knowledge and experience using websites: Yes / No 2) I have a Facebook account, and would be willing to login to this account on our Lab PC: Yes / No (All data will be deleted regarding your login information) 3a) I have used Shutterfly.com before, and have created customized products on the website: Yes / No 3b) How would you rate your product design experience(s) you ve had on Shutterfly? (1 - very confusing and unproductive, 10 - great and very productive): 4) Please estimate how many products you have created on Shutterfly.com: 5) Explain any other tasks you ve completed on Shutterfly: 112 Page

113 Appendix 3: Testing Recruitment Advertisement Paid Website Research Study! We are seeking both experienced, and inexperienced users of Shutterfly.com Payment: $100 Time it will take: 60 minutes What you will do: Accomplish some basic tasks on Shutterfly.com When: Whatever fits your schedule! Where: SJSU Usability Lab How: Please call us at to make an appointment. If it sounds like you, or someone you know is a good fit for this research study, please timothy.visich@sjsu.edu for more information. 113 Page