PNC.com, Weather.com & SouthWest.com Usability Analysis Tyler A. Steinke May 8, 2014 IMS 413
2 P a g e S t e i n k e Table of Contents Introduction 3 Executive Summary 3 Methodology 4 Results 4 Recommendations 6 Conclusions & Take-aways 7 Attachments 8
3 P a g e S t e i n k e Introduction PNC is a banking institution specializing in online banking and banking solutions for everyday use. PNC.com heavily utilizes online banking capabilities by providing customers with the ability to log into their personal banking accounts and perform online banking transactions Weather.com is a weather service provider for The Weather Channel. Weather.com delivers live updates on what is happening in local areas based off of the user s IP address and provided zip code. In addition, Weather.com is a hub for local and national weather related news. South West is a commercial airline specializing in international and domestic flights. SouthWest.com is an e-commerce website with the purpose of buying flight tickets in addition to being a resource for customers who have already bought tickets and are checking details. Executive Summary I conducted an on-campus usability test at Miami University in Oxford, OH during the time period of May 1 st and May 8 th, 2014. The purpose of this test was to assess the overall design aesthetics and user experience of PNC.com, Weather.com and Southwest.com A total of 30 participants were involved in the usability tests to ensure stable results. Each individual session lasted approximately 5 minutes. The specific website being tested differed throughout the course of the week. In general the participants found that PNC.com lacked appealing aesthetic design and a quality user experience based upon an average aesthetic score of 1.75/5. Comparatively, participants enjoyed the appealing layout of Weather.com based upon an average aesthetic score of 4.5/5. Participants found the quality of SouthWest.com to be in the middle of PNC.com and Weather.com with an average aesthetic score of 4/5. The test identified a few minor problems for each company website including: PNC.com The lack of a responsive website. Confusion over multiple apps with similar functionality. Cluttered webpages with too much text. Weather.com A Chaotic home page. An unclear login feature on desktop, mobile, and app. Cluttered webpages and application with too much text. SouthWest.com The lack of a consistent color scheme across platforms. Multiple calls to action on the home screen of the desktop site. An unclear log in feature on desktop, mobile, and app. This document contains the participant feedback, satisfaction ratings, ease or difficulty of completion ratings and recommendations for improvements. A copy of the questionnaires are included in the Attachments section.
4 P a g e S t e i n k e Methodology When approaching the task of creating a usability test and analyzing the results, I wanted to be able to keep the tests simple and compare the results of different website usability tests. My intent was to draw conclusions and compare the usability of the websites based upon design aesthetics rather than on functionality alone. My focus on the design aesthetics and navigation of each website resulted in the use of the question in following categories: 5 second test, demographic background, mobile, and visual analysis. All of these questions were 5-point scale from 1-5, multiple choice, or free response. Sessions I recruited Miami University students and personal friends. I provided information on what the test was and the reasoning behind performing the tests. Participants responded with a time that worked for them and I accommodated their schedules. Each Session lasted approximately 5 minutes. During the session, I explained the test and asked each participant to fill out a questionnaire (see Attachment A). Participants read the questions and were given a brief amount of time to interact with the website. I asked each participant to rate their interactions based upon a 5-point scale from 1-5, multiple choice, or free response. Participants All participants were attendees at Miami University. Thirty participants were scheduled over the course of one week of testing dates. All thirty of the participants were involved in testing between May 1 st and May 8 th. Of the thirty participants, eight were female and twenty-two were male. Results The average results for each website are summarized below. PNC.com Weather.com SouthWest.com Load Time 4.25/5 4.7/5 4.1/5 Initial Interest 2.5/5 4.75/5 3.3/5 Navigation Satisfaction 3.25/5 4/5 3.25/5 Scanability 3.5/5 4.5/5 4/5 Website Login 3.75/5 2.5/5 1.5/5 Content 3/5 4.75/5 4.25/5 Design Layout 1.75/5 4.5/5 4/5 Color Scheme 2.5/5 4.25/5 4.25/5 Clear Call to Action 4/5 4/5 4.5/5 Likeliness to Revisit 3.5/5 5/5 4/5 Overall Quality 4/5 4.8/5 4.5/5
5 P a g e S t e i n k e The demographic results of the participants are shown below. Gender Mobile Device 0.00% 26% 74% 100% Male Female iphone Other Web Browser 10% Level of Education Some Highschool 20% High School Graduate Some College 70% Chrome Safari Firefox 100% Associates Degree Undergraduate Degree Graduate or Masters Degree
6 P a g e S t e i n k e Recommendations PNC.com Responsive Web design Based upon the user feedback and data results, the design layout needs to be drastically improved. The design scored a 1.75/5 which is a fairly poor score. My suggestion is to redesign the website using a responsive web design. Less Text and Clutter When users review PNC.com they expressed oral and written feedback stating there is too much text and clutter on the PNC website. I suggest taking away some of the clutter and text and restructuring the information architecture of the home page with better navigation. Develop Mobile Browsing Experience Participants found that the mobile website and mobile applications are very similar. Because mobile browsing and mobile applications are different experiences, I suggest creating a unique mobile experience rather than mimicking the application experience. Combine Applications PNC currently has two native mobile applications PNC Virtual Wallet and PNC Mobile. In order for the user to be more efficient, I suggest combining the two applications into a single app that is well thought and more user friendly. Weather.com Simplify Home Page Based upon the user feedback, users were unsure of what caught their attention first. Because of the different colors and areas of attraction, the participant felt overwhelmed. In order to fix this problem, I suggest, simplifying the homepage and incorporating local weather alerts into the design and layout of the page. Less Text and Clutter Upon review of Weather.com, participants expressed that there was too much content on the page to digest. There are many videos and articles relating to weather that did not interest the participant. I suggest taking away some of the additional content and focusing on the main information of weather forecast and auxiliary information that the users desire. Emphasize the Log In Feature Through interpretation of the test results, it is inferred that participants were not able to find the log in feature because they rated it as 2.5/5. In order to improve this, I suggest highlighting the login feature more in order for users to have a more rich and optimized experience. SouthWest.com Consistent Color Scheme Across Platforms When looking at the different at the desktop, mobile, and application for SouthWest, the participants noticed how different the color scheme and brand style were. Although the desktop website scored high in overall color scheme, I suggest either changing the desktop color scheme and design to match the mobile and application or changing the mobile website and the
7 P a g e S t e i n k e application to match the color scheme and design of the desktop website. This will create a stronger brand personality and enhance the brand recognition across platforms. Single Call to Action Currently there are many different items highlighted on the home screen of SouthWest.com. Some of these include the hero banner, the top navigation, and the search feature. It is my suggestion to place specific emphasis on one of these items in order to have a single call to action and increase website conversions. Differentiate the Log In Feature Participants who reviewed SouthWest.com found that the log in feature was hard to find at a score of 1.5/5. I suggest differentiating this feature in some way in order for returning customers who have purchased a flight to log in and check their flight information. Conclusions and Take-aways Based upon these tests I have drawn several conclusions about the field of usability testing. The first item I concluded is that in general there tends to be a positive bias for answers. If you look at the data most of the data is in the upper range. I attribute this to the participants lack of web design and user experience knowledge. Second, I found that different types of participant groups lead to different results. I chose to look only university students which lead to very similar answers from participants. If I would change part of my process, I would have diversified my participant group more. The third take-away I discovered from this test is that participants prefer simple and short usability tests rather than long and drawn out surveys. Often, throughout administering the tests, participants felt that there were too many questions and that it took too long. Finally, in the future I will use electronic surveys in order to more efficiently perform test, group results, and analyze the data. I have found that through performing usability tests and analyzing the results I have a better understanding of everyday usability. By crafting the usability tests for the participants I gained a better understanding of the elements that make up usability and how to communicate those elements effectively. Overall, usability is an underrated tool that should be used in order to improve all parts of life.
8 P a g e S t e i n k e Attachments A) Which website are you testing? PNC.com Weather.com SouthWest.com USABILITY TEST How would you rate the time it takes the site to load? (1 slow - 5 fast) Upon entering the site, how interested were you in the website? (1 low - 5 high) What is the first thing that stands out to you and why? What type of media were you most interested in? Video Images Text Other What was your satisfaction level of the navigation? (1 Bad - 5 Great) How would you rate your ability to scan the text on this site? (1 Bad - 5 Great) What was the first impression you had about the site? Were you able to log into the website? (1 Unclear - 5 Clear) Was there any functionality that you think the site is missing? How would you rate the content of the website? (1 Bad - 5 Great) How would you rate the design layout? (1 Bad - 5 Great)
9 P a g e S t e i n k e How would you rate the effectiveness of the color scheme? (1 Bad - 5 Great) How clear was the call to action? (1 Unclear - 5 Clear) Based upon your experience, how likely would you be to revisit? (1 Unlikely - 5 Very likely) If you recommend the website, how would you describe it? How likely are you to recommend this website? (1 Unlikely - 5 Very likely) What is your level of education? Some High School High School Graduate Some College Associates Degree Undergraduate Degree Graduate Degree or More How familiar with web browsing are you? (1 low - 5 high) Which Devices do you browse on during a normal week (choose all that apply) Smartphone (Normal Size) Phablet (5 screen size or more) Desktop Computer Laptop Computer 7 Tablet 10 Tablet Gaming Console / Smart TV What platform are you browsing on? Desktop / Laptop Tablet Phone What type of browser did you use? Chrome Safari Firefox
10 P a g e S t e i n k e Internet Explorer Opera Other How would you rate the overall quality of this site? (1 Bad - 5 Great) Native App Usability: Was it easy to log in to the application? (1 Hard - 5 Easy) How would you rate the esthetics of the mobile native app? (1 Bad - 5 Great) Were you able to find the information you are looking for? (1 Hard - 5 Easy)