MOBILE APP USER TESTING GUIDING THE WAY TO SUCCESS Mobile App User Testing 1
THE TWO PARTS OF USER TESTING > Usability Testing A systematic observation under controlled conditions to determine how well people can use the product or service. > Concept Testing Tests the appeal and potential success of a new product or service before it is launched. For mobile this means testing how people use a specific application or website on the mobile device. WHAT WE THINK Testing will help you identify: navigation errors (failure to find something), presentation errors (failure to understand something), control usage problems (failure to use something), and understand how people use the service and get user feedback on the general appeal of the service. Usability Testing should take place throughout projects from the initial design to the last sprint to give you time to address the issues during the design and development process. TYPES OF USER TESTING > Guerrilla Usability Testing Informal user involvement with little or no budget available. > Lab Usability Testing User involvement in design phase to understand needs and use patterns. > Contextual Usability Testing User involvement in the right environment. > Remote Usability Testing User research with the user and researcher connected remotely from different locations. WHAT WE THINK If you test the app with your target users during development, the chances are pretty high that you will develop something the users want and find engaging. By doing at least one type of User Testing, you avoid ending up with a product that only contains your own company s requirements. Mobile App User Testing 2
MINIMUM TESTING REQUIREMENTS 14 people that do not have a personal interest in the app. A distribution tool for test versions of your app such as TestFlight. A survey tool to get anonymous feedback from test users. A test plan including the testing activities listed in this white paper. A project plan that takes both user feedback and implementation of this feedback into account during all phases of development. WHAT WE THINK Involving users in the development process doesn t have to be hard. All you need are a handful of free tools and a commitment to collect as much user feedback as possible so that your app can reach its full potential. TESTING MATERIALS > Low Fidelity Prototyping Quick mockup of ideas in an interactive form that brings them to life to elicit feedback. It could be a paper or digital format based on rough sketches or wireframes. > High Fidelity Prototype Interactive tappable mockups presenting the main screens and main functionality of the app without real data. > Real App Anything from a fully developed, feature complete app to a product that is close to launch. WHAT WE THINK You can do User Testing during any phase of your project. From presenting conceptual sketches of your app idea to exploring sophisticated tools that record the user s screen interaction. It doesn t matter which one you choose, just find one that works for you. The most important thing is that you do User Testing to begin with. Mobile App User Testing 3
OTHER TESTING ACTIVITIES > A/B Testing Offering alternative versions of your product to different users and comparing the results to find out which one performs better. > Surveys A quick and easy way to get feedback from a large group of people. Ideally they include a set of yes/no questions and also gives users the opportunity to provide additional feedback. WHAT WE THINK You can put as little or as much effort as you want into User Testing, but be aware that the payoff will be relative to the time spent. We have heard many clients claim there is not enough time to carry out User Testing, but common sense dictates that this is a key aspect of app and mobile web development you do not want to skip. Mobile App User Testing 4
RADEEUS USABILITY TESTING CASE STUDY Radeeus, a music app powered by Spotify, asked us to help them improve the user experience of their app. They wanted to know if an improved user experience would increase downloads and app engagement. We began by recruting a Focus group of potential app users that fit Radeeus target market. We compiled a group of 4 users without a Spotify account and 4 Premium Spotify account users who were all in the 20-32 age range and used to listening to music online. Mobile App User Testing 5
OUR PROCESS 1. PLAN THE USABILITY TEST 7. INCORPORATE ACCEPTED RECOMMENDATIONS 2. CREATE SCENARIOS & TASKS 6. CREATE & PRESENT THE REPORT 3. RECRUIT PARTICIPANTS 5. ANALYZE TEST DATA 4. CONDUCT THE USABILITY TEST Mobile App User Testing 6
FOCUS GROUP TASKS TASK 1 Create a user name & Log in. TASK 2 Connect to your Spotify account. TASK 3 Search, add, delete and reorder songs. TASK 4 Interact with other users. METHODS We were looking to get feedback and first reactions to each screen, specifically focusing on these questions. > Do you know how to do this? Is it intuitive enough? > Is it easy to do this task? Tell us if some part of the process has been specially difficult or easy to achieve for you. > How can we make this better? Do you have any idea to improve it? > What do you see here? Tell us more about this screen and what you would do next. > What is being asked of you? What would you do next if you were trying to complete this task? > Is this what you would expect to see? What would you expect that button to do if you pressed it? > Is there anything you feel is missing from this screen? Could there be something else that helps you with the task or do you think there are some elements that are not needed? > How do you feel about this task? Do you feel confortable or unconfortable with the task? Mobile App User Testing 7
FINAL USABILITY FINDINGS Mobile App User Testing 8
FINDING 1 Users feel frustrated when they realize they need a premium account to use the whole app, without a preview of what the app provides. FINDING 2 Users feel frustrated when they realize they need 3 login steps (FB, username and Premium account). Mobile App User Testing 9
FINDING 3 Users are easily confused the first time they see the different types of icons (star, follow, and thumb). Users were quite frustrated about the fact they could not unlike a selection. FINDING 4 Users feel it is very complicated to access the Play queue. The song s screen does not look like a clickable element, and the Play queue is not intuitive. Mobile App User Testing 10
FINDING 5 The search method is not intuitive enough because the top bars only appear once the user has begun the search. If a user wants to search for a song, it won t be found because the app is searching for a user with the name of the searched song. Maybe the top bars should appear before having to insert text in the search box. Mobile App User Testing 11
CONCLUSION Based on our research findings, Radeeus made some key design changes to improve the user experience of their app. The resulting app is more streamlined and intuitive for Radeeus users. Preliminary feedback has been positive, and the app will soon be launched. In this case, a few small changes made a big difference to Radeeus core audience. In today s mobile first world, where app stores have an almost overwhelming number of options, it s important to launch an app well. Bringing a mobile app to market before doing good user testing is a sure way to drive potential users in the direction of competitors apps. Building a beautiful, useful app that your users will love takes a bit more time and planning, but the results will show the extra effort is worth it in the end. Mobile App User Testing 12
ABOUT DMI DMI One Rock Spring Plaza 6550 Rock Spring Dr. Bethesda, MD 20817 DMInc.com DMI Sales Team U.S. Sales: 855.963.2099 Int l Sales: 240.200.5848 sales@dminc.com DMI IS THE WORLD S LEADING PROVIDER OF ENTERPRISE MOBILITY SERVICES AND SOLUTIONS. We build enterprise class mobile solutions that generate results for the world s top brands and businesses. Our mobile solutions combine the awardwinning user experience design that has made us one of the top creators of consumer apps, with the deep middleware and engineering expertise that we ve used to build and manage enterprise applications for the most demanding IT departments in the world. DMI mobility solutions improve business processes, tap new revenue streams, build customer loyalty, and increase employee productivity. And we offer a full range of Managed Services to securely set up, configure, and manage your mobile devices. THE PROOF We ve built more than 400 mobile apps in the past 12 months alone for more than 150 leading organizations like Bacardi, Toyota, Vodafone, The National Guard, Novartis, Unilever and Universal Studios. We offer brilliant creative and user experience: Our mobile app development group was named the Best Branded App Developer at the 2013 Mobile Entertainment Awards. We have 750,000 devices under management for 100+ clients, including many Fortune 500 companies like Johnson & Johnson, Sears, The Associated Press, Allergan, and more. We provide 24 x 7 x 365 mobile service support. DMI is the one call our customers need to make to resolve any issue devices, apps, infrastructure, even carriers. We offer a full range of security options that include Federal-grade hardware-based security, two-factor authentication, secure container, and sophisticated encryption solutions. With our expertise and economies of scale, we can provide mobility management at a higher service level and on average 20%-40% lower cost than most companies can do on their own. Pervasive excellence is our commitment to quality service. DMI is one of only a handful of companies that is CMMI L3 appraised for both application development and services, as well as ISO 9001:2008, ISO 27001:2005, and ISO 20000-1:2011 certified. Our average D&B Open Ratings performance score from our clients is 95/100. We deliver secure mobile and ecommerce solutions for more than 50 leading consumer and business brands. Mobile App User Testing 13