Are Senior Smartphones really better?

Transcription

1 Are Senior Smartphones really better? Nina Ascher, Stephan Schlögl Dept. Management, Communication & IT, MCI Management Center Innsbruck Abstract Giving our aging society it seems important that technical devices be also adapted to elderly users. Especially with smartphones there is a need for optimization (e.g. larger icons, simpler labels, etc.) as they may offer numerous benefits to senior citizens (e.g. more independence and security in everyday life). The goal of our work was to let seniors assess the usability of such an optimized smartphone and compare it to a more traditional model. Participants were given ten tasks to be completed both with a senior smartphone (i.e. the emporiasmart) and a traditional smartphone (i.e. the iphone 5S). Collected data included performance measures (i.e. completion rates, error rates, issues and times per task) as well as preference data (based on questionnaires and a short interview). Results show that the senior-friendly smartphone did out-perform the traditional device in all categories. In particular, the bigger icons and simpler navigation led to the device being perceived as more usable. 1 Introduction In 2017, 18.6% of the Austrian population was over 65 years of age a number which, according to forecasts, will grow to 25% by At the same time the use of smartphones has increased significantly, showing that in 2017 already 68% of Austrians aged 15 years or older owned a respective device 2. Given that smartphones are not only used as a means of communication between teenagers (or for work) but may also improve the overall quality of life, we also see a growing uptake in the senior citizen target group (Pak & McLaughlin, 2010). Yet, given potential age-related restrictions, elderlies often face problems when having to use traditional smartphones, whose features are usually optimized for the younger consumer market 1 [23/12/2017] 2 [10/01/2018] Veröffentlicht durch die Gesellschaft für Informatik e. V in R. Dachselt, G. Weber (Hrsg.): Mensch und Computer 2018 Workshopband, September 2018, Dresden. Copyright (C) 2018 bei den Autoren.

2 518 Ascher, N., Schlögl, S. (Fisk et al., 2012). Although there are some devices which specifically target elderly users, evaluations scrutinizing their level of senior-friendliness are still scarce. 2 Methodology The goal of our work was to let relevant end users assess the usability of a senior smartphone and compare results to a more traditional smartphone. That is, we asked 15 seniors to perform ten different tasks both on an emporiasmart 3 and on an iphone 5S, and collected feedback. In addition we recorded a number of prevelant usability messures. 2.1 Participants A total of 15 participants (9 female) aged between 60 and 78 years (M=69; SD=5.62) took part in this study. Only one of them had no previous experience with smartphones. One owned a Windows phone, three owned phones running Apple s ios and the rest (i.e. 10 participants) owned phones based on Google s Android operating system. 2.2 Tasks and Procedure To evaluate the usability of our two test phones (i.e. the emporiasmart and the iphone 5S), we designed ten common tasks (i.e. Tasks A J), each of which had to be performed on both devices. With the tasks we also thought about potential error opportunities (cf. Table 1), which eventually helped us compute an (admittingly artificial) error rate. Task Description Error opportunities Task A Create a new contact Enter first name incorrectly Enter last name incorrectly Enter first name and last name in one field Enter number incorrectly Task B Task C Call a contact from the contact list Set the alarm clock to a defined melody Call a wrong contact Do not call (just select) Set alarm time incorrectly Wrong alarm melody Alarm clock off Task D Write a WhatsApp message Select wrong contact 3 [02/06/2018]

3 Are Senior Smartphones really better? 519 Write empty message Write wrong message Do not send Task E Take a photo Make a video Take a wrong photo Task F Delete a photo Delete wrong picture Edit image Task G Add a birthday to the calendar Wrong date Wrong name Wrong reminder time Do not select birthday Do not save Task H Open a website Enter wrong website Error when entering the link Task I Task J Read a text message (i.e. SMS) Turn the smartphone into silent mode Table 1: Tasks and Respective Error Opportunities Read wrong text message Do not open text message Read WhatsApp message instead Use the +/- switch Choose another audio profile Tests were performed at participants homes, where they were first introduced to the test procedure, then asked for informed consent, and eventually given some time to familiarize themselves with the devices (Note: for the two tests the order of devices was counter balanced between participants so as to account for potential order effects). As for the emporiasmart this familiarization included a short tutorial called touch training, which each of the participants was asked to complete. Once ready, participants started with the above described tasks, each of which was followed by a short post-task questionnaire (Note: if a task could not be completed within a timeframe of 7 minutes, it was aborted and regarding the test statistic considered as failed). A final post-test questionnaire completed the test of a device. The same procedure was then applied with the second device. That is, participants who started with the emporiasmart moved to the iphone 5S, and those who started with the iphone 5S moved to completing the ten tasks with the emporiasmart. Finally, after testing the second device, participants were asked for their feedback regarding the devices user-friendliness. In total the entire procedure took approx. 2 hours per participant.

4 520 Ascher, N., Schlögl, S. 2.3 Measures Collected metrics included performance data (i.e. time per task), completion rate, error rate and encountered usability problems. Completion rate was measured binary (i.e.1=completed; 0=failed), and for the error rate we took the error opportunities identified for each task as a baseline. Also, subjective preference data was collected via two questionnaires, i.e. the SEQ questionnaire (Sauro & Dumas, 2009) to record perceived task difficulty ratings after each task and the SUS questionnaire (Brooke, 1996) to measure perceived usability after each device. Finally, a short interview concluded each test. The following result discussion focuses on task completion, experienced problems, and perceived usability (as we believe those are most relevant to this workshop). A more in-depth analysis of the remaining metrics shall, however, be subject to a future publication. 3 Results Table 2 shows the task completion rates and their respective adjusted Wald confidence intervals for both devices (Sauro, 2010). Table 3 and Table 4, respectively list the main problems identified with the two devices. Task Completion Rate emporiasmart adjusted Wald confidence interval Completion Rate iphone 5S adjusted Wald confidence interval Task A 93% 68% 99% 60% 36% 80% Task B 100% 82% 100% 93% 68% 99% Task C 93% 68% 99% 67% 42% 85% Task D 93% 68% 99% 87% 61% 98% Task E 93% 68% 99% 100% 82% 100% Task F 80% 54% 93% 87% 61% 98% Task G 73% 48% 90% 27% 10% 52% Task H 93% 68% 99% 20% 6% 46% Task I 100% 82% 100% 93% 68% 99% Task J 13% 2% 39% 0% 0% 18% Table 2: Device related task completion rates and their respective adjusted Wald confidence intervals Focusing on the emporiasmart, three issues seemed to be particular prevalent. The first of which concerns the audio profile, which was not found in the emporiasmart setting and thus prevented 11 of the 15 participants from turning the phone into silent mode (i.e. Task J). Also, Task H, i.e. opening a website, posed some challenges, as two participants were not able to identify the field into which they should input the address. Finally, three participants had difficulties finding special characters on the device s keyboard.

5 Are Senior Smartphones really better? 521 Number of Participants who Percentage and Problem experienced this problem Confidence Interval Audio Profile in Settings 11 71% (48% 90%) In input link not clear where 18% (2% 39%) 2 to enter Keyboard special characters 24% (6% 46%) 3 Table 3: Identified Problems with the emporiasmart Focusing on the iphone 5S, completion rates were consistently lower than with the emporia device. Particularly interesting seems the fact that none of the participants managed to put the phone into silent mode. And also tasks G (i.e. adding a birthday to the calendar) and H (i.e. opening a website) appeared to be rather difficult. With respect to concrete problems the tests identified six relevant issues inherent to the iphone. First, Safari was not recognized as an Internet browser, which might explain why many participants were unable to open a website. And also the calendar app was not recognized as such. Next, the keyboard was perceived as too small, so that (similar to the emporia phone) some of the participants did not find special characters. As for Task C (set the alarm clock) participants were confused to read No alarm clock and thus believed they opened the wrong application. Another significant problem concerned Task J (i.e. turn phone into silent mode), where approx. a third of the participants used the plus/minus buttons on the side of the device instead of the switch. Problem Number of Participants who Percentage and experienced this problem Confidence Interval Safari not recognized as internet browser 59% (36% 80%) 9 No alarm set 4 29% (10% 52%) Keyboard special characters 24% (6% 46%) 3 Turn phone silent with +/- 5 35% (15% 59%) No calendar entry 6 41% (20% 64%) Keyboard too small 4 29% (10% 52%) Table 4: Identified Problems with the iphone 5S The results of the SUS questionnaire support the better usability of the senior smartphone. With an average of (SD=18.11; CI=+/-10.03) the emporiasmart scored significantly higher than the iphone 5S (M=54.17; SD=23.45; CI=+/-12.98). A t-test confirmed this difference in perceived usability; t(28)=0.012; p<0.05.

6 522 Ascher, N., Schlögl, S. 4 Discussion Our results confirm a better usability for the senior smartphone model. From a performance point of view such is supported by significantly better completion rates. That is, adjusted task completion rates of the emporia device are all except for one higher than 70%, while with the iphone this level was reached by only half of the tasks. One reason for this difference may be found in the bigger sized icons and apparently simpler labeling of applications. The fact that labeling should be adapted so that it speaks the user s language (Leung et al., 2012) was particularly apparent during Task H where with the emporiasmart 93% of the participants were able to complete the task as the browser was labeled internet, whereas with the iphone only 20% understood that Safari is the right application to be used. Also, with respect to the preference-focused SUS, the emporia device showed significantly higher scores. The post-test interviews confirmed this result. In particular the large icons and the bigger keyboard were perceived as positive. Only the silent mode seems to pose some challenges both with the emporiasmart as well as the iphone device. Yet, putting a phone into silent mode seems to be generally a rather uncommon task among seniors. 5 Conclusion and Future Work In summary, our study results hint towards a better usability of the emporiasmart when compared with an iphone 5S. Aspects of effectiveness, efficiency and satisfaction were measured, all of which led to significantly better scores for the senior optimized smartphone. Yet, our study setup did not account for learning effects, which may be an area of further research. References Brooke, J. (1996). SUS - A quick and dirty usability scale. Usability evaluation in industry, 189(194), 4-7. Fisk, A. D., Rogers, W. A., Charness, N., Czaja, S. J. & Sharit, J. (2012). Designing for Older Adults: Principles and Creative Human Factors Approaches, Second Edition (2nd ed.). Hoboken: Taylor and Francis. Leung, R., Tang, C., Haddad, S., Mcgrenere, J., Graf, P. & Ingriany, V. (2012). How Older Adults Learn to Use Mobile Devices. ACM Transactions on Accessible Computing, 4(3), Pak, R. & MacLaughlin, A. (2010). Designing displays for older adults. Boca Raton, FL, USA: CRC Press. Sauro, J. (2010). A practical guide to measuring usability: 72 answers to the most common questions about quantifying the usability of websites and software. Denver, CO.: Measuring Usability LLC. Sauro, J., & Dumas, J. S. (2009). Comparison of three one-question, post-task usability questionnaires. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp ). ACM.