Usability Evaluation of Cell Phones for Early Adolescent Users

Transcription

1 Yassierli*, Melati Gilang Industrial Management Research Division, Faculty of Industrial Technology, Bandung Institute of Technology Jl. Ganesa 10 Bandung Indonesia ABSTRACT:. The increasing number of cell phone user in Indonesia has been followed by the spread of its usage in early adolescent. This research was started with a preliminary study through questionnaires which were randomly distributed to students at several middle schools in Bandung. The result surprisingly showed that 87% of them (n=156) own and use cell phone with their usage preference on two popular brand, A and B. Based on this finding, a laboratory testing was then conducted to investigate usability issues related to the current design of these two cell phones if used by early adolescent, considering their association with possibility of language and computer literacy. Therefore the objective of this research is to evaluate the usability of cell phone A and B through measurements of effectiveness, efficiency, and satisfaction. A total of 16 participants were involved in the test, consisting of 8 novices and 8 experienced users. They were asked to complete 8 scenarios consisting of calling a number with and without phonebook, saving an incoming call, reading and sending short message via SMS, utilizing camera, activating games, and using radio features. Data collected during the test included time to complete each task, number of error, and rate of satisfaction using a questionnaire. Results of the usability testing showed that overall about 88% of the scenarios given were successfully completed, meaning that both cell phones had relatively good usability in term of effectiveness. This particular result was supported by participant s subjective opinion that showed individual mean score always greater than 3 (the scale of 1-5, with 1 negative attitude and 5 positive attitude). However, based on efficiency measures (completion time and error rate), phone B seemed to have better usability than phone A. This might be due to the uses of broad menu hierarchy and resemble icons to represent menu, folder, and feature options in phone B. This study also noted three main aspects that need to be considered in cell phone design for early adolescent users, which were terminology, icon recognition, and menu hierarchy. Recommendations were made related to the use of broad menu hierarchy, resemble icons, and simple terminology. Keywords: usability testing, cell phone, early adolescent, features. 1. INTRODUCTION Usability has become the major aspect that needs to be considered by designers as well as consumers in making or using products. Consumer seems no longer tolerate the hard using product, but, in contrast, even willing to pay more on a usable (ergonomic) ones. International Standards Organization (ISO) has defined usability as the efficiency and effectiveness to use, and also the ability to satisfy user in using the product (Jordan, 1998). Cellular phone is one sample product that requires usable design to meet its user needs. In Indonesia, the number of cell phone user has increased drastically every year. Data from Indonesian Cellular Phone Association showed that there were 75 million cell phone users in Indonesia until March About half of the cell phone market is currently dominated by the young people, especially those under 25 years old. It can be expected that the may be an increasing number of cell phone user in early adolescent segment as well. Various cell phone designs have also entered the market with the increasing features and complexity. This complexity may result in unusable product if ergonomics aspects in human- centred design are ignored. A previous study (Chotimah, 2001) revealed that usability aspects appear among of the most important things to be considered by teenagers in choosing and buying a cell phone. Therefore, it is interesting to find out if there are any usability issues of the current cell phone design if the users are early adolescent person. It is expected that this user segment keeps growing and will become a prospective one in the near future. The purpose of the present study was to usability aspects of two selected cell phones that being owned by early adolescents. We hypothesize that there may some design issues since the current design is probably not specifically considered their physical and cognitive capabilities as well as limitations.

2 2. METHODOLOGY This research was started with a preliminary study. Questionnaires were distributed randomly to students at several middle schools in Bandung. The result showed that 87.8% of them (n=156) owned and have used cell phone with their preferences on two popular brands, A (61.05%) and B (23.08%). The usage was not only related to the using of cell phone for the main functions (i.e. calling and sending short message SMS), but also for utilizing several features such as camera, games, and radio. Results of this preliminary study were then used for the experimental design of usability testing. The usability testing was conducted in Ergonomics Laboratory at ITB. The two most common cell phones owned by early adolescent respondents (based on preliminary study) were selected as the testing object. Dependent measures were determined according to the mentioned ISO definition of usability, including measurements of effectiveness, efficiency, and user satisfaction. The effectiveness was measured as how user can meet their needs by using the product. Here, a product can be considered as usable when 50-70% of given tasks were done successfully. The efficiency was quantified as task completion time and number of error while doing the tasks (objective data obtained from the experiment). An error was counted for every unnecessary step taken. Satisfaction was measured through a questionnaire in order to obtain user attitude and subjective opinion for the ease level, preferences, and usability aspect of each cell phone. A Likert-type design in scale of 1-5 was used with 1 representing negative attitude and 5 associated with positive attitude. Two groups of respondent were involved in the usability testing: 8 novice users and 8 experienced ones. Each participant had to complete eight task scenarios as follows: 1: Calling a given number 2: Reading a short message 3: Saving a contact in phonebook 4: Calling a number from the phonebook 5: Sending a short message 6: Utilizing camera feature to take a picture 7: Activating radio feature and set the radio frequency 8: Activating game feature The participants were compensated for their involvement. The task order was set balanced within each group. Two video cameras were used to record all participant activities while performing the task. All measures were compared to find out the performance differences between both user groups in using both cell phone A and B. Pair T-test was used to find out if there is a significant difference between group and between cell phone. All experimental conditions (e.g. physical environment) were set to be the same across task sessions. Recommendations were then made based on identifications of the error and user difficulties, and also the strengths and weaknesses of each cell phone. 3. RESULTS & DISCUSSIONS 3.1 Effectiveness measurement Table 1 shows percentage of participant who successfully completed each scenario for both cell phone. The percentage computed the summation of both groups. According to Rubin (1994), if less than 70% of respondents unable to complete the task successfully, the task considered as difficult task. The results suggested that only task no. 3 that could be considered as a difficult task, especially for cell phone B. Table 1: Percentage of successfully completed task Cell phone A Cell phone B 1 100% 100% % 93.75% 3 80% 60% % 100% 5 100% 93.33% 6 100% 100% 7 100% 100% % 100% 2

3 In short, about 87.5% of the tasks given were successfully completed using both cell phones. Rubin (1994) stated that when 50 up to 70 percent assigned task are well completed using the product, then it is considered as usable one. 3.2 Efficiency measurement Results pertaining to the efficiency of cell phone which was measured by completion time and error rates are shown in figure 1 and figure 2. The data demonstrates differences between groups and between cell phones. It can be seen that the performance of the experienced group was better than the novice in general, although the difference was not statistically significant (p>0.05). The experienced group made less error and demonstrated shorter completion time than the novice one. In general, the performance of two groups seemed to be better when using cell phone B rather than A. This is based on completion time as well as error made. However, it should be noted that, again, the difference was not statistically significant (p>0.05). Performance Mean Time (second) A - A - B - B - Figure 1: Mean of completion time for each scenario Error Rate A - A - B - B - Figure 2: Mean of error for each scenario 3

4 3.3 User Satisfaction Measurement Results of the questionnaire are listed in Table 2, Table 3, and Table 4. It can be seen that the individual mean score for each task is always greater than 3. This suggests that participants, in general, provided positive subjective opinion on the cell phone in terms of ease of use, preference, and usability aspects. Table 2: Mean ratings on easy of use Cell phone A Cell phone B Cell phone A Cell phone B Table 3: Mean ratings on user preference Cell phone A Cell phone B Cell phone A Cell phone B Table 4: Mean scores of usability aspect No Usability Aspect A B A B 1 Navigation Screen Design & Lay Out Terminology Feed back Appropriateness to user task Comparisons between the cell phones and recommendations Interestingly, this study found out that there were two basic differences in user interface design between the two cell phones as follows: Menu hierarchy. Cell phone A was associated with deep menu hierarchy, while cell phone B used broad menu hierarchy. It seems that the latter is easier for early adolescent users. This may be one reason why cell phone B was better than cell phone A in terms of efficiency. Icon. In cell phone B, the design of icon menu was based on resemblance icon, for example, joy stick figure for game menu. In contrast, almost no such icon is available in cell phone A which seems to be characterized with text icon. Again, this icon design did help early adolescent users in browsing the menu. This may be another reason why cell phone B was better than cell phone A in terms of efficiency. Regardless this finding, it is not known why cell phone A is much more popular than cell phone B. Results of this study also noted that issue of terminology used in both cell phones. The reason why the task no 3 was difficult to be completed is due to the missing menu for save the number. It seems that this 4

5 task can be considered as a basic function. However, no direct menu for save the number is available after a phone call to- or from a non-recognized number is made. 5. CONCLUSION This study has noted several usability issues regarding of the current cell phone design owned by early adolescent users. Several recommendations have also been provided, especially for manufacturers and designers. However, some limitations are worth noting. This study was only focused on user-interface aspects, not including physical dimensions. Moreover, only two cell phones were investigated. The selection was done based on those commonly used by early adolescent persons taking consideration a limited preliminary study. It should be noted that a lot of types and variations of either cell phone A or cell phone B are available in the market. REFERENCES [1] Chotimah, R (2001). Usability Testing of cell phone with a focus on learnability. Unpublished thesis. Industrial Engineering Program, ITB, Bandung, 2001 [2] Indonesian Cellular Phone Association (2007). Indonesia Celullar Show 2007 Remaja dan Eksekutif Muda Paling Menggiurkan. (accessed on August 13, 2007). [3] Jordan, P. W. (1998). An Introduction to Usability. London: Taylor & Francis Ltd. [4] Nielsen, J. (1993). Usability Engineering. Boston: Academic Press, Inc. [5] Rubin, J. (1994). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York: John Wiley & Sons 5