Usability & User Experience: Preliminary Results from Evaluating an IPTV Community Platform

Transcription

1 Usability & User Experience: Preliminary Results from Evaluating an IPTV Community Platform Marianna Obrist 1, Sara Kepplinger 1, Elke Beck 1, Manfred Tscheligi 1, and Paul Muzak 2 1 ICT&S Center, University of Salzburg, Sigmund-Haffner-Gasse 18, 5020 Salzburg, Austria {marianna.obrist, sara.kepplinger, elke.beck, manfred.tscheligi}@sbg.ac.at 2 Telekom Austria TA AG, Lassallestraße 9a, 1020 Wien, Austria paul.muzak@telekom.at Abstract. The promises of IPTV and related changes of the traditional TV environment are broadly discussed in the EuroiTV community. In this paper we present preliminary, qualitative results from a user evaluation conducted for an IPTV community platform focusing on usability and user experience issues. Our results show that major obstacles in the first phase of developing an IPTV community application are the usability of the system as well as the motivation of people to share personal audiovisual content. Keywords: IPTV, user evaluation, community TV, user-generated content. 1 Introduction and Background We re entering an era in which people are participating rather than just receiving information (Jonathan Swartz, in [8]). IPTV could bring a breath of fresh air into local communities by providing new ideas for sharing and creating content, exploring new ways of doing things, and enriching the perspective of community members on local topics and social interaction. Strengthening the relation building among people via the television is one of the possibilities for the IPTV technology and earns attention [7], in particular considering trends such as user-generated content [3]. If we trust prognoses, personalization, personal TV as well as personal or local content will play a bigger role in the near future [8]. Field tests, like for example within the Lommel TV project [10] in Belgium or SAMBA in Brazil [11] have already collected experiences on community related TV services and applications. Leading players are already stepping into the market by offering services related to community or social topics [6] (e.g. Amigo TV Application [12]). Although, usability and user experience research in this area is still in its infancy. Within this paper, we try to fill this gap by presenting the results from a user evaluation of an IPTV community platform. 2 User Evaluation A first user evaluation of the IPTV platform (which was developed as part of the CITIZEN MEDIA project) was conducted in November The focus of the user M. Tscheligi, M. Obrist, and A. Lugmayr (Eds.): EuroITV 2008, LNCS 5066, pp , Springer-Verlag Berlin Heidelberg 2008

2 304 M. Obrist et al. evaluation was to discover usability problems as well as to gather preliminary, qualitative feedback from users experience with the community platform. 2.1 Tested IPTV Application The evaluated application was developed for supporting a local community in creating, uploading and sharing audio-visual content. The application, based on the DSL broadband IP technology, is an integrated part (walled garden see Figure 1) of the service range of a local telecommunication provider. In a longterm field trial, which started in January 2008, a limited number of test households have access to the IPTV platform and can create and share videos and pictures with other community members. Fig. 1. Start Page of the IPTV Application On one hand, the evaluated IPTV platform (see Figure 2) supports the traditional viewers role in front of the TV. On the television itself the user can watch the uploaded audiovisual content (either his own or of other users) and rate the content as well (e.g. rate the quality of a picture, rate the friendship with other users, rate the quality of a video). Another feature is the possibility of searching for certain other users and adding them as a friend. On the other hand, users can take an active role by using their PC or mobile phone. Through the PC people can upload videos, pictures and edit their profile. People are able to use their mobile phone to take pictures and display them on the IPTV platform in real time. Fig. 2. Multiple IPTV Platform Environments The decision of having multiple media in the IPTV environment builds on the idea to support the pre-experiences of people with each device. Existing media experiences with each device are supported instead of replaced by new interaction concepts.

3 Usability & User Experience: Preliminary Results from Evaluating an IPTV Participants A broad distribution of age, household structure and media experience was used for the recruitment of the test participants. The intention was to have a good representation of the demographics of the local community addressed within the CITI- ZEN MEDIA project. Eleven test participants five female and six male at the age between 23 years and 64 years participated in the evaluation of the IPTV platform. The average age was 40,73 years (SD 14,227). Two single households participated, four participants live in pairs, and four participants live in a three-person household and one participant counts four household members. All of the participants had Internet access. Three participants had three TV sets at home, four participants had two TV sets, two participants had one device and one participant had no TV at home anymore. Every participant used (at least) one mobile phone. In sum, the majority of the participants had basic experience with the TV, Internet, and mobile phones. Additionally, each participant was a member in at least one offline or online community. 2.3 Test Set Up The study was conducted in a laboratory environment using video and audio recording as well as eye-tracking (see Figure 3). Each participant was tested alone in a one-way mirror room, equipped with a classical IPTV set top box and remote control, connected to a flat screen TV (CRT-TV). Fig. 3. Laboratory Usability Test Environment 2.4 Procedure The test participants took part in a one and a half hour evaluation. Every participant carried out six realistic and typical tasks with the IPTV platform in front of the TV. The functionalities available through the PC (video upload) and through the mobile phone (picture upload) couldn t be included due to technical constraints. A main focus of the tasks performed was on searching and navigational issues. For each task, common data such as success rate, time needed and errors occurred were collected. Apart from a questionnaire addressing the socio-demographic data, two additional usability and user experience questionnaires were used (i.e. SUS System Usability Scale [2] and AttrakDiff [3]). Moreover, two qualitative interviews (a preand post-interview) were conducted. Additional data was collected through eyetracking in order to underline identified usability problems. Due to the limited space available, we will not present the eye-tracking results in this paper. The main results are summarized towards identified usability problems and first impressions and experiences users had with the platform.

4 306 M. Obrist et al. 2.5 Results Several problems, mostly related to the design of the user interface, but also related to the labeling and the general navigation on the IPTV platform were identified in this first user evaluation Success Rate and Usability The following table gives an overview on the number of tasks successful solved, the tasks solved with help, not solved tasks and the average rating of the degree of difficulty of each task on a scale from 1 to 5 (where 1 = very easy and 5 = very difficult). Table 1. Task / Success Overview Task Solved Solved with Help Not Solved Task Rating T1: Search for a User Profile ,42 T2: Find and watch a Video ,00 T3: Find the first Task of the Game ,25 T4: Add a New Friend ,92 T5: Find and rate a Picture ,67 T6: Look at the Game High Score ,83 Participants had the most problems to solve task 3 due to design and labeling problems (mainly related to navigation and orientation problems), as they couldn t find the task for the game offered within the platform. The SUS questionnaire revealed a score of 55,7 points (from 100) for the usability of the IPTV platform. Consequently there is room for improvements of the system and to support the users requirements Overall Impression and User Experience The results from the AttrakDiff Questionnaire [3] (indicating how users experience the system on a pragmatic and hedonic quality level) revealed that the user interface of the IPTV platform was generally rated as neutral. There is still a need for improvements to make the system more attractive to the users. In the post-interview, the majority of the participants were critical about the IPTV platform, but had also a lot of sympathy for the offered features and the provided content. Following statements give some insights in the overall impression participants had of the platform: As I am not a fan of pictures I would rather use that for the exchange of messages like in Skype in earlier days. Perhaps like for example: Ask a question to the community and expect an answer (female, 28) or Well, when I can show pictures to my friends, I would use it, if I am sure, that just my friends can see them (male, 62). Overall it can be summarized, that features or elements motivating the participants to use the platform were mostly community related. Participants mostly liked the possibility of getting to know other people and to look at their pictures. The friends list and picture gallery were mentioned in this case as motivating features for the usage. Features of the platform, which discourage the participants to use the platform, are mainly concerned with the usability of the platform. The participants most mentioned the fussiness of the platform and a lack of meaningful labeling of links, as well as lack of

5 Usability & User Experience: Preliminary Results from Evaluating an IPTV 307 transparency. On the question what shouldn t be changed on the platform, the participants most often mentioned the possibility to watch videos or pictures of friends and adding friends to the personal space at the platform. 3 Conclusions and Outlook In this paper an IPTV community platform was evaluated by focusing on usability and user experience aspects, which are influencing the acceptance of a new system. From previous research we know, that aspects related to ease of use community design both related to features, content variation and usability are important. Seventeen percent of the people that stop using communities or use communities less than before explain this fact by usability issues and 24 % by content issues [1]. Therefore it is important to take usability issues into account throughout the whole design and development process. As part of a one-year field trial just started in the project selected test households will be involved in future in situ evaluation activities. We will address user experience issues and in particular also sociability of the IPTV platform, as well as the question how the deployed IPTV application could enhance the community experience and social interaction in a rural environment. References 1. Brandtzæg, P.B., Heim, J.: Initial context, user and social requirements for the CITIZEN MEDIA applications. Participation and motivations in off- and online communities. P IST-41, CITIZEN MEDIA, D (2007), 2. Brooke, J.: SUS: A quick and dirty usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L. (eds.) Usability Evaluation in Industry. Taylor and Francis, London (1996) 3. Hassenzahl, M., Burmester, M., Koller, F.: AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Teubner, B.G. (ed.) Mensch & Computer 2003: Interaktion in Bewegung, pp (2003) 4. Jensen, J.F.: User-generated Content a Mega-trend in the New Media Landscape. In: Lugmayr, A., Golebiowski, P. (eds.) Interactive TV: Shared Experience, TICSP Adjunct Proceedings of EuroITV2007, pp (2007) 5. Nielsen, J.: Participation Inequality: Encouraging More Users to Contribute (2006), 6. Schmitt, D.: European IPTV: Market assessment and forecast. Screen Digest, UK (2005) 7. Weber, J., Newberry, T.: IPTV Crash Course. Mc Craw Hill Companies, NY (2007) 8. Wellman, B., Hogan, B.: Connected Lives: The Project. In: Purcell, P. (ed.) Networked Neighbourhoods. The Connected Community in Context, pp Springer, London (2006) 9. Zu Salm, C.: Zaubermaschine interaktives Fernsehen? TV-Zukunft zwischen Blütenträumen und Businessmodellen. Gabler Fachverlage, Wiesbaden (2004)