Automated Usability Evaluation during Model-based Interactive System Development

Transcription

1 Automated Uability Evaluation during -baed Interactive Sytem Development Sebatian Feuertack 1, Marco Blumendorf 1, Maximilian Kern 1, Michael Kruppa 2, Michael Quade 1, Mathia Runge 1 and Sahin Albayrak 1 1 DAI-Labor Techniche Univerität Berlin Ernt-Reuter-Platz 7,10587 Berlin Firtname.Latname@DAI-Labor.de 2 Deutche Forchungzentrum für Kuentliche Intelligenz GmbH Tripptadter Str.122, Kaierlautern Firtname.Latname@dfki.de Abtract. In thi paper we decribe an approach to efficiently evaluate the uability of an interactive application that ha been realized to upport variou platform and modalitie. Therefore we combine our Multi-Acce Service Platform (MASP), a model-baed runtime environment to offer multimodal uer interface with the MeMo workbench which i a tool upporting an automated uability analyi. Intead of deriving a ytem model by revere-engineering or annotating creenhot for the automated uability analyi, we ue the emantic of the runtime model of the MASP. Thi allow u to reduce the evaluation effort by automating part of the teting proce for variou combination of platform and uer group that hould be addreed by the application. Furthermore, by teting the application at runtime, the uability evaluation can alo conider ytem dynamic and information that are unavailable at deign time. Keyword: model-baed uer interface development, automated uability evaluation. 1 Introduction Uer interface (UI) are more and more required to upport everal context-of-ue. They need to be able to be run on everal platform, conider different type of uer and adapt to variou uage ituation. Thi poe new challenge when it come to the development of interactive application a well a their evaluation. In thi work we preent our approach combining a model-baed runtime ytem with an Automated Uability Evaluation (AUE) tool to provide the ability to evaluate UI that adapt at runtime. In order to attend to thee iue we combined two approache: The Mental (MeMo) workbench, a workbench for AUE and the Multi-Acce Service Platform (MASP), a model-baed framework for UI generation. -baed UI development approache [1, 2, 3, 4] already upport the generation of multi-platform uer-interface a well a context-of-ue adaptation. They contain emantic tored in a well-tructured form of declarative deign model. Thi allow tool to ait

2 developer at deign-time by detecting quetionable feature and by offering the help of automated advior. However, mot of thee approache do not conider ad-hoc adaptation to the context-of-ue which can only be calculated at runtime. The MASP allow the derivation of a UI from a et of executable model [5], defining the uer interface tate. Beide having the poibility to decribe adaptive UI, the model and the tate information can alo be utilized to upport the AUE of the UI. By working with abtract UI model, which ideally contain all required concept, the UI could be generated for any platform and thu uability evaluation could be done by conidering any platform without being forced to redefine the concept of the evaluation target. After we give an overview of related work in the reearch field of automated uability evaluation in the next ection, we elaborate on the combination of the two approache. We illutrate the reult of the evaluation uing the interactive Cooking Aitant we developed baed on the MASP (the Cooking Aitant ha alo been deployed a a demo application in our Ambient Aited Living Tetbed [6]) and we conclude with a ummary and outlook in ection 5. 2 Related Work AUE method can upport the evaluation proce, wherea they differ ignificantly in their degree of automation and the effort for evaluator [7]. The majority of AUE method i uually applied on already exiting ytem or prototype and therefore require re-contructing a ytem interaction model by revere-engineering or manually annotating the emantic of already exiting application [8, 9, 10]. The CogTool [11] i a tool to predict execution time for certain tak. The max model [12] i conidering cognitive apect via uer imulation for meauring acceibility of information within web ite and the PROSKIN project [13] i tracking uer data in order to aggregate it to higher-level profile to gain peronalized UI deign. The AIDE [14] tool focue on organizing the control of an interface by incorporating five metric (efficiency, alignment, horizontal balance, vertical balance and contraint) into the deign proce, while initial automated aitance ha been propoed by USAGE [15]. Furthermore, tet have been developed for certain apect of completene, conitency and command-reach ability [16]. -baed interface development environment, uch a TADEUS [17], upport imulation and modelchecking by tranlating the dialog model into a Petri net. In contrary to thee approache we are moving the AUE from deign-time to runtime in order to enable the evaluation of ad-hoc context-of-ue adaptation a well a conidering ytem dynamic that are unknown at deign-time, uch a data querie. We ue the MeMo workbench [18] to imulate different uer profile that perform certain tak and benefit in the way that uability iue are uncovered for a wide range of poible uer. Further on, we can imulate uer performing more error a uual to diagnoe the ytem behavior which i difficult to predict by real peron within complex ytem. The evaluation proce of the MeMo workbench i baed on a cognitive walkthrough (CWT) carried out by a uability expert and include a rule engine which contain a et of modifier rule extracted from the CWT methodology. Compared to the related work, our approach offer the following benefit:

3 No need for a manual re-creation of the pecification of an application which i fragile for introducing miundertanding and incompletene. All combination of platform and uer that are addreed by the interactive application can be efficiently targeted to an automated uer interface evaluation. By utilizing the model-baed run-time ytem, the evaluation can conider ytem dynamic and parameter, and context-of-ue variation. 3 -baed Automated Uability Evaluation In order to automate the uability evaluation for everal context-of-ue cenario, each pecifying a combination of a pecific platform, a certain group of uer and condition of the environment, we replaced the ytem interaction model (SIM) of the MeMo workbench with the runtime model of the MASP. The imulation tart with the MASP generating the initial repreentation of the uer interface for a certain context-of-ue. Thi repreentation i ent to the MeMo workbench by delivering a ytem interaction tate (SI tate) which conit of the current enabled et of interaction input and output tak. Baed on thi information and the current uer profile, the MeMo workbench chooe an interaction which i related to an input tak of the MASP. The correlated uer action will be performed and a new SI tate i generated until the uer goal ha been accomplihed. 3.1 Sytem Interaction State Generation In the MASP we are interpreting tak tree that define the temporal relation a the baic interaction flow for the interactive ytem. A domain model complete the tak model by providing content for the tak. The model define the data tructure and hold intance of thee tructure which are object that become acceible at runtime. The life-time of thee object i determined by the tak model, which alo reference the object in the deignated tak a we decribed in detail in [19]. Modifying object of the domain model happen either through the ervice model (1), connecting backend ervice to application tak (2) or by uer interaction (3) while entering and changing information. Uer interaction i mediated by the interaction model, detailing the interaction tak (4). Here we ditinguih input interaction tak (IIT) and output interaction tak (OIT), which identify the interaction on the highet level of abtraction. While OIT require no human intervention but preent information to the uer until they become diabled by another tak, IIT require human intervention uch a data input. The tak are alo annotated with the object that are read, modified, created or declared and refer to the related clae of the domain model. A reification of the interaction in term of detail i provided by the interaction model. It encloe an abtract interaction decription that i modality independent and a concrete interaction decription that add the modality dependent information. Additionally, by mapping between the interaction and the layouting model (5) preented in [20], abolute poition and element ize of the concrete interaction object are calculated baed on the context model (6) and filled with

4 information delivered by variou enor at runtime. Thu, each SI tate will be compoed of encapulated et of enabled tak. Each atomic tak i linked to the concrete domain object cla and the relevant cla attribute which are preented in the interface together with the expected uer operation (read, create or modify) (3). Further, the relevant interaction model element and the calculated layouting information are added for each enabled tak of the SI tate (5). Finally a cacading tyle heet defining the tyle for the graphic element of the concrete model i attached to the SI tate. Service (1) (2) Tak Domain Layouting (4) (3) (5) Interaction (6) Context Fig. 1. Overview of the involved runtime MASP model, eparating multiple level of abtraction. Each model comprie (part of) the runtime tate(). The relation between the model i formalized by mapping which upport the exchange of information between the model and keep the model ynchronized. Figure 1 how the different model and how they are related. At runtime each model contain additional tate information, allowing the derivation of the final UI according to the tate of the application. Through the utilization of mapping between the model, we upport an information flow at runtime, which allow identifying active element of the model baed on the enabled tak et. The derived SI tate allow the creation of a final UI which can be evaluated by an AUE tool a decribed in the next ection. From the AUE perpective, the runtime interpretation of the UI model create a SIM containing SI tate, which decribe the interface and interaction capabilitie of the evaluation target. The UI element contained in a SI tate are communication element with different interaction, e.g. a button can be ued with left click or right click a input interaction, wherea the information of the caption i carried through an output interaction from the ytem to the uer. Every input interaction and the ucceive SI tate, which i enabled after the interaction, are encapulated in a tranition to the ubequent SI tate. In the imulation phae the SIM i mapped to a dynamic SIM repreentation which i handed over to the imulator for interaction with the uer interaction model (UIM). The imulator continuouly preent SI tate to the UIM and map the input interaction, choen by the UIM depending on the current uer tak, to the correponding tranition of the SIM. 3.2 Uer Action Generation After the MASP preent the SI tate to MeMo, the information proceing unit (IPU) of the UIM evaluate all input and output interaction and manipulate the data torage which i ued by the planning unit for the final interaction election proce.

5 Both module are influenced by two type of uer attribute (UA). Static UA expre the characteritic of a uer group and cannot be influenced by the imulation proce, wherea dynamic UA (DUA) are flexible propertie of a uer. Fig. 2. The component of the uer interaction model and the interaction election proce. A hown in figure 2 the knowledge of the UIM i divided into three categorie. While the tak knowledge decribe the knowledge which ha to be exchanged to uccefully accomplih a tak, general knowledge decribe knowledge all uer group have in common and domain knowledge depend on the ytem to be evaluated, e.g. expert know where to look for required information in the ytem. During the analyi proce, the UIM ha to handle four different ituation: receiving new information (1), already known information (2), known information with differing value (3), or a requet for information (4). Situation 1 and 2 are eay to handle. Either the UIM already know what the ytem i preenting or the UIM tore the preented information in it data torage. Situation 3 i a mimatch between the knowledge of the UIM and the preented information by the ytem. The UIM can react in different way. If the mimatch refer to the tak knowledge, the UIM tore the wih for correction in it working memory, otherwie it domain or general knowledge will be corrected. If the requeted information in ituation 4 i contained in it knowledge, the UIM i able to repond to the requet. After analyzing the output interaction, the rule engine i called to modify the dynamic uer attribute according to DUA rule. The DUA influence the intention of the UIM, e.g. high frutration or irritation level increae the probability of a tak abortion, time preure reduce the probability for requeting help. Four intention are implemented o far: forward, cancel, help and abort. If the UIM ha the intention forward, it trie to tranfer it tak knowledge to the ytem and therefore prefer interaction which upport the tranfer. In cae that no interaction i preferred the UIM trie to navigate in order to analyze further tate. If the UIM i navigating and believe further navigation i not poible or reaonable the intention turn to cancel and the UIM trie to navigate back. In cae it could find neither interaction for knowledge tranfer nor meaningful interaction for navigation, the intention become help. The intention abort lead to an abort of the current tak. After the intention alteration the UIM tart to evaluate the available input interaction. Auming the intention i forward, the UIM trie to tranfer it tak knowledge to the ytem and therefore prefer interaction which upport the tranfer.

6 Thee interaction are et up with a higher probability of election. In cae that no interaction wa preferred by the IPU, the planning unit ytematically evaluate the UI object for navigational functionalitie (navigation object) in correlation with it knowledge and increae their interaction election probability. If the intention i not forward, the UIM trie to find navigation object related to it intention, e.g. the intention i help it prefer button labelled with?, i or Help. In cae that no interaction related to help or cancel can be identified, the intention will be et to abort and the UIM give up. Finally each interaction i et up with a probability of election, wherea the preferred interaction have higher probabilitie than alternative interaction. In order to modify thee probabilitie, the rule engine analyze the complete SI tate and generate fact for numerou attribute of the UI object. A dice throw elect the interaction according to the probabilitie. The elected interaction i given back to the MASP which generate a new SI tate. The interaction election proce will be tarted again until the tak i finihed or the UIM abort the proce. 4 Evaluation Proce and Reult All data collected during the imulation i captured with the help of a logging module and tored in log file. With the help of an internal view the deigner gain acce to imulation detail for each tak, uer group and iteration of the imulation. Thi way, the deigner can retrace every interaction the UIM ha choen. Thi information conit of the interaction element, the lit of triggered rule, the probability ditribution of the available interaction and further tatitic, e.g. execution time. A portion of thee data i viualized within an interactive graph (compare figure 3). Each node of thi graph repreent a SI tate which ha been paed by the UIM and each tranition repreent the choen interaction. Deviation from the hortet goal driven path are highlighted in different color Thi help the deigner to eaily uncover problematic SI tate and to find reaon for the deviation, becaue in the current tate of implementation the workbench i not offering thi level of critique by itelf. A decribed in ection 1 the MeMo workbench evaluated the interactive Cooking Aitant (CA) which i preented to the workbench by the MASP. In the following, evaluation reult of everal imulation run are decribed. With the help of the rule engine, everal limitation of the CA regarding it uability were expoed. Figure 3 illutrate a problematic iteration in which the UIM accidently choe a different meal from a lit of preented meal and did not dicover thi fact immediately in the tate RecipeDetail. In fact the probability of thi interaction i low, but the conequence might caue cot for the uer, e.g. buying ingredient for a different recipe a intended. In the ubequent tate a dialog in which the number of peron to whom the meal hould be prepared for wa preented to the UIM. In thi tate the name of the meal i not diplayed and therefore the UIM did not dicover it wrong meal choice. The ame problem occur in the tate ShoppingLit, where the UIM can decide whether to continue with the CA or prepare a hopping lit with the neceary ingredient. Finally the UIM i able to dicover it wrong meal choice in the lat tate of figure 3.

7 Fig. 3. An interactive graph which diplay uer interaction and preented SI tate. In thi dialog the information of the choen meal i diplayed and returned to the uer a output interaction. The only input interaction to undo the election i via a link to the tart page in the upper left corner of the creen. A a reult of it non-conform coding a a link it might not be recognized a an interaction element and the rule engine therefore reduce it interaction probability. Within a couple of iteration the UIM did not chooe thi interaction which ha the conequence that the uer find no proper interaction to correct the meal election and therefore abort the tak. Another reult of the evaluation wa the low contrat of the font compared to the underlying interaction element and furthermore the low contrat of ome button compared to the background, e.g. a mall white font wa ued on light blue button. The rule engine dicovered thi fact and reduced the interaction probabilitie for uer group repreenting elderly people with poor viion. Thee uer group had a higher probability of not finding the appropriate interaction element and therefore chooing an alternative interaction that deviated from the hortet goal driven path. Finally, uing tandard label for ome interaction element (e.g. Next ) to achieve higher coherence could improve the uability a well, which wa confirmed by further imulation done with the help of the workbench. 5 Concluion and Outlook In thi paper we preented our approach to utilize uer interface deign and runtime model to upport an AUE of the modeled ytem. Utilizing the model allow kipping the uually required manual annotation of the final UI with the underlying deign concept. The approach alo make the evaluation of multi-platform and context adaptive uer interface much more traightforward becaue the automated ytem i able to make aumption about context parameter or ued platform. Thu it can evaluate multiple variant of the UI much eaier. We have alo hown the evaluation of one of our application which revealed everal uability problem. For the near future we plan to extend the approach to conider more detail of the UI decription a well a to conider multimodal apect of the UI and extended context information (e.g. voice-baed feedback could help noticing the election of a wrong recipe earlier). We would like to upport the evaluation of different context ituation, e.g. imulating migrated or ditributed UI. Along with the challenge, we would alo like to enrich the perception of the UIM by concerning further mental

8 apect. Finally our goal i to automatically make contructive uggetion (beyond critique) for improving the uability. Reference 1. Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., and Vanderdonckt, J. A unifying reference framework for multi-target uer interface. Interacting with Computer, 15(3) (2003) 2. Coninx, K., Luyten, K., Vandervelpen, C., Van den Bergh, J., and Creemer, B. Dygime: Dynamically generating interface for mobile computing device and embedded ytem. In Mobile HCI (2003) 3. Mori, G., Paternò, F., and Santoro, C.. Deign and development of multidevice uer interface through multiple logical decription. IEEE Tran. Softw. Eng., 30(8) (2004) 4. Reichard, D., Forbrig, P., and Dittmar, A.. Tak model a bai for requirement engineering and oftware execution. In Proceeding of TAMODIA Blumendorf, M., Lehmann, G., Feuertack, S. & Albayrak, S. Executable for Human-Computer Interaction. In Proc. of DSV-IS Blumendorf, M., Feuertack, S., and Albayrak, S. Multimodal mart home uer interface. In Proc. of IUI4AAL Workhop on IUI 2008 (2008) 7. Ivory, M.Y. and Heart, M.A.. The tate of the art in automating uability evaluation of uer interface. ACM Comput. Surv., 33(4) (2001) 8. Tarta, A. and Moldovan, G. Automatic Uability Evaluation Uing AOP. IEEE International Conference on Automation, Quality and Teting, Robotic, IEEE Computer Society, 2, (2006) 9. Ritter, F. E., et al. High-level behavior repreentation language reviited. In ICCM Paternò, F.; Piruzza, A. & Santoro, C. Remote web uability evaluation exploiting multimodal information on uer behaviour, Springer (2006) 11. Teo, L. and John, B. E. Comparion of keytroke-level model prediction to oberved data CHI '06: CHI '06 extended abtract on Human factor in computing ytem, ACM Pre, (2006) 12. Lynch, G., Plamiter, S., Tilt, C.. The max model: A tandard web ite uer model. In 5th Conference of Human factory & the Web (1999) 13.Fine, N. & Brinkman, W. Informing intelligent environment: creating profiled uer interface EUSAI '04: Proceeding of the 2nd European Union ympoium on Ambient intelligence, ACM, (2004) 14. Sear, A.. Aide: a tep toward metric-baed interface development tool, In UIST Byrne, M.D., Wood, D., Sukaviriya, P.N., Foley, J.D., and Kiera, D.E.. Automating interface evaluation. In CHI Conference Companion (1994) 16. Braude, R.E. and Sibert, J.L.. Conmod: a ytem for conceptual conitency verification and communication. SIGCHI Bull., 23(1) (1991) 17. Elwert, T. and Schlungbaum, E.. ling and generation of graphical uer interface in the tadeu approach. In DSV-IS (1995) 18. Jameon, A., Mahr, A., Kruppa, M., Rieger, A., and Schleicher, R.. Looking for unexpected conequence of interface deign deciion: The memo workbench. In TAMODIA Feuertack, S., Blumendorf, M., and Albayrak, S. Prototyping of multimodal interaction for mart environment baed on tak model. In European Conference on Ambient Intelligence: Workhop on Driven Software Engineering for Ambient Intelligence Application (2007) 20. Feuertack, S., Blumendorf, M., Schwartze, V., and Albayrak, S.. -baed layout generation. In Proc. of Advanced Viual Interface (2008)