User-Centred Evaluation Criteria for a Mixed Reality Authoring Application Marjaana Träskbäck, Toni Koskinen, Marko Nieminen Helsinki University of Technology, Software Business and Engineering Institute Information Ergonomics Research Group P.O.Box 9600, FIN-02015 HUT, Finland E-mail: Firstname.Lastname@hut.fi http://www.soberit.hut.fi/ierg Abstract Mixed Reality (MR) applications are beginning to emerge in the industry. In future these applications will increasingly become general in various application fields. Currently the threshold for this development is that the authoring of MR applications consumes a lot of time and resources. Current MR authoring environments are designed and restricted for dedicated experts only. In this article the objective is to specify the user-centred evaluation criteria for the MR authoring environment for the non-dedicated MR application developers ( MR authors ) in the context of content provider. The study is conducted in a EU-funded project called AMIRE. In order to offer MR authoring tool for wide use, it has to be consistent with the commonly used tools and it has to be intuitive to use. The authoring tool should offer way to reuse MR components to enable cost effective and efficient development of MR applications. 1 Introduction Authoring can be categorized by the amount of programming it requires from the authors and by the development style, which may or may not be primarily script-based. Many authoring systems assume that authors will posses a well-developed "programming mentality", and their applications consist of rigidly defined events in logical sequences and hierarchies. In some authoring systems applications are constructed by entering text into an editor, and then compiling or interpreting this. Other authoring systems have a graphical interface design editor, but require extensive scripting to describe interactions. Some authoring systems attempt to dispense with a script entirely and use iconic constructs or menu commands for almost all operations. (Davies, P., Brailsford, T., 1994) Currently authoring of MR requires a lot of time and resources. It has to be hard coded and the reusability of existing work is very low. The authoring in such a way is not very cost effective and efficient. The criteria for authoring presented in this paper consider the authoring in a componentbased environment. Through reuse, adaptation and combination of existing building blocks synergies of previous solutions can be exploited (Dörner, Geiger and Paelke, 2002). A component in this paper is a software component in the sense of the component theory (Sametinger, 1997). It is a building block of a software system with an explicitly defined interface designed for reusability (Dörner et al., 2002).
Mixed reality is a particular subset of virtual reality related technologies that involve the merging of real and virtual worlds somewhere along the virtuality continuum, which connects completely real environments to completely virtual ones (Milgram 1994). While mixed reality applications are increasingly coming to various application fields (medical, military, entertainment, industry etc.)(azuma, 1997, Azuma et al. 2001), more heterogeneous group of professionals take part in the development process. In order to offer them the possibility to develop MR applications, we need to support them with appropriate tools. We need to find out the context in which they will develop the application and the level of knowledge they have in developing MR applications. In this paper, we present user-centred evaluation criteria for MR authoring environment, which can be used by the non-dedicated MR application developers. These criteria are created from the point of view of an application content provider. In order to make the tools usable for the diverse users of the authoring tool we need to consider usefulness of the tool. Usefulness can be divided into two categories: utility and usability. Utility offers information on what is needed for the functionality of the system to be sufficient and usability offers information how well users can use that functionality (Nielsen, 1993). In ISO 9241-11 usability is defined as being the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241-11). In this study the users are content providers of a multimedia company and the context is the work done in their office with desktop PC s or laptops and occasionally a site visit to the customers environment. The case study is conducted in a EU-funded project called AMIRE (Authoring Mixed Reality). The purpose of the project is to create an authoring tool that enables non-dedicated MR application developers efficiently create MR applications. This requires that the authoring is possible without detailed knowledge of the underlying technology. In AMIRE, user-centred design approach is used to address this topic. 2 Objective of the Study The objective of the study is to specify the user-centred evaluation criteria for the MR authoring environment for the non-dedicated MR application developers ( MR authors ). The evaluation criteria will be used to assess the forthcoming requirements, features and the implementation of the MR authoring environment. The ultimate goal is to support the creation of such an MR authoring environment that enables widespread use of mixed reality features in future applications. 3 Method and Implementation of the Study The research method applied in the study resembled a contextual inquiry (Beyer, H., Holtzblatt, K., 1998). Contextual inquiry is a combined ethnography-based on-site interview and observation method. The method was used to study the context-of-use of the MR author (the developer person) in the MR provider organisation (in this case a company called Talent Code Oy). On-site interviews and field observations were conducted in the real working environment of the MR provider. Two persons were interviewed and four persons were observed throughout their working day. From the results of this study, a context of use description of the MR author was created according to the model presented by ISO 9241-11. This model consists of the description of the users, tasks, equipment and tools, and environment of the forthcoming MR authoring tool.
The user-centred evaluation criteria were created for the component based authoring tool that enables content providers to produce MR applications easily. Easily means here that a typical developer in an MR provider organization can create the required MR applications with his/hers existing skills and knowledge in a specified time. 4 Results The contextual inquiry in the MR provider organisation revealed requirements that have to be met for the authoring tool to be usable for developers who are not familiar with a detailed level of low level coding and implementation of the MR applications. According to the results from the study, following issues contribute the evaluation criteria for the MR authoring tool. 4.1 User Users of the forthcoming MR authoring environment are familiar with multimedia and web design. They are familiar with and use multimedia development application like Macromedia Director and MR authoring tool should use the common user interface (UI) conventions. One user stated: The user interfaces that are currently in use are good and we are used to use them. 4.2 Tasks and Tools Working with customers is typically iterative prototyping in such a way that intermediate demonstration applications are presented. Therefore, the tool should support intermediate previews of the application. During the project the customer needs to see the previews and snapshots of the future application. This is especially important in the project proposal phase when the customer commits the company for the project. Site visit is done at the beginning of the project to collect information, but the content creation and editing itself should be possible to do off-site, in the developer s office. If the work can be done in the authors own office environment; and if the customer can test the application on the site, the efficiency of the use of the MR application increases. Good usability on the behalf of the AMIRE Authoring Tools UI and interaction also makes the work more efficient. Content provision is handled on a high, domain specific level (domain = customer application domain), meaning that the author does not need to touch the source code. The tool should be intuitive to use and it should resemble other commonly used tools (Macromedia Director and Flash in the AMIRE project). It should also be compatible with these tools. One user stated: We should be able to connect the MR-objects with other environments, like html-site or flash. The maintenance of the application must be easy and off-site maintenance should be possible. Application has to be robust and error resistant. 4.2.1 Use of Components Components are the heart of the component-based augmented reality framework built in AMIRE. The idea of the components is the reusability of them. Always when creating new components these should be usable in several customer application domains. MR application developers must be able to efficiently create and organise applications with the existing components. They need to able find the specifications easily, to select components from the library and to include them in a new application, allowing them to view the system at a high level of abstraction. The components must be able to be customised to fit the particular requirements posed by other components and by the application itself. Authoring process must
ensure that editing the properties of the MR components is readily available using easy-to-use standard API (Application Program(ming) Interface) (i.e. property window in Macromedia Flash). The developer of a component-based application has to be able to connect the component also by using API. Insertion of a MR component into the framework and the building up connections between the components as well as changing the parameters needs to be easy. The following was stated by one user: I can change some parameters that are easily available, not in the source code. Applications may require some functionality that cannot be supported by the existing components. In order to add this functionality, application author must be able to ask for a new component from a component developer. The application developer must have a way to initialise and co-ordinate the gathering of the components needed for the application. 4.2.2 Programming The authoring tool has to abstract the programming processes by defining the interfaces and basic behaviour of components. The level of programming required from the MR Provider should be equivalent to writing Flash Action Script in Macromedia Flash. An expert in the application field who is not familiar with programming has to be able to create an application with the authoring tool, i.e. no low level programming should be needed (i.e. C++). High-level programming is accepted (i.e. Java script, ECMA, Flash Action Script, Lingo). As a graphic designer stated: I do not want to touch the source code. The compiling after changing the parameters should be automatic like in Macromedia Flash. 4.3 Environment From the organisational standpoint, the authoring tool should not change the MR author s current development, or production, process. The tool should also decrease the time required for the technical development phase of the MR applications. The production process cycles are short and there is a constant lack of time so the development of MR with current tools is not reasonable. Similarly, the tool should make cost-effective development possible; the authoring of MR should be handled with low extra costs. 5 Conclusions and Discussion MR authors and MR content provider companies have their special characteristics that need to be taken into account in the development of the MR authoring environment. The user interface of the MR authoring application should be consistent with the widely used development environments; multimedia authoring is a strong related field here, and it should offer intuitive, quickly learnable and easy-to-use metaphors and interfaces for the developers. Intuitiveness can be increased with proper feedback, error messages and with a help function (context-sensitive help). The use of common user interfaces will help the introduction of the authoring tool in the MR provider organisations. The tools must adapt to the surrounding organisational and process environment. Dramatic change in the production process can lead to the lack of use of the authoring tool. Cost efficiency and quick creation of iterative prototypes must be employed. With the results presented in this paper, it is possible to develop the authoring tool that is specially targeted for non-dedicated MR application developers in the MR content provider companies.
6 Current State of the Project At this point of the AMIRE project a prototype of the AMIRE authoring tool is developed. This will be used in usability evaluations and it will be developed iteratively. The first version will be used by the content provider to author a demonstrator and it will also be used in usability evaluations and user tests. Acknowledgements This work is conducted in the AMIRE project, which is funded by the European Commission in the IST programme (IST-2001-34024). The AMIRE website can be found at www.amire.net. References Azuma, R. (1997). A survey of Augmented Reality. In Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355-385. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B. (2001). Recent Advances in Augmented Reality. IEEE Computer Graphics and Applications 21, 6 (Nov/Dec 2001), 34-47. Beyer, H., Holtzblatt, K. (1998). Contextual Design. San Fransisco: Morgan Kaufmann Publishers, Inc. Davies P., Brailsford T. (1994) New Frontiers of Learning Guidelines for Multimedia Courseware Developers in Higher Education. UCoSDA (ISBN 1-85889-062-4), 1994. Dörner, R., Geiger, C. and Paelke, V. (2002). Authoring Mixed Reality A Component and Framework-Based Approach. First International Workshop on Entertainment Computing (IWEC 2002) May 14-17, 2002 Makuhari, Chiba, JAPAN ISO/IEC (1999) 9241-11 Ergonomic requirements for office work with visual display terminals (VTDs) Part 11: Guidance on usability ISO/TC 159/SC 4 Milgram, P., Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems, Vol. E77-D, No.12 December 1994. Nielsen, J. (1993). Usability engineering. Boston: Academic Press Inc. Sametinger, J. (1997). Software Engineering with Reusable Components. Springer Verlag.