Designing Interaction Styles for a Mobile Use Context

Transcription

1 Designing Interaction Styles for a Mobile Use Context Steinar Kristoffersen 1 and Fredrik Ljungberg 2 1 Norwegian Computing Centre, Postboks 114 Blindern, N-0314 Oslo, Norway steinar@nr.no 2 Viktoria Institute, Box 620, SE Gothenburg, Sweden fredrik@viktoria.org Abstract. Direct manipulation, which is the dominating interaction style for mobile computers, fails to meet the conditions of many mobile use situations. In particular, it demands too much visual attention of the user. We introduce a new, complementing interaction style (and system) for mobile computers, called MOTILE, which addresses three main requirements of interaction with mobile computers: (1) no visual attention needed; (2) structured, tactile input, and; (3) the use of audio feedback. MOTILE relies on only 4 buttons for user input and hands free audio for feedback. Keywords: Mobile computers, interaction style, interface design. Introduction In the traditional office setting, the primary work tasks of users are inside the computer. The reason is simple: office work is to a large extent about processing documents, spreadsheets, s, etc., and those artefacts are inside the computer. In a mobile use context, this is not usually the case. Maintenance workers, sales personnel, and other mobile staff do not primarily process information inside the computer. Their main tasks are outside the computer, e.g., meeting clients, repairing broken equipment, and so on. Many of today's problems in the use of mobile computers derive from the fact that their interfaces are designed from the same principles as desktop computers. Desktop computers are designed on the premises of direct manipulation, with files and folders, drag and drop, and so on. Direct manipulation is an interaction style that demands a high degree of visual attention of the user. This may be suitable in the office (where the primary tasks are inside the computer), however it is not always the most suitable design guideline for the mobile computer. The reason is that direct manipulation paradigm is too exclusive for the mobile use context: it demands that the users attend to the virtual world inside the computer to an extent that does not agree with the conditions of the mobile use context, e.g., walking, driving, and repairing. In this paper, we analyse the problem of direct manipulation in the context of mobile computers, derive requirements for interaction styles of mobile computers, and introduce our candidate, called MOTILE. MOTILE is an interaction style and

2 system that seeks to complement current paradigms of interaction styles for mobile computers. The rest of this paper is structured as follows: we first describe the mobile computer and its interface, followed by an exploration of the problem of direct manipulation in the context of mobile use. We derive a set of requirements that interaction styles of mobile computers need to meet, that served as a foundation for the design of MOTILE, which we describe next. Lastly, we review related work and conclude the paper. Interacting with mobile computers In this section, we describe the current interaction styles for mobile computers, the problems that these seem to cause, and some requirements of interaction styles in mobile use contexts. The mobile interface Mobile computers (or PDAs) have up to now largely been used asynchronously. For user input, current models rely on keyboard, pen, or both. Three main categories of operating systems for mobile computers exist. Palm Pilot computers run on the Palm OS, Psion computers, and new mobile phones and hybrids (e.g., Ericsson R380) use EPOC, while Palm top computers like the Cassiopeia (Casio) and Ericsson s MC 16 run on Windows CE. All rely to a considerable extent on the direct manipulation interaction style. An interaction style defines the ways in which user input and output can take place. Primary interaction styles for the PC are menu selection, form fill-in, command language, natural language, and direct manipulation [3]. Direct manipulation is the dominating interaction style for desktop computing. In a direct manipulation environment, the user operates the computer by pointing, selecting, dragging, etc., visual objects on the screen, and the results of the operations are immediately visible on the screen. The basic components of direct manipulation interfaces are windows, icons, menus and point device [2], which is the reason such systems sometimes are called WIMP systems.

3 Figure 1. The three dominating mobile computer operating systems: EPOC, Windows CE, and PalmOS. Direct manipulation demands a high visual attention of the user. Consider, for example, the OAI (Object-Action-Interface) model of direct manipulation, which is frequently discussed in the literature. The OAI model [3, p. 205] suggests three principles: 1. Continuous representation of the objects and actions of interest with meaningful visual metaphors. 2. Physical actions or presses of labelled buttons, instead of complex syntax. 3. Rapid incremental reversible operations whose effect on the object of interest is visible immediately. The consequence of these principles is an interface that is very dependent on video. To input information, the user needs to find the right visual object on the screen, perform the operations (clicking, dragging, etc., on the right place), and receive video feedback ( perceiving the system state, see [4]). Let us now relate the dominating operating systems of mobile computers to the three principles of the OAI model. Continuous representation of the objects and actions of interest with meaningful visual metaphors. All three conform to this principle. Physical actions or presses of labelled buttons, instead of complex syntax. All three conform to this principle. The keyboard on most mobile computers is only used for word processing, and similar applications, not for navigating the operating system. Rapid incremental reversible operations whose effect on the object of interest All three conform to this principle.

4 The mobile use context The mobile use context is often heterogeneous, and for that reason it may be difficult to make general claims about it [1]. However, in many mobile situations, such as in mobile maintenance work in energy and telecommunication companies that we have studied, there is a set of common features of the use context, which in important ways differ from the traditional context of the office. Let us compare these two use contexts, using the categories of task, hand, attention, and dynamics: 1 Task The mobile user is often engaged with tasks outside the computer, e.g., implementing new equipment in the field. The office user, on the other hand, is often engaged in tasks inside the computers, e.g., manipulating a spreadsheet or writing a document. 2 Hands The mobile user often uses the hands to manipulate physical objects, e.g., tools and equipment, while the office worker often easily can place the hands on a keyboard. 3 Attention The mobile user may be involved in tasks outside the computer that demand a high level of visual attention, e.g., to avoid danger or monitor progress. The office user, on the other hand, can often easily direct a large degree of visual attention to the computer. 4 Dynamics The mobile user may move during the task, as opposed to the office user who often performs the task in one single location (e.g., writing a document at the desk). As we can see, the mobile and the stationary use context differs from each other in important ways. This suggests that interface guidelines from the office setting may not be suitable in the mobile setting, the reason being that the two use contexts simply are too different. Nevertheless, current mobile computers are equipped with interfaces and interaction styles from the office. This calls for new interaction styles that complement direct manipulation in the mobile environment. What requirements should such an interaction styles satisfy? Requirements When designing interaction styles for mobile computers, there seems to be at least three issues to consider. Using the terminology of Norman [4], these are: Executing actions should not demand a high degree of visual attention, among others, because, first: many users type on mobile computers with one finger on each hand, which is very difficult without looking at the keyboard;

5 second, localising the visual objects on the screen is very demanding visually. This calls for input methods that do not demand much video. Such methods seem likely to be very simple and structured. Perceiving the systems state should demand no or little visual attention, among others, because it may be very difficult to find a place for the mobile computer that makes the screen easily available during the entire work process. This calls for feedback and output methods that demand little or no video. Such methods may rely on audio, which is another implication. In most mobile situations, including rather extreme environments, users can rely on audio feedback. These implications served as the foundation for the design of the MOTILE interaction style method described next. MOTILE MOTILE is a technique (interaction style) and a system for operating mobile computers. It is based on the three implications discussed above: no or little visual attention, structured, tactile input and the use of audio feedback. MOTILE relies on only 4 buttons for user input, and hands free audio feedback. The technique comprises binary look-up in sets of virtual keyboards, currently with keyboards for: text input, moving the cursor, reading text and selecting links from web-pages, as well as reading and sending . The user encodes input by selecting and pressing regions on a touch screen. North switches between keyboards, East and West selects the next higher or lower half of the keyboard, respectively, and South executes the command (e.g., types a character or takes the user to the selected web-page). MOTILE informs the user about options and execution via a voice synthesiser. The visual channel that dominates the direct manipulation paradigm is replaced with a less obtrusive audio feedback conduit. MOTILE relies on semantic call-back, which entails using the system as a control panel, or ultra-thin client that sends encoded instructions to dedicated servers, rather than performing actions locally. Figure 2 shows an overview of the architecture. MOTILE consists of an ultra-thin client running on the palmtop terminal. The client communicates orientation (N, W, S, E) using a BreezeCom wireless LAN to the MOTILE server running on a Unix host. The simple messages are translated into selection, navigation and execution of semantic codes, which are organized in virtual keyboards. A perl script parses and returns elements of HTML from URLs. The users are continuously offered feedback on their actions via an echoserver, which uses a simple audio synthesiser to broadcast messages to a microwave radio. A headset is connected to the radio terminal carried by the user, thus affording non-obtrusive feedback through the audio channel. A separate server handles mail commands which it relays to sendmail. The current version of MOTILE is implemented for Windows CE using Waba and Solaris servers, but any platform dependencies are external to MOTILE itself; the

6 primary obstacle in this respect was finding drivers for Wireless LANs, and an effective voice synthesiser. The main contribution of the current MOTILE technique is twofold: First, it offers an input device platform on top of which to continue experimenting with new ways of operating mobile devices, and Second, it demonstrates that the visual feedback channel taken for granted in the direct manipulation paradigm may indeed be challenged. We are currently in the process of setting up a proper evaluation of the system. Thus far mainly the authors have been experimenting with the system during development only. Besides establishing that the technical configuration works well, very limited proof-of-concept may yet be elicited from this. We have found, however, that typing, reading and sending , browsing and selecting web-pages moving the mouse cursor works, albeit slowly and requiring more concentration (mental, rather than visual) compared to the desktop-based direct manipulation counterpart. An interesting avenue for further exploration is already established, namely to attempt using a combination of intelligent profiling based on use patterns, to re-use frequently types combinations, and tangible bits [5] to reduce the stress on the user during input operations. Getting audio feedback via the synthesiser and radio, rather than visual feedback, seems to work rather well. Echo server North, South, West, East Semantic coding MOTILE Mail server SMTP URLs Texts & links Http parser Virtual keyboards Figure 2. MOTILE.

7 Related work There are many contributions in HCI on multi-media and multi-modal input. Nomadic Radio, for instance, is a wearable system for providing background awareness for mobile users [6]. It is distinguished from MOTILE, which relies on audio as a directed foreground feedback mechanism concerning the state of the work and available operations. In most other HCI research, audio is viewed as a means for input. Our concern, on the other hand, is chiefly to use audio as a feedback mechanism. The issue of interfaces on mobile computers has been debated in several recent panels. However, the perspective has mainly been one of structured output in combination with open voice input [7]. The project reported in this paper, aims to support open audio output and structured tactile input. Our concern has not been auditory cues [8], although we of course see them as a useful complement to the design of MOTILE. Contributions on input devices for drawing, pointing and dragging complies with the direct manipulation paradigm [9], and we have chosen not to go into these in any detail here. One promising approach is to consider the integration or separation of input devices in terms of applications versus the mechanics of the system [10]. Rather than looking at perception as the key element, however, as Jacob et al. [10], we have looked at the social practices involved in the mobile situations that we aim to support. Conclusions We have explored the problems of applying direct manipulation in the mobile use context. We noted that the mobile use context in important ways differs from the office setting, which in turn has implications for the design of interfaces of mobile computers. Some important requirements that the interface of mobile computers need to meet are: no or little visual attention, structured, tactile input and the use of audio feedback. These requirements guided the design of MOTILE, a new and complementing interaction style (and system) for mobile computers. MOTILE offers a complementing interaction style for heterogeneous use contexts, in particular those where tasks other than operating the mobile computer may be the most important, users hands may be otherwise occupied, users may be involved in tasks outside the computer that demand a high level of visual attention, and users are highly mobile whilst performing the task. The direct manipulation paradigm seems too exclusive for the mobile use context. It rests upon the idea that users can direct very much of the attention to the computer, which, thus, circumscribes the task users want to accomplish. Accordingly, when the users are in situations where this is not the case, e.g., while driving a car or maintaining equipment in a hazardous environment, the assumptions of direct manipulation do not hold as well. Inasmuch as the mobile use context often is dynamic and involves tasks outside the computer, we argue that direct manipulation may not be suitable for mobile computers. Due to the dynamics of the mobile use context there is needed a set of interaction styles and interfaces for mobile computers

8 from which the users can choose according to the conditions of the particular situation. Future work involves exploring ways to enhance MOTILE, e.g., polar as well as binary look-up, by which the user can scan much faster simply by pressing the pertaining button longer or harder, pads with physical scale and orientation, rather than buttons, by which the user can jump relative to the magnitude of direction changes, and semi-automated completion of input sequences based on the frequency of previous user actions. We will also conduct a thorough evaluation of Motile. Acknowledgements The Mobile Informatics research program of the Swedish Information Technology Institute (SITI), and the MoBEE project with grants from the Research Council of Norway funded this research. References 1 Hinckley, K., et al. (1998) Two-handed virtual manipulation, ACM Transactions on Computer-Human Interaction, Vol. 5, No. 3, p Nielsen, J. (1993) Usability engineering, San Diego, CA: Academic Press. 3 Shneiderman, B. (1998) Designing the user interface. Strategies for effective human- Computer interaction, Third edition, Reading, MA: Addison-Wesley. 4 Norman, D. (1988) The psychology of everyday things, USA: Basic Books. 5 Ishii, H. and B. Ullmer (1997) Tangible bits: Towards seamless interfaces between people, Proceedings of ACM 1997 SIGCHI Conference on Human Factors in Computing Systems, ACM Press. 6 Sawhney, N. and C. Schmandt (1999) Nomadic Radio: Scalable and contextual notification for wearable audio messaging, in Proceedings of ACM 1999 SIGCHI Conference on Human Factors in Computing Systems, ACM Press. 7 Hindus, D., C. Schmandt, and C. Horner (1993) Capturing, structuring, and representing ACM Transactions of Information Systems, Vol. 11, No. 4, p Brewster, S. (1998) Using Nonspeech Sounds to Provide Navigation Cues, ACM Transaction on CHI, Vol. 5, No. 3, p MacKenzie, S., A. Sellen, and W.A. S. Buxton (1991) A comparison of input devices in element pointing and dragging tasks;, in Proceedings of ACM 1994 Conference on Human Factors in Computing Systems, ACM Press. 10 Jacob, R.J.K., et al. (1994) Integrality and Separability of Input Devices, ACM Transactions on Computer-Human Interaction, Vol. 1, No. 1, p