HMI ARCHITECTURE Piergiorgio Navone Advanced Product Dept - Centro Ricerche FIAT Strada Torino 50 10043 Orbassano (TO), Italy Tel: +39 011 9083 866 - Fax +39 011 9083 083 - e-mail: p.navone@crf.it Federico Braja Advanced Product Dept - Centro Ricerche FIAT Strada Torino 50 10043 Orbassano (TO), Italy Tel: +39 011 9083 140 - Fax +39 011 9083 083 - e-mail: f.braja@crf.it Roberto Montanari Advanced Product Dept - Centro Ricerche FIAT Strada Torino 50 10043 Orbassano (TO), Italy Tel: +39 011 9083 028 - Fax +39 011 9083 083 - e-mail: r.montanari@crf.it SUMMARY In the development of in-car informative systems Human Machine Interface (HMI) plays a key role for functionality, usability and style. This paper presents the advantages of a particular HMI architecture based on the concept of browser, borrowed from the Internet world. Fiat Research Centre has realised a modular in-car architecture for telematics services applying this concept. The solution seems to be very interesting particularly for open and distributed systems with a high variability in the configuration. The principal advantages are separation of look and functions, easiness in introduction of new applications and high modularity of the architecture that allows the reuse of the same components on different systems. Moreover the architecture has been designed in order to fit an automotive network with limited bandwidth, such a CAN (Controller Area Network). ARCHITECTURE DESCRIPTION The system we are speaking about can provide several services to the driver: traffic information, emergency call and breakdown assistance, route guidance, phone, e-mail, general information, car radio and air conditioning control, are some of the services the system can realise. The architecture has been designed to be modular and distributed over a network; the network is a CAN (Controller Area Network) bus with a transport protocol. HMI is one of the nodes of the network; every node can provide one or more functions and services are realised accessing different functions by means of the unique HMI. New services may be added increasing the number of nodes; the list of services is not known a priori. HMI Requirements All data oncoming from the car network are collected and shown to the driver through a shared HMI: it depends on the fact that complexity of the driving task and resulting workload is an important factor that affects the way the driver interacts with on-board information
systems. Integration of information may then facilitate decision making by the driver and prevent overloading the visual, acoustic or haptic channel of the driver. A shared HMI can also assure to the driver the possibility to use more than one function at the same time (i.e. phoning and using navigation system). In an open, distributed architecture the entity that controls the HMI cannot know a priori, at the design time, all the applications that can be present in the system so the architecture has to provide a common way of managing concurrent applications. The size of the display usually employed in the cars doesn t allow the use of windowing systems, as those used by personal computers, so the solution of the multitasking problem has to use a different approach. The model implemented by the browser is slightly different: there are many applications working at the same time and a unique browser; the browser presents to the user one page at a time; the user can navigate through different applications forward and backward accomplishing different tasks. In an open architecture it s very important to guarantee the consistency of the HMIs of different applications also when the applications are provided by external modules added to the core system, and not designed together with it. This requirement is very difficult to fulfil, but the problem can be split up into two parts: to ensure the consistency of the content and to ensure the consistency of the appearance and of the interaction with the user (look & feel). To do that the system architecture must allow the separation of the content from the look & feel: the entity that provides the function has to be responsible for the content as well as the entity that manage the HMI has to be responsible for the look & feel. Summing up the principal requirements for the HMI are: HMI is a shared resource: every node on the network can use the HMI Multitasking: HMI must manage several tasks at the same time Minimize network load: the interaction between HMI and applications lies on a car network with limited bandwidth Separation between contents and look & feel: the HMI fixes the look & feel, whereas the applications are responsible for the contents Markup Language The Internet browsers use the HTML (HyperText Markup Language) language for describing pages. For our browser we have chosen a similar language that is an extension of WML (Wireless Markup Language), the language used for WAP (Wireless Application Protocol) services. The WML contains constructs allowing the application to specify documents made up of multiple cards. An interaction with the user is described in a set of cards, which can be grouped together into a document (commonly referred to as a deck). Logically, a user navigates through a set of WML cards. Each card, in a deck, contains a specification for a particular user interaction. The choice of the WML, instead of HTML, is due to two major reasons: WML has been designed for small devices with limited resources (graphic capabilities, memory, etc.), but can fit also devices with a quite big colour display. WML is specified in a way that allows presentation on a wide variety of devices and allows vendors to incorporate their own HMIs. For example, WML does not specify how implementations request input from a user. Instead, WML specifies the
Figure 1 A typical page (a WML card) of the phone-book application displayed on a small monochrome display. The editing boxes, on the right hand side, cannot display the entire content so the browser provide a scroll mechanism. The page can also be scolled up and down to display other lines. The icons on the top of the display (new message, GPS state and GSM field strength) are not part of the XML document but are added by the browser itself. intent in an abstract manner. This allows WML to be implemented on a wide variety of input devices and mechanisms. The presentation of the information elements of a card can be done in different ways depending on the device capabilities. For example, certain user agents on devices with larger displays may choose to present all the information in a single card at once. Others, on the other hand, with smaller displays may break the content up across several units of displays (1). Functional Description As in the World Wide Web pages are addressed by URL (Universal Resource Locator). We have defined a special scheme for URL that identifies on board resources. A typical World Wide Web URL is: http://www.crf.it/page3.html?name=mike. The field http: is the scheme and identifies the URL syntax associated to a particular protocol; www.crf.it identifies the web server, the node of the web network; the next field page3.html locates a particular page on the server and after the question mark there is the parameter list, a list of variable names and values. We use a similar scheme to locate resources on the vehicle network; for example the URL ic:phone/call_menu locates the page call_menu on the node phone present on the network and the URL ic:phone_book/show_data?name=mike asks the phone_book for a page with relevant data for mike.
Figure 2 The rendering of the WML card showed in Figure 1 made by a browser with a high definition colour display. In this case the browser can display more information at a time. This browser uses the touch screen as a unique input device, so the keys for WML navigation and other applications are displayed on the page, even if they are not part of the WML deck. The technique of using special URL for addressing local (which means: not on the web but inside the car) resources has been used both in the Internet web browsers and in the WAP user agent for accessing the functions of the mobile phone (WTA - Wireless Telephony Applications)(2). In a page described with WML (or the appropriate markup language) every link (e.g. menu items, soft keys, icons, etc.) has an associated URL; when the user selects a link the browser sends a GET request to the node addressed in the URL. For example if the URL is ic:phone/call_menu the browser sends to the phone the message GET call_menu. The device that receives a GET request sends to the browser the appropriate WML page with the correct information end several new links. As in the web, a URL may also activate an action, for example the URL ic:phone/do_call?number=0119083866 asks the phone to make a call to the specific number. In this way we have realised a system with the following features: The browser doesn t know the complete structure of the menus Every node of the network knows the content and the links of his own pages Every node can introduce in a page links to other functions The user move through different applications following links, so he feels a single task, while in the system multiple applications are working
Display the home page Theuser select an item ssociated to the URL ic:app1/main_menu GET /main_menu Display deck main_menu WML Deck main_menu The user input some data and select a link with the URL ic:app2/action? dato1=22&dato2= abc Display the deck with the result of the action Browser Priority Manager GET /action3?dato1=2&dato2= str WML Deck action_result Application 3 push to the browser the URL ic:app3/new_event Application 1 Application 2 Application 3 The browser play a beep and display the new_event deck GET /new_event WML Deck new_event The user manage the new event then select BACK. The browser extract the previous activity from the history stack GET /action3?dato1=2&dato2= str WML Deck action_result Figure 3 The diagram shows the interaction between the browser and the applications. The user interface is shown in the left hand side. The user interacts with the browser that forward the user requests to the applications. Usually the applications answer to the requests of the browser (the GET command) but they can also PUSH an unsolicited event. A priority manager (represented in yellow) is necessary to filter the push requests. Note that in the example the user makes a complex activity using three different applications, but the applications don t interact each other directly. Furthermore note that, if the vehicle has a link with a wide area network, the three applications in the example might be on board or off board. The nodes that generate pages in WML don t know the characteristics of the actual device realising HMI (display characteristics, number and position of keys, other
input devices) The browser, that knows the characteristics of the actual input/output devices, gives to all pages generated by different nodes the same look & feel Architectural Constraints However an Internet browser as is isn t sufficient to realise the HMI for the most common services present on board. The first problem we encountered is the notification of unexpected asynchronous events, as an incoming telephone call, the reception of a message or an advice by the route guidance. For managing these events we had to add to our browser the push capability. Usually the applications wait for a GET request from the browser; when an asynchronous event occurs (e.g. the phone rings) the application sends to the browser a push request with the URL of the page that shall be displayed. The browser can manage the push in different ways: it can open a pop-up box or simply display a new page, perhaps warning the user with a sound. The new page gives the user the possibility to manage the new event (to answer the phone, to reject the call, etc.) and return to the previous activity. To do that the browser must have a navigation stack and the markup language must provide the control for backward navigation: WML provides all the necessary tags. The constraints implied by the browser model may be too strong to realise attractive and style effective interfaces for all the most common on board services. Applications requiring very close interaction with the HMI, special needs for the graphic (maps, animations, etc.) may be difficult to implement over a simple WML browser. Since that applications are few and known a priori we can introduce dedicated features on the HMI. Even though these additional features are beyond the concept of browser, they should interact with the browser scheme correctly. Let we introduce some examples: If the display is big enough we can dedicate a window for information from the car radio (program name, preset number, frequency, etc.) and from the mobile phone (network operator, field strength, new message icon, etc.). If the route guidance system has to display a moving map, or a dynamic bar for the next intersection distance, then the browser should open a special page interacting with the route guidance system directly. CONCLUSION AND FURTHER DEVELOPMENTS We realised with this architecture in several services: Phone Phone-book Traffic information (TMC) Climate control Emergency call
Figure 4 The front panel of the car radio realised with a page ad hoc. Off-board navigation Radio MP3 player News (WAP-like service) Travel information In our experiment we decided to provide two special cards to display navigation maps and the front panel of the audio system; we also introduced some dedicated keys for the audio system and for the phone. In our experiment we used as a markup language an extension of WML, that up to today seems to be the better solution for simple graphic interfaces. There are several new markup languages for page description that may be used for more advanced graphic: the XHTML (Extensible HyperText Markup Language) presents some interesting characteristics as the modularization. Another interesting development will be the voice browser; the speech recognition will be very important for accessing the on board services by the driver. There are many activities for developing voice browsers and markup languages for dialog description (e.g. the W3C Voice Browser working group). The browser may combine both graphic and speech access to the services. The architecture described has been implemented in several prototypes and fulfils the needs of modularity, architectural distribution and domain separation between contents and look & feel.
The browser concept introduces some constraints in the design of the HMI and of the services, that have to be structured as web applications. As we said these constraints may be partially avoided defining special methods for a few, well known applications. Finally the browser-based architecture may be a good solution when there are strong requirements of modularity and scalability in an open and distributed architecture. REFERENCES [1] WAP Forum, Wireless Application Environment Overview, April 30, 1998. [2] WAP Forum, Wireless Telephony Application Interface Specification, April 30, 1998.