Delivery of Services on Any Device From Java Code to User Interface

Transcription

1 Delivery of Services on Any Device From Java Code to User Interface Davide Carboni, Andrea Piras, Stefano Sanna, Gavino Paddeu CRS4 Center for Advanced Studies, Research and Development in Sardinia VI strada Ovest, Z.I. Macchiareddu, Uta (Cagliari), Italy {dadaista, piras, gerda, Sylvain Giroux Dept. of Mathematics and Computer Science, University of Sherbrooke Sherbrooke, Canada Abstract The design and the implementation of software for mobile computers and for pervasive computing environments entail several issues and lead to new requirements. Applications must handle at runtime the heterogeneity of delivery contexts in terms of devices features, network bandwidth, operating systems and so forth. The issues related to the access pertain both to delivery (can the device run the service?) and to user interfaces (how the user interact with the service through device?). This paper describes, on one hand, an approach where user interfaces are generated at run-time, and, on the other hand, a deployment strategy that, taking charge of distributing the code according to the device capacities, leads to the application apportioning. 1 Introduction The design and the implementation of software for mobile computers and for pervasive computing environments [Weiser, 1993] entail several issues. Enhancing physical surroundings with computing capabilities leads to new requirements: "A device is a portal into an application/data space, not a repository of custom software managed by the user. An application is a mean by which a user performs a task, not a piece of software that is written to exploit a device's capabilities. The computing environment is the user's information-enhanced physical surroundings, not a virtual space that exists to store and run software." [Banavar et al., 2000] Thus, applications must handle at run-time the heterogeneity of delivery contexts in terms of devices features, network bandwidth, operating systems and so forth. In fact, a user may want to access his personal data/applications from different devices in different context of use: from desktop PC to cellular phones. The issues related to the access belong both to delivery (can the device run the service?) and to the user interface (how the user interact with the service through device?). Delivery must cope with network bandwidth, processing power, operating systems, etc. Human-computer interaction must deal with screen size (if any), colors, and input/output modalities. The design and the implementation of a brand-new interface for each category of devices are tedious and error-prone tasks. Moreover, when applications get mobile or pervasive, one even does not know beforehand what devices will be involved. This paper describes an approach where user interfaces are generated at run-time in a mobile or pervasive context. On the one hand, there are rendering engines that, given a category of device, generate the appropriate

2 user interfaces (UI). These engines exploit reflection on Java code in order to understand which data can be accessed by the user and which actions are permitted on that data. On the other hand, there is a multi-device deployment strategy that, taking charge of distributing the code according to the device capacities, leads to the application apportioning [Banavar et al., 2000]. 2 Target Devices The access to an application from unpredictable delivery contexts involves the automatic UI generation at run-time. To support it, a categorization of devices according to their features is needed. In addition to devices physical features, our categorization is also based on the availability or not of a Java Virtual Machine (JVM) since Java is our privileged language for implementation of the aimed distributed applications. The many features already available in Java have been proven invaluable. Furthermore Java is widely used to develop network-based applications. Three main categories were sorted out: fat clients, thin clients, and Web-like clients. Fat clients groups together personal computers (PC), Unix workstations, and laptops. All of them are capable to run a standard Java Virtual Machine, to download mobile Java code, and are provided with WIMP (Windows Icons Mouse Pointing) input/output modalities. A part or all of the application code can be moved across the network to the target fat client device. For their part, thin clients can only run a limited JVM, for instance the Java Micro-Edition. They can run only a limited subset of Java and they cannot download nor execute mobile code. As a consequence, Java classes must be loaded from the local disc. Since the application code cannot reach the device, a service viewer written in Java has to be pre-installed on the device. This service viewer connects to the application and exchange data using an XML protocol germane to the Web-Service protocol. A service viewer generates an appropriate user interface according to the XML data received, and interacts with the remote object invoking its methods. Thin clients category groups together personal digital assistants, smart phones, and set-top boxes. The last category is Web-like clients. These devices support no JVM at all but they are equipped with a third-party browser (WAPenabled phones, VoiceXML [Voice, 2000], and HTML browsers). They can connect via HTTP to a Web server and download documents compliant to their specific mark-up languages. For instance, a WAP phone will work only with WML documents [Wireless, 2001], while a voice browser will work only with VoiceXML documents. 3 Rationale The rationale behind our approach is that the code of an interactive system can be decomposed in two main parts: models and views. Such a decomposition is borrowed from the object oriented programming practice in which the well known Model-View Control, (see Observer pattern in [Gamma et al., 1995]), design pattern has been formalized and widely used in the design of graphical user interfaces. Such a pattern organizes code for a clear separation of the logic (the Model) and the UI (the View-Controller). In object oriented systems models and views are objects, and objects are reactive systems that can be activated by the invocation the methods in their public interface (the set of operations that can be activated by a client). A model is independent from the delivery context while views are strictly tied to it. Let s take a pocket calculator application as an illustration. A calculator can be implemented as a model object which is activated whenever the user triggers a method that appends a new digit to the display or a method that selects the mathematical operation to perform. The model part is independent from the graphical user interface. With respect to the model part, the only thing to adjust at run-time is "where" such an object is loaded and executed. In the case of fat clients, the model moves across the network, is loaded by the client JVM and executed on the user s device. In the case of thin and

3 Web-like clients, an intermediate remote tier (remote with respect to the user s device), performs code loading, code execution, and communicates with the user s device using an ad-hoc protocol. On the contrary, a calculator view that is the user interface is strictly dependant on the user device. In the case of fat and thin clients, the view part of the application is generated at run-time by means of graphical components such as buttons, text fields, dialogs and so forth. Indeed, views are concrete interactors and must be defined by composing concrete widgets belonging to a specific toolkit. In the case of Web-like clients, view objects are only a mark-up presentation of the model: HTML for Web browsers, WML for WAP browser and so forth. So, in the latter case, the final presentation is delegated to a pre-installed third-party browser. 4 From Java Code to User Interfaces The issue turns into thus taking Java code of the model part of an application to generate a user device for a given class of devices. A rendering engine is an application that performs this rendering process. To address this issue we have developed MORE (Multi-platform Rendering Engine). MORE is on one hand a framework for defining and composing models, and on the other hand, a set of rendering engines that generates the view part of an application. At run-time, the rendering engine builds on-the-fly the right view by transforming models into structures of concrete and interactive objects. Widgets such as text fields, radio buttons, and more sophisticated controls, are deployed as placeholders for the data they refer to: strings, numbers, booleans and more complex data types like object arrays and collections. The rendering process uses reflection to extract directly from the code the displayable information, that is public accessors of fields and public methods. Complex models are viewed as composite graphical objects and the rendering process is applied recursively till atomic interactors are obtained. The advantage of this approach is that the same model can be accessed from any device for which has been implemented a rendering engine. The process of inspecting composite models to generate a graphical user interface can be summarized by the following algorithm: Render(a model object) : a view object Create a container for the actual platform //e.g. a JFrame in Java Swing toolkit For each displayable field of the model object: Look-up for a run-time editor for the field. If found: add the editor to the container. Else: add a link, e.g. a button, to reapply this rendering algorithm recursively to the value of the field For each method of the object: Add a trigger, e.g. a button, to execute the corresponding method. Return the container A run-time editor is a component aimed to write/read the value of an object field. For instance, if our model contains a field of type java.lang.string, the above algorithm looks up a run-time editor for such a type, that will likely be a text field graphic component. To foster the development of new applications, MORE provides run-time editors for main basic Java types: java.lang.string, java.util.number, java.lang.boolean, and java.util.calendar. Besides, it provides run-time editors for composite types like java.util.collection, and java.lang.object[]. For those types, the editor allow users to select, display, modify, remove, and add elements to the collection or to the object array. Finally, during the development of multimedia and geo-referenced applications, emerged the need for some new Java types: Situated and MultimediaResource. Situated is Java interface that provides information about the physical position of an object either in a Cartesian format (latitude and longitude) or in a topological format (city, street, number, etc.). An Situated[] component is displayed as a map centered on the barycentre of the array, in which the user can zoom in, zoom out, select an object and display it in

4 a separate view. The last type we added is the MultimediaResource type which is basically a wrapper to a multimedia file of type among JPEG, GIF, MP3, wav, and mpeg. 5 A Case Study : a Museum Guide for Mobile Users A museum wants to put on-line information on its collection of paintings. The information should be available from any device. So tourists may use their PCs or laptops at home to prepare their visit to the museum or they may use their handheld devices or their cellular phones in the museum to obtain a complement of information. To build and deploy this application, solely the model part has to be implemented in Java. The core classes are Museum and Picture (Fig. 1). A Museum instance contains a collection of Picture and the String member field city holds the name of the city. In both classes, read and write accessors methods are defined in compliance with the name pattern get<field>() and set<field>(). For each category of devices, an appropriate rendering engine will 1) use the method tostring() returns a text description of the whole object, 2) for basic type member field, generate text fields to access and to modify their value if allowed (set<field>() is public), 3) for aggregate data type, generate a set of widgets enabling to browse, select and view their content, and 4)generate buttons for other public methods. Then for any devices, the resulting generated user interface allows to browse the list of paintings, select one, and eventually invoke the operation export(). The rendering engine for fat clients yields to an interface based on Java Swing (Fig. 4) where browsing the collection and viewing items are performed at the same time in a single composite frame, while the rendering engines for handheld devices (Fig. 3) and WAP-enabled devices (Fig. 2), browsing the collection and viewing items are performed in two separate steps due to the severe screen size constraints. 6 Current Implementation Rendering engines were implemented for all the run-time editors described in 4 on the following platforms: fat clients such as laptops and workstations; thin clients such as palm and ipaq handheld computers, interactive TV, and smart phones; Web-like clients such as Web browsers, WAP browsers, and VoiceXML browsers. In the case of Web-like clients, the result of the rendering process is an intermediate XML representation which is then translated in a mark-up language suitable for the actual browser. MORE uses a XSLT processor and an XSL style sheets to perform these transformations. However information obtained by reflection is not always sufficient. On one hand, even if the fields may have names compliant with Java naming and coding conventions, the extracted name may still be difficult to read or understand. For instance, a field departuredate should be visualized as Departure date. Such a transformation could be done automatically decomposing strings according to the position of the capital letters. On the other hand, fields and methods of an object are not ordered in the code following precise criteria, thus the view is generated laying out components with a unpredictable position. Since views are automatically generated at run-time, programmers do not have complete control of the presentation. To address these issues, our rendering engine takes as input additional information, in form of a text file, which specifies aliases for names (allowing also the internationalization of the user interface), and how to layout fields and methods in the container. 7 Conclusion The framework we described provides immediate and significant advantages. MORE makes design once, display anywhere closer to reality. A programmer solely has to write the model in

5 Java. Reflection of Java code allows to extract the structure of Java types and our system uses this structure to generate a user interface. The model then can be deployed on any device for which there is a rendering engine. MORE is easily extensible as new devices categories emerge. So as technology evolves, it is not necessary to rewrite the UI but simply to adapt the rendering engine to this device category. This approach is more germane to Silica [Rao, 1991] and plug-ins than to model-based ones [Paterno, 2000]. Figure 1: UML Diagram for a simplified version of Museum information system. Figure 2: The WML Interface generated by the rendering engine for web-like client. Figure 3: The HTML Interface generated by the rendering engine for thin client (ipaq). 8 References Figure 4: The Java Interface generated by the rendering engine for fat clients. G. Banavar et al. (2000) Challenges: An Application Model for Pervasive Computing. MobiCom 2000, pp Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1995) Design Patterns, Addison-Wesley. Paterno, F. (2000) Model-based desing and evaluation of interactive applications, Springer. Rao, R. (1991) Implementational Reflection in Silica, ECOOP 1991, LNCS 512, Springer-Verlag, pp Voice extensible Markup Language (VoiceXML) v1.0 (2000), Wireless Markup Language v2.0, Wireless Application Protocol WAP-238-WML a, (2001), Weiser, M. (1993) "Some Computer Science Problems in Ubiquitous Computing," Communications of the ACM, pp