A Mobile Augmented Reality Demonstrator. Andrew Radburn 1

Transcription

1 A Mobile Augmented Reality Demonstrator Andrew Radburn 1 1 Ordnance Survey, Romsey Road, Southampton SO16 4GU, UK Tel. ( ) Fax ( ) Andrew.Radburn@ordnancesurvey.co.uk Crown copyright Reproduced by permission of Ordnance Survey KEYWORDS: augmented reality, magic window, mapping, text augmentation 1. Introduction Augmented Reality (AR) is a means of blending computer generated objects or labels with reality so that both appear to be a part of your natural environment. AR is beginning to mature as a subject field with applications moving from pure academic research into industrial and potential consumer areas. Mobile AR provides a good example of how a detailed geospatial database may be combined with location-aware computing to generate a compelling spatiotemporal application. This paper describes a working prototype for a hand-held augmented reality system containing data from Ordnance Survey, the national mapping agency of Great Britain. Ordnance Survey currently maintains a continuously updated digital database of the topography of Great Britain, including around 440 million man-made and natural landscape features. These features include everything from forests, roads and rivers down to individual houses, garden plots, and even pillar boxes. In addition to this topographic mapping, other layers of information are in the database, such as gazetteer information, data on the addresses of all properties and transport network information. This paper begins by describing the prototype platform s hardware and software. In Section 3 the results obtained are presented. Section 4 describes the conclusions and outlines for future work. 2. Prototype System The prototype augmented reality system is a hand-held device that functions as a magic window by displaying video images from a camera on the back of the device onto the screen on the front of it, thereby creating the effect of the user looking through a window. These images are then overlaid by relevant textual mapping information depending on the context of the user. The data to be overlaid is either stored locally on the device or streamed in via a wireless internet connection. Standard open interfaces have been used wherever possible (Open Geospatial Consortium, 2002). The purpose of the demonstrator is to show how real-life geospatial data can be visualised in ways different from the traditional paper map or GIS. The system is designed for outdoor use and uses GPS and orientation sensors, rather than a fiducial marker tracking approach, as it can then potentially be used anywhere in Great Britain Hardware The prototype runs on a Fujitsu Stylistic ST5021D Tablet PC. This provides a 1.1 GHz CPU with 1 Gb RAM and a 20 Gb hard disk. Connected to the PCMCIA slot via an adapter card is a SysOn GPS CF Plus Compact Flash GPS. The tablet PC was chosen as the screen has exceptionally good visiblity in daylight, and is currently used by Ordnance Survey surveyors in the field.

2 A Logitech QuickCam for Notebooks Pro is attached to the reverse side of the tablet PC via Velcro tape. Video images are captured at 640 x 480 pixels in 24 bit color at 10 frames per second. These images are then scaled to 800 x 600 pixels before being displayed on the screen. This scaling is entirely handled by the OpenGL graphics hardware and so is fairly fast. The orientation of the platform is provided by a digital compass. This is a Honeywell HMR3300, which is supplied in kit form and can detect rotation about all three axes. The kit comprises of a 2.4 cm by 3.6 cm PCB which holds a magnetoresistive sensor, this PCB connects to another PCB which converts the signals into standard RS232 output. The RS232 signal is finally converted by a Serial-to-USB convertor and connects to one of the USB ports on the tablet PC. Logitech Camera Video Fujitsu Tablet PC Screen GPS X,Y,Z Honeywell Digital Compass H,P,R Application Cache Internet WMS and WFS Database Figure 1: Schematic overview of the system 2.2. Software The software is written in Java 2 Platform, Standard Edition (J2SE ) version 1.5. The decision to use Java instead of C or C++ was mainly based on the availability of existing Java code and libraries written to handle Ordnance Survey data. The graphics generated by the application are rendered onto the screen via OpenGL using the JOGL Java library interface (soon to become JSR231). OpenGL was chosen as it will soon be available on mobile devices as OpenGL ES, thus making it easier to port the code onto other smaller and more portable platforms in the future. The camera is interfaced using Java Media Framework version 2.1.1e and provides a real-time video stream into the program. Currently the system is set up to process 10 frames per second, which seems to give a reasonably flicker-free experience to the user of the system. The input from the GPS is in NMEA format which is translated from WGS-84 into British National Grid via the OSTN02 transformation (Ordnance Survey, 2002), which has an accuracy of 0.1 m, inside the

3 accuracy of the GPS (under 10m depending on buildings). Data is streamed into the application on demand, centred on the current position of the user. Different datasets have a different area of demand. Data can be streamed in from a Web Feature Server (Open Geospatial Consortium, 2002) if the tablet has a WiFi or wireless internet connection. This data is then cached on the client device for situations where there is no connection Datasets The main dataset used is OS MasterMap Topographic Layer, which contains detailed geospatial features such as building and road outlines, and cartographic text features such as Electricity sub station, drain and road names. OS MasterMap data is ideal for use in AR as it has been collected at an accuracy of about 10cm and is continually kept up to date. OS MasterMap Address Layer is used to provide house names and numbers. Locality information is taken from the Ordnance Survey 1:50000 Scale Gazetteer. Points of interest information such as bus shelters and post boxes are taken from the PointX dataset. 3. Results The application has been field tested in various places, including urban environments such as Southampton and London and rural environments such as the New Forest in Hampshire. The different data layers can be turned on or off by the user to prevent the screen being filled with too much information at one time. The various layers have different colours to help quickly differentiate them to the user (see Table 1). A basic label placement algorithm is used, with the X/Y position taken directly from the base data and the Z calculated so that text labels further away are placed higher up the screen. The vertical position is calculated in a non-linear manner to counteract the lower position on screen the label would usually have by being further away. Locality information is designed to appear near the top of the screen, with closer labels such as street names lower down. When two labels are very close to each other in space, the software will reposition one of them either higher or lower to help reduce the impact of overlapping and occluding labels. The text is rendered onto the screen as a billboard, which dynamically renders the text so that the information is always oriented towards the users point of view. Layer Number Data set Colour Comments 1 Camera 2 OS MasterMap Text Generally Yellow Various colours Roads in White 3 OS MasterMap Address Pink House numbers Layer 4 50k Gazetteer Green Localities 5 PointX Light Blue Points of Interest 6 Direction Information Black Bottom of window Table 1: Datasets and colours The following figures 2, 3 and 4 show live screen captures from the application.

4 Figure 2: View with localities, house numbers and orientation Figure 3: View with localities, road names and cartographic text

5 Figure 4: Westminster Bridge In the basic version of the demonstrator, only text labels are displayed which makes it very efficient in the amount of data that has to be transmitted to the device. An alternative version has also been built that has a more detailed underlying model, but has much greater data volumes. The original 2D polygon data (figure 5) is streamed into the application as GML over a GPRS wireless internet link and converted onthe-fly into 3D polygons by addition of height data from a Digital Terrain Model (DTM) and an estimate of the height of the building (figure 6). The DTM is based on a ten metre grid, so interpolation is used to generate intermediate heights for polygon points. The 2D base data is also cached by the application for areas already visited. In figure 7, the road polygon has been coloured in red to show how close the registration is with the incoming video stream. Also in figure 7, an alternate rendering is shown for when the user is travelling in a moving vehicle, in this scenario the building numbers have been displayed as large textures attached to the vertical faces of the buildings. This could be of use in emergency applications to identify properties quickly. Figure 5: OS MasterMap Ordnance Survey Crown Copyright. All rights reserved. Figure 6: Underlying 3D buildings

6 Figure 7: Road polygons coloured red, numbers as building textures Calibration and registration is currently carried out manually by the user. The compass can be calibrated with an offset to allow for any errors in the setting up and the physical mounting of the compass device onto the tablet PC. This offset can also correct the difference between magnetic north (as used by the digital compass) and grid north (as used by the mapping). The field of view of the internal 3D model can be calibrated and adjusted to be approximately the same as the field of view of the incoming video stream. This is currently carried out by eye using an appropriate building to visually align the edges of the observed real building with the edges of the internal 3D model building. The 3D model can also be calibrated and adjusted for small offset errors in translation due to systematic GPS errors. The prototype performs fast enough to be of practical use, and is used to demonstrate new ways of visualising and using geospatial data. The screen is refreshed at over 10 frames per second, which is enough to give the user a reasonable experience. The user currently sees a slight lag between the overlaid text and the video feed when panning from side to side fast, this is due to timing differences between the incoming video feed and the digital compass. The system has some limitations due to the accuracy of the various elements. The heading from the compass is accurate to about +/- 0.5 degrees, the GPS is accurate to about +/- 2m, the video feed has lens distortions and a different focal length for vertical and horizontal, leading to a slightly distorted field of view. These errors are acceptable in the prototype as the text labels on the display are only labelling whole buildings, roads and localities rather than smaller individual features. Better position tracking could be obtained by using a real-time kinematic GPS system and OSNet, Ordnance Survey s real time GPS correction network. 4. Conclusions and further work This paper has described a prototype mobile hand-held Augmented Reality system using Ordnance Survey digital data as textual annotations over a real-world video feed.

7 Enhancements to the system will be made by better synchronization of the direction given by the digital compass with a direction calculated from the video stream using computer vision techniques. This should make the overlaid text appear to stick much better to the associated object or building and be less susceptible to errors in both direction and GPS position. Other research will be done into automatic calibration of the system, 3D text placement with overlapping label avoidance and the selection of labels depending on the current context of the user. 5. Disclaimer This article has been prepared for information purposes only. It is not designed to constitute definitive advice on the topics covered and any reliance placed on the contents of this article is at the sole risk of the reader. References Rekimoto, J. (1996) A hand held augmented reality system for collaborative design. Proceedings of Virtual Systems and Multimedia (VSMM96). Feiner, S., MacIntyre, B., Höllerer, T. and Webster, A. (1997), A Touring Machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment. Proceedings of the International Symposium on Wearable Computers. Ordnance Survey. (2002), National GPS Network Information Open Geospatial Consortium. (2002), Web Map Server, Web Feature Server and GML specifications. Tenmoku, R., Kanbara M. and Yokoya N. (2005), Annotating user-viewed objects for wearable AR systems. Proceedings of the 4 Th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp G. King, W. Piekarski, and B. Thomas. (2005), ARVino Outdoor augmented reality visualization of viticulture GIS data. Proceedings of the 4 Th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp Biography Andrew Radburn has worked in the Research and Innovation department of Ordnance Survey for over ten years. He has worked on projects with digital mapping data on mobile phones and is currently interested in writing applications to display mapping data in a mobile augmented reality environment.