Analysis of ARToolKit Fiducial Markers Attributes for Robust Tracking

Transcription

1 1 st International Conference of Recent Trends in Information and Communication Technologies Analysis of ARToolKit Fiducial Markers Attributes for Robust Tracking Ihsan Rabbi 1,2,*, Sehat Ullah 1, Muhammad Javed 2,3, Kartinah Zen 3 1 Department of Computer Science and IT, University of Malakand, Pakistan 2 Institute of Engineering and Computing Sciences, University of Science and Technology, Bannu, Pakistan 3 Faculty of Computer Sciences and Information Technologies, University Malaysia Sarawak Kuching, Malaysia Abstract The critical challenge in developing augmented reality applications is registration between virtual and real-world objects. This registration needs tracking of video camera position and orientation with respect to the real-world environment. Currently, marker-based tracking technique is the most utilized approach for tracking the camera position and orientation (pose). For the development of augmented reality applications different marker-based tracking toolkits are available. Among these toolkits ARToolKit is most widely used library. This paper analyzes various attributes of ARToolKit markers for more robust tracking. The attributes consist of marker sizes, marker distance from camera, the marker speed, the brightness in environment, the contrast level of lighting, and correlation between marker size and distance. Various experiments were conducted in order to analyze these factors. One can use these findings to design a robust marker-based augmented reality application. Keywords. Augmented reality; marker-based tracking; fiducial maker; ARToolKit. 1 Introduction Augmented Reality (AR) uses computer graphics, image processing and computer vision techniques to enhance the view of real-world by adding digital contents [1]. The users perform real-time interaction with real and virtual object. Augmented reality lies between the virtual world and real world as defined in the Reality- Virtuality Continuum [2] as shown in Fig. 1. Fig. 1. Reality-Virtuality Continuum [2] *Corresponding author: ihsanrabbi@uom.edu.pk IRICT 2014 Proceeding 12 th -14 th September, 2014, Universiti Teknologi Malaysia, Johor, Malaysia

2 Ihsan Rabbi et. al. /IRICT (2014) The above continuum consists of two extremes. At one extreme the complete virtual environment exists and the complete real environment exists at other extreme. Between these two extremes, the augmented virtuality and augmented reality exists. In augmented virtuality, virtual environment is augmented by real object(s) whereas augmented reality refers to add virtual contents to real-world objects. To develop an AR application tracking is the key challenge to be handled. Tracking means that as the user moves while using AR system, the virtual contents must remain aligned with the pose of real-world environment. This correct alignment of virtual information with the real-world environment is registration [3]. In AR applications, tracking is achieved through vision-based or sensor-based techniques. Vision-based tracking uses image based information to estimate the position and orientation of a camera and active sensors are placed in the real-world environment in sensor-based tracking [4]. Vision-based tracking approaches are categorized into marker-based and markerless. In prepared environment, marker-based tracking is widely used technique [5]. Markerless tracking uses feature-based or model-based methods for the calculation of camera pose [6]. It is required to develop fast and accurate tracking system with less efforts, lower costs and minimum changes in the environment [7]. ARToolKit [8] is an open source toolkit for the development of AR applications using marker-based approach. This paper analyzes the different attributes of ARToolKit markers for more robust marker-based tracking. These attributes consist of marker size, marker distance from camera, the marker speed, the brightness in environment, the contrast level of lighting, and correlation between marker size and distance. Experiments were conducted to produce the analysis of these factors. One can use the information achieved to design a robust marker-based augmented reality application. The section 2 elaborates the literature review in the field of marker-based tracking. The experimental design is presented in section 3. In section 4 the results obtained from experiments are explained. The conclusion and future direction are presented in section 5. 2 Literature Review Fiducial markers are placed in real environment for the development of AR applications. As each fiducial marker has a particular pattern inside, therefore the pose calculation relative to real environment is very easy. One should make multiple markers by using different patterns inside each marker. This enables a user to design

3 Ihsan Rabbi et. al. /IRICT (2014) several special markers for tracking the environment inside a huge building [9]. Researchers used marker-based approach for the development AR application of their interest. Some of these systems are as follow: Based on corner coordinates, a marker-based AR tracking [10] is designed to recognize and track unknown markers in real-time. Through this technique the robustness of marker tracking is increased and provides reliable tracking up to long distance. The color information is used for tracking using a mobile phone to detect color-coded markers [11]. Steinbis et al. [12] presented the development of more scalable markers from set of 3D cones. These markers have the capability to be used for tracking in outdoor and indoor environments. The advantage of these markers includes easy segmentation into regions. Maidi et al. [13] enhanced marker tracking accuracy and stability by combining the approaches of extended Kalman filter [14] and analytical method. Another real-time tracking approach [15] is developed having the capability of tracking and estimating 3D pose of weakly textured objects. Tracking-by-detection algorithm is used for tracking individual frame independently [15]. Marker-based tracking failure occurs when a user views the marker plane through a significantly oblique angle. To solve this problem, the method of samples modeling and reconstruction is used on markers. For the correction of marker template, linear filter method is used for correct pose estimation [16]. Seo et al. [17] provide proper solution to the key challenges in marker-based tracking i.e jitter and marker occlusion. The ARToolKit library is analyzed by researchers to enhance the marker-based tracking capabilities. Malbezin et al. [18] analyzed the marker tracking accuracy over long distance using ARToolKit library. They concluded that as the distance between marker and camera increases, more marker tracking errors are produced. Zhang et al. [19] evaluated different marker-based toolkits having fiducial markers in square shape. The evaluation parameters are marker reliability, usability, accuracy and efficiency for these toolkits. Abawi et al.[20] discovered the relationship between marker tracking accuracy and the marker distance from camera along with the rotation angle between camera and marker using ARToolKit. Rabbi et al. [21] extended the functionality of ARToolKit to unprepared environments. The primary motivation behind this work is to find the optimal values of multiple marker attributes to enhance the tracking functionality of marker-based systems. The marker attributes include marker sizes, marker distance from camera, the marker speed, the brightness in environment, the contrast level of lighting, and correlation between marker size and distance. Multiple experiments are performed for analyzing

4 Ihsan Rabbi et. al. /IRICT (2014) each attribute. One can easily use these information achieved through our analysis to design a robust marker-based augmented reality application. 3 Experimental Design We use ARToolKit library to analyze the tracking performance of a single marker. The experiments consist of analyzing different attributes of markers 3.1 Experimental Protocol The experiments are based on the analysis of each attribute. For each attribute a different experimental setup is designed. Each experiment is performed using the following setting: 1. For marker size the sample1 marker of different sizes is taken. The size ranges from 8 8 pixels to pixels. These markers are placed at the distance of one meter from the camera to be tracked. 2. To analyze the effect of distance (between marker and camera) on marker tracking, a pixels marker is placed at different distances. 3. For marker speed analysis, the speed along each axis is analyzed individually. The marker is pasted on hardboard containing a movable channel. The channel is freely movable along x-axis, y-axis and z-axis. The channel is moved along each axis individually. 4. After analyzing the marker size and marker distance individually, their correlation is analyzed. For this purpose different marker sizes are tracked at different distances from the camera. 5. The effect of environmental brightness level and lighting contrast are analyzed using pixel marker placed at one meter distance from the camera. The above experiments were performed by using Sony VAIO corei5 laptop having 2.4GHZ processor and 4GB RAM. A webcam having a resolution of pixels with NVIDA graphics card are used. 3.2 Experiments We performed a number of experiments to test each attribute of fiducial marker using ARToolKit Library. Separate modules are designed for individual attributes. Marker tracking errors are saved in text files for analysis. We performed the following experiments:

5 Ihsan Rabbi et. al. /IRICT (2014) The first experiment focuses on the tracking of markers having different sizes. The tracking errors of different marker sizes are recorded. For this purpose, we designed 40 different size markers. The smallest and largest marker sizes are 8 8 pixels and pixels respectively. These markers are pasted on hardboard and are placed in front of camera one by one to perform tracking. The designed module returns the marker size and tracking errors to a text file. For this experiment, we keep the marker distance constant i.e. one meter from camera. 2. In this experiment, the marker tracking distance is analyzed. For this purpose, we designed a module that tracks single marker and calculates the marker distance along with marker tracking errors. We consider a fixed size marker of pixels at different distances. The module gets the marker distance from camera and records their corresponding tracking errors. This information is saved in a text file for analysis. 3. The next experiment is analyzing the effect of marker speed at any direction in front of camera. For this purpose, we take a marker having fixed size of pixels to be tracked at different speed. A module is developed to record the marker speed and marker tracking errors in a text file. We analyze the marker speed along a specific direction. The fixed size marker is pasted on a hardboard. The hardboard is fixed to a movable channel. So the marker tracking errors are recorded along with the marker speed at x-, y- and z-axis. 4. The effect of marker size on marker distance from camera is analyzed in this experiment. A module is developed that records the marker size and marker distance from camera to a text file. The size of marker ranges from 8 8 pixels to pixels. These different markers are tracked through camera from various distances. 5. This experiment is performed to analyze the effect of environmental brightness level and lighting contrast on marker tracking. The brightness level for our experiment ranges from -64 to +64 having default value of zero. The contrast level ranges from 0 to +64 with default value of Experimental Results The data recorded for each attribute is analyzed through SPSS. SPSS Statistics is a software package used for statistical analysis. The analysis of each experiment is given as below: 4.1 Marker Size The experimental data got from marker size experiment is the marker tracking errors on each marker size. This data is analyzed and the result is shown in Fig. 2.

6 Ihsan Rabbi et. al. /IRICT (2014) Fig. 2. Effect of Marker Size on Marker Tracking The marker size is measured in P P pixels and the marker tracking errors ranges from 0 to 1. Here 0 means perfect track while 1 means complete tracking failure. The Fig. 2 indicates that the marker tracking errors increase when very small marker size is placed at one meter distance from camera. As the size of marker is equal or less than pixels and distance between marker and camera is one meter then ARToolKit has no ability to track those small size markers. 4.2 Marker Distance The effect of marker distance on marker tracking is recoded in a text file. The analysis is performed on this data as shown in Fig. 3. Fig. 3. Effect of Marker Distance on Marker Tracking

7 Ihsan Rabbi et. al. /IRICT (2014) Fig. 3 shows the effect of marker tracking while placing a fiducial marker at different distances from camera. The marker distance from camera has great effect on tracking performance. The increase in distance between maker and camera produces blurredness in the scene that causes marker tracking failure. To track a marker from longer distance needs larger marker size. 4.3 Marker Speed The movement of marker in front of camera is performed and the tracking errors are recorded in a text file. This movement is performed in x-, y- and z-axis direction. The motion is given to a marker at different speed to analyze the tracking errors. Each direction data is analyzed individually. Fig. 4 indicates the results produced during marker motion along x-axis, y-axis and z-axis. Fig. 4. Effect of Marker Speed on Marker Tracking along (a) x-axis (b) y-axis (c) z-axis The graph indicates that the movement of marker along y-axis causes more tracking errors than any other axis (Fig. 4 (b)). The errors produced by moving the marker along x-axis are also high as shown in Fig. 4 (a). The slow movement of marker along x-axis and y-axis has no significant effect on tracking process while high speed causes more errors. This high marker motion produces blurredness in scene that causes failure in marker detection. In Fig. 4 (c) the movement of marker at any speed along z-axis produces much smaller errors than any other axis. Therefore moving the marker at any speed away from camera or toward the camera has less effect in tracking performance. Zooming in and zooming out of marker in front of the camera at variety of speed produces less marker tracking errors.

8 Ihsan Rabbi et. al. /IRICT (2014) Correlation between Marker Size and Marker Distance The effect of marker size and marker distance from camera is analyzed and data is recorded. Fig. 5 draws the graph of marker tracking errors for different marker size at different distances. Fig. 5. The Effect of Marker Size and Distance on Marker Tracking Performance The graph presents the parameters of marker distance and its size along with marker tracking errors. The markers with larger size have less effect on marker tracking at different marker distances from camera. Similarly, the smaller marker size produces more tracking errors with longer distance. To track marker robust using ARToolKit at longer distance from camera, a larger size of the marker is to be taken. 4.5 Environmental Brightness Level and Contrast The data obtained from performing experiments on brightness level and contrast are analyzed and reported in Fig. 6. Fig. 6. Marker Tracking at Different (a) Brightness Level (b) Contrast Level

9 Ihsan Rabbi et. al. /IRICT (2014) The graph in fig. 6 (a) indicates the effect of brightness level of environment on marker tracking performance. As the brightness level of the environment is increased from its default level, the marker tracking produces more errors. The decrease of brightness level has no significant effect on marker tracking performance. Fig. 6 (b), indicates that as the contrast level is increased from its default value, the marker tracking errors increased. The better detection is performed at the contrast level of Conclusion and Future Work Fiducial marker attributes of ARToolKit is analyzed in this paper. This analysis leads to find the optimal values for each parameter to perform more robust marker tracking. We considered the attributes of marker sizes, marker distance from camera, the marker speed, the brightness in environment, the contrast level of lighting, and correlation between marker size and distance for our experiments. During each experiment the data about each attribute is recorded in separate files. These recorded data is analyzed through SPSS. We concluded from the analysis that the above marker attributes effects the marker tracking. The summary of each experiment is as below: The smaller marker size produces more errors. The detectible size of marker is pixels or greater. Below to this threshold the marker is not identified by ARToolKit. The longer distance from the camera causes more tracking errors. The movement of marker along y-axis at any speed produces the greater tracking than any other axis. The tracking performance is less affected during the marker movement at z-axis at different speed. When the marker size is larger, then it produces less marker tracking errors at any distance. The smaller marker size produces more tracking errors. The marker size is positively correlated with marker distance from camera. When the brightness level of marker is increased from its default value, it produces higher marker tracking errors. The decrease of environmental brightness level has no significant effect on marker tracking. The higher values of contrast level from the default level produce more marker tracking errors. The lower values of this level do not affect the tracking process. While performing experiments, we investigated that this toolkit produces false detection rate (false positive rate, false negative rate and inter-marker confusion rate). Therefore a procedure is needed to overcome the challenge of marker false detection rate.

10 Ihsan Rabbi et. al. /IRICT (2014) References 1. Siltanen, S. Theory and Applications of Marker-Based Augmented Reality. In: Science, V. (ed.), VTT Technical Research Centre of Finland, Milgram, P., Takemura, H., Utsumi, A. and Kishino, F. "Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum", In: SPIE Proceedings: Telemanipulator and Telepresence Technologies, pp , Azuma, R.T. "A Survey of Augmented Reality. Presence", Teleoperators and Virtual Environments, pp , Yang, P., Wu, W., Moniri, M. and Chibelushi, C.C. "A Sensor-based SLAM Algorithm for Camera Tracking in Virtual Studio", International Journal of Automation and Computing, vol. 05, pp , Shin, D.H. and Dunston, P.S. "Identification of Application Areas for Augmented Reality in Industrial Construction Based on Technology Suitability", Automation in Construction, vol. 17, pp , Comport, A.I., Marchand, E. and Chaumette, F. "A Real-time Tracker for Markerless Augmented Reality", 2nd International Symposium on Mixed and Augmented Reality, (ISMAR 03), pp , Rabbi, I. and Ullah, S. "A Survey on Augmented Reality Challenges and Tracking", Acta Graphica, vol. 24, pp , Fiala, M. "ARTag, An Improved Marker System Based on ARToolkit", National Research Council Canada, Publication Number: NRC: 47419, Naimark, L. and Foxlin, E. "Circular Data Matrix Fiducial System and Robust Image Processing for a Wearable Vision-Inertial Self-Tracker", International Symposium on Mixed and Augmented Reality, (ISMAR '02), pp , Ababsa, F. and Mallem, M. "Robust Camera Pose Estimation using 2D Fiducials Tracking for Real-Time Augmented Reality Systems" International Conference on Virtual-Reality Continuum and its Applications in Industry, pp , Möhring, M., Lessig, C. and Bimber, O. "Video See-Through AR on Consumer Cell Phones", 3rd ISMAR 04, pp , Steinbis, J., Hoff, W. and Vincent, T.L. "3D Fiducials for Scalable AR Visual Tracking", 7 th ISMAR '08, pp , Maidi, M., Didier, J.-Y., Ababsa, F. and Mallem, M. "A Performance Study for Camera Pose Estimation using Visual Marker Based Tracking", Machine Vision and Application, vol. 21, pp , Bishop, G. and Welch, G. "An Introduction to the Kalman Filter", SIGGRAPH, Donoser, M., Kontschieder, P. and Bischof, H. "Robust Planar Target Tracking and Pose Estimation from a Single Concavity", 10 th ISMAR'11, pp. 9-15, Ito, E., Okatani, T. and Deguchi, K. "Accurate and Robust Planar Tracking Based on a Model of Image Sampling and Reconstruction Process", 10 th ISMAR'11, 1-8, Seo, J., Shim, J., Choi, J.H., Park, J. and Han, T.-d. "Enhancing Marker-Based AR Technology Virtual and Mixed Reality", Part I, HCII, LNCS 6773, pp , Malbezin, P., Piekarski, W. and Thomas, B.H. "Measuring ARToolKit Accuracy in Long Distance Tracking Experiments", First ART'02, Zhang, X., Fronz, S. and Navab, N. "Visual Marker Detection and Decoding in AR Systems: A Comparative Study" ISMAR '02, pp Abawi, D.F., Bienwald, J. and Dörner, R. "Accuracy in Optical Tracking with Fiducial Markers: An Accuracy Function for ARToolKit", 3 rd ISMAR'04, Rabbi, I., Ullah, S., Rahman, S.U. and Alam, A. "Extending the Functionality of ARToolKit to Semi Controlled/Uncontrolled Environment", INFORMATION, vol. 17, pp , 2014.