DETECTION OF OBSTACLE AND FREESPACE IN AN AUTONOMOUS WHEELCHAIR USING A STEREOSCOPIC CAMERA SYSTEM

Transcription

1 DETECTION OF OBSTACLE AND FREESPACE IN AN AUTONOMOUS WHEELCHAIR USING A STEREOSCOPIC CAMERA SYSTEM Le Minh 1, Thanh Hai Nguyen 2, Tran Nghia Khanh 2, Vo Văn Toi 2, Ngo Van Thuyen 1 1 University of Technica Education HCMC, Vietnam 2 Internationa University, VNU, HCMC, Vietnam ABSTRACT Computer Vision has been a very new fied deveoping rapidy in recent years. One of the main advantages in this deveopment is that the human vision abiities such as eectronicay perceiving and understanding of images are provided to appy to autonomous vehices or mobie robots. In this paper, detection of obstace and freespace in an autonomous wheechair using a stereoscopic camera system is introduced. The image data captured from the rea word are obtained from a mutipe camera system caed the stereoscopic camera system. This paper wi represent two dimensiona (2D) and three dimensiona (3D) maps cacuated based on disparity map using geometric projection. From the 2D and 3D maps, one can detect obstaces and freespaces for the objective of autonomous wheechair contro. The experimenta resuts obtained show to iustrate the advantage of the proposed method. KEYWORDS: stereoscopic camera system, 3D and 2D maps, autonomous wheechair. 1. INTRODUCTION Computer Vision consists of finding the depth of a scene from severa views of it [1]. This depth is cacuated for obstace avoidance of an autonomous wheechair on its way using a stereoscopic camera system. Eectric wheechairs with the joystick or traditiona wheechairs controed by hands are necessary to assist the mobiity of disabed peope. However, in many cases, these wheechairs are not safe for users in the case of oss of contro. For this reason, additiona equipments incuding speciaized computer, sensors and contro system, are mounted on power wheechairs and nove agorithms are utiized to enabe them to be cassified as smart wheechairs [2-4]. In recent years, utrasound and aser sensors have been used to detect obstaces and freespaces in the environment surrounding mobie vehices [5-8]. For exampe, a mobie robot may be equipped with utrasonic sensors, heping to detect freespace and obstace for its mobie operation [9-11]. In addition to freespace detection, a Bayesian update theory is appied to compute the vaue, in a given direction, of FreespaceProbabiity at each point-mark [12]. The SAD correation method, using stereo cameras, has effectivey detected andmarks and corresponding images, in which bock matching methods are utiized. In recent years, these have served to estimate the disparity for pixes in a pair of stereo images [13-16]. In this paper, the SAD correation agorithm takes one pixe in an image from a region of pixes. It then finds its cosest match to a pixe in the corresponding region of the other image, and creates a disparity map [17, 18]. This project proposes an approach invoving the detection of freespaces and obstaces using a stereoscopic camera system. It is proposed that the camera system coects 3D information in an unknown environment. The eectric wheechair autonomousy detects both the height and width of freespaces and obstaces for coision avoidance. From the eft and right images captured by the camera system, the Sum of Absoute Differences (SAD) correation method wi be empoyed to determine stereo disparity. 244

2 Given this disparity map, a 3D point map is generated using a geometric projection, in which the height and width of a freespace or an obstace are determined. For the goa of detecting freespaces and obstaces, a 2D distance map is converted from this 3D map. Based on the 2D map, one can cacuate the autonomous avoidance of the eectric wheechair. 2. CALCULATION OF VISION IMAGES 2.1 Bock Matching Bock Matching agorithm uses a window with (2m + 1)x(2n + 1) size. The window wi be moved aong the epipoar ine [1] in the eft image from the pixe (x 0,y 0 ) in specific range of disparity d: d min d max (see Figure 1). Whie moving, the matching cost wi be cacuated by the SAD equation as foows: n SAD( x, y, d) ( ( I R ( x + k, y0 + m = k = n = m ( x0 + k + d, y0 0 I L + ) )) (1) in which I is the gray eve of the pixe (x i,y i ). After cacuating the SAD matching cost of a window s positions, the minimum cost wi be chosen: D = min { SAD( x, y, d)} (2) dmin dmax in which D is the disparity of the pixe (x 0,y 0 ). ) Figure 2: Fowchart of cacuation of a disparity map. 2.2 Cacuation of 3D Map The disparity D of a pixes in the image wi be used to find the depth Z using for geometric projection as shown in Figure 3. The depth distance wi be expressed in the foowing equation: f * b Z = (3) D in which Z is the distance from the camera to the rea object, f is the foca ength of the camera and b is the baseine ength of the camera. Figure 1: Bock matching of the eft and right images. The fowchart for cacuation of a pixe s disparity using equation (1) is shown in the Figure 2. Figure 3: Cacuation of the depth Z. The X and Y parameters of a pixe on the 3D map wi be then cacuated as in Figure 4 using the simiar triange projection in the 245

3 foowing equations: Figure 4: Cacuation of the X and Y. distance from obstace to the camera centre of the wheechair. The minimum distance Z imin of the coumn number i is computed in the foowing equation: n Z i = min Z (7) min j = 0 in which Z i min is the minimum Z of the coumn number I, j = 0 n is the numbers of the rows on the image. ij x X = Z (4) f y Y = Z (5) f in which (X, Y) is the position of the object P in rea word. Figure 5: Determination of pixes h Y h ). ( h The Y parameters of a pixes on the image wi be compared with the height h of the autonomous wheechair (h = h h h ) to make a 3D map. Therefore, the 3D map just consists of pixes with Y cassified to be the foowing equation: h Y (6) h h The fowchart for determining these pixes on the 3D map is shown in Figure Cacuation of 2D Map A 2D map is converted from the 3D map data. The 2D map wi be of the axes X and Z, in which the depth Z is considered as the Figure 6: Computation of the distance Z imin The fowchart for computing Z imin is shown in Figure 6. The distance Z min of the cosest object appearing in front of the camera is computed as foows: Z m = Z (8) min min i min i= 0 246

4 in which i = 0 m is the number of coumn. The width w of the freespace number is computed in the foowing equation: k = 2 X i r= k1 w (9) in which: r = k1, k2 is the first and the ast possition on the X axis where Z i Z. min Then, the width w of the freespace number wi be compared to the width w of the wheechair to see if it is reay freespace or it is obstace. If w > w, it wi be considered as freespace, otherwise it wi be considered as obstace. 3. EXPERIMENTAL RESULTS 3.1 Wheechair Hardware System in the Figure 8. Based on the eft and right images, one can compute the disparity map. Figure 8: The eft (a) and right (b) images. The environment has a board obstace and two freespaces (space 1 and space 2), in which there is a sma freespace (space 3) under the board obstace. The disparity map computed from the eft and right images using the bock matching agorithm with the window size of 7x7 is shown in the Figure 9. Figure 9: Disparity map. Figure 7: Eectrica wheechair with a stereoscopic camera system and other devices at Lab 104 of Internationa University, VNU, Vietnam. An eectrica wheechair was instaed with the Bumbebee stereoscopic camera system to produce 3D image information and some necessary devices such as Anaog to Digita Converter, DAQ, mini monitor (LCD) as shown in the Figure 7 in this paper. The height h of the wheechair is 1.1m and its width w is 0.8m. The camera is ocated at the height h = 0.8m from the ground. 3.2 Detection of Obstace The environment in this experiment has one obstace in front of the wheechair as shown The 3D map is computed using the geometric projection as shown in the Figure 10. Figure 10: The front view 3D point map. The 2D map of the environment is shown in the Figure 11. In this map, the camera system can recognize the board and the freespaces around it to be an obstace. Because the widths of the space 1 and space 2 are 0.8m, equa to the wheechair s width, and the height of the space 3 is 0.9m, 247

5 ess than the wheechair s height. Figure 14: Left (a) and right (b) images. The wheechair can recognize the chairs as an obstace and two freespaces on the eft and right side of the chairs in the 2D map (see Figure 15). The width of the freespace 1 is about 1m and that of the freespace 2 is 1.3m. Figure 15: 2D map of environment with obstace and freespaces. When detecting the obstace, the wheechair wi check if there is any freespace in the 2D map, from the eft to the right of 2D map, and chose the arger freespace to pass through. At this position on its way, the wheechair can recognize that the width of the freespace 2 is arger than that of freespace 1. Therefore, the wheechair made the decision to turn right to pass through the freespace 2 and avoid the obstace. After passing through the freespace 2, the track of the wheechair s motion was drawn as Figure 16. Figure 16: Wheechair avoidance of obstace. These experiments were shown to iustrate that an eectric wheechair instaed with the stereoscopic camera can avoid obstaces and pass through freespaces. In this paper, the wheechair was designed to avoid coision and go through freespaces based on 2D maps. 4. CONCLUSION A stereoscopic camera system mounted on an eectric wheechair can recognize obstaces and freespaces on its way. Given the eft and right images, the disparity map was determined. Therefore distance Z from objects in the scene to the camera centre of the wheechair was cacuated. The 2D map was obtained from the 3D map. Based on this 2D map, the wheechair was cacuated to detect obstaces and freespaces and autonomousy avoid and pass through. The experimenta resuts show that the proposed method is the effectiveness. 5. ACKNOWLEDGMENTS The authors woud ike to acknowedge the support of Internationa University and Vietnam Nationa University. 6. REFERENCES [1] G.Banchet, A.Buades, B.Co and J.M More, Fattening Free Bock Matching,in Journa of mathematica imaging and vision, vo. 41, num. 1-2, 2011, pp. 1. [2] [2] Y. Adachi, K. Goto, Y. Matsnmoto, and T. Ogasawara, "Deveopment of Contro Assistant System for Robotic Wheechair - Estimation of User's Behavior based on Measurements of Gaze and Environment," in Proceedings of the IEEE Internationa Symposium on Computationa Inteigence in Robotics and Automation, 2003, pp [3] Y. Kuno, T. Yoshimura, M. Mitani, and A. Nakamura, "Robotic wheechair ooking at a peope with mutipe sensors," in Proceedings of IEEE Internationa Conference on Mutisensor Fusion and Integration for 248

6 Inteigent Systems, 2003, pp [4] I. Moon, K. Kim, J. Ryu, and M. Mun, "Face Direction-Based Human-Computer Interface using Image Observation and EMG Signa for The Disabed," in Proceedings of the IEEE Imternationa Conference on Robotic and Automation, 2003, pp [5] D. An and H. Wang, "VPH: A New Laser Radar Based Obstaces Avoidance Method for Inteigent Mobie Robots," Proceedings of the 5th Congress on Inteigent Contro and Automation, 2004, pp [6] J. Borenstein and Y. Koren, "The Vector Fied Histogram-Fast Obstace Avoidance for Mobie Robots," IEEE Transaction on Robotics and Automation, vo. 7, pp , [7] I. Urich and J. Borenstein, "VFH+: Reiabe Obstace Avoidance for Fast Mobie Robots," in Proceedings of the IEEE Internationa Conf. on Robots and Automation, 1998, pp [8] Z. Xiang, Z. Xu, and J. Liu, "Sma obstace detection for autonomous and vehice under semi-structura environments," in Proceedings of IEEE Inteigent Transportation Systems, 2003, pp [9] J. Borenstein and Y. Koren, "Error Eiminating Rapid Utrasonic Firing for Mobie Robot Obstace Avoidance," IEEE Trans. on Robotics and Automation, vo. 11, pp , [10] S. Shova, J. Borenstein, and Y. Koren, "The Navbet A Computerized Trave Aid for the Bind Based on Mobie Robotics Technoogy," IEEE Transactions on Biomedica Engineering, vo. 45, pp , [11] I. Urich and J. Borenstein, "The GuideCane Appying Mobie Robot Technoogies to Assist the Visuay Impaired," IEEE Transactions on Systems, Man, and Cybernetics.-Part A: Systems and Humans, vo. 31, pp , [12] Y. L. Ip, A. B. Rad, and Y. K. Wong, "Autonomous Exporation and Mapping in an Unknown Environment," in Proceedings of the Internationa Conference on Machine Learning and Cybernetics, 2004, pp [13] Y. W. Huang, S. Y. Chien, B. Y. Hsieh, and L. G. Chen, "Goba Eimination Agorithm and Architecture Design for Fast Bock Matching Motion Estimation," IEEE Transactions On Circuits and Systems For Video Technoogy, vo. 14, pp , [14] J. Vanne, E. Aho, T. D. Hämääinen, and K. Kuusiinna, "A High-Performance Sum of Absoute Difference Impementation for Motion Estimation," IEEE Trans. on Circuits and Systems, vo. 16, pp , [15] M. Z. Brown, D. Burschka, and G. D. Hager, "Advances in computationa stereo," IEEE Transactions on Pattern Anaysis and Machine Inteigence, vo. 25, pp , [16] H. F. Ates and Y. Atunbasak, "SAD Reuse in Hierarchica Motion Estimation For The H.264 Encoder," in Proceedings of the IEEE internationa on Acoustic, Speech, and Signa proccessing [17] S. Gutiérrez and J. L. Marroquín, "Robust approach for disparity estimation in stereo vision," Image and Vision Computing, vo. 22, pp , [18] E. Izquierdo, "Disparity/segmentation anaysis: matching with an adaptive window and depth-driven segmentation," IEEE Trans. on Circuits and Systems for Video Technoogy, vo. 249

7 9, pp , Contact: Le Minh Facuty of Eectrica and Eectronics Engineering, University of Technica Education HCMC, Vietnam. Emai: Te:

8 251