A Panoramic Stereo Imaging System for Aircraft Guidance

Transcription

1 A Panoramic Stereo Imaging System for Aircraft Guiance Saul Thurrowgoo 1, Wolfgang Stuerzl 2, Dean Soccol 1, Manyam Srinivasan 1 1 Biorobotics Laboratory, Queenslan Brain Institute The University of Queenslan, Brisbane, Queenslan, 4072, an ARC Centre of Excellence in Vision Science, Australia 2 Computer Science Department VI, Technical University Munich, Boltzmannstr. 3, Garching, Germany s.thurrowgoo@uq.eu.au, stuerzl@in.tum.e,.soccol@uq.eu.au, m.srinivasan@uq.eu.au near-panoramic vision as well as near-panoramic range range information. Abstract This paper escribes an imaging system for aircraft guiance that provies panoramic stereo vision to facilitate surveillance, obstacle etection an avoiance, terrain following an laning. The system uses a single vieo camera in conjunction with a set of specially shape reflective surfaces to enable the etection of objects in the environment an estimation of their range through stereo. The avantage of this evice over systems that use optic flow information for visual guiance is that it oes not rely on the movement the aircraft. Therefore, it can be use even at low flight spees or hover Design of imaging system Curve reflective surfaces m A Plane mirrors B L m 1. Introuction There is consierable interest in the esign of guiance systems for UAVs that use passive sensing (such as vision), rather than active sensing which can be bulky, expensive an stealth-compromising. A recent tren in bioogically inspire vision systems for aircraft guiance has been to exploit optic flow information for collision avoiance, terrain following, gorge following an laning (e.g. [Chahl an Srinivasan, 1999; Barrows et al., 2003; Srinivasan et al., 2004; Ruffier an Franceschini, 2005; Zufferey an Floreano, 2006]). However, the acquisition of optic flow requires that the aircraft be in motion, which makes such systems unviable uring perios of low flight spee or hover. On the other han, the use of stereo vision enables estimation of range regarless of the aircraft s spee, an works even when the aircraft is stationary. Moreover, systems that rely on optic flow for extracting range information nee to iscount components of optic flow that are inuce by rotations of the aircraft, an use only those components of optic flow that are generate by the translational component of motion. The reason is that it is only the translatonal components of optic flow that provie information on the range to objects in the environment. Vision systems that exploit stereo information o not require this computationally elaborate proceure. Wie-angle stereo systems have previously been esigne for aircraft (e.g. [Tisse et al., 2007]). However, they typically use two cameras, which require synchronization an alignment. Furthermore, they usually use conventional fish-eye lenses that rarely cover visual fiels greater than a hemisphere. The system presente here uses a single camera, an can potentially eliver Camera Fig. 1 Schematic illustration of imaging system The system consists of a single camera that views a pair of curve, horizontally appose reflecting surfaces via a pair of angle plane mirrors, as shown in Fig. 1. Each of the curve surfaces is esigne to capture a nearpanoramic view of the environment. Their profiles are esigne to achieve constant elevational gain, like the panoramic reflective mirrors escribe in [Chahl an Srinivasan, 1997]. This angular gain is Unlike many other optical systems that have been use for panoramic imaging [Yagi et al., 1996; Nayar, 1997; Hicks an Bajcsy, 2001], the constant-gain esign use here ensures that a given angular elevation always subtens a constant number of pixels in the camera, thus making optimum use of the camera s resolution. This esign also simpifies the calculation of stereo isparities, as iscusse below. The effective viewpoint of each curve reflector is shown by the blue ot (A, B). (The viewpoint is not constant, but waners by a few mm epening upon the irection of view [Chahl an Srinivasan, 1997].) In effect, the system simulates a pair of human eyes, each with a near-panoramic fiel of view, an with a stereo baseline (interocular separation) corresponing to the istance between A an B (ca. 80mm). Each of the curve mirrors carries a 7 mm ia. axial aperture, through which a region

2 in front of the evice is viewe stereoscopically via reflection by a pair of auxiliary plane mirrors m, as shown by the green rays. This facilitates high-resolution viewing an ranging of objects irectly in front of the vision system (i.e. behin the camera). A lens L, associate with each auxiliary mirror, ensures that the camera can be focusse optimally to view the panoramic images as well as the frontal views. 3. Realization of imaging system Fig. 2 shows a realization of the system. Each curve reflector has a base iameter of mm an a base-totip height of mm. The istance between the bases of the reflectors is mm. The plane mirrors consist of 45 eg prisms with reflective bases (Emun Scientific) of imension 35 mm x 25 mm. An analog CCD camera (640x 480 pixels) with a FOV of 40 eg x 60 eg is use. The camera lens is focusse to a point 6mm below the tip of each reflector, to ensure that most of the If we assume (as an approximation) that the viewpoints of the curve mirrors are fixe at positions A an B (Fig. 1), then, for any plane passing through these points, the locus of constant isparity will lie on a circle in the plane that passes through A an B, with a raius equal to, 2Sin where is the istance between A an B (the stereo baseline) an is the angular isparity (Fig. 3). The centre of the circle (C) will be at a perpenicular istance of from the baseline (Fig. 3). 2Tan C A B Fig. 3. Locus of constant angular isparity in a plane. Fig. 2. View of realization of imaging system image capture by the curve reflectors is in focus. (The position of focus varies by a few mm, epening upon the irection of view [Chahl an Srinivasan, 1997]. The components are mounte rigily on a purpose-machine aluminium base, but with provision for minor ajustments of alignment. In the current harware version the auxiliary frontal stereo system has not been implemente, an this feature will not be iscusse any further. A B The fiel of view of the system (5.42 steraians mono, 4.05 steraians stereo) is not as large as it can potentially be, because (a) the brackets in the frame to support the whole assembly cause some occlusion of the visual fiel - - they were mae relatively large in cross section to ensure rigiity an resistance to impact; an (b) the boy of the camera that was use in this initial realization was relatively large, thus occluing some of the visual fiel. These esign shortcomings will be remove in subsequent realizations. 4. Loci of constant isparity Fig. 4. Surface of constant angular isparity in 3-D. In three imensions, the loci of all points that have a given isparity will therefore correspon to the surface of revolution that is generate by rotating the circle about the stereo baseline, as shown in Fig. 4. Thus, surfaces of constant angular isparity will have the shape of a torus

3 whose ring has a centre raius of thickness. Sin, an a 2Tan 5. Tests in an inoor laboratory environment Fig. 5a shows a raw image capture by the camera in an inoor laboratory setting. Figs. 5b an 5c show igitally unwarpe versions of the subimages from the left an right mirrors. The unwarping is one by representing the pixel positions in each subimage of the raw image in terms of polar coorinates (r, ) relative to the centre of each subimage, an replotting these as Y an X coorinates, respectively, in cartesian space. Because of the constant angular gain property of the curve mirrors, the Y isplacement of a given feature between the two unwarpe images is irectly proportional to the angular isparity of that feature. This facilitates irect an efficient computation of range. a 6. Offset measurement an compensation The Y isplacement between the two unwarpe images represents pixel isparity only if the optics of the system are perfectly aligne, an any systematic offsets are measure an correcte for. Since no alignment can be perfect, the offsets were etermine by viewing a istant point source (the moon, which, theoretically, shoul have zero isparity) an measuring the X an Y offsets between the moon s positions in the unwarpe left an right subimages. The offsets were measure for a wie range of viewing irections by mounting the system on a tripo an scanning the night sky uring a full moon. Fig. 6a shows one example of the raw images of the moon as capture by the left an right curve mirrors. Fig. 6b shows the positions of the moon in the unwarpe subimages. The X an Y offsets, interpolate between sample points, are shown in Fig. 7. These offsets, which are isparities cause by misalignments an lens istortions, shoul be subtracte out before the true isparity is measure. Once the offsets have been etermine in this way, true isparities can be measure for a stanar 3- environment (e.g. a texture chamber) an this stanar environment can be use to re-calibrate the offstes whenever necessary, without any nee to rely on the moon again. 7. System valiation an range calibration Having etermine the offsets, the next step is to calibrate the isparity reaings in terms of range. This was one by positioning the evice in front of a vertical, visually texture surface an acquiring images at various istances from the surface. The viewing irection was normal to the axes of the two curve surfaces, an was b c Fig. 5 Raw image of an inoor laboratory scene (a), an unwarpe versions of subimages from left mirror (a) an right mirror (b). irectly opposite to the camera s optical axis. At each istance, the Y isplacement of a frontal 41 x 41 pixel patch was measure between the two images, after the X an Y offsets were remove. This is the true isparity, after accounting for the X an Y offsets. The resiual Y isplacement was compute using a correlation algorithm

4 [Fua, 1993]. The results are shown in Fig. 8. The ata are well approximate by a rectangular hyperbola, as woul be expecte if isparity is inversely proportional to range. The isparity map compute for the scene of Fig. 5 is shown in Fig. 9. While this computation has not been groun-truthe, it is clear that there is goo qualitative agreement with the relative ranges to the various objects in the scene. 40 a Disp arity (p ix e ls ) b Fig. 6. Raw images of moon (a) an their positions in the unwarpe subimages (b) Range (mm) Fig. 8. Range calibration. Diamons show isparity reaings at ifferent ranges. Blue curve shows least squares fit of a rectangular hyperbola: Disparity = 19690/Range a b Fig. 7. Results of moon-base offset measurements: Measure X offset (a) an Y offset (b). Scale (Black to white) ranges from -30 to +30 (X) an 10 to 40 (Y). Fig. 9. Disparity map compute for scene of Fig. 5. Disparity is inversely proportional to range.

5 8. Test of system on an aircraft The system was then teste by mounting it on the unersie of a moel aircraft (Fig. 10). A raw image taken at low altitue an level flight is shown in Fig. 11. Unwarpe versions of this image, an the compute isparity map are shown in Fig. 12. As expecte, the isparity of the groun is high everywhere, inicating its proximity. The isparity is highest irectly below the aircraft an ecreases with more lateral viewing irections, as woul be expecte with flat horizontal terrain. Fig. 13 shows results from flight at a higher altitue, where the aircraft has a positive pitch attitue an is banking sharply to the right. As expecte, the isparities are generally much lower than in the case of Fig. 12, because of the greater altitue, an the groun is much closer towar the rear right-han portion of the system s visual fiel, because of the positive pitch an the sharp bank to the right. Thus, the isparity map provies information on the height of the aircraft above the groun, as well as its attitue relative to the groun. The results inicate that the vision system is well suite for use in tasks such as automate terrain following, attitue control an laning. Fig. 11. Raw image of terrain as capture by left an right mirrors Repeate offset calibration checks using the moon reveal that the system is quite robust to the vibrations that occur uring flight. A noteworthy aitional feature of the vision system is that the wie visual fiel of the evice permits imaging of a substantial portion of the horizon aroun the aircraft. As the attitue of the aircarft changes, the horizon shifts an istorts in preictable ways that epen upon the magnitue an polarity of the roll an the pitch. This offers the potential of monitoring (an controlling) aircraft attitue, which is a topic for future stuy.. Fig. 12. Unwarpe versions of the raw image in Fig. 11 (left an right, above) an isparity map (below) for level flight at a low altitue. 9. Conclusions Fig.10. View of system mounte on a moel aircraft. The camera faces backwars, but the imaging system captures a frontal view. This stuy has escribe the esign of a passive, nearpanoramic stereo vision sensor that can be use for surveillance, measurement an control of flight height, obstacle avoiance, an laning. A vieo camera is use in conjunction with two specially shape curve reflecting surfaces to achieve near-panoramic vision, an efficient computation of range through stereo. The avantage of using this visually-base metho of range computation, rather than the more traitional metho base on optic flow, is that information on range can continue to be obtaine even when the aircraft is stationary or moving slowly.

6 11. References [Barrows et al., 2003] G.L. Barrows, J.S. Chahl an M.V. Srinivasan. Biologically inspire visual sensing an flight control. The Aeronautical Journal, Lonon: The Royal Aeronautical Society 107:1069, , [Chahl an Srinivasan, 1997] J.S. Chahl an M.V. Srinivasan. Reflective surfaces for panoramic imaging. Applie Optics 36: , 1997 [Chahl an Srinivasan, 1999] J.S. Chahl, an M.V. Srinivasan. Panoramic vision system for imaging, ranging an navigation in three imensions. Proceeings, Fiel an Service Robotics Conference, Pittsburgh, August 29-31, , [Fua, 1993] P. Fua. A parallel stereo algorithm that prouces ense epth maps an preserves image features. Machine Vision an Applications 6 (1): 35-49, December Figure 13. Unwarpe images (above) an pixel isparity map (below) for higher altitue flight wih positive pitch an right bank The system also permits the acquisition of optic flow information, if esire, to assist visual guiance. Future work will explore close-loop tests of height an attitue control, an the use of the horizon for stabilization of attitue. 10. Acknowlegements This work was supporte partly by the US Army Research Office MURI ARMY-W911NF041076, Technical Monitor Dr Tom Doligalski, US ONR Awar N an the ARC Centre of Excellence Grant CE [Hicks an Bajcsy, 2001] R.A. Hicks an R. Bajcsy. Reflective surfaces as computational sensors. Image an Vision Computing, 19: , [Nayar, 1997] S.K. Nayar. Omniirectional vieo camera. Proceeings of the 1997 DARPA Image Unerstaning Workshop, May [Ruffier an Franceschini, 2005] F. Ruffier an N. Franceschini. Optic flow regulation: the key to aircraft automatic guiance. Robotics an Autonomous Systems 50: , [Srinivasan et al., 2004] M.V. Srinivasan., S.W. Zhang, J S Chahl, G Stange an M Garratt. An overview of insect inspire guiance for application in groun an airborne platforms. Proc Inst Mech Engnrs Part G, 218: , [Tisse et al., 2007] C-L. Tisse, O. Frank, H. Durrant-Whyte. Hemispherical epth perception for slow-flyers using coaxially aligne fisheye cameras. Proceeings, International Symposium on Flying Insects an Robots, Monte Verita, Ascona, Switzerlan, P. 123, Aug 12-17, [Yagi et al., 1996] Y. Yagi, W. Nishii, K. Yamazawa an M. Yachia. Stabilization for mobile robot by using omniirectional optic flow. Proceeings of the International Conference on Intelligent Robots an Systems, (Institute of Electrical an Electronics Engineers, Piscataway, N.J.), , [Zufferey an Floreano, 2006] J.C. Zufferey an D. Floreano. Fly-inspire visual steering of an ultralight inoor aircraft. IEEE Transactions on Robotics 22: , 2006.