A NEW GROUND-BASED STEREO PANORAMIC SCANNING SYSTEM

Transcription

1 A NEW GROUND-BASED STEREO PANORAMIC SCANNING SYSTEM R. Li*, L. Yan, K. Di, B. Wu Mapping and GIS Laboatoy, Dept. of Civil and Env. Eng. and Geodetic Science, The Ohio State Univesity 47 Hitchcock, 27 Neil Ave., Columbus, OH , USA {li.282, yan.351, di.2, Commission V, WG V-5 KEY WORDS: Panoamic Scanning System, Panoamic Camea, Multi-pespective Panoama, Geometic model, 3-D Mapping ABSTRACT: This pape intoduces a linescan-based steeo panoamic scanning system that employs an off-ais camea configuation mode. In this mode, the two steeo linescan cameas ae mounted off-ais on the hoizontal ba of a camea mast, appoimately equidistant fom the otation ais. Compaed to the on-ais mode used in many othe linescan-based panoamic scanning systems, the off-ais mode enables the system to acquie multi-pespective steeo panoamas that can povide unifom accuacy in all 36º diections at each depth. A pototype using a colo linea aay camea with RGB channels has been developed at the Mapping and GIS Laboatoy of The Ohio State Univesity. The futue vesion of this system will be a Mobile Panoamic Multi-spectal Scanne (MPMS) in which each camea will include multi-spectal channels such as VNIR (Visible and Nea Infaed) and SWIR (Shot Wavelength Infaed). This new system has the potential to be used in futue Mas landed missions fo the detection and mapping of mineals, wate and othe habitability signatues as well as fo suppot of Eath-based applications. 1. INTRODUCTION Linescan-based panoamic scanning systems ae based on otating linea aay sensos and ecoding image lines within a cylindical geomety. One of thei majo capabilities is quick image acquisition of any panoamic scene. Anothe is the acquisition of high-quality images having high esolution and homogeneous bightness and contast thoughout the entie image. These featues make linescan-based panoamic scanning systems suitable fo applications such as ecoding landscapes and city squaes, cultual heitage documentation, and data captue fo vitual eality applications. Eamples of two commecially available scanning systems used fo these puposes ae the EYESCAN M3 teestial digital panoamic camea, which was jointly developed by the Geman Aeospace Cente (DLR) and KST Desden GmbH, and the Sasta Digital DRS 5 camea developed by Spheon PanoCam. Thee ae two camea configuation modes applied in panoamic scanning systems: the on-ais mode and the off-ais mode, designating whethe o not the camea s pespective cente is designed to be on the otation ais. The panoamas acquied with these two modes ae efeed to as eithe singlecente panoamas (on-ais) o multi-pespective panoamas (off-ais). Many systems use the on-ais mode such as the EYESCAN M3 and the Sasta Digital DRS 5, and singlecente panoamas have been well eseached concening calibation, geometic modelling, and bundle adjustment of the data (Schneide and Mass, 23; 24; 25; 26). Othe eamples of the eseach and application of these systems include visualization and tetue mapping (Haala and Kada, 25), and data fusion with lase scannes (Scheibe et al., 24). Although they have a simple geometic model, single-cente steeo panoamas have a disadvantage in steeo vision, which is fomed by two cameas otating aound two sepaate aes; the 3-D infomation deived fom such two steeo panoamas by spatial intesection does not have unifom accuacy in all 36º diections, paticulaly in the baseline diection. As a esult, moe panoamas nomally have to be taken in ode to avoid weak intesection geometies. Othewise, we can fom a vetical baseline, but this would cause difficulties in steeoscopic viewing. This may limit the potential of on-ais panoamic scanning systems in situations whee constaints (e.g., vetical height) would not allow a vetical steeo configuation, fo eample, fo a space eploation vehicle. On the othe hand, while multi-pespective (off ais) panoamas have a moe complicated geometic model, they ae able to ovecome the poblem of nonunifom accuacy in diffeent diections. Futhemoe, the off-ais mode offes moe fleibility in system configuation. The panoamic camea system pesented is a Mobile Panoamic Multi-spectal Scanne (MPMS) that employs the off-ais camea configuation mode. The system can be mounted on eithe a manned o unmanned vehicle. The designed system will include two steeo multi-spectal cameas, each having 1 vetical linea aay CCDs epesenting visible (RGB), infaed (IR) and othe multi-spectal channels. With only one sweep, MPMS can acquie all the channels of steeo panoamic images to fully chaacteize the suounding envionment. This system can be used fo envionmental monitoing, homeland secuity, tanspotation, and landed planetay missions. The system intoduced in this pape is a pototype of MPMS that uses a colo linea aay camea with RGB channels. A futue vesion will include multi-spectal channels. In the following section, the steeo camea configuation and hadwae integation of the pototype ae descibed. Section 3 addesses the geometic model and steeo geomety of multipespective panoamas. In Section 4, sample panoamas * Coesponding autho. 723

2 The Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences. Vol. XXXVII. Pat B5. Beijing 28 acquied using the pototype and some peliminay 3-D mapping esults ae shown. An oveall summay is given in the final section. 2. THE MPMS PROTOTYPE - AN OFF-AXIS PANORAMIC SCANNING SYSTEM 2.1 Configuation of the Steeo Cameas In a linescan-based panoamic scanning system using an offais camea configuation mode, thee ae many ways to configue the steeo cameas egading thei heights, otation aes, otation adius, etc. A geometic model that consideed these factos was descibed in Huang et al. (21). Steeo camea mount Steeo ba Camea panoamas (the otation angle) can be pe-defined in this softwae, and does not have to be 36º. Focal Length 35 mm Vetical dimension of CCD 4,8 piels Channels RGB Vetical FOV 64.5º Image Columns (36º, vaying) 22, (e.g.) Data Volume (36º) 22 MB Recoding time (1 ms/column, 36º) 3.3 min Table 1 Majo paametes of the MPMP pototype 2.3 Geometic Model of Multi-pespective Panoamas The geometic model of multi-pespective panoamas is simila to that of single-cente panoamas as descibed in Schneide and Mass (23). It diffes, howeve, in that it has two additional paametes: R, the otation adius, measuing fom the otation ais to the camea s pespective cente, and Ω, the swing angle between the baseline and the camea s line of sight. The steeo computation is based on the off-ais steeo model. P Mast Rotay stage contolle j z i η Rotay stage y Ω c ξ R o Figue 1. Hadwae components of the MPMS pototype In the design of this pototype, both steeo cameas have the same height, shae the otation ais and ae equidistant fom the otation ais. With this configuation, shown in Figue 1, the acquied steeo multi-pespective panoamas will have not only unifom accuacy in all 36º diections in each depth, but also, in theoy, hoizontal epipola lines. This will allow the use of simplified algoithms to quickly find coesponding points, and povides the potential fo onboad applications. 2.2 Hadwae Integation The majo hadwae components of the MPMS pototype include steeo cameas (optical system, linea aay CCDs and housing), a fame gabbe, and a suppot system that includes a otay stage with contolle, emote sensing mast, and steeo camea ba (see Figue 1). In the pototype we have used only one camea. To obtain steeo panoamas, the camea is altenately mounted on the opposite ends of the steeo ba. The height of the mast is adjustable fom 7 cm to 13 cm, and diffeent mounting positions on the steeo ba fo the cameas allow fo baseline lengths fom 2 cm to 5 cm. Table 1 shows the main technical paametes of this pototype. Softwae was developed to integate the hadwae components fo image acquisition, whee the eposue time (also scan time of an image line) of the camea and the otation speed ae configued to ceate diffeent esolutions in the hoizontal diection. The hoizontal Field of View (FOV) of acquied Figue 2. Geometical modeling of multi-pespective panoamas Optical cente y Image point P (Object point) Ω c R ξ Figue 3. Top view: intenal cicle epesents the tajectoy of the camea s pespective cente; etenal cicle epesents the cylindical panoama o 724

3 The Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences. Vol. XXXVII. Pat B5. Beijing 28 Figues 2 and 3 illustate the geomety of the left panoama, which is acquied with the camea being mounted on the left of the steeo ba, as shown in Figue 1. Thee ae fou coodinate systems consideed in the geometic model, the object coodinate system (X-Y-Z), the local camea coodinate system (-y-z), the cylindical camea coodinate system (-ξ-η), and the image coodinate system (i-j). The local camea coodinate system (-y-z) is an auiliay coodinate system. Its oigin, o, is the intesection point of the otation ais and the steeo ba, and its z-ais coincides with the otation ais. The cylindical camea coodinate system (-ξ-η) is a coodinate system adapted to descibe this special camea geomety. In this system, is the adius of the cylindical panoama, ξ is the angle between the positive y-ais and the otation adius, measued clockwise, and η is the same as z. When the otation adius R, the swing angle Ω, and the camea s focal length c ae fied, is then fied 2 2 ( = R + c + 2RccosΩ ), and each point coesponds to unique (ξ, η) coodinates. The elationship between a point s object coodinates (X, Y, Z) and its image coodinates (i, j) is defined by a sequence of coodinate tansfomations. The fist tansfomation is a igid body otation and tanslation fom the object coodinate system (X-Y-Z) to the local camea coodinate system (-y-z): y = z X X Y Y Z Z whee (X, Y, Z ) ae the object coodinates of the oigin, o, of the local camea coodinate system; 11, 12,, 33 ae components of the otation mati In the second tansfomation, the point (, y, z) is pojected onto the cylindical panoama. R sin Ω ξ = actan( ) Ω acsin( ) y y cz η = 2 + y 2 R 2 sin 2 Ω R cos Ω (1) (2a) (2b) Equation 2b epesents the pin-hole pespective pojection of the linea aay cameas in the off-ais panoamic scanning system. The last tansfomation tansfoms (ξ, η) into the image coodinates (i, j): whee ξ +ξ i = A h N η + η j = 2 A v ξ o = the bias of coodinate ξ η o = the bias of coodinate η N = esolution of the linea aay CCD A v = piel size of the linea aay CCD (3a) (3b) A h = hoizontal angula esolution of a piel Thus the collineaity equations fo the left panoama ae: R sin i actan acsin / A h y 2 2 y Ω = Ω + ξ + N cz j / y R sin R cos = + η + Ω Ω A v (4a) (4b) The obsevation equations that descibe the tansfomation fom the object point to the image point ae given by incopoating Equation 1 and Equation 4. In Equation 4, N and A v ae fied values associated with the camea. A h is dependent on the eposue time and otation speed, while ξ o, η o, c and R ae acquied though calibation. Fo the ight panoama, the collineaity equations ae: R' sin ' i' actan ' acsin Ω ' (5a) = Ω + ξ / A h ' y 2 2 y + N' c' z j' = + η ' / A v ' (5b) y R' sin ' R'cos ' + Ω Ω whee η o ', ξ o', A h ', A v ', c', N', R', Ω' ae the coesponding paametes of the ight panoama. The paametes in collineaity Equations 4 and 5 have the following elationships, although some of them ae difficult to ealize: 1) if the same camea is used to take both left and ight panoamas, then c = c', A v = A v ', and N = N'; 2) if the two steeo camea positions ae equidistant fom the otation ais, then R = R'; 3) if the two swing angles ae the same, then Ω = Ω'; and 4) if the two steeo camea positions have the same height, then η o = η o '. If all the above conditions ae met, a point in the object space will be pojected onto the same image ow in the two steeo panoamas. Also, the image ows in the ight panoama will be the epipola lines of coesponding ows in the left panoama. 2.4 Steeo Geomety Figues 4 and 5 illustate the steeo geomety of multipespective panoamas acquied by this pototype system. The collineaity equations ae used to model the optical ays fom the left and ight cameas to point P. Fom Figue 5(c) we can see that fo any point at a constant distance fom the mast (on a cicle) in the object space, the steeo intesection angle is the same. That eplains why the off-ais configuation of the cameas enables the system to have unifom 3-D measuement accuacy in all 36º diections. But the accuacy vaies at diffeence distances in each adial diection. 725

4 The Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences. Vol. XXXVII. Pat B5. Beijing DATA ACQUISITION AND 3-D MAPPING TESTS Epeiments wee pefomed unde diffeent situations to test the system as well as detemine the optimal configuation of the camea as to eposue time, apetue and othe paametes. p p P Figue 4. Spatial intesection of multi-pespective panoamas (a) Left-eye view (b) Right-eye view (c) Spatial intesection Figue 5. Top view of the steeo geomety Fo eample, when taking pictues outdoos, eposue time can ange fom 8, ms to 2, ms accoding to actual weathe conditions. Fo indoo image acquisition, a 3, ms eposue time is usually selected. And unde fluoescent lights, a fluoescent-cut filte is used on the camea lens. Sample panoamas taken with the pototype ae shown in Figue 6. Figues 7 and 8 show eamples of a peliminay 3-D mapping test using a pai of 18º steeo panoamas that cove an aea of elatively simple teain (Figue 7). Image matching was pefomed using coss-coelation followed by a Least-Squaes Matching. Afte coesponding points wee found, gound positions wee calculated using Equations 4 and 5. Consequently, a DTM was geneated by Kiging intepolation. Then an othophoto was poduced by back-pojection using Equation 4. The deived mapping esults ae shown in Figue 8. The calibation of the camea is pending at this time. (a) Deset site nea Reno, Nevada (b) Ohio State Univesity Stadium Figue 6. Sample panoamas taken with the pototype (a) Left panoama (18º) (b) Right panoama (18º) Figue 7. Patial steeo panoamas used fo 3-D mapping 726

5 The Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences. Vol. XXXVII. Pat B5. Beijing 28 REFERENCES Haala, N., and Kada, M., 25. Panoamic scenes fo tetue mapping of 3-D city models. Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences, Belin, Gemany, Vol. 36, Pat 5/W8. (a) DTM and contou map (b) 3-D view: othophoto daped on the DTM Figue 8. 3-D mapping esults 4. CONCLUSIONS AND FUTURE WORK In this pape, we pesented a linescan-based panoamic scanning system developed as the pototype of a Mobile Panoamic Multi-spectal Scanne that employs an off-ais camea configuation mode. The system acquies 3-D infomation with unifom accuacy in all diections at each depth, and has the potential fo apid onboad pocessing. This system can be used to impove the pefomance of 3-D mapping, navigation and object-detection, all of which ae vey impotant fo time- and enegy-limited applications. Peliminay 3-D mapping tests wee pefomed to test the system and the poposed geometic model of the multipespective panoamas acquied with the pototype. Futue wok will concentate on the calibation of this system with paticula emphasis on the mechanical eos in the suppot system. We will continue ou wok on futhe assessment of the system and the automation of the data pocessing pocedue. ACKNOWLEDGEMENTS Reseach Funding fom the OSU Office of Reseach is acknowledged. Collaboation with NASA JPL, the Space Science Institute, and the Geman Aeospace Cente (DLR) is acknowledged. Huang, F., Wei, S., K., and Klette, R., 21. Geometical fundamentals of polycentic panoamas. 8th Intenational Confeence on Compute Vision, Vancouve, Canada, Vol. 1, pp Maki, J.N., Bell III, J.F., Hekenhoff, K.E., Squyes, S.W., Kiely, A., Klimesh, M., Schwochet, M., Litwin, T., Willson, R., Johnson, A., Maimone, M., Baumgatne, E., Collins, A., Wadswoth, M., Elliot, S.T., Dingizian, A., Bown, D., Hageott, E.C., Sche, L., Deen, R., Aleande, D., and Loe, J., 23. Mas Eploation Rove engineeing cameas. Jounal of Geophysical Reseach, Vol. 18, No. E12, 871 [DOI: 1.129/23JE277]. Scheibe, K., Scheele, M., and Klette, R., 24. Data fusion and visualization of panoamic images and lase scans. Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences, Desden, Gemany, Vol. 34, Pat 5/W16. Schneide, D. and Maas, H.-G., 23. Geometic modelling and calibation of a high esolution panoamic camea. Optical 3-D Measuement Techniques VI, Zuich, Switzeland, Vol. II, pp Schneide, D. and Maas, H.-G., 24. Development and application of an etended geometic model fo high esolution panoamic cameas. Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences, Istanbul, Tukey, Vol. 35, Pat B5, pp Schneide, D. and Maas, H.-G., 25. Combined bundle adjustment of panoamic and cental pespective images. Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences, Belin, Gemany, Vol. 36, Pat 5/W8. Schneide, D. and Maas, H.-G., 26. A geometic model fo linea-aay-based teestial panoamic cameas. The Photogammetic Recod, Vol. 21, No. 115, pp

6 The Intenational Achives of the Photogammety, Remote Sensing and Spatial Infomation Sciences. Vol. XXXVII. Pat B5. Beijing