IMAGE MOSAICKING FOR ESTIMATING THE MOTION OF AN UNDERWATER VEHICLE. Rafael García, Xevi Cufí and Lluís Pacheco

Transcription

1 IMAGE MOSAICKING FOR ESTIMATING THE MOTION OF AN UNDERWATER VEHICLE Rafael García, Xevi Cufí and Lluís Pacheco Coputer Vision and Robotics Group Institute of Inforatics and Applications, University of Girona Edifici Politècnica II, Escola Politecnica Superior Girona, Spain e-ail: Abstract: A coposite iage constructed by cobining a set of saller iages is known as osaic. Mosaics of the ocean floor are very useful in undersea exploration, creation of visual aps, navigation, etc. A feature-based osaicking ethod is proposed, based on textural paraeters of certain parts of the iage. s significantly help in the identification of given features of the iage. The tracking of these features over tie solves the atching proble in consecutive fraes. The correspondence is established with the aid of a cross-correlation algorith, applied to the colour coponents of the iage in the HSI space. Once the correspondence has been found a displaceent vector is obtained relating the features of two iages of the sequence. The otion paraeters between consecutive fraes are estiated through an error iniisation technique. Once the best transforation between two fraes has been found, iages are warped together coposing the osaic, and inforation about the vehicle otion is recovered. The experiental results on real iages show the effectiveness of the proposed ethod. Copyright 2000 IFAC Keywords: Coputer Vision, Motion estiation, Autonoous vehicles, Iage registration, Robot navigation, Marine Systes. 1. INTRODUCTION A osaic is a set of iages which have been cobined into a single and larger coposite iage (Marks, et al., 1995). Video osaics of the ocean floor are very useful for any applications, including undersea exploration, creation of visual aps, navigation, pipe inspection, etc. However, the application of standard coputer vision techniques for underwater iaging involves dealing with different probles. Underwater iages often lack distinct features (e.g., points, lines or contours) that are coonly exploited in terrestrial vision systes for tracking, positioning, navigation, etc. Moreover, the range is liited and the need for artificial light introduces any new properties to the iage, such as low contrast and non-unifor illuination. Quite often, sall observable particles suspended in the water show up as arine snow aking feature extraction difficult. For all these reasons, the processing of underwater iages appears as a challenging area of research. For ocean floor osaics, the individual iages foring the osaic are usually obtained by setting a caera on a ROV (Reotely Operated Vehicle) or AUV (Autonoous Underwater Vehicle). The caera is typically looking down to the bed of the sea, and the acquired iages cover a sall area of the ocean floor. By using this technique, the position and orientation of the underwater vehicle can be calculated by integrating the otions fro one iage to the next one (Xu and Negahdaripour, 1997; Negahdaripour, et al., 1998a). If our ai is siply to construct a ap of the bed of the sea, other ethods are found in the literature. Side scan sonar techniques have been applied to construct aps of the sea floor, see (Langer and Hebert, 1991). However, the reconstruction accuracy

2 is poorer than that provided by optical devices. Interesting results in colun-based osaicking can be found in the literature (Marks, et al., 1995) by using a constrained four-paraeter sei-rigid otion odel. Moreover, unconstrained iage osaicking can be obtained applying soother-follower techniques to reduce iage alignent errors within the osaic (Fleischer, et al., 1996). Soe of the projects involving experiental research in underwater iagery perfor osaicking in real tie (Negahdaripour, et al., 1998b), without the need of any additional inertial systes, since updating the position of the vehicle can be achieved solely fro visual inforation. Figure 1 shows how the otion of an underwater vehicle can be estiated fro a sequence of iages. Crosscorrelation Crosscorrelation Iage I k-1 Extraction Matching Estiation of vehicle otion Crosscorrelation Motion Registration between iages Iage I k Extraction Matching Estiation of vehicle otion Motion Registration between iages Iage I k+1 Extraction Fig. 1: Schee of the algorith for locating correspondence points between pairs of consecutive iages 2. BACKGROUND Mosaicking techniques can be divided into two categories: feature-based ethods, and featureless ethods. The first assue that feature correspondences between iage pairs can be established, and utilise these correspondences to find transfors which estiate the otion paraeters between iage pairs. This coputation of the otion paraeters is known as registration in the literature (Szeliski and Kang, 1995). The second ethod finds the transfors for iage registration by iniising a su of squared difference (SSD) function that involves soe paraeters. When applying these ethods to underwater iages, the extraction of features appears as a difficult task. Methods in this category typically assue that the change fro one iage to another be sall. In order to find the apparent otion fro one iage to another, good initial guesses for the paraeters of the transfor are required as initial values to find the optial solution. Failing that, the paraeter estiate process could fall into local inia (Szeliski, 1994). In this paper, a feature-based technique is proposed. The algorith is applied to a sequence of iages at a frae rate of 25 iages/second. In this way, the soothness requireent that pushes through the atching procedure is accoplished. Our ai is to use a projective transfor-based ethod for creating underwater osaics since no assuptions have to be ade in the caera otion. 3. THE ALGORITHM: MOSAIC CREATION In this section the proposed algorith for creating osaics fro a sequence of iages is exposed. The creation of the osaic is accoplished in three stages: feature selection and atching within iages, registration and osaic blending. 3.1 Feature atching The first step towards the estiation of the iage registration paraeters consists in finding feature correspondences. This is referred to as the atching proble, which is norally solved in aerial iages through the extraction of the high frequencies of the iage, discovering the outline of the objects present in the scene. However, in underwater applications, the nature of acquired iages presents serious difficulties when trying to atch iage corners or contours. Hence, a feasible alternative is to atch highly textured patches corresponding to given regions of the ocean floor. Our correspondence technique is a ixture of texture-based operators and correlation-based atching procedure. Large regions of the iages are used to find the atches. Thus, a significant percentage of the iage ust change for correspondence to fail. A colour caera and a standard frae-grabber copose the iaging syste of our underwater robot (Aat, et al., 1996). For every new iage in the sequence (I) a colour conversion is perfored, extracting the hue, saturation and intensity coponents of the iage. A set of texture paraeters is applied independently to every coponent, searching for distinguishable features in the iage. Once the set of candidate patches are defined, a sall window W i deliits the neighbourhood of every patch, and the grey-level of this window for the coponents hue, intensity and saturation are stored. When the next iage is acquired (I ), a new easure of textures is perfored in the vicinity of the initially defined windows for every coponent. In order to iprove the atching, a cross-correlation is carried out by taking the grey-level sub-iage of each and every patch and coparing it with the new iage. An

3 Original Iage Iage of texture patches Iage with atching points Low-cost Operators Identification of Interest Points Iage I Iage I T Iage I T Matching Module Low-cost Operators Identification of Interest Points Iage I Iage I T Iage I T Fig. 2. Schee of the algorith for locating points of iages I k and I k+1 in the coordinates syste of iage I k-1. attept is done in iage I for atching the previously selected features, by using both the texture-paraeters and cross-correlation criteria. Figure 2 shows the schee of the feature extraction procedure. The textural features used in this ipleentation have been chosen for their suitability for underwater iaging and their low cost in ters of coputing coplexity. A brief explanation of their ain characteristics is given below. For a ore detailed description see (Casals, et al., 1992). - Straightness: it defines the density of straight lines in a region. - Granularity: this paraeter quantifies the presence of a high density of non-concatenated gradients in a given patch. - Blurriness: it easures intensity changes in any direction within a region. A high value of this paraeter iplies that sooth changes are present. - Discontinuity: this paraeter easures the nuber of line discontinuities. - Abruptness: it indicates sudden changes of the direction of the lines within a region. Once the atching has been established a displaceent vector is obtained for every patch in the first iage. This vector relates the coordinates of the sae texture-feature in both iages. The assuption of large overlap of iage contents between the two fraes can be used to significantly reduce the coputational burden of the atching. This is achieved by liiting the search area in the second iage (I ). It is worth noting that iage otion that cannot be described by siple translation causes the displaceent vectors to be warped. Finally, the atching odule provides a list of coordinates of corresponding patches. Due to the error prone nature of the atching process, it is likely that a nuber of point correspondences will not relate to the sae 3D feature. For this reason, the next subsection is dedicated to the robust estiation of the otion paraeters taking into account the existence of isatches. 3.2 Global registration and osaicking This odule has the goal of estiating the otion paraeters between consecutive fraes. Starting fro the list of correspondences, a transforation atrix relating iages I and I has to be found. Projective transfors are coonly used since they perit arbitrary caera otions: translations, zooing, rotation around the optical centre, panning and tilting. The only constraint in osaicking applications is that the scene being recorded ust be planar. A projective transfor depicting the interfrae otion can be represented by a hoography, a linear function of projective iage coordinates (Szeliski, 1994): u'= Mu (1) where u' represents the iage coordinates of the projection of a 3D point P in the iage plane at tie instant k, and u represents the coordinates of the projection of the sae 3D point as projected the previous tie instant k-1. Expanding eq. (1) the following equation is obtained: ' ' x i ' yi ' xi yi 1 (2) where ( x i, y i ) and ( x i, yi ) denote a correspondence point in the present (I ) and previous iage (I), respectively; 11, 12,..., 32 are 8 paraeters that

4 deterine the projective transfor; and indicates equality up to scale. As we said before, the iage registration process consists in finding this 8- paraeter set, which is achieved through a least squares iterative process. Once the best transforation M has been found, iages I and I can be warped together. In order to construct the osaic, the first iage in the sequence is selected as a base frae. The osaic coordinate syste is placed at the origin of this reference frae. When a new iage has to be added to the osaic, atrix M provides its best fitting with respect to the previous iage (that at the beginning of the process will be the reference one). For every iage, the osaic is only updated within the regions where no inforation existed before. On the other hand, all the iages in the sequence are taken into account in order to keep a sooth track of the atching points. However, the osaic is updated with a new iage only every 10 fraes. In this anner, only the first easured value of a pixel updates the osaic. This is known as use-first ethod (Gracias and Santos-Victor, 1998). It has the drawback of the ipossibility of reoving transient data, such as a oving fish, but siplifies to a large extent the osaic construction. We plan to ipleent other techniques like a edian-based ethod, which effectively will reove oving objects whose intensity patterns are quiescent for less than half of the fraes. 4. RESULTS An experient on real data has been perfored, in order to check the suitability of the described texture paraeters, as well as cross-correlation estiation for the processing of underwater iages. The test sequence consists of a set of 180 underwater iages. Figure 3 shows a colour iage of the sequence and the result of perforing a contour detection on the luinance coponent. These iages have been taken fro our underwater robot, and belong to the iage sequence where the algorith is to be tested. If we were to analyse directly the iage of figure 3(b), it would be a difficult task to find a contour separating the ain regions of the iage. By applying to this iage a set of texture paraeters, iages like the one shown on figure 4(a) can be obtained. In this iage the texture coponents (texels) of one of the iages is shown. The result of finding the contours fro iage 4(a) is illustrated in figure 4(b). It can be observed how there is a contour separating the two ain regions of the iage. The points in the border of both regions provide considerable inforation about otion. For this reason cross-correlation is perfored for selected points over this edge line. Figures 4(c) and (d) illustrate the result of perforing the sae operation on soe other iage of the sequence. This tie the figure shows the result of applying the texture operator over the hue coponent of the iage. (a) (b) Fig. 3: (a) Original colour iage, and (b) extracted contours directly obtained over the luinance coponent. It should be noted that all the texture paraeters described in the previous section are applied to the three iage coponents hue, saturation and intensity, and then, the ain contours of the iage are obtained. In this way, the candidate points for coputing the cross-correlation are selected only fro the regions providing ore reliable contrast on the iage. Figure 5 illustrates the result of the crosscorrelation-based atching over two non-consecutive iages of the sequence. The atching is perfored for every iage in the sequence, applying a soothness constraint which facilitates the correspondence. In this way, the iage features can be tracked over the sequence. However, the osaic is not updated with every new iage of the sequence, but only when significant inforation can be added to the coposite osaic, speeding up the processing tie. Figure 6 shows the result of the resulting osaic with updates every 10 iages. The results here reported concern soe preliinary tests to check the ability of the algorith to recover inforation about the otion of the vehicle in different underwater situations. The extension of this ethod to ore coplex contexts, such as ore unifor iages including algae, sand, etc., will show the accuracy of this approach to be used as a position sensor in lieu of inertial-based positional systes.

5 (a) (b) (c) (d) Fig. 4: (a) Exaple of lineality textural feature obtained over the luinance coponent; (b) contours obtained fro (a). Cross-correlation is perfored for selected points over the resulting contours; (c) Exaple of lineality texture paraeter obtained over the hue coponent on another iage; (d) contours obtained fro (c). (a) (b) Fig. 5: Matched points between non-consecutive iages of the sequence 5. CONCLUSIONS AND FURTHER WORK A technique for creation of autoatic video osaics of the ocean floor has been presented. Our approach uses the iage textures over specially selected points to establish the correspondences fro one iage to the next one. Registration between fraes is coputed for every new iage of the sequence, but the osaic is updated every 10 iages, reducing the cycle tie of the algorith. Accuulation of iage alignent errors have been detected as the osaic increases in size. In the future, optial estiation techniques will be investigated to reduce these errors. Moreover, when oving objects are present in the scene, the registration between iages can suffer slight alterations. It sees that the use of a teporal edian filter on the overlapping pixels of the osaic would iprove the registration phase. REFERENCES Aat, J., J. Batlle, A. Casals and J. Forest (1996). GARBI: A Low Cost ROV, Constraints and Solutions. In Proceedings of the 6th IAPR Underwater Robotics. Casals, A., J. Aat, and A. Grau (1992). paraetrizacion ethod for iage segentation. In Proceedings of the Second European Conference on Coputer Vision, pp

6 Fleischer, S. D., H. H. Wang, S. M. Rock, and M. J. Lee (1996). Video Mosaicking Along Arbitrary Vehicle Paths, in Proceedings of the Syposiu on Autonoous Underwater Vehicle Technology, pp Gracias, N. and J. Santos-Victor (1998). Autoatic osaic creation of the ocean floor. In Proceedings of the OCEANS Conference, vol. 1, pp Langer, D. and M. Hebert (1991). Building qualitative elevation aps fro side scan sonar data for autonoous underwater navigation. In Proceedings of the IEEE International Conference on Robotics and Autoation. Marks, R.L., S.M. Rock, and M.J. Lee (1995). Realtie video osaicking of the Ocean floor. IEEE Journal of Oceanic Engineering, vol. 20, no.3, pp Negahdaripour, S., X. Xu, A. Khaene and Z. Awan (1998a). 3D otion and depth estiation fro sea floor iages for osaic based station keeping and navigation of ROVs/AUVs and high resolution sea floor apping. In Proceedings of the IEEE Workshop on Autonoous Underwater Robots, pp Negahdaripour, S., X. Xu and A. Khaene (1998b). A vision syste for real tie positioning, navigation and video osaicing of sea floor iagery in the application of ROVs/AUVs, in Proceedings of the 4th IEEE Workshop on Applications of Coputer Vision, pp Szeliski, R. and S.B. Kang (1995). Direct ethods for visual scene reconstruction. In Proceedings of the IEEE Workshop on Representation of Visual Scenes, pp Szeliski, R. (1994). Iage osaicing for tele-reality applications. In Proceedings of the IEEE Workshop on Applications of Coputer Vision, pp Xu, X., and S. Negahdaripour (1997). Vision-based otion sensing fro underwater navigation and osaicing of ocean floor iages, in Proceedings of the MTS/IEEE OCEANS Conference, vol. 2, pp Fig. 6: Mosaic creation fro a sequence of 180 iages. One iage every 10 is considered for actualising the osaic