Feature-based mage regstraton usng the shape context LEI HUANG *, ZHEN LI Center for Earth Observaton and Dgtal Earth, Chnese Academy of Scences, Bejng, 100012, Chna Graduate Unversty of Chnese Academy of Scences, Bejng, 100190, Chna Image regstraton s a basc and mportant process for mult-sensor or mult-temporal remote sensng. In ths letter, a new feature-based method named shape context s proposed for arborne mult-sensor mage matchng. Ths method has been found to be robust n handwrtten dgt and object recognton, and t s now ntroduced nto remote sensng mage matchng after some adjustments. In the proposed method, control ponts (CPs) are extracted on the reference mage, and edge features are extracted on the reference and the sensed mage, respectvely. The shape context explots feature smlarty between crcular regons of the two mages to fnd correspondng CPs on the sensed mage. Fnally, the sensed mage s warped accordng to the CPs usng thn-plate splne nterpolaton. Ths method s successfully appled to regster arborne optcal and mult-band synthetc aperture radar (SAR) mages n two experments, and the results demonstrate ts robustness and accuracy. 1. Introducton When dsasters such as earthquakes and floods occur, arborne synthetc aperture radar (SAR) and optcal mages are effcent and flexble data sources that can provde repeated mages wthn short tme ntervals. In key areas, large numbers of mages are acqured; these mages commonly need to be combned together to analyse the ground stuaton. Tmely mult-sensor mage regstraton can be a great challenge. Image regstraton s the process of overlayng two or more mages of the same scene taken at dfferent tmes, from dfferent vew ponts, or by dfferent sensors (Ztová et al. 2003). In remote sensng, t s a basc process for many applcatons, such as change detecton, fuson, mosacs (Chen et al. 2007) and Interferometrc Synthetc Aperture Radar (InSAR). Tradtonally, regstraton s performed wth control ponts (CPs) that are selected manually on mages. However, manual regstraton s a tme consumng task, and many automatc methods have been presented to decrease labour ntensty. In general, automatc regstraton falls nto two types: area- and feature-based technques. Area-based methods are relatvely accurate and are preferably appled when dstnctve nformaton s provded by grey values rather than by local shapes and structures (Ztová et al. 2003). Feature-based methods are typcally appled when the local structural nformaton s more sgnfcant than the mage ntensty nformaton. They allow the regstraton of mages of a completely dfferent nature and can handle mage dstortons to some extent. Feature-based methods bascally consst of four steps: feature extracton, feature matchng, transform model estmaton and mage transformaton. In ths letter, attenton s concentrated on feature matchng. In prevous research, many matchng strateges have been employed for remote sensng mages. The Hausdorff dstance measures the extent to whch each pont of a data set les near some pont of an mage set and vce versa (Huttenlocher et al. 1993), and t s mproved to reduce the computatonal complexty for use n remote sensng (Mount * Correspondng author. Emal address: hlhjsx@126.com
1999). Patch features are used n SPOT and ERS-1 mage matchng, and the area, permeter length, length and wdth are used to determne the smlarty between patches (Dare et al.2001). In urban areas, lne features can be extracted from roads and be matched wth the Modfed Iterated Hough Transform (Habb et al. 2005). Invarant-moment shape features are extracted and chan-code matchng s used (Da et al. 1999). In Wen et al. (2008), spatal relatons and organc feature smlarty are combned as a matrx, and ts global maxmum s assumed to be reached when two mages match well wth each other. Generally, the feature matchng strateges mentoned above rely on strong features, such as specal ponts, slands, closed lakes or straght lnes. A broadly sutable algorthm called shape context s ntroduced, even for weak feature areas. As far as the authors know, ths study s the frst tme that shape context has been used for remote sensng mages. The study areas consst of flat farmland, n the Schuan Provnce of Chna, and the usual feature s the feld rdge. The data are obtaned from arborne optcal and mult-band SAR mages. 2. Method The shape context s presented for object recognton on slhouette mages (Belonge et al. 2002), and t s brefly ntroduced n ths secton. In addton, some adjustments are made to make t ft for use wth complex remote sensng mages. 2.1 Shape Context Edge features extracted from mages are consdered as pont sets. For a pont on the frst mage, t s expected to fnd the best matchng pont on the second mage accordng to the edge features. It s dentfed that the dstrbuton over relatve postons s a more robust and compact, yet hghly dscrmnatve descrptor. For a pont p on the shape, a coarse hstogram h of the relatve coordnates of the remanng n-1 ponts s computed: h ( k) = #{ q p : ( q p ) bn( k) } (1) (Fgure 1) Ths hstogram s defned as the shape context of p. The bns that are unform n log-polar space make the descrptor more senstve to the postons of nearby sample ponts than to those of ponts further away. An example s shown n fgure 1(c). Consder a pont p on the frst shape and a pont q j on the second shape. Let C, j = C( p, q j ) denote the cost of matchng these two ponts. As shape contexts are dstrbutons 2 represented as hstograms, t s natural to use the χ test statstc: where (k) h and h j (k) the cost to match two pont sets. 2.2 Adjustments C, j j 1 2 K 1 h( k) hj( k) C( p, q ) = (2) 2 h( k) + h ( k) j denote the K-bn normalsed hstograms at p and q, respectvely; j j C s
The radus and orentaton of the crcular template are adjusted to make the algorthm adequate for remote sensng mage matchng. Accordng to the defnton of log-polar bns, pxels are ndexed by the rng number R and the wedge number W. The rad of the rngs surroundng the centre are r, 2r, 4r, 8r, and 16r. If r s set to two pxels as the radus of the smallest rng, the result becomes unstable, because the radus s too small to tolerate SAR mage dstortons. Therefore t s suggested that r equals four pxels. The radus s also related to the scalng. The spatal resolutons of a reference and a sensed mage are known, and ther rato s λ. λ can be used to adjust the crcular template sze when the two mages have dfferent spatal resolutons. If the radus used n the crcular template s r for a reference mage, then the radus should be λr for the correspondng sensed mage. The largest adjustment s that the crcular template s gven an orentaton. In slhouette mage matchng, one can use a relatve frame, based on treatng the tangent vector at each pont as the postve x-axs. For mult-sensor mages, ths does not work because edge features are too complex to stably determne the tangent vector. In a more practcal way, t s supposed that the SAR mage has a rotaton, compared to an ortho-mage; thus, the crcular template s gven a general rotaton angle α to correspond wth the SAR mage, as shown n fgure 2. (Fgure 2) Compared to the spaceborne data, the swath of an arborne SAR mage s much narrower. In addton, the study areas are flat farmland, so the dstorton arsng from terran can be neglected. The sde-lookng arborne radar mage s formed along the flght lne of an arplane, and the mage rotaton angle α can be estmated from the flght lne. In fgure 3, A s the startng pont and B s the end pont of the flght lne. The mage rotaton angle can be estmated from equaton (3): YB YA α = arctan (3) X B X A X A, YA, X B, Y are the coordnates of A and B, obtaned from a dfferental global postonng B system (DGPS) recever. The flght lne s desgned to be a straght lne that measures several klometres. (Fgure 3) The mage rotaton angle obtaned from the flght lne s only an estmated value, and t may be naccurate. However, t s confrmed that the shape context method s robust enough to tolerate small angle devatons. For an mage I, the mage s rotated through an angle θ, formng a new mage I, as n fgure 4. (Fgure 4) To regster the mages I and I, a rotaton angle θ + ε s assgned to the crcular template, where ε s the gven angle devaton. Regstraton s performed when ε ncreases from -20 to +20. As shown n fgure 5, when -3 <=ε <=3, the average coordnate devaton s wthn 0.4 pxels. In most cases, the angle devaton ε s between -1 and 1, whch translates to a devaton of about 0.2 pxels, accordng to fgure 5. The only dfference between mages I and I s the rotaton angle θ, so the coordnate devaton mentoned above arses from the
angle devaton ε. The experment demonstrates that the rotaton angle estmated from the flght lne s suffcent to obtan accurate regstraton results usng the shape context. (Fgure 5) 2.3 Matchng Procedure The entre matchng procedure was performed usng the followng steps. 1) Specal preparatons for the SAR mages. If the regstraton s performed on optcal and SAR mages, the SAR mage should be converted from slant range to ground range. All of the employed SAR mages are smoothed wth an enhanced-lee flter that s especally desgned to suppress speckle nose (Lee 1981). 2) Control ponts M = { m } are extracted from the reference mage wth the Harrs operator (Harrs et al. 1988). 3) Edge features are extracted from both the reference and the sensed mages wth the Canny operator (John 1986). 4) For each control pont m on the reference mage, the local edge feature dstrbuton s measured based on the shape context. Assumng that the canddate matchng pont s wthn an N N pxel area on the sensed mage, the matchng cost s calculated between m and each pont n the area wth equaton (2). A lower C, represents a hgher smlarty, and the pont j wth the lowest C, s selected as the correspondng pont of m j. The selected ponts n the sensed mage consttute a set of CPs S = { s }. 5) The nvald CPs are removed n ths step. The three ponts that take the lowest cost n shape context matchng are selected to estmate the affne transformaton parameters n equaton (4) x = a1x' + b1y' + c1 (4) y = a2x' + b2y' + c2 where a1, b1, c1, a2, b2, c2 are affne transformaton parameters, and ( x ', y' ) and ( x, y) are correspondng CPs n the reference mage and the sensed mage. Invald CPs are removed by comparng the dstance between the coordnates of the mapped control ponts and the actual ponts. 6) Thn-plate splnes (TPS) are used to warp the sensed mage. These are perhaps the most wdely used transformaton functons n mage regstraton wth nonlnear geometrc dfferences. The warpng of the sensed mage wth respect to the reference mage s accomplshed wth equaton (5), where ( x ', y' ) and ( x, y) are all of the vald CPs selected n step (5); W s the weght of the nonlnear radal nterpolaton functon K. The detals of TPS are ntroduced n Bentoutou et al. (2005). x y 3. Experments and analyss ( ) t WxK( ( x, y) x, y ) ' a11 a12 x x = + = 1 (5) + n ' a t 21 a22 y y WxK( ( x, y) ( x, y )) n = 1
To evaluate the robustness of the proposed regstraton method, two experments were performed. Frst, the method was performed for arborne optcal and SAR mages; second, t was then also performed for mult-band SAR mages. 3.1 Regstraton of Arborne Optcal and SAR mages The orgnal slant range SAR mage s converted to a ground range mage. Beng a rangng devce, radar records objects accordng to the dstance from the arcraft to the object, thus, formng a slant range mage. By applyng a correcton from slant range to ground range, the scale relatonshp between the mage and the ground becomes lnear. Ths s only an approxmaton and s based on the assumpton of level topography (Henderson et al. 1998). The pxel szes n the azmuth and the range drectons of the SAR data are both 0.5 m, whch accord wth the optcal mage. (Fgure 6) The arborne optcal ortho-mage, obtaned from ADS40, and the C-band SAR mage are employed n the frst group of experments. The rotaton angle estmated from the flght lne s 7.45 and s used to rotate the crcular template. The rotaton angle calculated from CPs usng the Cartesan coordnate system (Goshtasby et al. 1986) s 6.96, whch s close to the estmated value. The employed data and the matchng results are shown n fgure 6. 3.2 Regstraton of Mult-band SAR Images Arborne L-band and C-band SAR mages are employed n the second group of experments. The two mages arse from parallel flght lnes (perpendcular dstance of about 200 m), so the estmated rotaton angle between the two mages s zero. The spatal resolutons of the two knds of SAR data are the same. L-band radar operates wth a longer wavelength than C-band radar. The dfferent penetraton of the radar sgnals nto the ground and plants results n dfferent mage textures. The employed data and matchng results are shown n fgure 7. The regstraton s performed on slant range mages drectly because the two mages possess smlar dstortons. (Fgure 7) 4. Accuracy Analyss. To estmate the accuracy of the fnal regstraton results usng the proposed method, all of the vald CPs were used n the evaluaton. Parts of the CPs are lsted n table 1. Δ X and Δ Y are absolute devatons of CPs between the reference mage and the warped mage. The root mean square error (RMSE) nvolvng all of the CPs n experment A s 1.883 pxels n the X drecton and 1.752 pxels n the Y drecton, whle n experment B they are 1.138 and 0.964 pxels. The optcal and SAR mages have a lower matchng accuracy than the mult-band SAR mages. As mentoned above, mult-band SAR data arse from parallel flght lnes, so they have smlar dstortons and are more accurately regstered. (Table 1) 5. Conclusons
Ths letter develops a new method called the shape context, whch matches mages by comparng edge-feature dstrbutons n a defned crcular template. In ths study t s used on remote sensng mages for the frst tme after some practcal adjustments. The shape context method s nvarant to rotaton and scale when matchng the boundares of slhouette mages. However, t s hard to mplement t on remote sensng mages, whch are much more complex. For the employed arborne SAR data, the rotaton angle was estmated accordng to the flght lne, and the scale rato was supposed to be known; thus, the mage regstraton was performed wthout any manually selected CPs. The proposed method was valdated on optcal-sar mages and mult-band SAR mages n the experments. It s confrmed to be an effcent and broadly sutable feature-based method that can be used n mult-sensor regstraton. In future work, the accuracy and feasblty of the proposed method wll be tested on satellte sensor data usng orbt parameters. Acknowledgements Our research was supported by the Chnese Mnstry of Scence and Technology (Grant No. 2009CB723901, 2009AA12Z122, 2009AA12Z145). References BELONGIE. S., MALIK. J., Puzcha, J., 2002, Shape matchng and object recognton usng shape contexts. IEEE Transactons on Pattern Analyss and Machne Intellgence, 24, pp. 509-522. BENTOUTOU, Y., TALEB, N., KPALMA, K., and RONSIN, J., 2005, An automatc mage regstraton for applcatons n remote sensng. IEEE Transactons on Geoscence and Remote Sensng, 43, pp. 2127-2137. CHEN, F., WANG, C., ZHANG, H, 2007, Automatc matchng of hgh-resoluton SAR mages. Internatonal Journal of Remote Sensng, 28, pp. 3665-3678. DAI. X, KHORRAM. S., 1999, A feature-based mage regstraton algorthm usng mproved chan-code representaton combned wth nvarant moments. IEEE Transactons on Geoscence and Remote Sensng, 37, pp.2351-2362. DARE, P., DOWMAN, I., 2001, An mproved model for automatc feature-based regstraton of SAR and SPOT mages. Journal of Photogrammetry & Remote Sensng, 56, pp. 13-28. GOSHTASBY, A., STOCKMAN, G.C., PAGE, C.V., 1986, A regon-based approach to dgtal mage regstraton wth subpxel accuracy. IEEE Transactons on Geoscence and Remote Sensng, GE-24, pp. 390-399. HABIB, A., Al-RUZOUQ, R., 2005, Sem-automatc regstraton of mult-source satellte magery wth varyng geometrc resolutons. Photogrammetrc Engneerng & Remote Sensng, 71, pp. 325-332. HARRIS, C., STEPHENS, M., 1988, A combned corner and edge detector. In Fourth Alvey Vson Conference, UK, pp. 147-151. HENDERSON, F.M., LEVIS, A.J.(Ed.), 1998, Prncples and Applcatons of Imagng Radar, Manual of Remote Sensng, 3rd edton, vol 2. pp. 142-149 (New York: Wley). HUTTENLOCHER, D.P., G.A.Klanderman, W.A.Ruckldge, 1993, Comparng mages usng the Hausdorff Dstance. IEEE Transactons on Pattern Analyss and Machne Intellgence, 15, pp. 850-863. JOHN, C., 1986, A computatonal approach to edge detecton. IEEE Transactons on Geoscence and Remote Sensng, PAMI-8, pp. 679-698. LEE, J.S., 1981, Speckle analyss and smoothng of synthetc aperture radar mages. Computer Graphcs and Image Processng, 17, pp. 24-32. MOUNT, D.M., NETANYAHU, N.S., MOIGNE,J.L., 1999, Effcent algorthms for robust feature matchng. Pattern Recognton, 32, pp. 17-38.
WEN, G.J., LV. J., YU. W., 2008, A hgh performance feature-matchng method for mage regstraton by combnng spatal and smlarty nformaton. IEEE Transactons on Geoscence and Remote Sensng, 46, pp.1266-1277. ZITOVÁ, B., FLUSSER, J., 2003, Image regstraton methods:a survey. Image and Vson Computng, 21, pp. 977-1000. Fgure 1. Shape context computaton and matchng. (a) and (b) Sampled edge ponts of two shapes. (c) Dagram of log-polar hstogram bns used to compute the shape contexts. Fve bns for log r and 12 bns for θ are used. In the followng text t s called the crcular template. (d), (e), (f) Example shape contexts for reference samples marked by,, and< n (a) and (b). Each shape context s a log-polar hstogram of the coordnates of the rest of the pont set measured usng the reference pont as the orgn. (Dark=large value) Note the vsual smlarty of the shape contexts for and whch were computed for relatvely smlar ponts on the two shapes. In contrast, the shape context for < s qute dfferent. Fgure 2. Rotaton of hstogram bns. (a) Crcular template wthout rotaton. (b) Crcular template wth rotaton by angle α. The rotaton of the crcular template s n accord wth the flght lne. Fgure 3. Estmaton of rotaton angle from the flght lne of the arplane Fgure 4. Left mage I, a GeoEye mage downloaded Fgure 5. Curve dagram of rotaton angle matfrom the nternet; on the rght s mage I whch s chng accuracy. A total of 76 ponts are matched rotated through an angle θ =20 from mage I. on mages I and I of fgure 4. The averaged devaton of the 76 ponts s calculated when the rotaton angle of the crcular template changes from 0-40.
Fgure 6. Regstraton of optcal and SAR mages. (a) CPs on the reference mage. (b) CPs on the sensed mage (c) Warped sensed mage. (d) The matchng result. Fgure 7. Regstraton of mult-band SAR mages. (a) L-band SAR mage. (b) C-band SAR mage. (c) Matchng result of the L and C bands. Top left and bottom rght of mage c are from the C-band mage, and the other two parts are from the L-band mage. Table 1. Results of the accuracy evaluaton.