Perceptual Grouping for Building Recognition from satellite SAR Image Stacks

Transcription

1 Perceptual Grouping for Building Recognition from satellite SAR Image Stacks E. Michaelsen 1, U. Soergel 2, A. Schunert 2, L. Doktorski 1, and Klaus Jaeger 1 1 Fraunhofer-IOSB, Gutleuthausstrasse 1, Ettlingen, Germany eckart.michaelsen@iosb.fraunhofer.de 2 IPI, Leibniz Univ. Hannover, Nienburger Str.1, Hannover, Germany Abstract Modern high resolution satellite SAR sensors even allow analysis of building sub-structures like windows and balconies. In the amplitude data man-made objects usually appear either as salient bright lines or points embedded within dark background. The latter features may coincide also with so-called persistent scatterers (PS), whose phase history is exploited by time series analysis for 3D reconstruction and deformation monitoring. We apply principles of human gestalt perception for grouping urban objects such as entire facades. The analysis takes place both in the amplitude and the phase data. Fusion is possible at different levels, e.g: i) grouping results in one domain may focus search in the other domain, ii) both approaches allow to infer independently the 3D structure by exploiting complementary features (amplitude vs. phase), and iii) grouping as such is useful to introduce model knowledge. 1. Introduction The recent years saw the advent of a modern generation of SAR sensors like German TerraSAR-X (TSX) and Italian Cosmo-Skymed capable of mapping with spatial resolution of about one meter. In such data trihedral structures with edge length as small as only 6 cm (compared to 60 cm for sensors of ERS class) may cause dominant bright spots embedded into dark clutter [2]. As a consequence, about two orders of magnitude more such features are observed in the data. This offers new opportunities with respect to the analysis of urban structures based on SAR satellite images. For example, in Fig. 1a a TSX image taken over the city center of Berlin, Germany, is shown. Even though subject to SAR inherent layover with ground clutter, façade elements like windows and balconies form salient regular patterns consisting of many bright point-like primitives. Considering geometric constraints we may estimate from such features the building height from amplitude images. But there is an alternative and more accurate way to infer the 2.5D scene structure from SAR data: SAR Interferometry (InSAR). In recent years a special version of InSAR was developed initially almost exclusively used to determine surface deformation (e.g., due to mining activities): Persistent Scatterer Interferometry (PSI). This technique is tailored for urban scenes where man-made objects cause many scattering centers of high temporal stability, the PS. Those are identified in stacks of coregistered SAR images by a variety of techniques. Although it is not a must they often coincide with salient point structures in the amplitude data as discussed above. By PSI and more advanced techniques besides the deformation the 3D location of each PS can be determined with considerable accuracy [2]. However, common PSI processing schemes do not take advantage of the regular set-up of man-made structures and the possible redundancy due to the presence of many façade or roof objects on the very same building, but rather treat the set of individual PS as independent observations. In this contribution the same grouping steps as described for the amplitude data are also applied to PS data. Both reasoning lines can mutually support. We adapt previous own work using gestalt perception for building recognition on amplitude data [7] and also 2D grids of regular point scatterers [11]. The approach is extended to handle PSI results. Both processing lines are combined and compared. 2. Related work Structural knowledge-based computer vision in particular on aerial imagery has a remarkably long history [9]. Sophisticated production systems have been proposed e.g. in [4, 8]. Some of this work is being continued, including our own work [7] but often a certain lack of robustness was criticized. Attempts are known to unify statistical approaches with such /10/$ IEEE

2 syntactical or structural approaches [6]. It is difficult to achieve the low false alarm rate and high precision demanded e.g., for map update in an open world. 3. Radargrammetry and PSI Object extraction and elevation estimate may be conducted both from amplitude and phase data of SAR images. We introduce such methods briefly and accordingly choose suitable primitive objects for further process. 3.1 Radargrammetry Similar to optical images it is possible to infer the scene s 3D structure from radiometric features observed in the amplitude images. Those techniques refer under the umbrella term Radargrammetry. For example, a building s height can be deduced from its shadow cast on ground. Furthermore, based on image pairs SAR stereo is also possible. For such purpose, analogous to Photogrammetry the parallax of corresponding features is exploited [10]. In Fig 1a is displayedd the mean amplitude image of a stack of 11 TSX images taken over the city of Berlin, Germany. Groups of bright point scatterers embedded within dark clutter pop-up saliently in the image. In addition, we frequently observe also bright linear features often coinciding with object boundaries. The image of a group of horizontal façade elements along a building floor is usually in accordance with the façade s orientation relative to the sensor path. Since the viewing geometry of the sensor is known ( in the example from left to right), in case of multi-storey buildings we can a priori limit the search space for further floors to a 1D problem: starting from the highest floor located closest to the sensor the subsequent lower floors must lie on a straight line coinciding with the range coordinate of the SAR image (here: the image rows). Considering this model knowledge, after proper grouping and reasoning we may infer a building s height from a single SAR amplitude image by determining the spacing of its floors and counting their number. 3.2 PSI PSI is an extensionn of traditional InSAR: the phase difference of repeated data taken by satellite SAR sensors is exploited to infer object elevation and motion. The two basic limitations of repeat-pass InSAR are: i) temporal signal decorrelation and ii) phase delay caused by water vapor in the troposphere. PSI tackles both issues at the same time. It requires stacks of co-registered complex SAR images in which pixels are extracted that feature temporal persistent reflection properties. One common method of PS a) b) Fig. 1: SAR data, illumination from left: a) Stacked amplitude image, b) image a) superimposed with detected PS (color codes phase, 1 cycle ~ 15 m elevation) identification is to look for pixels that show smalll amplitude dispersion, i.e.., the ratio of standardd deviation to expectation value over the stack is small. Such PS are mostly caused by man-made Thus an irregular PS network of high density is found in cities and hardly any PS in rural scenery. The atmospheric phase screen is characterized by considerable long spatial correlation length compared to SAR pixel size, but short temporal correlation in the order of some hours. Hence, this error term is modeled as spatial low-passs and temporal high-pass signal and its estimate is subtracted from the phase of the PS network. Finally, the remaining phase term of each PS represents the local relativee elevation or motion, respectively, compared to a reference PS. In Fig 1b the extracted phase is superimposed as color for each detected PS. Obviously PS correlate with bright pixels in the amplitude image. The probability of small amplitude dispersion is high for highh signal-to-noise-ratio (i.e., for bright image pixel). Furthermore, we see that the phase cycles nicely run-up the facades. However, theree objects like façade and roof structures.

3 are still many outliers contained in this PSI result. Recall that neither model knowledge on object level nor grouping steps are carried out in PSI processing. Introducing such constraints should improve the results. In addition, we propose to do an independent analysis of the amplitude data: from the distance of rows in range direction we can infer the story height, and we can also count the number of stories. 3.3 Choice of primitive objects Summarizing the previous two sections we deal with point-like and line-like features in case of Radargrammetry and only point-like ones for PSI. In the former case we apply standard image processing techniques to extract primitive instances BrightPix and Line from the amplitude image. The PS candidates are hypothesized from the entire image stack using a standard PSI technique. For coding the primitives we use the concept BrightPix which was extended to store also the phase or height value. 4. Perceptual grouping by GESTALT system Table 1 lists the productions of the production system used here. Each such production has a hypothesis class on the left side, a mutual constraint on its parts, and two part classes. The system is used right to left implementing recognition from sets of primitive instances BrightPix and Line extracted from the SAR data. hypoth. constraint part0 part1 1 Spot proximity BrightPix BrightPix 2 RowSp proximity Spot Spot 3 RowSp good cont. Spot RowSp 4 Lattice proximity & RowSp RowSp similarity 5 Lattice good cont. RowSp Lattice 6 LoLine colinearity Line Line 7 LLattice close & LoLine LoLine parallel 8 LLattice good cont. LoLine LLattice Interpreting uncertain data using such combinatorial descriptions is a non-trivial endeavor. Complete and sound interpretation will usually not be feasible in the presence of large numbers of instances per image. There are means to overcome these problems mainly by avoiding exhaustive search and top-down control of the interpretation (details are given in [7]). Exemplarily Fig. 2 shows what top-down operations a newly inferred instance of the class RowSp will cause: Productions 2 and 3 constitute a syntactic recursive search for meaningful objects using good repetitive continuation as gestalt constraint. Following [3] the maximal meaningful elements should have high preference all hypotheses from this instance with other lefthand side are suppressed giving priority to further prolongation. Due to the Figure 2: Top-down inherent symmetry of control strategies such gestalts the of objects RowSp prolongation hypothesis searches in both directions where the yellow squares indicate. The generating vector g is measured with more precision if the row has many members so the search regions will become smaller and smaller. If the prolongation fails the top-down suppression of other hypotheses will be undone. So with high preference the maximal instance will trigger production 4 avoiding unnecessary combinatorial construction of possible sub-structures. Very important for a satisfying behavior is another local top-down mechanism: The construction of an instance always suppresses hypotheses for very close objects (closer than the adjacency parameter d, inside the black region) of the same type probably resulting from partially the same structure. Local inhibition should be combined by less local excitation [1]. So the brighter regions become more interesting, while the priority further away (in the grey surrounding) is not touched at all. The second parameter f specifying the excitation region is taken from higher knowledge - in our example from the proximity constraint of production 4. Inside the bright regions higher priority is given to hypotheses of productions 1, 2, and 3 respectively. Such mechanism makes the search much more focused and thus the performance more satisfying. Still this remains a declarative knowledge representation, where the procedural sequence is guided by the data not by a fixed program. After arbitrarily terminating the interpretation many variants of groupings are given. Among these a rank order is enforced recursively by picking the current best object and reevaluating all others suppressing those who are close to the one picked. The next one will then be further away. Fig. 3 displays the 20 best assessed results after 1800 cycles each processing 64 hypotheses (using amplitude data only). In yellow are the LLattice shown, which comprise interestingly the façade of the tower on the top left side. This is due to the close proximity of the underlying PS structure which is not resolved despite TSX high resolution and rather appears as line-like feature. In Fig. 4, we see a

4 subset of the previous result superimposed on the PS whose height is coded in color. The detected structure in the amplitude date fits nicely to the results from phase analysis. 6. Discussion Methodical: The use of a production system allows utilizing declarative knowledge. It is often stated that such knowledge should be learned from a set of representative training data instead of heuristically coded by an expert. We reply that learning inference rules requires very large, labeled, and representative data sets. In SAR-image analysis these are currently not available. Instead we exclusively use constraints known from Gestalt perception which are universally applicable. Given some representative labeled data one can learn the thresholds in the constraints using the theory of meaningful gestalts [3]. Parameters in the top-down control guiding the search can be inferred from the definition of the maximal meaningful element accordingly. Figure 3: Results selected objects Lattice (blue) and LLattice (yellow) Figure 4: Results: Some of the groupings selected and PS in phase-color Conclusive: The example shows that with a suitable search strategy some of the building façades will be instantiated using standard Gestalt grouping knowledge and combining it with knowledge about SAR imaging. Stacking images from successive orbits helps a lot in improving the signal to noise ratio thus allowing the recognition of more facades. We admit that human experts may recognize even more faint instances and thus give more complete results. Still, the fraction of buildings and building parts that do not appear at all on a single SAR perspective remains a problem. Satellite systems usually allow two perspectives on the scene (ascending and descending) which will reduce this problem, but only airborne SAR allows multi-perspectives such as from all four directions. References [1] M. Z. Aziz and B. Mertsching: An Attentional Approach for Perceptual Grouping of Spatially Distributed Patterns. In: F. Hamprecht et al. (eds.): Proc. 29th DAGM-Symposium, LNCS 4713, Springer, , [2] R. Bamler, M. Eineder, N. Adam, X. Zhu, S. Gernhardt: Interferometric Potential of High Resolution Spaceborne SAR. Photogrammetrie - Fernerkundung - Geoinformation, No. 5, pp , [3] A. Desolneux, L. Moisan, J.-M. Morel: From Gestalt Theory to Image Analysis. Springer, Berlin, [4] B. Draper, R. Collins, J. Brolio, A. Hanson, E. Riseman:. The Schema System. IJCV, (2): , [5] A. Ferretti, C. Prati, F. Rocca: Permanent scatterers in SAR interferometry. IEEE Trans. Geosci. Remote Sens. 39(1), 8-20, [6] C.-E. Guo, S. C. Zhu, Y. N. Wu: Modelling visual patterns by integrating descriptive and generative methods. International Journal on Computer Vision, 53 (1): 5-29, [7] E. Michaelsen, U. Stilla, U. Soergel, L. Doktorski : Extraction of Building Polygons from SAR Images: Grouping and Decision-Level in the GESTALT System. Pattern Recognition Letters, 31: , [8] T. Matsuyama, V.S.-S. Hwang : Sigma a Knowledgebased Image Understanding System. Plenum Press, New York [9] M. Nagao, T. Matsuyama: A Structural Analysis of complex Aerial Photographs. Plenum Press, New York, [10] U. Soergel, E. Michaelsen, A. Thiele, E. Cadario, U. Thoennessen: Stereo Analysis of High-Resolution SAR Images for Building Height Estimation in case of Orthogonal Aspect Directions: ISPRS Jour. of Photogr. & Remote Sensing, 64(5), , [11] U. Stilla, E. Michaelsen, U. Soergel, K. Schulz: Perceptual Grouping of Regular Structures for Automatic Detection of Man-Made Objects. IGARSS 2003, proceedings on CD, 2003.

5 Year: 2010 Author(s): Michaelsen, E.; Soergel, U.; Schunert, A.; Doktorski, L.; Jaeger, K. Title: Perceptual grouping for building recognition from satellite SAR image stacks DOI: /PRRS ( IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. Details: Institute of Electrical and Electronics Engineers -IEEE-; International Association for Pattern Recognition -IAPR-: 6th IAPR Workshop on Pattern Recognition in Remote Sensing, PRRS 2010 : , Istanbul, Turkey New York, NY: IEEE, 2010 ISBN: X ISBN: pp.