Building Extraction from Digital Elevation Models. U.Weidner. Version: May Printed: March 27, Institut fur Photogrammetrie

Transcription

1 U.Weidner Version: May 1995 Printed: March 27, 1996 Institut fur Photogrammetrie Universitat Bonn Nuallee 15 D Bonn Abstract: This paper describes an approach towards building extraction from Digital Elevation Models (DEMs). This approach is based on automatically generated DEMs with a resolution up to 0:5m and uses prior knowledge for the detection of buildings and their reconstruction. 1

2 Contents 1 Introduction 6 2 General Strategy 8 3 DEM Generation 9 4 Prior Knowledge and Models Parametric Models of Buildings : : : : : : : : : : : : : : : : : : : : : : Prismatic Models of Buildings : : : : : : : : : : : : : : : : : : : : : : : 15 5 Building Detection 18 6 Building Extraction Using Parametric Models 27 7 Building Extraction Using Prismatic Models 39 8 Extensions of the Approach 53 References 55 March 27,

3 List of Figures 3-1 Skyline versus DEM showing deciencies in describing buildings : : : : Real DEM-Skyline { x-axis: pixels with 5m ground resolution, z-axis: height in [m] : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : DEM automatically generated with MATCH-T, grid size 0:5 0:5m 2, x; y in [pixels], z in [m] : : : : : : : : : : : : : : : : : : : : : : : : : : : DEM of downtown area : : : : : : : : : : : : : : : : : : : : : : : : : : Reference coordinate system : : : : : : : : : : : : : : : : : : : : : : : : Eects of occlusions : : : : : : : : : : : : : : : : : : : : : : : : : : : : : DEM derived using a laser scanner : : : : : : : : : : : : : : : : : : : : Section (Scales: x in [m], z in [0:1m]) : : : : : : : : : : : : : : : : : : : Building with a at roof : : : : : : : : : : : : : : : : : : : : : : : : : : Building with a symmetric, sloped roof : : : : : : : : : : : : : : : : : : Building with a non symmetric, sloped roof : : : : : : : : : : : : : : : : Building with a hip-roof : : : : : : : : : : : : : : : : : : : : : : : : : : Ground plan of prismatic buildings : : : : : : : : : : : : : : : : : : : : DEM with buildings in hilly region : : : : : : : : : : : : : : : : : : : : Results of simple thresholding of original DEM : : : : : : : : : : : : : : Original data set and section : : : : : : : : : : : : : : : : : : : : : : : : Dierence (data set - opening) and section : : : : : : : : : : : : : : : : Histogram of dierence (data set - opening) : : : : : : : : : : : : : : : Sections: Original DEM, opening, and dierence (Scales: x in [m], z in [0:1m]) : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Histogram of dierence (original DEM - opening) : : : : : : : : : : : : Building detection : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Reference point of parametric buildings : : : : : : : : : : : : : : : : : : DEM related coordinate system (internal representation) : : : : : : : : Reconstruction of parametric models : : : : : : : : : : : : : : : : : : : 30 March 27,

4 6-4 Hill shading of result : : : : : : : : : : : : : : : : : : : : : : : : : : : : Ray-shading of result : : : : : : : : : : : : : : : : : : : : : : : : : : : : Overlay: DEM and ground plan of buildings : : : : : : : : : : : : : : : Labelled ground plan : : : : : : : : : : : : : : : : : : : : : : : : : : : : Projected models : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Label 115 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Label 128 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Outlines in grid : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Vectorization of building outlines : : : : : : : : : : : : : : : : : : : : : Classication of nodes : : : : : : : : : : : : : : : : : : : : : : : : : : : Direction of edges : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Alternatives for code = 5 and code = 10 (8-connectedness) : : : : : : : Alternatives for code = 5 and code = 10 (4-connectedness) : : : : : : : Elimination of straight line points : : : : : : : : : : : : : : : : : : : : : Corresponding triangle heights : : : : : : : : : : : : : : : : : : : : : : : Polygon after discretization noise elimination : : : : : : : : : : : : : : : Reconstruction of ground plan : : : : : : : : : : : : : : : : : : : : : : : Initial Segmentation of data set : : : : : : : : : : : : : : : : : : : : : : Segments' outlines in grid : : : : : : : : : : : : : : : : : : : : : : : : : Vectorization of outlines : : : : : : : : : : : : : : : : : : : : : : : : : : Polygons after discretization noise elimination : : : : : : : : : : : : : : DEM and superposed polygons : : : : : : : : : : : : : : : : : : : : : : Ground plans of simple buildings : : : : : : : : : : : : : : : : : : : : : Variance of gradient orientation : : : : : : : : : : : : : : : : : : : : : : Ridge points : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 54 March 27,

5 List of Tables 5-1 Control parameters for building detection : : : : : : : : : : : : : : : : : Control parameters for building extraction using parametric models : : Output parameters for building extraction using parametric models : : Node information : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Control parameters for building extraction using prismatic models : : : Output parameters for building extraction using prismatic models : : : 46 March 27,

6 1 Introduction During the last recent years the need for 3D data describing urban areas increased. This data is needed for a variety of applications such as town planning, architecture, microclimate investigations or transmitter placement in telecommunication. All these dierent applications need 3D city models, but of course the requirements concerning the quality of description are as various as the applications. Architects use CAD for the planning of new buildings. In order to give a realistic visualization of the planned building, e. g. from a pedestrian's point of view, within the neighbourhood of the existing buildings, they need a detailed description of these buildings including e. g. passages in buildings. The data also should be in a CAD format, in order to use the same tools. CAD-like descriptions are quite dierent from a description needed for transmitter placement. For this purpose a graph surface description seems to be sucient. Whatever kind and quality of description may be required, 3D information in urban areas is literally unavailable. Often only 2D information is provided in city maps, and possibly some height relevant information such as the number of oors is also provided. Nevertheless, in many cases this data does not fulll the requirements of the applications listed above. Therefore, 3D information has to be gathered. Possible sources of information are surveying, terrestrial and aerial photogrammetry, and laser scanning. Surveying and classical photogrammetric techniques are expensive mainly due to the higher amount of information required, but also because acquisition techniques are not developed for this. Techniques using digital images show promising results in performance (cf. Lang and Schickler 1993), but a breakthrough in 3D information extraction requires a great deal of automization. The possible degree of automization depends on the type of the sensor data and the complexity of the scene Type of sensor data As photogrammetric techniques rely on imagery, mostly aerial, automation requires the derivation of 3D coordinates using matching techniques. In spite of intensive research during the last few decades in automatic stereo vision, no approach has been able to prove its superiority in reconstructing the 3D-geometry of urban areas. There are two main reasons why computational stereo is hard in urban areas: the always present occlusions and the need to access vertical surfaces. Recently, Gabet et al addressed the problem of occlusions by using a sequence of dierent baseline-height ratios, thus allowing a view into the streets and simpler matching and increasing the accuracy from a weak to a strong ratio. Nevertheless, occlusions will still occur and limit the access of vertical walls. Laser scanners (Krabill et al. 1984, Krabill 1989) immediately provide 3D coordinates and are much more reliable in deriving surface data than stereo techniques. They are very precise in height, but up to now have less resolution in planimetry than comparable March 27,

7 aerial imagery. Due to their small viewing angle, techniques with laser scanners are not yet economical. Moreover they leave the same problem unsolved which is essential for deriving city models: vertical walls are not accessible. Integrating both approaches seems to be quite promising. Range data, either provided by techniques based on digital aerial images (Krzystek 1991) or directly gained from laser scanners, can provide 3D information such as high resolution Digital Elevation Models (DEMs), e. g. graph surfaces represented in a grid with increments of 0:5m. These DEMs, which include information about the topographic surface and the buildings, can be used for the detection of buildings and the extraction of their shape and heights, although the resolution of the parameters will be limited by the resolution of the grid. The information may then be used during image analysis (cf. Haala 1994, Baltsavias et al. 1995). Complexity of the scene The complexity and the interpretation of the scene are always connected with the two following questions: Which knowledge is available about the scene? How can the knowledge about the scene be linked with the sensor data and used for the interpretation? The knowledge about the scene can be used to set up models of the objects in the scene. Typical models for our purpose are parametric models (McGlone and Shufelt 1994), which describe a building by a small set of parameters, and prismatic models, which describe a building by its polygonal ground plane and a height (Herman and Kanade 1986), or union of blocks, which are able to describe complex buildings having parts with dierent heights (cf. Lin et al. 1994). Probably the most advanced building model has been used by Fua and Hanson 1987, who also were able to derive relational descriptions of complex buildings. All approaches published so far proved the feasability of automatic techniques for interpreting digital imagery. This paper describes some attempts to derive the 3D-form of buildings solely based on digital surface data. The geometric description of buildings seems to be an excellent intermediate representation for linking the sensor data with high level knowledge about the objects. The advantage over techniques relying on the intensity data directly is the invariance of the geometric description with respect to surface markings (colour, texture), geometric micro structures (bricks, possibly windows) and illumination effects (especially shadows). Of course reducing the available information to the surface geometry may cause diculties in discerning buildings from non-buildings such as trees in case their geometric appearance mimics building type objects. These cases could be resolved using some intensity information, e. g. colour. Nevertheless, it seems worthwhile to investigate the potential of techniques based purely on geometric information. The used object related knowledge also makes adaption to data of dierent resolution simple and transparent. March 27,

8 2 General Strategy Our approach towards automatic building extraction is object related, i. e. we use a geometric model of the buildings and their geometric properties directly. The strategy of our approach consists of three steps 1. automatic generation of a high resolution DEM, which delivers a graph surface as the description of the objects, 2. detection of buildings in the DEM and 3. reconstruction of a parametric or prismatic geometric description for each detected building. Automatic generation of a high resolution DEM may use various sources of information. The only requirements are that the data are provided digitally { for ease of automization { and that the data are a sucient dense description of the objects, e. g. providing a raster with increments between 0:5 and 5m. We use data derived by a surface reconstruction technique based on digital (grey value) imagery, described in Section 3. The second step is the detection of buildings (Section 5) with the goal of focusing attention on areas where buildings can be expected in order to trigger the computationally more involved geometric reconstruction of the next step. We rst compute an approximation of the topographic surface using mathematical morphology. The dierence between the measured DEM and the topographic surface contains the information about the buildings. The buildings are detected by thresholding the dierence data set. The threshold is chosen according to prior knowledge about the buildings. The third step is the reconstruction of the buildings. For this purpose dierent groups of models (cf. Section 4) are used, dependent on the complexity of the detected buildings. The rst group of models consists of parametric models of the buildings. These models are used for simple buildings, which can be described using a few parameters. Complex buildings and blocks of buildings are described using prismatic models, which constitute the second group. The next two sections cope with automatic DEM generation and the prior knopwledge and models for our approach. March 27,

9 3 DEM Generation This section about DEM generation deals with DEMs as graph surfaces, thus surfaces represented by z = z(x; y). Graph surfaces have conceptual deciencies for our purpose of building extraction, e. g. vertical walls or passages in buildings cannot be represented. Nevertheless, we use such a 2.5D description, because operational techniques yielding graph surfaces are available. Surface reconstruction algorithms may use several kinds of observations in order to compute the coordinates (x; y; z(x; y)) T. The main techniques for point determination are based on the principle of radiation, e. g. using range nders, or the principle of intersection, e. g. using images. Both techniques show some deciencies in the context of building extraction. The rst question that arises is: which surface is measured and can be reconstructed? In both cases, not only the topographic surface and the buildings are measured and represented in the DEM, but also objects which should not be included from the building extractor's point of view. These objects may be, depending on the applied technique, trees and bushes, but also cars. They are usually regarded as outliers in the data set, but they often cannot be distinguished from other small objects which may be of interest. Another problem concerns the visibility of the surface to be reconstructed. In order to gain as much information as possible about this surface, the data sets should contain information about vertical walls of the buildings, while at the same time including information about the surface between the buildings. These requirements are quite dierent compared to classical topographic applications. Therefore, the measurement design has to cope with trade-os concerning e. g. ying height, inuencing the precision and/or the resulution viewing angle, inuencing the eciency in terms of ying costs per area visibility of objects, inuencing the accessibility of the desired information. Laser scanning techniques (Krabill et al. 1984, Krabill 1989, Lindenberger 1993) have the advantage of directly gaining geometric information (point coordinates) about the surface and destinguishing between the tree canopy and the topographic surface beneath. Heights can be measured with an accuracy < 0:5m. In contrast to the requirements posed above, laser scanners do not measure points on vertical surface parts. Image based techniques normally use stereo pairs of aerial images. In the centre of a model, it allows a look into streets and yards, while vertical walls of the buildings cannot been seen. On the other hand, walls can be seen at the border of the stereo model, but of course problems due to occlusions of objects arise (cf. Remark 3.2). Recently some papers, e. g. Collins et al and Dang et al. 1994, have been published dealing with DEM generation including knowledge about buildings which can help to improve the results of DEM generation for the purpose of building extraction. March 27,

10 z x,y Fig. 3-1: Skyline versus DEM showing deciencies in describing buildings Fig. 3-2: Real DEM-Skyline { x-axis: pixels with 5m ground resolution, z-axis: height in [m] The high resolution DEMs we used are generated using the software package MATCH- T. The principle strategy of this automatic approach is described in Krzystek The basic idea is to use image and feature pyramids. Starting with the lowest resolution { top of the pyramids { and a plane as approximation of the DEM, homologous features are matched and their 3D coordinates and a rened DEM are computed. This step is carried out for each pyramid level using the DEM of the previous step as approximation until the highest resolution { bottom of the pyramids { is reached. Tests of the approach show that the accuracy of heights in open areas is comparable to the accuracy of the measurements by a human operator (Krzystek and Wild 1992). The algorithm was designed for topographic applications, treating objects such as buildings and trees as outliers. The reconstruction of surfaces needs regularization (Terzopoulos 1986). The main effects of standard regularization techniques are illustrated in Figure 3-1, where the solid line represents the buildings and the dashed line their surface description in the DEM; edges of buildings are rounded o. In order to avoid this eect, local adaptive regularization techniques have to be applied (Terzopoulos 1986, Weidner 1994). Even if March 27,

11 Fig. 3-3: DEM automatically generated with MATCH-T, grid size 0:5 0:5m 2, x; y in [pixels], z in [m] adaptive techniques are used, very small details may not be preserved in the generated DEM (see roof of the right building). Figure 3-2 shows a real DEM-skyline of a downtown area. For the special purpose of DEM generation for building extraction, the control parameters for MATCH-T were adapted. Buildings are no longer seen as outliers, but as the information of interest. As mentioned above, problems arise concerning the discrimination of buildings and other objects, e. g. group of trees, which still should be regarded as outliers. An example of an automatically derived DEM with a resolution of 0:5m is given in Figure 3-3. In order to give an impression of the DEM data in cities and with a lower resolution (5m) Figure shows a DEM of a downtown area. The black line marks the longitudinal section displayed in Figure 3-2. Remark 3.1 (Input Data) The input data for our approach is a graph surface. In order to compute datum parameters of the models, the coordinates of the origin of the grid has to be known. MATCH-T yields a graph surface given in a grid as shown in Figure 3-5, which also represents the reference coordinate system. All relevant information (x 0, y 0, x, y) are known. The increments x and y determine the possible resolution of the buildings' parameters length and width and inuence the results of the segmentation. 1 DEM supplied by INPHO GmbH, Stuttgart March 27,

12 Fig. 3-4: DEM of downtown area Remark 3.2 Figure 3-6 shows the entire DEM of the downtown area within the stereo model. The visualization indicates the eects due to occlusions. At the border of the model, the algorithm can only extract points on the roofs of the buildings and not in the streets due to occlusions. Therefore, buildings melt together, which makes segmentation a hard task or even impossible. Figure 3-7 shows a DEM 2 of a downtown area derived based on laser scanner data. This DEM has a ground resolution of 2 2m 2 and a height resolution of 0:1m. Figure 3-8 displays a section wich is marked white in Figure 3-7. Laser scanning allows a view into the streets within the entire data set. Nevertheless, it does not gain information about vertical walls. 2 DEM supplied by Dornier, Friedrichshafen March 27,

13 y dy dx x Fig. 3-5: Reference coordinate system Fig. 3-6: Eects of occlusions March 27,

14 Fig. 3-7: DEM derived using a laser scanner Fig. 3-8: Section (Scales: x in [m], z in [0:1m]) March 27,

15 4 Prior Knowledge and Models Object detection and reconstruction heavily depends on prior knowledge about the scene. The prior knowledge may be either generic or specic. In our approach the steps increasingly use generic and specic object related knowledge. Generic knowledge is used for the detection of buildings. The knowledge applied for this purpose is: buildings have vertical walls of a minimal height buildings have a minimal size building parts have a maximal size Based on this knowledge the control paramters for building detection can be xed. Specic knowledge helps to set up simple models of buildings with a few parameters, typically parametric models as given in section 4.1. Of course, such simple models only cover a limited number of buildings types and complex buildings can be hardly described using parametric models. The group of complex buildings can be described with more generic models such as prismatic models (section 4.2), using further generic knowledge about buildings. 4.1 Parametric Models of Buildings Parametric models are used for simple buildings, which can be described using a few parameters. For these parametric models we assume that the buildings are separate from each other and that the ground plan of the building is a rectangle. Examples for this group of models are given in Figure 4-1 showing a at building with 3 parameters and Figure 4-2 showing a building with a symmetric, sloped roof, parameterized using 4 parameters. Up to now, only these two models are integrated in our implementation. Other examples for parametric models are buildings with a non symmetric, sloped roof (Figure 4-3) and buildings with a hip-roof (Figure 4-4). The examples show the increase of the number of parameters needed. Although it is possible to introduce more and more parametric models with more complex parametrizations, their application will be restricted. Nevertheless, if such a model applies, the result of reconstruction is likely to be better than reconstructions only based on generic knowledge. 4.2 Prismatic Models of Buildings The rst fact we use is that the ground plans of buildings or building blocks are sets of closed polygons, as shown in Figure 4-5 (right) with polygons representing court yards. March 27,

16 z y height length width x,y width x Fig. 4-1: Building with a at roof z y height2 height1 length width x,y width x Fig. 4-2: Building with a symmetric, sloped roof z width2 y width2 height2 length height1 width1 x,y width1 x Fig. 4-3: Building with a non symmetric, sloped roof z height2 y lenght2 lenght1 height1 lenght2 width x,y width x Fig. 4-4: Building with a hip-roof March 27,

17 y x Fig. 4-5: Ground plan of prismatic buildings Neighbouring straight lines of the buildings' outlines and therefore neighbouring edges of the polygons are likely to be orthogonal (Figure 4-5, left). Non neighbouring straight lines may be parallel, which yields a restriction for the direction, or collinear, which yields restrictions for the direction and the position. Furthermore, these restrictions can be applied for polygon edges not only belonging to one polygon, but to the set of polygons. The reconstruction of prismatic models described in Section 7 only uses the fact, that the polygons are closed. The application of constraints such as orthogonality and parallelity is described in Weidner and Forstner March 27,

18 5 Building Detection Our approach towards building detection is outlined in the ow chart in Figure 5-8 and the several steps are given in Algorithm 5.1. Algorithm 5.1 (Building detection) 1. Minimum ltering of data 2. Maximum ltering of the minimum ltered data 3. Computation of the dierence between the data and the ltered data 4. Computation of the initial segmentation by thresholding 5. Computation of connected components and labelling 6. Computation of the labels' sizes and bounding boxes 7. Selection of valid labels 8. Computation of the maximum and minimum height of each label 9. Computation of the threshold for the rened segmentation 10. Computation of the rened segmentation by thresholding within bounding box The approach is based on the simple fact that buildings are higher than the (surrounding) topographic surface. Simple thresholding of the original DEM would only solve the problem of segmentation in at region but not in hilly regions. Figure 5-1 shows a DEM in such a hilly region. Simple thresholding would lead to results as given in Figure 5-2, and thus would not solve our problem. The information about the buildings is contained in the dierence between original DEM and an approximation of the topographic surface without buildings. Therefore, we rst compute an approximation of the topographic surface represented in the original DEM. For this purpose we use mathematical grey scale morphology (cf. Haralick et al. 1987). The rst step is minimum ltering, i. e. z = inffz(x; y)jx; y 2 Wg (5.1) March 27,

19 Fig. 5-1: DEM with buildings in hilly region Fig. 5-2: Results of simple thresholding of original DEM March 27,

20 112 flc05.dhm.row Fig. 5-3: Original data set and section with the structural element W, in this case a square window. In terms of mathematical morphology, minimum ltering is a special erosion, where the structural element w(x; y) has constant z values. z = z w (5.2) This step is followed by maximum ltering, i. e. z = supfz(x; y)jx; y 2 Wg (5.3) with the structural element W, which is a special dilation. Both steps perform an opening z = z w (5.4) z = z w (5.5) on the set of heights and deliver an approximation of the topographic surface. In order to eliminate all information of the buildings in the resulting image of the opening, the window size has to be chosen in such a way, that the structural element W is not entirely contained in a building's outlines. The window size can be xed using prior knowledge about the expected maximum size of the buildings or building parts. If the approximation for the topographic surface converges towards the original DEM in non building regions and interpolates within the building regions, the dierence data set d = z?zw, i. e. heights of original DEM minus topographic surface approximation, consists of the buildings approximately put on a plane. This holds if the topographic surface is smoother than the surface with buildings. Figure 5-3 shows the orignal data set and a section. The dierence data set d is displayed in Figure 5-4. Figure 5-5 displays the histogram of the dierence data set, indicating the eects mentioned above. March 27,

21 12 flc05.dhm-open.differ.row Fig. 5-4: Dierence (data set - opening) and section flc05.dhm-open.histogr Fig. 5-5: Histogram of dierence (data set - opening) Figure 5-6 displays the results for the DEM with buildings in a hilly region. The section is marked black in Figure 5-1. The next step towards building detection consists of thresholding the dierence data set. This is due to the fact that the buildings are put on a reference surface, which approximately is a plane (cf. Figure 5-5 and Figure 5-7). The threshold for segmentation can be easily motivated and derived from prior generic knowledge about the buildings. It can be chosen to be the expected height of vertical walls, e. g. the height of a oor. In order to identify the dierent segments during later processing, connected components are computed and each segment is labelled. Furthermore, the bounding box { with an additional margin { of each segment is computed. March 27,

22 ascona.dhm.col.145 ascona.dhm.opening.col ascona.dhm-open.col Fig. 5-6: Sections: Original DEM, opening, and dierence (Scales: x in [m], z in [0:1m]) ascona.dhm-open.histogr Fig. 5-7: Histogram of dierence (original DEM - opening) March 27,

23 MIN - MAX - Data -? Dierence? Initial Segmentation Rened Segmentation Fig. 5-8: Building detection March 27,

24 Figure 5-8 shows the result of the initial segmentation, including all segments found. It is obvious that some of the segments do not represent buildings, e. g. small segments representing trees. Based on the initial segmentation, we therefore select valid segments using as criteria the size of the segment; and, the position of the bounding box. The minimal size of valid segments is related to the expected minimal size of buildings. We furthermore reject segments, if their bounding box is at the margin of the data set, because it is then likely that the building is not contained in the data set entirely and therefore neither the parameters for parametric models nor the prismatic models would be extracted correctly. Figure 5-8 also indicates that pure global thresholding is not optimal. Therefore, a rened segmentation for each valid segment is computed, which takes the data within the segment's bounding box into account and locally adapts the threshold based on the height information within the labelled area and the bounding box without segments. Remark 5.1 (Step 1) Minimum ltering is an erosion z = z w (cf. Figure 5-8 top row, middle). Remark 5.2 (Step 2) Maximum ltering is a dilation. z = z w Remark 5.3 (Steps 1 + 2) Steps 1 and 2 form the opening of the set z = z w (cf. Figure 5-8 top row, right). Remark 5.4 (Steps 1 + 2) Minimum and Maximum ltering is sensitive to noise and outliers in the data. In order to deal with both, a dual rank lter as described in Eckstein and Munkelt 1995 can be used. March 27,

25 Remark 5.5 (Steps 1, 2) Both lters need the window size mm as control parameter. m is computed based on the control parameter area for minimum and maximum lter: s area for min=max filter m 0 = with x; y : grid increments xy If m 0 is even, then m = m 0 + 1, else m = m 0. Remark 5.6 (Steps 1, 2) For the examples given in the gures we used a square window. Such a structural element has the disadvantage that the result is rotation variant. The use of a circle as structural element delivers a rotation invariant result. The inuence of the use of the dierent structural elements has to be investigated. Remark 5.7 (Step 2) If area for minimum and maximum lter is chosen large with respect to the size of the buildings, the result of step 2 is an approximation of the DEM without buildings. On the other hand, a small window size is preferable in hilly regions, because the approximation of the topographic surface directly inuences the result of step 4, i. e. thresholding for the initial segmentation. The area for minimum and maximum ltering should be chosen to be area for filter > kexpected minimal size of buildings with k = 5 : : : 10. Remark 5.8 (Step 3) If the approximation obtained in step 2 converges towards the measured DEM, step 3 delivers the data of the buildings (approximately) put on a plane. This holds if the topographic surface is smoother than the surface with buildings. Remark 5.9 (Steps 1, 2, 3) Negative values in the dierence data set occur mainly at the margin of the data set, because the masks for minimum and maximum ltering are of a xed size m and are moved until the masks touch the border of the original data set. In order not to reduce the size of the data sets, the computed height values of the rows ( m + 1) and (#rows? m? 1) and the columns ( m + 1) and (#cols? m? 1) are transferred to the area of the margin. Remark 5.10 (Step 4) The threshold for the initial segmentation is given by the user. The height for the initial segmentation is related to the expected minimal height of the vertical walls of a building. Remark 5.11 (Step 6) In order to reduce the inuence of errors during the rst segmentation the bounding box is enlargened. This enlargement is predened by the user with the control parameter additional margin for bounding box. March 27,

26 Control parameter Remarks area for minumum and maximum ltering square, steps 1 and 2 cf. Remark 5.5, Remark 5.7 height for initial segmentation threshold for step 4, related to minimal height of walls cf. Remark 5.10 additional margin for bounding box cf. Remark 5.11 expected minimal size of buildings threshold for step 7 cf. Remark 5.12 Tab. 5-1: Control parameters for building detection Remark 5.12 (Step 7) The rst criterion for the selection of valid labels is the size of the label by thresholding using the control parameter expected minimal size of buildings, in order to exclude spurious segments due to bushes and trees. This criterion is applied for both groups of models. The second criterion, which is only applied for parametric models, is the position of the bounding box. If the bounding box coordinates are at the margins of the data set, the labels are rejected, because in this case only parts of the buildings may be contained in the data set and the parameters would not be correct. In case of prismatic building models, at least the building parts within the dataset can be extracted. Remark 5.13 (Step 9) The threshold for the rened segmentation of each segment is computed using the dierence between the maximum height of the initial segment and the minimum height of the related bounding box. The control parameters, which are needed for building detection, are listed in Table 5-1. March 27,

27 6 Building Extraction Using Parametric Models In this section the extraction of parametric models from DEMs is described. The approach is outlined in the ow chart in Figure 6-3 and the several steps are given in Algorithm 6.1. Algorithm 6.1 (Building reconstruction: Parametric models) 1. Computation of the maximum and minimum height of each rened segment and the mean height of the background within the bounding box 2. Computation of the point of gravity { position x; y { and the direction of the main axis { orientation { of each rened segment 3. Computation of the shape parameters for each segment 4. Computation of the reference point coordinates and the orientation of each segment in a reference coordinate system 5. Selection of the building model for each segment 6. Computation of the height parameters for each segment The form parameters for parametric models are the length, width and height for at buildings or the length, width, height of eave{base and height of ridge{eave for buildings with a symmetric, sloped roof (cf. Figures 4-1 and 4-2). We only integrated these two models in our implementation up to now. Besides these model related parameters, datum parameters, namely the three dimensional coordinates of a reference point and the orientation of the buildings in a reference coordinate system (cf. Figure 3-5), are needed. In order to determine the x; y coordinates of the reference point and the orientation of the buildings (cf. Figure 6-1), the point of gravity and the orientation of each rened segment using the heights within the segments as weights are computed. Length and width of the ground plan are the length and width of a rectangle approximating the segment and are computed as follows: length = length of the segment along the rst main axis width = width of the segment along the second main axis March 27,

28 y x Fig. 6-1: Reference point of parametric buildings col row Fig. 6-2: DEM related coordinate system (internal representation) The coordinates of the reference point are computed using the coordinates of the point of gravity, the orientation and the length. These coordinates and the orientation given in a DEM related coordinate system (cf. Figure 6-2) have to be transformed into the reference coordinate system. The z coordinate is computed taking the mean of heights within the background area of the bounding box. The other parameters, i. e. the heights, are model dependent. Therefore, we rst have to select the model which has to be applied. This selection uses the height information within a segment and prior knowledge about the minimal expected slope of roofs. If the slope within the segment is greater than the given minimal slope, the model with the symmetric sloped roof is chosen, otherwise the model of the at building. For buildings with at roofs (see Figure 4-1) the height parameter of the models is computed as March 27,

29 height = dierence between the mean height of the segment and the mean height within the background area of the bounding box For buildings with symmetric, sloped roofs (see Figure 4-2) the height parameters are computed as height1 = dierence between the minimum height of the segment and the mean height within the background area of the bounding box height2 = dierence between the maximum and minimum height of the segment In order to improve the robustness of the computed height parameters mean values for the k maximum or minimum heights could be used. Remark 6.1 (Step 2) In order to compute the point of gravity (position x; y) and the orientation of each rened segment, the heights within the segments are used as weights, i. e. r g = X S rz(r; c) = X S z(r; c) and c g = X S cz(r; c) = X S z(r; c) where S denotes the segment. Remark 6.2 (Steps 2, 3) The shape parameters are computed as length = length of the segment along the rst main axis width = width of the segment along the second main axis Instead of using these simple techniques, the point of gravity, the lenght and the width can be computed via more sophisticated, model-based techniques (cf. Luo and Mulder 1993). Remark 6.3 (Step 5) For the selection of the models, i. e. building with a at or a symmetric, sloped roof, the slope of the roof, i.e. dierence between maximum and minimum of each rened segment devided by one half of the width, can be used as criterion. This is quite closely related to using the magnitude of the mean gradient. The threshold for the selection is the minimal slope. Remark 6.4 (Step 6) The algorithm distinguishes for the height parameters between two types of buildings: Building with a at roof March 27,

30 ? Rened Segmentation - Estimation of Parameters? Generation of Buildings? Data Extracted Building Fig. 6-3: Reconstruction of parametric models March 27,

31 height = dierence between the mean height of the segment and the mean height of the background within the bounding box Building with a symmetric, sloped roof height1 = dierence between the minimum height of the segment and the mean height of the background within the bounding box height2 = dierence between the maximum and minimum height of the segment In order to improve the robustness of the computed parameters mean values for the k maximum or minimum heights could be used instead. Remark 6.5 In order to improve the extraction of height information, heights might also be derived from the opening of the DEM, e. g. taking the height information for the bounding box without labels from the opening as described above, or intersecting the vertical walls with the surface given by the opening, or a surface approximation for the bounding box without labels based on the heights of the original DEM by estimating the parameters of a tilted plane z(x; y) = ax + by + z 0 and computing the heights of the buttoms of the buildings as intersection of the vertical walls with this plane. The estimation of the plane's parameters should be done using robust estimation techniques in order to reduce the inuence of the round os at the buildings' borders. Remark 6.6 The direction of the ridge for buildings with a symmetric, sloped roof is assumed to be the direction of the rst main axis. Instead of this assumption, the ridge line should be detected using gradients or curvatures and this information should be used. The control parameters, which are needed for building extraction using parametric models, are listed in Table 6-1. The output parameters are listed in Table 6-2. Figure 6-4 and Figure 6-5 show two visualizations of the results. Figure 6-6 displays an overlay of the buildings' ground plans superimposed on the original DEM. Visual inspection of the result indicates good correspondance between ground plan information and the input data. March 27,

32 Fig. 6-4: Hill shading of result Fig. 6-5: Ray-shading of result March 27,

33 Control parameter minimal slope Remarks minimal expected slope for a sloped roof cf. Remark 6.3 Tab. 6-1: Control parameters for building extraction using parametric models Output parameter Remarks type of building 100 at roof 200 symmetric, sloped roof datum parameters coordinates x; y; z orientation parameters cf. Remark 6.4 additional: size, volume Tab. 6-2: Output parameters for building extraction using parametric models Fig. 6-6: Overlay: DEM and ground plan of buildings March 27,

34 Fig. 6-7: Labelled ground plan Figure 6-7 displays labelled ground plan information for the identication of buildings in Fig. 6-8, Fig. 6-9, and Fig Figure 6-8 shows the extracted building models (white lines) projected into the left image of the stereo pair. A qualitative evaluation indicates that the orientations of the extracted models t to the image information. Problems occur for the paramters lenght and width, although the overlay in Fig. 6-6 indicates a plausible t to the DEM data. Therefore, the reasons for the discrepancies occur during the DEM generation. An explanation of the eects may be that during DEM generation, interest points are found at the borders of the roofs. Due to low texture (cf. label 113) or shadows (cf. label 134), no interset points are found close to the building. Therefore, the regularization term within the reconstruction algorithm leads to interpolation between points at the roofs' borders and points on the ground, which are more or less far away from the building, and thus elongating the buildings sytematically. Furthermore, the round os at breaklines contribute to such eects, although we try to take these eects into consideration during the rened segmentation, and therefore the round os only have minor inuence, if a sucient number of points is extracted and used. Remark 6.7 In order to overcome the rst problem, the use of n images might be helpful. If n images are taken into account, an object is contained in these images from dierent views and information about also the vertical walls, which are not visible in two images, will be extracted, i. e. interest points are also found on these walls. Thus, n images are likely to deliver interest points and therefore a denser information about surface parts, which are not visible in two images. March 27,

35 Figure 6-9 allows a closer look on the result of DEM-analysis and the use of the extracted models (label 115) for the semiautomatic tool described in Lang and Schickler The upper image displays the projected model. An operator has only to check the selection of the right parametric model, and also can move the model (middle), before he starts the automatic measurement procedure. The result of automatic measurement is displayed in the lower image, also showing the extracted image edges (black lines). March 27,

36 Label 113 Label 116 Label 119 Label 121 Label 122 Label 124 Label 125 Label 126 Label 127 Label 130 Label 131 Label 133 Label 134 Label 135 Label 136 Fig. 6-8: Projected models March 27,

37 Model extracted by DEM-analysis Model modied by user interaction Model adjusted by matching Fig. 6-9: Label 115 March 27,

38 Model extracted by DEM-analysis Model adjusted by matching Fig. 6-10: Label 128 March 27,

39 7 Building Extraction Using Prismatic Models The extraction of prismatic models from DEMs consits of two parts: extraction of ground plan extraction of height information performed by the algorithm outlined in Figure 7-10 and Algorithm 7.1. Algorithm 7.1 (Building reconstruction: Prismatic models) 1. Vectorization of outlines for each segment 2. Elimination of polygon points on a straight line 3. Elimination of discretization noise of polygons 4. Extraction of height information The input data for the extraction of the prismatic models' polygons is the rened segmentation. The boundaries of each segment are simplied and approximated using knowledge about the expected structure of the buildings. This is performed by the following steps: 1. raster to vector conversion of the segments' outlines 2. elimination of discretization noise Raster to Vector Conversion In order to derive a polygon description of the segments, two approaches can be followed. The rst is directly based on the border pixels of the segments: Algorithm 7.2 (Raster to vector conversion based on border pixels) 1. Elimination of interior pixels of the segments 2. Computation of the number of polygons for each segment 3. Determination of clockwise ordered list of the outline points March 27,

40 Fig. 7-1: Outlines in grid Fig. 7-2: Vectorization of building outlines March 27,

41 First the interior pixels of the segments are eliminated, which delivers their outlines in the grid (Figure 7-1). This step is followed by the computation of the number of outline polygons for each segment using connected components. The next step determines clockwise ordered lists of the outline points with their x; y coordinates for each segment and outline polygon. These lists contain all outline points as shown in Figure 7-2. Remark 7.1 (Step 1) Figure 7-1 shows the result of interior pixel elimination. Computing the skeleton of this outline delivers a pixel chain in which the pixels are connected in a neighbourhood of the eight surrounding pixels, but also removes points marking corners of the segment. Therefore, the outline as shown in Figure 7-1 is used for further processing. Remark 7.2 (Step 2) The number of polygons for each segment is computed using connected components. Remark 7.3 (Step 3) For the determination of the lists of polygon points the lower right pixel of a polygon is taken as start point. The algorithm determines a clockwise ordered list of the outline points based on the neighbourhood of the eight surrounding pixels. Therefore, not all of the outline points in the grid are contained in the lists. This simple technique is sucient for simply shaped segments. Problems arise for complex shaped segments due to the fact, that lines can not be represented properly in a raster, leading to inconsitencies in topology. Examples are pixel chains connecting two parts of a segment and pixel chains of the segment between the background and interior holes of the segment. A more detailed discussion follows in Remark In order to overcome these diculties an other approach is pursued. This approach is based on the classication of congurations within a 2 2 neighbourhood (Figure 7-3). Each conguration denes the properties of the outline's polygon nodes. These properties are e. g. the number of polygon edges and, assuming oriented polygons, the direction of the edges (Figure 7-4). The main features of the classication scheme and the available node information, i. e. the direction to the next node, is gathered in Table 7-1. The algorithm for the vectorization is then given by Algorithm 7.3. March 27,

42 none cases cases none background object 1 2 background object Fig. 7-3: Classication of nodes Fig. 7-4: Direction of edges 5 case 1 case 2 5 case 1 case 2 10 case 1 case 2 10 case 1 case 2 Fig. 7-5: Alternatives for code = 5 and code = 10 (8-connectedness) Fig. 7-6: Alternatives for code = 5 and code = 10 (4-connectedness) Algorithm 7.3 (Raster to vector conversion based on node classication) 1. Classication of nodes code(r ; c + 1 ) = segment(r; c) + 2segment(r; c + 1) 2 +4segment(r + 1; c + 1) + 8segment(r + 1; c) 2. Determination of a clockwise ordered list of the outline points based on polygon edge direction Remark 7.4 (Step 1) segment(r; c) = 1 for segment pixels and 0 for pixels of the background. Remark 7.5 (Step 2) The codes for each segment are stored in a matrix. The codes allow to determine starting points for each polygon and to distinguish between outer and inner polygons of the segment. In our implementation running from top left to March 27,

43 code dr dc new code Tab. 7-1: Node information bottom right, the starting point for the outer polygon is determined by searching for a pixel with code(r; c) = 4. The starting point for an inner polygon has code(r; c) = 11. Code 0 or 15 indicate, that all pixels in the 22 neighbourhood belong to the background or to the object respectively. If a node is visited and evaluated, the code is changed as given in Table 7-1, right column. Remark 7.6 (Step 2) Table 7-1 displays the direction information for outer polygons, assuming 8-connectedness for the objects. Conguration 5 and 10 oer two alternatives (cf. Figure 7-5), which have to be decided upon dependent on the direction of the last evaluated node. If 4-connectedness for the objects has to be applied, the direction information for nodes with code = 5 and code = 10 has to be changed: code dr dc new code Again, two alternatives have to be taken into account (cf. Figure 7-6). March 27,

44 Fig. 7-7: Elimination of straight line points Remark 7.7 (Step 2) In order to extract polygons which point lists are clockwise ordered, the direction information given in Table 7-1 for outer polygons has to be modied for inner polygons by inverting the directions of the arrows displayed in Figure 7-4. Both approaches deliver a vectorization from pixel to pixel. In order to reduce the number of points in this vectorization, points on straight lines between two neighbouring points are eliminated (Figure 7-7). Discretization Noise Elimination After the elimination of straight line points, a merging algorithm is applied in order to eliminate discretization noise. This merging algorithm successively eliminates points where the corresponding triangle height d i in the triangle formed by the points i? 1, i and i + 1 (cf. Figure 7-8) is the minimum of the polygon until the minimum triangle height in an iteration is greater than a prexed threshold or the prexed minimum number of points is reached. For this merging the minimum height of a triangle has to be given as a threshold. This minimum height is closely related to the resolution x; y of the input data. If the minimum height d min is chosen q d min < k x 2 + y 2 only points lying approximately on a straight line are eliminated. We used k = 2 3. An example of the result is shown in Figure 7-9. Remark 7.8 (Step 2 of Algorithm 7.1) The elimination of points on a straight line between two points is performed as the merging in step 3 of Algorithm 7.1. The March 27,

45 P i+1 P i d d i+1 i P i-1 Fig. 7-8: Corresponding triangle heights Fig. 7-9: Polygon after discretization noise elimination threshold is (numerical) zero for the corresponding triangle heights. But here it is not an iterative scheme, but all points with corresponding triangle height smaller than the threshold are eliminated at once. Remark 7.9 (Step 3 of Algorithm 7.1) Instead of the scheme for discretization noise elimination used here other approaches can be applied (Douglas and Peucker 1973, Pan and Forstner 1994). March 27,

46 Control parameter threshold for discretization noise Remarks triangle height cf. Remark 7.9 Tab. 7-2: Control parameters for building extraction using prismatic models Output parameter Remarks polygons list of points x; y height of polygon z height height of prisms Tab. 7-3: Output parameters for building extraction using prismatic models Height Determination Finally, the heights for the prismatic models are computed analogous to the heights of at buildings in the previous section (cf. also Remark 6.5). If parts of a block of buildings have dierent heights, further segmentation within the polygons is of course necessary. The control parameters, which are needed for building extraction using prismatic models, are listed in Table 7-2. The output parameters are listed in Table 7-3. Figure 7-10 shows the result for a complex building in a downtown area. Taking into account the DEM resolution of 5 5m 2 the result is quite remarkable as the basic structure of the building with two court yards is extracted. However, the upper of the two exclaves is a group of trees mimicing to be part of the building. Here a simple analysis of the colour of the corresponding image area would solve this problem. This fact indicates the limitations of purely geometry based approaches. Remark 7.10 (Raster to Vector Conversion) A closer look onto Figure 7-10 indicates some problems of the raster to vector conversion using Algorithm 7.2. Figure 7-10, middle of top row shows the outlines in the grid. It is obvious that the complex building has three holes, whereas the vectorization in Figure 7-10, right of top row only shows three polygons, two of them representing court yards. The third hole is missed. For this fact there are two reasons: The rst reason is the resolution of the grid compared to the size of building parts. In this example the upper exclave is connected to the main part of the building only by one pixel wide structures. Therefore, the outline of the third hole is not seperated from the outer outline of the building and connected components only deliver three polygons. Increasing the resolutionis likely to reduce the probability of such cases, where the polygons are no longer simple closed polygons. The second reason is the fact that up to know the algorithm does not recognize, if the March 27,

47 Rened Segmentation Outlines ? Vectorization Vectorization after Elimination of Straight Line Points ? Vectorization after Elimination of Discretization Noise Fig. 7-10: Reconstruction of ground plan March 27,

48 Fig. 7-11: Initial Segmentation of data set polygon branches out, and does apply a post-processing in such cases. Therefore, the algorithm has to be extended in order to cope with these cases. Nevertheless, the results indicate some conceptual deciencies of representing the outlines in a grid. The use of crack edges helps to overcome these deciencies. Therefore, Algorithm 7.3, which is based on the idea of crack edges is more appropriate and is used. Figure 7-11 shows the initial segmentation of the data set given in Figure 3-7. Segmentation seems to be easier as the laser scanner supplies a dense point set for the description of the surface, also including points in the streets. Therefore, it is likely that edges (vertical walls) are sharper than edges in data sets derived by images based techniques. Figure 7-12, Figure 7-13 and Figure 7-14 show the outlines, the rst vectorization and the polygons after the elimination of discretization noise. The results for this data set are obtained with the same parameters as the results for the DEM shown in Figure 3-4. The only parameters which have to be changed are the data set specic parameters, e. g. the size of pixels. Figure 7-15 displays the original DEM and the extracted polygons. The small white points represent the points of the vectorization after the elimination of straight line points, whereas the large white points represent the vectorization after discretization noise elimination. Figure 7-16 shows the polygons extracted for simple buildings. A qualitative comparison of the results using parametric models shown in Figure 6-4 and Figure 6-5 and March 27,

49 Fig. 7-12: Segments' outlines in grid the result of the prismatic models, indicates that in this case parametric models are preferable to prismatic models, as they incorporate more specic prior knowledge than prismatic models. Shape extraction using polygons as described above only makes use of the knowledge that the polygons are closed. Prior knowledge such as orthogonality constraints can be used for further simplication of the polygons (cf. Weidner and Forstner 1995). March 27,

50 Fig. 7-13: Vectorization of outlines Fig. 7-14: Polygons after discretization noise elimination March 27,

51 Fig. 7-15: DEM and superposed polygons March 27,

52 Fig. 7-16: Ground plans of simple buildings March 27,

53 8 Extensions of the Approach Possible extensions of the approach described here can be devided into two groups dealing with 1. the exploitation of more prior knowledge 2. the exploitation of more information of the data Extensions of the rst group are given in Weidner and Forstner 1995 exploiting more generic knowledge about the buildings, e. g. using constraints such as orthogonality. Furthermore other constraints, e. g. symmetries, and semantic knowledge about rows of buildings, should be investigated. The second group includes further work for parametric models, which will focus on the integration of other parametric models such as buildings with non symmetric sloped roofs or buildings with hip roofs. The discrimination between the models needs exploitation of the structure of the roofs, either based on gradient or curvature information, e. g. the variance of the gradient orientation (Figure 8-1), or by nding local maxima (Figure 8-2). Furthermore only simple techniques are used for the computation of the models' parameters up to now. In order to improve the accuracy of parameters, template matching for the estimation of the point of gravity and the orientation should be investigated and robust statistics used for the estimation of the height parameters. Nevertheless, the resolution of the parameters will always depend on the resolution of the DEM grid. In order to deal with complex buildings consisting of parts with dierent heights more appropriately, discrimination of dierent parts using the height information within the region circumscribed by the extracted polygons with the aim of deriving a building graph is necessary (cf. Fua and Hanson 1987). For this purpose the height information can be exploited using gradient based techniques or an approach analogous to Kweon and Kanade 1994, an approach for the extraction of topographic features based on the simulation of ooding und tracking of segments. Gradient or curvature based segmentation techniques should also be investigated for the initial segmentation. In order to deal with complex roof structures, techniques for range images (Arman and Aggarwal 1993) should also be investigated for their applicability in this context. These techniques address either the segmentation or the classication of surfaces, or both, e. g. Homan and Jain 1987 deals with segmentation and classication, whereas Newman et al concentrate on the model-based surface classication. In the context of building extraction, integrated approaches for segmentation and classication seem to be of more interest, as roofs mainly consist of tilted planes, thus reducing the problem almost to mere segmentation (Li 1990). Up to now, the user has to dene which models should be applied for the entire data set, either parametric or prismatic models. In areas where simple and complex models are included, this is not acceptable, because of two facts. March 27,

54 Fig. 8-1: Variance of gradient orientation Fig. 8-2: Ridge points 1. Complex buildings cannot be described by parametric models. Therefore, prismatic models have to be applied to the entire data set. 2. Buildings, which t to the parametric models, should be reconstructed using these models, because they include more specic knowledge and therefore yield better results compared to prismatic models. Therefore, further work will also include work about automatic selection of the group of models, which should be applied for each detected building individually. March 27,