A Survey of Methods to Extract Buildings from High-Resolution Satellite Images Ryan Friese CS510

Transcription

1 A Survey of Methods to Extract Buildings from High-Resolution Satellite Images Ryan Friese CS510 There are many things in this world that are simply amazing. Some of these things are crafted by nature, such as the mighty Grand Canyon, the luscious rain forests of the tropics, and even a tiny flower on a high mountain top trying to send its roots down through cracks in the rock. Some things are also manmade, the Great Pyramids of Egypt, the Taj Mahal, and the Sistine Chapel all come to mind. While all these things are amazing in their own right, there might be something going on behind the scenes that is even more amazing. I am of course talking about the power of human vision, the ability we have that lets us easily comprehend patterns, colors, and shapes. It is not until I started looking into the field of computer vision that I realized what makes humans be able to comprehend images and scenes with little thought, is not as trivial or as simple as it may seem. Computer Vision is a broad area of study with research happening in many areas such as: face detection, video tracking, object recognition, scene restoration, and countless others. Each of these areas can be further refined into more specific sub-areas. For this paper, I will be talking about a specific sub-area on object recognition. First though, let s take a look at exactly what object recognition is. In its most basic form, object recognition is exactly what it sounds like, the ability to recognize objects. We as humans can perform this very fast and efficiently,

2 even when the objects are seen from different viewpoints, different sizes, different levels of lighting, and even when the objects are partially obscured from view. Despite this being so easy for humans, object recognition is still a challenge for computer vision systems. Sub areas within object recognition can range from, being able to read text off a printed circuit board, to finding a human face in a picture. There are many applications of object recognition. For this paper I am going to address the problem of extracting buildings from high resolution satellite images. Being able to extract buildings from high resolution satellite images is an open research topic in the field of Photogrammetry. There are many applications which require accurate information about the data contained in satellite images. This information is needed by many agencies such as land management agencies, urban development planners, the U.S. Military, and many others. There are several problems in being able to extract buildings properly, these include different viewing angles, surrounding vegetation, shadows, and other objects which obscure the edges of the buildings which are being detected. To solve this problem of building detection, many strategies have been used. In this paper I will give a brief survey of several different techniques, I will compare their similarities and differences, and discuss the results from these techniques. I will be examining 5 different solutions, they are; Texture classification and analysis used by (1) and (2), neural networks used by (3), active contour models used by (4), local Gabor features used by (5), and model based detection used by (6). The goal of all these solutions is to be able to extract vector data representing buildings. It could be then used in various applications which make use of the vector data. These solutions aim to automatically generate this vector data with as little human editing as possible. Let us

3 first take a look at what these techniques mean, and then we will begin comparing and analyzing the approaches. We will start with texture classification and analysis as according to (1) and (2). The idea behind a textural analysis approach is to use a specific texture pattern or patterns which will then be compared to the satellite image. The algorithm looks for areas in the satellite which are similar to what is found in the texture. This allows the image to be characterized into specific areas such as buildings, vegetation, and water. To be able to separate these areas apart from each other, there must be a way of comparing similarity between the texture and the image. There are various ways to measure similarity as is seen in the approaches of (1) and (2). While both approaches are able to identify buildings by using textures, the way to perform this measurement is different. In the approach used by (1), each specific area a texture represents is considered a class. For example, different classes could be building textures, vegetation textures, water textures, and so on. These classes are modeled as probability distributions across a texton library. These textons are basically the averaged combination of all the image patches in a given class which are supplied from a training set. Training sets are also used in approaches (3) and (4). The textons are then compared to 5x5 image patches in the satellite image. This allows each region in the image to be classified by using a nearest-neighbor classifier using the X 2 statistic. A region is considered similar only if its distance to any class is less than a learned threshold, otherwise that region is rejected. This application of training and meeting a threshold is used in (3) and (5) as well. The approach in (2) makes use of GLCM(Grey Level Co-occurrence Matrix) and GLDV(Grey Level Difference Vector) to perform the similarity measure. A GLCM texture considers the relation between two neighboring pixels the second order texture. The co-occurrence matrix is formed from the

4 target image by running a kernel mask over the image. The mask can be 3x3, 5x5 7x7 The neighboring pixels can be in one or more of eight defined directions. Typical directions are 0, ±45, ±90, ±135 and 180. The use of multi-directed masks is also seen in (5). Using directed mask means the GLCM texture is dependent upon both directionality and kernel size. The values in the GLCM texture can be interpreted using some known measures. These measures include angular second moments which measure the extent of pixel orderliness, contrast which measures how many elements don t lie along the main diagonal of the GLCM, dissimilarity which measures how different the elements are from each other, entropy which measures randomness, energy which measures the extent of pixel pair repetitions, and homogeneity which is the measure of uniformity of the co-occurrence matrix. Homogeneity is also an important measurement used in (4). The GLDV measure is the sum of the diagonals from the GLCM. When the GLCM and GLDV textures are transformed back into image space, the authors of (2) state that both textures show and find the detection of shadow zones, classification of building types, and the recognition of pavement. Comparing both (1) and (2), the use of textures are integral to their algorithms and the detection of buildings in satellite imagery. They are vastly different though, (1) creates an averaged texture from a training image, and then compares it to the test image. It will select areas which are under a given threshold for similarity, but it does not outline the actual boundaries of the image, but rather gives a general area of where the building is. (2) ends up using a kernel mask to turn the entire image into a texture which ends up highlighting and bringing out various zones in the image, this does not in and of itself outline the building boundaries, but rather makes the edges more apparent.

5 Another viable approach to building extraction is the use of artificial neural networks as proposed by (3). Similar to (1) and (4) this approach must first be trained on a set of training images before being able to perform the extraction. This approach will initially perform segmentation on the image by using a seeded region growing algorithm. What this algorithm does is evenly distribute seed points across the entire image; it then compares the seed point s value to neighboring pixels and adds them to the region if the neighboring pixels are below a set threshold. The region growing algorithm continues recursively with the newly added pixels until there are no more neighboring pixels which fall below the set threshold. This algorithm will find the homogenous roof regions in the image. After the region growing algorithm, this approach uses many features to classify the regions of the image. The features include: area, perimeter, mean color and intensity, roundness, compactness and structural features. The values of these features help determine if a region is a building or not. For example, if the area of a region is more than pixels then that region is assumed to not be a building. The roundness feature can help determine whether a region is a building by calculating the ratio of the regions area to the square of its perimeter, and discarding any regions which have a low ratio, or a high roundness. A high roundness means the object could be something like a tree. Since individual buildings normally maintain the same color and intensity throughout their rooftops, the mean color and intensity features are used to measure variations in color allowing detection of continuous building areas. These features for a given region are then set as the input to the neural network. If the output from the neural network is greater than a specified threshold, the region will be recorded as a building. Thresholds are also used in approach (1) and (5). This approach is capable of finding and outlining the boundaries of the building.

6 We will now examine the approach presented in (4). This approach uses an active contour model to perform the building extraction. A contour model, also known as a snake, is a model that is used to delineate object outlines in images. It accomplishes this by minimizing the energy associated to the current contour. This approach uses a homogeneity and similarity measure which is similar to (2). This approach segments the image into regions so that each region will have pixels inside it which have maximal homogeneity and similarity. This model has the ability to extract objects without obvious edges and it is not sensitive to noise. The model finds the building boundaries by finding whether a given contour curve is inside or outside the building region. The model will iterate until the curve finds the minimal energy of the region, which indicates it has hit the boundary. This is similar to the way approach (6) operates. To perform the actual building extraction, this approach runs through 4 steps. The first step is to run the image through an image smoothing Gaussian filter, approach (5) must also apply a preprocessing filter to the image. The next step is taking the smoothed image and as with (1)and (3) introduce training data into the system. This training data contains points which are inside the boundaries of the buildings. The third step implements the active contour model and the buildings boundaries are extracted. The fourth and final step is an accuracy assessment. This approach along with (3) is capable of finding and outlining actual building boundaries. Moving onto our next approach for building extraction, we are presented with (5). This approach uses Gabor filters and spatial voting to find and extract the centers of buildings in a satellite image. As we will see, I believe this approach could be used in conjunction with (4), but more on that later. This approach makes use of Gabor filter sets to detect building properties in the image. A two dimensional Gabor set is mathematically the product of a two dimensional

7 Gaussian and a complex exponential function, which otherwise can be thought of as a Gaussian filter modulated by a sinusoidal wave. Similar to (4) there is some preprocessing that must be done to the image in the form of a 5x5 median filter to try and eliminate some of the noise which might be present in the image. This is done to help prevent false detections. There is no guarantee that all the buildings in a given image will all be oriented the same way, to account for this problem, approach (5) generates a set of six Gabor filters to account for multiple directions. This implies that the same image must be convolved with a Gabor filter six times, one for each filter in the set. This will result in 6 separate Gabor spaces. Each Gabor space will be examined for local maximums which indicate a building property. To prevent false detections a threshold is set to eliminate weak maximums. These collected building properties are then used to generate descriptor vectors. For each feature three descriptors are extracted; the feature location, the possible distance to the center of the building, and the features dominate orientation. These descriptors are then used to describe the direction towards the buildings center. The authors of (5) note that this approach only works when buildings are brighter than the background they are on, this is because when the building is brighter, the second highest edge will be located close to the building center, allowing the direction to the building center to be found. If the building isn t brighter, then the second highest edge does not have to be close to the center, thus, no information about the direction towards the center of the building can be found. Once the direction towards the buildings center has been calculated, all the features then vote for possible building center locations to create a voting matrix. Once the voting matrix has been calculated, buildings locations can be determined based upon where local maximums in the matrix occur. Again to avoid false positives a threshold is enforced to

8 eliminate weak maximums. This approach differs from (3)and (4) by not being able to detect the outlines of buildings but is instead being able to detect an approximate center location of a building. The final approach we will be examining is presented in (6). This approach is a modelbased building detection algorithm. It is model-based in the aspect that it uses a prior-based variational framework to extract multiple buildings from a single image. What this means is this approach tries to align evolving contours with a prior building shape. The use of evolving contours is also used in (4). There are multiple prior building shapes, these are used to represent differently shaped buildings. These prior building shapes can be scaled so that they can fit differently sized buildings. The way this approach works, is it first applies an arbitrary curve to the entire image. It then tries to minimize the energy, similar to (4), in the bounded region by matching one of the prior building shapes to the region. This results in one or more regions bounded by a prior shape. The algorithm then continues this same process of minimizing energy in a bounded region and matching prior building shapes to contours. This continues until the algorithm converges and all the energy in the image is minimized. The final building shapes are then assigned to the best fitting prior building shape. This approach like (3) and (4), is able to extract building boundaries from the image, but since the resulting boundaries are a best fitted prior building shape, there might be slight variations from the actual boundary of the building. After examining all 6 approaches, we can notice a few common trends and similarities across the techniques. One of the most common aspects in the approaches is the requirement

9 of training data needing to be introduced into the system before building extraction can be performed. Training data is needed in approaches (1), (3), (4), and (6). The extent of the training data changes from approach to approach. For example both (1) and (3) require training images which represent the buildings the will be trying to extract. (4) needs information on where seeding points are located in the image. (6) just needs general building templates which will be used to approximate building shapes in the image. The use of training data allows these approaches to better identify the objects they are searching for, at the cost of some initial set and manual data creation. Another problem these algorithms tried to solve was the detection of false positives, or regions being highlighted as buildings when they actually weren t. Approaches (1), (3), and (5) all tackle this problem by introducing thresholds into their systems. These thresholds force regions in the image to meet a certain condition; otherwise they will be thrown out and not considered a building. After further analyzing the approaches, I found an interesting relationship, approaches (3), (4), and (6) all have an iteration mechanism as part of their systems. These three approaches are also the only three approaches which actually find and highlight the boundaries of the buildings. In the cases of (4) and (6), an energy minimization algorithm is implemented to find the building edges, while in (3) a region growing algorithm is implemented. Now that we have begun discussing the actual results of the approaches, let us examine what the other 3 approaches provide. Approach (1) fails to provide a concrete outlining of the buildings; instead it supplies regions which have matched specific textures. Since 5x5 pixel patches are used in the comparison, the regions are blocky. Approach (2) states they can produce outlines of buildings, but they fail to fully explain how exactly they do this. I assume

10 they take the filtered images which highlight and define separate areas and perform additional analysis to extract the building regions and boundaries. Approach (5) is the last approach which fails to concretely provide building boundaries. It instead provides information about where the expected center of a building is located. If the exact shape of the building is not important, this information can be used to create generic shapes which represent buildings in the image. There might also be a way this data about building centers could be used with another approach. As I stated earlier approach (4) requires a set a training data. The type of information contained in this training data is information about seed points which are inside building regions in the image. This seems to me to be exactly what the output of approach (5) is providing. This means a new system could be created which combines approach (5) with approach (4) thus eliminating the need to manually create the training data, allowing the system to become more automatic. The task of extracting building boundaries from high-resolution satellite images is an easy yet tedious and time consuming task for humans to perform. This extracted data is also an essential part of many applications used in agencies around the world. If the extraction of building boundaries could become an automated process it would speed up the collection of this data and remove the need for a human to manually find and create the boundaries. Throughout this paper I have reviewed and analyzed several different approaches attempting to automatically extract building boundaries from high resolution images. I believe there are some very promising results from individual approaches such as those illustrated in (3), (4), and (6). I also believe that a combination of several approaches could lead to a more fully automated system, all but eliminating the need for any human interaction. In closing, after

11 performing this research I have a greater understanding on the subject object detection and in a grander scope, computer vision as a whole.

12 Bibliography 1. Computer aided generation of stylized maps. Adabala, Neeharika, Varma, Manik and Kentaro, Toyama. s.l. : Microsoft Research India, Urban Feature Characterization using High Resoultion Satellite imagery: Texture Analysis Approach. Jeon, So Hee, Kwon, Byung-Doo and Lee, Kiwon Automated Building Extraction from High-Resoultion Satellite Imagery Using Spectral and Structural Information Based of Artificial Neural Networks. Lari, Zahra and Ebadi, Hamid Automatic Building Extraction from High Resoultion Aerial Images Using Active Contour Model. Ahmady, Salman, et al., et al Building Detection Using Local Gabor Features in Very High Resotution Satellite Images. Sirmacek, Beril and Unsalan, Cem Automatic Model-Based Building Detection from single Panchromatic High Resolution Images. Karantzalos, Konstantions and Nikos, Paragios