T O B C A T C A S E E U R O S E N S E D E T E C T I N G O B J E C T S I N A E R I A L I M A G E R Y

Size: px

Start display at page:

Download "T O B C A T C A S E E U R O S E N S E D E T E C T I N G O B J E C T S I N A E R I A L I M A G E R Y"

Caitlin Brooks
5 years ago
Views:

1 T O B C A T C A S E E U R O S E N S E D E T E C T I N G O B J E C T S I N A E R I A L I M A G E R Y Goal is to detect objects in aerial imagery. Each aerial image contains multiple useful sources of information. Several object classes that could be interesting to detect are: Cars and transportation vehicles Traffic markings on the roads like cross-overs, direction arrows, round points, Calculating the amount of solar panels in a city area Calculating the amount of swimming pools in a city area We will discuss some of the approaches that were tested in this case. DETECTING OF CARS FROM AERIAL IMAGERY A first tryout was building a car detector to detect parked and driving cars to have an idea on how much traffic is driving through a Belgian city center. We started from aerial images of 5000x5000 pixels. Since they were too big to efficiently handle them, we have reduced the size towards 500x500 pixels. We made sure there was an overlap of about 100 pixels to ensure we do not loose objects that are exactly on the border. This resulted in a set of 150 car images that were manually annotated step by step. A sample of these manual annotations can be seen on the following page. In a first step all these annotations were cut out to have a set of positive training data: Background images were created by cutting out all positive regions. This ensures that the detector is applicable in the same situation, making it robust to the background variation specific to a city center. An example can also be seen on the following page.

2 A first remark: we noticed that building classifiers based on actual negative data yielded much better result than the approach followed by general pedestrian detectors. There just a large set of random images is used as background in order to model everything except pedestrians.

Since cars in aerial imagery appear in all possible orientations, again the dominant gradient approach with cutoff region fitting was applied here.

3 Since cars in aerial imagery appear in all possible orientations, again the dominant gradient approach with cutoff region fitting was applied here. The following parameters were used to reach satisfying results. No smoothing was applied, probably due to the size of the input samples. Details don t contain enough pixel information to disturb the dominant gradients. This was done by setting the smoothing filter to [1,1] and a sigma =1. An angle step of 5 degrees was applied to yield best results. It seemed that here and there some objects still got translated towards the horizontal axis and not the vertical axis. This is mainly due to the sharp gradients of the windows of the cars, which are stronger than the actual car boundaries, for example when motion blur is polluting the boundaries. An extra manual rotation tool was created which can solve this problem. Run through all samples using hotkeys. A key can be pressed to rotate 90 or 180 degrees. Instead of simply removing them, we corrected the samples this way, resulting in valuable information not being lost. After this setup we concluded that making a car for only upright cars was somewhat overkill. Therefore we kept the up facing and down facing cars together and made a detector independent of that orientation. It introduced extra variation to the class model which made it more useful in several cases. However all further processing is still in the idea of rectifying all cars to a similar orientation.

Looking at these images, we still noticed a lot of overburden, useless information that will influence the model way too much. Therefore a little manual step was introduced 1.

Define the average height and width around these images : w = 28px / h = 56px 4.

4 Looking at these images, we still noticed a lot of overburden, useless information that will influence the model way too much. Therefore a little manual step was introduced 1. Select 5 10 vehicles samples 2. Cut them down to contain only the car using a simple image processing software 3. Define the average height and width around these images : w = 28px / h = 56px 4. Use this to define a ratio parameter being H/W = 2 for this application Then feed this set together with the ratio parameter to the remove_overburden.exe application. This application takes the height of the image, and reduces the width according to the ratio parameter. This results in clean cars for training. The average measurements will be used for model training! Examples can be seen below.

TRAINING AND TESTING THE CAR DETECTOR TRAINING LBP CLASSIFIER USING 214 POSITIVES AND 1000 NEGATIVES The first detector training was stopped at 20 stages.

5 TRAINING AND TESTING THE CAR DETECTOR TRAINING LBP CLASSIFIER USING 214 POSITIVES AND 1000 NEGATIVES The first detector training was stopped at 20 stages. However it seemed the detector was far not powerful enough to detect cars but removing all false positives in the same time. We can clearly see that a lot of false detections are made. However, in order to reduce the false detections, constraints can be set. Let s also reduce the detections by maximum and minimum detection size = original scaling size * The size is known since these images are taken on a constant height. This yielded the following result.

6 This removes already a lot of false positives, however, by simple looking at min and max scale for cars in this context. We notice that large detections just disappear, since they are not natural for the object. We increase the amount of overlapping detections and in meanwhile adapt the image. We shift it minimal in order to get vertical roads. This is necessary because we only had a single orientation trained in our model.

CONCLUSIONS 1. Add more positive training variables more variation than expected 2.

7 Using 3 overlaps as criteria this result was achieved for a part of the image. Results are better, but basically, this shows that there is a need for lots more of positive and negative examples, since there is a lot of variation in object and background information. CONCLUSIONS 1. Add more positive training variables more variation than expected 2. Add more negative training variables it is clear that the background is still not modelled enough, resulting in clear background like roads, being sampled as actual cars TRAINING Training of stages further than level 20 takes a long time. However, it is still progressing so we won t limit it and await the result. Added a test that switches the image horizontally and then applies the detection model again. This should result into more detections, and show if model is influenced by direction of the actual car.

MORE TESTS ON SINGLE WINDOWS We limited the testing data to debug further any useful conclusions.

So in a very constrained environment the model actually does exactly as expected.

objects like round points, traffic arrows, crossroads, however due to the limited time models were not

8 MORE TESTS ON SINGLE WINDOWS We limited the testing data to debug further any useful conclusions. Below you see the original versus the detection image. So in a very constrained environment the model actually does exactly as expected. However keep in mind that there is NO background information or clutter here. So this should be used with caution. APPLYING THE SAME TECHNIQUE TO OTHER OBJECTS It was suggested to use the same approach on other objects like round points, traffic arrows, crossroads, however due to the limited time models were not trained. Some example applications in aerial imagery that still should be tested: Another interesting approach is taking a look at railroads since they have a very similar structure. This is especially interesting in combination with the dominant gradient general approach.

9 SOFTWARE THAT IS USED FOR PROCESSING THIS DATA USB Stick > Software > cpp_windows > cut_out_annotations.cpp USB Stick > Software > cpp_windows > dominant_orientation.cpp USB Stick > Software > cpp_windows > rotate_batch_segments.cpp USB Stick > Software > cpp_windows > remove_overburden.cpp USB Stick > Software > cpp_windows > flip_images.cpp

DIGITAL SURFACE MODELS OF CITY AREAS BY VERY HIGH RESOLUTION SPACE IMAGERY

DIGITAL SURFACE MODELS OF CITY AREAS BY VERY HIGH RESOLUTION SPACE IMAGERY Jacobsen, K. University of Hannover, Institute of Photogrammetry and Geoinformation, Nienburger Str.1, D30167 Hannover phone +49