IMPROVEMENT OF BACKGROUND SUBTRACTION METHOD FOR REAL TIME MOVING OBJECT DETECTION INTRODUCTION

Transcription

1 IMPROVEMENT OF BACKGROUND SUBTRACTION METHOD FOR REAL TIME MOVING OBJECT DETECTION Sina Adham Khiabani and Yun Zhang University of New Brunswick, Department of Geodesy and Geomatics Fredericton, Canada ABSTRACT Accurate moving object extraction is a critical stage in a measurable 3D photogrammetric surveillance system. Several methods have been introduced in spatial and frequency domain by now. Background subtraction is a basic method in spatial domain which has great potentials to be implemented in the live videos due to its simplicity and high rate. However, simple subtraction will not lead to the expected results due to the different kind of noise (e.g. illumination changes noise) and contaminations. In this research, in order to improve the precision of this method in extracting the shape of moving objects, an interactive thresholding has been applied to the subtractions. Moreover, to cover the holes which have been produced due to the same tone of moving objects and background, two methods of colour processing and adaptive filtering is introduced. For this purpose and to reduce the noises and falsedetections, morphological operators have been utilized. The implemented strategies show improvements in the overall accuracy of object detection and separation. Keywords: Remote Sensing, Artificial intelligence, Moving object detection, Extraction, Background subtraction, Spatial domain, Morphological operators, Thresholding INTRODUCTION Automatic moving object extraction is an essential task in many surveillance projects. In many of these projects, it is necessary to detect moving objects on the scene in order to track or locate them through the interested scene. Hence, it is essential to accurately extract the boundaries of various moving objects in the scene. Extracted boundaries could be used in shape analysis stage or precise measurements. In a photogrammetric system, it is necessary to do the measurements accurately in various cameras. In our project, it has been focused on a live photogrammetric surveillance. In this project, it is tried to detect the moving objects and estimate their 3D location through a surveillance video system. Hence, in order to meet the requirements of the photogrammetric measurements, it is essential to extract the moving objects coordinates in every image to be able to estimate their 3D location in the object coordinate system. Image coordinates should be measured with highest possible accuracy in order to obtain more reliable estimation on 3D location of moving objects. For a live-stream continuous capturing, it is important to extract moving objects completely automatic and in the fastest possible period. In order to keep the frame rate of a live video, the whole processing period on each frame must be less or at most equal to the frame refreshing interval. As an instance, for a 25 frame per second live video, each frame must be processed in less than 40 milliseconds. As result, in order to sustain the time requirements of live video streams such as rate, the utilized algorithms should not be time consuming and of course it must be completely automatic. In order to meet the requirements of the photogrammetric system, first of all, it is necessary to detect and separate the moving objects through the scene of interest. Secondly, it is needed to measure the disparity of the respective images of each object. In this paper it has been tried to introduce a method to detect and separate the moving objects through a scene. Moreover, it has been attempted to extract the boundaries of moving objects accurately. This algorithm could be implemented on each image separately and the image measurements could be sent to the control module for further process. In this paper, a spatial domain method of object extraction based on the background subtraction method has been proposed. In the next section, the implementation of the background subtraction method is noted. In this section, some interactive thresholding strategies have been considered to reduce the illumination noise of the system. In the filtering section, implemented morphological filter to the resulting image of subtraction step has been noted. This filter has been applied to the image to fill the gaps of detected object and make the several parts of an object

2 consolidated. Furthermore, some utilized color processing and adaptive filtering has been considered to reduce the remained noise. BACKGROUND SUBTRACTION AND THRESHOLDING Back ground subtraction is the most common method of object detection. In this method, a reference image or a set of reference images are utilized as a base (Spagnolo et al, 2006). This reference image is accepted as a model of background and it is assumed that the reference image is a background, empty of any moving object. By subtracting the frames of the video from the reference image, the different areas could be extracted as the container of the moving objects. In practice, the simple subtraction will not lead to required results. Camera noise, minute illumination changes (especially at indoor places), color noise (for color cameras) and etc. contaminate the final results (Fig. 1). Figure 1. Simple background subtraction results of a scene without (Up) and with (down) moving object. As shown in figure 1, even a scene without moving objects has some differences with the reference image. In order to reduce such noise, it is common to use one (Elhabian, et al, 2008) or multiple (Wang and Yung, 2010) thresholds to separate the meaningful changes and the noise on the image. In this project, an interactive thresholding has chosen to purify the images from these kinds of noise. In this project, according to the mean sensed illumination of each frame, a single threshold is chosen for respective frame. In this stage, the values over the threshold are assigned value of 1 and the ones below the threshold are assigned 0 in the subtracted image.

3 Figure 2. Applied different thresholds for the background subtraction method. As seen in figure 2 and as it has been expected, by choosing the higher thresholds, probability of illustration of such noise will be reduced in the subtracted image. However, by increasing the value of the threshold, it will be more possible that a moving object or even some parts of a moving object become hidden in the final subtraction image, In other words, to reduce the noise completely, the higher thresholds are more helpful. While to demonstrate the complete moving object lower thresholds are demanded. In lower illumination condition, difference of received energy from the foreground and background to the sensor is normally lower than the parameter in higher illumination condition. Hence, as is expected, lower thresholds are led to better demonstration of the moving object. However, usually in lower illumination conditions, camera noise has seen more often. In this project, suitable threshold value has been extracted visually for two lowest and highest illumination cases. However, this threshold could be derived by attention to the mean of kappa coefficient of each candidate threshold value (or a range of candidate thresholds) on a few respective frames under a fixed illumination condition. The interactive thresholding method which has been used in this project is based on the highest and lowest defined thresholds. In our project, as soon as the system senses a change in mean illumination of the scene, the threshold value will be interpolated by reference to the highest and lowest derived thresholds. It is noticeable that the reference image should be updated after a severe change in the scene illumination. FILTERING Applying the suitable threshold to the subtraction stage, the noise of the images will be removed from the subtracted image. However, as could be seen in the figure 3, color or intensity noises (1), interlacing lines (2) and object holes (3) are three common remained errors in the subtracted image. Color noise and intensity noise are manufacturing noise which could be seen more in the multispectral images. Low-light conditions or changing in the temperature of the camera will increase these noises in the scene. Although the thresholding has a positive effect on removal of this noise, as its intensity is somehow random it would be shown in the subtraction results. Moreover, Interlacing lines are produced due to the capturing technique which has been used in many security cameras including the utilized test cameras. Furthermore, the similarity between a part of moving object and the background may cause holes inside of a moving object.

4 Figure 3. Moving Object extraction, Up: Original Panchromatic image (Left), Respective extraction image (Right), Down: Original MS image (Left), Respective extraction image (Right). Wren, et al, (1997) and Fang, et al, (2006) suggested different methods using color features to make the differencing process more accurate. Fang, et al, (2006) introduced a method using the color information differences to recognize the moving objects. They utilized the differences of the color properties of the moving objects and background in the YUV color system and extracted the moving objects from the scene. Although color properties make the subtraction more reliable, the subtraction is also affected by the color noise of the frames. In this project, in order to remove the noises, a combination of a closing and opening morphological operator has been applied to the subtraction image (Parker, 2000). The operators have been implemented by a simple 5x5 featuring element. By using the opening filter, the erosion part of the morphological operator removes the noises among the scene and the dilation phase of the operator attempts to keep the size of the extracted subjects. Moreover, a closing operator has been utilized to fill the internal holes of the moving object. CONCLUSION In this paper, an object detection method based on the background subtraction was introduced. As noted in the previous sections, illumination changes and cameras systematic effects are the main causes of error on the final subtraction product. In order to detect moving objects accurately, a combination of interactive thresholding and filtering have been utilized to improve the method of background subtraction method. Interactive thresholding shows its ability in reducing the camera noise and some illumination random noise. However, it makes the subtraction results more sensitive to the similarity of background and foreground. This may

5 cause some empty points inside of the extracted moving objects. In addition to the remained noise and the produced holes, interlacing lines of the camera could cause some errors around the extracted moving objects. In order to reduce the remaining effects, a combination of opening and closing operators has been used. Although the thresholding and filtering improve the results of subtraction stage, some remaining issues are still need improvements. The small difference between the intensity of the background and moving objects is still a challenging problem in the process. Moreover, false detection of the moving objects shadow causes some complications in the process. In addition to the noted problems, the sudden changes of the mean illumination of the scene, caused by passing of a moving object from the view of illumination resource is somehow problematic. REFERENCES Elhabian, S., K. El-Sayed, S. Ahmed, Moving Object Detection in Spatial Domain using Background Removal Techniques State-of-Art, Recent Patents on Computer Science, Volume 1, Number 1, pages 32-54, January Fang, X., W. Xiong, B. Hu, L. Wang, A Moving Object Detection Algorithm Based on Color Information, International Symposium on Instrumentation Science and Technology, vol. 48, pp , Parker, J.R. Algorithms for Image Processing and Computer Vision. (Wiley) 432 p Spagnolo, P., T.D. Orazio, M. Leo and A. Distante, Moving object segmentation by background subtraction and temporal analysis, Image Vision Computing 24. pp Wang L. And N.H.C. Yung, Extraction of Moving objects from their Background based on multiple adaptive threshold and boundary evaluation, IEEE Trans. Intelligent transportation systems vol. 11, pp , Wren, C. R., A. Azarbayehani, T. Darrell, and A. P. Pentland, Pfinder: Real-time tracking of the human body, IEEE Trans. Patt. Anal. Mach. Intell., vol. 19, no. 7, pp , Jul