Efficient Shadow Removal Technique for Tracking Human Objects

Efficient Shadow Removal Technique for Tracking Human Objects Aniket K Shahade Department of Information Technology Shri Sant Gajanan Maharaj College of Engineering, Shegaon, Maharashtra, India aniket.shahade11@gmail.com Gajendra Y Patil Department of Information Technology Shri Sant Gajanan Maharaj College of Engineering, Shegaon, Maharashtra, India gajendrapatilgp@yahoo.com Abstract This paper deals with the shadow removal algorithm for tracking human object with the background subtraction and occlusion detection technique. This is implemented by initially considering a reference frame and using its background information. When a new object enters into the frame, the foreground image and background image are derived using the reference frame which was taken earlier as Most of the times, the shadow from background information mixes with the foreground object hence results in intricate tracking process. The algorithm used involves modeling of the desired background as a reference model which is later used in background subtraction to produce foreground pixels which are the deviation of the current frame from the reference one. Here, morphological operations will be used for identifying and removing the shadow. The occlusion is one of the most common events in object tracking and centroid of each object are used for detecting the occlusion and identifying each object separately. Video sequences are captured and detected with the proposed algorithm. Keywords Object tracking; background subtraction; shadow removal. I. INTRODUCTION Object tracking, by definition, is to track an object (or multiple objects) over a sequence of images. Object tracking, in general, is a challenging problem. Difficulties in tracking objects can arise due to abrupt object motion, changing appearance patterns of the object and the scene, non-rigid object structures, object-to-object and object-to-scene occlusions, and camera motion. Tracking is usually performed in the context of higher-level applications that require the location and/or shape of the object in every frame. In Computer Vision, object tracking is considered as one of the most important tasks. Various methods have been proposed and reported both in academia and industry at large numbers for real-time applications. II. LITERATURE REVIEW A new method for real-time tracking of non-rigid objects seen from a moving camera was proposed by Comaniciu, D et al [7]. The central computational module was based on the mean shift iterations and found the most probable target position in the current frame. The dissimilarity between 978-1-4799-7169-5/14/$31.00 2014 IEEE the target model (its color distribution) and the target candidate were expressed by a metric derived from the Bhattacharyya coefficient. The theoretical analysis of the approach shows that it relates to the Bayesian framework while providing a practical, fast and efficient solution. The capability of the tracker to handle in real-time partial occlusions, significant clutter, and target scale variations is demonstrated for several image sequences. In template-based approach category, mean-shift method [7] and Kernel-based tracking method [8] have been proposed, where the color histograms of the target object were constructed using a Kernel density estimation function. Since, the color histogram is invariant feature for rotation, scaling and translation, it is considered as one of the suitable feature for handling the problem of change in the scale, rotation and translation of target object. The object tracking is carried out by comparing the color histogram of the template and the target object. However, mean-shift method is not suitable for 3-D target object and monochromatic object. In case of monochromatic target object, even small variation in illumination, produces narrow histogram pattern and tracking often fails. In object tracking problem, the object representation is the difficult aspect. Various ways of representing or describing target objects have been proposed such as object appearance, image features [9, 10], target contour [11,12] and color histogram [8]. In both appearance-based and color histogram based approaches, the region of the object has to be defined for describing the target. Thus, if some of the background pixels are mixed with the defined region, the tracking may fail. Tao Zhao et. al. [2] track the multiple human in crowded environment. They presented a method that can track humans in crowded environments, with significant and persistent occlusion by making use of human shape models in addition to camera models, the assumption that humans walk on plane and acquired appearance models. In this paper, A Bayesian framework of the multi-object tracking problem, includes a color-based joint likelihood which enables simultaneously detection and tracking. An efficient MCMC-based approach to compute the optimal solution: The design of reversible dynamics to explore the solution space and the use of informed

proposal probabilities from image features for faster convergence. The extension of the mean-shift tracking to incorporate background information in the context of a stationary camera. Several authors have proposed methods to identify shadows in outdoor environments. Cucchiara et al [13] classify pixels into foreground/background using the HSV (Hue, Saturation and Value) color space, since in this space chromaticity and luminosity components can be easily decoupled. This decoupling exploits the assumption that an area cast into shadow often results in a significant decrease in intensity whilst maintaining a similar chromaticity [14]. Thus, their classification criteria for shadows are: 1) that the hue and saturation components of the surface s color should not change significantly and 2) that the value component should decrease. One unsolved problem is the specification of the procedure for selecting the appropriate classification thresholds. McKenna et al [14] used similar assumptions, and define a background model with Gaussian distributions for each of the pixel channel chromaticity values. That enabled confidence measures to be generated after the background subtraction process, based on the probabilities of a particular pixel value belonging to each distribution in the model. If a pixel value is classified as foreground using the intensity distribution, and background uses the chromaticity model, then, overall, it is classified as shadow. In addition, a third classification method was designed to distinguish between shadows and darker objects that are of a similar color to the background, using gradient and texture information as the discriminate. One difficulty was that the edge of a shadow will manifest a gradient, just as the edge of a dark object did. To overcome this, both background models were recursively updated: however, they were susceptible to sudden environmental changes; and, in the case of the third classification method, computationally expensive. Horprasert et al [15] developed a novel algorithm for color images whose contained shading and shadow and detect moving objects from a static background scene. This algorithm was a robust and efficiently computed background subtraction that was able to cope with local illumination change problems, such as shadows and highlights, as well as global illumination changed. They developed a color model, which separated brightness from chromaticity component. During a period of scene inactivity, a statistically generated 4- tuple background model was learned. The model components comprise: a pixel RGB mean and variance; and chromaticity and brightness distortion components. Color Model Color model separated the brightness from the chromaticity component. Fig. 1. illustrates the proposed color model in three-dimensional RGB space. Consider a pixel, i, in the image; let E i = [E R (i); E G (i); E B (i) ] represent the pixel s expected RGB color in the reference or The line OE i passing through the origin and the point Ei is called expected chromaticity line. Next, let I i = [I R (i); I G (i); I B (i)] denote the pixel s RGB color value in a current image that we want to subtract from the background. Basically, we want to measure the distortion of I i from E i. We do this by decomposing the distortion measurement into two components Fig.1. Fig.1. Color Model Illustrate the proposed color model in the three dimensional RGB color space; the background image is statistically pixel-wise modeled. E i represents an expected color of a given i th pixel and I i represents the color value of the pixel in a current image. The difference between I i and E i is decomposed into brightness (α i ) and chromaticity (CD i ) components. This method classifies a given pixel into four categories. A pixel in the current image is Original background (B) if it has both brightness and chromaticity similar to those of the same pixel in the Shaded background or shadow (S) if it has similar chromaticity but lower brightness than those of the same pixel in the This is based on the notion of the shadow as a semi-transparent region in the image, which retains a representation of the underlying surface pattern, texture or color value [16]. Highlighted background (H), if it has similar chromaticity but higher brightness than the Moving foreground object (F) if the pixel has chromaticity different from the expected values in the Then, a further period of statistical learning is required, to estimate appropriate thresholds for the foreground, background and shadow classes. That entails the construction of normalized histograms for chromaticity and brightness, then choosing the threshold to obtain an assumed detection rate. A significant advantage of this technique is the automatic determination of threshold values. The

disadvantages are that the background model is not adaptive, the detection rate needs to be known, and it is computationally expensive. Second problem is that it may suffer from dynamic scene change such as extraneous event in which there are new objects deposited into scene and become the part of background scene. The main objectives of Saravanakumar et al [17] was to developed multiple human object tracking approach based on motion estimation and detection, background subtraction, shadow removal and occlusion detection. In the approaches morphological operations were used for indentifying and removing the shadow. Human motion can be detected at a certain distance in tracking application. Normalized Cross Covariance (NCC) can be used to detected shadow. The occlusion has also been dealt effectively. Applications like Visual surveillances, content based video retrieval, and precise analysis of athletic performance fixed cameras are used with respect to static background (e.g. stationary surveillance camera) and a common approach of background subtraction is used to obtain an initial estimate of moving objects. First perform background modeling to yield reference model. This reference model is used in background subtraction in which each video sequence is compared against the reference model to determine possible variation. The variation between current video frames to that of the reference frame in terms of pixels signifies existence of moving objects. The variation which also represents the foreground pixels are further processed for object localization and tracking. Ideally, background subtraction should detect real moving objects with high accuracy and limiting false negatives (not detected) as much as possible. At the same time, it should extract pixels of moving objects with maximum possible pixels, avoiding shadows, static objects and noise. The mode model was chosen to perform the background modeling, which provided better results. If the absolute difference between the current pixel and the mode modeled background pixel is larger than a threshold, then that pixel is considered as foreground object. RGB values of current frames pixels subtracted with that of background modeling frame. The mean of absolute difference of red value, green value and blue value are found. If the absolute difference greater than threshold, indicates the foreground pixels else background pixels. Foreground pixels are detected by calculating the Euclidean norm. Where 1,,,, 0, I is the current pixel intensity value B is the background intensity value and T is the foreground threshold. Kazuki Nakagami et. al [18] has developed simplified shadow removal approach by using interim result of transformed domain GMM (Gaussian Mixture Model) foreground segmentation. The approach was based on the fact that the spatial frequency distribution did not change from the background in the shadow areas. Due to employing gray level picture processing and to utilize only low frequency components in the transformed domain, the resultant shadow removal approach drastically reduced the amount of processing, compared to the conventional shadow removal approaches based on pixel based color components processing. Julio Cezar Silveira et,al [19] proposed a small improvement to an existing background model, and incorporated a novel technique for shadow detection in grayscale video sequences. The proposed algorithm works well for both indoor and outdoor sequences, and does not require the use of color cameras. Shadows were also correctly detected and removed in this indoor footage, and valid foreground moving objects were correctly segmented. One drawback of the proposed technique was the misclassification of valid foreground objects as shadows in video sequences containing a homogeneous background with homogeneous (and darker) foreground objects. III. ANALYSIS OF PROBLEM Shadow detection and removal in various real life scenarios including surveillance system, indoor outdoor scenes, and computer vision system remained a challenging task. Shadow in traffic surveillance system may misclassify the actual object, reducing the system performance. There are many algorithms and methods that help to detect a shadow in image and remove such shadow from that image. Nowadays, surveillance systems are in huge demand, mainly for their applications in public areas, such as airports, stations, subways, entrance to buildings and mass events. In this context, reliable detection of moving objects has been the most critical requirement for any surveillance systems. In the moving object detection process, one of the main challenges is to differentiate moving objects from their shadows. Moving cast shadows are usually misclassified as part of the moving object making the following analysis stages, such as object classification, tracking, or to perform inaccurate. In traffic surveillance, system must be able to track the flow of traffic. Shadows may lead the misclassification of traffic, due to that exact traffic flow is difficult to determine. It will become major drawback of a surveillance system. Shadow detection and removal is an important task in image processing when dealing with the outdoor images. Shadow occurs when objects occlude light from light source. Shadows provide rich information about the object shapes as well as light orientations. Some time we cannot recognize the original image of a particular object. Shadow in image reduces the reliability of many computer vision algorithms. Shadow often degrades the visual quality of images. Shadow removal in an image is an important pre-processing step for computer vision algorithm and image enhancement.

IV. IMPLEMENTATION Here present our algorithm of the Shadow detection and removal. Each design decision will be presented and rationalized, and sufficient detail will be given. Simulation Results For Human object with single shadow. Start Sequence of Frames No Motion Detected? Background Modeling Background Subtraction/ Foreground Extraction Draw Bounding Box and Human Object Tracking Morphological Process Shadow Detection and Removal Fig. 3.Background subtraction and shadow removal for Human object with single shadow. (a) Video frame (b) Background subtraction (c) Object detection (d) shadow removal For Single object with two shadows in opposite direction. End Fig. 2. Data Flow Diagram V. EXPERIMENTAL RESULTS The experimental results are presented to show that the proposed methods can achieve promising performance in background subtraction and foreground object extraction. This system detects and tracks the moving objects exactly and removed the shadow efficiently. In this approach, the background scene is modeled using a set of background image frames. A. Basic Steps Obtaining the image The first and probably most significant challenge was capturing the images with shadow to the computer. Issue involved with this challenge is: the video captured must only include the object with shadow. Detecting the object and remove shadow The object detection was the most challenging aspect of this dissertation. Then apply object detection and shadow removal algorithm to detect object and remove its shadow. Fig. 4.Background subtraction and shadow removal for Single object with two shadows in opposite direction.(a) Video frame (b) Background subtraction (c) Object detection (d) shadow removal

VI. CONCLUSION In this paper, an approach capable of detecting motion and extracting object information which involves human as an object has been described. The algorithm involves modeling of the desired background as a reference model for later used in background subtraction to produce foreground pixels which is the deviation of the current frame from the reference frame. The deviation which represents the moving object within the analyzed frame is further processed to localize and extracts the information. REFERENCES [1] Khan, Z., Balch, T. and Dellaert, F. (2004) An MCMC-based Partical Filter for Tracking Multiple Interacting Targets, 8th European Conference on Computer Vision (ECCV), Proceedings, vol.4, pp.279-290. [2] Tao Zhao, Ram Nevatia, (2004) Tracking Multiple Humans in Crowed Environment, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'04) vol. 2, pp.406-413. [3] Sidenbladh, H. and Black, M.J. (2001) Learning image Statistics for Bayesian Tracking, IEEE International Conference on Computer Vision (ICCV), Vol.2, pp.709-716. [4] Hartigan, J. and Wong, M.(1979) Algorithm AS136:A K-Means Clustering Algorithm, Journal of the Royal Statistical Society. Series C (Applied Statistics), vol. 28, No.1. pp. 100-108. [5] Heisele, B. (2000) Motion-based Object Detection and Tracking in color Image Sequence, 4th Asian Conference on Computer Vision. [6] Heisele, B., Kressel, U. and ritter, W. (1997) Tracking Non-Rigid Moving Objects Based on Color cluster Flow, Conference on Computer Vision and Pattern Recognition, Proceeding, pp.253-257. [7] Comaniciu, D., Ramesh, V. and Meer, P. (2000) Real-time Tracking of Non-rigid Objects using Mean shift, IEEE Conference on Computer Vision and Pattern Recognition (CVPR'00), Vol.2, pp.142-149. [8] Comaniciu, D., Ramesh, V. and Meer, P. (2003) Kernel Based Object Tracking, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol.25, No. 5, pp.564-577. [9] Collins,R.and Liu,Y(2005), Online Selection of Discriminitive Tracking Feature, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol.27, No. 10, pp.1631-1643. [10] Nguyen, H.T. and Semeulders, A.(2004) Tracking aspects of the Foreground against the Background, 8th European Conference on Computer Vision (ECCV), Proceedings, vol. 2, pp.446-456. [11] Kass,M.,Witkin, A., and Terzopoulos, D.(1988) Snakes:active contour modules, International Journal of Computer Vision, vol.1, No. 4, pp.321-331. [12] Isard, M. and Blake, A. (1996) Contour tracking by stochastic propogation of conditional density, 4th European Conference on Computer Vision, Proceedings, vol.1, pp.343-356. [13] R. Cucchiara, C. C. Grana, M. Piccardi, and A. Prati, Detecting moving objects, ghosts, and shadows in video streams, PAMI, vol. 25, no. 10, pp. 1337 1342, October 2003. [14] S.J. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, and H. Wechsler, Tracking groups of people, Computer Vision and Image Understanding, vol. 80, no. 1, pp. 42 56, October 2000. [15] T. Horprasert, D. Harwood, and L.S. Davies, A robust background subtraction and shadow detection, in Asian Conference on Computer Vision (ACCV 2000), Taipei, Taiwan, January 8-11 2000 [16] P.L.Rosin and T.Ellis, Image difference threshold strategies and shadow detection, Proc. The sixth British Machine Vision Conference, 1994 [17] S.Saravanakumar, A.Vadivel, C.G.Saneem Ahmed, Multiple human object tracking using background subtraction and shadow removal techniques, 2010 International Conference on Signal and Image Processing. [18] Kazuki Nakagami,Toshiaki Shiota and Takao Nishitani Low Complexity Shadow Removal on Foreground Segmentation, ICASSP 2011 [19] Julio Cezar Silveira Jacques Jr, Claudio Rosito Jung, and Soraia Raupp Musse. Background Subtraction and Shadow Detection in Grayscale Video Sequences, IEEE Proceeding of the XVIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 05) pp: 530-1834/05, 2005. [20] Rajni Thakur, Shveta Chadda, Navjeet Kaur, Review on Shadow Detection and RemovalTechniques/Algorithms, IJCST Vol. 3, Issue 1, Jan. - March 2012 ISSN : 0976-8491 (Online) ISSN : 2229-4333 (Print). [21] Mathworks http://www.mathworks.com [22] Shiuh-Ku Weng, Chung Ming Kuo and Shu-Kang Tu, Video object tracking using adaptive Kalman filter, Journal of Visual Communication and Image Representation, Volume 17, Issue 6, pp.1190-1208, 2006.