FOREGROUND OBJECT EXTRACTION USING FUZZY C MEANS WITH BIT-PLANE SLICING AND OPTICAL FLOW

Transcription

1 FOREGROUND OBJECT EXTRACTION USING FUZZY C EANS WITH BIT-PLANE SLICING AND OPTICAL FLOW SIVAGAI., REVATHI.T, JEGANATHAN.L 3 APSG, SCSE, VIT University, Chennai, India JRF, DST, Dehi, India. 3 Professor, SCSE, VIT University, Chennai, India. msivagami@vit.a.in, revathi.theerthagiri@vit.a.in, jeganathan.l@vit.a.in ABSTRACT: This paper address the problem of extrating the foreground objets.we proposed a novel tehnique for foreground objet extration using Fuzzy--means with Bit-plane sliing and optial flow. odeling the bakground is a hallenging task in foreground extration. Before modeling the adaptive bakground image, the image is proessed by Lab olor model and Bit-plane sliing. Then the frame is modeled as a bakground using Fuzzy C - means algorithm with threshold.the foreground is extrated based on this model. The bakground model is updated at regular intervals of time. At last, the optial flow is applied to the foreground extrated image to eliminate the errors aused due to the movement of bakground objet suh as tree leaves, et. The videos are taken from the weizmann dataset and examined for this method. This method yields better results than the previous algorithms with respet to memory onsumption and quality of the extrated image. KEYWORDS: Foreground extration, Fuzzy C-eans, optial flow, G, Sliing. K-means, Bit-Plane. INTRODUCTION Identifying, moving objets from a video sequene is a fundamental and ritial task in many omputer-visions appliations. Foreground objet extration is also known as bakground subtration. The onventional approahes for the foreground objet detetion are bakground subtration, temporal differening, orrelation, olor based segmentation and optial flow. Bakground subtration is a quik and dirty way of loalizing moving objets []. It is the ommon approah for foreground extration,whih identifies moving objet is made up of a olor, whih differs from those in the bakground. Typial bakground subtration methods label in motion every pixel at a time t, whose olor is signifiantly different from the ones in the bakground [],[],[3]. Temporal differening [4],[5],[6] is used in dynami environments. The resultant image is obtained from the differene of urrent frame and the previous frame by applying ertain threshold value. Optial flow tehniques are also used in dynami environments to detet the moving objets using the flow vetor. Optial flow is the pattern of apparent motion of objets, surfaes, and edges in a visual sene aused by the relative motion between an observer and the sene. The optial flow tehnique is used for motion detetion, objet segmentation, time-to-ollision, et.[7].the foreground extration may falsely detet the moving bakground objets as foreground, to avoid that optial flow method is used. The luster based tehniques are also used for foreground extration. Clustering algorithms label the unlabeled data, based on the similarity measure between the data patterns. There are two types of lustering. They are hierarhial and partition lustering. The hierarhial lustering is onsidered as non-parameterized lustering and it is further divided into agglomerative and divisive. Agglomerative based lustering, at first takes N single point lusters and merge lusters to beome larger and larger. There are three types of agglomerative lustering i) single-link algorithm [8] ii) omplete-link algorithm [8] and iii) minimum-variane algorithm [8]. Divisive lustering splits the entire dataset into a single point luster. It is a reverse proess of agglomerative lustering. The partition based lustering is onsidered as parameterized lustering. It uses an iterative optimization tehnique to minimize the 337

2 objetive funtion. The ommonly used parameterized lustering is k-means lustering, whih is the most popular and easily used lustering algorithm. The lustering proess inludes harateristi representation, similarity measurement, olleting data points, data abstration and output validation. K-means is the simplest unsupervised learning algorithm. It follows a simple and easy way to lassify a given data set through a ertain number of lusters (assume k lusters) fixed as a priori. The main idea is to define k entroids, one for eah luster. These entroids should be plaed in a unning way beause of different loation auses different result. So, the better hoie is to plae them as muh as possible far away from eah other [9].The other lustering method is the one based on learning a mixture of Gaussians[0],[][]: lusters onsider as Gaussian distributions. The Expetation-aximization algorithm whih is used in pratie to find the mixture of Gaussians that an model the data. The Fuzzy C-eans is soft lustering. It lusters the data based on the membership value. The main differene between fuzzy lustering and other lustering tehniques is that it generates fuzzy partitions. The main advantage of this method is, identifying the pixel membership in eah luster to segment the overlapped objets aurately [9]. All the partition based lustering needs number of entroids as a parameter. The number of entroids an be deteted using many ways one of them is histogram.the image is ompressed to redue the memory by using Bit-Plane sliing. Bit-Plane sliing is a tehnique in whih image is slied at different planes. It bit level ranges from 0(LSB) to 7(SB).. RELATED WORKS There are many tehniques present in the literature for foreground objet extration. The most foreground detetion method uses either the temporal or spatial information of the image sequene. The onventional foreground extration tehniques are bakground subtration [3], temporal differening [5] and optial flow [7]. Bakground subtration [3] takes several seonds of frames to model eah pixel of a bakground with a normal distribution. Then subtrat the urrent image from the bakground image and apply threshold to get the foreground objet. Temporal differene [3],[4] uses pixel-wise differene between two or more onseutive frames in an image to detet the moving regions. It is adaptive to dynami environments. Optial flow based motion detetion uses harateristis of flow vetors of moving objets over time to detet moving regions in an image sequene. Bakground subtration is a simple tehnique used before, now temporal differene is used for dynami foreground extration and to yield better results the optial flow method is used. Olivier Barnih and ar Van Droogenbroek [] proposed a universal bakground subtration tehnique. Rita Cuhiara, assimo Piardi and Andrea Prati [4] proposed a method to identify the moving objet with high auray and low false negatives.andar Kulkarni [9] proposed a tehnique for the foreground extration that, the spatial histogram of a single bakground image is modeled as a Gaussian mixture model. To extrat the foreground, input frames are ompared with urrent bakground model and the foreground pixels are lassified aording to the intensity differenes. To mitigate errors aused due to the movement of the bakground objets, the optial flow method is used. The paper [9, 4] uses optial flow for foreground extration.. Lab Color Spae Color spae defined by CIE, based on one hannel for Luminane (L) and other two olor hannels for a* and b*. The a* axis is green at one extremity (represented by -a), and red at the other (+a). The b* axis have blue at one end (-b), and yellow (+b) at the other. Lab olor spae is muh more intuitive than RGB. The LAB olor spae is also the "devie independent" olor spae. In this model the olor differenes whih orrespond to the distane when measured alorimetrially. The a axis extend from green (-a) to red (+a) and the b axis from blue to yellow (+b).the brightness inreases from the bottom to the top of the three dimensional model. The luminane ranges from 0 to 00, the A omponent ranges from green to red and the B omponent ranges from blue to yellow. By means of this model we an handle olor regardless of speifi devies. This olor spae is better suited to many digital image manipulations than the RGB spae, whih is typially used in image editing programs. For example, the Lab spae is useful for sharpening images and the removing artifats in JPEG images or images from digital ameras and sanners.. Histogram The histogram is the basis for numerous spatial domain proessing tehniques []. The histogram of the digital image is alulated using the following disrete funtion: 338

3 H (r) k =n k () Where, r k- the k th intensity value n k - is the number of pixels in the image with intensity r k. (k ranges from 0 to L-). The normalized histogram is given by P(r k )=n k /N () Where, is the total number of rows in the image. N is the total number of olumns in the image. K ranges from 0 to L -..3 Bit-Plane Sliing Bit-plane sliing is a tehnique in whih the image is slied at different paths. The bit level arranges from 0 to 7.The 0 represents the least signifiant bit and 7 represent the most signifiant bit. The higher order bit usually ontains most of the signifiant visual information. A lower order bits ontain subtle details. It is the lossy ompression tehnique whih makes less storage spae than the JPEG ompression. The steps in implementing the bit-plane sliing: i) Take the input image. ii) Convert the olor image to the gray level image. iii) Generate the resultant k bit level image based on the level of bit plane..4 Fuzzy C-eans Fuzzy lustering, otherwise known as soft lustering. Fuzzy C-eans (FC) is a method of lustering whih allows one piee of data to belong to two or more lusters. This is frequently used in pattern reognition. It is based on minimization of the following objetive funtion. This lustering method allows eah pattern to be assigned to multiple lusters. This algorithm updates the luster entroids iteratively..5 Optial Flow Optial flow [7],[5] is an approximation of the loal image motion based upon loal derivatives in a given sequene of images. That is, in D it speifies how muh eah image pixel moves between adjaent images. Thus the omputation of deferential optial flow is, essentially, a two-step proedure:. easure the spatial-temporal intensity derivatives (whih is equivalent to measuring the veloities normal to the loal intensity strutures). Integrate normal veloities into full veloity, for example, either loally via a least squares alulation or globally via regularization. The optial flow is used to eliminate the errors aused due to the movement of bakground objets suh as tree leaves, et. The optial flow algorithms are lassified into four approahes. They are gradient, phase, region and feature based methods.the gradient based method is also known as differential methods. There are three gradients based methods: Luas Kanade, Horn and Shunk and Proesmans. The phase based method finds the veloity of motion using a band - pass filter. Region based method alulates the flow vetor by using the displaement of the pixel between two onseutive frames. There are two regions based methods. They are i) differene and ii) orrelation. Feature mathing based methods alulates the flow vetors by measuring the displaement of image features. The two features based methods are Harris and Sale Invariant Feature Transform (SIFT). 3. PROPOSED ETHODOLOGY The proposed method is onerned with stati amera video images. The tehniques used in this system are a CIELAB olor model, histogram, Fuzzy C-eans with Bit-Plane Sliing and Optial Flow. The main purpose of the FCBPSOF foreground extration method is to present a fast algorithm with a less memory utilization towards other algorithms. The proposed algorithm is given below: 339

4 Input image BPI k - Bit-plane information for the bit k.r- Remainder. Color onversion (RGB to LAB) Bit-plane sliing (lab image to bit-slied image) Finding entroids using histogram 3.3 Foreground Extration (Fuzzy--means): The spatial histogram is applied to the binary slied image to find the entroids. The bakground image is modeled using Fuzzy C means algorithms with the entroids as input. j p n = u i= j= p ij x i j (4) Color onversion (RGB to LAB) Applying fuzzy--means to model the bakground n u i= = j n u i= p ij. xi p ij (5) Extrating foreground from the bakground model Eliminating falsely deteted foreground using optial flow Truly extrated foreground image 3. Color Conversion: In this system the RGB olor spae is onverted to the lab olor spae. The RGB olor is devie dependent. And so for effiiently proessing the image, the RGB image is onverted to the Lab olor spae. The Lab olor model is derived from CIEXYZ olor spae and it is devie independent. The RGB image is not diretly onverted to Lab olor model, first the image RGB is onverted to XYZ olor spae and then to Lab olor model. 3. Bit-plane Sliing: The Bit-plane sliing tehnique[6] is applied to the lab olor image. This tehnique redues the memory spae. Here, only the most signifiant bits are taken for the further proessing and by eliminating the proessing and storage of other bits. BPI k ( i, j ) = R floor (, ) ( 3 ) I i j k where, I original image. where, p is a real number greater than. u ij is the degree of membership of x i in the luster j, x i is the i th of d-dimensional measured data, j is the d- dimension enter of the luster. uij = k = xi xi p Then the foreground extration is done by omparing eah input frame I with the urrent bakground frame. The bakground frame is modeled at regular interval of time to inorporate the hanges in the environment.foreground is extrated using the equation below F(x, y) =55, if[(i(x, y)-(x, y))>t) (6) F(x, y) =0, otherwise (7) where, - bakground frame,i-urrent frame, T threshold value to detet the foreground objet. The threshold of the outdoor video image may not be too low or high.if the threshold is high some of the foreground objet is eliminated and if it is low few bakground pixels are wrongly onsidered as foreground pixels. In this system the threshold value ranges from to Optial Flow The gradient [5] based Horn and Shunk optial flow is applied to the foreground extrated image to eliminate the errors aused due to the movement of the bakground objet like tree j k 340

5 leaves, flags, eletrial wires hanging on the road, et. If optial flow has not applied, then the above movements are onsidered as foreground objets. I I I ( x + u, y + v) = I ( x, y) + u + v t t x y u =, v = t t Where, I/ x and I/ y are the derivatives of x and y oordinates of the pixel position in an image. u and v are the veloities with respet to x and y diretion. (8) 4. EXPERIENTAL RESULT AND ANALYSIS The proposed algorithm is implemented in mat lab version 7.3 on Windows XP platform. The videos are taken from Weizmann dataset. The foreground extration results obtained using proposed algorithm is better than the previous algorithms with respet to memory onsumption and better objet extration. The memory mentioned in the table (table-) inludes data memory and proessing memory for eah tehnique. 4. Figures 4.. Run Figure. Bakground Frame Figure. Current Frame Figure 3. G Figure 4. K-eans Figure 5. FC Figure 6. FCBPSOF Figure7. FCBPSOF 34

6 4.. Jump Figure 8.Bakground Frame Figure 9. Current Frame Figure 0. G Figure. K-eans Figure. FC Figure 3. FCOF Figure.4. FCBPSOF 4. Evaluation ethods Segmentation is the initial proess in the foreground extration, so the quality of the segmentation is evaluated first. If the segmentation yields good results, then only the further proessing steps in foreground extration give the perfet result. If the segmentation evaluation fails to give the best results, then obviously the further proessing tends to fail. For appropriate performane evaluation of the proposed system should ontain the following to be performed in the step by step manner ) Segmentation based evaluation ) Clustering based evaluation 3) Foreground Extration based evaluation The segmentation evaluation methods are PSNR, J and Goodness value.the lustering evaluation methods onsist of Fuzziness in partition matrix U, Fukuyama-sugeno index and Xie-Beni index. The foreground extration evaluation methods are Reall, Preision and F- measure. At last the whole system performane is evaluated by the exeution time and the memory onsumption. 34

7 4.. Segmentation Evaluation ethods The PSNR [7] is alulated by the following funtion: PSNR= 0 * log0 (56^ /SE) (9) - Number of lusters and µ ik embership degree The higher value of I, and lower value if I gives good result of lustering. where, SE is the ean Squared Error. The higher value of PSNR indiates the good segmentation. The goodness funtion [5is given by the following equation: F I = (0) where, I is the image to be segmented, is the number of regions in the segmented image, A i is the area or i th region number of pixels and e i he sum of the Eulidean distane of the olor Vetors between the original image and the segmented image of eah pixel in the region. The smaller the value of F gives the better segmentation. The J Value [] is used as the riterion to estimate the performane of the segmented region. where, J k is J = ( ) k J k J omputed N k over the region k, k is the number of pixels in the region k, and N is the total number of pixels in the image. The lower value of J gives the better segmentation. 4.. Clustering Evaluation ethods Clustering evaluation is performed by means of speial indexes alled lustering validity indexes. The lustering validity indexes are i) Fuzziness in partition matrix U ii) Fukuyama-sugeno index iii) Xie-Beni index 4... Fuzziness in partition matrix U There are two methods to measure the fuzziness degree. I I A () (3) where, I and I -validity indexes Number of data items n ei i= ( U) = µ i= k= ( U) = ln( µ µ ik ik i= k= ik ) I Fukuyama-sugeno index This index enables the relationship of partition with geometri harateristis of lustered data. The minimum value gives good result of lustering m I ( U, V, X) ( ) 3 = i = µ ik x k vi A k= ( x v ) ( 4) k A v = k = x (5) x- data V-mean value of the data where, Number of data items - Number of lusters and µ ik embership degree v i luster enters Xie-Beni index This index is given by the formula = µ m where, i= k= ik k i (6) Numbe min { } i, j vi v j r of data items - Number of lusters and µ ik embership degree V luster enters x- Data The minimum value of this gives good result of lustering Foreground Extration Evaluation ethods The reall, preision and F-measure [6] are the three methods whih is used to evaluate the performane of the foreground extration. Reall is defined as the ratio of the assigned foreground pixels (AFP) to the true foreground pixels (TFP). Reall=AFP/TFP (7) k x v 343

8 Preision is defined as a ratio of the true foreground pixels (TFP) to the assigned foreground pixels (AFP). Preision=TFP/AFP (8) F-measure ompares the performane, onsidering both the reall and preision simultaneously. F-measure= pr p + r (9) where, p - preision and r - reall. High reall, high preision and high F-measure shows the high performane of the proposed system. 4.4 Tables Table : Result for the Evaluation ethods of Segmentation Finally the system is evaluated based on exeution time and memory onsumption. The exeution time and the memory required for the proposed system are less when ompared to G and K-means. 4.3 Complexity Analysis of FCBPSOF ethod The time omplexity of our algorithm depends on following parameters m- value of fuzzifier n - Number of data points (mxn) d - Number of dimensions () - Number of lusters This proposed algorithm takes O(n 4 ) polynomial time. S. No Image ethods Exeution time (illiseonds) emory used (Bytes) Good -ness Funtion J PSNR (db) G e K-means e Run FC e FCOF e Proposed method(fcbpsof) e G e K-means e Jump FC e FCOF e Proposed method(fcbpsof) e

9 Table : Result for the Evaluation ethods of Clustering S. No Image ethods I I I 3 I 4 Run G K-means FC FCOF Proposed method(fcbpsof) G Jump K-means FC FCOF Proposed method(fcbpsof)

10 Table 3: Result for the Evaluation ethods of Foreground Extration S. no Image ethods Reall Preision F-measure G K-means Run FC FCOF Proposed method(fcbpsof) G K-means Jump FC FCOF Proposed method(fcbpsof) CONCLUSION AND FUTURE ENHANCEENTS The proposed system uses Fuzzy C- eans for foreground extration, the Bit-plane sliing to redue the memory spae and optial flow for eliminating false foreground pixels. It takes less memory and less exeution time when ompared with other methods. The performane is evaluated using reall, preision, F-measure and four lustering index funtions. In future GPU is used to speed up the proess to handle real time video images and the membership onstraint in the fuzzy lustering will be eliminated. ACKNOWLEDGENT This work has been funded by the Department of Siene and Tehnology, New Delhi, India. REFERENCES: [] ing, Tien, pao, Foreground detetion in multi-amera surveillane system, IPPR onferene on omputer vision, Graphis and image proessing, 006. [] Olivier Barnih and ar Van Droogenbroek ViBe: A Universal Bakground Subtration Algorithm for Video Sequenes IEEE Transations on Image Proessing, Vol:0, Jun-0. [3] L. Y. Liu, N. Sang, and R. Huang, Bakground subtration [4] using shape and olour information, Eletroni Letters, vol. 46, no., pp. 4 43, 00 [5] Rita Cuhiara, assimo Piardi and Andrea Prati, Deteting moving objets, 346

11 ghosts, and shadows in video streams IEEE Transations on Pattern Analysis and ahine Intelligene Vol:5.Ot. 003 [6] C. Li, Y. Li, Q. Zhuang, Q. Li, R. Wu and Y. Li, oving objet segmentation and traking in video, in Pro. ahine Learning and Cybernetis, Vol.8, pp , Aug [7] C. Kim and J. Hwang, "Fast and automati video objet segmentation and traking for ontent-based appliations," IEEE Transations on Ciruits and Systems for Video [8] Tehnology, vol., pp. -9, 00. [9] A. Bab-Hadiashar and D. Suter. Robust optial flow omputation. International Journal of Computer Vision, 9:59 77, 998. [0] A.k.Jain,.N.urthy, P.J.Flynn, Data lustering: A review AC Computing Surveys, Vol. 3, No. 3, September 999. [] J. B. aqueen, "Some ethods for lassifiation and Analysis of ultivariate Observations, Proeedings of 5-th Berkeley Symposium on athematial Statistis and Probability", 967, :8-97. [] Q.Zhou and J. Aggarwal Traking and lassifying moving objets from video in IEEE Pro. PETS Workshop, 00. [3] Y.Benezeth, P..Jodoin, B. andar kulkarni, Histogram based foreground objet extration for indoor and outdoor senes, ICVGIP, 00. [4] D. Lee, Effetive Gaussian mixture learning for video bakground subtration, [5] IEEE Trans. Pattern Anal. ah. Intell., vol. 7, no. 5, [6] pp , ay 005 [7] S. Cheung and C. Kamath. Robust tehniques for bakground subtration in urban traffi video. in Pro. of the VCIP, 004. [8] A. Elgammal, D. Harwood, and L. Davis. Non-parametri model for bakground subtration. in ECCV, 000 Workshop Appliations of Computer Vision, 998,pp. 8. [9]. C. Y. A new gradient based optial flow method and its appliation to motion segmentation 6th Annual Conferene of the IEEE Industrial Eletronis Soiety, :5 30, 000 [0] Gonzalez, Digital image Proessing, Pearson Eduation India, 009 [] Leyuan liu, Nong sang, etris for objetive Evaluation of bakground subtration algorithms, International [] onferene on Image and Graphis,