Optimization of Image Processing in Video-based Traffic Monitoring

Transcription

1 Optimization of Image Proessing in Vieo-base Traffi Monitoring Fei Zhu, Jiamin Ning, Yong Ren, Jingyu Peng Shool of Computer Siene an Tehnology, Soohow University, Suzhou, Jiangsu, 5006, P.R.China Applie Tehnology College of Soohow University, Suzhou, Jiangsu, 535, P.R.China, phone: zhufei@sua.eu.n Abstrat The vieo-base traffi monitoring systems have been wiely use. The system usually reas real time monitoring vieo an onverts them into images for proessing. However, suh systems are often limite by image proessing algorithms an on t behavior as expete. We hereby stuy optimization approahes for ifferent image proessing stages. As getting a binary image is important an funamental to image proessing, we first present an aaptive threshols approah for binary onversion. The approah hooses threshols aoring to eah pixel an its spae information. Then we introue a three-imension Gaussian filter for enoising. We also propose a Gaussian moel base bakgroun generation approah so that we an upate bakgroun at any time, thus getting ri of a major limitation of bakgroun subtration approah. Moving objets shaow etetion an removing mehanism is also ae in. We put forwar a burst noise eliminating algorithm that uses several ontinuous frames to wipe off suen noise in real worl monitoring. Finally, we assert that removal of the refletion light ban is to alleviate negative effet from high-energy refletion light. We onlue that aing in white light will reue olor saturation. In the area affete by refletion light ban, there are muh more white light involve than other area. Base on the onlusion, we use Gaussian moel an an improve four iretion Sobel operator to ahieve refletion light ban elimination from image. Inex Terms Digital images, image proessing, image quality, image olor analysis. I. INTRODUCTION The rapi evelopment of image proessing failitates the appliation of vieo-base monitoring systems in ifferent omains an areas. Vieo-base traffi monitoring system, a typial kin of wiely use monitoring system, an ahieve automati surveillane by vieo obtaine from monitoring system []. However, many vieo-base traffi monitoring systems, although their onveniene an low ost, on t perform as well as expete. Most vieo-base traffi monitoring systems involve image proessing, inluing image graying, binary onversion, an enoising []. Image proessing is an essential part to the monitoring system. Optimizing image proessing is able to Manusript reeive Marh 9, 0; aepte May 7, 0. This work was supporte by the natural siene founation of Jiangsu Provine uner Grant No. BK0375. improve monitoring effet. We hereby fous on optimization of the vieo-base traffi monitoring system, mainly in image proessing optimization. We will talk about optimization for image proessing in ifferent phrases of proessing routine. In the first setion, we present an aaptive threshols approah for binary onversion. Next, we will introue a three-imension Gaussian filter to remove noise in the image. As a most popular approah, bakgroun subtration, although being effetive, is usually limite by bakgroun real time etetion. We take avantage of a Gaussian moel to generate bakgroun with whih we an refresh the bakgroun ynamially at any time. We also put forwar a burst noise eliminating algorithm to remove its interferene. Finally, we use a novel approah to erase refletion light ban whih tens to isturb monitoring baly. II. BINARY CONVERSION WITH ADAPTIVE THRESHOLDS As olor information of an image is not so muh of use, images are usually onverte to binary ones. Image binary onversion is so important that sometimes it largely etermines affet final result. In a binary image, typially blak an white are use for eah pixel. White is often use to represent foregroun objets an blak is for bakgroun ones. The simplest metho of binary onversion is by threshol [3]. Eah pixel is set to if its gray value is not less than some threshol or otherwise set to 0, as () f x, y, g x, y threshol, 0, otherwise, where g(x, y) is the gray value of pixel (x, y). We an see that threshol is a key to arry on image segmentation. An ieal threshol is the one that remains as muh feature information as possible [4]. Aaptive threshol, whih take the position information of the pixel into onsieration, is a better solution than a single preefine threshol. We arry out image binary onversion with aaptive threshols metho, as follows [5]. Step : Segment the image into foregroun part an bakgroun part with (). Set eah pixel of whih grey value is less than threshol to 0; otherwise it will be set to () 9

2 threshol N LiP Li i N i i N i L h Li h Li, () where h(l i ) represents the ourrene of the gray value L i, an P(L i ) is its ourrene probability. Step : Get threshol by (3). Set eah pixel of whih gray value is less than threshol to 0; otherwise set it to :,,,,,, f x y f x y f x y,,,,,, f x, y f x, y f x, y, f x y f x y f x y u f x y f x y f x y,,,, f x y f x y f x y v f x y f x y f x y, (3) u threshol, v where f(x,y) is the gray value of pixel (x,y) an θ is minor sign less integral. Step 3: Get an aaptive winow A of pixel (x,y). Get a loal aaptive threshol 3. Set eah pixel of whih gray value is less than threshol 3 to 0; otherwise it will be set to f x, y threshol 3, (4) M where M is the total pixel number of A. The following two figures are onverte by threshol. The left one is proesse by a single preefine threshol metho an the right one is proesse by our aaptive threshols metho. We an see the segmente objets in the right image are istint, but they are fuzzy in the left one, espeially for the vehile in the enter of the image. Fig.. The left one is proesse by onversional single threshol metho an the right one is proesse by aaptive threshols metho. Obviously the right one is more istint than the left one. III. DENOISING WITH THREE-DIMENSION GAUSSIAN FILTERING Binary onverte images usually have lots of noise. Hene we nee image enoising to reue negative effet ause by noise. There are many ommonly use algorithms, suh as neighborhoo averaging, meian filtering an Gaussian filtering. A. Neighborhoo Averaging Filtering Neighborhoo averaging filtering (NAF) [6] assigns average gray value of a pixel an its neighborhoo pixels to the pixel, as (5) g x, y (, ), M f x y (5) x, y S where S is oorinate set of neighborhoo pixels, an M is total number of pixels in S. NAF approah works goo in most ases but it tens to make image fuzzy. B. Meian Filtering With meian filtering (MF) [7], gray value of eah original pixel will be replae with the meian value of a preefine winow W. (6) shows how to get meian value. MF approah an keep etails of image an protet ege from being fuzzy. However the image is still possible to get fuzzy. g( x, y) meian { f ( x k, y l), k, l W}. (6) C. Gaussian Filtering Gaussian Filter [8] is a linear smoothing filter whose weight relies on shape of Gaussian funtion. Gaussian filter works espeially effiient in removing the noise with normal istribution (Gaussian istribution) [8].One-imension Gaussian funtion is as (7) g x x sigma e, (7) where sigma etermines with of Gaussian funtion. D. Three-imension Gaussian Filtering In our experiment [9], we use a three-imension Gaussian filter (3DGF), with the kernel funtion: W 3 sigma, xw sigma f x e, (8) sigma f x, f i g x iw where W is the with of Gaussian funtion. Data ontext winow [-3 sigma, 3 sigma] overs most filter oeffiient. We teste filtering using the three approahes with ifferent parameters. Table I shows the proessing time, an Fig. shows filtering effets. TABLE I. PROCESSING TIME OF THREE APPROACHES WITH DIFFERENT PARAMETERS. SIZE OF TESTING IMAGE: CPU:INTEL CORE I GHZ. MEMORY: 8GB. Approah Time(s) NAF(3 3) 0.5 MF(3 3) 0.6 GF(sigma=0.065) DGF(sigma=0.065)

3 a) NAF (winow size:3 3) b) MF (winow size: 3 3) ) GF (sigma=0.065) ) 3DGF (sigma=0.065) Fig.. Denoising effets of three approahes with ifferent parameters. As we oul see, GF ha least proessing time an 3DGF took seon plae. As for the effet, 3DGF, the best one, ientifie the ars as well as lines of traffi lanes in the enter of images; while two ars an some part of the traffi lanes were so inistint in resulting image by GF. In general, 3DGF ha the best ost effiieny. IV. DYNAMIC BACKGROUND REFRESHING Bakgroun subtration is a pratial approah to etet moving objet in vieo monitoring [0]. However, in pratie, environmental fators will isturb monitoring. For example, sun light illumination variation an ifferent weather onitions make environmental bakgrouns so ifferent. As a result, onventional bakgroun subtration without bakgroun refreshing perioially an ynamially usually oes not work well in real worl. We here introue a ynami bakgroun refreshing metho so as to alleviate impat from environment. A. Bakgroun Detetion We fin the bakgroun of monitoring vieo, taken as a whole, tens to hange graually within a ertain time perio rather than suenly an sharply. The hange pixels generally pertain to a Gaussian istribution with mean value μ an variane σ. We hereby utilize Gaussian funtion to moel ifferent states of a pixel throughout all time perios. Consequently, given time t, the states of pixel (x, y) are a time sequene vetor. Eah pixel is subjet to Gaussian istribution P z t e xu, z I x, y, k, x,, M; y,, N; k,, t, (8) where I is an image sequene an (x, y, k) represents the pixel (x, y) of frame k. We use the moel to fit the new oming frame of pixel. Given a pixel, if the values of H (hue), S (saturation), an V (brightness) omponents [] are all between μ an σ, it will be set as a bakgroun pixel; otherwise it is a foregroun one. However, in HSV [] moel, if the value of S omponent is less than some threshol, the image will be regare as a monohromi one. As a result, H omponent beomes not so useful an olor preision of the image reflete by omponent of S rops with ereasing of V omponent value. Hene, we an see that H omponent is only involve in some eiate ases, S omponent is use to exam whether the image is hromati or monohromi, an V omponent is essential all through the proessing. Let H, S, V be the H, S, an V omponent value of a pixel respetively; let H, S, V be the H, S, an V omponent value of a new oming pixel respetively; let S T be a threshol of saturation; let σ be the stanar eviation. We an use the following riterion to examine whether the oming pixel is a bakgroun one or a foregroun one. There are altogether four kins of ases: Case : if both S an S are less than S T, we will hek whether variation of V omponent is not more than variane σ. If true, the pixel is regare as a bak one. Conition: ( S ST ) AND ( S ST ) ; Criterion: V V. Case : if S is less than S T an S is greater than S T, we will hek whether both variation of V omponent an variation of S omponent are not more than variane σ. If true, the pixel is regare as a bak one. Conition: ( S ST) AND ( S ST) ; Criterion: V V AND S S. Case 3: if S is less than S T an S is less than S T, we will hek whether both variation of V omponent an variation of S omponent are not more than variane σ. If true, the pixel is regare as a bak one. Conition: ( S ST) AND ( S ST) ; Criterion: V V AND S S. Case 4: if both S an S are greater than S T, we will hek whether variation of V omponent, variation of S omponent, an value of S multiply osine the variation of H omponent are not more than variane σ. If true, the pixel is regare as a bak one. Conition: ( S ST) AND ( S ST) ; Criterion: V V AND S S AND S os H H. After bakgroun etetion, those unmathe pixels will be kept unhange an those mathe ones will be upate by the (9): x, max min, x, where σ min is a threshol of noise, α represents the moel learning rate, an x enotes the omponents of H, S an V. As the initial istribution of pixels is unknown, we set the omponent values of the pixels in first frame to be the mean (9) 93

4 value of Gaussian istribution an variane to be zero. After that we save values of H, S an V omponents, as well as orresponing mean values an variane values. Then we will ompare the values with the save ol values. Finally we will upate the ones whih have the same values as the save ol values, an leave alone others. B. shaow etetion The shaow of the moving objets will make objet separation omponent inorretly etet objet along with its shaow as moving objet. If the subsequent proessing is base on etete moving objet, it will result in mistake []. Hene, we nee to etet an remove the shaow of moving objets. We fin that ompare with the objet part, the shaow part is arker an has less olor. By this riterion, we put forwar an approah to etet an remove the shaow of moving objets. We ompare the value of the bakgroun pixel with that of shaow pixel. If the variation of H omponent value an the variation of S omponent value are both less than some given threshol, it an be etermine as a shaow pixel. The riterion is showe as follows. V That is, give a pixel (x, y), if an S S Ts, V an H H TH, we will set S omponent value of (x, y) to ; otherwise we will set it to 0. Here T s an T h enote the threshols of S omponent an H omponent respetively. As the V omponent value of the pixel in shaowe area is generally less than that of the pixel in unshaowe area, γ an β are less than. Meanwhile β is use to measure the strength of brightness. The greater β shows the brighter the light the pixel has. By this, we an remove the shaow of moving objets. The following figures show us the effet of getting separate moving objets with shaow removing. Fig. 3. The left figure is snapshot from monitoring system in urban roa. The right figure is the greye image after separating moving objets an removing it shaow. Fig. 4. The left figure is snapshot monitoring system in highway. The right figure is the greye image after separating moving objets an removing it shaow. The left two figures are snapshots from monitoring system in urban roa an in highway. The right figures are orresponing greye image after separating moving objets an removing it shaow. We an see that the moving objets are effetely ientifie an separate; the shaows are also remove. V. BURST NOISE ELIMINATING Burst noise, suh as flying bir an falling leaves, is an unexpete an suen interferene from outsie an usually exists for a very short time span [0]. Due to the harateristi that most of the burst noise merely exists in several ontinuous frames, we will take avantage of several anterior an posterior frames to remove the burst noise. First, we get subtration of urrent frame F n an previous frame Fn-, namely Dif t. Then we get subtration of urrent frame an urrent bakgroun, namely Dif b. We an see that Dif b inlues urrent moving objets an noise. As the time span between two ontinuous frames is very short, the burst noise woul generally exist in both frames. Therefore we an remove the burst noise by their subtration. Thus, Dift only has relative moving information. After that, we get moving objets an bakgroun frames Dif t an Dif b. The algorithm that segments moving objets from image with burst noise removing is shown as follows. Step : Get frame subtration Dif t Dif b. Dif x, y F x, y F x, y. (0) t n n Step : Get subtration of urrent frame an bakgroun Dif x, y F x, y B x, y. () b n Step 3: Given pixel (x, y), if Dif b (x, y) an Dif t (x, y) are both, it is regaring as a moving pixel an will be set to ;otherwise it will be set to 0. Step 4: Repeately eal with eah pixel in the frame with step to step 3. Step 5: Get total movement value by summing up the results returne from step 4. Step 6: If the total movement is greater than the preetermine threshol, it will be treate as a moving objet; otherwise it is a non-moving one. VI. REFLECTION LIGHT BAND ERASING Many objets, suh as groun an water, an reflet light. Yet these refletions an be ignore as they perturb monitoring so slightly. However, light ban, whih is ause by highly refletive objets suh as glass, is a kin of lustere refletion light with high energy. It will isturb monitoring very muh [], [5], [9], as it is often inorretly regare as moving objets. Here we will introue an approah to remove the refletion of highly refletive objets. We have foun that a light ban has an obvious ege with symmetrial brightness, whih is the part with highest energy in refletion light. After binary onversion, the variation of light aroun light ban will beome more istint. We an see the istinguishing hange of brightness in Fig. 5. Inee, it is 94

5 neither environment hange of light, nor shae, an nor burst noise. Fig. 5. The image with refletion light ban. Few of existing bakgroun upating algorithms an shaow eliminating algorithms an eal with the refletion light ban [],[],[0]. In our approah, we regar removal of the refletion light ban as eliminating high-energy part of image. We have foun that the low-beam of ar healamp is asymmetri on both sies of the referene axis while the full-beam of ar healamp is symmetri. In fat, the brightness of reflete light is with a graient istribution. In partiular, the graient istribution of the part that is not affete by the refletion is uniform an similar to the graient istribution of the bakgroun. Deteting the ege of light ban is an important an funamental step of light ban eliminating. There are many operators for ege etetion algorithm, suh as Canny operator [], Sobel operator [3], an Prewitt operator [3]. As the Canny ege etetor uses a multi-stage algorithm, although usually returning goo results, it is more time onsuming than some other ones [4]. Both Sobel operator an Prewitt operator are isrete ifferentiation operators. The Prewitt operator uses averaging filtering [3] to ompute an approximation of the graient of the image intensity funtion; while the Sobel operator uses weighte filtering [3] to get an approximation of the opposite of the graient of the image intensity funtion. Taking into aount of brightness variation an real-time proessing requirement, we here use improve four iretion Sobel operator [5] to a weight to the pixels near the enter an the pixels near the ege highlight so as to get the graient approximation of brightness funtion, thus improving auray an fastening the operating spee. The improve four iretion Sobel operator for onvolution operation are given in Table II. TABLE II. IMPROVED FOUR-DIRECTION SOBEL OPERATOR FOR CONVOLUTION OPERATION We utilize the above Sobel operators to get eah graient omponent of V omponents of bakgroun image an pening image, as (): V x y V x y V x y V x y V x y V x y V x y V x y V x y V x y V x y V x y gx 3* V x, y 0* V x, y 3* V x, y 3*, 0*, 3*,, g y 3* V x, y 0* V x, y 3* V x, y 3*, 0*, 3*,, gl 3* V x, y 0* V x, y 3* V x, y 3*, 0*, 3*,, gr 3* V x, y 0* V x, y 3* V x, y 3*, 0*, 3*,, where g x, g y, g l, an g r are graient omponents for x-iretion, y-iretion, left iagonal iretion an right iagonal iretion. We use g x, g y, g l, an g r to get x an y iretion graient an iagonal iretion graient, as (3): G g g G g g x y l r,. (3) As brightness graient istribution of the light ause by iffuse refletion is ifferent from the one ause by full refletion, we an then we an test whether a given region is affete by full refletion light, as (4) t t b b G G G G threshol, (4) where G t enotes the testing image, an G b represents the bakgroun. When a full refletion light omes, we examine the variation between V omponent graient of the region an that of the bakgroun. As the energy istribution of the full refletion light rapi elines with the expansion of range of irraiation, while at the same time the iffuse refletion light elines in a slower spee rate, along with iretional harateristis of the full refletion light, it is more effetive to separate bakgroun an foregroun by using graient ifferene of the two iretions than by using graient of single iretion. As we have known it will ause a large variation of the bakgroun saturation if the target light is projete to the bakgroun. We onlue that the saturation of the testing image, S t, is greater than that of the bakgroun one, S b. Aing in white light will reue olor saturation. When using bakgroun subtration to extrat moving objets, we take avantage of the above jugment rule. If the following onitions are satisfie, we an onsier the image is affete by full refletion light rather than moving objet t t b b G G G G threshol. (5) Fig. 6 shows us the effet of refletion light ban erasing. The left one is image of Fig. 6 is the binary onverte image 95

6 an the right one is the orresponing generate greye image. We an see that most of isturbing refletion light ban is eliminate. we utilize the harateristis of H, S an V omponent of refletion light ban an onlue that removal of the refletion light ban is eliminating high-energy part of image. Base on the rule, we propose an approah to remove refletion light ban. The experiments show that the optimization approahes an algorithms prove to improve effeteness an effiieny of system. Fig. 6. The image that is after refletion light ban erasing. However, we an see from Fig. 6 that although most of refletion light ban is remove, the ege pixels that are near the light ban are also orroe. To solve the problem, we use Canny algorithm [] to etet the ege of the refletion light ban. The left figure of Fig. 7 is the ege etete by Canny before refletion light ban erasing; the right figure of Fig. 7 is the ege etete by Canny after refletion light ban erasing. Fig. 7 shows us that Canny algorithm an eal with ege orrosion. Fig. 7. The left figure is the ege etete by Canny algorithm before refletion light ban erasing; the right figure is the ege etete by Canny algorithm after refletion light ban erasing. VII. CONCLUSIONS A vieo-base traffi monitoring system must be apable of working in various weather an illumination onitions. However, it has a lot of prerequisites an onstraints. In this paper, we fous on optimizing image proessing in vieo-base traffi monitoring. We first improve the threshol-base image binary onversion approah. Different from onventional approah, out aaptive threshol take into spae information as well an proves to be a better one. Then we make a traeoff between time ost an effet, introuing a three-imension Gaussian filtering enoising approah. We use a Gaussian moel to generate bakgroun, so that we an refresh bakgroun any time, thus throwing away a major limitation of bakgroun subtration approah. At same time, we also introue metho to etet an remove shaow of moving objets so that the separate moving objets will not be isturbe by their shaows. As burst noise will often interfere monitoring, we, taking avantage of the harateristi that most of the burst noise merely exists in several ontinuous frames, put forwar an algorithm to remove burst noise by several anterior an posterior frames. Aiming at eliminating refletion light ban, 96 REFERENCES [] P. Kumar, S. Ranganath, W. M. Huang, K. Sengupta, Framework for real-time behavior interpretation from traffi vieo, IEEE Transations on Intelligent Transportation Systems, vol. 6, no., pp , 005. [Online]. Available: [] J. Zhou, D. Gao, D. Zhang, Moving Vehile Detetion for Automati Traffi Monitoring, IEEE Transations on Vehiular Tehnology, vol. 56, no.. pp. 5 59, 007. [Online]. Available: [3] H. Zhang, J. E. Fritts, S. A. Golman, Image segmentation evaluation: A survey of unsupervise methos, Computer Vision an Image Unerstaning, vol. 0, no., pp , 008. [Online]. Available: [4] O. Miller, A. Averbuh, Color image segmentation base on aaptive loal threshols, Image an Vision Computing, vol. 3, no., pp , 005. [Online]. Available: [5] F. Zhu, A Vieo-Base Traffi Congestion Monitoring System Using Aaptive Bakgroun Subtration, in Pro. of the ISECS, IEEE, 009, pp [6] D. Barash, Nonlinear iffusion filtering on extene neighborhoo, Applie Numerial Mathematis, vol. 5, no, pp., 005. [Online]. Available: [7] W. W. Boles, M. Kanefsky, M. Simaan, A reue ege istortion meian filtering algorithm for binary images, Signal Proessing, vol., no., pp , 990. [Online]. Available: [8] P. F. Johannes, D. Haeyer, Gaussian filtering of images: A regularization approah, Signal Proessing, vol. 8, no, pp. 69 8, 989. [Online]. Available: [9] F. Zhu, L.Y. Li, An Optimize Vieo-Base Traffi Congestion Monitoring System, in Pro. of the Knowlege Disovery an Data Mining, IEEE, 00, pp [0] C. C. Chiu, M. Y. Ku, L. Y. Liang, A Robust Objet Segmentation System Using a Probability-Base Bakgroun Extration Algorithm, IEEE Transations on Ciruits an Systems for Vieo Tehnology, vol. 0, no 4, pp , 00. [Online]. Available: [] T. F. Chan, S. H. Kang, J. H. Shen, Total Variation Denoising an Enhanement of Color Images Base on the CB an HSV Color Moels, Journal of Visual Communiation an Image Representation, vol., no. 4, pp , 00. [Online]. Available: [] J. Canny, A Computational Approah To Ege Detetion, IEEE Trans. Pattern Analysis an Mahine Intelligene, vol. 8, no. 6, pp ,986, [3] I. E. Abou, W.K. Pratt, "Quantitative esign an evaluation of enhanement/thresholing ege etetors, IEEE Transations on Image Proessing, Vol. 67. No. 5, pp , 979. [4] L. J. Ding, A. Goshtasby, On the Canny ege etetor, Pattern Reognition, vol. 34, no. 3, pp. 7 75, 0. [5] H. Fari, E. P. Simonelli, Differentiation of isrete multi-imensional signals, IEEE Transations on Image Proessing, vol. 3, no. 4, pp , 00. [Online]. Available: