Fire Recognition in Video. Walter Phillips III Mubarak Shah Niels da Vitoria Lobo.

Transcription

1 Fire Recogitio i Video Walter Phillips III Mubarak Shah Niels da Vitoria Lobo {wrp65547,shah,iels}@cs.ucf.edu Computer Visio Laboratory Departmet of Computer Sciece Uiversity of Cetral Florida Orlado, Fl 3286

2 Fire Recogitio i Video Abstract This paper presets a system for automatic fire detectio i video images. Our system uses color ad motio iformatio gaied from video sequeces to locate fire. Ulike previous methods, this method is early uiversally applicable because of its isesitivity to camera motio. Two specific applicatios ot possible with previous algorithms are the recogitio of fire i the presece of global motio ad the recogitio of fire i etertaimet ad movies for possible use i a automatic ratig system.. Itroductio Existig methods of visual fire detectio rely almost exclusively upo spectral aalysis usig expesive camera equipmet. This approach, however, is still very vulerable to false alarms, especially caused by images of the su. I additio, this limits fire detectio to those who ca afford the high price of the expesive sesors that are ecessary to implemet these methods. There are two methods that seem promisig [][2]. However, both of these rely upo ideal coditios. I the first, camera iitializatio requires the maual creatio of rectagles based upo the distace of portios of a scee from the camera. Because camera iitializatio is so difficult, the camera must also be statioary. The secod method [2] is based upo strictly greyscale images. Though computatioally iexpesive, this method oly works where there is very little that may be mistake for fire (i Aircraft dry bays, where there is o su, for example). Oce agai, the camera must be statioary for this method to work. The method described i this paper, employig oly color video iput, does ot require a fixed camera, ad is desiged to detect fire i early ay eviromet. I this method, first, a color predicate is built usig the method preseted i sectios 2. ad 2.2 of this paper. The small subsets of color video sequeces are examied, ad based upo both the color properties, ad the temporal variatio (sectio 3), a probability is assiged to each pixel locatio idicatig the likelihood that each pixel is a fire pixel (sectio 4). Based upo some coditios also preseted i sectio 4, we ca determie if this test has bee reliable. The reaso this is a effective combiatio is explaied i sectio 5. If the test to fid fire has bee successful, a erode operatio (sectio 6) is performed to remove spurious fire pixels. This is followed by a regio growig algorithm desiged to fid fire ot idetified i the iitial testig (sectio 7). The overall fire fidig algorithm is described i sectio 8. The results preseted i sectio 9 show the effectiveess of this algorithm. Future work ad coclusios follow i sectios 0 ad, respectively. 2.. Color Detectio Rather tha makig use of spectral iformatio obtaied via models of fire (which may igore slight irregularities ot cosidered for the type of fire beig observed), our system is based upo first creatig a color predicate through traiig usig test data from which the fire has bee isolated maually (see figure ). This is accomplished usig the ski detectio algorithm described i [3]. Usig other meas, it is very difficult to isolate the colors that are withi fire. This modificatio allows for icreased accuracy if traiig sequeces are available for specific kids of fires, while allowig for a geeric fire look-up table if traiig sequeces are ot available (assumig the user ca create a geeric, all-purpose fire probability table). This results i a fuctio, which we shall call Colorlookup, which, give a (R,G,B) triple, will retur a boolea. For our tests, we foud that usig about te traiig images from several of our traiig sets to be sufficiet to costruct a effective color predicate. Figure : Some traiig images ad their maually created masks 2.2. Color i Video Fire is gaseous, ad as a result, i additio to becomig traslucet, it may disperse eough to become udetectable, as i figure 2. This ecessitates that we average the fire estimate over small widows of time. A simple way to compute the probability of a pixel havig fire s color over a sequece is by temporally averagig the probability that a pixel is fire over time.

3 Figure 2: Traslucet fire with a book burig behid it. More precisely: Colorlookup( Pi ( ) i where Colorlookup is the boolea color predicate produced by the ski-detectio algorithm, is the umber of images i a sequece subset, P i is the i th frame i a sequece subset. P i ( is the (R,G,B) triple foud at I ( Pi ( ) I ( Pi ( ) i 2 DIFFS ( locatio ( i the i th image. For our calculatios, we have determied that may be a legth greater tha or equal to 3 images. 3. Fidig Temporal Variatio Color aloe is ot eough to idetify fire. There are may thigs that share the same color as fire, but that are ot fire, such as a desert su ad red leaves. The key to distiguishig betwee the fire ad the firecolored objects is the ature of their motio. The motio of fire will cause the positio of flames betwee two cosecutive frames (take at 30 fps) to be i completely differet locatios (see figure 3). Our system makes tests for this by checkig for a high average variatio of Figure 3: Flicker i two cosecutive images Figure 4: The Su i this image is fire colored. This is ot detected as fire by our system because of low temporal variatio. itesity over a umber of images. For a sequece subset cotaiig images this temporal variatio may be defied as: where P i is the i th frame i a sequece of images, ad I is a fuctio that give a (R,G,B) triple, returs the itesity (which is (R+G+B)/3 ). The highest possible temporal variatio occurs i the case of flicker, that is, whe a pixel is chagig rapidly from oe itesity to aother. This geerally occurs oly i the presece of fire. This is i cotrast to global motio or motio of rigid bodies, which produces lower temporal variatio as a result of the relatively smooth itesity gradiet possessed by most objects i the spatial domai which ca cause oly slight variatios as objects chage locatios withi a image, eve i the presece of global motio. By first correctig for the chage of o-fire pixels it is possible to determie if fire-colored pixels actually represet fire. This is most easily doe by: ) Decidig which pixels are fire cadidates by thresholdig the Color matrix. 2) Fidig the average chage i itesity of all ofire pixels 3) Subtractig this average value from the value i DIFFS at each locatio. More precisely: First compute ofirediffs y, 0 DIFFS( y, 0 The lower summatio represets the umber of pixels i the image that are determied to be fire colored.

4 ad the compute σ ( DIFFS( ofirediffs Figure 4 shows the importace of the temporal variatio because ow the algorithm correctly rejects the part of the scee that is fire-colored. 4. Heuristic Aalysis Sice the test to fid fire is directly depedet upo both color ad temporal variatio, it is best expressed by a simple test: if color( ad σ ( > k Fire( 0 otherwise where k is a experimetally determied costat. This is a measure of the temporal variatio of the firecolored pixels. There are several exceptios that give reaso that merely fidig Fire is ot eough. The first of these occurs specifically i sulight. As a result of reflectio, sulight ca cause ew light sources to appear ad disappear. For that reaso, there are ofte a few pixels i a image cotaiig the su that score high eough o the temporal variatio test to be recogized as fire, though most do ot. Therefore, sequeces cotaiig a high umber of fire-colored pixels, but with a low umber of fast movig-fire colored pixels must usually be set ito a fire ulikely/udetectable class. Specifically, this meas coutig the umber of pixels i Fire that are s ad comparig it to the umber of pixels that have color(. If the umber of fire colored pixels is less tha some threshold, the we say that there is o fire i the sequece at all. For our tests, this threshold was 0. If the umber of pixels detected as fire is greater tha this threshold, but the ratio of fire pixels to fire-colored pixels is high, the the sequece must be placed ito the fire ulikely/udetectable class. For our tests, at least oe out of every thousad fire-colored pixels must be foud as fire. There is oe case that cotais fire that this method is uable to detect: if a sequece is recorded is close eough to a fire, the fire may fully saturate the images with light, keepig the camera from observig chages or eve colors other tha white. Therefore, if cotrast is very low ad itesity is very high, as i figure 5, sequeces must be put ito a fire likely/udetectable class. 5. Correlatio Issues It is possible that motio ad color iformatio could result i the same iformatio so that kowig oe is the same as kowig the other. I order to determie the correlatio, we took a radom samplig of 8,000 poits from video data used i our experimets. For each poit, we stored. The value of DIFFS 2. The value of COLOR We the compared them usig this formula to fid ρ, the correlatio coefficiet: ( xi µ x )( yi µ y ) ρ ( σ σ ) x y where x i is the i th sample take from Color, y i is the I th sample take from DIFFS, is the size of the sample, µ ad µ y are the sample meas of Color ad Diffs, ad σ σ y are their sample variaces. The correlatio we measured by this method was.072, idicatig that these two cues are idepedet. 6. Dealig with Reflectio: Figure 5: Fire Likely/Udetectable Before Erosio Fire Detected. Figure 6: Reflectio o groud detected at lower left. I this ad all examples, the detected locatio of fire is outlied i white. Oe of the largest problems i the detectio of fire is the reflectio of fire upo the objects ear the fire. However, barrig surfaces with high reflectivity, such as mirrors, reflectios ted to be icomplete. A erode operatio ca elimiate most of the reflectio i a image. For our study, the followig erode operatio worked the best: examie the eight-eighbors of each pixel. Remove all pixels from Fire that have less tha five eighteighbors. Figure 6 shows the results of this stage.

5 7. Regio Growig These values cotaied i the boolea image created from the erosio stage will cotai oly the most likely fire cadidates; to have avoided false positives thus far, our coservative strategy will ot have detected all of the fire i a sequece subset. But this is ot a accurate measure of the total quatity of fire i the sequece subset. For example, some of the fire i a sequece will ot appear to be movig because it is right i the ceter of the fire. Hece, i order to fid the rest of the flame, it is ecessary to grow regios usig color aloe. To fid the total quatity of fire pixels i the sequece subset, the followig regio growig algorithm is applied:. Create a ew boolea image, Fire ad a variable, dist ad set Fire Fire(, dist 0 2. For all pixels i Fire that are adjacet to pixels such that Fire(, if >(k +dist) set Fire 3. Fire( Fire 4. dist dist+k 3 5. Loop to step 2. Where dist is a threshold that begis at zero ad is icremeted, ad k 3 is a experimetally determied costat; i our experimets we used This method is applied util it there is o chage from oe step to the ext (i.e., whe o steps have ay effect o Fire) 8. Algorithm for Fire Detectio The steps i the algorithm are the followig:. Maually select fire from images ad create a color predicate usig the algorithm i [3]. Ulike [3], do ot threshold. Create a fuctio that, give a (R,G,B) triple, returs a real umber. Call this Colorlookup. 2. For cosecutive images, calculate color ad DIFFS, DIFFS ( i 2 I ( P ( ) I ( P i i ( ) ColorLookup( Pi ( ) i where Colorlookup is the predicate created i step #. 3. Calculate the motio of the part of the image that is ot fire ad subtract it from each value i sigma. First calculate: ofirediffs DIFFS( ad the calculate: σ ( DIFFS( ofirediffs where the summatio is over <k, ad k is a experimetally determied costat. 4. Create a fire boolea image, if color( ad σ ( > k Fire( 0 otherwise where k is a experimetally determied costat. 5. Classify sequece as fire likely/udetectable if average itesity is above some experimetally determied value. 6.a. Calculate the umber of possible fire cadidates by addig together all the values of Fire. Call this umber Numfire. b. Calculate the umber of pixels detected as fire by coutig the umber of pixels i Fire that are above some threshold value. Call this umber Foudfire. c. Calculate Foudfire/Numfire. If this value is less tha some experimetally determied costat, classify the sequece as fire ulikely/udetectable. 7. Examie the eight-eighbors (the eight adjacet pixels) of each pixel. Remove all pixels from Fire that have less tha five eight-eighbors that are. 8.a. Create a ew boolea image, Fire ad a variable, dist ad set Fire Fire(. dist 0 b. For all pixels i Fire that are eight eighbors of pixels such that Fire(, if >(k +dist) Fire c. Fire Fire d. dist dist+k 3 e. Loop to step b. where dist is a costat that begis at zero ad is icremeted, ad k 3 is a experimetally determied costat.

6 Sequece# Legth Frames w/fire False + False - Descriptio A fire i a fireplace A settig su A fire at ight Su i the desert A ma s face Fire burig i the street 20 feet from the camera Figure 7: All measuremet are i umber of frames 9. Experimetal Results: Figure 8: The su is ot recogized, eve with bal This motio. method has bee effective for a large variety of coditios (see figure 7). False alarms, such as video of the su movig (see figure 8) are ot detected by this method because i all realistic sequeces, the rate of global motio is almost always much less tha the expected speed of the fire. Figure 9: Very bright image ad very dark image; detectio occurs i both cases. Lightig coditios also have o effect upo the system; it has bee able to detect fire i a large variety of fire images, as i figure 9. Certai types of fires, such as cadles, blow torches, ad lighters, are completely cotrolled, ad always bur exactly the same way without flickerig (see figure 0). Ufortuately, the algorithm fails for these cases because of the lack of temporal variatio. However, these cases are ot usually importat to recogize because cotrolled fires are ot dagerous. 0. Future Work The ext step i the developmet of this algorithm would be error reductio. There are three equatios stated i this algorithm that have costats that must be determied experimetally. The error i this method ca be reduced by employig traiig to determie these values. Because of the low computatioal demad ecessary for this algorithm, it is also possible to use it as part of a robust, real-time system for fire detectio. Aother directio would be to distiguish betwee differet types of fires. Fially, predictig fire s path i video would be iterestig.. Coclusio This paper has preseted a robust system for detectig fire i color video sequeces. This algorithm employs iformatio gaied through both color ad temporal variatio to detect fire. We have show a variety of coditios i which fire ca be detected, ad a way to determie whe it caot. Through these tests, this method has show promise for detectig fire i real world situatios. Ackowledgemets The author of this paper would like to thak xxxxx ad yyyyy for their replies to questios about their previous work. I additio, much gratitude goes to zzzzz at our Uiversity, without whose previous work much of this would ot have bee able to be accomplished. Refereces []Healey, G., Slater, D., Li, T., Drda, B., Goedeke, A.D. A system for Real-Time Fire Detectio, IEEE Cof Computer Visio ad Patter Recogitio, 994. [2]Foo, S. Y. A rule-based machie visio system for fire detectio i aircraft dry bays ad egie compartmets, Kowledge-Based Systems, vol [3]Kjedlse R, Keder, J. Fidig Ski i Color Images, 996 Face ad Gesture Recogitio P Figure 0: Detectig a match or cadles meas detectig based mostly upo color.