A Simple Cigarette Butts Detection System

Transcription

1 A Simple Cigarette Butts Detection System 0. Introduction It's likely to see cigarette butts which people carelessly throw away everywhere. For the sake of environment, they must be cleaned up by someone. And to make robots the someone, the detection of cigarette butts is essential. This paper presents a MATLAB-based cigarette butts detection system which is an algorithm composed of Color Segmentation, Image Enhancement and Feature matching. 1. Image Processing The test image: This procedure mainly aims to turn the original image to binary image which is segmented according to color and helpful for the later identifications based on shapes after some image enhancement 1.1 Color Segmentation For each pixel, if its color is close enough to the target color, take it. The distance is defined as: Distance=((R-a).^2+(G-b).^2+(B-c).^2).^(1/2) (suppose the target color is (a,b,c)) Since butts commonly are white or orange, so I choose the following two colors as the target color:

2 White(255,255,255), Orange(191,153,78) Though these two colors are chosen, they're not actually ideal. As it's too tricky to consider all the situations like the highlight, the shadow, the best way is to decide the parameter by machine learning, which requires an abundant set of samples that is unavailable to some degree. In case that sometimes one of these two might be the background of the other, which may cause mix-up when the image is transferred to a binary image, there are two matrices 'whitei' and 'orangei' to contain white or orange like pixels respectively. This can't be avoided if the color of butts is uncertain so that the chosen area should include both white and orange pixels, set as 1 in binary image., The result of orangei: The result of whitei:

3 1.2 Image Enhancement The following processes are done in both matrices. To save the space, whitel will be taken as an example. BW2 = bwareaopen(bw,40); figure;imshow(bw2); This is a function in MATLAB Image Process which removes from a binary image all connected components (objects) that have fewer than 40 pixels, producing another binary image, BW2. Result: se = strel('diamond',2); BW3 = imclose(bw2,se); figure;imshow(bw3); This is a morphologically close image operation. It's used to smooth the outline, fill gaps and cracks on the outline and some small holes. Result:

4 BW4 = imfill(bw3,'holes'); figure;imshow(bw4); This is applied to fill the holes. Result: 2. Feature Matching [B,L] = bwboundaries(bw5,'noholes'); stats = regionprops(l,'area','centroid','convexarea',perimeter,'minoraxislength','majoraxislength'); " bwboundaries" function traces the exterior boundaries of objects, And "regionprops" function measures a set of properties for each connected component (object) in the binary image. This is to get the boundary and properties of each connected component. Then for each component, to identify if it's rectangle, the shape of cigarette butts, there are some metrics to calculate. axisratiometric: it's the ratio of 'MajorAxisLength' to 'MinorAxisLength'.('MajorAxisLength' is a scalar specifying the length (in pixels) of the major axis of the ellipse that has the same normalized second central moments as the region. 'MinorAxisLength' is similar but for the minor axis) It's a rectangle so that length in one direction is longer than the other. In this paper,the threshold is set to 1.95, according to the experience. Surely there is much more appropriate value by machine learning. arearatiometric: it's the ratio of practical area to the production of 'MajorAxisLength' and 'MinorAxisLength'.

5 It's a rectangle so that the area should equal to the production of long and short edges. So the more the region is like a rectangle, the closer this value is to 1. In this paper, the threshold is set to perimeter2areametric: it's the ratio of perimeter to area. For cigarette butts, this value is usually below areametric: it's the practical area. Because when the area is too small, then the features of shapes becomes less significant, the cigarette butts of size less than 40 pixels are ignored. 3.Conclusion Why it works? If its color is white or orange and its shape meets all these metrics, the component is likely to be a rectangle and a cigarette butt. What I tried? Before do color segmentation, I immersed myself in template matching, edge detection, Hough transform and so on. In fact, I still believe these ways may work as long as they are implemented properly. How it works? Here is how it works finally:

6 Limitations: Generally speaking, if the cigarette butts are single color and a typical rectangle and the contrast between the color of butts to the background is significant then the result is perfect. Besides this system can't work well when butts overlap each other. If the butt has two kind of colors, then it will be detected twice. This image has highlight and shadow which may dim the edge between different colors.

7 Grey and white is some kind of close here. So the result has flaws. And any further adjustment may harm the generality. The wrongly labeled point is another trade-off for the shadow may happen on white cigarette butts.

8 Journal: Jan 27 run Brian's code to see what I can get Jan 28 change the mask from circle to rectangle don't work, don't know the size and the orientation Jan 29 set some thresholds for the size like length from 20 to 80 pixels every 1 pixel orientation from 0 to 360 degree every 1 degree it may need too many calculation, each pixel needs to run 'rotate' 360 times and change length or width at least 60 times and for one butt, it may use different masks. or it's needed to run each mask for the whole image. too complex Jan 30 try Hough transform it's messy. first of all need to have strict 'line'. likely there is a break point. so need to mend the line to make it bolder. and even have the line. the way of it to detect rectangle is just too strict to meet in practice. Jan 31 try edge detection too many edges are connected, have tried a lot of parameter and thresholds Feb 1 try to have a better threshold iterative threshold and other ways to set threshold, but they are under some specific assumption. so not work well Feb 2 change to color based, choose those pixels whose are white not bad has removed a lot of background already Feb 3 do some filter and image processing

9 filter don't look good. it dims the edge. and the sharp angle of rectangle. "imhist" no helping, "bwareaopen" great! "imclose" works but not sure diamond is the best choice of the "strel", Feb 4 do some adjustment in parameters not very effective hard to meet all sample images Feb 5 try HSV some said it's better than RGB don't quite understand the principal, the V value is strange. Feb 6 try Hough + color segmentation need edge detection which requires grey scale image, but need color to do segmentation, give up. Feb 7 try to detect both white and orange color of butts try to think of a metric can recognize rectangle the result is OK, but the thing is that if the background color is one of them two, then all fails. the area/perimeter looks like various do not have any feature. Feb 8 fix the white and orange problem and keep thinking how to recognize rectangle have to save them into two matrices, but the problem is if the butt is not finished with two colors. It will be detected twice. But it is acceptable. find "regionprops", good function! now I have area and the rough length and width of a connected component. Feb 9 try to find the metrics of rectangle have the value, but not satisfied Feb 10 final adjustment tired I really need to write a machine learning code to let it adjust it itself. but I don't have the data. It's hopeless for me to collect it myself. don't have enough time now. keeping doing it only causes over fitting. Let the parameters go.