Research Article Scene Semantics Recognition Based on Target Detection and Fuzzy Reasoning

Transcription

1 Research Journal of Applied Sciences, Engineering and Technolog 7(5): , 04 DOI:0.906/rjaset ISSN: ; e-issn: Mawell Scientific Publication Corp. Submitted: Januar 9, 03 Accepted: Februar 5, 03 Published: Februar 05, 04 Research Article Scene Semantics Recognition Based on Target Detection and Fuzz Reasoning Weiliang Liu, Changliang Liu and Yongjun Lin Department of Automation, North China Electric Power Universit, Baoding, China State Ke Laborator of Alternate Electrical Power Sstem with Renewable Energ Sources, Beijing, China Abstract: In order to get better image semantics recognition, a recognition sstem based on object detection and fuzz reasoning is presented in this stud. The sstem contains four parts: image preprocessing, feature etraction, target recognition and fuzz reasoning machine. Compared with other methods, the outputs of target detectors are fuzzed, the fuzz relationships between targets are etracted and fuzz inference is performed using fuzz automata. The eperiment indicates that this method could overcome the problems of false positive and false negative of pattern classifiers and perform relativel more accurate image semantics recognition than other eisting methods. Kewords: Feature etraction, fuzz reasoning machine, image preprocessing, semantics recognition, target detection INTRODUCTION As one kind of important information resources, image contains far more knowledge than words. Content-Based Image Retrieval (CBIR) has man advantages compared with other image retrieval method and becomes the mainstream of image retrieval technolog. However, CBIR can t support semantics retrieval due to it mainl adopts the bottom image visual features such as color, teture, shape, etc. Semantics-based image retrieval technolog is considered as a more ideal and valuable image retrieval method, therefore, how to identif image semantics well using the knowledge reasoning sstem has become a research hotspot. Image Semantics refers the meaning of the scene or behavior in image, which consists of targets and their relations. In order to get image semantics, man specific image processing and recognition problems must be solved such as image segmentation, reconstruction, edge detection, feature etraction, target detection, etc. In recent ears image processing and detection technolog have made great progress, such as iris identification (Boles and Boashah, 998), face recognition (Viola, 00) and so on. But for some more comple target identification, such as severe non-rigid target detection, is still has great difficulties (Li and Hu, 005). Therefore, research work of how to perform relativel precise semantics recognition based on inaccurate target detection is ver meaningful. Due to the uncertaint of eisting target detection technolog, detection result could be epressed as fuzz Fig. : Flowchart of Image semantics recognition sstem information, which the fuzz function could deal with. For better realization of image semantics recognition, this stud proposes a scene semantics identification sstem based on fuzz reasoning. The sstem consists of the following four parts: image preprocessing, feature etraction, target detection and fuzz reasoning machine. Figure gives the sstem framework of the presented scene semantics recognition. Compared with other methods, the outputs of target detectors are epressed b fuzz information, fuzz relationships between targets are etracted and fuzz reasoning is performed using fuzz automata. The simulation indicates that this method could overcome the problems of false positive and false negative of target detectors and perform relativel more accurate image semantics recognition than other eisted methods. Image preprocessing: The aim of image preprocessing is to perform better feature etraction and target detection. The image collected b record equipment includes not onl the targets, but also some noise. With the influence of light or other reasons, the image ma Corresponding Author: Weiliang Liu, Department of Automation, North China Electric Power Universit, Baoding 07003, China This work is licensed under a Creative Commons Attribution 4.0 International License (URL: 970

2 Res. J. App. Sci. Eng. Technol., 7(5): , 04 be vague, which makes the net feature etraction and eact matching difficult. Image preprocessing contains image noise reduction, image smooth, image enhancement, etc. Here eisting image processing technologies are adopted, such as the two-dimensional wavelet transform, Gabor filters, etc. FEATURE EXTRACTION The aim of feature etraction is to get a group of "fewer but better" classification feature. Because the generation of feature is associated with specific target, there lack of a unified feature etraction method which is suitable for various targets in theor. Therefore, for different target, different feature etraction methods are adopted here. For face, Haar wavelet etraction method is adopted (Viola, 00) For human bod, gradient directed graph characteristics (HOG) is adopted (Dalal and Triggs, 005; Dalal, 006); For some background such as grassland, sea, beach, etc, using color and teture features ma be a good choice. This section introduces the Gabor filter, which is widel used in target detections like fingerprint detection, iris recognition and so on. -D Gabor function can be epressed as: + + j( u+ v) δ δ g(, ) = e () πδ δ And its Fourier Transform is: where p (, ) is the input image of the channel, h e (, ), h o (, ) are respectivel the even Gabor filter and odd Gabor filter. Without losing of universalit, isotrop Gabor filter can be used (Tan, 995): (,,, θσ, ) (,, σ) cos π ( cosθ sinθ) (,,, θσ, ) = (,, σ) sin π ( cosθ + sinθ) h f = g f + h f g f e o where g(,, σ) is the Gaussian function: + σ g e (6) (,, σ) = (7) πσ f, Ө, σ in (6) and (7) are respectivel the important parameters of Gabor filter, i.e., spatial frequenc, phase and spatial constant. The Fourier Transform of even Gabor filter and odd Gabor filter in Eq. (6) are: and ( ω, ω, f, θσ, ) ( ω, ω, f, θσ, ) ( ω, ω,, θσ, ) + ( ω, ω,, θσ, ) H f H f He = H f H f ( ω, ω,, θσ, ) ( ω, ω,, θσ, ) H o = H( ω, ω, f, θ, σ) H ( ω, ω, f, θ, σ) j π σ ω cosθ + ω sinθ ( f ) ( f ) π σ ω + cosθ + ω + sinθ ( f ) ( f ) (8) (9) ( ω ω ) gˆ, u v π δ ω + δ ω π π () In practice, convolution is usuall performed using Fourier Transform, that is: The two-dimensional Gabor function is the result of translation of a two-dimensional Gaussian function at the two frequenc aes. The two-dimensional Gaussian function is a two-dimensional smoothness function, while the two-dimensional Gabor function is a two-dimensional band-pass filter. Performing convolution on Gabor function on g (, ) and -D teture image p (, ), that is: (, ) = (, ) (, ) (3) f g p Good result of image edges could be obtained. Usuall, the Gabor function is designed as: (, ) g π π + + j Assume the model of ever channel is: (, ) e(, ) o(, ) e(, ) = e(, ) (, ) (, ) (, ) (, ) q = q + q q h p qo = ho p (4) (5) 97 (, ) = (, ) (, ) = ( ω, ω) ( ω, ω,, θσ, ) (, ) = (, ) (, ) = ( ω, ω) ( ω, ω,, θσ, ) q e he p FFT P He f q o ho p FFT P Ho f (0) where,, =,. Ever pair of Gabor filters, h e (, ), h o (, ), correspond to given spatial frequenc and direction. The frequenc information and direction information could be etracted at the same time. As to the aim of teture recognition, it is unnecessar to choose the filter parameter space that covers the whole frequenc field (Tan, 995). The smaller the center frequenc is, the larger scale of etracted teture feature is. Usuall the frequenc which is the eponent of is chosen. For eample, in the fingerprints recognition algorithm, we choose, 4, 8, 6, 3, 64 as center frequenc respectivel. For ever center frequenc, four phases are selected: =,,. In this wa, there are 4 channels of Gabor filter. To the filter result of ever channel, the eception and deviation are etracted as its features. To ever input image, multicenter Gabor filter could etract 48 features in total.

3 Res. J. App. Sci. Eng. Technol., 7(5): , 04 TARGET RECOGNITION There are usuall two was to perform target recognition. One is multi-scale window detection means, which in turn etracts fied shape, different scale windows from the image and their features are sent into classifier to determine whether it is a target; The other is the segmentation means, which segments the image into different areas first, then determines the interest region, etracts the features, then classifier is used to determine whether it is a target. According to the scene semantics to be identified, the related interested targets are determined b epert sstem and then are etracted respectivel. The eisting target recognition classifier, such as support vector machine and neural network, often give an output value O i and then comparison is performed with fied threshold t i and the result is "either-or". It is difficult to select a threshold for comparison for this method is eas to cause the false positive and false negative. In order to enhance the follow-up processing robustness, here to the output of target recognition classifier is fuzzed, namel represent recognition results using membership degree. According to the characteristics of output decision value, the form of membership functions is as follows: Ai( ) o ( i t i) min(, e Ti + Tµ ), if oi t i 0 Tµ, else = () where, A i represents the first i kind target set, represents target, A i () represents the membership degree, T i is an constant, T u is the minimum membership degree. In order to save calculation time, it is necessar to avoid getting too much targets. For the first i kind targets, we onl keep at most N i targets that membership degree is bigger than T u, so the target recognition result is a target set: X A µ { ( ),,...,, 0 i i = ij i ij > T, j = N N N } FUZZY REASONING Scene semantics is sentenced b fuzz rule base of epert sstem. The inputs of fuzz reasoning machine are membership degrees and attributes of all targets and the output is the membership degrees to the current scene semantics. The form of fuzz rules is as follows: if ( A( ) is C) and( A( ) is C) and( f (, ) is C)then D ; if ( A( ) is C) and( A( ) is C) and( f (, ) is C)then D ; if ( A( ) is C) and( A( ) is C) and( f (, ) is C)then D3 ; where, i (i =,,, n) represents the detected first i kind target, f k ( i, j ) (i, j, k =,, n) represents the first k fuzz relation measure function of i and j and its scope is [0, ]; C k (k =,,, n) is fuzz set that represents the intensit of the relation and its scope is [0, ]; D k (k =,,, n) is fuzz set that represents the conclusion about scene semantics and its scope is [0, ]; For simplified calculation, the membership function of C k and D k is triangle function. According to the important degree of knowledge, the fuzz rules are given different credibilit, which helps launch reliable conclusion. When perform fuzz reasoning, take out a target and its attributes from each target set X i as the input of the automata. Here the former part matching of the rule is performed, if none target is detected, namel N = 0, then the related rules will not participate in the fuzz reasoning. Fuzz operation adopts Single-point fuzz method and operation adopts minimum calculation rule, implication operation adopts Mamdani-minimum calculation rule, snthetic operation adopts the maimum-minimum calculation rules and clearness operation adopts center of gravit method (COG). Suppose there are k target class and at most Ni targets is detected for first target class i, then there are output values at most, take a maimum of these output values as the ultimate decision value. EXPERIMENT and then attribute etraction is performed. Attribute etraction refers to etract the location, size and other information of the target. For multi-scale window detector, the center of the detected window is the target position and the length together with the width of the detected window represents the target size information. For segmentation detector, the center of the Minimum Boundar Rectangular (MBR) is the position of the target and the length together with the width of the MBR represents the target size information. These Attributes are etracted to perform the fuzz reasoning, such as analsis the spatial relations between targets. Each target class has an attribute set B i, = { j,,... n i}. B i = ij 97 Suppose the wanted scene semantics I is "Referee shows a red card in the football match", which is shown in Fig.. Through human eperience it could be easil known that in this scene: There must be the referee's handheld red card, in addition to a great degree the face or the human bod of referee or plaers will appear; Handheld red card is higher than face and is similar with face in size, etc. According to the eperience we can determine the interested target set and establish the fuzz rules and its credibilit. There are 3 interested target sets, i.e., handheld red card A, face A, stand human bod A 3.,, 3 represents ever kind of targets. C, C, C 3 represents that membership degree is weak, general, strong respectivel; D represents it is

4 Res. J. App. Sci. Eng. Technol., 7(5): , 04 Fig. : Referee shows a red card in the football match Fig. 3: Target detectors impossible that scene semantics is I, D represents it is possible that scene semantics is I, D 3 represents it is etremel possible that scene semantics is I. Three detectors are trained respectivel, i.e., handheld red card detector, face detector and human bod detector. Handheld red card detector takes the RGB averages of image sub-block as features, face detector takes the Haar wavelet characteristics as features and human bod detector adopts the HOG features. Three kinds of detector use multi-scale window measure method and the detection result is shown in Fig. 3. It is obviousl that false negative is eliminated b fuzzing the detection results. There is false positive in human bod detection and face detection and the detected position of human bod is not particularl accurate. Ninet images are collected as positive samples from internet that the image semantics is "Referee shows a red card in the football match" and negative samples are 000 images selected from INRIA database, the content of which include landscape, characters, sports, etc. Multilaer filter method is usuall adopted for identifing semantics of specific situations and achieved satisfied effect (Lijuan et al., 00; Yin et al., 004). A corresponding decision diagram for semantics recognition is established according to the thought of multilaer filter, which is shown in Fig. 4. The Precision and Recall are two universal criterions to evaluate the performance of recognition algorithm. Assume the images which actuall accord with the semantics is {Relevant}, the images which are sentenced to accord with the semantics b recognition sstem is {Retrieved} and the images which not onl actuall accord with the semantics but also are sentenced to accord with the semantics b recognition sstem is {Relevant} {Retrieved}, then Precision is: {Relevant} I {Retrieved} () Precision= {Retrieved} Fig. 4: Decision diagram for semantics recognition 973

5 Res. J. App. Sci. Eng. Technol., 7(5): , 04 target detectors are fuzzed, the fuzz relationships between targets are etracted and fuzz inference is performed using fuzz automata. The eperiment indicates that this method could overcome the problems of false positive and false negative of pattern classifiers and perform relativel more accurate image semantics recognition than other eisting methods. ACKNOWLEDGMENT Fig. 5: Comparison of two methods which indicates the recognition sstem s abilit to reject the negative samples. Meanwhile Recall is: {Relevant} {Retrieved} Recall= {Relevant} I (3) which indicates the recognition sstem s abilit to accept the positive samples. Higher Precision and Recall means the better performance of the recognition sstem. Eperiments of Comparison of the two methods are performed using MATLAB and OpenCV on a computer and Fig. 5 shows the evaluation curve of recognition effect. It is obviousl that the method presented b this stud is better than multi-level filter method on the whole. This is because when the threshold in multilaer filter is bigger, false negative happens, which causes a low recall. On the contrar, when the threshold in multilaer filter is smaller, false positive happens, which causes a low precision. The method in this stud fuzz the outputs of the target detector first, hence false negative is avoided; Then fuzz reasoning is performed to reduce the false positive, so better result is obtained. CONCLUSION In order to get better image semantics recognition, in this stud, a scene semantics recognition sstem based on fuzz reasoning is presented. The sstem contains four parts: image preprocessing, feature etraction, target recognition and fuzz reasoning machine. Compared with other methods, the outputs of This stud was supported b the Fundamental Research Funds for the Central Universities of North China Electric Power Universit and State Ke Laborator of Alternate Electrical Power Sstem with Renewable Energ Sources. REFERENCES Boles, W. and B. Boashah, 998. A human identification technique using images of the iris and wavelet transform. IEEE T. Signal Proces., 46(4): Dalal, N., 006. Finding people in images and videos. Ph. D. Thesis, Institute National Poltechnique de Grenoble. Dalal, N. and B. Triggs, 005. Histograms of oriented gradients for human detection. IEEE International Conference on Computer Vision and Pattern Recognition, pp: Li, F.Z. and K.H. Hu, 005. Proceeding of non-rigid motion analsis. J. Image Graph., 0(): -3. Lijuan, D., C. Guoqing, G. Wen and Z. Hongming, 00. A hierarchical method for nude image filtering. J. Comput. Aid. Design Comput. Graph., 4(5): Tan, T.N., 995. Teture edge detection b modeling visual cortical channels. Pattern Recogn., 8(9): Viola, P., 00. Rapid target detection using a boosted cascade of simple features. Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp: Yin, X.D., D. Tang, J. Deng, et al., 004. Contentbased method for nude image filtering. Comput. Automat. Measurement Control, (3):