2. METHODOLOGY 10% (, ) are the ath connected (, ) where

Size: px

Start display at page:

Download "2. METHODOLOGY 10% (, ) are the ath connected (, ) where"

Tyrone Powell
5 years ago
Views:

1 Proceedings of the IIEEJ Image Electronics and Visual Computing Workshop 01 Kuching Malaysia November UNSUPERVISED TRADEMARK IMAGE RETRIEVAL IN SOCCER TELECAST USING WAVELET ENERGY S. K. Ong W. K. Lim Faculty of Engineering Multimedia University Cyberjaya Malaysia. ABSTRACT Trademark is a distinctive symbol used by businesses to distinguish its products or services from other similar entries. Trademark plays a vital role in the growth of business as it represents the reputation and credit standing of the owner. Many methods have been proposed to identify classify and retrieve trademarks. However most methods need query images being processed and features stored prior to the recognition and retrieval process. In this paper an unsupervised content-based trademark image retrieval system is proposed by query database information and human monitoring process are excluded. By given an input soccer telecast image the regions of interest are first localized followed by feature analysis and finally trademark is extracted. With the emphasis on utilizing closed region filling method with implementation of inverting and non-inverting binary maps and incorporated with regional wavelet energy information certain categories of trademarks are successfully localized and retrieved. 1. INTRODUCTION Trademark represents the reputation of a product s quality. It is able to convey information of the producer s reliability while widening its popularity. With the ever increasing number of registered trademarks to the extent of more than three million worldwide manual human search approach is no longer practical [1]. Since the advancement of computer researchers have paid considerable attention to contentbased image retrieval (CBIR). In the 90 s numerous query systems had being proposed in trademark image retrieval and recognition. In ARTISAN a shape-based trademark retrieval system was developed by utilizing the idea of multi-component matching []; Lam et al. used Fourier descriptors and moment invariants as features for image retrieval [3]; Jain and Vailaya used edge direction histogram to retrieve in large trademark image database [4]; Gao et al. presented a two-stage logo detection using spatial-spectral saliency (SSS) and partial spatial context (PSC) [5]. Later wavelet was brought into trademark image recognition. Hesson and Androutsos proposed a 5-dimensional color wavelet cooccurrence histogram to retrieve similar trademark images [6]; Jian and Xu used wavelet-based edge detector and shape descriptor in the process of retrieving the trademark images [7]; Kekre et al. implemented Approximation Haar wavelet pyramid to do image detection [8]. Query system requires database storage query images to be preprocessed before detection and only capable to retrieve limited images depending on data library. Unsupervised retrieval is one of the solutions to encounter the issue listed above. Soccer telecast is chosen because it is one of the most popular sports in the world. Trademark images appeared in telecast has high influence to the business market too. Furthermore trademark appearance in different scales orientations as well as motion blurred and occluded scenarios makes retrieval processes challenging. Recently Watve and Sural introduced automated DABA algorithm for soccer video; by classify the input frames to long shot and closed-up shot combined with 'Hue Slicing' algorithm and edge detection for template matching [9]. Bagdanov et al. proposed a compact representation of trademarks and video frame content based on SIFT feature points for detection and retrieval of trademarks that appear in sports video [10]. Kuo et al. implemented a gradient based average approach for broadcast video logo detection which focus on opaque transparent and animated logo type and remove it using spatial-temporal image restoration technique [11]. Despite advancement mentioned above the existing findings still suffer limitations. Most of the research methods require appropriate images size and format prior to the detection and retrieval. The systems in the literature focused mainly on textual or wording based trademarks which fail to address trademarks formed by shapes and other trademark images complexity. Furthermore they also do not emphasize on the regional information of the trademark images. Here we proposed an unsupervised trademark image retrieval system. The challenges are: trademark information prior to the detection and retrieval remained unknown compared to existing query image system. Trademark image retrieval without reference image used to get low precision rate. However unsupervised trademark image retrieval could significantly reduce the data storage therefore helps in reducing retrieval time. It is also able to detect trademark images not in the database which improve trademark searching ability.

. METHODOLOGY To address the issues faced while preserving the accuracy and efficiency in trademark image detection and retrieval works we propose an unsupervised content-based trademark image

The processes are region of interest localization followed by features extraction and lastly trademark is detected and retrieved from the input image.

The general framework of the proposed method is shown in figure 1. small portion of the total sports scenes yet it shall be formed by certain amount of pixels in order to be visible.

2 . METHODOLOGY To address the issues faced while preserving the accuracy and efficiency in trademark image detection and retrieval works we propose an unsupervised content-based trademark image retrieval method based on wavelet approach. Once a given image is loaded it underwent three major processes. The processes are region of interest localization followed by features extraction and lastly trademark is detected and retrieved from the input image. For this paper input images were extracted from soccer telecasts with the frame resolution of 640-by-368 pixels. The region of interest consist samples from -by-11 pixels to 300-by-81 pixels. The general framework of the proposed method is shown in figure 1. small portion of the total sports scenes yet it shall be formed by certain amount of pixels in order to be visible. A mapping to the input image is performed to extract the trademark ROIs for further processing. 0% < ℎ 10% are the ath connected region under connected component analysis of inverting binary map and bth connected region under connected component analysis of non-inverting binary maps. Resultant ath and bth connected regions are then. represented by general notation with is the label area percentage of respect to the total pixels of input frame... Candidate label features extraction Figure 1: General framework of proposed method.1. Region of interest (ROI) detection In this section we focus on the preliminary stage in detecting the regions of interest for further processing. The image is first sharpened and grey scaled. Image sharpening is used for weak edges exploration. The enclosed sections of the image are then filled up and converted to binary. The enclosed regions filling is implemented due to trademark images usually formed in enclosed areas. Next inverting and non-inverting binary maps are created; with both undergo morphology closing process. Inverting binary map and non-inverting binary map extraction are as follows: 1 are the inverting binary map the non-inverting binary map and the thresholded enclosed filling input image respectively. The output maps are fed for connected component analysis by labels area thresholding determines the regions of interest (ROIs) location and those falling out of range will be filtered. The label area threshold value shall be in between 0% to 10% of the total pixels. Trademark regions usually occupy only Wavelet transform is used for trademark features extraction. Wavelet is a short wavelike mathematical function which its transform allow data to be processed under different frequencies and resolutions to get a desired output. Moreover wavelet transform is compactly support and shows good detection on discontinuities or sharp spikes. The wavelet representation of a two-dimensional function is as follows: Ψ Ψ ( ) ( ) Ψ ( ) ( ) and Ψ ( ) ( ) Ψ ( ) ( ) 1 x n y m Ψ is the scaling function and is the mother wavelet. There are some characteristics that differentiate trademark images from natural images. Trademark images are designed to have visual impact containing multiple homogeneous elements and purely abstract patterns. So trademarks usually have clear color transition blocks sharper boundaries and higher complexity around its region. In this paper we proposed to use wavelet energy as the main tool to extract trademark features. This is because wavelet energy is sensitive to changes between a pixel and the surrounding. The wavelet energy is defined to be Ψ

3 is the coefficient of wavelet transformed image. Wavelet decomposition analysis is applied to the extracted trademark ROIs image resulting 4 elements namely approximation coefficients horizontal vertical and diagonal details coefficients. For better extraction of trademark features we proposed the localized wavelet energy ( ) By using the approximation wavelet coefficients the extracted images are split to pixels overlapping sub-windows for wavelet energy calculation. After energy value comparison with preset threshold on the approximation element of wavelet transform a rectangular boundary is generated for regions with high wavelet energy loss value and the area cropped as a candidate label. All localized candidate labels shall considered potential trademark images. All of them are finalized and served as the output of trademark identification. Last each finalized label boundary location is mapped back to the input image; with trademark regions are being highlighted. cropped randomly from input images which consists of 31 clear and visible trademarks 10 blurred or occluded trademarks and 30 non-trademarks with different image resolution. Threshold percentage is set to 1.5%. According to samples calculation 54 out of 71 samples are correctly justified. Only samples of trademarks are not detectable. Table 3 is made by contingency examination as shown in table 1 [1]: Retrieved Not retrieved Trademark Non Trademark True Positive (TP) False Positive (FP) True Negative (TN) False Negative (FN) Table 1: Contingency examination Next precision and recall for retrieval result were calculated. Precision rate defined as the rate of retrieved result that is trademarks; while recall rate known as the rate of trademarks in total retrieval. Table 4 summarizes the overall precision and recall rate for the algorithm. Based on Table 3 tabulation precision and recall percentage found to be 71.07% and 95.00%. The samples and its corresponding overlapping wavelet energy plots are displayed in figure 4 and 5. The equations of precision and recall computation are as follows: 3. RESULT AND DISCUSSION The research work uses MATLAB as the platform to develop the algorithm. This preliminary work only considers Haar wavelet with levels of decomposition. Input image samples are from random screen shots of soccer telecast containing long shot and closed-up shot scenes as well as non-playing field scenes. Examples are shown in figure. Figure 3 displays the output of good moderate and poor region of interest detection. Good detection successfully detected and localized above 80% of the trademarks appeared in the input image. Moderate detection secured 50% to 80% of the trademarks as poor detection has only less than 50% of the total trademark successfully found. The region of interest is then saved for further processing. Table shows the statistics of human vision detected trademarks compared to region of interest detection using the research algorithm. As shown in the table there are 1 frames which the trademarks in the frames are all detected; 3 frames with above 80% trademark detection; 5 frames with above 50% trademark detection and 9 frames with less than 50% trademark detection. So 0 out of 9 frames detected more than half of the trademarks on a given image. This statistics display a constructive result as about 70% of trademarks on the input images are well localized. Nondetectable trademarks are mostly blurred occluded or too small region for the algorithm to recognize its existence. Table 3 shows the overlapping sub-window wavelet energy loss percentage record. Total of 71 samples were + + TP FP and FN are number of true positive images number of false positive images and number of false negative images respectively. Figure 6 shows the wavelet energy plot against extracted ROIs of input image. Experiment result shows that wavelet energy value around trademark regions are generally higher than non-trademark areas thus justified the wavelet energy study precision. This shows that wavelet energy is suitable as one of the feature vectors to recognize color transition and content complexity which served as the nature of trademark images. 4. CONCLUSION In this paper we proposed an unsupervised contentbased trademark image retrieval algorithm on sports scenes using wavelet energy. We showed that wavelet energy approach has returned a constructive precision rate which enable us to locate trademark images on a given sports scene without human supervision monitoring or input information required. In future work we will combine color; texture and shape feature integration incorporating geometry features for better retrieval recognition and classification of trademark images.

4 (c) (d) Figure 4: Sample images for wavelet energy study. 3 (c) (d) are the true positive images true negative images false positive images and false negative images. Figure : Some examples of input image. (c) (d) (e) (f) Figure 3: Corresponding ROI detection. (d) (f) are the good moderate and poor ROI detection. Detected Trademark In A Given Soccer Telecast Image # of Frames Trademark detection 100% 1 80% Trademark detection < 100% 3 50% Trademark detection < 80% 5 Trademark detection < 50% 9 Total 9 Figure 5: Overlapping 4-by-4 sub-window wavelet energy plot. 4 are overlapping wavelet energy plot of 3(i) and (ii). Table : Trademark localization result Wavelet Energy Study # of Samples True Positive 38 True Negative False Positive 15 False Negative 16 Total 71 Table 3: Trademark wavelet energy retrieval result Precision rate 71.07% Recall rate 95.00% Table 4: Overall precision and recall rate Figure 6: 4-by-4 Sub-window wavelet energy plots from detected ROI. Figure 5 and 5 are plotted from figure and (c).

5 5. REFERENCES [1] Trend in Total Trademark Applications (011) [online] available: [] J. P. Eakins J. M. Boardman K. Shields Retrieval of Trademark Images by Shape feature - The ARTISAN Project Department of Computing University of Northumbria at Newcastle Newcastle upon Tyne NE1 8ST United Kingdom [3] C. P. Lam J. K. Wu B. Mehtre STAR A System for Trademark Archival and Retrieval Proceeding of nd Asian Conference on Computer Vision Singapore Vol. 111 pp [4] A. K. Jain and A. Vailaya Image Retrieval using Color and Shape Pattern Recognition 9(8) pp [5] K. Gao S. Lin Y. Zhang S. Tang D. Zhang Logo Detection based on Spatial-Spectral Saliency and Partial Spatial Context IEEE International Conference on Multimedia and Expo pp [6] A. Hesson D. Androutsos Indexing and Retrieval of Compound Color Objects using Co-occurrence Histograms of Color and Wavelet Features IEEE International Conference on Image Processing (ICIP) pp [7] M. Jian L. Xu Trademark Image Retrieval Using Wavelet-based Shape Features International Symposium on Intelligent Information Technology Application Workshops (IITA) pp [8] H.B. Kekre S. D. Thepade A. Maloo Query by Image Content using Color-Texture Features Extracted from Haar Wavelet Pyramid IJCA Special Issue on Computer Aided Soft Computing Technologies for Imaging and Biomedical Applications (CASCT) pp [9] A. Watve S. Sural Soccer video processing for the detection of advertisement billboards Pattern Recognition Letters Volume 9 Issue 7 pp [10] A. D. Bagdanov L. Ballan M. Bertini and A. D. Bimbo Trademark matching and retrieval in sports video databases In Proceedings of the international workshop on Workshop on multimedia information retrieval (MIR '07). ACM New York NY USA pp [11] C.-M. Kuo C.-P. Chao W.-H. Chang J.-L. Shen ``Broadcast Video Logo Detection and Removing'' Intelligent Information Hiding and Multimedia Signal Processing 008. IIHMSP '08 International Conference pp Aug [1] C. D. Manning P. Raghavan and H. Schutze An Introduction to Information Retrieval Cambridge University Press Cambridge England 009.

Trademark Matching and Retrieval in Sport Video Databases

Trademark Matching and Retrieval in Sport Video Databases Andrew D. Bagdanov, Lamberto Ballan, Marco Bertini and Alberto Del Bimbo {bagdanov, ballan, bertini, delbimbo}@dsi.unifi.it 9th ACM SIGMM International