Searching Image Databases Containing Trademarks

Transcription

1 Searching Image Databases Containing Trademarks Sujeewa Alwis and Jim Austin Department of Computer Science University of York York, YO10 5DD, UK and October 5, Introduction Trademarks play an important role in providing unique identity for products and services in the marketing environment and trademark classification systems should be able to ensure that the existing trademarks are distinct to avoid confusions. Traditionally, classification of trademarks is based on limited vocabulary descriptions. Most of the patent offices use manually assigned codes to represent these descriptions such as human beings, animals, geometrical figures. But, it has been shown that all these methods suffer many problems including: the assignment of classes to trademarks is subjective, the classes become either too specific or too broad depending on how users use the classes. there is no mechanism to handle the generation of new classes, there is a large fraction of images with little or no representational meaning which makes such a classification extremely difficult. This motivates the need to investigate the potential of content based image retrieval techniques in solving this problem. Retrieval of images by shape feature is still an unsolved problem. though there has been considerable research into this topic [1.2,3]. Defining a shape similarity criteria which can reflect human reasoning on visual perception can be seen as a challenging problem. There is evidence which shows that that many of the existing shape similarity assessment methods fail in identifying some of the images perceived as similar by humans [4.5]. This evidence motivates the effort in using concepts from visual cognitive psychology for investigating new shape retrieval systems. Trademark image retrieval provides a good avenue of investigation in this regard since an effective trademark retrieval system should necessarily be able to retrieve images which humans perceive as similar. In this study, we investigate a new trademark image retrieval system based on features extracted using Gestalt feature extraction methods. During retrieval, we utilize alternative feature interpretations in four different modules. To obtain the final similarity scores, we combine evidence from these modules. Though this framework may be able to capture perceptual similarity of trademark images, the high computational requirements creates the need for an efficient and low cost computational platform. There have been numerous attempts to solve a range of problems using neural networks. However, many of the neural network architectures suffer from long training time and inefficient hardware implementation. -Associative memory architectures perform better than many other methods in this respect. Pattern matching capabilities offered by correlation matrix memory networks( CMM) [6] under the framework of AURA [7] provide a number of features that could be exploited to obtain an efficient search engine for the proposed system. Apart from its fast and low cost hardware implementation of the network, it offers the ability to parallelise the search mechanism by presenting input patterns and obtaining output pattern simultaneously. With this integration we hope to use findings from visual cognitive psychology under the neural network framework, in an attempt to integrate advantages from both fronts. The rest of this paper is organised as follows: section 1 and section 2 describe the feature extraction and similarity assessment processes respectively. Combination of the retrieval modules is described in section 3 while section 4 describes the CMM based implementation of the system. 2/ 1

2 2 Feature Extraction Feature extraction process goes through several steps starting from edge extraction and decomposition, as shown in figure 1. Local feature vector of the image consists of length and orientation of lines and curvature of curves. Gestalt feature extraction methods proposed by Sarkar and Boyer [8], and Lowe [9] are used in extracting the following perceptual relationships; end-point proximity, parallelism, cc-linearism and co-curvilinearism. We extract gestalt features on segment level rather than on boundary level as performed in.artisan [lo]. Extraction of closure is based on the end-point proximity relationships. This method extracts alternative interpretations of closed figures which may not be able to obtain using standard pisel based linking methods. In the next step, we extract features of closed figures namely; circularity, directionality, straightness, complexity, right-angleness, aspect-ratio, sharpness, stuffedness. The co-linear and co-curvilinear relationships allow to obtain a new grouped interpretation of the same image which is again subjected to the above process of feature extraction. -~ 1 edge exmcuon contour decomposrrion, local perceptual feature exmction from the raw image i.- from the raw image preparation of the Gestalt image I 1 4 pxceptual feature exusction from the Gestalt image I from the Geswlt image Figure 1: The overview of the feature extraction phase. Figure 2: Figure 2.b shows the co-linear and co-curvilinear segments while figure 2.c parallel segments extracted using the image in figure 2.a. 212

3 Figure 3: Some of the closed figures extracted using the image in figure 3.a. Figure 4: Figures 4.b and 4.d show the grouped images obtained using images in figures 4.a and 4.c respectively. 3 Similarity assessment In this phase? we use local features as well as the features of closed figures of both original and grouped images: in separate modules. 3.1 Using local features Similarity assessment using local features is based on graph representations of the image in which nodes represent the segments of the image and arcs represent different perceptual relationships between the segments. During this process query graph is compared against all the model graphs in the database. Initial matching possibilities between nodes of the query graph and the model graphs are obtained using local features of the segments. Elimination of unplausible candidates at the nodes is performed under the relaxation by elimination framework [ll] using upper bound probability estimations from perceptual neighbours. 3.2 Using features of closed figures Similarity assessment between query image and the model image is performed by considering the feature vectors of each closed figure to obtain candidate matching possibilities for each query figure. This can be performed in two different ways; either in a symbolic fashion by discretising the feature components or calculating distance measures between feature vectors of the corresponding query figure and the model figure. In the next step upper bound support for each matching possibility is obtained from the contextual neighbourhood. 4 Combination of retrieval modules We have investigated different strategies for combining evidence from retrieval modules and the overall effect of the process. Rank positions obtained for each image from different modules can be used to 213

4 evidence from '. "'0 step 1: candidate matches from model graphs step 2: optimization of evidence using constraint propogation under relaxation by elimination framework ( 4 Figure 5: Local features based similarity assessment framework. obtain minimum, maximum or mean ranking or sum of the reciprocal values [12]. New rank order can be obtained as the outcome of each process. Alternatively, similarity scores for each image can be combined in a probabilistic framework using the Dempster-Shafer mechanism [13]. It has been observed that average combined results are better than average results delivered by any individual retrieval module CMM based implementation The AURA system is aimed at fast combinatorial searching and high performance knowledge base system design. The unique feature of ALRA has been its partial matching capability which would be essential to enable a system to deal with real world problems. The input-output relationship to the CMM which is the basic building block of AURA, has the form of pre - condition- > post - condition The system is trained with input-output associations inter-related by rules or predicates. 5.1 Training the system Training phase is aimed at storing the feature information in such a way that it will provide an efficient retrieval mechanism Local perceptual features based modules Three different CMMs are used to store end-point proximity, parallelism and cc-linearity or co-curvilinearity associations. The input and output patterns in the associations comprise of tags which represent nodes in the graph and in training, unique patterns are generated to represent them Retrieval using features of closed figures In training the network, CMM stores relationships between the input patterns which represent the feature vector and output patterns which represent closed figure identification numbers. The training set consists of a number of associations equal to the total number of closed figures in the database. 214

5 6 Results We have completed the first phase of the evaluation of the system using similarity judgement data from trademark examiners. The second phase is aimed at using similarity judgement data from a set of human subjects. The table 1 summarises average normalized recall (R,) and precision (P,) measures [14] obtained for ten queries while the table 2 summarises results obtained by combining the retrieval modules as explained in section 4. Retrieval module I R, 11 P, Local features (raw image) - I I Features of closed figures (raw image) Local features (gestalt image) Features of closed figures (gestalt image) Table 1: Effectiveness of the retrieval modules. 1 Method of combination I R, I1 P, 1 Mean rank 1 Dempster-Shafer mechanism I J Table 2: Comparison of performance of different combination strategies. References [l] s. Sclaroff and A. Pentland. Object recognition and categorization using model matching. In Proceedings of the 2nd CAD Based Vision Workshop, pages > [2] W. I. Grosky and 2. W. Jiang. A hierarchical approach to feature indexing. Image and Vision Computing, vol. 5, pages , [3] R. Mehrotra and J. E. Gray. Similar-shape retrieval in shape data management. IEEE Computer vol 28, no 9, pages 57-62, [1] D. Mumford. Mathematical theories of shape: Do they model perception. Geometric methods in computer vision-spie, vol 2185 pages 2-10, [3] B. Scassellati. Retrieving images by 2-D shape: a comparison of computation methods with human perceptual judgements. Proceedings SPIE, vol 1285, [6] M. Turner and J. Austin. Matching Performance of Binary Correlation Matrix Xemories. Neural Networks, Elsevier Science, [7] J. Austin, J. Kennedy, K. Lees The Advanced Uncertain Reasoning Xrhitecture. In Proceedings of the Artificial Neural Networks and Ezpert Systems Conference? June 1995 [SI S. Sarkar and K. Boyer. Computing perceptual organization in computer vision. World Scientific Publishers, [9] D. G. Lowe. Three-dimensional object recognition from single two-dimensional images. Artificial Intelligence, vol 31, pages , [lo] J. P. Eakins, bl. E. Graham, J. M. Boardman et al. Retrieval of trademark images by shape feature. British Library Research and Innovation Report 26, [ll] M. Turner and J. Austin. A neural relaxation technique for chemical graph matching. In Proceedings of the Fifih International Conference on Artificial Neural Networks, Cambridge, UK, Editor. M Xranjan, IEE Publishers, July [12] S. Alwis and J. Austin. A novel architecture for trademark image retrieval systems. In Proceedings of the challenge of image retrieval, [13] J. M. Jose, D. J. Harper. A retrieval mechanism for semi-strucmred photographic collections. Lecture notes in computer science 1308, pages , [la] C. J. Van Rijsbergen. Information retrieval. Butterwoths, London, The Institution of Electrical Engineers. Printed and published by the IEE, Savoy Place, London WC2R OBL, UK. 21.5