MULTIRESOLUTION LOGO RECOGNITION

Transcription

1 MULTIRESOLUTION LOGO RECOGNITION M. CORVI Research & Development Department Elsag Bailey, un azienda di Finmeccanica S.p.A. Via Puccini 2, Genova, ITALY E. OTTAVIANI Research & Development Department Elsag Bailey, un azienda di Finmeccanica S.p.A. Via Puccini 2, Genova, ITALY A wavelet-based multiscale algorithm for the localisation and identification of logos on document images is presented. The algorithm relies only on the graphical appearance of the logo. The identification is measured by the correlation coefficient between the logo and its occurrence within the document and the localisation is achieved by a three-step procedure involving the images at different scales of resolution. The algorithm is robust against rotations and dilations of the pattern, and interference with other graphical elements. 1 Introduction Everyday documents like mail, advertisements, official forms, etc., usually have a logo that helps the reader to identify the source of the document. The logo is generally a very conspicuous pattern easily indetifiable within the document, and is often located in key points, such as the upper-left or the upper-right region of the page. A potential application field for the Logo Recognition Algorithm is the automated document handling, e.g., mail sorting (by class), bank check reading and invoices handling. The recognition of the logo provides an input for a high level analysis of the document structure (the layout) in the context of the new approaches to document image analysis 6-8. The logo pattern itself is an iconic feature that can therefore be used as a strong and reliable feature in a document-image database application, useful both for image indexing and for image retrieval. The use of the logo in database indexing has received little attention in spite of its potentialities 2-5.

2 Doermann, Rivlin and Weiss 3,4 address the logo based query by extracting the logo characteristic components: lines, circles, and text. These are organised in geometrically invariant features and are used for the recognition of the logo. In this work we assume that a logo is a graphical pattern with clear and simple geometrical features. The geometrical features can be lines, arcs, shaded areas, regular shapes, simple textures. Like Doermann et al. 3,4 Logo Recognition Algorithm strongly relies on the presence of such features. However, unlike the work of Doermann et al., in our approach the individual features are not extracted explicitly from the image. The logo is treated as a whole entity, and its recognition is based on the pattern matching of the picture itself rather than of features extracted from the picture. This global approach has the advantage of the independence of the recognition from the identification of the individual features, at the same time maintaining the computational burden to an acceptable level. The algorithm consists of an analysis of the images, followed by the localisation and identification of the logo pattern. The logo template image and the document test image are decomposed with a multiscale analysis based on a discrete wavelet transform (DWT) a' la Mallat 9 with biorthogonal spline wavelet of Cohen et al. 10, obtaining two sequences of residue-detail image pairs at different scales. The recognition of the pattern consists of two interdependent functions: the localisation and the identification. The localisation carries out a local registration of the logo image within the test image 11. The identification is measured by a function which evaluates the similarity between the logo image and the area of the document image where the logo has been localised. A reliable identification is important in indexing and retrieval applications for discriminating among several logos, and a good localisation is crucial for it. On the other hand the precise and accurate localisation is important for the definition of a reference co-ordinate system that can be utilised in automatic document handling applications. An estimate of the identification of the logo is necessary for the process of localisation. The localisation is described by the parameters of the mathematical transformation that maps the logo image inside the document image. This transformation is assumed to be composed of a translation, a rotation and a dilation. The recognition of the pattern achieves the localisation of the given logo in three steps, at the same time carrying out the identification. First a rough localisation of the logo is obtained based on the coarsest scale

3 residue images; next this localisation is improved by considering the detail images at an intermediate scale; lastly the localisation is refined in order to achieve a strong and robust identification. This paper is organised as follows: the next section describes the algorithm; the third section discusses the implementation and the results of numerical tests; the final section is devoted to the conclusions and further comments. 2 The Logo Recognition Algorithm The Logo Recognition Algorithm tackles a particular image registration problem consisting of the localisation of a small template image, the logo, in a larger test image. A registration algorithm 12 is characterised by four elements: the features used for the pattern matching; the class of transformations T allowed in the registration; the similarity function C measuring the quality of the matching; the search strategy used to find the best transformation. We suppose that the template image (pattern) I has size 2 N and the test image I' has size 2 M, where M>N. The class of transformations that relate the template image and its occurrence within the test image is made of the affine transformations obtained by composing a rotation R (by an angle t), a dilation D and a translation (by a vector a) T(x) = a + D R(x) (1) where x is a 2-dimensional vector representing a point in the template image, R(x) is the vector R(x) = (cos(t) x1 + sin(t) x2, -sin(t) x1 +cos(t) x2) (2) and D multiplies both vector components by the dilation factor d. These transformations form a 4-parameter subgroup of the group of affine transformations. This subgroup is sufficient for the applications to digitised documents in which the logo pattern is often rotated or dilated but is not distorted.

4 We further assume that there are no radiometric changes between the template image and its occurrence in the test image. However slight differences between the two images are allowed, as the logo might be drawn in the test image with a style different from that in the template image, and the digitising conditions maybe different. Therefore we assume that the two images I(x) and I'(T(x)) are similar, for x in the domain of the image I. The logo recognition algorithm aims to find the transformation T that maximises the similarity function of the pattern matching. For this function we use the cross-correlation, C( I, I'; T) = [ I( x) u ][ I' ( T( x)) ut ] 0 where u0 and S0 are the mean and the standard deviation of the template image I, and ut and ST are the mean and the standard deviation of the test image I', restricted to the range of T (i.e. to the region where the logo had been localised). The features used in the Logo Recognition Algorithm are obtained from a multiresolution analysis of the two images. Both the template image and the test image are transformed with a discrete wavelet transformation (DWT) a' la Mallat 9. This is a reiterated transformation, each iteration of which produces four images obtained by low-pass and high-pass filtering (in the X and Y directions) the generic input image denoted J, and decimating the filter outcomes: S 0 S T JHH = HX HY J (3) JHG = HX GY J (4) JGH = GX HY J JGG = GX GY J where H and G are the low-pass and high-pass filters, respectively. The subscripts denote the direction along which the filter is applied. The filter coefficients are deduced from the theory of biorthogonal wavelets applied to the cardinal splines functions. If the generic image J has size 2 P the 4 transformed images have size 2 P-1. The transformed image JHH is the residue image at scale level K=N-(P-1), where N is the log2 size of the original image. It will be denoted JK, and it is the input for the next iteration step. The other three images are the detail images at this scale. In particular J0 will denote the

5 original image, and is the input for the first iteration. The DWT of the template image and the test images is repeated up to the scale L. The search strategy is the most important component of the algorithm. It consists of three phases and attempts to find the best registration 11 of the template image inside the test image. The first phase achieves a rough estimate of the translation vector a by maximising the function C(IL, I'L;a), with respect to a. Here IL and I'L are the residue images at the coarsest scale L. This phase is crucial for the successful logo recognition, and relies heavily on the logo being a conspicuous pattern in the document. The translation parameter a is estimated by maximising, at scale L, the correlation between the logo image and an equal-size image positioned at a in the test image. This exhaustive procedure is fast since it is carried out on images of small size, and the set of possible values of a is limited. In the second phase of the search strategy the algorithm improves the translation parameter a while estimating the rotation angle t and the dilation factor d. It is assumed that the logos contain simple geometric features such as lines, arcs, and geometrical shapes. Therefore, the algorithm tries to correlate a sort of "edge images" of the template image and of the portion of the test image where the logo has been roughly localised in the first phase. As "edge images" we use the gradient images of the residues at the scale L1. For each edge image E the centre of mass c is obtained, x E( x) c =. (5) E( x) Next the radial density r, relative to c, is computed, r = 2 x c E( x) E( x) where x denotes the image point with co-ordinates. Then the radial distribution of edges is obtained, 1/2 2 θ( t) = x c E( x) K where the sum runs over the points x = (c1 + cos(t) k, c2 + sin(t) k), k being a positive integer, and t is the angle variable. By maximising the cross-correlation of the two radial edge distributions e(t) and e'(t) for the template image and the test image, (6) (7)

6 respectively, we obtain the rotation angle t that relates the logos in the two images. The two functions are evaluated in 256 points and the correlation is calculated with a FFT 12 (fast Fourier transform). From the ratio of the two radial densities the dilation factor is estimated, d = r / r' (8) where r is the radial density of the template edge image, and r' is that of the test edge image. Finally the translation parameters are adjusted to the new estimates: anew = aold + c' - d Rt(c) (9) where c is the centre of mass of the logo edge image, and c' is that of the test image. The last phase of the search strategy is devoted to the fine tuning of the localisation transformation T. The transformation parameters are improved by maximising the cross correlation between residue images from the scale L2 down to the scale L3. At each scale the parameters are adjusted with the Newton interpolation method 12. For each parameter p of the transformation group, three correlations are evaluated at values p- p, p, and p+ p. The maximum of the correlation and the corresponding value of the parameter are computed by quadratic interpolation. This procedure is repeated until for each parameter the maximum lies within a distance p/2 from p. The variation p must be chosen large enough to avoid too many repetition of the procedure, but cannot be too large otherwise the regularity of the correlation function is missed. We found that a=1 pixel (for each component), d=0.05 (a dilation of 5%), and t=0.05 rads (corresponding to about 3 degrees) are reasonable values. 3 Numerical Implementation and Results The Logo Recognition Algorithm has been implemented in C on a Digital Alpha workstation 200 at 100 MHz. The logo template images have size 128x128. This size is large enough to contain most of the logos extracted from digital images of documents scanned at 160 dpi. The test images have size 512x512.

7 The DWT 9 was carried out up to the scale L=4. The first phase used the residue images at this scale. The second used detail images at scale L1=2. The third used residue images at scales from L2=3 down to L3=1 included. The execution gives the following CPU times: DWT 0.22 sec 1-st phase 0.02 sec 2-nd phase 0.04 sec 3-rd phase 0.36 sec Tab.1 CPU times The two most computing intensive parts of the algorithm are the image analysis by the DWT, and the final search phase. Overall the Logo Recognition Algorithm takes about 0.6 seconds to verify the presence of the given logo in a document image. The numerical test was carried out on a set of US mail pieces, and aimed to retrieve a stamp with the "dove" (Fig.1). The test set consisted of 35 images of letters (see Fig.2 for an example). The "dove" stamp appeared in six images, all of which with another stamp as well. The other images contained different stamps. Fig.1 The "dove" The frame in Fig.2 represents the region where the logo has been localised. The results of the Logo Recognition Algorithm on the six images with the dove stamp are reported in the table 2 below. For a comparison, the highest values of the similarity for the images without the "dove" stamp are C1=0.31, C2=0.52, and C3=0.54. It is evident that the algorithm can clearly discriminate the presence of the stamp from the absence. These results too show that the values of the similarity after the first and second phase are not sufficient for such a discrimination, and that the third phase is necessary for a robust pattern identification.

8 Fig.2 A part of a typical US mail Image C1 C2 C3 dove dove dove dove dove dove Tab.2 Results of the algorithm

9 4 Conclusions We have presented a logo recognition algorithm based on a multiscale image decomposition obtained through a discrete wavelet transform. The algorithm is very robust against arbitrary rotations and dilations of the pattern. It is also successful when the pattern is perturbed by some interference with other patterns. The dependence on the wavelet filter choice is not very strong. We have tested the algorithm with different choices of biorthogonal filter derived from splines 10. The biorthogonal spline filters are particularly interesting for numerical implementations since they have a finite number of coefficients, that are furthermore multiples of dyadic fractions. For applications to document images, low order splines appeared more appropriate, but the results did not significantly favour a particular spline. It is a three-phases procedure based on the DWT-transformed images. The first phase of the localisation is essential for the success of the recognition. The crucial point is to apply a low-pass filter to the images in order to obtain "thumbnail" copies of them for the first phase, and to use low and high passed versions richer in information for the other phases. The most important part of the algorithm is the search strategy and the features used for it. In specific applications it is possible that this first localisation is enhanced by restricting the search focus using a-priori knowledge about the position of the logo. The second phase attempts to estimate the rotation angle and the dilation factor using the distribution of the linear elements of the images. This phase requires the logo to be composed by simple geometrical linear features, and can be hampered when the logo strongly interferes with other graphical patterns. The third phase aims to maximise the similarity function between the logo and the region of the document image where it has been localised. Many functions resulted successful, but the normalised cross correlation seems to be the best. A future development of the algorithm might take into account also the other choices for the translation parameter a, resulting from the first phase of the localisation. The algorithm would analyse all the relevant choices locally and select the best only after the second phase. As the first two phases of the localisation do not cost much CPU, the time performances should not be impaired, while making the algorithm even more robust.

10 5 References [1] UMD Logo Database [2] F. Kanehara, S. Satoh, and T. Hamada, A flexible image retrieval using explicit visual instructions, Int. Conf. Docum. Analysis and Recogn., Montreal 1995, p [3] D. Doermann, E. Rivlin et al., Logo recognition, University of Maryland Report, CAR-TR-688, CS-TR-3145, Oct [4] D. Doermann, E. Rivlin, and I. Weiss, Applying algebraic and differential invariants for logo recognition, Machine Vision and Applic. Vol.9 (1996), p.73. [5] M. Corvi, Wavelet based logo recognition, SIMAI Int.Conf [6] A. Dengel, and A.L. Spitz Proc. Document Analysis Systems, Kaiserslautern, Germany, [7] O'Gorman, and R. Kasturi (eds.) Document Image Analysis, IEEE Comp. Soc. Press [8] R. Casey, D. Ferguson, K. Mohiuddin, and E. Walach, Intelligent Form Processing System, in Document Image Analysis, IEEE Comp. Soc. Press, p [9] S.G. Mallat, A theory for multiresolution signal decomposition: the wavelet representation, IEEE Trans. PAMI, 11 (1989) 674. [10] A. Cohen, I. Daubechies, et al., Biorthogonal bases of compactly supported wavelets, Comm. Pure Appl. Math. 45 (1992) 485. [11] M. Corvi, G. Nicchiotti, Multiresolution image registration, Int. Conf. Image Process., Washington DC,1995. [12] L.G. Brown, A survey of image registration techniques, ACM Comp. Surveys, 24 (1992) 325.