Fast Image Matching on Web Pages

Transcription

1 Fast Image Matching on Web Pages HAZEM M. EL-BAKRY Faculty of Computer Science & Information Systems, Mansoura University, EGYPT NIKOS MASTORAKIS Technical University of Sofia, BULGARIA Abstract In this paper, a fast method for image matching on web pages is presented. Such method relies on performing cross correlation in the frequency domain between the web image and the image given in the user query. The cross correlation operation is modified. Instead of performing dot multiplication in the frequency domain, image subtraction is applied in two dimensions. It is proved mathematically that the number of computation steps required for the proposed fast matching method is less than that needed by conventional matching. Keywords Fast image subtraction, frequency domain, cross correlation, image matching. I. Strategy of Fast Image Matching by Applying Cross Correlation in the Frequency Domain First an image is given in the user query. Then, each sub-image in the web image is tested for the presence or absence of the image in the given query. At each pixel position in the web image, each sub-image is tested for matching with the image in the given query. Each sub-image in the web image is subtracted in two dimensions from the given image in the query, which has the same size as the subimage. This subtraction is done in the spatial domain. When the final result is zero this means that the subimage under test contains the required image (given the user query) and vice versa. Thus, we may conclude that this searching problem is cross correlation in the spatial domain between the web image under test and the image given in the user query. In this section, a fast algorithm for image matching based on applying two dimensional cross correlations between the tested web image and the image given in the user query is described. The image given in the user query can be represented as a sliding window moving pixel by pixel on the web image. The convolution theorem in mathematical analysis says that a convolution of f with h is identical to the result of the following steps: let F and H be the results of the Fourier transformation of f and h in the frequency domain. Multiply F and H in the frequency domain point by point and then transform this product into spatial domain via the inverse Fourier transform [1]. As a result, these cross correlations can be represented by a product in the frequency domain. Thus, by using cross correlation in the frequency domain a speed up in an order of magnitude can be achieved during the detection process [1-35]. Here, the theory of cross correlation is modified. To achieve the best matching, instead of dot multiplication, matrix subtraction is performed. In the matching phase, a sub-image X of size mxz (sliding window) is extracted from the tested web image, which has a size PxT. Let W be the two dimensional image given in the user query. This image has a size of mxz and can be represented as mxz matrix. The output of this matching h can be calculated as follows: z m h = X(j,k)- W (j,k) (1) j= 1k = 1 Eq.(1) represents the result for a particular sub-image X. It can be computed for the whole image Ψ as follows: m/2 z/2 h(u, v) = Ψ(u + j, v+ k)-w(j, k) j = m/2 k = z/2 (2) Eq.(2) represents the proposed modified cross correlation operation. In general, for any two functions f and g, their modified cross correlation can be performed by applying two dimensional subtraction. These modified cross correlations can be obtained by: g(x,y) Θf(x,y) = g(m,z)- f(x+ m,y + z ) m= z= Where Θ denotes the modified cross correlation. Therefore, Eq.(2) can be written as follows: (3) h( i) = WΘΨ (4) ISSN: Issue 4, Volume 5, April 2009

2 where h(i) is the matching result when the sliding window is located at position (u,v) in the web image Ψ and (u,v) [P-m+1,T-n+1]. Now, the above cross correlation can be expressed in terms of the Fourier Transform: WΘ Ψ = F 1( F( Ψ ) F * ( W )) (5) (*) means the conjugate of the FFT for the image given in the user query. Hence, by evaluating this cross correlation, a speed up ratio can be obtained comparable to conventional image matching. II. Analysis of the Proposed Fast Image Matching Algorithm The complexity of cross correlation in the frequency domain can be analyzed as follows: 1. For a tested web image of NxN pixels, the 2D-FFT requires a number equal to N 2 log 2 N 2 of complex computation steps. Also, the same number of complex computation steps is required for computing the 2D-FFT of the image given in the user query. 2. The inverse 2D-FFT is computed. So, one backward and two forward transforms have to be computed. Therefore, for an image under test, the total number of the 2D-FFT to compute is (2+1)N 2 log 2 N The web image and the image given in the user query should be subtracted in the frequency domain. Therefore, a number of complex computation steps equal to N 2 should be added. 4. The number of computation steps required by the fast image matching is complex and must be converted into a real version. It is known that the two dimensional Fast Fourier Transform requires (N 2 /2)log 2 N 2 complex multiplications and N 2 log 2 N 2 complex additions [36]. Every complex multiplication is realized by six real floating point operations and every complex addition is implemented by two real floating point operations. So, the total number of computation steps required to obtain the 2D-FFT of an NxN image is: ρ=6((n 2 /2)log 2 N 2 ) + 2(N 2 log 2 N 2 ) (6) which may be simplified to: ρ=5n 2 log 2 N 2 (7) Performing complex subtraction in the frequency domain also requires 2N 2 real operations. 5. In order to perform cross correlation in the frequency domain, the image given in the user query must have the same size as the web image. Assume that the the image given in the user query has a size of (nxn) dimensions. So, the search process will be done over sub-images of (nxn) dimensions. Therefore, a number of zeros = (N 2 -n 2 ) must be added to the image given in the user query. This requires a total real number of computation steps = (N 2 -n 2 ). Moreover, after computing the 2D-FFT for the image given in the user query, the conjugate of this matrix must be obtained. So, a real number of computation steps = N 2 should be added in order to obtain the conjugate of the image given in the user query. Also, a number of real computation steps equal to N is required to create butterflies complex numbers (e -jk(2πn/n) ), where 0<K<L. These (N/2) complex numbers are multiplied by the elements of the input image or by previous complex numbers during the computation of the 2D-FFT. To create a complex number requires two real floating point operations. So, the total number of computation steps required for the fast image matching becomes: σ=(2+1)(5n 2 log 2 N 2 ) +2N 2 + N 2 +(N 2 -n 2 ) +N (8) which can be reformulated as: σ=(3)(5n 2 log 2 N 2 ) +(4N 2 -n 2 ) +N (9) 6. Using a sliding window of size nxn for the same image of NxN pixels, (2n 2-1)(N-n+1) 2 computation steps are required for image matching when using traditional two dimensional subtraction. The theoretical speed up factor η can be evaluated as follows: 2 2 (2n - 1)(N - n + 1) η = (10) (15N log 2 N ) + (4N - n ) + N The theoretical speed up ratio given by Eq.(10) with different sizes for the web images and different in size user images (given in user query) is listed in Tables An interesting property with Fast image matching is that the number of computation steps does not depend on either the on size of the image given in the user query or the sub-image extracted from the web image. The effect of (n) on the number of computation steps is very small and can be ignored. This is in contrast to conventional image matching in which the number of computation steps is increased with the size of both the extracted subimage and the image given in the user query. III. Conclusion A new fast method for image matching has been presented. This has been achieved by modifying the theory of cross correlation. Instead of performing dot multiplication, image subtraction has been applied in two dimensions. The modified cross correlation has been applied in the frequency domain between web image and the image given in a query. It has been proven mathematically that the number of computation steps required for the presented fast image matching is less than that needed by conventional image matching. This method reducers ISSN: Issue 4, Volume 5, April 2009

3 the time required (ex. by neural networks) for image matching especially when the two images are exactly the same. References [1] R. Klette, and Zamperon, "Handbook of image processing operators, " John Wiley & Sonsltd, [2] H. M. El-Bakry, " Fast Virus Detection by using High Speed Time Delay Neural Networks," Accepted for publication in journal of computer virology. [3] H. M. El-Bakry, A New Neural Design for Faster Pattern Detection Using Cross Correlation and Matrix Decomposition, Accepted for publication in Neural World journal. [4] H. M. El-Bakry, " New Fast Principal Component Analysis For Real-Time Face Detection," Accepted for publication in MG&V Journal. [5] H. M. El-Bakry, " Fast Record Detection in Large Databases Using New High Speed Time Delay Neural Networks," Accepted for publication in International Joint Conference on Neural Networks, June 14-19, 2009, Atlanta, USA. [6] H. M. El-Bakry, New Faster Normalized Neural Networks for Sub-Matrix Detection using Cross Correlation in the Frequency Domain and Matrix Decomposition, Applied Soft Computing journal, vol. 8, issue 2, March 2008, pp [7] H. M. El-Bakry, "A Novel Modified Hopfield Neural Network for Perfect Calculation of Magnetic Resonance Spectroscopy," International Journal on Biomedical Science, vol. 3, no. 1, 2008, pp [8] H. M. El-Bakry, and N. Mastorakis, "A New Technique for Detecting Dental Diseases by using High Speed Neuro-Computers," WSEAS Trans. on Signal Processing, vol. 8, issue 2, October 2008, pp [9] H. M. El-Bakry, and Nikos Mastorakis, "A Real- Time Intrusion Detection Algorithm for Network Security," WSEAS Transactions on Communications, Issue 12, vol. 7, December 2008, pp [10] H. M. El-Bakry, and Nikos Mastorakis, "A New Fast Forecasting Technique using High Speed Neural Networks," WSEAS Transactions on Signal Processing, Issue 10, vol. 4, October 2008, pp [11] H. M. El-Bakry, and Nikos Mastorakis, "A Perfect QoS Routing Algorithm for Finding the Best Path for Dynamic Networks," WSEAS Transactions on Communications, Issue 12, vol. 7, December 2008, pp [12] H. M. El-Bakry, and Nikos Mastorakis, "A Novel Fast Kolmogorov s Spline Complex Network for Pattern Detection," WSEAS Transactions on Systems, Issue 11, vol. 7, November 2008, pp [13] H. M. El-Bakry, and Nikos Mastorakis, " A Modified Hopfield Neural Network for Perfect Calculation of Magnetic Resonance Spectroscopy," WSEAS Transactions on Information Science and Applications, issue 12, vol. 5, December 2008, pp [14] H. M. El-Bakry, New Fast Principal Component Analysis for Face Detection, Journal of Advanced Computational Intelligence and Intelligent Informatics, vol.11, no.2, 2007, pp [15] Hazem M. El-Bakry and Mohamed Hamada, " New Fast Decision Tree Classifier for Identifying Protein Coding Regions," Lecture Notes in Computer Science, Springer, ISICA 2008, LNCS 5370, 2008, pp [16] Hazem M. El-Bakry and Mohamed Hamada, "A New Implementation for High Speed Neural Networks in Frequency Space," Lecture Notes in Artificial Intelligence, Springer, KES 2008, Part I, LNAI 5177, pp [17] Hazem M. El-Bakry, and Nikos Mastorakis "New Fast Normalized Neural Networks for Pattern Detection," Image and Vision Computing Journal, vol. 25, issue 11, 2007, pp [18] Hazem M. El-Bakry and Nikos Mastorakis, "Fast Code Detection Using High Speed Time Delay Neural Networks," Lecture Notes in Computer Science, Springer, vol. 4493, Part III, May 2007, pp [19] H. M. El-Bakry, "Fast Painting with Different Colors Using Cross Correlation in the Frequency Domain," International Journal of Computer Science, vol.1, no.2, 2006, pp [20] H. M. El-Bakry, "New High Speed Normalized Neural Networks for Fast Pattern Discovery on Web Pages," International Journal of Computer Science and Network Security, vol.6, No. 2A, February 2006, pp [21] Hazem M. El-Bakry, and Qiangfu Zhao, "Fast Normalized Neural Processors For Pattern Detection Based on Cross Correlation Implemented in the Frequency Domain," Journal of Research and Practice in Information Technology, Vol. 38, No.2, May 2006, pp [22] Hazem M. El-Bakry, and Qiangfu Zhao, "Fast Time Delay Neural Networks," International Journal of Neural Systems, vol. 15, no.6, December 2005, pp [23] Hazem M. El-Bakry, and Qiangfu Zhao, "Speeding-up Normalized Neural Networks For Face/Object Detection," Machine Graphics & Vision Journal (MG&V), vol. 14, No.1, 2005, pp [24] Hazem M. El-Bakry, "A New High Speed Neural Model For Character Recognition Using Cross Correlation and Matrix Decomposition," International Journal of Signal Processing, vol.2, no.3, 2005, pp ISSN: Issue 4, Volume 5, April 2009

4 [25] Hazem M. El-Bakry, and Qiangfu Zhao, "A Fast Neural Algorithm for Serial Code Detection in a Stream of Sequential Data," International Journal of Information Technology, vol.2, no.1, pp , [26] Hazem M. El-Bakry, and Qiangfu Zhao, "Fast Complex Valued Time Delay Neural Networks," International Journal of Computational Intelligence, vol.2, no.1, pp , [27] Hazem M. El-Bakry, and Qiangfu Zhao, "Fast Pattern Detection Using Neural Networks Realized in Frequency Domain," Enformatika Transactions on Engineering, Computing, and Technology, February 25-27, 2005, pp [28] Hazem M. El-Bakry, and Qiangfu Zhao, "Sub- Image Detection Using Fast Neural Processors and Image Decomposition," Enformatika Transactions on Engineering, Computing, and Technology, February 25-27, 2005, pp [29] Hazem M. El-Bakry, and Qiangfu Zhao, "Face Detection Using Fast Neural Processors and Image Decomposition," International Journal of Computational Intelligence, vol.1, no.4, 2004, pp [30] Hazem M. El-Bakry, and Qiangfu Zhao, "A Modified Cross Correlation in the Frequency Domain for Fast Pattern Detection Using Neural Networks," International Journal on Signal Processing, vol.1, no.3, 2004, pp [31] Hazem M. El-Bakry, and Qiangfu Zhao, "Fast Object/Face Detection Using Neural Networks and Fast Fourier Transform," International Journal on Signal Processing, vol.1, no.3, 2004, pp [32] H. M. El-Bakry, and H. Stoyan, FNNs for Code Detection in Sequential Data Using Neural Networks for Communication Applications, Proc. of the First International Conference on Cybernetics and Information Technologies, Systems and Applications: CITSA 2004, July, Orlando, Florida, USA, Vol. IV, pp [33] H. M. El-Bakry, Face detection using fast neural networks and image decomposition, Neurocomputing Journal, vol. 48, 2002, pp [34] H. M. El-Bakry, Human Iris Detection Using Fast Cooperative Modular Neural Nets and Image Decomposition, Machine Graphics & Vision Journal (MG&V), vol. 11, no. 4, 2002, pp [35] H. M. El-Bakry, Automatic Human Face Recognition Using Modular Neural Networks, Machine Graphics & Vision Journal (MG&V), vol. 10, no. 1, 2001, pp [36] J. W. Cooley, and J. W. Tukey, "An algorithm for the machine calculation of complex Fourier series," Math. Comput. 19, 1965, pp Table 1 The Theoretical Speed up Ratio for Images with Different Sizes (n=20,25,30). Image size Speed up ratio (n=20) Speed up ratio (n=25) Speed up ratio (n=30) 100x x x x x x x x x x x x x x x x x x x x ISSN: Issue 4, Volume 5, April 2009

5 Table 2 The Theoretical Speed up Ratio for Images with Different Sizes (n=35,40,45). Image size Speed up ratio (n=35) Speed up ratio (n=40) Speed up ratio (n=45) 100x x x x x x x x x x x x x x x x x x x x Table 3 The Theoretical Speed up Ratio for Images with Different Sizes (n=50,55,60). Image size Speed up ratio (n=50) Speed up ratio (n=55) Speed up ratio (n=60) 100x x x x x x x x x x x x x x x x x x x x ISSN: Issue 4, Volume 5, April 2009

6 Table 4 The Theoretical Speed up Ratio for Images with Different Sizes (n=65,70,75). Image size Speed up ratio (n=65) Speed up ratio (n=70) Speed up ratio (n=75) 100x x x x x x x x x x x x x x x x x x x x Table 5 The Theoretical Speed up Ratio for Images with Different Sizes (n=80,85,90). Image size Speed up ratio (n=80) Speed up ratio (n=85) Speed up ratio (n=90) 100x x x x x x x x x x x x x x x x x x x x ISSN: Issue 4, Volume 5, April 2009

7 Table 6 The Theoretical Speed up Ratio for Images with Different Sizes (n=95,100,105). Image size Speed up ratio (n=95) Speed up ratio (n=100) Speed up ratio (n=105) 100x x x x x x x x x x x x x x x x x x x x Table 7 The Theoretical Speed up Ratio for Images with Different Sizes (n=110,115,120). Image size Speed up ratio (n=110) Speed up ratio (n=115) Speed up ratio (n=120) 200x x x x x x x x x x x x x x x x x x x ISSN: Issue 4, Volume 5, April 2009

8 Table 8 The Theoretical Speed up Ratio for Images with Different Sizes (n=125,130,135). Image size Speed up ratio (n=125) Speed up ratio (n=130) Speed up ratio (n=135) 200x x x x x x x x x x x x x x x x x x x Table 9 The Theoretical Speed up Ratio for Images with Different Sizes (n=140,145,150). Image size Speed up ratio (n=140) Speed up ratio (n=145) Speed up ratio (n=150) 200x x x x x x x x x x x x x x x x x x x ISSN: Issue 4, Volume 5, April 2009

9 Table 10 The Theoretical Speed up Ratio for Images with Different Sizes (n=155,160,165). Image size Speed up ratio (n=155) Speed up ratio (n=160) Speed up ratio (n=165) 200x x x x x x x x x x x x x x x x x x x Table 11 The Theoretical Speed up Ratio for Images with Different Sizes (n=170,175,180). Image size Speed up ratio (n=170) Speed up ratio (n=175) Speed up ratio (n=180) 200x x x x x x x x x x x x x x x x x x x ISSN: Issue 4, Volume 5, April 2009

10 Table 12 The Theoretical Speed up Ratio for Images with Different Sizes (n=185,190,195). Image size Speed up ratio (n=185) Speed up ratio (n=190) Speed up ratio (n=195) 200x x x x x x x x x x x x x x x x x x x Table 13 The Theoretical Speed up Ratio for Images with Different Sizes (n=200,205,210). Image size Speed up ratio (n=200) Speed up ratio (n=205) Speed up ratio (n=210) 300x x x x x x x x x x x x x x x x x x ISSN: Issue 4, Volume 5, April 2009