Report on Image Processing (ECE 8741) Project. Fast Multiresolution Image Querying implementation of paper by Jacobs, Finkelstein, Salesin.

Transcription

1 Report on Image Processing (ECE 8741) Project Fast Multiresolution Image Querying implementation of paper by Jacobs, Finkelstein, Salesin. Author: Keywords: wavelet-signature, multiresolution, image-search, similarity-analysis 05/14/04 1

2 Table of Contents Introduction...2 Theory...2 Preprocessing...3 Querying...3 Implementation...4 Results...4 User painted queries...4 1) boat.png (256x256)...4 2) magic.png (256x256)...5 Existing images as query...6 1) flowers.redlilies.jpg (800x600)...6 2) planepic.jpg (500x357)...7 Future work...8 Acknowledgement...8 Introduction Searching for images with keywords is successful only upto a point. Most often than not, a user can't describe an image in words which limits her search. But user can sometimes specify more effectively her search criteria using a simple conceptual drawing or existing similar image. Fast multiresolution image querying is a technique which uses Haar wavelet based signatures to perform such a search. Previous work As given in their paper [1] earlier projects doing content based image querying include IBM's QBIC, work of Hirato and Kato and some others. The approach used here is uses wavelets allowing the query to be of any resolution. Authors of the paper also have an implementation which runs on SGI platoform. 2

3 Theory This report just briefly describes the overall operation. For more information please read the original paper [1] or a decriptive chapter on the same paper [2]. The python code for this project will also be made available for download on author's website Any further questions should be directed to ajitdatar@gmail.com Preprocessing The prprocessing stage stores the signature of each target image in an offline database in an easy to search format. Following are the main steps 1. Resize the image to predefined size (128x128 in current implementation) 2. Convert it to YIQ color model 3. Do Haar decomposition 4. Truncate all other coeffcients except T(0,0) to m largest values. Or in other words, keep m largest Haar wavelet coeffcients, and discard others (m=30 in my implementation) 5. Quantize the coeffcients to +1 (positive) or -1 (negative) 6. Add the image to one of the six search arrays (positive and negative for each of the 3 color channels). Eg If an image has a large positive coeffcient for Y color channel at (3,100) then it will be added to the list at location (3,100) in search array Y + The database also stores the links to original images. Note that the original image is not modified at all in the preprocessing stage. Querying Querying uses the metric given in [1] for signature comparison and generates a score for each image. The images with least score are the best matches and n such images are returned as n matches to the query. Following is the L q metric. Reader is encouraged to refer [2] for discussion. 3

4 Equation 1 The overall steps for querying are, 1. Obtain query image as user-painted image or existing 2. picture.preprocess the query image to obtain signature Q 3. For each target image T use the metric in equation 1 to generate a score. 4. Sort the scores in increasing order. 5. Return first n images as n first matches to the query. Implementation The paper was implemented using python ( Following are the 3 modules. preprocess.py image preprocessing db.py database manipulation query.py query matching It uses following command line frontends to put the 3 modules together. infodb.py information regarding the database. queryfile.py specift an image file as query. adddir.py recursively add all the images in the directory to database 4

5 Results Following are some query images and first 20 matches for them. The database consists of ~ 1000 images from variety of categories. User painted queries 1) boat.png (256x256) Time taken is less than 31 sec 5

6 2) magic.png (256x256) Time taken is less than 30 sec 6

7 Existing images as query 1) flowers.redlilies.jpg (800x600) Time taken is less than 1 min 9 sec 7

8 2) planepic.jpg (500x357) Time taken is less than 2 min 45 sec 8

9 Future work Here are some general directions for future work. 1. Decent GUI 2. Grayscale image storing and querying 3. Effect of change of weights on the query 4. Effect of number of wavelet coefficients (both in preprocssing as well as querying stage) 5. Scaling of database 6. Adding category information to search as an option Acknowledgement I thank Professor Rocio Alba Flores (UMD) for her guidance in this project. I also thank my friends for providing me some test images for user-painted queries. Bibliography 1: Jacons, Finkelstein, Salesin, Fast Multiresoultion Image Querying, : Stollnitz, DeRose, Salesin,, 9