[FMIQ] Ajit Datar. Fast Multiresolution Image Querying

Transcription

1 Fast Multiresoution Image Querying [FMIQ] Ajit Datar 1

2 Background Implementation of FMIQ paper by Jacobs et al (see references) Use of Haar wavelet based signatures Small signature database Search time linearly proportional to numer of imagess Uses color information for search Implemented in python (cross platform) using offline database 2

3 In a nutshell... All the images are preprocessed to store their wavelet signatures. User paints a query or submits an existing picture as query, no other information is given. Signature of the query is generated All the images are scored according to the match between query-signature and image signature. Images are sorted by score and search results are presented 3

4 What is a wavelet signature? It is basically just a collection of m largest Haar decomposition coefficients quantized to 2 levels (positive or negative) Value at (0,0) gives us the average intensity level for that color plane Rest of the coefficients give us the detail of the image (we keep only m greatest coefficients for our signature) 4

5 Database representation After pre-processing, images are added to Search arrays for fast searching, according to values of their non-zero wavelet coeff for each color channel. 6 search arrays -> 2 (positive/negative) for each color channel Each location in the search array stores a list of images having a matching coefficient at that location. Eg location (3,4) in the negative search array for Y color channel stores the list of images having a large negative coefficient at location (3,4) for Y color channel in its signature. 5

6 Querying Metric is the method of comparison Which after simplification becomes... (called L q metric 6

7 Implementation 3 modules preprocess, db, query Images can be incrementally added to an existing database (flat file database implemented as python shelves) Couple of command-line frontends to these modules to glue them together adddir add all images in the direcotry to db queryfile show matching images to this file infodb return info about the database Current database holds images almost 1000 images from a wide range of categories such as animals, paintings, actors, space, cars, aviation, scenes, cartoons... 7

8 Some observations Smaller databases with images from the same/similar category gives better matching Wide range of categories widens the search An existing db image when given as query would show up as the 1 st / 2 nd match. Background color plays an important role. Average search time for database of ~1000 images 2-3 minutes with current implementation 8

9 Future work Decent GUI Grayscale image storing and querying Effect of change of weights on the query Effect of number of wavelet coefficients (both in preprocssing as well as querying stage) Scaling of database Adding category information to search as an option 9

10 References - [ Jacobs, Finkelstein, Salesin ] Wavelets for Computer Graphics [Stollnitz, DeRose, Salesin ] Wavelet based image similarity analysis [Rocio Alba Flores et al] 10

11 boat.png (< 31 sec) Questions/Demo magic.jpg (< 30 sec) flowers.redlilies.jpg ( < 1 min 9 sec) painted Come and try it yourself! photos planepic.jpg (< 2 min 45sec) 11