Different types of images. Taxonomy of fixed images. Introduction to image databases. Chapter VI: Image Databases

Transcription

1 Visual nformation Systems ifferent types of images Taxonomy of fixed images ntroduction to image dataases mages Simple Picture irect Complex yrid Picture ndirect Chapter V: mage ataases rawing Visual Surrogate iagram Map/plan Logo 6.1 Generalities 6.2 Retrieval ased on key-words 6.3 Retrieval ased on content 6.4 Physical structuring of image dataases 6.5 Example: mage Rover 6.6 Conclusions 1

2 Several types 6.1 Generalities Collections of fixed images Very ig rasters Raster maps erial photos Satellite images Video Sequences films Retrieval ased on key-words typically a user gives a list of key-words) ased on content user gives an imageexample query y example) colors shapes textures spatial relations igh and low level characteristics igh Level Characteristics nature and semantics of oects present on image and their relations example : a oat on the sea at sunset Low Level Characteristics pixels colors textures etc. example : images with 40 % of light lue. nterrogation of image ases Retrieval ased on key-words Retrieval ased on content Retrieval ased on colors Retrieval ased on shapes Chapter V: mage ataases 2

3 6.2 Retrieval ased on key-words Each image is descried y a list of keywords operation named indexing) Generally from 3 to 10 key-words given y the author or y an expert Those key-words are regrouped in a thesaurus with 3 relations synonymy genericity / specificity nnotations dditional nformation Retrieval of key-words characterizing MM documents Lists of oects persons etc. Origin of the document author device date etc.) Generally made visually Seldom named manual indexing" Example of a geographic thesaurus Other example of thesaurus Europe nimal taly France Switzerland elgium ird Mammal Snakes Fish Piedmont Savoie ourgogne retagne Migrating ird Carnivore erivore Primate Chaméry nnecy lpes Rennes og Wolf Tiger Cow Chapter V: mage ataases 3

4 Queries oolean list of key-words N OR EXCEPT) Noise = too much non relevant documents Silence = too less or asence results Example : looking for photos dealing with the culture of cauliflowers in ustralia Reformulation R : culture and cauliflower and ustralia Transformation of the query ased on thesaurus R : culture except civilization)and cauliflower or vegetale)andustralia or Oceania) ON LNE Text-only Query Visualization Relevance feedack Keyword: each Search engine Manual nnotations OFF LNE MM 1 st Generation Search System Chapter V: mage ataases 4

5 Plant dataase 6.3 Retrieval ased on content Generally ased on an example Query-yexample Content-ased mage Query query y example) Colors Shapes Texture Search Engine MM ON LNE Text-only or visual query rowsing Search Engine Multidimensional ndexing Visualization Relevance feedack Spatial Relations utomatic Extraction Manual nnotations MM 2 nd Generation Search System OFF LNE Chapter V: mage ataases 5

6 Structuring a video flow Segmentation Manual vs. automatic Segmentation Splitting in episodes scenes shots Semantics same actor same place etc.) ON LNE Relevance feedack Content-ased Retrieval Text-only or visual query rowsing Visualization Retrieval ased on colors Search Engine Multidimensional ndexing Navigation Retrieval ased on shape Retrieval ased on parameters Retrieval ased on spatial relations utomatic Extraction - Manual nnotations Segmentation of video OFF LNE MM Search System for Video Chapter V: mage ataases 6

7 6.3.1 Retrieval ased on colors Example of histograms Containing a color in a certain proportion Similarity of colors in the whole image Similarity of only a part ased on an oect having some particular color etc. istograms of regions Solution Gray level histogram istogram of colors for the image-query Q ) and for other images ) ttention: same resolution same colorencoding system lack White Chapter V: mage ataases 7

8 Chapter V: mage ataases 8 Comparison of histograms Comparison y test χ 2 type) ut weak performances )) ) ) ) ) ) Q n Q n Q Q = = = = Method of Swain and allard ntersection of histograms = = = n Q Q n Q 1 1 ) )) ) min ) ntersection of histograms Colors Frequence loworld University of California erkeley images

9 MRS MRS Multimedia nalysis and Retrieval System) University of llinois at Urana-Champaign University of California at rvine emos Simplicity Stanford university images Similarity Chapter V: mage ataases 9

10 6.3.2 Retrieval ased on shape The user gives for instance gives a shape manually draught template) Comparison with other images Linear deformations translations rotations scaling Other transformations warping) Example horse Example of results Chapter V: mage ataases 10

11 Snakes ctive Contours Example Original shape is deformed to reach gradually some other shape Oectives To follow edges the est possile To minimize deformation energy Shape deformation 44 Chapter V: mage ataases 11

12 6.3.3 Retrieval ased on parameters Structure Principle : each image is descried y a set of parameters issued from image processing until several dozens of parameters index feature extraction) the image-query is analyzed to evaluate those parameters comparison distance) Retrieval ased on spatial relations Usual features Symolic Proections 2 strings) Using Egenhofer relations Using Jungert operators Signature Chapter V: mage ataases 12

13 Chang 2 strings escription of an image Principle : d x-proection : car < tree < house y-proection : car and house < tree c a a = u v) a=d<a=<c a=a<=c<d) = same place < left-right normal 2 strings : ad<a<c aa<c<d) Pattern matching Conclusion aout 2 strings nitial mage d a c nteresting tool ifficulties of description when overlaps Prolems of zones with holes non connected oects) Example of queries : other operators c c a a a Chapter V: mage ataases 13

14 Encoding with 2 strings Jungert operators 1/2) u : v : y x < x < e y = C = x y < e < x e y < e = C x y < Ce x < Ce y < center) < center) = center) = center) C C % Side y side Min) > Min ) Max) < Max ) Length) < Length) Jungert operators 2/2) Signature [ ] \ Min) = Min ) Length) < Length) Max) = Max ) Length) < Length) Min) < Min ) Length) Length) 1 Encoding an image with 2 strings or more exactly with 2 strings Lee Yang Chen) = part encodage) 2 ashing Function / Max) > Max ) Length) Length) Chapter V: mage ataases 14

15 Retrieval y signature 6.4 Physical Structuring of mage ataases Query Signature Generator mage set of encoded pixels set of descriptors = parameters) Possily some recognized pictorial oects Signature Matching mage Signatures Signature candidate 2 -strings Query Processor mage ataases Zillions of images usually same format) n access system ased on descriptors ig raster unique very ig image several illions of pixels) Stored position of some pictorial oects escriptors Query escriptors: any characteristics of an image Low level efore interpretation) igh level after interpretation); classifiers Several dozens possily point in a n-dimension space Examples Colors: encoding histogram etc. Texture: granularity direction repetitiveness etc. Let e a set of descriptors Let e a query-image nalysis to evaluate descriptors Relevance feedack Several query-images Chapter V: mage ataases 15

16 Relevance feedack mage query spatial query Point query n-dimension space) Research of k-nearest neighors Multipoint query Original mage mages found ut non relevant New image Relevant mages Physical ccess Method escriptors n-dimension space istances Euclidian etc. lgorithms: Research of k-nearest neighors Solutions : R-tree or R + -tree VP-tree Vantage Point Tree) VP-tree Retrieval of neighors distance etween points n-dimension) We start from a point named Vantage point) which will e the center of a "circle" so that 50 % of the points will elong to this "circle" Then we continue in a tree-like manner Vantage point : arycenter The farthest point Chapter V: mage ataases 16

17 E C E F Example C F 6.5 Example: mage Rover Semantic ssociations oston University M. emo er/demo.html. Semantic ssociations Color ssociations Orientation ssociations Chapter V: mage ataases 17

18 Color ssociations Orientation ssociations 6.6 Conclusion ifficulty of retrieving images ased on contents Passing from pixels to semantic oects Low and high level descriptors edicated search engines Often disappointing results Using semantic networks and ontologies emos lt.aspx Chapter V: mage ataases 18