SISTEMI PER LA RICERCA

Transcription

1 SISTEMI PER LA RICERCA DI DATI MULTIMEDIALI Claudio Lucchese e Salvatore Orlando Terzo Workshop di Dipartimento Dipartimento di Informatica Università degli studi di Venezia

2 Ringrazio le seguenti persone, coinvolte nel progetto SAPIR Raffaele Perego Fausto Rabitti Paolo Bolettieri Fabrizio Falchi Tommaso Piccioli Matteo Mordacchini

3 WHAT AND, MORE IMPORTANTLY, WHY??! Multi-Media M Objects: text, web pages (Google, Yahoo!, ) images (Flickr bought by Yahoo!, ) video (YouTube bought by Google, ) audio etc.... Why we are interested in MM objects? 1 image every 10 documents. Increasing multimedia content on the web. Yet, search in audio-visual content is limited to associated text and metadata annotations. 3

4 OK, BUT GOOGLE ALREADY DOES THE JOB... We mean to search by content! Google does text-based t (annotations) ti search Do we have enough text? How is the quality of such text? Is it correlated with the actual content? Look at this: ISTI-CNR (Pisa) and Max Planck Kunsthistorische Institute project on Florentine Coat of Arms 4

5 SISTEMI PER LA RICERCA DI DATI MULTIMEDIALI 5 CONTENT-BASED SEARCH Shape Selection Results

6 SISTEMI PER LA RICERCA DI DATI MULTIMEDIALI 6 CONTENT-BASED SEARCH Shape Selection Results

7 WHAT IS THE ADDED VALUE? Content-based search for disambiguation: What about searching for sapphire on Flickr? SISTEMI PER LA RICE ERCA DI DATI MULTIMEDIALI 7 of 21

8 CONTENT-BASED, I.E. SIMILARITY SEARCH Objects are unknown distances between objects is known Metric Space assumption: symmetry identity triangle inequality Distance functions inducing a metric space: Minkowski distances, edit distance, jaccard distance... Typical queries: Range or knn Applications: 8

9 SIMILARITY SEARCH Photo Search Objects are unknown query: distances between objects is known Metric Space assumption: symmetry identity triangle inequality Distance functions inducing a metric space: Minkowski distances, edit distance, jaccard distance... Typical queries: Range or knn Applications: 9

10 Photo Search SIMILARITY SEARCHMedical Data Search Objects are unknown distances between query: objects is known Metric Space assumption: symmetry identity triangle inequality Distance functions inducing a metric space: Minkowski distances, edit distance, jaccard distance... Typical queries: Range or knn Applications: 10

11 Photo Search SIMILARITY SEARCHMedical Data Search 3D Shape Search Objects are unknown distances between objects is known Metric Space assumption: symmetry query: (haemoglobin identity molecule) triangle inequality Distance functions inducing a metric space: Minkowski distances, edit distance, jaccard distance... Typical queries: Range or knn Applications: 11

12 SIMILARITY SEARCH Objects are unknown distances between objects is known Metric Space assumption: symmetry identity triangle inequality Distance functions inducing a metric space: Minkowski distances, edit distance, jaccard distance... Typical queries: Range or knn Applications: photos, 3D shapes, medical images but also text, dna, graphs, etc. etc. 12

13 SISTEMI PER LA RICERCA DI DATI MULTIMEDIALI 13 FEATURE-BASED APPROACH From the object space to the feature space

14 SISTEMI PER LA RICERCA DI DATI MULTIMEDIALI Generalized Hyper-plane p Tree DATA STRUCTURES Vantage Point Tree 14

15 DATA STRUCTURES Vantage Point Tree Generalized Hyper-plane p Tree Multiple sub-trees may hold the solution They perform well with high dimensionality data 15

16 I HAVE NEVER SEEN ANYTHING LIKE THAT IN THE WEB!!! Why giants like Google and Yahoo! are not using content-based search? Recent studies confirm that centralized solutions are not scalable! A single standard PC would need about 12 years to process a collection of 100 million images. Why? Feature extraction is expensive! feature extraction vs. words in a web-page Searching is expensive! similarity search vs. boolean search 16

17 I HAVE NEVER SEEN ANYTHING LIKE THAT IN THE WEB!!! Why giants like Google and Yahoo! are not using content-based search? Recent studies confirm that centralized solutions are not scalable! A single standard PC would need about 12 years to process a collection of 100 million images. Why? Feature extraction is expensive! feature extraction vs. words in a web-page Searching is expensive! similarity search vs. boolean search 17

18 PARALLEL FEATURE EXTRACTION We crawled Flickr to gather Image Ids A set of clients were deployed on the EDEE Grid: Retrieve 1000 imaged Ids Download Image Extract MPEG-7 features Parse Flicker Photo-page to obtain additional metadata Send metadata to our centralized repository We have about 50 millions images with features it seems that Flickr as at least 1 billion images Yahoo! images has at least 2 billion images and they grow exponentially! Our metadata collection is called CoPhIR It is largely the largest publicly available collection 18

19 PARALLEL SEARCH The search space is mapped into a linear interval This is mapped onto a distributed network thanks to some DHT-based algorithm Each node of the network holds an M-Tree SISTEMI PER LA RICE ERCA DI DAT TI MULTIMEDIALI 19

20 OUR PROPOSAL Metric cache: Cache is widely used in traditional search engines Trivial: i Store results previous queries Return results when submitted query is stored Less trivial... Store results of previous queries Best-effort query answering (with guarantees) ) Use past queries to optimize database queries. 20

21 A CACHE WITH ONE ENTRY... q i is the query in cache with its k-nn. r i is the radius of the query. q j is a new query. If d(q i, q j )<r i then the cached objects at tdistance R ji d(q i, q j ) q i r i q j R ji = r i - d( q i, q j ) from q j are the top-k results of the new query. 21

22 A CACHE WITH MANY ENTRIES Given the new query q x find the largest R xi corresponding to the cached query q i. The cached objects within distance R xi from q x are the most similar to the query. r l q i r i r j Additionally, one could use other objects in the q l q x cache to provide an approximate answer. 22 q j

23 SISTEMI PER LA RICERCA DI DATI MULTIMEDIALI 23 SOME RESULTS...

24 SISTEMI PER LA RICERCA DI DATI MULTIMEDIALI 24 SOME RESULTS...

25 SISTEMI PER LA RICERCA DI DATI MULTIMEDIALI 25 SOME RESULTS...

26 THE END.... Grazie! =)