Image Information Mining (IIM): Where do we go?

Transcription

1 Image Information Mining (IIM): Where do we go? Klaus Seidel and Mihai Datcu IGARSS 2004

2 The paradox People have normally trouble in caching more than 7 items at a time We design systems to enable people to analyse TeraBytes of data!

3 1. The beginning: short reminder in Florence, Conference on Data Base Techniques for Pictorial Applications aiming at integrating data bases with picture processing 1990 the field got a name Content Based Image Retrieval CBIR 1998 CBIR gets married with Data Mining and KDD 2000 a new field is born: Image Information Mining

4 1. The beginning: short reminder The problem: Search and extract information from image archives Bottleneck 1. Scene description and image representation Bottleneck 2. Visual information and semantics Bottleneck 3. Image acquisition/storage and interpretation Bottleneck 4. Off line and on line storage Bottleneck 5. Local and remote access to data Bottleneck 6. Technical culture and education Bottleneck 7. Budget

5 1. The beginning: short reminder What has been done 1980 Text annotation of images 1990 Indexing images by colour, shape, texture 1993 Query by image feature similarity 1997 Query by examples and relevance feedback 2000 Semantic learning adaptation to the user conjecture

6 Anno 2004: what did we achieve? The progress: A pre-operational prototype for EO data, information an knowledge access and distribution -- KIM and KES Interactive semantic image annotation Knowledge formalization and ontology learning Large data volume management Multi-sensor and heterogeneous data management Human perception of massive data volumes Optimal solutions for technological bounds (storage, speed, GUI, )

7 The paradox People have normally trouble in caching more than 7 items at a time We design systems to enable people to analyse TeraBytes of data!

8 Communication as solution

9 KIM - Knowledge driven Information Mining in remote sensing image archives Semantic Definition Objectives: a prototype system of a next generation architecture to help the users to gather relevant information rapidly, manage and add value to the huge amounts of historical and newly acquired satellite data-sets Image Information Mode Learn Mode Label Information Name Description Label_1 Pan Mode Reset Similar labels View Order Gallery Store A posteriori Map Models contribution Model1 Model2 Search Search parameters Set label Set a label Probability Separability Coverage Set collections Set Satellites and/or Sensors Collection_1 Satellite_1 Collection_2 Satellite_2 Collection_3 Collection_4 Sensor_1 Collection_5 Sensor_2 Date Area Upper bound(lat/lon) From: yyyy/mm/dd Lower bound(lat/lon) To: yyyy/mm/dd Textual annotation Stored Queries Set a stored query Details Query_1 Query_2 Apply Query_3 Result Data: ERS, Landsat, Ikonos, MERIS, E-SAR, DEDALUDS Search Store Query Parameters Confirm Selection Evaluators and users: ESRIN, CNES, EUSC, NERSC, DLR Output: prototype system

10 KES - EO domain specific Knowledge Enabled Services Objectives: a scalable prototype applicable to a number of fields supporting image information mining and other related user interactions, knowledge acquisition and sharing within user communities, multi - type data handling (image, text, GIS), up to semantic interactions and knowledge communication Geology Hydrology Water Drainage Mountain River Glacier Sea Grassland Agriculture Domain Ontology Semantics Labels Data: ERS, Landsat, HR SAR and optical (data volume ~200 scenes) Evaluators and users: ESRIN, EUSC, DLR Output: prototype system Spectral Texture Spectral SAR Features Images

11 KIMV - KIM Validation for EO archived data exploitation support Objectives: to implement, test and evaluate a quasioperational environment for simple access also as MASS Services to enhanced image selection functions (image selection by combinations of standard spatio temporal - parameters and image information content queries) Administrator Client Learning Client Enhanced image selection client Internet Direct Connection Internet KIMV SYSTEM KIMV Server MASS Internet MASS Toolbox Web Services Data: MERIS, ERS, Landsat, SPOT (data volume ~5 000 scenes) Evaluators and users: ESRIN, EUSC, CNES, DLR, universities, industry (total ~15 users) Output: system pre-operational Ingestion chain monitor Existing Archive (disk or juke box) Direct Connection Direct Connection Parallel Ingestion Chain JDBC RDBMS

12 KEO - Knowledge-centric Earth Observation Objectives: a prototype system and environment to foster the enlargement of EO data utilisation, and in particular of the large archives of multi-mission and multi-temporal images, provide a better support to research, value-adding industry, service providers and EO user communities, like scientific investigations, risk and disaster management, or in the GMES programme Data: TerraSAR, ENVISAT, ERS, Landsat, SPOT (data volume ~100GB/day.) Evaluators and users: ESRIN, DLR, EUSC, CNES, universities, industry. Output: operational system

13 The European Image Information Mining Coordination Group IIMCG Members: ASI, CNES, CNR, DLR, EC- IST, ESA, ETHZ, EUSC Chair: Sergio D Elia, ESA/ESRIN IIMCG focus: research and technological activities for automated and user centred extraction of information from EO images and image archives in support to content understanding Events: ESA-EUSC Conferences on Knowledge driven Information Management in Earth Observation data, ESRIN, Frascati, 2002, EUSC Madrid, 2004, ESRIN, Frascati, 2005

14

15 EO today EO is one of Europe s successes EO business: a disappointment? A need for information and services, not for data Use the potential of EO missions synergy

16 The Opportunities Historical archives (ERS 1/2, Landsat, etc. etc.) ENVISAT (we work on MERIS!) Archives of high resolution images: SPOT, IKONOS, Quickbird, etc. Ground segment payload of TerraSAR Global Monitoring for Security and Stability (GMES) Reconnaissance and surveillance

17 The future: what to do in theory, in technology and for applications? Think differently!

18 The future: next 3 years Educate, develop, apply, use Push and valorise the existing concepts and technology to its limits to bring into operation the existing systems for EO applications Prototype new applications in EO and other fields: astronomy, medicine, biology, Establish formation programs to enlarge the usage of the technology and promote education programs to prepare future development

19 The future: next 5 years Think differently to design new - not yet existing - concepts: Information theory of semantic coding Theory of knowledge representation and communication Multidimensional data and information representation Theory of information and complexity of massive data sets Re-investigate unconventional technologies: optical processing, holography, parallel computing, etc.

20 The future: next 15 years New theories, new technologies Biological and neural information processing: theory and machines Quantum information Intuitive and conscious computing Intuitive and conscious sensors