Session 2 A virtual Observatory for TerraSAR-X data

Transcription

1 Session 2 A virtual Observatory for TerraSAR-X data 2nd User Community Workshop Darmstadt, May 2012 Presenter: Mihai Datcu and Daniela Espinoza Molina (DLR)

2 This presentation contains contributions from: Quartulli, M; Datcu, M; 2012, Accelerated Compression Based CBIR, in preparation Dumitru, C. O. ; Datcu, M, TN KLAUS, Dec Blanchart, P; Ferecatu, M; Datcu. M; 2011, Cascaded Active Learning for Object Retrieval using Multiscale Coarse to Fine Analysis, in the proceedings of IEEE Conference on Image Processing (ICIP), Bruxelles, 2011 Shiyong Cui; Datcu, M.; 2011, Coarse to fine patches-based multitemporal analysis of very high resolution satellite images, Multi-Temp, Page(s): Popescu, A. A.; Gavat, I.; Datcu, M.; 2011, Contextual Descriptors for Scene Classes in Very High Resolution SAR Images, IEEE GRSL Bratasanu, D.; Nedelcu, I.; Datcu, M.; 2011, Interactive Spectral Band Discovery for Exploratory Visual Analysis of Satellite Images, IEEE JSTARS, Espinoza Molina, D.; Gleich, D.; Datcu, M.; 2011, Evaluation of Bayesian Despeckling and Texture Extraction Methods Based on Gauss Markov and Auto-Binomial Gibbs Random Fields: Application to TerraSAR-X IEEE TGRS (IEEEXPLORE early access) Birjandi, P.; Datcu, M.; 2010, Multiscale and Dimensionality Behavior of ICA Components for Satellite Image Indexing, IEEE GRSL, Volume: 7, Issue: 1, Page(s): Blanchart, P.; Datcu, M.; 2010, A Semi-Supervised Algorithm for Auto-Annotation and Unknown Structures Discovery in Satellite Image Databases, IEEE JSTARS, Volume: 3, Issue: 4, Part: 2, Page(s): Gueguen, L.; Datcu, M.; 2008, A Similarity Metric for Retrieval of Compressed Objects: Application for Mining Satellite Image Time Series IEEEE Transactions on Knowledge and Data Engineering, Volume: 20, Issue: 4, Page(s): Daschiel, H.; Datcu, M.; 2005, Design and evaluation of human-machine communication for image information mining, IEEE Transactions on Multimedia, Volume: 7, Issue: 6, Publication Year: 2005, Page(s): Datcu, M.; Seidel, K.; 2005, Human-centered concepts for exploration and understanding of Earth observation images IEEE TGRS, Volume: 43, Issue: 3, Page(s): Daschiel, H.; Datcu, M.; 2005 Information mining in remote sensing image archives: system evaluation IEEE TGRS, Volume: 43, Issue: 1, Page(s): Datcu, M.; et al.; Information mining in remote sensing image archives: system concepts IEEE TGRS, Volume: 41, Issue: 12, Part: 1, Page(s):

3

4

5 The Content of VHR EO Archives EO Archive = O { } EO Products EO product = image + xml file image = x pixels pixel = 16/32 /1000s bits, real/complex/multiband xml file = a lot about the imaging mode, orbit, position, timing TOTAL = many PetaBytes

6 Sensor resolution & Information Content 10m 4m 2m 1m 4 objects > 100 objects

7 Spatial Context & Information Content Left: Example of typical ATR image (MSTAR database: T-72 battle tank): one isolated target on homogeneous background. Middle and right: HS TerraSAR-X examples of industrial/urban sites; middle patch: broad view of a harbor; right patch: close view of a district with skyscrapers. Conversely to ATR, we no longer have isolated objects, but rather complicated arrangements of targets. The number of individual objects is so high that we should work with Scene Classes instead of specific Objects

8 Spatial Context & Information Content At meter resolution, if context is ignored (the analysis windows being used are relatively small as compared to the object scale), very different scene classes may be confused. The example shows a bridge (left) and buildings (right) which are comprehensible only when the window size is sufficiently large to incorporate relevant context. Otherwise, both scene classes look similar and can be confused.

9 Scene Categories & Information Content 1 HS TerraSAR-X Scene = up to image patches (100 x 100 m)

10

11 Scene Evolution categories & Information Content Agriculture Forest House Lake Road

12 Multispectral and Spatial & Information Content WorldView 8 bands, 2 meter: spectral classes Spatial categories

13 Topology & Information Content A symbolic language?

14 Semantics & Information Content Define semantic annotation models

15 Product Metadata & Information Content XML file contains information about productcomponent, annotation, imagedata, missioninfo, acquisitioninfo, sceneinfo, etc Ontology queries Extraction of Metadata from XML annotation file

16 Number of acrchived scenes Number of archived scenes Number of archived TerraSAR-X scenes by country and incidence angle Spain Finland < < < < <60 Mid-range incidence angle 60 and larger 0 < < < < <60 Mid-range incidence angle 60 and larger The figure shows the number of archived TerraSAR-X scenes acquired over Spain and Finland at specific incidence angles. Almost half of the nearly 60 acquisitions over Spain were acquired with a mid-range incidence angle of between 30 and 40. Of a total of only six acqisitions over Finland two were acquired at each of the three 10 incicence angle ranges between 20 and 50.

17 The data archive The data access

18 Few Challenges The data and information processing performance The scalability issues need more detailed analysis as the volume, diversity and heterogeneity of data is continuously growing The end to end IIM process, i.e. the communication of the EO and related data content to the EO user needs special new solutions Fill the instrument gap also the semantic gap Provide an enriched index for the product catalogue Provide a base for a new type of interactive value added EO products Design the image and data representation for further auto-annotation and data mining processing.

19 TELEIOS Architecture

20 TELEIOS Architecture

21 Concept of Knowledge Discovery Query, Data Mining & Knowledge Discovery Data Sources Data Model Generation DBMS Visual Data Mining Users Interpretation Understanding

22 Functional Concepts Browsing: Navigation in the archive for discovering things. The direction, and goals of the navigation are changing based on the content of the queries. It gives a global image of the archive content. Searching: Navigation in the archive with a clear objective, the results of each query is used for directing the navigation. Annotation: Creation of new catalogue (index) entries for the archive. The result of browsing or searching may be used to annotate the results.

23 Exploration/Query Concepts Numeric and predefined queries: classical geographic position, time, senor type, etc. The output is a list of products in the specified parameters Semantic an numeric queries: by use of supporting frames a complex question can be formalized. The output is a list of products explained by the query sentence. Image similarity queries: search of similar images with a given example. The output is a ranked list of images. KDD: interactive search supported by relevance feedback mechanisms. The output is the desired product, the related products (containing similar information), semantically explained, or new, previously unknown information. Visual Data Mining: Visual exploration of the whole archive. The output are outliers, interesting groups, associations, etc.

24 Queries using numerical descriptors and predefined keywords These state-of-the-art queries are based on basic image metadata such as coordinate systems, acquisition time, type of product, etc. The system allows functionalities such as navigations through the globe map, listing the available products in the image database, and placing an order for the desired products

25 Agenda (2/2)

26 Semantic and numerical queries Semantic Labels: The user can enter a simple label in the form of text or select an item from the available labels in the catalogue to perform the query (as for example forest ). We observe that these labels are pre-defined labels previously obtained as results of the image annotation process. Query language and Ontologies: Here, the queries can be performed either using only semantics or semantics and spatial content in the form of text or numerical entries. The queries based on spatial content are performed by using the image descriptors. The query language can rely on a query template in order to avoid mistyping and helping the user. Query Builder enabling queries by using semantics, topological relations, different operators, and numerical descriptors

27 Agenda (2/2)

28 Query by example The knowledge discovery starts with a query based on image content as example. The user selects an image; later, its content is passed as example for the query, the system activates the data mining methods and retrieves the results The user selects the content of his interest by clicking on the image and the system retrieves the relevant images ranked according to some metrics

29 29

30 KDD, Active Learning and Auto-annotation KDD: interactive search supported by relevance feedback mechanisms. The output is the desired product, the related products (containing similar information), semantically explained, or new, previously unknown information learn the targeted image category as accurately and as exhaustively as possible minimize the number of iterations in the relevance feedback loop sys Query image user

31

32 Warning: not all categories can be learned!

33 Dec. Agenda (2/2)

34 Functions 1 Exploration & browsing: fast and controlled by Relevance Feedback 2 Search: fast reaching the target 3 Category learning, image groping and annotation: fast and well controlled by Relevance Feedback and Active Learning

35 Visual Data Mining Interactive exploration and analysis of very large, high complexity, and non-visual data sets. The analyst can navigate in a multi-modal space and interactively search and refine relevance criteria to explore patterns of interest in the data, and perform their in-depth analysis. Provides ensample views (projections) of the entire archive

36 Image Archive View Ontology and Knowledge Graph Jain IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 2, FEBRUARY

37 Visual Data Mining Representation of the data in the 3D space using a Laplacian eigenmap Semantically consistent groupings appear inside the data

38 Rapid Mapping: Sources of Information and the Product EO Image Maps Maps GIS Text Manual integration Reports Statistics

39 Rapid Mapping

40 Rapid Mapping

41 Interactive Map

42 Interactive Map City

43 Interactive Map City

44 Interactive Map City Industry

45 Interactive Map City Industry