Session 2 A virtual Observatory for TerraSAR-X data 2nd User Community Workshop Darmstadt, 10-11 May 2012 Presenter: Mihai Datcu and Daniela Espinoza Molina (DLR)
This presentation contains contributions from: Quartulli, M; Datcu, M; 2012, Accelerated Compression Based CBIR, in preparation Dumitru, C. O. ; Datcu, M, TN KLAUS, Dec. 2011 Blanchart, P; Ferecatu, M; Datcu. M; 2011, Cascaded Active Learning for Object Retrieval using Multiscale Coarse to Fine Analysis, in the proceedings of IEEE Conference on Image Processing (ICIP), Bruxelles, 2011 Shiyong Cui; Datcu, M.; 2011, Coarse to fine patches-based multitemporal analysis of very high resolution satellite images, Multi-Temp, Page(s): 85-88 Popescu, A. A.; Gavat, I.; Datcu, M.; 2011, Contextual Descriptors for Scene Classes in Very High Resolution SAR Images, IEEE GRSL Bratasanu, D.; Nedelcu, I.; Datcu, M.; 2011, Interactive Spectral Band Discovery for Exploratory Visual Analysis of Satellite Images, IEEE JSTARS, Espinoza Molina, D.; Gleich, D.; Datcu, M.; 2011, Evaluation of Bayesian Despeckling and Texture Extraction Methods Based on Gauss Markov and Auto-Binomial Gibbs Random Fields: Application to TerraSAR-X IEEE TGRS (IEEEXPLORE early access) Birjandi, P.; Datcu, M.; 2010, Multiscale and Dimensionality Behavior of ICA Components for Satellite Image Indexing, IEEE GRSL, Volume: 7, Issue: 1, Page(s): 103 107 Blanchart, P.; Datcu, M.; 2010, A Semi-Supervised Algorithm for Auto-Annotation and Unknown Structures Discovery in Satellite Image Databases, IEEE JSTARS, Volume: 3, Issue: 4, Part: 2, Page(s): 698 717 Gueguen, L.; Datcu, M.; 2008, A Similarity Metric for Retrieval of Compressed Objects: Application for Mining Satellite Image Time Series IEEEE Transactions on Knowledge and Data Engineering, Volume: 20, Issue: 4, Page(s): 562-575 Daschiel, H.; Datcu, M.; 2005, Design and evaluation of human-machine communication for image information mining, IEEE Transactions on Multimedia, Volume: 7, Issue: 6, Publication Year: 2005, Page(s): 1036 1046 Datcu, M.; Seidel, K.; 2005, Human-centered concepts for exploration and understanding of Earth observation images IEEE TGRS, Volume: 43, Issue: 3, Page(s): 601 609 Daschiel, H.; Datcu, M.; 2005 Information mining in remote sensing image archives: system evaluation IEEE TGRS, Volume: 43, Issue: 1, Page(s): 188 199 Datcu, M.; et al.; Information mining in remote sensing image archives: system concepts IEEE TGRS, Volume: 41, Issue: 12, Part: 1, Page(s): 2923-2936
The Content of VHR EO Archives EO Archive = O {1 000 000} EO Products EO product = image + xml file image = 30 000 x 30 000 pixels pixel = 16/32 /1000s bits, real/complex/multiband xml file = a lot about the imaging mode, orbit, position, timing TOTAL = many PetaBytes
Sensor resolution & Information Content 10m 4m 2m 1m 4 objects > 100 objects
Spatial Context & Information Content Left: Example of typical ATR image (MSTAR database: T-72 battle tank): one isolated target on homogeneous background. Middle and right: HS TerraSAR-X examples of industrial/urban sites; middle patch: broad view of a harbor; right patch: close view of a district with skyscrapers. Conversely to ATR, we no longer have isolated objects, but rather complicated arrangements of targets. The number of individual objects is so high that we should work with Scene Classes instead of specific Objects
Spatial Context & Information Content At meter resolution, if context is ignored (the analysis windows being used are relatively small as compared to the object scale), very different scene classes may be confused. The example shows a bridge (left) and buildings (right) which are comprehensible only when the window size is sufficiently large to incorporate relevant context. Otherwise, both scene classes look similar and can be confused.
Scene Categories & Information Content 1 HS TerraSAR-X Scene = up to10 000 image patches (100 x 100 m)
Scene Evolution categories & Information Content Agriculture Forest House Lake Road
Multispectral and Spatial & Information Content WorldView 8 bands, 2 meter: spectral classes Spatial categories
Topology & Information Content A symbolic language?
Semantics & Information Content Define semantic annotation models
Product Metadata & Information Content XML file contains information about productcomponent, annotation, imagedata, missioninfo, acquisitioninfo, sceneinfo, etc Ontology queries Extraction of Metadata from XML annotation file
Number of acrchived scenes Number of archived scenes Number of archived TerraSAR-X scenes by country and incidence angle Spain Finland 40 3 30 2 20 1 10 0 <20 20 - <30 30 - <40 40 - <50 50 - <60 Mid-range incidence angle 60 and larger 0 <20 20 - <30 30 - <40 40 - <50 50 - <60 Mid-range incidence angle 60 and larger The figure shows the number of archived TerraSAR-X scenes acquired over Spain and Finland at specific incidence angles. Almost half of the nearly 60 acquisitions over Spain were acquired with a mid-range incidence angle of between 30 and 40. Of a total of only six acqisitions over Finland two were acquired at each of the three 10 incicence angle ranges between 20 and 50.
The data archive The data access
Few Challenges The data and information processing performance The scalability issues need more detailed analysis as the volume, diversity and heterogeneity of data is continuously growing The end to end IIM process, i.e. the communication of the EO and related data content to the EO user needs special new solutions Fill the instrument gap also the semantic gap Provide an enriched index for the product catalogue Provide a base for a new type of interactive value added EO products Design the image and data representation for further auto-annotation and data mining processing.
TELEIOS Architecture
TELEIOS Architecture
Concept of Knowledge Discovery Query, Data Mining & Knowledge Discovery Data Sources Data Model Generation DBMS Visual Data Mining Users Interpretation Understanding
Functional Concepts Browsing: Navigation in the archive for discovering things. The direction, and goals of the navigation are changing based on the content of the queries. It gives a global image of the archive content. Searching: Navigation in the archive with a clear objective, the results of each query is used for directing the navigation. Annotation: Creation of new catalogue (index) entries for the archive. The result of browsing or searching may be used to annotate the results.
Exploration/Query Concepts Numeric and predefined queries: classical geographic position, time, senor type, etc. The output is a list of products in the specified parameters Semantic an numeric queries: by use of supporting frames a complex question can be formalized. The output is a list of products explained by the query sentence. Image similarity queries: search of similar images with a given example. The output is a ranked list of images. KDD: interactive search supported by relevance feedback mechanisms. The output is the desired product, the related products (containing similar information), semantically explained, or new, previously unknown information. Visual Data Mining: Visual exploration of the whole archive. The output are outliers, interesting groups, associations, etc.
Queries using numerical descriptors and predefined keywords These state-of-the-art queries are based on basic image metadata such as coordinate systems, acquisition time, type of product, etc. The system allows functionalities such as navigations through the globe map, listing the available products in the image database, and placing an order for the desired products
Agenda (2/2)
Semantic and numerical queries Semantic Labels: The user can enter a simple label in the form of text or select an item from the available labels in the catalogue to perform the query (as for example forest ). We observe that these labels are pre-defined labels previously obtained as results of the image annotation process. Query language and Ontologies: Here, the queries can be performed either using only semantics or semantics and spatial content in the form of text or numerical entries. The queries based on spatial content are performed by using the image descriptors. The query language can rely on a query template in order to avoid mistyping and helping the user. Query Builder enabling queries by using semantics, topological relations, different operators, and numerical descriptors
Agenda (2/2)
Query by example The knowledge discovery starts with a query based on image content as example. The user selects an image; later, its content is passed as example for the query, the system activates the data mining methods and retrieves the results The user selects the content of his interest by clicking on the image and the system retrieves the relevant images ranked according to some metrics
29
KDD, Active Learning and Auto-annotation KDD: interactive search supported by relevance feedback mechanisms. The output is the desired product, the related products (containing similar information), semantically explained, or new, previously unknown information learn the targeted image category as accurately and as exhaustively as possible minimize the number of iterations in the relevance feedback loop sys Query image user
Warning: not all categories can be learned!
Dec. Agenda (2/2)
Functions 1 Exploration & browsing: fast and controlled by Relevance Feedback 2 Search: fast reaching the target 3 Category learning, image groping and annotation: fast and well controlled by Relevance Feedback and Active Learning
Visual Data Mining Interactive exploration and analysis of very large, high complexity, and non-visual data sets. The analyst can navigate in a multi-modal space and interactively search and refine relevance criteria to explore patterns of interest in the data, and perform their in-depth analysis. Provides ensample views (projections) of the entire archive
Image Archive View Ontology and Knowledge Graph Jain IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 2, FEBRUARY 2008 36
Visual Data Mining Representation of the data in the 3D space using a Laplacian eigenmap Semantically consistent groupings appear inside the data
Rapid Mapping: Sources of Information and the Product EO Image Maps Maps GIS Text Manual integration Reports Statistics
Rapid Mapping
Rapid Mapping
Interactive Map
Interactive Map City
Interactive Map City
Interactive Map City Industry
Interactive Map City Industry