Nonparametric Multiscale Multimodal Models for Detection/Recognition John Fisher & John Reif Eagle Eye Software (919)

Transcription

1 Rochester Y October 999 onparametric Multiscale Multimodal Models for Detection/Recognition John Fisher & John Reif Eagle Ee Software (99) reif@cs.duke.edu

2 Multiscale Multimodal Models for ATR Rochester Y October 999 Project Summar Objectives Demonstration of nonparametric multiscale multimodal statistical models for image analsis. ATR/AFE Technologies Algorithms relies on registered multi-modal data Demonstrations use the Positive Sstems RGB and near IR imager Output Product target map listing of targets and probabilities of occurrence John Fisher & John Reif Eagle Ee Software Page of 0

3 Multiscale Multimodal Models for ATR Rochester Y October 999 Project Concept Multimodal image registration (an admittedl challenging task) is onl the first step in the processing of such data. The task of higher-ordered processing (e.g. detection/classification) remains. We present an application of multi-modal multiscale statistical models for classification. John Fisher & John Reif Eagle Ee Software Page 3 of 0

4 Multiscale Multimodal Models for ATR Rochester Y October 999 Project Concept Adopting a multiscale snthesis view of multimodal images we can infer a statistical model of an image class which captures both the structure and variabilit of man-made objects. Using nonparametric multiscale statistical models originated b De Bonet and Viola (Teture Recognition Using a on-parametric Multi- Scale Statistical Model CVPR 998). Etensions to Snthetic Aperture Radar and ATR (Fleible Histograms: A Multiresolution Target Discrimination Model De Bonet Viola Fisher 998). Combination of nonparametric densit estimator and multiscale analsis pramids. Allows for natural fusion of joint measurements of multiple modalities. John Fisher & John Reif Eagle Ee Software Page 4 of 0

5 Multiscale Multimodal Models for ATR Rochester Y October 999 Computational Flow Diagram Fusion/ data reduction registration or joint measurement John Fisher & John Reif Eagle Ee Software Page 5 of 0 Multiscale Analsis Inference Fleible Histogram Model Detection Map

6 Multiscale Multimodal Models for ATR Rochester Y October 999 Fusion Data/Reduction Sample joint piel statistics over diverse region RGB IR Eigenvalue decomposition over multi-dimensional piels Keep first two components (85% energ) well approimated b Letterman Medical Comple John Fisher & John Reif Eagle Ee Software Page 6 of 0 e = / 6 * ( R + G + B) + / * IR e = / * IR - / 6 * ( R + G + B)

7 Multiscale Multimodal Models for ATR Rochester Y October 999 John Fisher & John Reif Eagle Ee Software Page 7 of 0 At each location a parent vector is etracted. This vector consists of the multiresolution wavelet decomposition at that location. V()={ } coarse fine Multiresolution parent vector ( ) ( ) ( ) ( ) [ ø æ ø æ ø æ ø æ ø æ ø æ = M M M F F F F F F F F F V K K M K r Fleible Histogram Model

8 Multiscale Multimodal Models for ATR Rochester Y October 999 Fleible histogram I MODEL B ()= 8 I MODEL parent vector B measuring the frequenc with which locations with similar parent vectors occur a fleible histogram is etracted. John Fisher & John Reif Eagle Ee Software Page 8 of 0

9 Multiscale Multimodal Models for ATR Rochester Y October 999 The histogram for the image measured with respect to the model is compared to the the histogram for the model measured with respect to itself. B()= 8 Discrimination via histogram comparison I MODEL I MODEL c = S (B-B ) /B B ()= 3 John Fisher & John Reif Eagle Ee Software Page 9 of 0 I TEST

10 Multiscale Multimodal Models for ATR Rochester Y October 999 The frequenc of locations in the image which have a parent structure whose components are each within a threshold of the parent vector of some location in the model is given b: c r R V r = ' '» R V r B r r = Q ' Z ' ' where [ B( ) - B' ( ) ] B( ) ( ( ) ( ) ) - T - V I -V I' ' D Q ( a) = ' ' if a > 0 0 otherwise A difference measure is calculated b taking chi-square difference between each such frequenc count in the model and test image which approimates of the Kullback-Leibler divergence. ( ( I ) V( I ' ' ) r ( ( I ) V( I' ' ' ) Similarit can be measured b simpl negating the distance.» D ( p q) John Fisher & John Reif Eagle Ee Software Page 0 of 0 S = -c

11 Multiscale Multimodal Models for ATR Rochester Y October 999 SAR Eample [De Bonet Viola Fisher] Eample images Samples from fleible histogram John Fisher & John Reif Eagle Ee Software Page of 0

12 Multiscale Multimodal Models for ATR Rochester Y October 999 Model Based Detection Optical (RGB) ear IR John Fisher & John Reif Eagle Ee Software Page of 0 Positive Sstems Data

13 Multiscale Multimodal Models for ATR Rochester Y October 999 Model Based Detection PCA Fusion John Fisher & John Reif Eagle Ee Software Page 3 of 0

14 Multiscale Multimodal Models for ATR Rochester Y October 999 Model Based Detection Multi-scale feature etraction John Fisher & John Reif Eagle Ee Software Page 4 of 0

15 Multiscale Multimodal Models for ATR Rochester Y October 999 Model Generation fusion John Fisher & John Reif Eagle Ee Software Page 5 of 0 Q. How do we generate models? A. In nonparametric framework models are derived from eamples. The eample data capture the spatial coherence while the histogram captures the joint scale and modal properties.

16 Multiscale Multimodal Models for ATR Rochester Y October 999 Matched Filter matched filter template formed from fused image Q. Is it the features? A. O!! Template formed from features performs much worse. Matched filter is not robust to edges or changes in orientation. John Fisher & John Reif Eagle Ee Software Page 6 of 0

17 Multiscale Multimodal Models for ATR Rochester Y October 999 Fleible Histogram Detections fusion multi-scale features John Fisher & John Reif Eagle Ee Software Page 7 of 0

18 Multiscale Multimodal Models for ATR Rochester Y October 999 Comparison fleible histogram matched filter Both methods perform similarl on buildings with the same orientation. The fleible histogram approach however does a much better job on buildings with orientations different from the original model. John Fisher & John Reif Eagle Ee Software Page 8 of 0

19 Multiscale Multimodal Models for ATR Rochester Y October 999 Model Based Detection Precision Recall - rank ordering of detections Matched filter Fleible Histogram John Fisher & John Reif Eagle Ee Software Page 9 of 0

20 Multiscale Multimodal Models for ATR Rochester Y October 999 Further Investigation Testing with more modalities Landsat Radar Registration Robustness how does performance degrade with precision of registration algoritms can robustness to misalignment be built into the test (via multiscale representation) Registration Refinement can learned joint statistical models be used to refine an eisting registration algorithm John Fisher & John Reif Eagle Ee Software Page 0 of 0