Binary Hierarchical Classifier for Hyperspectral Data Analysis

Transcription

1 Binary Hierarchical Classifier for Hyperspectral Data Analysis Hafrún Hauksdóttir A intruduction to articles written by Joydeep Gosh and Melba M. Crawford Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 1/16

2 The Articles Hierarchical Fusion of Multiple Classifiers for Hyperspectral Data Analyses A Hierarchical Multiclassifier System for Hyperspectral Data Analysis Investigation of the Random Forest Framwork for Classification of Hyperspentral Data Best-Bases Feature Extraction Algorithm for Classification of Hyperspectral Data Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 2/16

3 Hyperspectral Data Hyperspectral sensor simulataneously acquire information in hundreds of bands. Hyperspectral image is a three dimensional array I(x,y,d) where (x,y) denotes a pixel location in the image and d denotes a spectral band. Analysis of hundred of simualtanous channel of data necessitates the use of either feature selection or extraction. Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 3/16

4 Feature Selection/Extraction Goals Reduce number of samples required to train the classifier Reduce computational complexity Improve classification accuracy Improve generality Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 4/16

5 Key Desired Properties of Extractors 1. Class dependent, different subsets of classes are best distinguished by different feature sets Hierarchical Classifier 2. Exploit band ordering Best bases or preprocessing to combine highly correlated adjecant bands 3. Transformations should maximize discrimination among classes Fisher discriminant Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 5/16

6 Hierarcical clustering Family Tree of Classes Tree constructed sequentially based on series of binary questions and splitting rule which maximizes decrease in impurity of parent and child nodes Maintains natural groupings, alike or similar classes tend to end up as siblings in the decision tree. CART Minimizes the impurity of the nodes using a sequence of binary tests. Small sample size produces enormous trees if output space is large. BHC Minimizes the entropy within nodes and maximizes the covariance between nodes. Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 6/16

7 Binary Hierarchical Classifier Classifier specific to each internal node (Fisher discriminant, SVM) Feature extraction/selection in each internal node Cleaf nodes, C-1 internal nodes Top down tree constructed via deterministic simulated annealing Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 7/16

8 Fisher s Discriminant The goal is to maximize the discriminant: τ(w) = wt Bw w T Ww where W is the covariance matrix within sub-class, W = P(ω α )Σ α + P(ω β )Σ beta and B is the covariance between sub-classes B = (µ α µ beta )(µ α µ beta ) T. The Fisher projection that maximizes the discriminant is given by w = W 1 (µ α µ β ) Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 8/16

9 Selection techniques Exhaustive search Guarantee optimal solutions Heuristic methods Typically produce sub-optimal solutions Developed to reduce computational complexity Traditionally implemented via sequential forward selection or backward elimination Various simulated annealing and genetic algorithms and varius other algorithms Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 9/16

10 Example Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 10/16

11 Band combining A band combining step is performed on highly correlated, spectrally adjecent bands prior to the partitioning of meta-classes. Used to reduce the number of inputs relative to the number of training data points. Bands are aggregated until a user defined radio, R, between the number of training samples for the respective meta-classes and input dimension is achieved. Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 11/16

12 RF RS and BB BB: best bases Used to first consturct the hierachy Highly correlated layers are combined to maximize the discriminant between classes RS: Random subspace Then random subspace sampling is performed at each node of the tree. Discriminant vector is constructed for each random subspace. RF: Random forest Random forest extends the random subspace by incorporating random subspace feature selection in the actual development of the tree. Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 12/16

13 Random Forrest of BHC trees Selecting sub-samples of original data, creating classifiers, and developing a classifier for each sample. Combine results from individual classifiers Perform poorly for extremely small sample sizes as ensemble methods cannot overcome lack of diversit Decision boundaries for individual classifiers typically simple Generalization typically superior if the classifiers are diverse Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 13/16

14 Comparison Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 14/16

15 The Soft Part BHC uses GAMLS algorithm to split each node into meta-classes which are seperated by Fisher distance. GAMLS is a determanistic simulated annealing algorithm and can be seen as a generalisation of the fuzzy c-means clustering. The pairwise classifier at node n can be either a soft or a hard classifier. The soft classifier generates the posterior probability P(Ω 2n x, Ω n ) and P(Ω 2n+1 x, Ω n ) where Ω n is the parent node and Ω 2n and Ω 2n+1 the children. The hard classifier maps the input x into one of the two class labels corresponding to the two child nodes of class Ω n Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 15/16

16 Binary Hierarchical Classifier for Hyperspectral Data Analysis Hafrún Hauksdóttir A intruduction to articles written by Joydeep Gosh and Melba M. Crawford Binary Hierarchical Classifierfor Hyperspectral Data Analysis p. 16/16