TREE SPECIES CLASSIFICATION USING RADIOMETRY, TEXTURE AND SHAPE BASED FEATURES

Transcription

1 TREE SPECIES CLASSIFICATIO USIG RADIOMETRY, TEXTURE AD SHAPE BASED FEATURES Maria S. Kulikova 1, Meena Mani 1, Anuj Srivastava 2, Xavier Descomes 1, Josiane Zeruia 1 1 Ariana Research Group, IRIA/I3S Sophia Antipolis Ceex France firstname.lastname@inria.fr, mmani@ucla.eu to join Meena Mani 2 Department of Statistics Floria State University Tallahassee, FL anuj@ani.stat.fsu.eu ABSTRACT We consier the prolem of tree species classification from high resolution aerial images ase on raiometry, texture an a shape moeling. We use the notion of shape space propose y Klassen et al., which provies a shape escription invariant to translation, rotation an scaling. The shape features are extracte within a geoesic istance in the shape space. We then perform a classification using a SVM approach. We are ale to show that the shape escriptors improve the classification performance relative to a classifier ase on raiometric an textural escriptors alone. We otain these results using high resolution Colour InfraRe (CIR) aerial images provie y the Sweish University of Agricultural Sciences. The image viewpoint is close to the nair, i.e. the tree crowns are seen from aove. 1. ITRODUCTIO Interest in applying remote sensing to forests goes ack to the 1920s when aerial photographs were first use to assess forest inventory. Remote sensing is now wiely employe in forest management where the aerial information is comine with measurements taken on the groun to stuy the ioiversity of the forest ecosystem. The methos evelope in forest image processing aim to facilitate the task of forest inventory an assessment. The most useful parameters otaine from aerial images an groun measurements are ensity (of planting), age of trees, stem volume, tree species composition, an information such as iotops an haitats that have ecological value. In orer to otain information aout the iversity of the forest species an stem volume, the classification of tree crowns into species is necessary. Prior to classification, we nee to segment the image into iniviual tree crowns. Segmentation techniques such as template matching, ege etection an others that have een employe for this application are iscusse in [5], [13]. A few approaches have een propose to classify the trees into species. One metho is the Signature Generation Process where for every crown extracte, a class of signatures is create from the multispectral ata of initial image, (cf [10]). A likelihoo maximisation technique is use to lael the crown. Some crowns with signatures too far from a succesful match remain unclassifie. In another stuy [6], tree crowns are manually elineate to avoi ias ue to a etection. 50 trees for each class verifie against the groun truth are selecte. 7 parameters such as the multispectral average of pixels in the crown (average on each of the ans) an the illuminate part of the crown, as well as the multispectral value of the tree crown (most lighte pixel) are then compute. Spectral signatures of a crown or a region within a crown are evelope y The work of the first author has een partly supporte y a grant of the French Government. This work has een conucte within the IRIA ARC Moe e Vie joint research program ( an IRIA/FSU associate team SHAPES ( comining means an covariance patterns. The istinct characteristics allow us to regroup elineate crowns in foreste populations [7]. There are some limitations with this metho ue to the close spectral signature of species like the re cear an the fir for instance [8]. In [4], a classification ase on reflectance is use to separate the conifers from the eciuous trees. The internal structure an shaing within a crown offer other ifferentiating criteria. One such measure, the proportion of re an clear pixels to the total numer of pixels, can ientify irch trees. Another helps ifferentiate aspens from spruces. A hierarchy of criteria is set forth to classify the crowns. Classification accuracy using this strategy was 75%. Raiometry an texture analysis have een use extensively in remote sensing applications. In this paper, we incorporate information otaine from stuying tree crown shapes to improve classification performance. As in the Erikson [4] stuy, the tree classification is performe on the four most prevalent species of in Sween: orway spruce, Scots pine, irch, an aspen. Two of these species are coniferous while the other two are eciuous. We select 48 crowns (12 per class) from the high resolution CIR images (Fig.1). Their contours, represente y a set of points, are then extracte in orer to stuy their shapes. The support vector machine (SVM), a supervise learning metho, was chosen for the classification. An important property of this classifier is that uring the training process, only a small suset of the training set vectors are selecte as support vectors. This reuces the computational cost an provies etter generalization, so that, for instance, when new samples far from the ecision ounary are introuce, the existing support vectors remain unchange. Figure 1: An example of one of the images use in this work (resolution 3 cm), c Sweish University of Agricultural Sciences.

2 2. SUPPORT VECTOR MACHIE w.x + = 1 Optimal Separating Hyperplane SVMs are a set of supervise learning methos of classification an regression, which consists in fining the maximum separation (margin) etween classes using a set of oservations calle the training ata. SVMs are also calle maximum margin classifiers since they minimize the empirical classification error an maximize the geometric margin simultaneously [15]. y i = 1 x l ξ l Support Vectors 2.1 Linear SVM Separale case Given a training ata (x i,y i ), i = 1,...,, where x i R m an y i { 1,1}, that enotes to which class x i elongs, SVM looks for the Optimal Separating Hyperplane that maximizes the istance etween the closest training samples of the two classes calle also the margin. The iviing hyperplane is efine as following w x + = 0. The vector w is a vector normal to the hyperplane an is the offset parameter allowing us to increase the margin. So the classifier is given y f : x R m sign(w x+) { 1,1}, i.e. all the training ata satisfy the following constraints: { w xi + 0 if y i = +1 w x i + 0 if y i = 1 That coul e escrie y a set of inequalities: y i (w x i + ) 1, i (1) To fin the hyperplane which gives the maximum margin 2/ w (for etails see [2]) we shoul minimize w 2, suject to constraints (1). This leas to the following quaratic optimization prolem: w 2 min (w,) 2 By introucing the Lagrange multipliers λ i,i = 1,..,, one for each inequality constraints (1), the prolem ecomes ual [2]: max (L(λ) = λ i λ 1 2 λ i λ j y i y j x i x j ), j=1 suject to λ i y i = 0, 0 λ i, i, which is a convex quaratic optimization prolem suject to linear constraints. Thus, the solution is given y w = λ iy i x i an = y i w x i, for i : λ i 0. The classification function ecomes: onseparale case f (x) = λ i y i x i x + For nonlinearly separale ata, slack variales ξ i an a regularization parameter C are introuce to eal with misclassifie samples, i.e. to relax the constraints (see Fig. 2). By introucing them into the optimization prolem, we otain: w 2 min (w,) 2 +C ξ i, suject to : y i (w x i + ) 1 ξ i, ξ i 0, i. x i margin = 2 w ξ k x k w.x + = 1 Figure 2: SVM Classifier. y j = 1 This quaratic prolem using the Lagrange multipliers ecomes: max (L(λ) = λ i λ 1 2 λ i λ j y i y j x i x j ), j=1 2.2 onlinear SVM suject to λ i y i = 0, 0 λ i C, i. For the cases where the ecision function is not a linear function of ata, the nonlinear classifier was create y applying the kernel trick (originally propose y Aizerman) to maximum-margin hyperplanes (cf [1]). The resulting algorithm is formally similar, except that every ot prouct x i x j is replace y a non-linear kernel function K(x i,x j ) = Φ(x i ) Φ(x j ), with Φ : R m H an H eing an Eucliean space generally of a higher imension. This allows the algorithm to fit the maximum-margin linear hyperplane in the transforme H. Thus, the classification function ecomes: f (x) = λ i y i K(x i,x) +. In regar to kernels, there exists a mapping Φ, if K(x i,x j ) satisfy the Mercer s conition [16]. 3. FEATURE CREATIO AD CLASSIFICATIO RESULTS As mentione in the introuction, 48 tree crown contours were selecte. The contours were carefully elineate manually to preserve important tree crown shape information. The SVM classification is performe using a Gaussian kernel K(x,x ) = exp( x x 2 ). Since 2σ 2 the ataase is quite small, for each experimental run, 50% of the samples were picke at ranom to form the training set. First, we performe the classification using only raiometric attriutes. We then a texture features an finally, we inclue the shape escriptors. After each step, we calculate the performance. To evaluate the performance, the average performance, P, of a set of experiments is compute. 5% of the values at the high an low en of the performance scale are exclue from this calculation. P can e expresse as e P = ( w P i P w P )/( e w ). w=1 =1

3 P i = c / is the performance of experiment, e the numer of experiments, c the numer of correctly classifie trees, the numer of trees an w an the numers of the worst an the est results respectively. The maximum performance P max is also given (from which we can get the est comination for training an test sets). Then, we present the confusion matrix for the est performance run. A confusion matrix is a visual representation of actual versus preicte classifications. 3.1 Raiometry ase features For vegetation an lan use monitoring, CIR or false-colour film offers a richer set of information than natural colour film. The false colours that escrie the three channels are Re(IR), Green(Re), Blue(Green), [12]. CIR photography is commonly use to iscriminate live healthy trees from ying vegetation. In our stuy, CIR allows us to istinguish the conifers from eciuous trees. As was pointe out in [4], the 4 classes of trees - aspen, irch, spruce an pine - can e easily ientifie as eciuous or coniferous from first orer statistics (the mean an stanar eviation, compute from the histogram of pixel intensities on the image). This is ecause eciuous trees reflect a sustantially greater percentage of infrare light. Classification ase exclusively on these escriptors gives low performance with the average performance eing P = 0.54 an the maximum, P max = The confusion matrix for one of the est values of P after repeate classification runs is shown elow a c where (a)aspen, ()irch, (c)spruce, ()pine. From this matrix, we can see that there are a sustantial numer of false positives. For example, 50% of aspens are classifie as irch, while 33% of the irch trees are classifie as either aspen or spruce. This classifier is especially weak in ifferentiating etween coniferous an eciuous classes, i.e. etween aspen an irch or etween spruce an pine. 3.2 Texture ase features To further istinguish within the eciuous an coniferous classes, we performe texture analysis using gray level co-occurrence matrices (GLCM), which is well aapte for characterising microtextures. A co-occurrence matrix is a two-imensional quantitative representation of spatial relationship [9]. Let {I(x,y), 0 x 1, 0 y 1} enote a image with G gray levels as escrie in [3]. The G G gray level co-occurrence matrix P is efine as is a measure of the homogeneity of the texture. If the gray level transitions are roughly uniformly istriute, the energy has a low value. Conversely, textures which have ominant gray level transition moes have higher energy values. The contrast feature i (i j) 2 P (i, j) j is a measure of local variation present in an image. We selecte this feature to exploit the istinctive features present in the four types of crown surface. Spruce trees, for example, isplay a raial pattern while aspen have ranom light an ark regions. Since these two features are inepenent of the size an shape of the crown surface, they represent pure texture characteristics. The evaluation of classifier performance allows us to etermine the optimal couple of parameters ( an irection). In practice we have chosen = 1 an 135 egree irection. By incorporating these two texture features into the classifier, we were ale to separate the trees into 4 classes with the average performance P = 0.71 an the maximum performance P max = The confusion matrix for one the est values of P after repeate classification runs is shown elow: We can see that the texture information allows to improve the classification results for esciuous, especially for aspens (a). 3.3 Shape ase features To further improve the previous results, we propose to stuy tree crown shapes in a so calle shape space, using the representation of planar shapes y their angle functions, cf [11]. We consier the tree crown contours as continuous an close curves in R 2. The representation of such curves in the shape space is invariant to rigi rotation an translation, an to scaling in R 2. Let us efine these properties. Curves α = (α 1 (s),α 2 (s)) are parametrize y arclength s, where α : R R 2 with perio 2π satisfying conition of constant spee along the curve α (s) = 1, s. We can write α (s) = e jθ(s), where θ : R R an j = 1 associating C with R 2. θ is calle irection function or angle function. For every s, θ(s) gives the angle etween α (s) an the positive ascissa x, cf Fig.3: a c P (i, j) = ((r,s),(t,v)) : I(r,s) = i,i(t,v) = j. The entry (i, j) of matrix P is the numer of occurrences of the pair of gray levels i an j which are a istance = (x,y) apart. Since is a isplacement vector, (r,s), (t,v), such that (t,v) = (r + x, s + y), an. is the carinality of a set. GLCMs are a compact representation of pairs of pixel values in relation to each other. They are an example of the secon orer statistics as efine y Julesz an several useful features can e compute from them. We generate 9 such matrices for each tree, each matrix representing a ifferent irection or istance. Two texture features, energy an contrast were extracte from the GLCMs. The energy term, given y i P 2 (i, j) j Figure 3: Examples of shapes an corresponing angle functions. On the Fig.4 we can see more complicate graphs of angle function representing some crowns.

4 Figure 4: Examples of crowns of four species, their contours, corresponing angle functions θ, an θ = θ θ 0 at the ottom. (a)aspen, ()Birch, (c)spruce, ()Pine. We assume L 2 to enote the space of real functions R R of the perio 2π an square integrale on [0,2π] with scalar prouct f 1, f 2 = 0 f 1 (s) f 2 (s)s an f = f, f. Let θ(s) e the angle function of a planar shape. For the unit circle the function is θ 0 (s) = s. For any other close curve the angle function can e written as θ = θ 0 + f, where f L 2. The space θ 0 + L 2 is an affine space the elements of which are ifferent y an element of L 2. The properties of curve invariance mentione aove are guarantee y the following conitions (cf [11]): 1. The prolem of scaling can e simply resolve y fixing the length of the curve, for instance y 2π. Let c e a close contour, then we otain: s = s = 2π; c 0 2. An aition of a constant to the angle function θ is equivalent to the rotation of the curve in R 2. To guarantee the invariance of the curve to this action, we eal with the functions θ, the mean values of which are equal to a constant on [0,2π]. So, let 1 θ(s)s = π, where constant π is chosen to inclue the ientity function θ 0 in the restricte set, since 1 2π θ 0 (s)s = 1 ss = π; 3. Finally, to e close, the curves must satisfy following conition: exp( jθ(s))s = 0. 0 We efine C θ 0 + L 2 as the set of all the elements of θ 0 + L 2 satisfying the conitions 1-3 escrie aove. Or more formally, efine the map φ = (φ 1,φ 2,φ 3 )) : (θ 0 + L 2 ) R 3 : φ 1 = 1 θ(s)s φ 2 = 1 cos(θ(s))s φ 3 = 1 sin(θ(s))s. From now on, we can efine C as C = φ 1 (π,0,0), which is calle the pre-shape space, ecause the same curve with ifferent initial points (s = 0) correspon to ifferent elements of the space C. But we will eal with the shape space S, where such curves are represente y one element of S efine y C/D, where D is the unite circle. We are going to escrie the tree features create using such a representation of the crown contours. To o it some tree crown shape analysis was one to etermine the information that coul allow us to recognize the species. Let us look at Fig.4. We can see that the aspens have an irregular structure, the convexities/ranches sticking out of the oy of crowns. The ranches of spruces are more regular an have a more raial irection. The irch an pine crown contours are more roun, where irches are the smoothest. The first feature, chosen to e incorporate in the classifier, is a istance to a circle (θ,θ 0 ), which is compute using the geoesic on the shape space (see [11] for etails). For the given shapes θ i,i = 1,...,M, we perform a vector of istances to a circle : v c (θ) = {(θ i,θ 0 ), i = 1,2,...,M} R, where M is the numer of trees. We also inclue geometrical escriptors ase on θ = θ θ 0. Firstly, we translate the property that some species have more regular structure of the crowns than others, y a measure of contour elasticity: v e ( θ) = θ(s) 2 s, 0 v e ( θ) = {v e ( θ i ), i = 1,2,...,M} R. In regar to spruces, generally they have ig ranches/convexities an not numerous in comparison with the convexities of irch crown. This criterion is reflecte y the angle function uner the form of the local maxima numer. v ( θ) =, v ( θ) = {v ( θ i ), i = 1,2,...,M} R. Then, the pine crown shape is quite close to that of irch ut certain roughnesses are a little it igger, some ranches sticking out. Thus we can qualify the crown contour irregularities ue to the ranches, leaves an shaows: v µ ( θ) = µ = 1 n n θ(s k ), k=1 v µ ( θ) = {µ i, i = 1,2,...,M} R, where s k = k t=1 c t, c t = p t+1 p t chor lengths of contour an p t R 2,t = 1,...,n orere collection of points approximating the contours, v Var ( θ) = Var = 1 n n 1 (µ i θ(s k ) ) 2, k=1 v Var ( θ) = {Var i, i = 1,2,...,M} R. Finally, we use the ensity function of the angle function in orer to have information aout the quantity of ig an small convexities of the crown contour. For that, we calculate the histograms of θ, h θ (x) = car{ θ (s) = x,s [0,2π],x R}, an then Eucliean istances (h 1 (x),h 2 (x)) = < h 1 (x),h 2 (x) >. Then we create the following features associate to each shape, one istance for every class l. Such a istance is calculate as the istance to the nearest element of the class l: l min (h i) = min{(h i,h j )} j=1,... m l R,

5 v ( θ) = { l min (h i), i = 1,2,...,M, l = 1,..., l } R, where l is the numer of classes an m l is the numer of shapes of each class in the training set. ow we are forming a long vector using iniviual features an apply SVM on that feature vector. Otaine results: the average performance P = an the maximum of performances P max = An example of confusion matrix is given as follows: Shape ase features allow us to improve the ientification of species within the conifers an eciuous trees. 4. COCLUSIO In this paper we show that the performance of a classifier ase on conventional spectral an texture characteristics can e improve y incluing shape escriptors in the feature set. By incorporating shape features, the classification performance mean improve y aout 4% for 6 samples per class while the maximum performance was 87.5%. To create the new escriptors, the shapes of crowns were analyse using the angle function representation. Such representation allowe us to preserve the tree crown characteristics that associate it with one class or another. The limits impose y the ataase size i not give us much freeom to experiment with the training to test sample size ratio. We i, however, oserve that the performance mean tene to increase when samples were ae to the training set (see fig. 5). These results are encouraging ut they nee to e valiate on a larger numer of images an tree crowns. Figure 5: Performance means an maxima. Future work will consist in automatic extraction of tree crowns using shape information. a c REFERECES [1] B. E. Boser, I. M. Guyon an V.. Vapnik, A Training Algorithm for Optimal Margin Classifiers, in Fifth Annual Workshop on Computational Learning Theory, Pittsurg, [2] C. J. C. Burges, Tutorial on Support Vector Machines for Pattern recognition. Kluwer Acaemic Pulishers, Boston. pp [3] C. H. Chen, L. F. Pau, P. S. P. Wang, The Hanook of Pattern Recognition an Computer Vision (2n Eition), pp , Worl Scientific Pulishing Co., [4] M. Erikson, Segmentation an Classification of Iniviual Tree Crowns, Ph.D. thesis, Sweish University of Agricultural Sciences, Uppsala, Sween, [5] M. Erikson, Species Classification of Iniviually Segmente Tree Crowns in High-Resolution Aerial Images using Raiometric an Morphologic Image Measures, Remote Sensing of Environment, vol. 91, pp , April [6] F. A. Gougeon, Comparison of Possile Multispectral Classification Scheme for Tree Crown Iniviually Delineate on High Spatial Resolution MEIS Images, Canaian Journal of Remote Sensing, 21(1), pp. 1 9, [7] F. A. Gougeon, Vers l inventaire forestier automatisé : reconnaître l arre ou la forêt?, CD-ROM u 9 eme Congrès e L Association quéécoise e téléétection, May [8] F. A. Gougeon, D. G. Leckie, I. Scott an D.Paaraine Iniviual Tree Crown Species Recognition: the ahmint Stuy, Proc. of the International Forum on Automate Interpretation of High Spatial Resolution Digital Imagery for Forestry, pp , Pacific Forestry Center, Victoria, British Columia, Canaa, Feruary [9] R. M. Haralick, Statistical an Structural Approaches to Texture, Proceeings of the IEEE, vol. 67, pp , [10] D. G. Leckie, F. A. Gougeon,. Walsworth an D. Paraine, Stan elineation an Composition estimation using Semi- Automate Iniviual Tree Crown Analysis, Remote Sensing of Environment, vol. 85, pp , [11] E. Klassen, A. Srivastava, W. Mio an S. H. Joshi, Analysis of Planar Shapes Using Geoesic Paths on Shape Space, IEEE Transaction on Pattern Analysis an Machine Intelligence, vol. 26, no 3, pp , March [12] G. Perrin, Etue u Couvert Forestier par Processus Ponctuels Marqués, Ph.D. thesis, Ecole Doctoral e Centrale Paris, Octoer [13] G. Perrin, X. Descomes an J. Zeruia. A on-bayesian Moel for Tree Crown Extraction using Marke Point Processes, Research Report 5846, IRIA, France, Feruary, [14] A. Srivastava, S. H. Joshi, W. Mio an X. Liu, Statistical Shape Analysis: Clustering, Learning, an Testing, IEEE Transaction on Pattern Analysis an Machine Intellegence, vol. 27, no 4, pp , March [15] V.. Vapnik, Statistical Learning Theory. John Wily an sons, inc., [16] V.. Vapnik, The ature of Statistical Learning Theory. Springer Verlag, ew-york, 1995.