Statistical Shape Models Using Elastic-String Representations

Transcription

1 Statistical Shape Models Using Elastic-String Representations Anuj Srivastava, Aastha Jain, Shantanu Joshi,andDavidKaziska Florida State University, Tallahassee, FL, USA Indian Institute of Technology, N. Delhi, India Air Force Institute of Technology, Dayton, OH, USA Abstract. To develop statistical models for shapes, we utilize an elastic string representation where curves (denoting shapes) can bend and locally stretch (or compress) to optimally match each other, resulting in geodesic paths on shape spaces. We develop statistical models for capturing variability under the elastic-string representation. The basic idea is to project observed shapes onto the tangent spaces at sample means, and use finite-dimensional approximations of these projections to impose probability models. We investigate the use of principal components for dimension reduction, termed tangent PCA or TPCA, and study (i) Gaussian, (ii) mixture of Gaussian, and (iii) non-parametric densities to model the observed shapes. We validate these models using hypothesis testing, statistics of likelihood functions, and random sampling. It is demonstrated that a mixture of Gaussian model on TPCA captures best the observed shapes. Introduction Analysis of shapes is emerging as an important tool in recognition of objects from their images. As an example, one uses the contours formed by boundaries of objects, as they appear in images, to characterize the objects themselves. Since the objects can occur at arbitrary locations, scales, and planar rotations, without changing their appearances, one is interested in the shapes of these contours, rather than the contours themselves. This motivates the development of tools for statistical analysis of shapes of simple, closed curves in R.Astatistical analysis is beneficial in many situations. For instance, in cases where the observed image is low quality due to clutter, low resolution, or obscuration, one can use the contextual knowledge to impose prior models on expected shapes, and use a Bayesian framework to improve shape extraction performance. Such applications require a broad array of tools for analyzing shapes: geometric representations of shapes, metrics for quantifying shape differences, algorithms for computing shape statistics such as means and covariances, and tools for testing competing hypotheses on given shapes. Analysis of shapes of planar curves has been of a particular interest recently in the literature. Klassen [] have described a geometric technique to parameterize curves by their arc lengths, and to use their angle functions to represent P.J. Narayanan et al. (Eds.): ACCV, LNCS 8, pp.,. c Springer-Verlag Berlin Heidelberg

2 Statistical Shape Models Using Elastic-String Representations and analyze shapes. Similar constructions for analysis of closed curves were also studied in [, ]. Using the representations and metrics described in [], [] describe techniques for clustering, learning, and testing of planar shapes. One major limitation of this approach is that all curves are parameterized by arc length, and the resulting transformations from one shape into another are restricted to bending only. Local stretching or compression of shapes is not allowed. Mio [] resolved this issue by introducing a representation that allows both bending and stretching of shapes to match each other. The geodesic paths resulting from this approach seem more natural as interesting features, such as corners, are better preserved while constructing geodesics, in this approach. This representation of planar shapes is called an elastic string model. Our goal in this paper is to use elastic string model to study several probability models for capturing observed shape variability. Similar to approaches presented in [, ], we project observed shapes onto the tangent spaces at sample means, and further reduce their dimensions using PCA. Thus, we obtain a low-dimensional representations of shapes called TPCA. On tangent principal components (TPCs) of observed shapes we study: (i) Gaussian, (ii) nonparametric, and (iii) mixture of Gaussian models. The first two have been studied earlier for non-elastic shapes in []. To study model performances, we: (i) synthesize random shapes from these models, (ii) test amongst competing models using likelihood ratio, and (iii) compare statistics of likelihood on training and test data. This framework leads to stochastic shape models that can be used as priors in future Bayesian extraction of shapes from low-quality images. To illustrate these ideas we have used shapes from the ETH databases. Rest of this paper is organized as follows. Section summarizes elastic-string models for shape representations. Section proposes three candidate probability models for capturing shape variability, while Sections and study these probability models via synthesis and hypothesis testing. Elastic Strings Representation Here we summarize the main ideas behind elastic-string representations of planar shapes, originally described in Mio et al [].. Shape Representation Let α: [, π] R be a smooth parametric curve such that α (t), t [, π]. The velocity vector is α (t) = e φ(t) e jθ(t), where φ: [, π] R and θ :[, π] R are smooth, and j =. The function φ is the log-speed of α and θ is the angle function. φ(t) measures the rate at which the interval [, π] is stretched or compressed at t to form the curve α; φ(t) > indicates local stretching near t, andφ(t) < local compression. Curves parameterized by arc length have φ. We will represent α via the pair (φ, θ) and denote by H the collection of all such pairs. Parametric curves that differ by rigid motions or uniform scalings of the plane, or by re-parameterizations are treated as representing the same shape. The pair

3 A. Srivastava et al. (φ, θ) is already invariant to translations of the curve. Rigid rotations and uniform scalings are removed by restricting to the space, C = {(φ, θ) H : Z π e φ(t) dt =π, π Z π θ(t)e φ(t) dt = π, Z π e φ(t) e jθ(t) dt =}, C is called the pre-shape spaces of planar elastic strings. There are two possible ways of re-parameterizing a closed curve, without changing its shape: (i) One is to change the placement of origin t = on the curve. This change can be represented as the action of a unit circle S on a shape (φ, θ), according to: s (φ(t),θ(t)) = (φ(t s),θ(t s) +s). (ii) Re-parameterizations of α that preserve orientation and the property that α (t), t, are those obtained by composing α with an orientation-preserving diffeomorphism γ :[, π] [, π]. Let D be the group of all such mappings. These mappings define a right action of D on H by (φ, θ) γ =(φ γ +logγ,θ γ). () denotes composition of functions. The space of all (shape-preserving) reparametrization of a shape in C is thus given by S D. The resulting shape space is the space of all equivalence classes induced by these shape preserving transformations. It can be written as a quotient space S =(C/D)/S. What metric can used to compare shapes in this space? Mio [] suggests that, given (φ, θ) H, leth i and f i, i =,, represent infinitesimal deformations of φ and θ, resp., so that (h,f )and(h,f ) are tangent vectors to H at (φ, θ). For a, b >, define (h,f ), (h,f ) (φ,θ) as a h (t)h (t) e φ(t) dt + b f (t)f (t) e φ(t) dt. () It can be shown that re-parameterizations preserve the inner product, i.e., S D acts on H by isometries. The elastic properties of the curves are built-in to the model via the parameters a and b, which can be interpreted as tension and rigidity coefficients, respectively. Large values of the ratio a/b indicate that strings offer higher resistance to stretching and compression than to bending; the opposite holds for a/b small. In this paper we fix a value of a/b that balances between bending and stretching.. Geodesic Paths in Shape Spaces An important tool in this shape analysis is to construct geodesic paths, i.e. paths of smallest lengths, between arbitrary two shapes. Given the complicated geometry of S, this task is not straightforward, at least not analytically. One solution is to use a computational approach, where the search for geodesics is treated as an optimization problem with iterative numerical updates. This approach is called the shooting method.givenapairofshapesα (φ,θ )and α (φ,θ ), one solves: min Ψ (α ; g) (s (α )) γ () s S,γ D,g T α (C)

4 Statistical Shape Models Using Elastic-String Representations where Ψ t (α; g) denotes a geodesic path startingatashapeα in the direction g, and parameterized by time t. Also, is the L norm on H. Basically, one solves for the shooting direction g such that the geodesic from α in the direction g gets as close to the orbit of α under shape preserving transformations []. Let d(α,α ) g denote the length of geodesics connecting the shapes α and α. This construction helps define the exponential map: exp α (g) =Ψ (α; g) and its inverse exp α (β) =g such that Ψ (α; g) =β.. Sample Mean of Shapes Since the shape space S is nonlinear, the definitions of sample statistics, such as means and covariances, are not conventional. Earlier papers [7, 8] suggest the use of Karcher mean to define mean shapes as follows. For α,...,α n in S, and d(α i,α j ) the geodesic length between α i and α j, the Karcher mean is defined as the element µ Sthat minimizes the quantity n i= d(µ, α i). A gradientbased, iterative algorithm for computing the Karcher mean is presented in [8, ]. Shown in Figure are some examples of three classes of shapes dogs, pears, and mugs used in the experiments here, and the Figure shows Karcher means of shapes in these three classes. Let µ be the mean shape and for any shape α, let g T µ (S) be such that Ψ (µ; g) =α. Then, α called the exponential of g, i.e. exp µ (g), and conversely, g =exp µ (α). As described next, statistics of α are studied through statistics of its map onto the tangent space at the mean. Fig.. Examples of three classes of shapes dogs, pears, and mugs from the ETH database that are studied in this paper, with the numbers used in test and training Statistical Shape Models Our goal is to derive and analyze probability models for capturing observed shapes. The task of learning probability models on spaces like S is difficult for two main reasons. Firstly, they are nonlinear spaces and therefore classical statistical approaches, associated with the vector spaces, do not apply directly. Secondly, these are infinite-dimensional spaces and do not allow component-by-component modeling that is traditionally followed in finite-dimensional vector spaces. The solution involves making two approximations. First, we project elements of S onto the tangent space T µ (S), which is a vector space, and therefore, better suited to statistical modeling. This is performed using the inverse exponential map exp µ. Second, we perform dimension reduction in T µ (S) using PCA. Together,

5 A. Srivastava et al. Fig.. In each case, left image shows the Karcher mean of shapes and right shows plots of the singular values of sample covariance matrix these two approximations given rise to TPCA representation. These ideas were first proposed for landmark-based shape analysis in []. To start TPCA, we use the Gram-Schmidt algorithm to find an orthonormal basis of the given vectors: Set i =andr =.. Set Y i = g r i j= Y j,g r Y j.. If Y i,y i, Set Y i = Y i / Y i,y i, i = i +,r = r +, and go to Step. Else If r<k Set r = r +andgotostep. Else Stop Say the algorithm stops at some i = n k. Sonowwehaveann-dimensional subspace Y spanned by an orthonormal basis with elements {Y,Y,...,Y n }.The next step is to project each of the observed vector into Y as follows. Let x ij = g i,y j and define a vector x i =[x i,x i,...,x in ] R n. Then, the projection of g i into Y is given by n j= x ijy j.eachg i T µ (S) is now represented by a smaller vector x i R n.next,weperformpcainr n using the projected observations {x, x,...,x k }. That is, from their sample covariance matrix C R n n, find its singular value decomposition C = UΣU T, and use the first d-columns of U to form a basis for the principal subspace of R n,withd n. The vector x R n maps to a smaller vector a R d such that x = d j= a ju j. The choice of d is made using the singular values of C; shown in Figure are the plots of singular values of C for the three classes: dogs, pears, and mugs.. Probability Models on TPCs We impose a probability model on α implicitly by imposing a probability model on its tangent principal components (TPCs) a. What probability models can be used in this situation? In this paper, we study the following three models: nonparametric, Gaussian and mixtures of Gaussian. The first two models were studied for non-elastic shapes in [].. Nonparametric Model: Assuming that the TPCs, a j s, are statistically independent of each other, one can estimate their probability densities directly from the data using a kernel estimator. Let f () j, j =,...,d be the kernel estimate of the density function of a j,thej th TPC of the shape α. Intheexperiments presented here we used a Gaussian Kernel. Then, assuming independence of

6 Statistical Shape Models Using Elastic-String Representations 7 TPCs, we obtain: f () (α) = d j= f j(a j ). Shown in Figure are some examples of estimated f () for several js. For each shape class, we display three examples of non-parametric density estimates for modeling TPCs. Dogs Pears Mugs Fig.. We show three examples of modeling TPCs in each class. For each example, the left figure shows nonparametric estimate f () while the right figure shows the mixture of Gaussian f () (using cross-lines) drawn over observed densities (plain lines).. Gaussian Model: Let Σ R d d be the diagonal matrix in SVD of C, the sample covariance of x i s. Then, we can model the component a j as a Gaussian random variable with mean zero and variance Σ jj. Denoting the Gaussian density function as h(y; z,σ ) exp( (y πσ z) /(σ )), we obtain the Gaussian shape model f () (α) = d j= h(a j;,σ jj ).. Mixture of Gaussian: Another candidate model is that a j follows the density ( d K ) f () j (α) = p k h(a j ; z k,σk), p k =, j= k= a finite mixture of Gaussian. For a given K, EMalgorithmcanbeusedto estimate the means and variances of components. Based on empirical evidence, we have used K = in this paper to estimate f () from observed data. Shown in Figure are some examples of estimated f () for some TPCs. In each panel, the marked line shows the estimated mixture density and, for comparison, the plane line shows the observed histograms. k Empirical Evaluations We have analyzed and validated the proposed shape models using: (i) random sampling, (ii) hypothesis testing, and (iii) statistics of log-likelihoods. We describe these results next.

7 8 A. Srivastava et al. Fig.. Sample shapes synthesized from the nonparametric model (top) and the mixture model (bottom) Shape Sampling: As a first step, we have synthesized random shapes from the three probability models f (i), i =,,. In each case the synthesis involves generating a random TPC according to its probability model- kernel density, Gaussian density or mixture of Gaussian- and then reconstructing the shape represented by that set of TPCs. For the generated values of TPCs, we form the vector x = d j= a ju j, and the tangent direction g = n i= x iy i, and eventually the shape α = exp µ (g). Shown in Figure are examples of random shapes generated from the models f () (top row) and f () (bottom row). We found that all three models seem to perform reasonably well in synthesis, with f () and f () being slightly better than f (). Testing Shape Models In order to test proposed models for capturing observed shape variability, we use the likelihood ratio test to select among the candidate models. For a shape α S, the likelihood ratio under any two models is: f (m) (α) f (n) (α) = and the log-likelihood ratio is l(α; m, n) d f (m) j (a j ), m,n =,,, f (n) j (a j ) j= d ( ) log(f (m) j (a j )) log(f (n) j (a j )). j= If l(α; m, n) is positive then the model m is selected, and vice-versa. Taking a large set of test shapes, we have evaluated l(α; m, n) for each shape and have counted the fraction for which l(α; m, n) is positive. We define: P (m, n) = {i l(α i; m, n) > } k where k is the total number of shapes used in this test. This fraction is plotted versus the component size d in Figure, for two pairs of shape models: P (, ) in,

8 Statistical Shape Models Using Elastic-String Representations dogs pears mugs Fig.. P (m, n) plotted versus vs d, for each of the three classes. Top row: m =, n =, and bottom row: m =,n =. the top row and P (, ) in the bottom row. P (m, n) >. implies that model m outperforms n. Two sets of results are presented in each of these plots. The solid line is for the test shapes that were not used in estimation of shape models, and the broken line is for the training shapes that were used in model estimation. Also, we draw a line at. to clarify which model is performing better. As these indicate, the mixture model seems to perform the best in most situations. On the training shapes, for pears and mugs, the nonparametric model is better than the mixture model. This result in expected since nonparametric model is derived from these training shapes themselves. However, on the test shapes, the mixture model is either comparable or better than the other two models. We conclude that for this data set, the mixture model is better for capturing variability in both training and test shapes. Furthermore, it is efficient due to its parametric nature. Statistics of Model Likelihoods Another technique for model validation is to study the variability of a modelbased sufficient statistic when evaluatedonbothtrainingandtestshapes.in case the distributions of this statistic are similar on both training and test shapes, this validates the underlying model. In this paper, we have chosen the sufficient dogs pears mugs dogs pears mugs Fig.. Histograms of ν (i) (α) for test (solid) and training shapes (broken). First three are for nonparametric model, and the last three are for mixture of Gaussians.

9 A. Srivastava et al. statistictobeproportionaltonegative log-likelihood of an observed shape. That is, we define ν (i) (α) log(f (i) (α)), where the proportionality implies that the constants have been ignored. Shown in Figure are some examples of this study for the nonparametric (first three) and the mixture model (last three). These plots shows histograms of ν (i) (α) values for both test and training shapes, for each of the three shape classes. It is evident that the histograms for training and test sets are quite similar in all these examples, and hence, validate the proposed models. Acceptance/Rejection Under Learned Models: In the final experiment, we performed acceptance/rejection for each test shape under the mixture model, f (), for each shape class (dogs, pears, and mugs). Using threshold values estimated using training data of each class, we compute the value of ν () (α) foreach test shape α; if it is below the threshold we accept it, otherwise we reject it. For example, we have dog reject ν () > (α) < κ dog. dog accept This is done for each of the three classes dogs, pears, and mugs, and the results are summarized in the next table. This table lists the percentage of times a shape from a given test class was accepted by each of the three shape classes. For example, test shapes in dog class were accepted 9.7% times by shape model for dog class,.7% by pear model, and.8% by cup model. Also,.7% of test shapes in dog class were rejected by all three models. Since a shape can be accepted by more than one model, the sum in each row can exceed %. Notice that the test shapes also include other objects such as horses, cows, apples, cars, and tomatoes. Some of the cows (%) are accepted under dog model, but are easily rejected under pear and mug models; most of the cows (%) are rejected under all three models. Tomatoes are mostly accepted by pear and mug models. Overall, the mixture model f () demonstrates a significant success in capturing shape variability and in discriminating between object classes. It also enjoys the efficiency of being a parametric model. Test class Dog Accepts (%) Pear Accepts (%) Cups Accepts (%) No Accepts (%) Dogs Pears Cups Horses Apples Cows Cars Tomatoes

10 Conclusion Statistical Shape Models Using Elastic-String Representations We have presented results from statistical analysis of planar shapes under elastic string models. Using TPCA representation of shapes, three candidate models were presented: nonparametric, Gaussian, and a mixture of Gaussian. We evaluated these models using (i) random sampling, (ii) likelihood ratio tests, (iii) similarity of (distributions of) sufficient statistics on training and test shapes, and (iv) acceptance/rejection of test shapes under the models estimated from the corresponding training shapes. All three models do reasonably well in random sampling and likelihood ratio test. However, the mixture model emerges as the best model for capturing shape variability and efficiency. We therefore conjecture that mixture of Gaussians are sufficient for modeling TPCs of observed shapes for use as prior shape models in future Bayesian inferences. References. Klassen, E., Srivastava, A., Mio, W., Joshi, S.: Analysis of planar shapes using geodesic paths on shape spaces. IEEE Pattern Analysis and Machine Intelligence (March, ) 7 8. Younes, L.: Optimal matching between shapes via elastic deformations. Journal of Image and Vision Computing 7 (999) Michor, P.W., Mumford, D.: Riemannian geometries on spaces of plane curves. Journal of the European Mathematical Society to appear (). Srivastava, A., Joshi, S., Mio, W., Liu, X.: Statistical shape analysis: Clustering, learning and testing. IEEE Transactions on Pattern Analysis and Machine Intelligence 7 () 9. Mio, W., Srivastava, A.: Elastic string models for representation and analysis of planar shapes. In: Proc. of IEEE Computer Vision and Pattern Recognition. (). Dryden, I.L., Mardia, K.V.: Statistical Shape Analysis. John Wiley & Son (998) 7. Le, H.L., Kendall, D.G.: The Riemannian structure of Euclidean shape spaces: a novel environment for statistics. Annals of Statistics (99) 7 8. Karcher, H.: Riemann center of mass and mollifier smoothing. Communications on PureandAppliedMathematics (977) 9