Multivariate distribution theory by holonomic gradient method. Akimichi Takemura, Shiga University June 28, 2016, IMS-APRM 2016

Transcription

1 Multivariate distribution theory by holonomic gradient method Akimichi Takemura, Shiga University June 28, 2016, IMS-APRM 2016

2 Outline 1. An advertisement and overview of my research on holonomic gradient method (HGM) 2. Definition and properties of holonomic functions 3. Example: ball probability for multivariate normal distribution 1

3 Advertisement of HGM HGM is a common ground, where statistics, algebra and numerical analysis meet. It is a general method and can be applied when functions are holonomic. It evaluates a function by numerically solving differential equations. It already has some success stories. It is numerically very accurate and with proper mix of symbolic and numerical computations, it is often faster than existing methods. 2

4 References of HGM Please google hgm takemura or hgm takayama to find other success stories. Then you find This page lists 25 papers. You also find CRAN package for HGM by Takayama, containing many R commands based on the results of published papers. 3

5 Origin of my research on HGM Origin: a problem session by Takayama in September, HGM proposed in Holonomic gradient descent and its application to the Fisher-Bingham integral, Advances in Applied Mathematics, 47, N 3 OST My coauthors on HGM: H.Hashiguchi, J.Hayakawa, T.Koyama, S.Kuriki, N.Marumo, H.Nakayama, K.Nishiyama, M.Noro, Y.Numata, K.Ohara, T.Sei, C.Siriteanu, N.Takayama. 4

6 Definition and properties of holonomic functions Univariate homogeneous case: A smooth function f is holonomic if f satisfies the following ODE 0 = h k (x)f (k) (x) + + h 1 (x)f (x) + h 0 (x)f(x), where h k (x),..., h 1 (x), h 0 (x) are rational functions of x. Example: for f(x) = exp( x 2 /2), we have 0 = f (x) + xf(x). 5

7 Multivariate case (for simplicity let dim = 2) A smooth f(x, y) is holonomic if for each of x and y, fixing the other variable arbitrarily, we have a holonomic function in x and y. Namely, if there exist rational functions h 1 0(x, y),..., h 1 k 1 (x, y), h 2 0(x, y),..., h 2 k 2 (x, y) in x, y such that k 1 i=0 k 2 i=0 h 1 i (x, y) i xf(x, y) = 0, h 2 i (x, y) i yf(x, y) = 0. 6

8 Consider r 1 x r 2 y f(x, y). If r 1 k 1 or r 2 k 2, we can always compute this, by recursively applying the differential equations. If we keep numerical values of i x j yf(x, y) in the range i = 0,..., k 1 1, j = 0,..., k 2 1, then we can always compute other higher-order derivatives. We can approximate i x j yf(x + x, y + y) by the values { i x j yf(x, y)} i=0,...,k1 1,j=0,...,k

9 k 2 standard monomials (0, 0) k 1 We usually only need to keep a subset of { x i yf(x, j y)} i=0,...,k1 1,j=0,...,k 2 1 in memory. The subset is given by the set of standard monomials obtained by the division algorithm based on a Gröbner basis. 8

10 Which functions are holonomic (Zeilberger(1990))? Polynomials and rational functions are holonomic. exp(rational), log(rational) are holonomic. f, g : holonomic f + g, f g : holonomic f(x 1,..., x m ) : holonomic f(x 1,..., x m )dx m : holonomic Restriction of a holonomic f to an affine subspace is holonomic. 9

11 Holonomocity is also defined for generalized functions. In multivariate case, integration over a region defined by polynomial inequalities is OK. From the above properties, it is often easy to tell that a given function is holonomic, i.e., it must satisfy a differential equation with rational function coefficients. The problem is to find the explicit form of the differential equation (either by computer or by hand). 10

12 Virtually all standard multivariate normal distribution theory is holonomic. For many standard results, the differential equations are not known. Some differential equations are known, such as the differential equations for hypergeometric functions of a matrix argument by Muirhead. Even when they are known, people have not been using them numerically. 11

13 An example by Toshinori Oaku Algorithms for integrals of holonomic functions over domains defined by polynomial inequalities, J. Symbolic Comput. (2013) by T.Oaku gives the following example: Consider the probability of P (X 3 Y 2 ) under bivariate normal distribution: v(t) = e t(x2 +y2) dxdy. x 3 y 2 12

14 v(t) satisfies the following ODE. 0 = (216t 4 4 t + (32t t 3 ) 3 t + (224t t 2 ) 2 t + (326t t) t + 70t + 15)v(t). HGM allows this kind of new distribution theory! 13

15 Example: ball probability for multivariate normal distribution Holonomic gradient method for distribution function of a weighted sum of noncentral chi-square random variables. Computational Statistics. T.Koyama and A.Takemura Let X be a d-dimensional normal random vector with the mean µ and the covariance matrix Σ: X N d (µ, Σ) 14

16 P (X X 2 d r2 ) of the ball with radius r is written as 1 exp x 2 (2π) d/2 Σ 1/ x2 d r2 ( 1 ) 2 (x µ) Σ 1 (x µ) dx. By rotation, we can assume that Σ is a diagonal matrix without loss of generality. Hence X 2 = X X 2 d is the weighted sum of independent noncentral chi-square random variables. 15

17 The integral for the infinitesimal interval r < (x x 2 d) 1/2 < r + dr is the Fisher-Bingham integral. In statistical interpretation, the conditional distribution of X given X = r is the Fisher-Bingham distribution. The Fisher Bingham integral is an integral on the sphere S d 1 (r) = { t R d t t 2 d = r2} of radius r, with parameters λ 1,..., λ d, τ 1,..., τ d. 16

18 It is defined by f(λ, τ, r) = S d 1 (r) exp ( d i=1 λ i t 2 i + d i=1 τ i t i ) dt. Here, dt is the volume element on S d 1 (r) with dt = r d 1 S d 1, S d 1 = Vol(S d 1 (1)) = 2πd/2 Γ(d/2). S d 1 (r) 17

19 Let Σ = diag(σ 2 1,..., σ 2 d), µ = (µ 1,..., µ d ). Put new parameters as λ i := 1 2σ 2 i, τ i := µ i σ 2 i. Then the ball probability is written as: P (X Xd 2 r 2 ) d i=1 λi = exp π d/2 ( 1 4 d i=1 τ 2 i λ i ) r 0 f(λ, τ, s)ds. 18

20 Let F = ( f,..., f, f,..., f ). τ 1 τ d λ 1 λ d Then, the vector F satisfies the ODE: r F = P r F. Note that f(λ, τ, s) = r 2 ( f + + f ). λ 1 λ d Hence we do not need f as an element of F. 19

21 The matrix P r can be written as P r = 1 r 2r 2 x O y 1 y O 2r 2 x d + 1 y d y d r 2 y 1 O 2r 2 x O r 2 y d 1 2r 2 x d + 2, where O denotes off-diagonal block of 0 s and 1 denotes off-diagonal block of 1 s. We can solve the differential equation starting from an appropriate initial value. 20

22 Laplace approximation Suppose σ 1 > σ 2 σ d, i.e., λ 1 > λ 2 λ d Then as r f(λ, τ, r) f(λ, τ, r), where f(λ, τ, r) = (e rτ 1 + e rτ 1 ) exp ( r 2 λ 1 π (d 1)/2 1 d i=2 (λ 1 λ i ) 1/2. d i=2 τ 2 i 4(λ i λ 1 ) ) Similar expressions for derivatives of f(λ, τ, r). 21

23 Numerical Experiments Let d = 3 and σ 1 = 3.0, σ 2 = 2.0, σ 3 = 1.0, µ 1 = 0.01, µ 2 = 0.02, µ 3 = 0.03, i.e., λ 1 = , λ 2 = 0.125, λ 3 = 0.5, τ 1 = , τ 2 = 0.005, τ 3 = By the HGM, we can compute the probability ( d P i=1 X2 i < r ), 2 starting from the small initial value r

24 Ball Probability r Figure 1: CDF 23

25 Ratio of FB and its Laplace approximation (Fisher Bingham Integral)/(Laprace Approximation) r Figure 2: Ratio of FB and Laplace approx. 24

26 Derivatives: (Differantial of FB)/(Laprace Approximation) df/d\tau_1 df/d\tau_2 df/d\tau_3 df/d\lambda_1 df/d\lambda_2 df/d\lambda_ r Figure 3: Ratios of derivatives of FB and its LA 25

27 This suggests that HGM for this case is numerically very accurate. Also we can easily compute CDF for up to dimension d =

28 Summary I gave definition of HGM I discussed the example of ball probability in detail. For other problems google hgm takemura. Thank you for your attention! 27