MULTISCALE RECONSTRUCTION OF PHOTON-LIMITED HYPERSPECTRAL DATA. Kalyani Krishnamurthy and Rebecca Willett

Transcription

1 MULTISALE REONSTRUTION OF PHOTON-LIMITED HYPERSPETRAL DATA Kalyani Krishnamurthy and Rebecca Willett Department of Electrical and omputer Engineering Duke University, Durham, N ABSTRAT This paper combines recent innovations in intensity estimation for marked Poisson processes and multiscale nonparametric function estimation using generalized linear models (GLM) to perform photon-limited hyperspectral image reconstruction. Inspired by spectroscopic images of solar flares, which are very intense at some energies but very weak at others, we develop a multiscale intensity estimation method which can adapt to spatial inhomogeneities, spectral emission lines, and large spectral dynamic ranges. This approach exploits the fact that boundaries between different physical structures exist at every spectral band, even when the contrast is very small in some of those bands. Incorporating this denoising method into a generalized expectation-maximization method allows very faint features to be accurately reconstructed from blurry, photon-limited observations. Index Terms Poisson processes, Wavelet transforms, Multidimensional signal processing, Hyperspectral imaging 1. HYPERSPETRAL INTENSITY ESTIMATION Hyperspectral imaging is a common task in a variety signal processing applications, including astronomical imaging [1], fluorescence microscopy [2], and remote sensing. The spectral dimension contains information about different materials properties which aid in understanding the underlying phenomenon. Effective utilization of the information contained in the spectral dimension leads to significant improvement in photon limited spatio-spectral intensity estimation problems. The spectra associated with each spatial location, while relevant to application specific objectives, do not directly lead to improved reconstruction capabilities in data starved scenarios, in which the number of observed photons is very small. In fact, in some applications, the spectral dimension can pose additional challenges. One such application is RHESSI solar spectroscopic imaging [1]. RHESSI is a spacecraft that makes indirect measurements of hyperspectral data corresponding to solar flares. Here, the spectra decay exponentially as a function of photon energy and span a wide range of energies for example, a single flare can have close to photons with energies close to one kev and only a few photons with energies close to ten MeV. Because of the very large dynamic range and photon-limited noise, many reconstruction techniques, such as those based on wavelets [3], can potentially be very inaccurate. Recent work in the context of marked Poisson processes, however, has demonstrated that the effective use of marks is highly useful in data-starved Poisson process R. Willett was partially supported by NSF award number NSF-F and DARPA award numbers HR and N intensity estimation [4]. This paper builds upon that work by incorporating multiscale generalized linear models in the context of high dynamic ranges and by incorporating the regularization method into an expectation-maximization framework for Poisson inverse problems Problem Formulation Assume that we measure noisy observations of a spatially- and spectrally-varying intensity, where the noise obeys a Poisson distribution associated with photons hitting the detector array. Let f denote the N N M true three-dimensional intensity, where the first two dimensions correspond to the spatial locations, and the third dimension corresponds to the spectral energies. Our observations (in this section) are thus assumed to be of the form y Poisson(f). Our goal is to infer f from the measurements y as accurately as possible, effectively utilizing the spectral information contained in the third dimension. 2. MULTISALE SPATIO-SPETRAL INTENSITY ESTIMATION The approach described in this paper is roughly based upon denoising Haar wavelets with a hereditary constraint. Taking advantage of correlations in the data between both spectral bands and spatial locations, the proposed method entails the following two stages: Stage 1: Perform hereditary Haar Poisson intensity estimation in the spatial dimensions, with each leaf of the resulting unbalanced quad-tree decomposition corresponding to a spectrum (or series of marks). Stage 2: Smooth each spectrum (or mark dimension) using a multiscale nonparametric generalized linear model fit [5]. We will elaborate upon both these stages below Stage 1 In the first stage we compute an initial estimate by determining the ideal partition of the spatial domain of observations (assumed to be [0, 1] 2 ) and use maximum likelihood estimation to fit a single, mean spectrum to each square in the optimal partition. The space of possible partitions is a nested hierarchy defined through a recursive dyadic partition (RDP) of [0, 1] 2, and the optimal partition is selected by pruning a quad-tree representation of the intensity. This gives our estimators the capability of spatially varying the resolution to automatically increase the smoothing in very regular regions of the intensity and to preserve detailed structure in less regular regions. In general, the RDP framework leads to a model selection problem that can be solved by a tree pruning process. Each of the terminal squares in the pruned spatial RDP could correspond to a region

2 of intensity which is spatially homogeneous or smoothly varying (regardless of the regularity or irregularity between the spectral bands). Such a partition can be obtained by merging neighboring squares of (i.e. pruning) a complete RDP to form a data-adaptive RDP P and fitting spectra to the data on the terminal squares of P. Specifically, given the partition P, f(p) can be calculated by finding the average spectrum fit to the observations over each cell in P. Thus a Stage 1 intensity estimate, b f, is completely described by P. This provides for a very simple framework for penalized likelihood estimation, wherein the penalization is based on the complexity of the underlying partition [6]. The goal here is to find the partition which minimizes the penalized likelihood function: where bp arg min [ log p(x f(p)) + pen 1 (P)], f1 b f( P) b P p(x e f) = N 1 Y Y M 1 i,j=0 k=0 (e e f i,j,k e f x i,j,k i,j,k )/x i,j,k! denotes the likelihood of observing x given the estimate f( b P) and where pen 1 ( b P) is the penalty associated with the estimate f( b P). We penalize the estimates according to a codelength required to uniquely describe each model with a prefix code; the penalties are discussed in detail in [6, 7]. The resulting estimator b f 1 is referred to as the spatial penalized likelihood estimator (PLE). This approach is very similar to the image estimation method described in [6], with the key distinction that Stage 1 as described above forces the spatial RDP to be the same at every spectral band. This constraint makes it impossible for the method to perform spatial smoothing at some spectral bands but not others Stage 2 Stage 1 performed spatial smoothing irrespective of the smoothness associated with the spectra. In many applications, such as astronomical spectral imaging, spectra can accurately be modeled by smooth functionals added to relatively isolated emission lines [1]. Furthermore, the dynamic range of the spectra may be very large. Stage 2 exploits the smoothness in the spectral dimension and accounts for the large dynamic range by fitting a piecewise exponential curve to each spectrum, thereby improving the accuracy of the estimate obtained in Stage 1. Nonparametric curve estimation using multiscale generalized linear models was proposed in [5] and shown to be near minimax optimal for broad classes of intensities. More specifically, Stage 2 consists of pruning an RDP of each unique spectrum in b f 1 (there is one for each cell c b P). This pruning method is similar to the spatial pruning described above: we consider a collection of binary RDPs of [0, 1], and select the optimal partition by pruning a binary tree representation of the spectrum. The result is a data-adaptive binary RDP Q with an exponential curve fit to the data on each terminal interval of Q, for each unique spectrum. Analogously to Stage 1, for cell c b P, we set bq c arg min [ log p(x c f c(q)) + pen 2 (Q)], Q bf 2,i,j,k X c bp(f( c Q c)) i,j,k I {(i,j,k) c}, where x c is the spatial sum of the observations in cell c of b P, f c(q) is the piecewise exponential function fit to x c over the RDP Q, and pen 2 (Q) is a penalty on the size (complexity) of Q. 3. UPPER BOUNDS ON APPROXIMATION ERROR The estimate computed by the method above is piecewise constant in the spatial dimensions and piecewise exponential in the spectral dimension. In this section we describe a class of piecewise smooth hyperspectral images and bound the rate at which the approximation error decays as a function of the number of constant and exponential pieces in the best approximation. In particular, assume that f is a sampled version of a continuous domain function f, and let x, y, and λ denote the two spatial dimensions and one spectral dimension of f, respectively, so that f i,j,k = Z (i+1)/n Z (j+1)/n Z (k+1)/m i/n j/n k/m f(x, y, λ)dλ dy dx. Let the log of the true intensity be denoted g log f. To facilitate this analysis, we make the following assumptions about g. First, assume g belongs to the family of functions that are piecewise Höldersmooth in the spatial dimensions and Besov smooth in the spectral dimension, and that < l g u <. We say that g(x, y, λ) is spatially piecewise Hölder- (α, γ) smooth if, for any fixed λ 0 [0, 1] g(x, y, λ 0) = g 1(x, y, λ 0)I {H(x) y} +g 2(x, y, λ 0)I {H(x)<y} where each surface g j(x, y, λ 0) for j = 1, 2 is Hölder-α for α (0, 1], so that g j(x, y, λ 0) g j(x 0, y 0, λ 0) α((x x 0) 2 + (y y 0) 2 ) α/2 for any (x 0, y 0) [0, 1] 2 [8]. Also, H(x) is Hölder-γ for γ (0, 1] so that H(x) H(x 0) γ x x 0 γ In other words, g(x, y, λ 0) consists of two Hölder smooth surfaces separated by a Hölder smooth boundary. Further assume that for any fixed (x 0, y 0) [0, 1] 2, the one-dimensional function g(x 0, y 0, λ) is in the Besov space Bp β (L p([0, 1])) for β (1, 2] and 1/p = β+1/2. We refer the reader to [9] for more details on Besov spaces. We consider k-term approximations formed when Ω [0, 1] 3 is partitioned into k non-overlapping cuboids (which are of the same sidelength in the first two dimensions) and g is approximated with the same linear spectrum at each spatial location within each unique cuboid, so that the resulting approximation is piecewise linear in the spectral dimension and piecewise constant in the spatial dimension. We now derive upper bounds on the rate at which the error of the best approximation of this type decays as a function of k. In particular, the approximation error between g and its best k-term approximation, denoted bg k, can be bounded using the triangle inequality: g bg k L2 (Ω) min 1 m k g egm L2 (Ω) + eg m bg k L2 (Ω) (1) where eg m is formed by partitioning Ω into m cuboids, and approximating g on each cuboid independently with one spectrum at each spatial location. Each of these cuboids is of length one in the third (spectral) dimension and the sidelengths of each cuboid in the first two (spatial) dimensions are equal. In other words, we form a spatial partition of Ω, so that eg m is spatially piecewise constant. For cuboid i, we let (x i, y i ) denote the spatial location of the upper left corner of the cuboid, and set eg m(x, y, λ) mx I {(x,y) i }g(x i, y i, λ). i=1

3 From here, bg k can be formed by approximating eg m on each cuboid with a free-knot piecewise linear spectrum with p k/m pieces. Because of the boundary H(x) separating different smooth spatial regions, the m cuboids will not be uniformly sized. Instead, the best m-cuboid approximation eg m will have smaller cuboids near the boundary and larger cuboids away from the boundary. In particular, consider partitioning Ω into m 2 cuboids of spatial sidelengths 1/m and spectral sidelength one. We denote by 1 the set of all cuboids that intersect the boundary. Because of the Hölder-γ condition on the boundary H(x), it can be shown that the boundary passes through O(m 2 γ ) cuboids. Merge all cuboids not containing the boundary into larger cuboids; after merging there are O(m) cuboids; denote this set as 2. Now 0 g eg m L2 (Ω) X Z g(x, y, λ) eg m(x, y, λ) X Z g(x, y, λ) eg m(x, y, λ) 2 A The L error between g and eg m can be bounded above by g(x, y, λ) eg m(x, y, λ) u l. Since each of these O(m 2 γ ) cuboids has volume m 2, the L 2 error over 1 can be bounded by X Z g(x, y, λ) eg m(x, y, λ) (m 2 )m 2 γ = O(m γ ) Over the set 2, g consists of a Hölder smooth surface with smoothness parameter α. This leads to the following: r 2 g(x, y, λ) eg m(x, y, λ) α m g(x, y, λ) eg m(x, y, λ) 2 L 2 ( 2 ) = X Since there are O(m) cuboids in 2, Thus,! α 2 O(m α 1 ) g(x, y, λ) eg m(x, y, λ) 2 L 2 ( 2 ) = O(m α ). 1/2 g eg m L2 (Ω) = `O(m γ ) + O(m α ) 1/2 = O m min(γ,α)/2 (2) Let us now bound the second part on the right hand side of (1). Note that because each spectrum in g (and hence eg m) is in the Besov space B β p (L p([0, 1])), we have that for any fixed (x 0, y 0) [0, 1] 2 Z eg m(x 0, y 0, λ) bg k (x 0, y 0, λ) 2 dλ pp 2β where p k/m is the number of linear pieces in bg k (x 0, y 0, λ) and p is a constant that does not depend on p. This yields eg m bg k L2 (Ω) ZZZ = eg m(x, y, λ) bg k (x, y, λ) 2 dxdydλ ZZ «1/2 pp 2β dxdy«1/2 = O(p β ). (3) Applying (2) and (3) to (1) we have n g bg k L2 (Ω) O min 1 m k m min(γ,α)/2 + O((k/m) β ) o. Differentiating the expression being minimized above with respect to m and equating it to zero, we find «β min(γ,α) g eg m L2 (Ω) O k min(γ,α)+2β. 4. SPETRAL IMAGE REONSTRUTION The above multiscale method for spatio-spectral denoising can now be used to help reconstruct a blurred noisy spectral image. We can model our observations as x Poisson(Hf), where f is the vectorized spatio-spectral data cube we wish to reconstruct, H is the operator induced by the spectral imaging system, and x is the vectorized collection of measurements. To solve this challenging inverse problem, we solve the following optimization problem: bf = arg min ef F n log p(x H e f) + pen( e f) o, where F is the collection of estimators corresponding to a pruned spatio-spectral tree as described in Section 2, and pen( f) e = P c P b pen 2( Q b c) + pen 1 ( P b ); this penalty is proportional to the number of cells in the pruned RDP and hence the penalty term encourages solutions with small numbers of leaves. We compute the solution to this problem using a Generalized Expectation- Maximization algorithm, which in this case is a regularized version of the Richardson-Lucy algorithm [10, 11, 12]. The method consists of two alternating steps: E-step: y (t) = b f (t). H T (x./h b f (t) ), where. and./ denote element-wise multiplication and division, respectively. M-step: ompute b f (t+1) by denoising y (t) as described in Section 2. In the experiments below, the method is initialized with b f (0) = 1 and terminated when b f (t+1) b f (t) 2 2/ b f (t) 2 2 < SIMULATION RESULTS To see the effectiveness of the proposed method, we simulated a spatio spectral hypercube of size that has different degrees of smoothness in the spatial and the spectral dimensions. Motivated by solar hyperspectral imaging applications, we built a phantom datacube that models a solar flare. Fig. 1a shows a portion of solar flare. The green smooth surface represents the sun s surface; the bluish red spot represents the sun spot from which the solar flares originate and the red lines show a part of the flare loop [1]. Fig. 1b shows the spectra associated with each color shown in Fig. 1a. The spectrum shown in red in Fig. 1b corresponds to the flare lines in Fig. 1a and the green spectrum in Fig. 1b corresponds to the spectrum associated with the sun s surface in Fig. 1a. The bluish red sun spot in Fig. 1a has a combination of the blue and the red spectra shown in Fig. 1b. One key factor that makes this problem challenging is the high dynamic range of the flare spectra. onsider the flare spectra shown in red in Fig. 1b. The expected number of photon counts that are observed in energy band one is close to 10 4, whereas the expected

4 (a) ounts Quiet Sun Spectrum Flare Spectrum Sunspot spectrum Spectral Band Fig. 1. Spatio-spectral intensity (a) True Image. orresponding spectra. number of photon counts observed in energy band sixty-four is approximately Also, the spectrum has few sharp emission lines in addition to the exponentially decaying component. The spectra shown in blue and green on the other hand has relatively few photon counts Intensity estimation Ignoring the correlations between different spectral bands and denoising each spectral band independently using the an RDP-based method as described in [6] yields the result displayed in Fig. 2(e) and (f); important structures are oversmoothed, particularly in low intensity bands. In this and later simulations, we measure error according to ε = 1/(N 2 M) X i,j,k( f i,j,k b f i,j,k )/ p f i,j,k ; this measure does not over-emphasize spectral bands with large photon counts. The error associated with this image-wise denoising approach is ε = If we were to denoise the hypercube from noisy measurements using just the spatial estimation procedure described in Stage 1, we would achieve the result described in Fig. 2(g) and (h). The error associated with Stage 1 alone is given by ε = ; despite the small increase in error over the image-wise approach, we see that key spatial structures are preserved. Performing spectral smoothing of Stage 2 in addition to the spatial smoothing described in Stage 1 offers dramatic improvements over the estimate obtained by treating each spectral band individually as shown in Fig. 2(i) and (j). The error associated with this estimate is given by ε = This simulation result shows that the flare edges and the other finer details such as the sun s surface are captured well by our proposed approach as opposed to the estimate obtained using the hereditary Haar estimation method Deconvolution To demonstrate the usefulness of the proposed approach in spatiospectral image reconstruction problems, we used the same flare test image as before and blur it to model an instrument response. What we observe at the detector is the image that is blurred and noisy. The test image at spectral band 38 is shown in Fig. 3a. Fig. 3b shows the noisy observation of the blurred image. The reconstruction results using Richardson-Lucy algorithm without any regularization is shown in Fig. 3c. Fig. 3d and Fig. 3e show the reconstruction results using the hereditary Haar denoising in the GEM algorithm and using the proposed denoising in the GEM algorithm. From the results, we can see that the GEM algorithm using the proposed denoising algorithm significantly outperforms the other two. 6. DISUSSION This paper demonstrates that spatio-spectral intensity estimation using GLM fits in the spectral domain can offer significant advantages in the context of photon-limited hyperspectral image reconstruction. The proposed two-stage intensity estimation approach preserves spatial inhomogeneities across all spectral bands simultaneously, using high-contrast boundaries in some bands to prevent oversmoothing of low-contrast boundaries in other bands. In addition, the nonparametric multiscale GLM fits proposed in [5] exploit the smooth variation in intensities across spectral bands while performing well in the face of high dynamic ranges. It is important to note, however, that this two-stage approach does not necessarily yield estimates computed on the optimal partition of the entire spatio-spectral data cube. In fact, more accurate estimates could potentially result from searching over every possible recursive dyadic partition of [0, 1] 3, unconstrained by our proposed two-stage partition. However, such a search would be very computationally intensive, particularly for the large data sets of interest. The above two-stage approach approximates this more global search; a more careful analysis of the accuracy-computation tradeoff is an important aspect of future work. 7. REFERENES [1] R. P. Lin, B. R. Dennis, and A. O. Benz (Eds.), The Reuven Ramaty High-Energy Solar Spectrscopic Imager (RHESSI) - Mission Description and Early Results, Kluwer Academic Publishers, Dordrecht, [2] W. Denk, J.H. Strickler, and W.W. Webb, Two-photon laser scanning fluorescence microscopy, Science, vol. 248, no. 4951, pp , [3] I. Atkinson, F. Kamalabadi, and D. L. Jones, Wavelet-based hyperspectral image estimation, 2003, vol. 2, pp [4] R. Willett, Multiscale intensity estimation for marked poisson processes, in Proc. IEEE Int. onf. Acoust., Speech, Signal Processing IASSP, [5] E. Kolaczyk and R. Nowak, Multiscale generalised linear models for nonparametric function estimation, Biometrika, vol. 92, no. 1, pp , [6] E. Kolaczyk and R. Nowak, Multiscale likelihood analysis and complexity penalized estimation, Annals of Stat., vol. 32, pp , [7] R. Willett and R. Nowak, Platelets: a multiscale approach for recovering edges and surfaces in photon-limited medical imaging, IEEE Transactions on Medical Imaging, vol. 22, no. 3, pp , [8] A. P. Korostelev and A. B. Tsybakov, Minimax theory of image reconstruction, Springer-Verlag, New York, [9] R. A. DeVore, Nonlinear approximation, Acta Numerica, vol. 7, pp , [10] W. Richardson, Bayesian-based iterative method of image restoration, J. Opt. Soc. of Am., vol. 62, pp , [11] L. B. Lucy, An iterative technique for the rectification of observed distributions, Astron. J., vol. 79, pp , [12] R. Nowak and E. Kolaczyk, A multiscale statistical framework for Poisson inverse problems, IEEE Trans. Info. Theory, vol. 46, pp , 2000.

5 (a) (c) (d) (e) (g) (a) (c) (d) (f) (h) (e) Fig. 3. Spatio-spectral deblurring results. (a) Spatial variation of intensity band in spectral band 38. Noisy blurry observations. (c) Image reconstruction using Richardson Lucy algorithm with no regularization; ε = (d) Image reconstruction using hereditary Haar denoising as the M-step of the GEM algorithm; ε = (e) Image reconstruction using the two stage denoising procedure in M-step of the GEM algorithm; ε = (i) (j) Fig. 2. Spatio-spectral analysis results. (a) Spatial variation of intensity at spectral band 38. Spatial variation of intensity at spectral band 46. (c) Noisy observations at spectral band 38. (d) Noisy observations at spectral band 46. (e) Hereditary Haar image denoising [6] applied to every image separately, see at spectral band 38; ε = (f) Hereditary Haar image denoising applied to every image separately, see at spectral band 46. (g) Stage 1 result at spectral band 38; ε = (h) Stage 1 result at spectral band 46. (i) Result after proposed two-stage approach at spectral band 38; ε = (j) Result after proposed two-stage approach at spectral band 46.