A VOLUME-BASED METHOD FOR DENOISING ON CURVED SURFACES

Transcription

1 A VOLUME-BASED METHOD FOR DENOISING ON CURVED SURFACES Harry Biddle Double Negative Visual Effects London, UK Ingrid von Glen, Colin B. Macdonald, Tomas März Matematical Institute University of Oxford Oxford, UK vonglen macdonald ABSTRACT We demonstrate a metod for removing noise from images or oter data on curved surfaces. Our approac relies on in-surface diffusion: we formulate bot te Gaussian diffusion and Perona Malik edge-preserving diffusion equations in a surface-intrinsic way. Using te Closest Point Metod, a recent tecnique for solving partial differential equations (PDEs) on general surfaces, we obtain a very simple algoritm were we merely alternate a time step of te usual Gaussian diffusion (and similarly Perona Malik) in a small 3D volume containing te surface wit an interpolation step. Te metod uses a closest point function to represent te underlying surface and can treat very general surfaces. Experimental results include image filtering on smoot surfaces, open surfaces, and general triangulated surfaces. Index Terms Image denoising, Surfaces, Partial differential equations, Numerical analysis. 1. INTRODUCTION Denoising is an important tool in image processing and often forms a crucial step in image acquisition and image analysis. On general curved surfaces, denoising and te corresponding scale space analysis of images, as well as oter image processing tasks ave seen some interest [1]. For example, Kimmel [2] studies an intrinsic scale space for images on parametrically-defined surfaces via te geodesic curvature flow. More recently, Spira and Kimmel [3] use te same flow for segmentation of images painted on parametrically-defined manifolds. Bogdanova et al. [4] perform scale space analysis as well as segmentation for omnidirectional images defined on various sapes. Our aim ere is to provide simple numerical realizations of te Gaussian diffusion filter and te nonlinear Perona Malik [5, 6, 7] edgepreserving diffusion filter for images on general surfaces. Fig. 1 sows an example comparing te two approaces. In applications, image processing on curved surfaces can occur even wen data is acquired in tree-dimensional volumes. For example, Lin and Seales [8] propose CT scanning of scrolls in order to non-destructively read text written on rolled up documents. In digital image-based elasto-tomograpy (DIET), a new metodology for non-invasive breast cancer screening, te surface and te surface data are measured from strobed ligting wit several calibrated digital cameras [9]. In suc applications, it may be useful to perform denoising and oter tecniques on surfaces embedded in tree-dimensional volumes. Denoising on surfaces is also related to Te work of all autors was partially supported by Award No KUK- C made by King Abdulla University of Science and Tecnology (KAUST). (a) (b) (c) Fig. 1. Denoising on te surface of a fis using our algoritm. Te original noisy surface data is sown in (a), Gaussian diffusion gives (b) wit blurry edges wereas (c) sows tat Perona Malik diffusion removes te noise wile maintaining sarp edges. surface fairing if te surface data is a eigt-field wic gives a perturbed surface of interest as a function on a reference surface [10]. In tis context te Perona-Malik model and variations of it ave already been considered in te literature [10, 11, 12]. A common teme in many tecniques (e.g., [2, 3, 4] above) is te use of models based on partial differential equations (PDEs). Applying PDE-based models on surfaces requires some representation of te geometry and tis coice is significant in te complexity of te resulting algoritm. A common coice is a parameterization of te surface; owever tis introduces artificial distortions and singularities into te model, even for te simplest surfaces suc as speres. On more complex geometry, joining up small patces is typically required. On a triangulated surface, PDEs can be discretized using finite element metods, e.g., [13]. Alternatively, implicit representations suc as level sets (e.g., [14]) or te closest point representation employed ere can be used. Tese tend to be flexible wit respect to te geometry altoug level sets do not give an obvious way to treat open surfaces. Using level set metods in a 3D neigbourood of te surface also requires introducing artificial boundary conditions at te boundary of te computational band. In tis work we use te Closest Point Metod [15] for solving bot te Gaussian and Perona Malik diffusion models on surfaces. Using tis tecnique, we keep te resulting evolution as simple as possible by alternating between two straigtforward steps: 1. A time step of te model in tree dimensions using entirely standard finite difference metods. 2. An interpolation step, wic encodes surface geometry and makes te 3D calculation consistent wit te surface problem. One benefit is tat te Closest Point Metod uses a closest point function [15, 16] to represent te surface, and terefore does not require modification of te model via a parameterization nor does

2 (a) (b) (c) Fig. 2. Our computational tecnique, illustrated ere in two dimensions, is based on a volume representation of a surface (a) were eac grid point (voxel) knows te closest point on te surface. On a uniform grid of points surrounding te surface, we ten alternate between two steps: (b) evolving te Perona Malik equation using a finite difference stencil at eac grid point; and (c) replacing te value at eac grid point wit te value at its closest point, using bi-cubic interpolation. it require te surface to be closed or orientable. Te Closest Point Metod as already been used successfully in PDE-based image segmentation [17] and PDE-based visual effects [18]. In 2, we demonstrate te Closest Point Metod by applying it to te standard diffusion equation. Tis gives a metod for Gaussian filtering on surfaces. 3 reviews te Perona Malik model for edge-preserving diffusion and adapts it to te Closest Point Metod. Tis is followed by numerical experiments in SOLVING DIFFERENTIAL EQUATIONS ON SURFACES We begin by demonstrating our approac on te Gaussian diffusion equation using te Closest Point Metod [15]. Tis flow does indeed remove noise from te image but it also results in considerable blurring of edges. In order to preserve edges we extend te approac to te nonlinear Perona Malik model in 3. Te tecnique presented ere is more general tan eiter equation and can be applied to a wide variety of surface differential equations Surface functions and images Suppose we ave a surface function u 0 : S R wic takes eac point on te surface S and returns a real number. We will tink of tis function as representing image data on te surface were te values of u 0 encode intensity (for colour images, we could replace u 0 wit a vector function u 0 : S R 3 ). We can blur suc a surface image by solving te surface PDE t = Su, (1) for a finite time from initial conditions consisting of our image u 0. Here S is te Laplace Beltrami operator, te surface intrinsic analogue of te Laplacian diffusion operator. We solve (1) by embedding te surface in a iger dimensional space (typically a treedimensional volume) Representing te surface We represent te surface S using a closest point representation: eac point x R 3 is associated wit its geometrically closest point on te surface cp(x) S. Fig. 2a sows a closest point representation of a curve (an example of a one-dimensional surface embedded in two dimensions). Using tis representation, we can extend values of a surface function u into te surrounding volume by defining v : R 3 R by v(x) := u(cp(x)). (2) Notably, v will be constant in te direction normal to te surface. Tis feature is crucial because it implies tat an application of a Cartesian differential operator (suc as te gradient or Laplacian) to v is equivalent to applying te corresponding intrinsic surface differential operator to u. Tese matematical principles are establised in [15, 16] Te Closest Point Metod We want to solve te tree-dimensional diffusion equation v t = v subject to te constraint tat v(x) = v(cp(x)) (i.e., tat v remains constant in te normal direction). One simple approac to solving tis constrained problem is to discretize in time and alternate between advancing te tree-dimensional PDE in time and reextending te results [15]. Using te forward Euler time-stepping algoritm wit step size τ, tis gives te following two-step process ṽ n+1 := v n + τ v n = v n + τ ( v n xx + v n yy + v n zz), (3a) v n+1 := ṽ n+1 (cp(x)). (3b) Note tat te temporary ṽ n+1 contains te result of a forward Euler step using te Cartesian Laplacian, but tis migt not be constant in te normal direction, so we re-extend to obtain v n+1. For additional accuracy, tis can easily be extended to igerorder Runge Kutta or multistep metods [15]. Implicit metods are also practical [19, 20] and useful if te problem is stiff Te role of voxels We discretize te tree-dimensional embedding volume, typically wit a uniform Cartesian grid as tis makes te algoritm straigtforward to implement and analyze. We will tink of tis treedimensional voxel grid as our discrete image wit te voxel size parameter specifying scale. Tis could be appropriate if te surface and image data were acquired in tree dimensions using e.g., CT scans [8]. In tat case, te grid parameter could be naturally cosen by te resolution of te CT macine. In oter cases, may simply be a numerical parameter. Acquiring closest point representations is a researc area in its infancy and for now we assume suc a representation is given. However, it is straigtforward to convert triangulations into tis formation [21] or to perform steepest descent on level-set representations [16] Interpolation Because we are working on a discrete grid, cp(x) is likely not a grid point and tus we need a way to approximate te extension operation of (3b). One approac is to use interpolation on a stencil consisting of te surrounding grid points nearby cp(x) [15]. We use a standard tri-cubic interpolant wic gives good accuracy and stability; a scematic of te extension is sown in Fig. 2c. Te numerical properties of te Closest Point Metod are fairly well-understood [15, 16, 19, 20, 22] and igly accurate computations ave been done up to fift-order accuracy [21]. In some cases, stability analysis as also been performed [22, 23]. Te operations can also be accelerated using a multigrid strategy [23] altoug we do not do so ere.

3 2.6. Banding We ave described te algoritm as taking place on te voxel grid of a full cube. Tis will work fine in practice but many of te grid points are not required and tus te code can be made more efficient by working on a narrow band of points surrounding te surface. For any particular discretization, te minimum bandwidt is easily computed as a small multiple of and using a wider band will ave no effect on te solution. Notably, no artificial boundary condition need be imposed at te edges of te band [15]. 3. PERONA MALIK DENOISING Te Perona Malik equation [5, 6] is a classic modification of Gaussian diffusion tat employs edge preservation by varying te diffusion coefficient across te image, penalising it at edges and structures tat sould be preserved in te image. Te equation in its general form is t = div ( g( u ) u ), (4) were te image is defined on a finite rectangle Ω R 2 via a function u : R 2 R tat describes te pixel intensity at any given point, for example were one is wite and zero is black. If te diffusion coefficient g : R R is taken to be a constant, we recover te Gaussian diffusion equation. Instead, we take te popular coice 1 g(s) =. (5) 1 + s2 λ 2 Edges are detected via te gradient: a large gradient u λ means tat we are close to an edge and te diffusion almost stops, since g 0. A small gradient u λ means tat we are away from edges, ence we filter (locally) wit Gaussian diffusion, since g 1. Here λ is a tunable parameter tat controls te sensitivity of te sceme to visual edges; it gives a tresold to separate noise from edges Perona Malik on surfaces To formulate Perona Malik diffusion on a surface S, we simply replace operators wit in-surface operators: t = divs ( g( Su ) Su ). (6) Te original noisy surface image is te initial condition u 0 : S R. We solve tis equation for t [0, t final], were te final time is a second parameter wic controls te amount of denoising Numerical discretization We start wit te tree-dimensional Cartesian form of (4) v t = x y [ ( ) ] g vx 2 + vy 2 + vz 2 v x + ] [g(...) v y + ] [g(...) v z, z and discretize wit a textbook-standard finite difference sceme for parabolic equations. Combined wit forward Euler, te timeevolution in 3D is tus given by ṽ n+1 ijk = v n ijk + τ [ g n i+1/2,j,k D+v x ijk n g n i 1/2,j,k D v x ijk n + g n i,j+1/2,k D y + vn ijk g n i,j 1/2,k D y vn ijk + g n i,j,k+1/2 D+v z ijk n g n i,j,k 1/2 D v z ijk n ]. (7a) Here D α + and D α indicate forward and backward finite differences, in te direction indicated by te superscript α, on a grid spacing of. Te expressions involving g between grid points are calculated as te average of te values at te two neigbouring grid points, e.g., g n i+1/2,j,k = 1 2 (gn i+1,j,k + g n ijk), g n i 1/2,j,k = 1 2 (gn i 1,j,k + g n ijk), (7b) were te nodal gijk n are computed using central finite differences Dc α applied to g( u ) ) gijk n = g ( (D c x v nijk )2 + (Dc y v nijk )2 + (Dc z v nijk )2. (7c) As in (3), after advancing te solution in time using te above sceme, we perform a re-extension of te surface values v n+1 ijk = ṽ n+1 (cp(x ijk )), (8) were ṽ n+1 (cp(x ijk )) is approximated wit tri-cubic interpolation on te surrounding grid points (see Fig. 2c). Te numerical sceme (7) uses a stencil sown in Fig. 2b. Te extension (8) uses te tri-cubic interpolation stencil sown in Fig. 2c. To determine te minimal computational band 2.6, we must be able to apply former stencil at eac point in te latter [15]. In 3D, tis results in a band wit a widt of 4.9. We note tat te computational band scales wit te area of te surface and not wit te volume of te embedding space. In practical computations, we find tis band contains less tan 10% of te (teoretical) voxels of te bounding box of te surface (and te difference is even more significant in igly-resolved calculations). For a cosen number of time-steps, te running time scales linearly wit te number of voxels in te computational band. 4. RESULTS Te results of our algoritm are demonstrated in Fig. 1, wic sows denoising on te surface of a triangulated fis [24]. Gaussian diffusion blurs te image, wile Perona Malik diffusion removes noise wile preserving sarp edges. Our implementation uses MATLAB. Parameters used were = 0.002, λ = 5, and 100 time steps of size τ = = Te fis is about 1 unit long. Te computational band consists of voxels surrounding te fis. Eac of te 100 time-steps takes about 1 second on an Intel Core i7 CPU running at 3.20 GHz. Our surface images ave a range of [0, 1]. In our experiments, we apply additive noise in te embedding volume, normally distributed wit amplitude 0.3, mean 0 and standard deviation 1. However, Perona Malik is not limited to tis noise model. Fig. 3 sows an example of computing on te surface of a globe. In Fig. 3c we sow tat our approac results in a uniform application of denoising over te surface. We also demonstrate tat if one first maps te surface image data onto a plane (easy enoug ere wit a spere but quite ard in general), and naively applies te standard

4 (a) (b) (c) Fig. 3. Denoising on a unit globe. (a) Original noisy surface data. In (b) te data was projected to a 2D image were Perona Malik denoising was performed. We note artifacts near te pole, wile (c) sows Perona Malik diffusion on te surface using our approac gives a uniform treatment all over te surface. We use = 0.01, λ = 5 and 100 time steps of size τ = Te computational band consists of voxels surrounding te surface. Eac time step takes about 0.4 seconds. (a) (b) Fig. 5. Perona Malik denoising of an image texture-mapped onto an open fisbowl of unit radius. (a) Noise added in te embedding volume. (b) Denoised image, using = 0.007, λ = 5, and 109 time steps of τ Te computational band consists of voxels and eac time step takes about 0.6 seconds. (Potograp by te first autor.) 1.2 total average area 1 area 2 area 3 average intensity timesteps Fig. 4. Te average grey level over te image is preserved using Perona Malik denoising. Te average grey level over te small regions sown is also preserved. planar Perona Malik, distortions from te parameterization appear in te image. In particular, if te noise is added in te 3D embedded volume, ten projection onto a 2D plane can easily distort te noise so muc as to make denoising wit te (standard) planar Perona Malik impractical (see Fig. 3b). Altoug tis approac can be ameliorated by a significant modification of te PDE (involving te metric tensor and artificial boundary conditions), tis must be done for eac surface. Tis contrasts wit our approac were te standard 3D Perona Malik PDE is applied independent of te specific geometry. Te average intensity of te entire surface image sould be constant (as can be seen from te divergence teorem on (6)). Fig. 4 sows tis is te case using our algoritm. Additionally, in cartoonlike images, we can expect te average intensity in certain smaller subregions to be invariant. Fig. 4 also sows tis form of contrast preservation. Fig. 5 sows anoter example were an image as been texture mapped onto a curved surface. Te denoising algoritm is performed in te embedding volume. Te surface in tis case is open, wic is easily andled by our algoritm a omogeneous Neumann condition is automatically applied by te re-extension [15]. 5. SUMMARY AND CONCLUSIONS In tis paper, we describe a numerical realization of nonlinear diffusion filters for images on general surfaces. We obtain our filters by combining te in-surface PDE models of Gaussian or Perona Malik edge-preserving diffusion wit te Closest Point Metod. Te re- sulting filter is simple: it alternates a time step of te corresponding diffusion PDE in a 3D volume surrounding te surface wit a reextension step. In tis approac we use te closest point function to represent te surface. Notably, only te re-extension step evaluates te closest point function, and te PDE need not be transformed as in oter approaces to surface PDEs. Te fully discrete algoritm uses standard finite difference tecniques on a voxel grid to approximate te diffusion operators and tri-cubic interpolation to do te re-extension. Our experiments demonstrate tat te filter works well and tat we can andle complex geometries. Here we ave considered Gaussian and Perona Malik diffusion on surfaces but te Closest Point Metod is not limited to tese two models. Indeed, te Closest Point Metod applies generally to surface PDEs and could be useful in oter surface image processing tasks. Applications oter tan denoising include deblurring and inpainting and are part of ongoing researc. 6. REFERENCES [1] R.-J. Lai and T.F. Can, A framework for intrinsic image processing on surfaces, Comput. Vis. Image Und., vol. 115, no. 12, pp , [2] R. Kimmel, Intrinsic scale space for images on surfaces: Te geodesic curvature flow, in Grapical Models and Image Processing. 1997, Springer-Verlag. [3] A. Spira and R. Kimmel, Geometric curve flows on parametric manifolds, J. Comput. Pys., vol. 223, [4] I. Bogdanova, X. Bresson, J.-P. Tiran, and P. Vandergeynst, Scale-space analysis and active contours for omnidirectional images, IEEE Trans. Image Process., vol. 16, no. 7, [5] P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, in Proceedings of IEEE Computer Society Worksop on Computer Vision, [6] P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, IEEE Trans. on Pattern Analysis and Macine Intelligence, vol. 12, [7] J. Weickert, Anisotropic Diffusion in Image Processing, B.G. Teubner Stuttgart, 1998.

5 [8] Y. Lin and W.B. Seales, Opaque document imaging: Building images of inaccessible texts, in Proc. ICCV 05, 10t IEEE International Conference on Computer Vision vol. 1, [9] R.G. Brown, C.E. Hann, and J.G. Case, Vision-based 3D surface motion capture for te DIET breast cancer screening system, Int. J. Comput. Appl. T., vol. 39, no. 1, pp , [10] M. Desbrun, M. Meyer, P. Scröder, and A.H. Barr, Anisotropic feature-preserving denoising of eigt fields and bivariate data, in Grapics Interface, 2000, pp [11] U. Clarenz, U. Diewald, and M. Rumpf, Processing textured surfaces via anisotropic geometric diffusion, IEEE Trans. Image Process., vol. 13, no. 2, pp , [12] C.L. Bajaj and G. Xu, Anisotropic diffusion of surfaces and functions on surfaces, ACM Trans. Grapic., vol. 22, no. 1, pp. 4 32, [13] G. Dziuk and C.M. Elliott, Surface finite elements for parabolic equations, J. Comp. Mat., vol. 25, no. 4, [14] M. Bertalmìo, L.-T. Ceng, S. Oser, and G. Sapiro, Variational problems and partial differential equations on implicit surfaces, J. Comp. Pys., vol. 174, no. 2, [15] S.J. Ruut and B. Merriman, A simple embedding metod for solving partial differential equations on surfaces, J. Comput. Pys., vol. 227, no. 3, [16] T. März and C.B. Macdonald, Calculus on surfaces wit general closest point functions, SIAM J. Numer. Anal., vol. 50, no. 6, [17] L. Tian, C.B. Macdonald, and S.J. Ruut, Segmentation on surfaces wit te Closest Point Metod, in Proc. ICIP09, 16t IEEE International Conference on Image Processing, [18] S. Auer, C.B. Macdonald, M. Treib, J. Scneider, and R. Westermann, Real-time fluid effects on surfaces using te Closest Point Metod, Comput. Grap. Forum, vol. 31, no. 6, [19] C.B. Macdonald and S.J. Ruut, Te implicit Closest Point Metod for te numerical solution of partial differential equations on surfaces, SIAM J. Sci. Comput., vol. 31, no. 6, [20] I. von Glen, T. März, and C.B. Macdonald, An embedded metod-of-lines approac to solving PDEs on surfaces, submitted, [21] C.B. Macdonald and S.J. Ruut, Level set equations on surfaces via te Closest Point Metod, J. Sci. Comput., vol. 35, no. 2 3, [22] T. März and C.B. Macdonald, Consistency and stability of closest point iterations, in prep., [23] Y.-J. Cen and C.B. Macdonald, Te Closest Point Metod and multigrid solvers for elliptic equations on surfaces, submitted, [24] M. Attene and C. Pizzi, Fis, in te AIM@SHAPE repository ttp://sapes.aimatsape.net/view.pp? id=262, 2005.