Balanced Multiresolution for Symmetric/Antisymmetric Filters

Transcription

1 Balanced Multiresolution or Symmetric/Antisymmetric Filters Mahmudul Hasan Faramarz F. Samavati Mario C. Sousa Department o Computer Science University o Calgary Alberta Canada {mhasan samavati smcosta}@ucalgary.ca Abstract Given a set o symmetric/antisymmetric ilter vectors containing only regular multiresolution ilters the method we present in this article can establish a balanced multiresolution scheme or images allowing their balanced decomposition and subsequent perect reconstruction without the use o any extraordinary boundary ilters. We deine balanced multiresolution such that it allows balanced decomposition i.e. decomposition o a high-resolution image into a low-resolution image and corresponding details o equal size. Such a balanced decomposition makes on-demand reconstruction o regions o interest eicient in both computational load and implementation aspects. We ind this balanced decomposition and perect reconstruction based on an appropriate combination o symmetric/antisymmetric extensions near the image and detail boundaries. In our method exploiting such extensions correlates to perorming sample (pixel/voxel) split operations. Our general approach is demonstrated or some commonly used symmetric/antisymmetric multiresolution ilters. We also show the application o such a balanced multiresolution scheme in real-time ocus+context visualization. Keywords: multiresolution reverse subdivision balanced decomposition perect reconstruction lossless reconstruction ocus+context visualization contextual close-up symmetric extension antisymmetric extension. Introduction Applications that acilitate multiscale D and D image visualization and exploration (see [LHJ99 WS05] or example) beneit rom multiresolution schemes that decompose high-resolution images into low-resolution approximations and corresponding details (usually wavelet coeicients). Several subsequent applications o such a decomposition constructs the corresponding wavelet transorm. This wavelet transorm can then be used to derive low-resolution approximations o the entire image as well as high-resolution approximations o a region o interest (ROI) on demand. Reconstructing the high-resolution approximation o a ROI involves locating the corresponding details rom a hierarchy o details within the wavelet transorm. One such hierarchy o details resulting rom only two levels o decomposition o a 5 56 Blue Marble image [VEN0] is shown in Figure. For the purpose o demonstration we created the wavelet transorm in Figure using the short ilters o quadratic B-spline presented by Samavati et al. [SB0 SBO07]. In practice images that require multiscale visualization are larger in size and may require more levels o decomposition. For each level o decomposition in this particular example the image was irst decomposed heightwise and then widthwise. The rectangles that contain details are numbered in the order o their creation during the two levels o decomposition. Given such a hierarchy o details i we need to reconstruct the high-resolution approximation o a ROI located in the low-resolution (coarse) image shown in the top-let rectangle in Figure we have to locate the corresponding details in some or all o the rectangles that contain details (numbered.....) depending on the expected level o resolution. Locating these details will be straightorward i each o the heightwise and widthwise decomposition steps decomposes an image into two halves o equal size one hal corresponding to the coarse image and the other hal corresponding to the details. Among B-spline wavelets only the ilters obtained rom Haar wavelets provide such a balance in decomposition [Haa0 SDS96]. However because Haar wavelets and the associated scaling unctions are not continuous it would be beneicial to achieve such a balanced decomposition or the ilters obtained rom higher order scaling unctions and their wavelets. May 0

2 (.) (.) (.) (.) Figure : Hierarchy o details in a wavelet transorm resulting rom two levels o decomposition o a 5 56 Blue Marble image. The size o the image in this igure is not representative o its original size. There exist many sets o local regular multiresolution ilters obtained rom higher order scaling unctions and their wavelets (see [SBO07 CDF9 Dau9 Mey90] or example). They require special treatments to handle image and detail boundaries (such as the use o extraordinary boundary ilters) and result in unequal numbers o coarse and detail samples ater decomposition. Thereore in this article we present a method to devise balanced multiresolution schemes or such ilters using an appropriate combination o symmetric/antisymmetric extensions near the image and detail boundaries which correlate to sample split operations. To guarantee a perect (lossless) reconstruction without the use o any extraordinary boundary ilters our method requires each o the given decomposition and reconstruction ilter vectors (kernels) to be either symmetric or antisymmetric about their centers. Many existing sets o local regular multiresolution ilters such as those associated with the B-spline wavelets [SBO07] biorthogonal and reverse biorthogonal wavelets [CDF9 Dau9] and Meyer wavelets [Mey90 Dau9] exhibit such symmetric/antisymmetric structures. Additionally we show the use o such a balanced multiresolution scheme in an application that exploits real-time ocus+context visualization. This article is organized as ollows. In section we present the notations used throughout the article. Next we ormulate the problem deinition in section which is ollowed by a brie survey o the existing related work in section. Section 5 presents our method or devising a balanced multiresolution scheme accompanied by two examples. We demonstrate the application o a balanced multiresolution scheme devised by our method in real-time ocus+context visualization with experimental results in section 6. In section 7 we discuss some characteristics o our method with possible directions or uture work. Finally section 8 concludes the article. We also provide two appendices with additional examples o balanced multiresolution schemes devised by our method.. Notation In this article we adopted the notations or representing multiresolution operations used by Samavati et al. in [SBO07]. Multiresolution operations are speciied in terms o analysis ilter matrices A k and B k and synthesis ilter matrices P k and Q k. Given a column vector o samples C k a lower-resolution sample vector C k is obtained by the application o a downsampling ilter on C k. This can be expressed by the matrix equation C k = A k C k. The details D k lost ater downsampling are captured using B k as ollows: D k = B k C k. This process o obtaining the low-resolution sample vector C k and the corresponding details D k rom a given highresolution sample vector C k is known as decomposition. Note that the sequences o samples along each dimension o

3 an image can be treated independently during decomposition. Thereore any such sequence o samples can orm the column vector o samples C k or decomposition. The process o recovering the original high-resolution sample vector C k rom the previously obtained low-resolution sample vector C k and the corresponding details D k is known as reconstruction. The reconstruction process requires the reinement o the low-resolution sample vector C k and the corresponding details D k by the application o synthesis ilters P k and Q k as ollows: C k = P k C k + Q k D k. This equation reverses the prior application o A k and B k on the given high-resolution sample vector C k. Thereore decomposition and reconstruction are inverse processes satisying [ ] A k [ P B k k Q ] [ ] I 0 k =. 0 I I we recursively decompose a high-resolution sample vector C k into its coarser approximations C l C l+... C k and details D l D l+... D k then the sequence C l D l D l+... D k is known as a wavelet transorm. Here l < k and C l is the very coarse approximation o the dataset. Each o C l+... C k C k can be reconstructed rom the wavelet transorm C l D l D l+... D k. To simply the notations or the rest o this article we may omit the superscript k or the kth level o resolution assuming F = C k C = C k D = D k A = A k and B = B k P = P k and Q = Q k. We urther assume that the matrices are o appropriate size to satisy the ollowing equations: C = AF () D = BF () F = PC + QD. () Let a and b denote the ilter vectors containing the nonzero entries in a representative row o A and B respectively. Similarly let p and q stand or the ilter vectors containing the nonzero entries in a representative column o P and Q respectively. Furthermore let sizeo (V) represent the number o elements in vector V and the widths o ilter vectors a and b be represented by w a and w b respectively i.e. sizeo (a) = w a and sizeo (b) = w b.. Problem As mentioned earlier the method we present in this article can be used to devise a balanced multiresolution scheme or a given set o regular multiresolution ilters comprised o ilter vectors a b p and q such as the set given in (). I each o a b p and q is either symmetric or antisymmetric our method will guarantee a perect reconstruction without the use o any extraordinary boundary ilters. Due to the regularity present in an image structure the multiresolution methods applicable to curves and regular meshes are also applicable to images (see subsection ). For end point and boundary interpolations extraordinary ilters (as opposed to regular ilters) are used in multiresolution methods or curves and regular meshes respectively. However the use o extraordinary ilters at image boundaries or boundary interpolation assigns incongruous importance to the image boundaries. So or D or D image decomposition the general practice is to use symmetric extensions near the image boundaries to avoid boundary case evaluations using extraordinary ilters [SBO07]. However an arbitrary choice o symmetric extension or decomposition while using a given set o multiresolution ilters may eventually lead to the use o extraordinary boundary ilters or a perect reconstruction. This can also make on-demand reconstruction o image parts corresponding to a ROI computationally untidy near the image boundaries. Thereore a careul setup o symmetric/antisymmetric extensions or both decomposition and reconstruction is required which can be obtained by our presented method. Hence we deine our problem statement as ollows. Problem deinition. Given a set o regular multiresolution ilters in the orm o symmetric/antisymmetric ilter vectors a b p and q devise a balanced multiresolution scheme applicable to a high-resolution column vector o samples F that satisies:

4 (i) C = AF and D = BF analogous to equations () and () where F F through symmetric extensions at its boundaries and the nonzero entries in each row o A and B correspond to the regular ilters in the given ilter vectors a and b respectively; (ii) sizeo (C) = sizeo (D) i.e. a balanced decomposition; (iii) sizeo (C) + sizeo (D) = sizeo (F) i.e. a compact representation o the resulting wavelet transorm hereater reerred to as a balanced wavelet transorm; and (iv) F = PC + QD analogous to equation () where C C and D D through symmetric/antisymmetric extensions at their boundaries and the nonzero entries in each column o P and Q correspond to the regular ilters in the given ilter vectors p and q respectively.. Related Work In the next three subsections we review the existing related work within the ollowing three categories: multiresolution symmetric and antisymmetric extensions and ocus+context visualization... Multiresolution Regular meshes. Here we review the multiresolution methods applicable to curves and tensor-product meshes (suraces and volumes) given their applicability to multidimensional images due to their regular structure. Hierarchical representation o multiresolution tensor-product suraces was made possible due to the pioneering work o Forsey and Bartels [FB88]. They localized the editing eect in a desired manner on tensor-product suraces through hierarchically controlled subdivisions. This was done by adding iner sets o B-splines onto existing coarse sets. However it resulted in an over-representation because the union o the sets o basis unctions rom dierent resolutions did not orm a set o basis unctions. Adding complementary basis unctions to the coarse set o basis unctions is a possible way to resolve the problem o over-representation. This means o supporting multiresolution is closely aligned to the wavelet theory approach to multiresolution [SDS96]. Wavelet representations o details may however introduce undesired undulations as pointed out by Gortler and Cohen [GC95]. Furthermore under this approach optimizing the behaviour o the analysis (decomposition) using least squares is diicult due to the need to support interactive mesh manipulations [ZSS97]. Samavati and Bartels pioneered in their work on a mathematically clean and eicient approach to multiresolution based on reverse subdivision [SB99 BS00 BGS06 BS]. Under this approach during the analysis each coarse vertex is obtained by eiciently solving a local least squares optimization problem. The use o least squares optimization reduces the undesired undulations. Additionally the resulting wavelets provide a much more compact support compared to the conventional wavelets or curves and regular suraces. Some o the examples demonstrating the application o our proposed method use multiresolution ilters resulting rom this approach (see the examples in section 5 or instance). Images. Notable existing approaches obtaining a multiresolution representation supporting context-aware visualization o D images include the wavelet tree [WS05] segmentation o texture-space into an octree [LHJ99 PTCF0 PHF07] octree-based tensor approximation hierarchy [SGM ] and trilinear resampling on the Graphics Processing Unit (GPU) coupled with the deormation o regularly partitioned image regions [WWLM]. For D images the wavelet-based time-space partitioning (WTSP) tree was proposed in [WS05]. In [WS05] Haar [Haa0 SDS96] and Daubechies s D [Dau88] wavelets were used to construct the wavelet transorms in each node o the wavelet and WTSP trees... Symmetric and Antisymmetric Extensions As mentioned earlier we achieve balanced decomposition and subsequent perect reconstruction based on the use o an appropriate combination o symmetric and antisymmetric extensions near the image and detail boundaries. In the literature symmetric and antisymmetric extensions were used in the context o various types o wavelet transorms [LL00 KZT0 AW0 LS08]. In contrast our proposed method allows the construction o a balanced wavelet transorm. Figure shows three types o extensions as deined in [KNI9]. Consider a sequence o n samples (... n ) corresponding to a column vector o samples [... n ] T where n N and n. Figure (a) (b) and (c)

5 n n n- (a) Hal-sample symmetry. n n- n- (b) Whole-sample symmetry. - - n - n - n- (c) Hal-sample antisymmetry. Figure : Symmetric and antisymmetric extensions. show the extended sequences obtained through hal-sample symmetric whole-sample symmetric and hal-sample antisymmetric extensions respectively at both ends o (... n ). Whole-sample antisymmetry not shown in Figure can be obtained by negating the samples in the extensions o Figure (b). Note that in real applications the types o extensions at both ends o a sequence do not necessarily have to be the same (as used in Figure or example). To be consistent with the coloring used in Figure rom this point orward in this article notations and igures may use red purple and green to denote the samples introduced by hal-sample symmetric whole-sample symmetric and hal-sample antisymmetric extensions respectively... Focus+Context Visualization Because we chose to demonstrate the use o a balanced multiresolution scheme resulting rom our method in a real-time ocus+context visualization application here we review some o the notable related work. (a) A circular ROI. (b) A rectangular ROI. Figure : Traditional ocus+context visualization in medical illustrations. (a) Thrombosis in human brain. Copyright Fairman Studios LLC. Used with permission. (b) Aneurysm in human brain. Copyright University o Washington Medicine Stroke Center. In many visualization tasks it is useul to simultaneously visualize both the local and global views o the data possibly at dierent scales which is known as ocus+context visualization. One approach to implement ocus+context 5

6 is to use the metaphor o lenses [TSS 06 WWLM HMC]. This metaphor is inspired by techniques used in traditional medical (see Figure ) technical and scientiic illustrations [Hod0]. Our implemented approach to ocus+context visualization o multidimensional images is closest to the technique presented by Taerum et al. or the visualization o small-scale clinical volumetric datasets [TSS 06]. In their approach the resolution o a given D image is reduced by one level using reverse subdivision [SB99 BS00] which is rendered during user interactions to achieve interactive rame rates. The D image is rendered in the original resolution while there is no user-interaction. The ROI identiied by a query window is enlarged by the application o B-spline subdivision to allow dierent levels o smoothness. Thereore the authors used only three dierent levels o resolution. In contrast our implementation or multiresolution visualization o images provides a true multiresolution ramework where the resolutions o both the coarse image (providing context inormation) and the enlarged ROI (providing ocus inormation) can be controlled by the user. 5. Methodology: Balanced Multiresolution In this section we explain and demonstrate by examples how our method achieves balanced decomposition and subsequent perect reconstruction by choosing an appropriate combination o symmetric and antisymmetric extensions near the image and detail boundaries. 5.. Balanced Decomposition We deined balanced decomposition as the task o decomposing a high-resolution image into a low-resolution image and corresponding details o equal size. Balanced decomposition o a D image o dimensions w h s results in an image o dimensions w h s ater one level o widthwise heightwise and depthwise decomposition. To allow l levels o balanced decomposition we need the ollowing conditions to be satisied: w = l m h = l n and s = l p where m n p Z +. Disregarding the third dimension iners the same idea or a D image. Once the ideal dimensions are known the high-resolution image should be uniormly resampled to those dimensions beore the application o our balanced decomposition procedure. Given the decomposition ilter vectors a and b to achieve a balanced decomposition o a column vector containing an even number o ine samples F we irst decide on the type o symmetric extension to use or decomposition based on the parity o w a and w b. Then an extended column vector o ine samples F is obtained rom F through the chosen type o symmetric extension such that sizeo (F ) ensures the generation o sizeo (F)/ coarse samples and sizeo (F)/ detail samples by a subsequent application o ilter vectors a and b on F respectively. Demonstration by example. Beore we outline the general construction or the balanced decomposition process here we demonstrate how it works or a given set o decomposition ilter vectors. In this example we consider the decomposition ilter vectors a and b rom ollowing set o local regular multiresolution ilters [SB0 SBO07]: a = [ b = [ p = [ q = [ ] ] ] () ]. The ilter vectors in () are known as the short ilters o quadratic (third order) B-spline [SBO07] and were constructed by reversing Chaikin subdivision [Cha7]. Recall rom section that ilter vectors a and b contain the nonzero entries in a representative row o analysis ilter matrices A and B respectively. For the purpose o demonstration assume that we are given a ine column vector o 8 samples F = [... 8 ] T on which we have to perorm a balanced decomposition. Provided sizeo (F) = 8 a balanced decomposition should result in column vectors o coarse samples C = [ c c c c ] T and detail samples D = [ d d d d ] T. In Figure we present one possible setup to obtain such a balanced decomposition. It shows the application o equations C = AF and D = BF analogous to equations () and () where F = [ ] T. First note that F was obtained by extending the given sample vector F by extra samples. In general when the dilation 6

7 c d c d c d c d Figure : Balanced decomposition o 8 ine samples. actor is a given column vector o ine samples F with sizeo (F) = n or n Z + does not have enough samples to accommodate n shits o both a and b or generating n coarse and n detail samples respectively. The number o extra samples x required or a balanced decomposition can be obtained by the general ormula: x = max(w a w b ) + (n ) n (5) x = max(w a w b ). (6) Here we explain how equation 5 evaluates x. We need at least max(w a w b ) ine samples to obtain both c and d which explains the irst term on the right-hand side o equation 5. Next because the dilation actor is every additional samples will guarantee the generation o an additional pair o c i and d i. Here i {... n} because we want to generate {... n} = n more coarse samples and n more detail samples to achieve a balanced decomposition. This indicates the need or an additional (n ) ine samples justiying the addition o the second term on the righthand side o equation 5. Thereore subtracting n i.e. the sizeo (F) in the third term gives us the required number o extra samples. For the amilies o multiresolution ilters we consider in this article w a and w b are either both even or both odd. For example see the decomposition ilter vectors obtained rom B-spline wavelets [SBO07] biorthogonal and reverse biorthogonal wavelets [CDF9 Dau9] and Meyer wavelets [Mey90 Dau9]. The multiresolution ilter vectors obtained rom most such wavelets and their scaling unctions are available in commonly used mathematical sotware packages such as MATLAB [MAT]. For the given ilter vectors a and b in () because both w a and w b are even observe that the extension o F by extra samples to obtain F was achieved by hal-sample symmetric extension at both ends o F. Here we would have used whole-sample symmetric extension instead i both w a and w b were odd. Use o an appropriate type o symmetric extension is required to avoid the use o any extraordinary boundary ilters or a perect reconstruction. We justiy our choice o symmetric extension or a balanced decomposition later in subsection 5.. Finally as shown in Figure the ilter vectors a and b in () are applied to the samples in F to obtain C and D in order to complete the balanced decomposition process. For instance the coarse sample c and the detail sample d are computed rom the irst samples in F as ollows: c = + + (7) d = +. 7

8 Note that the total contribution o in the construction o c is written as + in (7) through an implicit sample split operation. A similar sample split is observed in the construction o d as shown in (7). Thereore the symmetric extensions at both ends o F implicitly lead to a number o sample split operations during decomposition. Thereore or n Z + a balanced multiresolution scheme based on the short ilters o quadratic B-spline given in () can make use o the matrix equations and c c c n n n n d d d n n n n or the decomposition process analogous to equations () and (). General construction. Now we present our general approach or achieving a balanced decomposition. Given the symmetric/antisymmetric decomposition ilter vectors a and b containing only regular ilters carry out the ollowing steps to achieve a balanced decomposition o a ine column vector o samples F where sizeo (F) = n or a suitably large n Z +.. Determine x the number o extra samples required or a balanced decomposition using equation (6).. I both w a and w a are even extend F with x extra samples using hal-sample symmetric extension to obtain F. Use whole-sample symmetric extension instead i both w a and w a are odd. Justiication o our choice o symmetric extension can be ound in subsection 5.. To avoid giving inconsistent importance to any end (boundary) o F: (a) I x is even introduce x/ samples at each end o F. (b) I x is odd introduce x/ samples at one end and x/ + samples at the other end o F. Let us reer to the end at which x/ + samples are introduced as the odd end. Alternate between the ends o F as the choice o the odd end during multiple levels o decomposition.. To obtain C and D such that sizeo (C) = sizeo (D) use equations C = AF and D = BF analogous to equations () and (). 5.. Perect Reconstruction Given the reconstruction ilter vectors p and q that can reverse the application o the decomposition ilter vectors a and b to achieve a perect reconstruction o the column vector o ine samples F rom its prior balanced decomposition into C and D we irst reconstruct as many interior samples o F as possible by the application o p and q on C and D using equation (). To evaluate the samples near each boundary (end) o F we orm a square system o linear equations based on the prior construction o corresponding boundary samples in C and D where the unknowns constitute the boundary samples o F yet to be reconstructed. Symbolically solving two such square systems or the two boundaries o F reveals the extended versions o C and D (denoted by C and D respectively) required or a perect reconstruction by the application o p and q using equation F = PC + QD analogous to equation (). Demonstration by example. Here we demonstrate how we perorm a perect reconstruction o F ollowing its balanced decomposition to C and D by means o an example beore giving the general construction or our perect 8

9 reconstruction process. In this example we consider the reconstruction ilter vectors p and q given in (). Recall rom section that ilter vectors p and q contain the nonzero entries in a representative column o synthesis ilter matrices P and Q respectively. This example to demonstrate our perect reconstruction process is an extension o the example shown in Figure. So rom the resulting column vectors coarse samples C = [ c c c c ] T and detail samples D = [ d d d d ] T in subsection 5. we now want to reconstruct the corresponding column vector o ine samples F = [... 8 ] T. Figure 5: Reconstruction o 6 o the 8 ine samples. In Figure 5 we show the application o the ilter vectors p and q to the samples in C and D respectively. For instance the ine sample is reconstructed rom the irst two coarse samples and the irst two detail samples as ollows: = c + c + d d. Note that the application o the ilter vectors p and q to the samples in C and D in Figure 5 let two samples and 8 near the two ends o F not reconstructed. Note that having two samples near the boundaries o F yet to reconstruct is speciic to this example. The example in subsection 5. receives 5 samples yet to reconstruct at this stage. Now to reconstruct we orm the ollowing system o linear equations based on the prior construction o c (as shown in Figure ) to which made some contribution during decomposition: c = + + (8) = c + ( ) = c c + c + d d + ( c + c + d ) d = c d. (9) Although it appears rom equation (9) that is not reconstructed using regular ilters our prior appropriate choice o symmetric extension to obtain F rom F (justiied later in subsection 5.) guarantees that we can rewrite using the regular ilter values rom p and q in (). This is achieved by a rearrangement o the right-hand side o equation (9) which is implicitly equivalent to perorming two sample split operations: = c + c + (d ) d. (0) 9

10 c c c c c c d d d d d d Figure 6: Perect reconstruction o 8 ine samples. This rewriting step is important because it allows the reconstruction o ine samples near the boundaries o F without the use o any extraordinary boundary ilters. Equation (0) now yields the introduction o one extra coarse sample through hal-sample symmetric extension and one extra detail sample through hal-sample antisymmetric extension or the reconstruction o as shown in Figure 6. We use a similar approach to determine how to reconstruct the boundary sample 8 resulting in 8 = c + c + d (d ) () as relected in Figure 6. This concludes the perect reconstruction process. Thereore based on our indings rom (0) and () or a given column vector o n ine samples or a suitably large n Z + we get = c + c + (d ) d n = c n + c n + d n (d n). So a balanced multiresolution scheme based on the short ilters o quadratic B-spline given in () will make use o the matrix equation c d c d c d n c n d n c n dn or the reconstruction process analogous to equation (). 0

11 General construction. Now we describe our general approach to achieve perect reconstruction. Given the symmetric/antisymmetric reconstruction ilter vectors p and q containing only regular ilters that can reverse the application o the decomposition ilter vectors a and b carry out the ollowing steps to perectly reconstruct the column vector o ine samples F rom its prior balanced decomposition into C and D.. Assume that F = [ F l T F m T F r T ] T where Fl and F r respectively contain some samples at the let and right boundaries o F and F m contains the remaining interior samples o F. To reconstruct the samples in F m use the equation F m = PC + QD analogous to equation (). The samples in F l and F r are yet to be reconstructed. (In the example above we had F l = [ ] Fm = [... 7 ] T and Fr = [ 8 ]. Note that Fl and F r may contain more samples; or instance the F l and F r encountered in 5. have and samples respectively.). To reconstruct the samples in F l : (a) Form a system o linear equations based on the prior construction o some coarse and detail boundary samples to which the ine samples in F l made some contributions during the decomposition process. It should be a q q system where q = sizeo (F l ) and the unknowns are the samples o F l. (For example see the system ormed by equation (8) and the system ormed by the two equations in (9).) (b) Solving the system ormed in step (a) symbolically will evaluate the samples in F l as a linear combination o some samples rom C and D. (For example see equation (9) and the two equations in (0).) (c) Rewrite the linear combination(s) o coarse and detail samples on the right-hand side(s) o the equation(s) obtained in step (b) using the regular ilter values rom the ilter vectors p and q as coeicients. Such rewriting o ine samples here correlates to perorming sample split operations. This will reveal the ollowing two pieces o inormation applicable to the let boundaries o C and D or a perect reconstruction: (i) the type o symmetric/antisymmetric extension that must be used and (ii) the number o extra samples that must be to introduced. (For example see equation (0) and the equations in ().). Use an approach similar to that in step to reconstruct the samples in F r. Note that steps - above allow the generation o C and D respectively rom C and D such that condition (iv) o the problem deinition given in section is satisied. 5.. Choice o Symmetric Extension or Decomposition Claim. For a given set o symmetric/antisymmetric multiresolution ilter vectors a b p and q even values o w a and w b imply the use o hal-sample symmetric extensions at the image boundaries during a balanced decomposition to ensure a perect reconstruction only using the regular reconstruction ilters rom p and q. On the other hand odd values o w a and w b imply the use o whole-sample symmetric extensions instead. Proo outline. We outline the proo by means o an example that makes use o the ilter vectors containing only regular ilters a = [ ] a a a a b = [ b b b b ] p = [ p p p p ] q = [ q q q q ]. The widths o the ilter vectors a b p and q in () are assumed to be as in the case o the ilter vectors containing the short ilters o quadratic B-spline in (). So here w a and w b are even. Next two possible balanced decompositions o a ine column vector o 8 samples F = [... 8 ] T are shown by the use o hal-sample and whole-sample symmetric extensions at its boundaries in Figures 7(a) and 7(b) respectively. ()

12 a a a a b b b b a a a a b b b b a a a a b b b b a a a a b b b b (a) Balanced decomposition using hal-sample symmetric extension. a a a a b b b b a a a a b b b b a a a a b b b b a a a a b b b b (b) Balanced decomposition using whole-sample symmetric extension. Figure 7: Balanced decomposition o 8 ine samples. Now our goal is to perectly reconstruct F rom the column vectors o coarse samples C = [ c c c c ] T and detail samples D = [ d d d d ] T using only the regular reconstruction ilters vectors p and q rom () as shown in Figure 8. p p q q p p q q p p q q p p q q p p q q p p q q p p q q p p q q Figure 8: Perect reconstruction o 8 ine samples. We intend to evaluate the unknowns in Figure 8 which are αc i {c c c c } βc j {c c c c } γd k {d d d d } and δd l {d d d d } near the boundaries o C and D. Once evaluated these will reveal the type o symmetric/antisymmetric extensions to be used at the boundaries o C and D to ensure a perect reconstruction using only the regular reconstruction ilters. Here α β γ δ {+ } represent the signs o c i c j d k and d l respectively. When negative they allow the representation o antisymmetric extensions. Now let us try to evaluate αc i. As shown in Figure 8 αc i contributes to the reconstruction o. I we consider the balanced decomposition shown in Figure 7(a) and try to evaluate ollowing our general approach rom subsection 5. we get c = a + a + a + a = a + a c a a + a a a + a

13 = = a c (p c + p c + q d + q d ) a + a a + a a (p c + p c + q d + q d ) a + a ( ) ( a p a p a p a p c + a + a ( a q a q + a + a ) d + a + a ( a q a q a + a ) c ) d. () Next i we consider the balanced decomposition shown in Figure 7(b) and try to evaluate ollowing our general approach rom subsection 5. we get c = a + a + a + a = = = a c a + a a a a a c a + a a (p c + p c + q d + q d ) + a (p c + p c + q d + q d ) a ( ) ( a p a p a p a p a p a p c + a a ( ) ( a q a q a q a q a q a q + d + a a ) c ) d. () Let the ilter values multiplied to c and c in the reconstruction o be denoted by w(c ) and w(c ) respectively. In () w(c ) = a p a p a +a (5) w(c ) = a p a p a +a which result rom using hal-sample symmetric extension at the let boundary F or a balanced decomposition. On the other hand in () w(c ) = a p a p a p a (6) w(c ) = a p a p a p a which result rom using whole-sample symmetric extension instead. Now according to Figure 8 is reconstructed as ollows: = p (αc i ) + p c + q (αd k ) q d. (7) I we consider αc i = c in equation (7) or example then w(c ) = p + p and w(c ) = 0. I c is substituted in Figure 8 in place o αc i it would then reveal the need or hal-sample antisymmetric extension or the let boundary o C to be used during reconstruction. In this manner Table lists the suicient conditions or all possible values o αc i. Note that each possible value o αc i yields a particular type o extension (listed in Table ) or the let boundary o C. Now i we substitute the actual values o the corresponding regular ilters o quadratic B-spline rom () in (5) and (6) we ind that (5) only satisies the suicient conditions under case I (i.e. αc i = c ) in Table and (6) does not satisy the suicient conditions under any o the cases. Recall that (5) was obtained by the use o hal-sample symmetric extension on the let boundary o F or a balanced decomposition. This implies that the use o hal-sample symmetric extension at the let boundary o F or a balanced decomposition will ensure the perect reconstruction o that boundary olely using regular reconstruction ilters. Similarly or the regular ilters o quadratic B-spline rom () we can show that βc j = c γd k = d and δd l = d ; and they all require the use o hal-sample symmetric extension at the boundaries o F or a balanced decomposition.

14 Table : Suicient conditions or symmetric and antisymmetric extensions. Case Suicient Conditions αc i Type o Extension w(c ) = p + p I c Hal-sample symmetry w(c ) = 0 w(c ) = p + p II c Hal-sample antisymmetry w(c ) = 0 w(c ) = p III c Whole-sample symmetry w(c ) = p w(c ) = p IV c Whole-sample antisymmetry w(c ) = p In the above manner we can show that or any set o symmetric/antisymmetric ilter vectors a b p and q where w a and w b are even hal-sample symmetric extension can be used at the boundaries o a column vector o ine samples or a balanced decomposition to ensure a perect reconstruction only using the regular reconstruction ilters rom p and q. A similar proo can be outlined to show that odd values o w a and w b imply the use o whole-sample symmetric extension instead. 5.. Further Demonstration by Example The example in this subsection illustrates the use o decomposition ilter vectors o odd width or a balanced decomposition as opposed to the even width o decomposition ilter vectors in the previous example (subsections 5. and 5.). Further examples are provided in Appendix A. Balanced decomposition. Here we demonstrate our general approach described in subsection 5. using the decomposition ilter vectors a and b rom ollowing set o local regular multiresolution ilters [BS00 SBO07]: a = [ ] b = [ ] 8 8 p = [ ] (8) 8 8 q = [ ] The ilter vectors in (8) are known as the inverse powers o two ilters o cubic (ourth order) B-spline [SBO07]. We explain the balanced decomposition process using the decomposition ilter vectors in (8) through the example shown in Figure 9. Similar to the previous example shown in Figure here we have a column vector o 8 ine samples F = [... 8 ] T that we want to decompose into the column vectors o coarse samples C = [ c c c c ] T and detail samples D = [ d d d d ] T. Figure 9 shows one possible balanced decomposition using our general approach presented in subsection 5.. Step o our general construction given in subsection 5. reveals that 5 extra samples are required to ensure a balanced decomposition. As noted earlier w a and w b or the ilter vectors in (8) are odd. So according to step whole-sample symmetric extension is used to introduce extra samples at one end and extra samples at the other end o F to obtain the extended column vector o ine samples F = [ ] T. Finally according to step the ilter vectors a and b rom (8) are applied to F to obtain C and D by means o the equations C = AF and D = BF analogous to equations () and (). Thereore or n Z + a balanced multiresolution scheme based on the inverse powers o two ilters o cubic B-spline given in (8) can make use o the matrix equations

15 c 8 8 d c 8 8 d c 8 8 d c 8 8 d Figure 9: Balanced decomposition o 8 ine samples. and c c = n c n n n n n d d = dn n n n or the decomposition process analogous to equations () and (). Perect reconstruction. Here we demonstrate our general approach described in subsection 5. using the reconstruction ilter vectors p and q given in (8). They can reverse the application o the decomposition ilters vectors a and b rom (8). Given the column vectors o coarse samples C = [ c c c c ] T and detail samples 5

16 D = [ d d d d ] T (obtained as shown in Figure 9) we now want to perectly reconstruct the column vector ine samples F = [... 8 ] T Figure 0: Reconstruction o o the 8 ine samples. Figure 0 shows the reconstruction o F m = [ 5 ] T according to step o our general construction given in subsection 5.. F l = [ ] T and Fr = [ ] T are yet to be reconstructed. Next ollowing step (a) o our given general construction we orm the ollowing system o linear equations in unknowns ( and in F l ): c = d = The equations in (9) were obtained rom Figure 9 which shows how c and d were computed during decomposition. Note that in (9) we can replace and 5 with the corresponding linear combinations o coarse and detail samples rom Figure 0. Then ollowing step (b) solving the system ormed by the equations in (9) gives = c d + d (0) = 7 8 c + 8 c + 8 d + d + 8 d. Now according to step (c) the equations in (0) can be rewritten as ollows such that the coeicients o the coarse and detail samples are all regular ilters rom (8): = c + c + d + ()d + d () = 8 c + c + 8 c + 8 d + 8 d + 8 d + 8 d. This rewriting required two implicit sample split operations on the right-hand side o each equation in (). Finally ollowing step o our general construction to reconstruct F r we orm the ollowing system o linear equations in unknowns ( 6 7 and 8 in F r ): c = c = d = The equations in () were obtained rom Figure 9 which shows how c c and d were evaluated during decomposition. Observe that in () we can replace and 5 with the corresponding linear combinations o coarse and detail samples rom Figure 0. Then solving the system ormed by the equations in () gives 6 = 8 c + c + 8 c + 8 d + 8 d + d 7 = c + c + d d 8 = c + c + d + d. (9) () () 6

17 Now the equations in () can be rewritten as ollows such that the coeicients o the coarse and detail samples are all regular ilters rom (8): 6 = 8 c + c + 8 c + 8 d + 8 d + 8 d + 8 d 7 = c + c + d + ()d + d () 8 = 8 c + c + 8 c + 8 d + 8 d + 8 d + 8 d. c c c c c c d d d d d d d Figure : Perect reconstruction o 8 ine samples. As we mentioned in the general construction given in subsection 5. note that the equations in () and () yield a speciic type o symmetric extension or each boundary o C and D as shown in Figure. Thereore based on () and () or a given column vector o n ine samples (n Z + ) we get = c + c + d + ()d + d = 8 c + c + 8 c + 8 d + 8 d + 8 d + 8 d n = 8 c n + c n + 8 c n + 8 d n + 8 d n + 8 d n + 8 d n n = c n + c n + d n + ()d n + d n n = 8 c n + c n + 8 c n + 8 d n + 8 d n + 8 d n + 8 d n. So a balanced multiresolution scheme based on the inverse powers o two ilters o cubic B-spline given in (8) can make use o the matrix equation c d c d c d cn 0 0 dn n dn c n c d n n dn

18 or the reconstruction process analogous to equation (). 6. Application in Focus+Context Visualization Multiscale D and D image visualization applications oten exploit query window-based ocus+context visualization or image exploration and navigation purposes. A low-resolution approximation is rendered to provide the context and a selected portion o that low-resolution approximation deining the ocus also known as the ROI is rendered as a close-up in high-resolution. While such visualization is supported by an underlying wavelet transorm it is necessary to reconstruct the high-resolution approximation o the ROI on demand rom the low-resolution approximation and corresponding details. Here the use o a balanced wavelet transorm constructed by our proposed method makes locating the details straightorward. For instance observe the reconstruction o interior samples in Figures 5 and 0. I the irst coarse sample or the reconstruction o a ine sample is c i then irst detail sample to use in the reconstruction o that ine sample is d i. This may not have been the case i we had an unequal number o coarse and detail samples rom decomposition. Also the only additional step required to reconstruct the ine samples near the boundaries is the use o speciic symmetric/antisymmetric extensions because our method completely eliminates the need or extraordinary boundary ilters. 6.. Overview o Visualization Tool We have implemented a visualization tool prototype named Focus+Context Studio to test our presented balanced multiresolution ramework or images. It robustly allows real-time multilevel ocus+context visualization o largescale D and D images supported by multiple movable query windows deining ROIs at dierent resolutions. It currently uses the balanced multiresolution scheme we devised using the short ilters o quadratic B-spline in () as described in the examples shown in subsections 5. and 5.. At the moment all the query windows are samples in dimension. To acilitate ocus+context visualization and exploration o a D image our prototype currently allows the query windows identiying the ROIs to move back and orth through sequential slices interactively by the use o mouse scroll wheel and alternatively the up and down arrow keys on the keyboard. When the query windows move rom one slice to another the low-resolution approximation o the context and the high-resolution approximations o the ROIs are updated on the ly in real-time. For D images currently it only perorms widthwise and heightwise decompositions which keeps the number o D slices intact or depthwise volume exploration. 6.. Experimental Results Here we present the experimental results produced by our Focus+Context Studio prototype. The n-tuples (n ) used in the captions o Figures and are deined as ollows: (image dimensions C d F r F r... F r m ) where d is the number o levels o (widthwise and heightwise) decomposition or the context and r i ( i m) is the number o levels o reconstruction or deriving the high-resolution approximation o the ith ROI. F r i appears in the n-tuple in a position determined by the let-to-right and top-to-bottom ordering o placement or the high-resolutions approximations o the ROIs. Figure shows various scenarios or ocus+context visualization o D images using our prototype. Figures (a) and (b) show multilevel ocus+context visualization o large-scale D images showing the topographic and bathymetric shading o northwestern North America and the topographic shading o Long Island respectively. Similar multilevel ocus+context visualization is shown or a diseased lea in Figure (c). Such multilevel ocus+context visualization is motivated by the need or more manageable utilization o screen space and visualization o the context at a higher resolution while maintaining interactive rame rates. Next or a D image Figures (d) and (e) show dierent levels o decomposition or the context and dierent levels o reconstruction or the high-resolution approximation o the ROI using our developed tool. The D image used in this example is an abdomen slice rom the male rom the Visible Human Project [NLM]. One advantage o allowing multiple query windows corresponding to multiple ROIs is the ability to draw comparisons between similar ROIs when required. Figure () shows such a comparison scenario between the tropical storm Parma (on the let) and typhoon Melor (on the right). Another such scenario comparing the ice near the coasts o Greenland and Alexander Island is shown in Figure (g). 8

19 (a) Topographic and bathymetric shading o northwestern North America: ( C6 F F 5 F 5 F F ). (c) Frangipani rust: (096 8 C6 F 6 F F ). (d) Male abdomen: (78 8 C F ). (b) Topographic shading o Long Island: (89 C5 F F F ). () Comparison o Parma (on the let) and Melor (on the right): ( C6 F F ). (e) Male abdomen: (78 8 C F ) (g) Comparison o ice near the coasts o Greenland and Alexander Island on a Blue Marble image: ( C6 F F F F ). Figure : Focus+context visualization o D images at various resolutions.

20 January December February November March October April September May August June July Figure : Focus+context visualization o time-lapse imagery monthly global images: (50 75 C F ).

21 Slice 65 Slice 66 Slice 7 Slice 7 Figure : Focus+context visualization o a D image ( C F F ).

22 Our developed prototype is also suitable or the visualization and exploration o time-lapse imagery. For instance Figure shows unique rames rom the interactive transition through the slices o monthly global images. The order o rames is shown by directions marked on the curved-arrow in the middle. The ROI covers most o northwestern North America and shows the transition rom one winter to the ollowing winter. Figure shows an example o visualization and exploration o a D image in our prototype. For the purpose o demonstration the transition through 0 o the 50 slices that the query windows were constrained to move back and orth through are shown in Figure. The original dataset corresponds to the head o the emale rom the Visible Human Project [NLM]. This head dataset contains a total o 77 D slices each o dimensions among which 50 sequential slices were loaded into our prototype or this example. 7. Discussion and Future Work Our method can be used to devise a balanced multiresolution scheme or any set o given regular multiresolution ilter vectors. However i the scheme would only make use o regular reconstruction ilters is determined by the properties o the given multiresolution ilter vectors. I the given ilter vectors are symmetric/antisymmetric then our method can devise a balanced multiresolution scheme that only uses regular ilters. Otherwise some extraordinary boundary reconstruction ilters are introduced (see Appendix B or instance). The balanced multiresolution schemes devised by our approach can also be applied to open curves and tensor product meshes (suraces and volumes) in applications where boundary interpolation is not important but a balanced decomposition is preerred or reasons such as partitioning the curve or the mesh into even and odd vertices. Such a partitioning allows the storage o coarse vertices and details in even and odd vertices respectively as proposed in [OSB07]. However some o the devised balanced multiresolution schemes may support boundary interpolation only in the context o subdivision i.e. when we only consider the result o PC in order to increase the resolution o C. For example the ilters o second order B-spline in (A.) and the short ilters o third order B-spline in () lead to such boundary interpolating subdivisions. There is a number o directions or uture research. In this article we covered the commonly used types o symmetric and antisymmetric extensions. It would be useul to investigate and develop extension types that can be utilized to devise balanced multiresolution schemes or near symmetric and asymmetric ilter vectors in order to ensure a perect reconstruction solely by the use o regular ilters. To start with the devised balanced multiresolution scheme given in Appendix B or Daubechies asymmetric D ilters may provide some insights. In addition urther investigations are needed or an in-depth understanding o the relations between the symmetry/antisymmetry exhibited by the ilter vectors parity o their widths and the determined types o symmetric/antisymmetric extensions required or a perect reconstruction using only regular ilters. For instance compare the multiresolution ilter vectors containing the inverse powers o two ilters o ourth order B-spline in (8) and the wide and optimal ilters o ourth order B-spline in (A.). In these two sets the corresponding ilter vectors have the same widths and they are all symmetric. Now observe that the two balanced multiresolution schemes we devised using these two sets o ilter vectors suggest exactly the same type o symmetric extensions or the column vectors o ine coarse and detail samples. Thereore the determined types o symmetric/antisymmetric extensions are not dependant on actual ilter values. Several other such scenarios are shown in Table A.. 8. Conclusion In this article we presented a novel method or devising a balanced multiresolution scheme primarily applicable to images using a given a set o symmetric/antisymmetric ilter vectors containing regular multiresolution ilters. A balanced multiresolution scheme resulting rom our method allows balanced decomposition and subsequent perect reconstruction o images without using any extraordinary boundary ilters. This is achieved by the use o an appropriate combination o symmetric and antisymmetric extensions at the image and detail boundaries correlating to implicit sample split operations. Balanced decomposition o an image makes locating the details corresponding to a ROI straightorward in a hierarchy o details within the wavelet transorm representation o that image. In order to support smooth multiresolution representations o images beyond Haar wavelets and the associated scaling unctions and still exploit the advantages o a balanced decomposition we used our method to devise balanced