A Real-Time Volume Rendering Architecture Using an Adaptive Resampling Scheme for Parallel and Perspective Projections

Transcription

1 A Real-Tme Volume Renderng Archtecture Usng an Adaptve Resamplng cheme for Parallel and Perspectve Proectons Masato Ogata TaaHde Oham y Hugh C. Lauer y Hanspeter Pfster y Mtsubsh Electrc Informaton Technology Center Amerca z Abstract Ths paper descrbes an obect-order real-tme volume renderng archtecture usng an adaptve resamplng scheme to perform resamplng operatons n a unfed parallel-ppelne manner for both parallel and perspectve proectons. Unle parallel proectons, perspectve proectons requre a varable resamplng structure due to dvergng perspectve rays. In order to address ths ssue, we propose an adaptve ppelned convoluton bloc for resamplng operatons usng the level of resoluton to eep the parallel-ppelne structure regular. We also propose to use mult-resoluton datasets prepared for dfferent levels of grd resoluton to bound the convoluton operatons. The proposed convoluton bloc s organzed usng a systolc array structure, whch wors well wth a dstrbuted sewed memory for conflct-free accesses of voxels. We present the results of some experments wth our software smulators of the proposed archtecture and dscuss about mportant techncal ssues. CR Categores and ubect Descrptors: I.3.1 [Computer Graphcs]: Hardware Archtecture - Graphcs Processors; I.3.3 [Computer Graphcs]: Pcture/Image Generaton - Dsplay Algorthms. Addtonal Keywords: Volume Graphcs, Volume Renderng, Raycastng, Raytracng, Parallel Proecton, Perspectve Proecton, centfc Vsualzaton, Real-Tme, ystolc Array. 1 INTRODUCTION Volume vsualzaton technques are becomng avalable and popular. Ths s due to the ncreasng avalablty of scentfc data generated by a varety of computer smulatons, medcal data obtaned by MRI and CT scanners, and geologcal, oceanographc, and meteorologcal data collected from varous sensors, and also due to the decreasng costs of hgh-performance computng requred for volume renderng. One of the notable characterstcs shared by these volume data s the sheer amount of data elements to be processed n renderng. Ths requres a huge amount of computng resources for anmated vsualzaton, whch s essental to observe some physcal phenomena. Another characterstc of the data s that they can not be represented by surfaces as n the conventonal polygon-based graphcs; ogata@merl.com, vstng from Mtsubsh Electrc Corporaton y foham,lauer,pfsterg@merl.com z 21 Broadway, Cambrdge, MA 2139, U..A. the volume data may nclude complcated nternal structures and shapeless features. Because of these characterstcs, fast drect volume renderng methods are n hgh demands. The most popular volume renderng algorthm s the raycastng algorthm, whch casts rays from the center of proecton nto a volume, computes samples along the rays, and accumulates the sampled values to determne the pxel values on the screen. In many cases, each sample s computed from eght voxels surroundng the sample pont by lnear nterpolatons. Each resamplng operaton s relatvely smple, but the total number of resamplng operatons s very large, and the tme spent for the operatons s domnant n the renderng tme. Because of ths, a raycastng-based volume renderng system could be consdered a resamplng machne. The renderng operatons for parallel proectons are very regular and amenable to parallel-ppelne processng. The operatons for perspectve proectons, however, are varable due to dvergng perspectve rays. Ths processng varablty adversely affects the parallel-ppelne structure for parallel proectons and has been a maor obstacle for hardware mplementaton of a perspectve proecton system. In ths paper, we propose a real-tme volume renderng archtecture usng an adaptve resamplng scheme for both parallel and perspectve proectons. The proposed archtecture s structured as a sample-parallel machne to perform resamplng operatons n a unfed parallel-ppelne manner for both types of proectons. The processng varablty ssue s addressed by an adaptve ppelned convoluton bloc for resamplng voxels usng the level of grd resoluton. The convoluton area can be arbtrarly large because of dvergng perspectve rays. Mult-resoluton datasets are prepared for dfferent resoluton levels to address ths ssue. Ths paper s organzed as follows; secton 2 descrbes related wor, secton 3 presents some ssues for real-tme perspectve proectons and the ey deas for the proposed archtecture, secton 4 shows a proposed hardware structure, secton 5 presents some experments wth the archtecture smulator, secton 6 descrbes future wor, and secton 7 concludes the paper. 2 RELATED WORK 2.1 Renderng Methods Renderng methods are categorzed nto two groups: mage-order and obect-order. Each method has advantages and dsadvantages for structurng a real-tme volume renderng archtecture based on some form of parallel processng. The mage-order method casts rays from a screen nto a volume. The number of rays to cast s determned by the screen sze and resoluton. The ray-parallel scheme parallelzes renderng operatons on a ray bass [4]. The maor dsadvantage of ths scheme s that one voxel s accessed by multple rays for resamplng, ncreasng the total number of memory accesses. There are some optmzaton technques avalable for ths scheme. Early ray termnaton and coherence encodng are two examples to reduce the number of memory accesses [6]. However, they requre a varable resamplng

2 structure, mang the ppelne structure complcated. The obect-order method, on the other hand, maps a volume onto a screen. The splattng scheme splats each voxel onto the screen [9]. One of ts maor dsadvantages s that flter ernels of dfferent szes are requred for perspectve proectons, The voxel-parallel scheme reads a voxel once and retans t untl all the samples requrng the voxel are computed [8]. The maor advantages of ths scheme are the reducton of the memory accesses and the fxed resamplng structure. The dsadvantage s that there are no maor optmzaton technques avalable for ths scheme. The optmzaton technques for the ray-parallel scheme can not be used for ths obect-order scheme, because they brea the fxed resamplng structure. 2.2 hear-warp Algorthm The shear-warp algorthm provdes a unfed framewor for formulatng the renderng operatons for both parallel and perspectve proectons [5]. Gven a vewng vector, the algorthm fnds the prncpal vewng axs, the axs most parallel to the vewng vector, to permute a volume so that the prncpal vewng axs be the z axs. Raycastng s performed on the permuted volume. The resultng mage s an ntermedate mage formed on the base plane, the plane perpendcular to the vewng vector, whch ncludes the front face of the volume. It s warped to produce the fnal screen mage. Raycastng s effectvely a shear operaton that shears the voxel grd of the volume to parallelze the rays at the base plane. The parallelzed rays are all perpendcular to the base plane. In parallel proectons, the rays proceed n the sheared voxel grd. In perspectve proectons, the rays proceed n the sheared and progressvely scaled voxel grd. The grd scalng depends on the dstance between the center of proecton and the resamplng poston n the z drecton; the grd becomes fner as the z poston ncreases. Each pxel n the base plane mage s computed by compostng the samples taen along the perspectve ray startng at the pxel poston on the base plane. A sample s estmated from the voxels n ts neghborhood. Because of the progressvely scaled grd, the samples along a perspectve ray have to be computed from progressvely larger groups of voxels. The shear-warp algorthm can be mplemented by both software and hardware. For hardware mplementaton, however, there are several crtcal ssues to be addressed, such as how to compute samples along perspectve rays n the progressvely scaled voxel grd wth a fxed amount of hardware resources and how to perform resamplng operatons n a unfed ppelne manner for both parallel and perspectve rays. 2.3 pecalzed Hardware The EM-Cube archtecture [7] s a real-tme volume renderng archtecture under development. It s derved from the Cube-4 archtecture [8] wth sgnfcant modfcatons and extensons for VLI mplementaton. The archtecture s based on the shear-warp algorthm, but supports parallel proectons only. It s a voxel-parallel archtecture wth the sewed voxel memory for conflct-free voxel accesses. Each renderng ppelne computes a sample at each slce of voxels; a slce s defned as a group of voxels wth the same z coordnates n the volume space permuted by the vewng vector. A renderng ppelne computes samples for dfferent rays at dfferent slces. The compostor accumulates samples for each ray and stores the resultng pxels n the pxel memory to generate the base plane mage, whch s warped to produce the fnal screen mage. The renderng operatons are performed n a parallel-ppelne structure wth multple renderng ppelnes. The archtecture computes samples slce by slce. It s desgned to explot the geometr- Voxel Memory MEM MEM MEM MEM Resamplng Module MEM: Memory Modules Renderng Ppelnes Pxel Memory Fgure 1: Proposed sample-parallel archtecture. cal regularty n renderng operatons for parallel proectons; all the samples at a slce have the same offsets from the reference ponts, snce all the rays are parallel and proceed from the ponts wth the same offsets from the reference ponts toward the same drecton wth the same steps. The archtecture s well organzed for parallel proectons. It s, however, not obvous to facltate perspectve proectons n ths archtecture, because the varable resamplng structure for the perspectve rays does not ft well n ts parallel-ppelne structure. The ray-slce-sweep algorthm s proposed for parallel and perspectve proectons n the Cube-4L archtecture [1]. It s stll under nvestgaton for mprovements. VIRIM s a real-tme parallel renderng system for perspectve proectons [2]. It uses a Gaussan flter mas to compute samples from neghborng voxels. The resamplng operatons are performed n the rotator board and the resultng samples are sent to the mult-dp board for raytracng. It s a parallel system, but not organzed n a complete parallel-ppelne structure for VLI mplementaton. 3 PROPOED ARCHITECTURE 3.1 ample-parallel Archtecture The EM-Cube archtecture s a voxel-parallel archtecture that provdes contnuous streams of voxels from the voxel memory and feeds them to multple renderng ppelnes, each performng resamplng operatons. Its parallel-ppelne structure s tuned for parallel proectons. The archtecture s basc assumpton s that a sample can always be computed from two neghborng voxels n one dmenson, a notable characterstc of the parallel rays. Ths assumpton does not hold for perspectve proectons, because the number of voxels requred for resamplng ncreases as the dstance of the resamplng pont from the center of proecton ncreases. It s not easy to nclude n the archtecture the varable resamplng structure requred for perspectve proectons. In order to address ths ssue, we shft a focus from voxels to samples and reorganze the renderng archtecture as a sampleparallel archtecture to provde a unfed parallel-ppelne structure for both parallel and perspectve proectons. As shown n Fg. 1, the proposed archtecture places a resamplng module between the voxel memory and renderng ppelnes. Ths module s specalzed for resamplng wth a varable number of voxels n the resamplng area. All the resamplng functons are moved from the renderng ppelnes to ths resamplng module to organze the renderng ppelnes n a fxed parallel-ppelne structure for the other renderng operatons.

3 Base Plane Eye Data et Parallelzed Ray H n Ray Prncpal Vewng Axs Warped Image on screen Eye Transform wth Matrx H lce z lce +m Multple Resoluton Data Coverage Area Convoluton Area W : hear-hrn Grd Data : hear-hrn Mult-Resoluton Grd Data : Resamplng Grd Data Fgure 2: hearng and scalng n perspectve proectons. (a) Resamplng wth Convoluton Wthout MRD L to 1 Wth MRD Convolver L to L (b) Ppelned Convolver Fgure 4: Convoluton wth mult-resoluton datasets. Fgure 3: heared and scaled grd of a 16 3 dataset. 3.2 Perspectve Proectons The shear-warp algorthm produces a base plane mage as an ntermedate mage, whch s warped to produce the fnal screen mage. The warpng operatons are dfferent from those for parallel proectons due to dvergng perspectve rays. Fg. 2 shows the perspectve rays parallelzed at the base plane by a shearng matrx H. Theyare parallel to the prncpal vewng axs and perpendcular to the base plane. The base plane mage s produced by the parallelzed perspectve rays. The shearng transformaton shears and progressvely scales the voxel grd as shown n Fg. 3, where the voxel grd becomes fner as the dstance from the base plane ncreases. tartng from a poston n the frst slce, a parallelzed perspectve ray proceeds n the progressvely scaled grd to compute a sample at each slce. The computed samples are accumulated by a compostor to produce the fnal pxel value n the base plane mage. One of the smple resamplng operatons s to average the values of the voxels n the neghborhood of a sample pont. A more general operaton s convoluton over the voxels n the resamplng area. There are two mportant ssues for the ppelne mplementaton of the convolver: convoluton area and structure. The convoluton area s determned by the resoluton of the scaled voxel grd at the resamplng pont. The convoluton area sze W n one dmenson s computed by: W =1+=Z ; (1) where s the slce number or the dstance from the base plane and Z the dstance between the eye poston (the center of proecton) and the base plane. The sze of convoluton area s equvalent to the dstance between the two neghborng perspectve rays at slce. It can be arbtrarly large wth large values of and/or small values of Z. Ths s llustrated n Fg. 4(a). nce the hardware mplementaton can only use a fxed amount of resources for convoluton, t has to gnore the voxels outsde the convoluton area, producng a low-qualty or alased base plane mage. The convoluton structure s another ssue. The underlnng archtecture pars a memory module and a renderng ppelne so that each ppelne can tae one voxel along wth a neghborng voxel through a sdeway communcaton and produce one sample computed from the two voxels n each dmenson. In ths structure, the number of nputs (voxels) s equal to the number of outputs (computed samples). Ths holds for parallel proectons, but not for perspectve proectons. Because of the progressvely scaled grd, the number of outputs s equal to or less than the number of nputs, requrng a varable convoluton structure, as llustrated n Fg. 4(b). 3.3 Mult-Resoluton Datasets The control of the varable convoluton area s a crtcal ssue for the hardware mplementaton of the convolver. In order to address the ssue, we propose to use mult-resoluton datasets prepared for dfferent levels of grd resoluton. It s a 3D verson of the mpmappng scheme for texture mappng [1]. As shown n Fg. 4(a), a data n a mult-resoluton dataset represents the area covered by a certan number of orgnal voxels. The covered area s larger n a coarse dataset than n a fner dataset. The use of mult-resoluton datasets can reduce the number of data requred for convoluton. By selectng an approprate mult-resoluton dataset dependng on the resoluton of the scaled voxel grd, the archtecture can always use a bounded number of data for resamplng regardless of the number of orgnal voxels coverng the same convoluton area. It also maes the ppelne convolver smple because the number of outputs s equal to the number of nputs as shown n Fg. 4(b). Fg. 5 shows the effect of usng the mult-resoluton datasets for the volume of thesameszenfg.3. It s not necessary, but very practcal, to use powers of 2 for mult-resoluton data, as llustrated n Fg. 6, where L s the fnest, and L3 the coarsest. The memory overhead to store mult-resoluton datasets for a volume of sze n 3 s less than n 3 =7, whch s not consdered a very large overhead.

4 Normalzed C.. hear & hrn Matrx H hear-hrn C.. * cale Up wth K (Usng MRD) cale-up C..!+ Floor Compostng C.. Fgure 7: Four coordnate systems. Fgure 5: heared and scaled grd wth mult-resoluton datasets. Lc Ls =1 Ld V!+ V!+ V!+ -1,, +1, L s = K*L d Resoluton Fne Coarse Level L L1 L2 L3 : Voxel : Geometrcal Coverage ( 8x8x8 voxels ) Fgure 6: Power-of-2 mult-resoluton datasets. : Grd n cale-up Coordnates : Resampled Grds n Compostng Coordnates : Grd n Compostng Coordnates V!+ : Voxel n Grd (,) : ample Fgure 8: Grds n scale-up and compostng coordnates. 3.4 ewed Memory The sewed memory organzaton s a technque to store voxels n separate memory modules so that voxels n a slce can be accessed n parallel wthout any memory conflct regardless of the vewng drecton [3]. It does not requre multple volume copes. The proposed archtecture uses t to store mult-resoluton datasets. Consder a system wth N p renderng ppelnes for a volume of sze n 3. nce each ppelne s connected one-to-one to a memory module, the number of memory modules s equal to the number of ppelnes. Let (x; y; z) be a poston ndex set of a data. Then the memory module number n p and the memory module address p for the data are computed as follows: m = (x + y + z) modn; (2) n p = m mod N p; (3) p = bm=n pc + yn=n p + zn 2 =N p; (4) where n s a multple of N p. It guarantees that voxels n any slce denoted by (; y; z), (x; ; z),or(x; y; ) can be accessed n parallel wth no memory conflct. For the mult-resoluton sewed memory, a memory address s specfed by (L; x; y; z), wherel s the resoluton level. Each mult-resoluton dataset can be stored n the sewed memory as f t were an orgnal volume dataset. Mult-resoluton data are accessed by the followng addressng scheme: where m = (x + y + z )modn ; (5) n p = m mod N p; (6) p = bm=n pc + y n =N p + z n 2 =N p; (7) K =2 L ; x = b x K c; y = b y K c; z = b z K c; n = b n c: (8) K 3.5 Convoluton Area The resoluton level s an ndcator of the convoluton area sze at each slce n the progressvely scaled voxel grd to choose an approprate mult-resoluton dataset. The power-of-2-based resoluton level s defned by L = b log 2 W c; (9) where W s the convoluton area sze n equaton (1). We use four 2D coordnate systems and transformatons between them to determne the convoluton area for a gven resoluton level. The four coordnate systems are the normalzed, shearshrn, scale-up, and compostng coordnate systems, as llustrated n Fg. 7. The normalzed coordnate system defnes the orgnal voxel grd at slce. It s equvalent to the base plane coordnate system. The shear-shrn coordnate system defnes the voxel grd sheared and scaled by a shear-shrn matrx. The scale-up coordnate system defnes a grd scaled up by a scalng factor K. Ths s the effect of usng mult-resoluton datasets; a dataset coverng a larger area effectvely scales up the grd. The compostng coordnate system defnes a pxel grd whch concdes wth the orgnal voxel grd n the normalzed coordnate system. The sequence of transformatons from the normalzed coordnate system to the compostng coordnate system entals the sequence of grd changes. The procedure to determne the convoluton area s descrbed as a sequence of transformatons between the coordnate systems as follows: 1) transform a grd pont at (; ; ) n the normalzed coordnate system nto a grd pont at ( ; ; ) by the shear-shrn matrx; 2) scale up the grd pont at ( ; ; ) usng the scalng factor K to a grd pont at ( y ; y ; y ); and 3) apply the floor operaton to the grd poston ( y ; y ; y ) to get the fnal poston (^; ^; ^). The scalng factor K s gven n equaton (8). The mult-resoluton data v y ;; for convoluton are addressed by (L; b=kc; b=kc; b=kc), as shown n equatons (5)-(8). Note that ther geometrcal postons are gven by ( y ; y ; y ).

5 Weghtng Functon w1 w2 w3 V -1, d1 d3 d2 V -drecton V +1, d4 d5 d6 W = w ( d ) -drecton w4 w5 w6 Weghtng Functon Mem Mem Mem Mem W 1 1D 1D 1D 1D 1D 1D 1D 1D W 6 -drecton 1D Convolver -drecton 1D Convolver : Grd Data n cale-up Coordnates V : Voxel : ample : Grd Data n Compostng Coordnates d : Dstance W : Weght Fgure 9: 2D convoluton. d 1 d2 d 3 a b c d a b c d a b c d a b c d W = w ( d ) d4 d5 d6 g 1D a W f b Unt Delay Arthmetc unt g:= b f:= a+ bw The grds n the scale-up and compostng coordnate systems are shown n Fg. 8, where L s and L c are the grd spacngs n the scaleup and compostng coordnate systems, respectvely. Note that L c s equal to the orgnal voxel spacng. Because of the power-of-2 mult-resoluton datasets, the followng condton holds: : : Grd Data ( a - d ) n cale-up Mem: Memory Modules Coordnates 3 Grd Data n Compostng : Resamplng Grd Coordnates Fgure 1: Ppelned 2D convolver. 3.6 ample Estmaton L s L c < 2L s: (1) ample values are estmated by convoluton over voxels or multresoluton data n the convoluton area. We assume that the convoluton ernel or the set of weghts s symmetrc for,, and drectons and separable; that s, w = w w w. Consder M M data for sample estmaton. We apply a 1D convoluton to and drectons ndependently, as llustrated n Fg. 9. For smplcty, we assume that M = 3. In 1D, a sample value s can be computed by s = w 1v y,1 + w2vy + w3vy +1 ; (11) where v y,,1 vy,andvy +1 are data n the convoluton area and wm are weght functons defned by w m = w m(^, y m); for m =; 1; 2 (12) where ^ s the sample poston and y m the data poston. The ndependent applcaton of the 1D convoluton to and drectons smplfes the 2D convoluton structure. Frst, three 1D convolutons n the drecton are computed: " s;,1 # s ; = s ; vy,1;,1 v y ;,1 v y +1;,1 v y,1; v y ; v y +1; v y,1;+1 v y ;+1 v y +1; " w 1 # w 2 : w 3 (13) Then a 1D convoluton for the drecton s performed to compute the fnal sample value ^s ;: ^s ; = w 4s ;,1 + w 5s ; + w 6s ;+1: (14) The set of weghts depends on the flter functon chosen for convoluton. Dependng on the vewng drecton, the dstance between sample postons vares from 1 to 3p 3 n both parallel and perspectve proectons. To mae correctons for ths varable dstance, an opacty table can be used to provde corrected opacty values by usng the dstance as an address, as descrbed n [5]. 4 HARDWARE TRUCTURE 4.1 Ppelned 2D Convolver Fg. 1 llustrates an mplementaton of a ppelned 2D convoluton for a 3 3 convoluton area. The operatons are dvded nto two groups: one group for the drecton and the other for the drecton. In ths example, the dataset sze n one dmenson and the number of ppelnes are both 4. Table 1 shows several snap shots of the ppelne operatons of the 1D convoluton for the -drecton. In ths example, one slce of 4 4 voxels s processed wth the sewed memory organzaton. The 1D convoluton structure for the drecton s the same, but has dfferent tme delays due to the dfferent tmngs of neghborng voxels n the drecton. 4.2 Convoluton Kernels The prevous ppelned 2D convolver wors fne for parallel proectons. However, t does not wor for perspectve proectons because of the msalgnment of the poston of the voxel group for convoluton wth the poston of the ernel center, as shown n Fg. 11(a). Because of ths msalgnment, the convolver does not produce the correct samples for perspectve proectons. Fg. 11(b)

6 Table 1: nap shots of ppelne data flow. Out Ppe Ppe 1 Ppe 2 Ppe 3 Mem a b c d Mem d1 a1 b1 c1 (a; ; ) (b; ; ) (c; ; ) (d; ; ) Mem c2 d2 a2 b2 (d1; ; ) (a1; ; ) (b1; ; ) (c1; ; ) (a; b; ) (b; c; ) (c; d; ) (d; a; ) Mem b3 c3 d3 a3 (c2; ; ) (d2; ; ) (a2; ; ) (b2; ; ) (d1; a1; ) (a1; b1; ) (b1; c1; ) (c1; d1; ) (a; b; c) (b; c; d) (c; d; a) (d; a; b) Mem a4 b4 c4 d4 (b3; ; ) (c3; ; ) (d3; ; ) (a3; ; ) (c2; d2; ) (d2; a2; ) (a2; b2; ) (b2; c2; ) (d1; a1; b1) (a1; b1; c1) (b1; c1; d1) (c1; d1; a1) (p; q; r) =p + q + r Voxels cale-up Coordnates Compostng Coordnates Voxels 2 4 cale-up Coordnates Compostng Coordnates lce : Voxel : Kernel Center Poston : Convoluton (a) Flterng area of Nave Ppelne Convolver lce Kernel Center Postons (b) Flterng area of Ideal Ppelne Convolver shows a ppelned convolver that produces correct samples. Table 2 also shows the two convolvers. In ths example, 1D convolutons over 3 voxels are performed for 4 ppelnes. In the worst case, the grd spacng L s n the scale-up coordnate system s half the grd spacng L c n the compostng coordnate system, when we use power-of-2 mult-resoluton datasets. In ths case, the convolver tang N voxels produces only N=2 samples. Gven the ernel center poston, whch corresponds to the base plane memory address, and the slce number, t s possble to compute the postons of voxels n the convoluton area n advance or on the fly. The order of voxel reads cannot be controlled, but the poston of the ernel center can be controlled by choosng one of the voxel postons n the convoluton area as the poston of the ernel center. For example, the leftmost voxel poston can be the ernel center poston. nce each voxel poston corresponds to a renderng ppelne, the renderng ppelne correspondng to the ernel center poston receves a vald sample; the other ppelnes receve nvald samples. Ths s llustrated n Table 2. To control ths stuaton, a vald/nvald flag s attached to each sample, ndcatng the valdty of the sample to the recevng ppelne. Invald samples are dscarded at the composton stage. Convoluton centers are not necessarly algned wth the sample postons. It causes an error f a fxed set of weghts s used. Ths msalgnment error can be corrected by computng convoluton weghts usng a loo-up table addressed by the offsets of sample postons. 4.3 Adaptve Ppelned Convolvers Fg. 12 shows a bloc dagram of an adaptve 2D convolver. Let n be the sze of a dataset n one dmenson, whch s generally greater than the number of ppelnes N p. The selector s controlled by the status of ppelne, that s, whether or not the ppelne s processng the leftmost voxel n the current group of voxels. The delay of n=n p s used to delay the operaton for the tme for one scanlne of voxels. It s actually confgured as a varable delay element for dfferent resoluton levels. For a gven scalng factor K representng a resoluton level, ths delay element causes a delay of (n=k)=n p. Fg. 13 llustrates an adaptve 3D convolver. The structure s a drect extenson of the adaptve 2D convolver wth the -drecton 1D convolver added at the output of the 2D convolver wth a delay of n 2 =N p. Ths delay element s also a varable delay element that causes a delay of (n=k) 2 =N p for the scalng factor K. The varable delay element can be mplemented usng a FIFO memory addressed n a crcular manner by a sngle ponter for both read and wrte operatons. The cycle tme from one locaton to Fgure 11: Voxel postons vs. ernel centers. Table 2: Kernel centers and voxel groups. Convolver 1 (nave) Convolver 2 (correct) Kernel voxels ernel voxels center center 1 v1; v2; v3 1 v1; v2; v3 2 v2; v3; v4 2 v2; v3; v4 3 v3; v4; v5-4 v4; v5; v6 3 v4; v5; v6 5 v5; v6; v7-6 v6; v7; v8 4 v6; v7; v8 7 v7; v8; v9-8 v8; v9; v1 5 v8; v9; v1 9 v9; v1; v11-1 v1; v11; v12 6 v1; v11; v12 11 v11; v12; v13-12 v12; v13; v14 7 v12; v13; v14 13 v13; v14; v15-14 v14; v15; v16 8 v14; v15; v16 15 v15; v16-16 v16 9 v15; v16 the same locaton determnes the delay tme, whch can be easly changed by changng the maxmum address value. 4.4 Renderng Tmngs The proposed renderng archtecture s organzed wth the resamplng module consstng of the ppelned convolvers, the multresoluton sewed memory, the renderng ppelnes, and the pxel memory. nce the real-tme processng capablty of the proposed archtecture depends on ts renderng performance, we estmate the tmngs for renderng volumes of practcal szes. The renderng tme s drectly related to the number of resamplng operatons to perform, whch can be reduced by the use of mult-resoluton data. It s upper-bounded by the total number of resamplng operatons wthout mult-resoluton datasets, that s, only wth orgnal voxels. nce the resamplng and other renderng operatons can be fully ppelned, the ppelne cycle tme can be equal to the memory access tme T m. For a gven set of renderng parameters s chosen, accesses to the voxel memory are regular and determnstc. A double

7 Mem Mem Mem Mem g n/np n/np n/np n/np n/np n/np n/np n/np Fgure 12: Adaptve ppelned 2D convolver. g := b f := a+ bw a W f b Latch Unt Delay elector n: Data et Dmenson N p : Number of Ppelnes bufferng technque can be used to provde contnuous data streams from the voxel memory to the resamplng module and renderng ppelnes. The pxel memory does not requre much bandwdth on average, because the pxel wrte operatons are bursty but ntermttent. A smple pxel bufferng technque wth a FIFO memory wll be enough for pxel wrte operatons. The voxel and pxel memores can be mplemented by DRAM (ynchronous DRAM) chps to explot ther burst access mode. Let n 3, N p,andn f be the volume sze, the number of renderng ppelnes, and the number of mage frames generated per second. The total number of samples N s to compute n each ppelne for one second s gven by Mem Mem Mem Mem W 4 n /Np n /Np n /Np n /Np n /Np W7 W7 W7 W7 W8 W8 W8 W8 W9 W9 W9 W9 n /Np n /Np n /Np Fgure 13: Adaptve ppelned 3D convolver. -drecton 1D Convolver 2D Convolver For each second, N s = n 3 N f =N p: (15) N st m 1: (16) For a gven set of parameters T m, N f,andn p, the maxmum dmenson of volume that can be rendered s gven by n 3p N p=(n f T m): (17) Assumng that T m =8ns as n a 125-MHz DRAM chp and N f =3frames/second, the volume dmensons computed for several values of N f and N p are shown n Table 3. These values verfy that the proposed archtecture can render volumes of practcal szes n real-tme. Voxel loadng, renderng, and pxel readng can be ppelned usng a double-bufferng technque for the voxel and pxel memores. Ths allows the proposed archtecture to render tme-varyng volume datasets n real-tme. Table 3: Maxmum volume dmensons. N f N p (# ppelnes) (frames/sec) Mult-resoluton datasets are generated before a volume dataset s loaded nto the voxel memory. They can be generated off-lne by software. The set of operatons to compute a sngle data from eght data at a lower level ncludes seven addtons, one dvson (shft), and eght memory reads, and one memory wrte; n 2 (n, 1) 2 =4 sets of operatons are requred for a volume of n 3. Although the number of operatons can be reduced by usng some optmzaton technques, such as bufferng and ppelnng, t seems stll dffcult to perform these operatons on the fly wth the current technologes. It may be reasonable, however, to perform these operatons usng several frame perods for practcal applcatons. 4.5 calablty Addng renderng ppelnes ncreases the volume sze for a fxed frame rate or the frame rate for a volume of fxed sze. There are no archtectural problems n addng renderng ppelnes, because 1) n the voxel memory nterface, each memory module n the voxel memory s connected one-to-one to a ppelne n the resamplng module; 2) n the resamplng module, the resamplng ppelnes communcate only wth the left and rght ppelnes; 3) n the nterface between the resamplng module and renderng ppelnes, each renderng ppelne s connected to one resamplng ppelne; 4) n nter-ppelne communcatons, each ppelne communcates only wth the left and rght ppelnes; and 5) n the pxel memory nterface, the number of pxels to wrte s ndependent of the number of ppelnes. Therefore, the proposed archtecture s scalable.

8 5 EPERIMENT We bult a software smulator to smulate the ppelne data flow of the proposed archtecture and verfy the concept for both parallel and perspectve proectons. To compare mages, we also bult a screen-to-obect raycastng renderer to smulate a texture mappng method that computes slces of samples perpendcular to the vewng vector and accumulates them to produce the fnal mage. We conducted several renderng experments wth these smulators. Fg. 14 shows a perspectve mage rendered from an opaque cube of sze 64 3 to verfy the perspectve proecton. The flter ernel based on the Lagrange formula s used n resamplng. Fgs. 15 and 16 show two perspectve mages rendered from an opaque checer-board cube of sze (spatal frequency of 64 Hz) to explore the alasng problem; a fully opaque dataset gves the worst case for alasng. The mage n Fg. 15 s generated by usng the nearest neghbor voxel values n resamplng, showng the alasng problem clearly. The mage n Fg. 16 s generated by usng a box flter ernel n resamplng, showng the antalasng effect by convoluton usng mult-resoluton datasets. Fgs. 17, 18, and 19 show the mages rendered from the engne bloc of sze used n Lacroute s renderng experments [5] wth a manually adusted opacty table. Fg. 17 shows a perspectveproecton mage, and Fg. 18 shows a parallel-proecton mage for a comparson purpose. These two mages are generated by usng a ernel based on the Lagrange formula. Fg. 19 s a perspectve-proecton mage generated by the screen-to-obect raycastng renderer wth nterpolatons usng mult-resoluton datasets. The number of the slces taen for ths mage s 258, about the same number of slces (256) used n Fg. 17. The two mages n Fgs. 17 and 19 loo comparable n qualty. 6 FUTURE WORK The proposed archtecture provdes a freedom for adustng the convoluton ernel. Choosng a convoluton ernel for the best mage s a challengng problem. Especally, t s an mportant topc to fnd the ernel to maxmze the antalasng effect. The current archtecture smulator computes samples only at ntegral slce postons. ubslcng, that s, tang samples between ntegral slce postons, s expected to mprove the mage qualty. How much can the subslcng mprove the mage qualty? What s the lmtaton of subslcng? mlarly, the mage qualty can be mproved by addtonal samplng n the x and y drectons. nce the proposed archtecture casts a fxed number of rays, mages generated by rays startng at dfferent offsets need to be blended. Ths s an nterestng technque worth explorng for the mprovement of mage qualty. Error analyss s an essental wor for hardware mplementaton, whch s most lely to use fxed-pont arthmetc. The applcaton of resamplng by convoluton to renderng a class of rregular volumes s another topc for study. 7 CONCLUION We have proposed a real-tme volume renderng archtecture usng an adaptve resamplng module for both parallel and perspectve proectons. It s organzed as a sample-parallel archtecture wth a unfed parallel-ppelne structure. The archtecture can be easly organzed n a systolc array structure for mplementaton on an AIC chp. The resamplng module s the ey feature of the proposed archtecture to address the processng varablty problem caused by the dvergng perspectve rays. It s placed between the voxel memory and the renderng ppelnes for resamplng by convolutons over groups of data to mae the parallel-ppelne structure regular. The use of mult-resoluton datasets s another ey feature to reduce the number of resamplng operatons. We have demonstrated that t also contrbutes to antalasng. We have shown that ppelned 2D and 3D convolvers can be mplemented wth 2M and 3M elements, respectvely, by explotng the propertes of the sheared and scaled grds. They are a contrast to M 2 and M 3 elements requred to mplement general 2D and 3D convolvers. We have also descrbed the adaptve convolvers wth varable delays. ACKNOWLEDGEMENT We would le to than M. Maeda, K. Muro, and H. Taayama of the Vson-21 proect n Mtsubsh Electrc Corporaton for gvng us an opportunty of ths research and all the members of the volume graphcs proect at Mtsubsh Electrc Informaton Technology Center Amerca for ther support. Especally, many thans to Beverly chultz for her valuable comments on the paper. References [1] I. Btter and A. Kaufman. A ray-slce-sweep volume renderng engne. In Proceedngs of the 1997 IG- GRAPH/EUROGRAPHIC Worshop on Graphcs Hardware, pages , August [2] T. Günther, C. Polwoda, C. Renhart, J. Hesser, R. Männer, H.-P. Menzer, and H.-J. Baur. Vrm: A massvely parallel processor for real-tme volume vsualzaton n medcne. Computers & Graphcs, 19(5):75 71, [3] A. Kaufman and R. Baalash. Memory and processng archtecture for 3d voxel-based magery. IEEE Computer Graphcs & Applcatons, 8(6):1 23, November [4] G. Knttel and W. trasser. A compact volume renderng accelerator. In Proceedngs of the IEEE ymposum on Volume Vsualzaton, pages 67 74, October [5] P. G. Lacroute. Fast volume renderng usng a shear-warp facrorzaton of the vewng transformaton. Techncal Report CL-TR (Ph.D. Dssertaton), Computer ystems Laboratory, tanford Unversty, eptember [6] M. Levoy. Effcent ray tracng of volume data. ACM Transactons on Graphcs, 9(3): , July 199. [7] R. Osborne, H. Pfster, H. Lauer, N. McKenze,. Gbson, W. Hatt, and T. Oham. Em-cube: An archtecture for lowcost real-tme volume renderng. In Proceedngs of the 1997 IGGRAPH/EUROGRAPHIC Worshop on Graphcs Hardware, pages , August [8] H. Pfster. Archtecture for real-tme volume renderng. Ph.d. dssertaton, Department of Computer cence, tate Unversty of New Yor at tony Broo, [9] L. Westover. Footprnt evaluaton for volume renderng. In Proceedngs of the ACM IGGRAPH 94, pages , October [1] L. Wllams. Pyramdal parametrcs. In Proceedngs of the ACM IGGRAPH 83 Conference, pages 1 11, July 1983.

9 Fgure 14: Perspectve proecton wth convoluton. Fgure 17: Perspectve proecton wth convoluton. Fgure 15: Perspectve proecton by resamplng the nearest neghbor voxels. Fgure 18: Parallel proecton wth convoluton. Fgure 16: Perspectve proecton wth convoluton. Fgure 19: creen-to-obect perspectve proecton by nterpolatons usng mult-resoluton datasets.