Volumetric display system based on threedimensional scanning of inclined optical image

Transcription

1 Volumetric display system based on threedimensional scanning of inclined optical image Daisuke Miyazaki, Kensuke Shiba, Koji Sotsuka, and Kenji Matsushita Department of Physical Electronics and Informatics, Graduate School of Engineering, Osaka City University, , Sugimoto, Sumiyoshi-ku, Osaka , Japan ABSTRACT: A volumetric display system based on three-dimensional (3D) scanning of an inclined image is reported. An optical image of a twodimensional (2D) display, which is a vector-scan display monitor placed obliquely in an optical imaging system, is moved laterally by a galvanometric mirror scanner. Inclined cross-sectional images of a 3D object are displayed on the 2D display in accordance with the position of the image plane to form a 3D image. Three-dimensional images formed by this display system satisfy all the criteria for stereoscopic vision because they are real images formed in a 3D space. Experimental results of volumetric imaging from computed-tomography images and 3D animated images are presented Optical Society of America OCIS codes: ( ) Displays; ( ) Three-dimensional image processing References and links 1. T. Miyashita and T. Uchida, "Cause of fatigue and its improvement in stereoscopic displays," Proc. SID 31, (1990). 2. Y. Nojiri, H. Yamanoue, A. Hanazato, M. Emoto and F. Okano, Visual comfort/discomfort and visual fatigue caused by stereoscopic HDTV viewing, in Stereoscopic Displays and Virtual Reality Systems XI; A. J. Woods, J. O. Merritt, S. A. Benton, and M. T. Bolas; eds., Proc. SPIE 5291, (2004). 3. M. Emoto, T. Niida, and F. Okano, Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television, J. Disp. Tech. 1, (2005). 4. B. G. Blundell and A. J. Schwarz, Volumetric Three-Dimensional Display Systems, (John Wiley & Sons, New York, 2000). 5. I. N. Kompanetsa and S. A. Gonchukov, Volumetric displays, in Current Research on Image Processing for 3D Information Displays, V. V. Petrov, ed., Proc. SPIE 5821, (2005). 6. A. C. Traub, "Stereoscopic display using varifocal mirror oscillations," Appl. Opt. 6, (1967). 7. S. Suyama, M. Date, and H. Takada, "Three-dimensional display system with dual-frequency liquid-crystal varifocal lens," Jpn. J. Appl. Phys. 39, (2000). 8. P. Soltan, M. Lasher, W. Dahlke, N. Acantilado, and M. McDonald, "Laser-projected 3-D volumetric displays," in Cockpit Displays IV: Flat Panel Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE 3057, (1997). 9. C.-C. Tsao and J. Chen, "Volumetric display by moving screen projection with a fast-switching spatial light modulator," in Projection Displays IV, M. H. Wu, ed., Proc. SPIE 3296, (1998). 10. G. E. Favalora, J. Napoli, D. M. Hall, R. K. Dorval, M. G. Giovinco, M. J. Richmond, and W. S. Chun, 100 Million-voxel volumetric display, in Cockpit Displays IX: Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE 4712, (2002). 11. D. Miyazaki and K. Matsushita, "Volume-scanning three-dimensional display that uses an inclined image plane," Appl. Opt. 40, (2001). 12. D. Miyazaki, M. Lasher, and Y. Fainman, "Fluorescent volumetric display excited by a single infrared beam," Appl. Opt. 44, (2005). 13. C. W. Tyler and R. D. Hamer, Analysis of visual modulation sensitivity. IV. Validity of the Ferry-Porter law, J. Opt. Soc. Am. A7, (1990). 14. A. Sullivan, DepthCube solid-state 3D volumetric display, in Stereoscopic Displays and Virtual Reality Systems XI, A. J. Woods, J. O. Merritt, S. A. Benton, M. T. Bolas, eds., Proc. SPIE 5291, (2004). 15. E. Downing, L. Hesselink, J. Ralston, and R. Macfarlane, A three-color, solid-state, three-dimensional display, Science 273, (1996). (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12760

2 1. Introduction Information processing technology for three-dimensional (3D) images is undergoing rapid progress and is being widely adopted in various fields. The development of displays that produce natural-looking stereoscopic images is expected to make such 3-D image systems more useful. The majority of commercially available stereoscopic displays are based on binocular parallax. These displays have some problems, including visual confusion and fatigue caused by inconsistencies in the 3D visual information, such as between focus and binocular convergence [1-3]. Volumetric displays can provide 3D images, which satisfy all the criteria of stereoscopic vision without the need for special glasses because the 3D real images that they produce are made up of light spots arranged in 3D space by optical scanning [4-5]. In volumetric display systems, a light spot or a two-dimensional (2D) image is rapidly scanned in 3D space. A realistic full 3D image can then be observed as a result of persistence of vision. Researchers have proposed various 3D scanning methods using, for example, rotation of a projection screen [8-10], and movement of an image plane with a varifocal lens [6,7]. We have previously described the basic principles of a volumetric 3D display system in which an inclined 2D image is moved in lateral directions perpendicular to an optical axis using a mirror scanner [11, 12]. In those reports, we presented experimental results of a preliminary volumetric display system using an 8 8 array of light emitting diodes to confirm the principles of our method [11]. In this paper, we report experimental results of a volumetric display system, which produces higher resolution 3D images using a vector-scan cathoderay-tube (CRT) monitor. In Section 2, we recap the principles of the volumetric display system using an inclined image plane, and we discuss some advantages and problems of this method. In Section 3, we describe the volumetric display system that we constructed. In Section 4, we describe experimental results of stereoscopic 3D images formed by the volumetric display system, and we give some conclusions in Section Principle of volumetric display using inclined image plane Figure 1 is a schematic diagram of our proposed volumetric display system. A 2D display device is placed obliquely in an optical imaging system, in which a rotating mirror (mirror scanner) is inserted. In this figure, the 2D display device is placed perpendicularly to the xz plane and is inclined with respect to the yz plane in the object space of the imaging system. An inclined planar image is formed in the image space of the optical system. When the mirror is rotated about a shaft parallel to the y-axis, the inclined planar image in the image space, whose coordinate system is x', y', z', is moved laterally in the x' direction. A locus of the moving image can be observed as a series of moving afterimages as a result of the high-speed rotation of the mirror. Inclined cross-sectional images of a 3D object are displayed on the 2D display device in accordance with the position of the image plane. A 3D real image is thus formed as a stack of 2D cross-sectional images. (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12761

3 Fig. 1. Schematic diagram of the volumetric display system using an inclined 2D display device and a mirror scanner. In a non-telecentric imaging system, the magnification of an image varies with the depth. Therefore, cross-sectional patterns that are displayed on the 2D display device must be deformed in advance by taking into consideration this change in magnification to form a nondeformed 3D image. In general, the distance moved by an image in response to rotation of the mirror varies according to the distance from the lens. This causes the angle of the inclined planar image to change with the angle of the mirror. Therefore, it is necessary to prepare sectional images at angles corresponding to the changing angle of the image plane. A 4-f optical system composed of two lenses and a mirror scanner placed at the common focal plane of both lenses is telecentric in both image and object spaces. In this optical system, neither the magnification nor the distance that the image moves depends on the depth. Fig. 2. The relationship of the 3D image region and the movement of the inclined image plane. The angle of the 2D display device affects the scanning efficiency in terms of scanning distance and number of scanning planes required. Let the widths of the display region of a 3D image be, a, b, and c in the x', y', and z' directions, respectively. We assume that the angle of the inclined image plane, θ is constant for each shifted image position of the mirror scanner. Thus, the image must be moved by the mirror scanner through a distance, d = a + c tanθ, (1) in order to scan the entire 3D image region, as shown in Fig. 2. Thus, the distance, d, becomes shorter at a larger angle of the inclined image plane. If the frame rate of the 3D images is f, and the pixel interval in the x' direction of the 3D image is g, the refresh rate of the 2D display device, r, is required to be (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12762

4 r = g f a + c tan θ. (2) Accordingly, a larger angle of the inclined image plane moderates the refresh rate requirement of the display device. In addition, the size of the 2D image, which is parallel to the xz plane, needs to be c/sin θ. Thus, the required area for the 2D display device is smaller for larger angles of the inclined image plane. In conclusion, a larger angle of the image plane is therefore more desirable. In the experimental system we constructed, the angle of the inclined image plane was 45 degrees because it was easy to generate cross-sectional images from voxel data at this angle and to match the pixel pitch for every point of the 3D image. 3. Experimental volumetric display system using a vector-scan CRT monitor A schematic diagram and a photograph of the constructed system are shown in Figs. 3 and 4, respectively. Concave mirrors were used as optical imaging elements because it is easier to produce concave mirrors with larger diameter and larger numerical aperture compared with refractive lenses. The diameter of the concave mirror was 152 mm, and the focal length was 152 mm. Two concave mirrors were arranged adjacent to each other in the vertical direction. A vector-scanning CRT monitor was used to display cross-sectional images of the object to be viewed. The CRT monitor was inclined upward by 45 degrees relative to the optical axis. A galvanometric mirror scanner of dimensions 100 mm in length and 50 mm in width was used. The long sides of the mirror were aligned parallel to the horizontal direction. The galvanometric mirror was controlled by a triangular wave signal synchronized to the CRT monitor signal produced in each frame. A real optical image of the 2D image displayed on the CRT monitor was formed by the two concave mirrors and the galvanometric mirror. The distance from the monitor to the concave mirror and the distance from the galvanometric mirror to the two concave mirrors were adjusted to be equal to the focal length of the concave mirror in order to make the optical system telecentric. The optical image was moved in the vertical direction by rotating the mirror on a horizontal shaft. Fig. 3. Schematic diagram of the volumetric display system. (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12763

5 Fig. 4 Photograph of the volumetric display system. To produce flicker-free 3D images, every 3D scanning period must be completed within the afterimage (persistence) threshold of the human eye, namely, a few tens of milliseconds [13]. It is necessary to switch the cross-sectional images displayed on the CRT monitor at a frequency equal to the product of the 3D frame rate and the number of cross-sectional images. Since it is difficult to achieve such a high frame rate using a raster-scan display device, the vector-scan CRT monitor was used in this experiment. The vector-scan CRT monitor can draw an image consisting of a small number of lines quickly. The display area was 82 mm high and 102 mm wide. The brightness controller generated a brightness signal for the CRT monitor, to turn the monitor off in the return path of the reciprocating motion of the galvanometric mirror to avoid displaying an image twice. Display data stored in a computer was converted into an analog signal and sent to the vector-scan CRT monitor via a 24-bit video interface. The red, green, and blue (RGB) color signals of the video interface were used for the x-coordinate, the y-coordinate, and the brightness, respectively. The allocation of the signals can be exchanged, because the formats of the RGB signals is equivalent. A 60-Hz vertical synchronizing signal of the video interface was used as a trigger to generate a signal for controlling the galvanometric mirror. Each crosssectional image could be drawn in a period corresponding to the 38-kHz frequency of the vector-scan CRT monitor. The resolution of each cross-sectional image was because the red and green signals from the video interface, specifying the x- and y-coordinates, were 8-bit signals. The memory capacity of the video interface restricted the number of sampling points of each cross-sectional image and the number of cross-sectional, which were 1280 and 1040, respectively. (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12764

6 (a) (b) Fig. 5. Square wave shapes displayed on the vector-scan CRT monitor with frequencies of (a) 2.2 MHz and (b) 5.8 MHz. Fig. 6. Frequency response of vector-scan CRT monitor. We drew square wave patterns on the CRT display to measure its frequency response. Figure 5 shows the images displayed the CRT monitor when square waves of frequencies 2.2 MHz and 5.8 MHz were input. The broken lines show the ideal square waves responses. The waveforms were distorted due to the limited time response of the CRT monitor. Figure 6 shows the degradation of the amplitude of the square wave pattern versus frequency. The maximum modulation frequency of the signal input to the CRT monitor from the VGA interface was about 35 MHz, which was too high for drawing the required patterns. To reduce the temporal frequency of the signal, we inserted dark points for each point, corresponding to the transition time of the CRT, to prevent unwanted bright lines appearing during the transition period. We observed the quality of several 3-D images, which contained the various numbers of the dark points. We decided to use nine dark points, because it was the minimum number for keeping the quality of the 3-D image. (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12765

7 4. Experimental results of stereoscopic image formation by the volumetric display system Here, we show experimental results of stereoscopic 3D images formed by the volumetric display system. We used 3D computer tomography (CT) images as the object data. A stack of cross-sectional CT image slices was processed using 3D graphics software to extract regions of interest, to interpolate the slices, and to smooth the 3D images. In principle, a 3D image formed by the volumetric display was a phantom image; that is to say, a hidden (occluded) part of the object could be seen through a front part of the object even though the slices overlapped each other. To solve this problem, we removed hidden surfaces of the object using 3- D graphics software. Only the front-most points on contours of the object were retained in each cross-sectional image. We determined the coordinates and the degrees of brightness of the points on the contours of the cross-sectional images inclined at 45 degrees to display the those images on the vector-scan CRT monitor. Figures 7(a) and 7(b) show photographs of a stereoscopic 3D image observed from the front and from the left at an oblique angle, respectively. The 3D images changed smoothly depending on the observer position. We confirmed that the 3D images had natural depth perception. The maximum viewing angles in the horizontal and vertical directions were about 30 degrees and 15 degrees, respectively. The image region was a cube of 6 cm on each side. These image size and viewing angle can be predicted from the observation region restricted by the numerical aperture of the optical system described in reference [11]. If the sum of the width of a 3-D image l i and the effective aperture of a mirror scanner l m is smaller than the diameter of a lens D (l i + l m < D), the width l o of an observation region at the distance from the image to an observer s is expressed by l o = sl m l i. (3) f If l i + l m > D (l i < D), l o is expressed by l o = s(d l ) i l i. (4) f The maximum viewing angle can be estimated by 2 tan -1 (l o /2s). The estimation of the experimental system is in agreement with the observed results. Figure 8 shows a movie of a spinning 3D image. A 3D movie of the spinning object was created from the CT images using the 3D graphics software. Each frame of the 3-D movie was coded for the volumetric display in advance and was transferred from the computer to the CRT monitor continually to display a continually moving 3-D image. There were undesired vertical trails from bright points in the 3D images formed by the experimental system. They were caused by persistence of the phosphors of the CRT monitor. In order to form clearer images, it will be necessary to use phosphors with low persistence, or another type of high-speed display device. (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12766

8 (a) (b) Fig. 7. Three-dimensional images formed with the volumetric display. The 3D images were viewed (a) from the front and (b) from the left at an oblique angle. Fig. 8. (100KB) Movie of a volumetric image of a spinning human skull. 5. Discussions One of the keys in developing a practical volumetric display is an effective 3-D scanning method. Table 1 shows a comparative table of the proposed method and other major volumetric display methods, which based on a rotating screen [10], a stack of switchable liquid crystal scattering shutters [14], two-beam activation of upconversion fluorescence [15], and a moving image plane with a varifocal lens [6], respectively. Some advantages of our method, such as the uniformity of voxel attribute are caused by translational motion of an optical real image without changing the focal length of an imaging system. Moreover, optical image formation in the air without a project screen is advantageous for constructing an interactive system to access the 3D image directly using 3D positioning devices. The restriction of viewing angle by the numerical aperture of the optical system is a disadvantageous point due to observation of an optical real image. Large lenses with a large numerical aperture and a large scanning mirror are required in order to increase the observation area. However, a large numerical aperture causes larger optical aberrations, and it is not straightforward to construct a large mirror scanner. Especially, aberrations of curvature and distortion of the image field affected the (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12767

9 distortion of the 3-D images in the experiments. These issues and tradeoffs must be considered when designing and constructing the system. The proposed volumetric display method is suitable for 3D imaging applications, which require accurate grasp of the 3-D structure of volumetric data, such as 3D medical imaging by x-ray computerized tomography or magnetic resonance, because of the uniform voxel attribute. The experimental system is, however, not suitable for displaying 3D images containing many valued voxels because of the restriction of the memory capacity and the time response of the CRT display unit. Moreover, luminous persistence of the CRT disturbed fine imaging. These disadvantages were caused by the performance of the display unit used in the experimental system. The quality of a 3-D image can be improved by using a 2D display device, which has high time-response and low persistence, such as a DMD used in other volumetric display systems [10, 14]. Monochromatic image is another disadvantage of the prototype system. Multicolored imaging can be achieved by providing three display units for three primary colors in the similar way as other volumetric displays [10, 14]. Table 1 Comparison of the proposed method with conventional volumetric displays. System Limitation of Size of 3- D image space Attributes of voxel Viewing angle Rotating screen [10] Size of Rotating screen restricted by the driving power Enlargementof voxels along the rotation direction as the distance from the rotation axis. Invisible voxels on the screen parallel to the viewing direction. 360 Stack of switchable liquid crystal scattering shutters [14] Limitation of the image depth by the depth of focus of the projection optical system Larger separation of voxels in depth direction than other direction Limitation of the number of voxels by transparency of the panel < 180 Two beam activation of upconversion fluorescence [15] Size of fluorescent medium Beam alignment problem for small voxel Gohst voxel problem 360 Moving image plane with a varifocal lens [6] Size of varifocal mirror Change of voxel size depending on depth due to the change of magnification Depending on numerical aperture of the optical system Proposed method Size of optical imaging components Uniformity of size, density, and brightness Depending on numerical aperture of the optical system (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12768

10 Table 1 Comparison of the proposed method with conventional volumetric displays. (Continued) System Color reproduction Optical aberrations Interaction with the 3D image space Rotating screen Achieved Small Impossible Stack of switchable liquid crystal scattering shutters Achieved Small Impossible Two beam activation of upconversion fluorescence Difficult of individual activation of each wavelengths Small Impossible Moving image plane with a varifocal lens Possible by improvement of a display unit Certain aberrations due to large numerical aperture and varifocal lens Possible Proposed method Possible by improvement of a display unit Certain aberrations due to large numerical aperture Possible 6. Conclusions We have demonstrated a volumetric display system based on 3D scanning using an inclined image plane. Three-dimensional images consisting of pixels can be formed using a vector-scan CRT monitor for displaying 2D cross-sectional images and a galvanometric mirror scanner for scanning a 3D volume. It was possible to observe stereoscopic 3D images in which all stereoscopic visual factors were satisfied. Improvement of the CRT display unit, especially suppression of the luminous persistence and the frequency response, is required to achieve fine image formation. Acknowledgment This research was partially supported by the Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Scientific Research (C), , 2005, (C) 2006 OSA 25 December 2006 / Vol. 14, No. 26 / OPTICS EXPRESS 12769