Three-dimensional integral imaging for orthoscopic real image reconstruction

Size: px

Start display at page:

Download "Three-dimensional integral imaging for orthoscopic real image reconstruction"

Scot Weaver
6 years ago
Views:

1 Three-dimensional integral imaging for orthoscopic real image reconstruction Jae-Young Jang, Se-Hee Park, Sungdo Cha, and Seung-Ho Shin* Department of Physics, Kangwon National University, 2-71 Republic of Korea ABSTRACT Integral imaging is one of the attractive autosteroscopic three-dimensional(3d) displays. The 3D image can be composed by integrating elemental images with different perspectives. Due to the geometry of integral imaging system, pseudoscopic imaging is an inevitable problem regardless of image type such as imaginary and real. Various methods are presented to improve the image quality and viewing parameters but few of trials for solving pseudoscopic imaging problem are presented. We propose and compare two methods for orthoscopic 3D real image reconstruction in integral imaging system using elemental image transformation. Theoretical analyses of elemental image transformation for 3D real image reconstruction are described in details and the experimental results are presented. Keyword: Three-dimensional display, integral imaging, orthoscopic, pseudoscopic, image processing 1. INTORDUCTION Integral imaging(inim) was first proposed by Lippmann in 198 for 3D image reconstruction 1. The InIm system is composed of the lens array on the image side for capturing elemental images, and the other lens array that is set on the display device on the display side for 3D image reconstruction. The elemental images, which are generated from the object by using lens array, are inverted small images. Unlikely other 3D display method, InIm provides continuous perspectives of 3D information in the proper viewing area. InIm has many advantages such as full color, full parallax and continues viewing points etc., but on the other hand it has several serious disadvantages such as narrow viewing area and pseudoscopic imaging for practical application. The pseudoscopic imaging is that the reconstructed image is viewed from the opposite direction that used for image pickup. In InIm system elemental images are formed by micro-lenses as inverted real images during pickup process and the 3D image is formed at the symmetric position as the object. Due to this fundamental geometry of InIm system, pseudoscopic imaging is an inevitable problem regardless of image type. Many methods are proposed to overcome the disadvantages such as narrow viewing point 2-6 and the pseudoscopic image problem. To solve pseudoscopic image problem, orthoscopic-pseudoscopic image-conversion optics, which needs a complex lens system that uses specially designed prisms, was proposed. 5 However, it was not useful to apply practical applications due to the complexity of optical system. There are two more methods for solving pseudoscopic problem. One is the elemental image rotation about the center of the elemental image. These conversion can be carried out during image pickup or image reconstruction either GRIN lens array or electronically. In this case, however, reconstructed 3D image is not real but virtual. The other method is based on the two-step integral imaging, which was used in case of holographic display. 6 In this paper, we propose and analyze two methods for pseudoscopic-orthoscopic conversion; twostep integral imaging using continuous perspective 3D image synthesizing and direct transformation of elemental images. We also present experimental results to reconstruct orthoscopic 3D real image. 2. 3D IMAGE RECONSTRUCTION IN INTEGRAL IMAGING SYSTEM In general, two kinds of reconstructed images are available in imaging system according to the viewing direction. One is the orthoscopic image, which has the same depth information with object, and the other is the pseudoscopic image, which has the reversed depth. Figure 1(a) shows the configuration of the pseudoscopic real image reconstruction. During the pickup step 3D object is decomposed into the elemental images, which keep the perspective information at Holography, Diffractive Optics, and Applications II, edited by Yunlong Sheng, Dahsiung Hsu, Chongxiu Yu, Byoungho Lee, Proceedings of SPIE Vol (SPIE, Bellingham, WA, 25) X/5/$15 doi: /

2 corresponding viewing angle. In the pseudoscopic reconstruction, when we use an identical lens array for pickup and display steps, the reconstructed image is the same as the object except depth information. It is possible to convert pseudoscopic image to orthoscopic image by using elemental image conversion as shown in Fig. 1(b). In this case, the elemental image should be rotated in a 18-degree arc about the center of the each elemental image and the 3D image can be reconstructed as virtual image as shown in Fig. 1(b). To perform this conversion, two sets of lens array or a gradient-index-lens-array were used between pickup and display steps. 7 When we consider whole optical system, in general, small numerical aperture(na) can be provided with virtual imaging. Therefore, the orthoscopic real image reconstruction is demanded for practical applications. Fig. 1. Basic principle of integral imaging (a) Pickup and pseudoscopic real image reconstruction, (b) Orthoscopic virtual image reconstruction after elemental image conversion. 2.1 Two-step integral imaging for orthoscopic real image reconstruction Two-step imaging to realize orthoscopic image reconstruction is well known technique in holographic display and it is possible to apply the same technique to the InIm system. However, the viewer should submit to serious image degradation according to two-step imaging in InIm, because the resolution of InIm is limited to the several important parameters and the absolute value might be very smaller than the object itself. Figure 2 shows an example of two-step InIm system. The system is composed of three squared parts-left, center, and right with dotted lines as shown in Fig. 2. The left square represents the pickup process, which can be preformed with optical method with real object or computer generation technique without real object. The first part of center square represents display step to reconstruct pseudoscopic real 3D image and the second part of the center square means the pickup process of second step. The pickup and display process appeared in center square can be performed with optical method or with computer using image synthesizing technique. The optical method provides two-dimensional processing and fast speed, but on the other hand the image resolution should be degraded seriously. Jang et. al has tried to solve this problem with nonstationary micro-optics but the improvement in resolution was limited and the fast moving part cause trouble in many cases 4. Therefore, we propose the 3D image synthesis technique by using the integration of elemental images. This method is based on the image mapping method with continuous perspectives. In this method, each elemental image is decomposed into small sections, which are selected and integrated into the proper perspective without image degradation. The reconstructed image, which located in center square, as shown in Fig. 2 can be stored in the form of many perspective 2D images. The elemental images for last reconstruction, which are located in overlapped area of center and right squares, can be generated easily by using synthesized 2D images. As shown in right square box of the Fig. 2, the final image is 38 Proc. of SPIE Vol. 5636

3 reconstructed in the same direction of the object in left square. Although the concept of this method is very clear and the results obtained at each stage are useful, this process needs a lot of calculation steps. If we divide elemental image into MxN parts and total number of lens in lens array is given P, the total calculation steps is MxNxP, which is huge calculation steps, because MxN determines the resolution of reconstructed 3D image.. Fig. 2. Schematic diagram of two-step integral imaging for orthoscopic real image reconstruction. The whole system is composed of three parts according to the roles; pickup, synthesis, and reconstruction. Image synthesis can be performed electronically. hgggggggiggggggggjgggggggggkggggk GGGGGGGlG Fig. 3. Schematic diagram to explain basic concept of orthoscopic real image reconstruction. Each column from left presents the object(a), elemental images(b), 1st conversion results(c), 2nd conversion results(d, D ), and reconstructed 3D image(e). Proc. of SPIE Vol

4 2.2 Direct conversion of the elemental images for orthoscopic real image reconstruction As describe in section 2.1, two-step InIm method is not suitable for dynamic or video image, because it needs long time to obtain the calculation results. To reduce the calculation time, direct conversion of the elemental image is investigated. The direct conversion is very similar with the conversion for orthoscopic virtual image reconstruction. In this case, elemental image should be revolved 18 degree about the center of elemental image. In addition to the conversion, the converted elemental images move to the proper position on the other side of lens array. The proposed direct conversion method can be applied to two steps; from B to C and C to D, as shown in Fig. 3. The column B in Fig. 3 represents the elemental images which are captured with lens array and column C is the first converted elemental images. The first conversion, which is performed between column B and C, is similar to elemental image conversion for orthoscopic virtual image reconstruction but the elemental image is remained at the pickup position. The second conversion, which is performed between column C and D, is the resizing of elemental images. In general, the size of elemental image is given as a function of the distance between object and the lens array. So, the magnification is also depended on the location of the point in the object. In the direct conversion, the depth and the magnification of the object should be changed at the same time. Fig. 4. The coordinate system for object, elemental image, and integrated imaging point. Figure 4 shows the coordinate system for pickup and display related to direct conversion method. To analyze the elemental image direct conversion, we need to consider three coordinate systems; for object, integrated point, and elemental image as shown Fig. 4. The x en, s ) plane represents the elemental image plane in pickup process and the ( ( x, s en I ) plane represents the elemental image plane on display process. In addition, we can also assign the point on the object to the ( x, z ). In the same way, the point on the reconstructed image has been assigned as ( x I, z I ). Let P n and s denote the pitch of the lens array and the imaging distance of a lens array, respectively. Then, we have the imaging point x and s en x s = n en 1 Pi z + i= x z s ( n = 1,2,3,...) (1) 382 Proc. of SPIE Vol. 5636

5 s fz = (2) f z Therefore, the elemental image conversion can be considered with the assigned coordinates and the mapping function in Fig 5. The step (1) and (3) in Fig. 5 show the relation between the object and imaging point in pickup process, and the integrated and elemental image point, respectively. The step (2) shows the direct conversion and is composed of two stages. First stage is the rotation of elemental images and second stage is the ration adjusting to correct distortion of reconstructed image. Fig. 5. The elemental image conversion for orthoscopic real image display. (1) Pickup process, (2) elemental image transformation, and (3) display process. The point ( x eni, s ) on the elemental image plane is the mapping of imaging point ( x, z i ) step, the relation of between xen and x en2 can be represented as follows: x x en en on the object. In the second = xen 1 + b (3-a) 2 = xen 1 a (3-b) In the second step, elemental image rotation in 18 degree and ratio adjustment of each element image are performed. The converted point just after rotation is x' x + a (4-a) en2 = Proc. of SPIE Vol

6 x' x b (4-b) en = After ratio adjustment, the coordinate for the points can be formulated. x' ' x' ' x + b = (5-a) en2 x' en2 xen 1 + a x a = (5-b) en x' xen 1 b In this method, we chose a point that is the center of three points for using the datum conversion. Actually, any other point can be a datum point for transformation. When we use other point as an axis of rotation, the image distortion is increased. Therefore, the proper choice of axis point is very important in this operation. In case of voluminous object, center of object should be best point for direct conversion. 3. EXPERIMENTAL RESULTS To verify the principle of orthoscopic real 3D image reconstruction, we have performed the experiment with computer generated elemental images and the optically picked up elemental images. Figure 6 shows the experimental results obtained from two-step InIm method. As shown in Fig. 6, the image conversion from pseudoscopic to orthoscopic is preformed successfully. From the experimental results, we can see that the image resolution is improved, and the flipping is appeared at the smaller viewing angle than low depth object. Fig. 6. Orthoscopic real image reconstructed from two-step integral imaging method with different image depth(d). (a) d=1p, (b)d=3p, and (c)d=5p, where p is the pitch of the lens in lens array. Fig.7 shows the experimental results using direct conversion method. The conversion has been performed with two objects with the distance at 6.5cm (face of 5 spots) and 8cm (face of 3 spots). The size of the object is 2x2x2 cm 3 and the distance two between objects is 1.5cm. The lens array using the direct conversion is 4 4 and the focal length of elemental lens is 4cm. The square type elemental lens is 1cm in side. The picture of elemental image in Fig.7 is the result of the direct conversion of original elemental image and the bottom of Fig.7 shows reconstructed images using this elemental image. In the reconstructed imagefrom the series of 3D images with different perspectives, it is clear that the reconstructed image is orthoscopic real as described in Section 2.2. According to this result, the direct conversion method is useful to realize orthoscopic real image in InIm system. 384 Proc. of SPIE Vol. 5636

Fig. 7. Reconstructed orthoscopic real image with direct conversion method. 4. CONCLUSION Orthoscopic real image reconstruction in 3D display system is very useful for practical application.

7 Fig. 7. Reconstructed orthoscopic real image with direct conversion method. 4. CONCLUSION Orthoscopic real image reconstruction in 3D display system is very useful for practical application. In this paper we have proposed and compared two methods for orthoscopic real image reconstruction working in the InIm system; two-step and direct conversion InIm. In the two-step InIm method, the pseudoscopic 3D image reconstruction and the elemental image regenerating have been performed electronically to reduce the image degradation to a minimum. We have demonstrated that the resolution of the reconstructed image much better than that of the optical two-step imaging system. To reduce the calculation time, we have proposed the pseudoscopic to orthoscopic conversion method based on the direct conversion of elemental image. We have verified the principle of this method by the experimental results. ACKNOWLEDGMENTS This work was supported by University ITRC(MSRC in Kangwon National University) project of Korea Ministry of Information and Communication. * Corresponding author, shinsh@kangwon.ac.kr REFERENCES 1. G. Lippmann, La photographic integrale, C. R. Acad. Sci., , S. H. Shin and B. Javidi, Speckle-reduced three-dimensional volume holographic display by use of integral imaging, Appl. Opt. vol. 41, , S. H. Shin and B. Javidi, Viewing-angle enhancement of speckle-reduced volume holographic threedimensional display by use of integral imaging, Appl. Opt. vol. 41, , B. Lee, S. Jung, and J. H. Park, Viewing-angle-enhanced integral imaging by use lens switching, Opt. Lett. Vol. 27, , 22. Proc. of SPIE Vol

8 5. J. S. Jang and B. Javidi, Improvement of viewing angle in integral imaging by use of moving lenslet arrays with low fill factor, Appl. Opt. vol. 42, , J. S. Jang and B. Javidi, Two-step integral imaging for orthoscopic three-dimensional imaging with improved viewing resolution, Opt. Eng. vol. 41, , G GN. Davies, M. McCormick, and L. YangSGThree-dimensional imaging systems : a new developmentsgappl. Opt.G vol. 27, , 1988.G G 8. H. E. Ives, Optical properties of a Lippmann Lenticulated sheet, J. Opt. Soc. Am. Vol. 21, , J. Arai, F. Okano, H. Hoshino and I. Yuyama, Gradient-index lens-array method based on real time integral photography for three-dimensional images, Appl. Opt. vol. 37, , Proc. of SPIE Vol. 5636

Vision-Based 3D Fingertip Interface for Spatial Interaction in 3D Integral Imaging System

International Conference on Complex, Intelligent and Software Intensive Systems Vision-Based 3D Fingertip Interface for Spatial Interaction in 3D Integral Imaging System Nam-Woo Kim, Dong-Hak Shin, Dong-Jin