Uniform angular resolution integral imaging display with boundary folding mirrors

Transcription

1 Uniform angular resolution integral imaging display with boundary folding mirrors Joonku Hahn, Youngmin Kim, and Byoungho Lee* School of Electrical Engineering, Seoul National University, Gwanak-Gu Sillim-Dong, Seoul , South Korea *Corresponding author: Received 27 October 2008; revised 10 December 2008; accepted 10 December 2008; posted 11 December 2008 (Doc. ID ); published 14 January 2009 Uniform angular resolution integral imaging display is proposed. Conventionally, inefficient bias of the perspectives around boundaries is disregarded and it is impossible to display the field of view over full spatial resolution. However, the proposed display has four boundary folding mirrors and these mirrors fold the view volumes correctly to form uniform angular resolution within them. The distribution of perspectives and field of view in the proposed system are analyzed and the removal of this boundary effect is confirmed experimentally Optical Society of America OCIS codes: , Introduction Integral imaging (InIm) is one of the most feasible directional displays that give different perspectives according to view directions. This technique displays a three-dimensional (3D) object with full parallax without any viewing aids, such as glasses, and it does not require any diffuser screen that constrains the directions of perspective [1 3]. Therefore, the elemental images are focused on the image plane following the lens law; this image plane is commonly called the central depth plane (CDP). In this CDP, directions of perspective are distributed not only horizontally but also vertically, resulting from a two-dimensional lens array. The number of perspectives is understood as angular resolution. Figure 1 shows the spatial and angular resolutions on the CDP of an InIm display. There is a trade-off relation between the spatial and the angular resolutions [4 6]. Therefore, the angular resolution is increased at the cost of the reduction of spatial resolution. The viewing angle of the InIm display is defined as the extent of the perspectives at the fixed position of the CDP /09/ $15.00/ Optical Society of America In an InIm display, many methods to widen the viewing angle have been proposed. In one method, the surface of elemental images is embossed to widen the individual viewing angle [7]. In another method, both elemental images and lens array are curved to enhance the field of view [8]. However, these studies are concerned only with the viewing angle in the bulk region. In the boundaries of an InIm display, the viewing angle is not as uniform as that in the bulk region, and the perspective views around the boundaries are biased to the outward direction, which is inefficient for creating the field of view. The clues to removing the effect of boundary can be found in a multiprojection display such as a hologramlike display. In these displays, two mirrors are utilized to fold left- and right-side perspectives [9 11]. The folding mirrors make the additional virtual projections by reflection and the folded view volumes efficiently form the fields of view over the full spatial resolutions. However, these multiprojection displays have only a one-dimensional projection array and every projection optic is positioned apart from the others. Therefore, the asymmetric diffuser that is usually called a holographic screen is necessary and full parallax is impossible. We propose an InIm display with boundary folding mirrors. This display gives uniform angular resolution within four boundaries. Figure 2 shows 504 APPLIED OPTICS / Vol. 48, No. 3 / 20 January 2009

2 Fig. 1. (Color online) Spatial and angular resolutions on central depth plane of InIm display. a schematic of the proposed InIm display. These four folding mirrors extend to CDP whose location is determined by the focal length of the lens array and the gap between the elemental image and the lens array. In Section 2, the angular resolution and distribution of perspectives in the InIm display are described and the effects of boundary folding mirrors are expressed analytically. In Section 3, the field of view of the proposed InIm display is compared with that of the conventional InIm display and elemental images of the proposed system are explained and experimental results are presented and discussed. In Section 4, conclusions and perspective are given. 2. Angular Resolution in Proposed Integral Imaging Display In directional displays, the angular resolution is given by the number of perspectives. In a view volume by a single lens, the perspective is a function of its position. Therefore, the angular resolution is equal to the number of overlapping-in-view volumes at a given position. Figure 3 shows the view volume and directions of perspective in the InIm display. Since the CDP is an image plane of the elemental image, Fig. 3. (Color online) View volume and directions of perspective in InIm display. the chief ray of an individual lens is important to describe the directions of perspective and the view volume generated by this lens is represented as the existence of the chief ray in the space. The view volume generated by the lens positioned at ðx nm ; y nm ; z L Þ is defined by dgap ðx x nm Þ dgap ðy y nm Þ V nm ðx; y; zþ ¼rect rect : wðz z L Þ wðz z L Þ ð1þ Here, d gap is the gap between the lens array and the elemental image and w is the lens spacing of the lens array. The function rectðþ is a rectangular function, which is only true (i.e., gives numerical value of 1 as a function output) when the absolute value of the input argument is less than 1=2. Otherwise, its functional output is 0. Therefore, the view volume defined in this paper is a Boolean value. For example, this view volume includes the position ðx nm ; y nm ; z z L Þ. This means that the extension of a line connecting two points, ðx nm ; y nm ; z L Þ and ðx nm ; y nm ; z z L Þ, passes through the corresponding elemental image. Therefore, the view volume is shaped as two pyramids with the summits in contact [12]. Since angular resolution is the number of imbricate view volumes, angular resolution at a position ðx; y; zþ is given by R A ðx; y; zþ ¼ XN X M n¼1 m¼1 V nm ðx; y; zþ; ð2þ Fig. 2. (Color online) Schematics of InIm display with boundary folding mirrors. where N and M are the number of lenses in the horizontal and vertical directions, respectively, as shown in Fig. 3. The directions of perspectives are determined by the positions of individual lenses within the corresponding view volumes. At a given position ðx; y; zþ, the distribution of perspectives is the set of directions of chief rays passing through the lens array. Then the distribution of perspectives from the imbricate view volumes are defined by 20 January 2009 / Vol. 48, No. 3 / APPLIED OPTICS 505

3 D P ðx; y; zþ ¼f~r 0 j~r 0 ¼ð~r ~r nm ÞV nm ð~rþ for 1 n N; 1 m Mg: ð3þ Here ~r and ~r nm mean the position ðx; y; zþ and the position of lens ðx nm ; y nm ; z L Þ, respectively. From the point of view in angular resolution, the proposed InIm display is compared with the conventional InIm display. To make the issue clear, the analysis is restricted on the xy plane but it does not lose the generality. Figure 4 shows the imbricate view volumes and directions of perspective view on the CDP in the conventional InIm display. In Fig. 4(a), the number of overlapping-in-view volumes decreases around the boundaries and the angular resolution also decreases across the boundaries. As shown in Fig. 4(b), the local viewing angle is related to the angular resolution and the resultant viewing angle from perspectives becomes narrow near the boundaries. Moreover, in the region over the boundaries, only the outgoing directions of perspectives remain. Figure 5 shows the angular resolution and the distribution of viewing angle on the CDP in the conventional InIm display shown in Fig. 4. In Fig. 5(a), the viewing angle has the maximum value in the bulk region and decreases coming up to both upper and lower boundaries. As shown in Fig. 5(b), the distribution of perspectives has sawtooth shape in the bulk region since each direction of perspective changes according to the relative positions from the individual lenses. Moreover, there is only the outgoing direction the bias in perspectives results in a reduction of the field of view, which can be displayed over the total viewing angle. This phenomenon is discussed in more detail in Section 3. In the proposed InIm display with boundary folding mirrors, all view volumes are folded at the mirror planes. Since the number of mirror planes is four, there are eight reflected positions per a single lens. The original position of lens ðx nm ; y nm Þ and its reflected positions are represented as 8 < 2x 11 w x nm for p ¼ 1 x nmp ¼ x : nm for p ¼ 0 ; ð4aþ 2x N1 þ w x nm for p ¼ 1 8 < 2y 11 w y nm for q ¼ 1 y nmq ¼ y : nm for q ¼ 0 : ð4bþ 2y N1 þ w y nm for q ¼ 1 The resultant folded view volume is defined by V nm ðx; y; zþ ¼ X1 X 1 p¼ 1 q¼ 1 V nmpq ðx; y; zþ: ð5þ Here, V nmpq ðx; y; zþ is a part of the folded view volume from nine positions, which are the original position of the lens and its eight reflected positions. Since the boundary folding mirrors have finite length, V nmpq ðx; y; zþ is defined by d mirror ðx x nmp Þ V nmpq ðx; y; zþj ðp;qþ¼ð0;0þ ¼ H ðx nmp x 11 þ w=2þðz z L Þ þ 1 d mirror ðx x nmp Þ H ðx nmp x N1 w=2þðz z L Þ þ 1 d mirror ðy y nmq Þ H ðy nmq y 11 þ w=2þðz z L Þ þ 1 d mirror ðy y nmq Þ H ðy nmq y 1M w=2þðz z L Þ þ 1 dgap ðx x nmp Þ rect wðz z L Þ rect dgap ðy y nmq Þ wðz z L Þ ; ð6aþ d mirror ðx x nmp Þ V nmpq ðx; y; zþj ðp;qþ ð0;0þ ¼ H ðx nmp x 11 þ w=2þðz z L Þ 1 d mirror ðx x nmp Þ H ðx nmp x N1 þ w=2þðz z L Þ 1 d mirror ðy y nmq Þ H ðy nmq y 11 þ w=2þðz z L Þ 1 d mirror ðy y nmq Þ H ðy nmq y 1M w=2þðz z L Þ 1 dgap ðx x nmp Þ dgap ðy y nmq Þ rect rect : ð6bþ wðz z L Þ wðz z L Þ in perspectives around the boundaries and, then, the distribution of perspectives in the conventional InIm display is roughly a parallelogram shape. Therefore, Here, H½Š is the unit step function and d mirror is the length of the boundary folding mirrors, which is designed by 506 APPLIED OPTICS / Vol. 48, No. 3 / 20 January 2009

4 Fig. 5. (Color online) (a) Angular resolution and (b) distribution of perspective on CDP in conventional InIm display. Fig. 4. (Color online) Conventional InIm display with (a) imbricate view volumes and (b) directions of perspective on CDP. d mirror ¼ fd gap : f þ d gap ð7þ As previously mentioned, the angular resolution is the number of overlapping-of-view volumes and the angular resolution of the proposed InIm display is given by R A ðx; y; zþ ¼ XN X M X 1 X 1 n¼1 m¼1 p¼ 1 q¼ 1 V nmpq ðx; y; zþ: ð8þ The distribution of perspectives from the folded imbricate view volumes are defined by D P ðx; y; zþ ¼f~r 0 j~r 0 ¼ð~r ~r nmpq ÞV nmpq ð~rþ for 1 n N; 1 m M; p; q ¼ 1; 0; 1g: ð9þ Figure 6 shows the folded imbricate view volumes and directions of perspective views on the CDP in the proposed InIm display. In Fig. 6(a), the angular resolution is uniform since the reflected parts of the view volume complement the angular resolution near the boundaries. In this system, the viewing angle is also uniform and the directions of perspective are not biased, as shown in Fig. 6(b). Figure 7 shows the angular resolution and the distribution of perspectives on the CDP in the proposed InIm display shown in Fig. 6. In Fig. 7(a), the viewing resolution uniformly has the same value as the maximum in the bulk region. This effect is understood as the rotation of the outside part into the inside. As shown in Fig. 7(b), the distribution of perspectives is roughly a rectangle shape where the upper and lower sides have a sawtooth shape in the bulk region. This phenomenon is also understood as the rotation of the outside part into the inside. 3. Field of View of Proposed Integral Imaging Display In an InIm display, the field of view means the space where the 3D objects to be displayed are positioned. In the proposed InIm display, the field of view is different from that of the conventional InIm display. Though their dimensions and shapes are equal to each other, the locations are different. Figure 8 shows the movement of the field of view by means of boundary folding mirrors. Here, the total viewing angle is the viewing angle defining the viewing zone where the whole field of view can be observed. It has the same value as the local viewing angle in the bulk region, which is determined by Ω ¼ 2 arctanðw=2d gap Þ: ð10þ 20 January 2009 / Vol. 48, No. 3 / APPLIED OPTICS 507

5 Fig. 7. (Color online) (a) Angular resolution and (b) distribution of perspectives on CDP in proposed InIm display. Fig. 6. (Color online) Proposed InIm display with (a) folded imbricate view volumes and (b) directions of perspectives on CDP. In an InIm display, the view volume generated by an individual lens is shaped as two pyramids with the summits in contact, as represented by Eq. (1). The field of view can be defined as the region where the full perspective exists. The objects in this region can be watched by observers positioned at an infinite distance from the display within the viewing angle. Therefore, the field of view of the InIm display is a rhombus shape. Observers are positioned only in front of the display and the viewing zone, which is defined as a perfectly overlapped region of all the view volumes located in front of the display. Within this viewing zone, the observers can watch the whole field of view with full perspective. In the conventional InIm display, the largest cross section in the field of view is located on the lens array plane, as shown in Fig. 8(a). On the other hand, in the proposed system, the folding mirrors move the field of view into the front of the observer, as shown in Fig. 8(b). This phenomenon occurs because an additional lens array effectively exists in the mirror folded region. Here, the width w folded of this mirror folded region is determined by w folded ¼ w ðm x 1Þ=2 : ð11þ Here, is the ceiling function that converts a real number to the smallest integer not less than it and M x means the horizontal magnification by the lens law, which is given by M x ¼ f =ðd gap f Þ: ð12þ In the case that M x is not an integer, Eq. (11) is valid but the uniformity of angular resolution is broken. In comparison with the conventional InIm display, the field of view in the proposed InIm display has a larger cross section on the CDP. In general, the closer to the CDP 3D objects are positioned, the more correctly they are represented. Therefore, the proposed system uses the angular resolutions efficiently and displays the field of view over the full spatial resolution. The boundary folding mirrors make the field of view move toward observers and the proposed method has the advantage only for the real display mode when the CDP is located in front of the lens array. On the other hand, in the virtual display mode, the CDP is located behind the lens array. Then the field of view in the proposed InIm display has a smaller cross section on the CDP than that in the conventional InIm display. Therefore, it is improper to use this method for the virtual mode. In the proposed system, the reorganization is necessary to generate the set of elemental images. Figure 9 shows this reorganization relation. The additional lenses in the mirror folded region do not actually exist and the corresponding elemental images should be reorganized. In Fig. 9, we assume that the horizontal number of lenses is seven and that the magnification M x is 5. Each number on the CDP represents the spatial portions with the same width as 508 APPLIED OPTICS / Vol. 48, No. 3 / 20 January 2009

6 Fig. 10. (Color online) Experimental setup. Fig. 8. (Color online) Movement of field of view by boundary folding mirrors: (a) field of view in the conventional InIm display and (b) field of view in the proposed InIm display. the spacing w. Since the magnification M x is 5, two lens rows per boundary need to be reorganized. Every numbered portion is traceable from the CDP to elemental images. As expected, the elemental image near the boundary contains the mirror folded region. In this elemental image, two separate portions of it represent the same positions on the CDP with two different perspective views. Then the same numbers appear again in one elemental image. In this paper, viewpoint vector rendering is applied to generate raw elemental images [13]. It is the computer generation algorithm based on directions of perspectives. Therefore, the location of the field of view is easily defined and folding the elemental images is clearly calculated. The proposed system is embodied with a Fresnel lens array and the first surface mirrors. Figure 10 shows the experimental setup. The lens array is composed of 13 8 lenses with a focal length of 22 mm and a spacing of 10 mm. The set of elemental images has a pixel resolution, which is projected by a full high-definition (full HD) projector. The magnification M x is five and the resultant length d mirror of the boundary folding mirrors is 132 mm. Figure 11 shows a set of elemental images reorganized for the proposed InIm display. Here, the dashed lines separate the set of elemental images into individuals and the solid lines are the borders of the mirror folded regions. As previously discussed, in the elemental images near the boundaries, one object appears twice. For example, there are two blue striped balls in the far left elemental images. Figure 12 shows the perspective views of the embodied system. Each image is the view depending on the position of the CCD. The dashed rectangles are Fig. 9. (Color online) Reorganization relation in generating set of elemental images. Fig. 11. (Color online) Reorganized set of elemental images for the proposed InIm display. 20 January 2009 / Vol. 48, No. 3 / APPLIED OPTICS 509

7 Fig. 12. (Color online) Perspective views of the proposed InIm display: (a) upper left view, (b) upper center view, (c) upper right view, (d) middle left view, (e) middle center view, (f) middle right view, (g) lower left view, (h) lower center view, and (i) lower right view. the areas of the lens array unfolded by mirrors. That is, the outside regions of the dashed rectangle are the reflected images. In the conventional InIm, it is impossible to avoid the reduction of area displaying the image for the observer positioned apart from the center. And the conventional technique cannot display the outsides of the dashed rectangles. In Fig. 12, the maximum width of the outside region is two times the lens spacing. This experimental result agrees with the calculation that the width w folded of the mirror folding region is twice as wide as the lens spacing w from Eq. (11). 4. Conclusion A uniform angular resolution InIm display with boundary folding mirrors is proposed. By this proposed method, the angular resolution near the boundaries is complemented with the undesirable perspectives outside the boundaries and the bias of the perspectives around the boundaries is removed. Therefore, the field of view in the proposed display is moved and its cross section on the CDP becomes larger than that in a conventional system. It is desirable to display the field of view over the full spatial resolution. By experiments, the proposed InIm display is confirmed to form a uniform angular resolution and the field of view is displayed in agreement with the design parameters. It is expected that the proposed technique can be one of the most effective techniques for compensating the decrease of angular resolution in the boundaries of the InIm display. This work was supported by the Korea Science and Engineering Foundation (KOSEF) and the Ministry of Education, Science and Engineering of Korea through the National Creative Research Initiative Program (R ). References 1. B. Lee, J.-H. Park, and S.-W. Min, Three-dimensional display and information processing based on integral imaging, in Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer, 2006), pp B. Javidi and F. Okano, eds., Three Dimensional Television, Video, and Display Technologies (Springer, 2002). 3. H. Liao, M. Iwahara, N. Hata, and T. Dohi, High-quality integral videography using a multiprojector, Opt. Express 12, (2004). 4. T. Georgiev, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, Spatio-angular resolution tradeoffs in integral photography, in Proceeding of the 17th Eurographics Workshop on Rendering (Eurographics, 2006), pp A. Stern and B. Javidi, 3-D computational synthetic aperture integral imaging (COMPSAII), Opt. Express 11, (2003). 6. M. Hain, W. von Spiegel, M. Schmiedchen, T. Tschudi, and B. Javidi, 3D integral imaging using diffractive Fresnel lens arrays, Opt. Express 13, (2005). 7. S.-W. Min, J. Kim, and B. Lee, Wide-viewing projection-type integral imaging system with an embossed screen, Opt. Lett. 29, (2004). 8. Y. Kim, J.-H. Park, S.-W. Min, S. Jung, H. Choi, and B. Lee, Wide-viewing-angle integral three-dimensional imaging system by curving a screen and a lens array, Appl. Opt. 44, (2005). 510 APPLIED OPTICS / Vol. 48, No. 3 / 20 January 2009

8 9. T. Balogh, T. Forgács, T. Agocs, O. Balet, E. Bouvier, F. Bettio, E. Gobbetti, and G. Zanetti, A scalable hardware and software system for the holographic display of interactive graphics applications, in Eurographics Short Papers Proceedings (Eurographics, 2005), pp T. Agocs, T. Balogh, T. Forgacs, F. Bettio, E. Gobbetti, G. Zanetti, and E. Bouvier, A large scale interactive holographic display, in Proceedings of IEEE Conference on Virtual Reality (IEEE, 2006), p Y. Takaki, High-density directional display for generating natural three-dimensional images, Proc. IEEE 94, (2006). 12. J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, Undistorted pickup method of both virtual and real objects for integral imaging, Opt. Express 16, (2008). 13. S.-W. Min, K.-S. Park, B. Lee, Y. Cho, and M. Hahn, Enhanced image mapping algorithm for computer-generated integral imaging system, Jpn. J. Appl. Phys. 45, L744 L747 (2006). 20 January 2009 / Vol. 48, No. 3 / APPLIED OPTICS 511