UNIVERSITY OF CINCINNATI

Transcription

1 UNIVERSITY OF CINCINNATI DATE: June 1, 2001, hereby submit this as part of the requirements for the degree of: Doctorate of Philosophy (Ph.D.) in: Electrical and Computer Engineering and Computer Science It is entitled: Three-Dimensional Display Systems Implemented with A Micromirror Array I, Jun Yan Approved by: Stephen T. Kowel Fred R. Beyette, Jr. Jeffrey H. Kulick Chia-Yung Han Joseph T. Boyd Chong H. Ahn

2 Three-Dimensional Display Systems Implemented with A Micromirror Array A dissertation submitted to the Division of Research and Advanced Studies of the University of Cincinnati in partial fulfillment of the requirements for the degree of DOCTORATE OF PHILOSOPHY (Ph.D.) in the Department of Electrical and Computer Engineering and Computer Science of the College of Engineering 2001 By Jun Yan B.S., Fudan University, 1991 M.S., Fudan University, 1994 M.S.E., University of Alabama in Huntsville, 1999 Committee Chair: Dr. Stephen T. Kowel

3 Abstract A novel approach for three-dimensional (3-D) display systems implemented with a micromirror array was proposed, designed, realized and tested. The major advantages of this approach include: (1) micromirrors are reflective and hence achromatic (panchromatic), (2) a wide variety of displays can be used as image sources, and (3) time-multiplexing can be introduced on top of space-multiplexing to optimize the viewing-zone arrangements. Real-time auto-stereoscopy and motion parallax were the goals for these single-user 3-D display systems. First, auto-stereoscopy allows an observer see left and right images without any special eyewear or head-tracking devices. Second, different pairs of stereoscopic images can be seen according to the viewer s head position under horizontal displacement, denoted by series of viewing zones, so horizontal motion parallax is provided. These 3-D display systems use two spatial light modulators (SLM). The first one acts as the image source, which is relayed onto the second SLM, a micromirror array. Micromirrors redirect the light into appropriate viewing zones. We used backlit transparencies and a color CRT as the first SLM, which exemplifies the wide acceptance of image sources. Three simplifications in the optical design were made to lower the actuation requirements of the micromirrors. First, a collecting lens was introduced so that the micromirrors needed uniform actuation in one dimension (horizontal). Second, an interleaved actuation profile of the micromirrors was introduced to dedicate odd columns of micromirrors for the right eye views and even columns for the left ones. Finally, a doubleopening pupil was used to further lower the actuation requirements of the micromirrors. A two-view (left and right) 3-D autostereoscopic display system was first constructed. Left and right eye views in the forms of both still and motion 3-D scenes were displayed and viewers were able to fuse the stereo information. A multi-view (2 left and 2 right) 3-D autostereoscopic display system was then proposed. Although there are issues to work out moving micromirrors and providing more viewing zones the experimental results proved our design idea. This is a promising step toward a personal multi-view auto-stereoscopic 3-D display system.

4

5 Acknowledgements First and foremost, I would like to express gratitude to my advisor, Dr. Stephen T. Kowel. His guidance and instruction throughout this project greatly added to my research and study experience, and will continue be beneficial to my professional and personal life. Secondly, I would like to thank my professors and colleagues for this project. Dr. Chong H. Ahn and his group, especially Hyoung J. Cho, made a major contribution to the fabrication of the prototype device. I would also like to thank Dr. Jeffrey H. Kulick and Dr. Gregory P. Nordin at The University of Alabama in Huntsville. Thirdly, I am profoundly grateful to the support from my professors: Dr. Fred R. Beyette, Jr., Dr. Joseph T. Boyd, Dr. Chia-Yung Han, and Dr. Tomas D. Mantei at University of Cincinnati; Dr. John O. Dimmock, Dr. Joseph M. Geary, and Dr. Alexander D. Poularikas at The University of Alabama in Huntsville; Professor Diechi Sun and Professor Fuming Li at Fudan University. Finally, I would like to thank my family for their support. I dedicate this thesis to my parents.

6 Table of Contents Abstract Acknowledgements Table of Contents. 1 List of Figures 3 List of Tables. 5 List of Symbols.6 Chapter 1 Introduction Review of 3-D display systems Diffractive partial-pixel 3-D displays Micromirror arrays and their applications Proposal for MEMS 3-D display systems Overview of this dissertation.. 20 Chapter 2 Optical Design for MEMS 3-D display Conceptual design Advantages of MEMS approach Generic optical design for a MEMS 3-D display. 28 1

7 Chapter 3 Design and Fabrication of Micromirror Arrays Masks design Fabrication processes Finished micromirror arrays and actuation methods Initial testing and discussion.. 44 Chapter 4 Testing of the Optical System Image sources preparation Optical system setup and adjustments Testing with single-opening pupil Testing with double-opening pupil 51 Chapter 5 Prototype 3-D Display Systems Two-view autostereoscopic 3-D Display Multiple-view autostereoscopic 3-D Display Discussions.. 61 Chapter 6 Conclusion Summary of this work Future work suggestions 65 References Publications List. 71 Biography. 72 Appendix A Program Listing for Image Source Preparation

8 List of Figures 1-1. Holographic recording and reconstructing. a) The photographic plate records the interference pattern produced by the light waves scattered from the object and a reference wave. b) The hologram, after processing, is illuminated with the reference wave. Light diffracted by the hologram appears to come form the original object [29] Simplified drawing of the second generation MIT holographic display: a number of holographic sub-elements are segmented by the multi-channel Bragg cells and raster scanners [31] A glass cube used as the volumetric display medium. a) Energy level diagram. If an active ion of the glass absorbs energies from laser beams of λ 1 and λ 2, it will fluorescent in visible light. b) Two scanned, intersecting laser beams are used to address voxels and draw objects [20] Three-view lenticular display [36] D display implemented with diffractive partial pixels Partial pixel gratings for a) 16-view monochrome display and b) 4-view color display TI DMD structure: binary positioned micromirror integrated with CMOS controlling circuitry [41] A micromirror cross connect: a) chip structure; b) device architecture [46] Lucent LamdaRouter: a) one micromirror; b) routing architecture [47] a) Using diffractive partial pixels to form 16 views simultaneously. b) Using a single scanning micromirror to form 16 views sequentially General MEMS IC Vision Architecture Projection with scanning Initial system: a) left viewing zone and b) right viewing zone Directional angles α and β for a spherical contour Optical ray-tracing layout for a) left viewing zone, and b) right viewing zone. After introducing the collecting lens, only the uniformly horizontal actuation is required a) Left and right viewing zones separation and b) Interleaved micromirror profile for left and right viewing zones Optical system design with the double-opening pupil (inset): a) L1 and R1 viewing zones, b) L2 and R2 viewing zones Optical system analysis Geometric arrangement of (-S' 1 +S' 2 ), θ e, and θ p Mask 1 for membrane and anisotropic etching of a membrane Mask 2 for electroplating a permalloy layer and the layers structure when electroplating Mask 3 for micromirror hinges and the structure Alignment of three masks Fabrication process for magnetic-actuated micromirrors Fabrication process for electrostatic-actuated micromirrors Fabricated mirror arrays with electroplated permalloy films: (a) view from the top and (b) view from another angle

9 3-8. Electrostatic and magnetic actuation methods Initial testing setup for the micromirror arrays Camera positioning for left- and right-eye pictures in POV-Ray Interleaving of left- and right-eye pictures in Matlab Image source preparation: a) color testing b) stereo image testing Optical system adjustment: accurate positioning of the relaying lens and the collecting lens Actual picture of an almost perfect image relay onto the micromirror array: one to one correspondence between the image source and the micromirror array The color test result for single-opening pupil: a) the image source; b) left image and c) right image. The left-right separation is 22 mm The stereo image testing result for single-opening pupil: a) the image source, b) left image, and C) right image with 22mm separation The color testing result for double-opening pupil: pictures taken at L1, L2, R1 and R2 viewing zones. The separation between L1 and R2 is 66 mm The stereo image testing result for double-opening pupil: pictures taken at L1, L2, R1 and R2 viewing zones. The separation between L1 and R2 is 66 mm Example Image source on the color CRT Translate an object according to the clock in POV-Ray Pictures taken from the left (L1) and right (R2) viewing zones, which are 66mm apart Alternatively open and close the double-opening pupil at the relaying lens: a) pupil at +y position is open at one time 2mt 0 +1, and b) pupil at -y position is open at time 2mt Example Image source sequence on the color CRT A pair of commercial 3-D glasses synchronized with a CRT [53] Optical ray tracing for the multiple-view (2 left and 2 right) 3-D display: a) at time 2mt 0 +1, interleaved L1 and L2 views are displayed, pupil at +y position is open, b) at time 2mt 0, interleaved R1 and R2 views are displayed, pupil at -y position is open Design criteria for 8 pairs of viewing zones. 63 4

10 List of Tables Table 4-1 Optical system parameters (units in mm, paraxial approximation). 48 5

11 List of Symbols f x and f y spatial frequency in x- and y- directions ϕ x and ϕ y viewing angles in x- and y- directions λ wavelength D x and D y dimension in x- and y- directions D hgram θ scan r o and r i r' o and r' i S RLMM S MMVZ R m x and m y total data of a hologram scanning range required for left and right viewing zones ray positions at the object plane and the image plane ray directions at the object plane and the image plane distance between the relaying lens and the micromirrors distance between micromirrors and viewing zones radius coordinates of a micromirror α, β, and γ directional angles with respect to x-, y-, and z- axes θ vz θ mmx and θ mmy f r f c S 1 S 2 θ p θ dopen θ e W T L1 and L2 R1 and R2 deflection angle required for left and right viewing zones micromirror deflection angles about x and y axis focal length of the relaying lens focal length of the collecting lens distance from the micromirror array to the collecting lens distance from the relaying lens to the collecting lens physical scanning range scanning angle introduced by double-opening pupil effective scanning angle width of the membrane thickness of the wafer 1 st and 2 nd left viewing zones 1 st and 2 nd right viewing zones 6

12 Chapter 1 Introduction Chapter 1 Introduction 1.1 REVIEW OF 3D DISPLAY SYSTEMS A real-time, full-color, and wide viewing-angle three-dimensional (3D) display has been a goal of display technology for a long time [1-17]. Physicians could use 3D MRI (Magnetic Resonance Imaging) and CT (Computed Axial Tomography) images, architects could model true 3D designs, teachers could illustrate 3D examples, and even consumers could enjoy 3D televisions and games. Additionally, the development of 3D display systems has received substantial efforts in recent years [18-23], as part of intense international competition to develop advanced display systems such as 3-D workstations and virtual reality systems. Unfortunately, none of the 3D technologies, holographic, volumetric, and stereoscopic, could be a general approach for 3D display systems. Nevertheless, each of these three approaches is unique because it solves one or several aspects of the general problem and can be applied to some specified fields. First, holographic technology can provide still, monochrome, and wide-viewing-angle holograms [24]. Second, volumetric displays can provide a full-colored, wide-viewing-angle display for wireframes [20]. Finally, stereoscopic and auto-stereoscopic displays can be a real-time, full-color 3D display for a single user [25, 26]. 7

13 Chapter 1 Introduction Human Factors The human visual system perceives depth through a number of visual cues, which can be categorized into monocular cues, binocular cues and motion parallax [22]. First, monocular cues include accommodation (relaxation or contraction of the eye muscles to focus on an object), texture, shading, occlusion, relative brightness of objects, relative size of objects, relative motion of objects, and hazing (blurring of distant objects). Second, binocular depth cues are convergence (the amount the eyes must rotate toward each other in order to focus on the same point of an object) and stereoscopic disparity (the difference in the two images formed on the retinas of the left and right eyes). Finally, vertical and horizontal motion parallaxes are relative motions of objects due to the observer motion, even if the objects are still. That is to say, a moving observer will obtain relative depth information even the objects are stationary. The relative importance of the above visual cues is summarized by Wickens [22]. In binocular cues, only stereoscopic disparity creates a compelling sense of 3D, although 10% of the population cannot perceive it. Additionally, some cues, such as vertical motion parallax, are not important and often can be eliminated to reduce information content (for saving data rate) [27]. Computer-generated 3-D graphics or real imagery taken by a camera can provide monocular cues. Stereoscopic systems can supply binocular cues. Several ways, for example head tracking and holography, can fulfill the motion parallax requirement Holographic 3D displays The ideas of holographic imaging can be traced back to 1920 s, when several scientists tried to improve x-ray crystallography. After several decades of little success in realizing 8

14 Chapter 1 Introduction those ideas, researchers made some breakthroughs and also took advantage of the invention of the laser in the early 60's. Afterwards, an explosive growth of activity took place, and holography soon found a very large number of scientific applications [28, 29]. As a result, holography has ceased to be a novelty and has become a well-established and valuable technique in art and advertising, high-resolution imaging, information processing, security coding, holographic optical elements, and nondestructive testing and strain analysis. a) b) Figure 1-1. Holographic recording and reconstructing. a) The photographic plate records the interference pattern produced by the light waves scattered from the object and a reference wave. b) The hologram, after processing, is illuminated with the reference wave. Light diffracted by the hologram appears to come form the original object [29]. The unique characteristic of holography is the recording of the complete wavefront, which can be regenerated from the processed hologram merely by illuminating it once again with the reference wave as shown in Figure 1-1. However, this very characteristic leads to 9

15 Chapter 1 Introduction the biggest problem in the development of a real-time holographic display: the tremendous bandwidth and processing requirements. The holographic spatial frequency f x and f y are the reciprocals of the hologram fringe periods: f f x y sin ϕ = λ sin ϕ = λ x y (1 1) where ϕ x and ϕ y are viewing angles in x- and y- directions, respectively, and λ is the wavelength. The sampling rate, by Nyquist criterion, should be twice the maximum data frequency. So the total data of a hologram of dimension D x D y will be: D hgram 4DxDy ( 2 f xdx )(2 f ydy ) = sin ϕ sin (1 2) 2 x ϕ λ = y As an example, a 100mm 100mm monochrome (at λ = 630nm) hologram with a 30 viewing angle, 8 bit grayscale resolution and 30 frames per second rate would require a data rate of bits/s, which is far beyond current capabilities. Many efforts have been taken to lower the bandwidth requirements [30] or make trade-offs [18]. One of the most promising holographic 3D displays, an acousto-optic modulated pseudo-wavefront reconstruction, has been under development at the Media Lab of MIT (Massachusetts Institute of Technology). The up-to-date version (second generation) of the MIT holographic display is shown in Figure 1-2, where a number of holographic sub-elements are segmented together to reach a dimension of 150mm 75mm with a viewing angle of 30. Vertical motion parallax was also dropped to save bandwidth. As a result, the total data rate is at 9.12 Gbit/s [31], which still need further reduction in practice. 10

16 Chapter 1 Introduction Figure 1-2. Simplified drawing of the second generation MIT holographic display: a number of holographic sub-elements are segmented by the multichannel Bragg cells and raster scanners [31]. On the other hand, still and monochrome holograms have already penetrated the medical imaging market. Multiple CT, MRI, or ultrasound scans builds up a composite 3D image, which is exposed in the holographic setup [24]. The resulting hologram can be viewed thorough a special light box. Hospital discharge summary data, in 1993, indicated that ~1.09 million surgical procedures were performed on patients in areas for which these holograms have shown clinical promise [24]. This example shows the potential of holography for monochrome and still 3-D display Volumetric 3D displays In the 60's, Traub and Ketchpel [1] individually demonstrated that translation or rotation motion of a synchronized 2D image source can provide a volumetric 3D display in which the display medium itself is three-dimensional and consists of discrete volume elements referred to as voxels, or volumetric pixels. These ideas have been actively explored throughout the 11

17 Chapter 1 Introduction decades with several commercializing attempts [32]. More recently, real volume media are under development by stacking up fiber-optical pigtails [33] or waveguides [21]. As an example, Figure 1-3 shows the idea of using the two-step upconversion fluorescence in a glass cube, the volume medium. Because voxels can be set to be transparent or photo-luminescent, a 3D wireframe can be drawn in real time [20]. Of course the laser beams must be rapidly scanned to provide motion pictures. a) b) Figure 1-3. A glass cube used as the volumetric display medium. a) Energy level diagram. If an active ion of the glass absorbs energies from laser beams of l 1 and l 2, it will fluorescent in visible light. b) Two scanned, intersecting laser beams are used to address voxels and draw objects [20]. But volumetric displays have two intrinsic drawbacks. First, a voxel cannot appear opaque and occlude voxels behind it. Hence the display is limited to frameworks. Second, the maximum depth of a 3D scene that can be displayed is restricted by the actual physical size of the display medium. The bandwidth requirement for volumetric displays is not as huge as holographic ones, but it is still a technical difficulty. 12

18 Chapter 1 Introduction Stereoscopic and auto-stereoscopic 3-D displays Many of us remember the fun of seeing a stereo movie. By wearing a pair of special glasses (usually consisting of perpendicular polarizers), almost all the actors and objects seem to be suspended in three dimensions. Sometimes you even reached out and tried to touch them. This example shows how important stereoscopic disparity is in human 3-D perception. Tremendous efforts have been made in the areas of theoretical study of providing stereo signals [34], general display add-on stereo products [25], commercial stereo displays [35], and 3-D video transmission and hardware design [23]. Figure 1-4 shows a lenticular system for auto-stereoscopic displays [36], which doesn t require special glasses because the lenticular sheet will refract light to the viewing zones. Lenticular approaches are extensively studied and attempted as commercial products. Figure 1-4. Three-view lenticular display [36]. However, because another layer of optical elements (the lenticular sheet) is placed between the pixels and the viewer, the overall optical performance might be degraded. A theoretical study has shown that using current electronic display devices, it is difficult to match the resolution of a lenticular display to traditional 2-D displays [34]. 13

19 Chapter 1 Introduction 1.2 DIFFRACTIVE PARTIAL-PIXEL 3-D DISPLAYS The IC (Integrated Circuit) Vision project, conducted by a research group at The University of Alabama in Huntsville (UAH) and the University of Cincinnati (UC), implements displays delivering a series of stereoscopic images to a viewing region [26, 37-39]. Since an observer would see a particular single pair of stereo images depending on the head position, both stereoscopic disparity and horizontal motion parallax are provided. The partial pixels architecture is shown in Figure 1-5. Each pixel of the display is divided into 8 pairs of partial pixels, which correspond to the appropriate viewing zones. The partial pixels are diffractive elements, which were implemented as arrays of fixed amplitude Figure D display implemented with diffractive partial pixels. gratings etched in chrome on a quartz substrate. Prototype real-time 3-D devices for monochromatic and color displays were demonstrated. The monochromatic display (designed for 630 nm) had a dimension of about 1.2" 0.8" and 16 views, 8 for the left eye and 8 for 14

20 Chapter 1 Introduction the right. The color device was designed for three prime colors, 440 nm, 550 nm and 630 nm. It had a similar dimension and 4 views. The partial pixel gratings for these prototype devices are shown in Figure 1-6. Both devices produced clearly visible and easily fused 3-D scenes. The color display exhibited vivid color as well. Partial Pixel Grating µm a) 45 µm b) Figure 1-6. Partial pixel gratings for a) 16-view monochrome display and b) 4-view color display. The major challenge of developing the diffractive partial-pixel display is how to make a larger screen display while maintaining color and high resolution. In the partial-pixel prototypes of IC Vision displays the spatial light modulator (SLM) - a liquid crystal display - pixels were spatially multiplexed for eight stereo pairs (16 views), thus the resolution was degraded by a factor of four in each direction. This is a severe restriction given the difficulty of increasing pixel count by 16. As an example, a VGA ( ) display with 16 views will need million partial pixels, far beyond current capabilities. 15

21 Chapter 1 Introduction 1.3 MICROMIRROR ARRAYS AND THEIR APPLICATIONS Micro-Electro-Mechanical-System (MEMS) technology has been developed rapidly from early ideas and demonstrations [40] to major commercial success [41] and futuristic spotlight [42] during last 20 years. Biological and optical MEMS are among the most attractive MEMS directions recently. A broad review of all optical MEMS applications is beyond the scope of this dissertation. So the discussion is concentrated on micromirrors and especially micromirror arrays, which are widely used optical MEMS devices and of special interest to our research project. Micromirrors not only offer common advantages associated with the micro-fabrication, like miniature size and low cost, but also provide other advantages including: 1) High efficiency (reflectivity can be larger than 90%); 2) Wide band (the reflectivity can be almost a constant for a wide range of wavelength); 3) Scalability (micromirrors can be formed into 1-D or 2-D arrays for additional functionality) Digital Micromirror Device (DMD) DMD from Texas Instruments (TI) is possibly the most successful product of optical MEMS so far [41]. As shown in Figure 1-7, these micromirrors, are 16µm 16µm in dimension, and can switch between two positions (±10 ) in about 15µs. When used with a fixed input light source, micromirrors at one position will reflect the light to pass the output pupil, while those at the other position will reflect the light to a stop, preventing it to pass. Hence the on and off states are achieved by positioning the micromirrors. Control of the positions is simply the same as addressing a SRAM (static random access memory), because the micromirrors are integrated with CMOS SRAM circuitry underneath. Micromirror array sizes of SVGA ( ) and SXGA ( ) are currently available. 16

22 Chapter 1 Introduction Figure 1-7. TI DMD structure: binary positioned micromirror integrated with CMOS controlling circuitry [41]. The major advantage of a DMD over other SLM technologies, like an active liquid crystal device (LCD), is the superior brightness. There are two major reasons: 1) The reflectivity of the DMD aluminum surface is much higher that the transmittance of the liquid crystal; 2) DMD has much higher fill factor of about 90% [43]. As denoted by its name, DMD is a digital device, making it perfect as a digital SLM. If grayscale is needed, we can alter the duty cycles of the micromirrors to achieve temporal modulation [41], with the tradeoff of the slower responding time. This is the basic concept behind TI s Digital Light Processing (DLP) technology [44], which is quickly gaining market share of digital projectors, large video walls, digital cinemas, and digital home entertainment systems. Several other companies offer similar micromirror devices. For example, Samsung Electronics has a micromirror structure with a size of 50µm 50µm [45], a deflection angle 17

23 Chapter 1 Introduction of about 9, and a resonance frequency of 25kHz ~ 50kHz. The applications are based on the same principles Flip-up micromirror array Figure 1-8 shows an 8 8 array of flip-up micromirrors driven by scratch-drive actuators [46] for high port-count wavelength-division-multiplexed (WDM) cross connects. By connecting several chips together, higher port count is achievable. However, switch loss becomes impractically high when optical path get too long, - beyond ports. a) b) Figure 1-8. A micromirror cross connect: a) chip structure; b) device architecture This architecture is a promising candidate for the large-scale optical cross connects that are needed in next-generation optical-transport networks. A good angular alignment (0.1 ), careful selection of micromirror size (> 400µm to minimize loss), and improvement of the micromirror flatness (by multiple material-layer structures) are important design criteria for the cross connects. This application is a good example of taking the advantage of wavelength-independence of reflectance. The device can handle WDM signals, with desirable increase of wavelength channels of optical networks management. 18

24 Chapter 1 Introduction dimensional actuated micromirror array Lucent Technologies WaveStar TM LambdaRouter is an electrostatically actuated, 500µm diameter, Si surface micromachined 2-axis tilting micromirror array [47]. Arrays of , , and even larger can be implemented with complexity scaling linearly as a function of port count. The micromirrors are capable of large, continuous, controlled, DC tilt in any direction (about 10 around x- and y- axis) at moderate actuation voltages (less than 170V). The optical path length scales only as square root of N, where N is the number of input ports. But when N increases, alignment tolerance of these optical elements becomes a major problem. This architecture is also an example of continuous 2-dimentional actuation profiles, which could be very helpful for our project. a) b) Figure 1-9. Lucent LamdaRouter: a) one micromirror; b) routing architecture [47]. 1.4 Proposal for MEMS 3-D display systems To overcome the difficulty of the diffractive partial-pixel 3D display, we propose to use a micromirror array to redirect each view by using scanning micromirrors. Because this is a 19

25 Chapter 1 Introduction reflective approach, we can produce 3 prime colors using just one micromirror. By time multiplexing of different views, synchronizing between the image source and the scanning micromirrors, we propose to use one scanning micromirror to redirect light to appropriate viewing zones, eliminating the need for multiple partial pixels. The system is significantly simplified. The other advantage of the MEMS approach is the less complicated requirement for the image source. First, we don t need a large number of pixels as long as they can be refreshed fast enough for time multiplexing. Second, reflectivity is not influenced by the collimation of the incident light. We can use a variety of image sources, including an ordinary CRT (cathode ray tube). As the comparison, we are restricted to LCD with collimated illumination as the image source in the diffractive approach. 1.5 Overview of this dissertation For our 3-D display systems, MEMS and optics were designed and realized in parallel. We begin our conceptual design and detailed optical analysis for our proposed system in Chapter 2. We first derived the actuation requirements of micromirrors for autostereoscopic 3-D display systems, where 2-dimensional actuation profile is required for our initial design. Facing the fabrication difficulty of such MEMS structure, we first introduced a collecting lens to lower the actuation requirements to be a uniform and horizontal deflection. Moreover, we modified the optical system with an interleaved actuation profile and a double-opening pupil, to lower the deflection angle requirements further for micromirrors. In Chapter 3, we discuss our design and fabrication of the first micromirror array for our 3-D system. The idea was quite straightforward: first etching a thin membrane, and then 20

26 Chapter 1 Introduction forming our micromirror structure on it. We realized two designs, one for electrostatic actuation and the other for magnetic actuation. Our initial optical testing realized actuation angles of ±0.8, which are very small. Fortunately, the optical system modifications introduced in Chapter 2 can compensate for the small actuation angle. In Chapter 4 we present the optical testing of our system, with backlit transparencies used as image sources. First, we used interleaved color stripes as our image source, and a clear separation between colored left and right views were observed. Second, we interleaved computer-generated left and right pictures as our image source, and the left and right eye views were again clearly separated. When testing with a single-opening pupil, the left and right viewing zone separation was 22mm, not big enough for average human binocular separation. However, when we tested with a double-opening pupil, the left and right viewing zone separation was 66mm. One can see the left view with his left eye and the right view with his right eye simultaneously. 3-D scenes can be fused in the observer s mind. Chapter 5 shows our prototype system of a two-view (left and right) autostereoscopic 3- D display. We used an ordinary color CRT as our image source, so that we can update the 3- D scenes in real time. Several computer-generated, full-color 3-D animations have been displayed, and viewed successfully by several people. Consequently, we proposed a multipleview (2 left and 2 right) 3-D display. We can use an alternative shutter for the doubleopening pupil, and synchronize the shutter with the CRT, thus four different viewing zones can be provided. As the conclusion, we discuss our results and suggest several future work directions in Chapter 6. We discuss our design and realization of 3-D display systems with a micromirror array, the advantages and the technical difficulties of this approach. We point out follow-up 21

27 Chapter 1 Introduction directions for this project: Larger deflection angle and moving micromirrors, scalability of the display, and some fine-tuning of the prototype system. We also point out related research and development directions, such as 3-D image acquisition, 3-D still and motion picture format, and 3-D input devices. We believe our work is a promising step toward a personal multi-view autostereoscopic 3-D display system. 22

28 Chapter 2 Optical Design for MEMS 3-D display Chapter 2 Optical Design for MEMS 3-D display 2.1 CONCEPTUAL DESIGN Micromirrors for view-scanning / frame sequencing We first proposed to use a micromirror array for a multiple-view autostereoscopic 3-D display. As shown in Figure 2-1, we can use one scanning micromirror to redirect the views sequentially instead of using partial pixels. Thus we eliminate the factor of partial-pixel number in each pixel by time multiplexing, and the factor of prime colors by reflection. The micromirror size should be smaller than the human eye resolution limit (about 280µm at 300mm viewing distance). 16 partial pixels of 40mm 25mm each One scanning micromirror of about 160mm 160mm a) b) Figure 2-1. a) Using diffractive partial pixels to form 16 views simultaneously. b) Using a single scanning micromirror to form 16 views sequentially. 23

29 Chapter 2 Optical Design for MEMS 3-D display Figure 2-2 depicts a conceptual design of the whole system. Two spatial light modulators (SLM) are used. The first one acts as the image source, on which the timesequential views are painted. These views are relayed on the second SLM, a micromirror array, which redirects these views into the appropriate viewing zones. Of course the painting of different views (refreshing of the first SLM) and the redirection of the viewing zones (scanning of the micromirrors) should be tightly synchronized. Frame N Frame N+1 Figure 2-2. General MEMS IC Vision Architecture. Let us consider a SVGA array of 280µm 280µm micromirrors, making up an 8" 6" screen. If we supply 8 pairs of views, the refreshing rate and scanning speed will be 16 times the normal video refreshing rate, Hz = 960 Hz. Given the viewing slit size of 6µm 24

30 Chapter 2 Optical Design for MEMS 3-D display 10µm, the normal human interocular distance of 65 mm, and a viewing distance of 300 mm, the scanning range is θ scan = ± tan 1 ( ) = ± 10.7 (2 1) These requirements for scanning micromirrors are currently feasible. The horizontal size of a viewing slit is designed to be about 6 mm, which will just fill the human pupil size. However the vertical size should be much larger, otherwise it will be difficult for a person to find the views Micromirrors for both picture-generating and view-scanning A micromirror array can generate pictures the same way as a TI DLP does. If micromirrors can be positioned at (n+1) orientations, which can be used as n on positions and 1 off positions, then a micromirror array can generate n views of a picture by varying the duty cycle at every on position. Notice we need to introduce at least one common off position for the picture-generating process. Since a National Television System Committee (NTSC) TV field lasts 16.7 ms (refreshing rate of 60 Hz), each of the primary colors must be displayed in about 5.6 ms. Given that the micromirrors of the DMD have less than a 20µs switching time, and 16 views are provided, a 4-bit-per-color grayscale (2 4 = 16 shades) is possible with a single DMD (16 views 16 shades 20 µs 5 ms). If three DMD chips are used to provide the primary colors, = 2 12 = 4096 possible colors can be generated. These grayscale and color shades might not be enough, and could be improved by other system modifications. 25

31 Chapter 2 Optical Design for MEMS 3-D display Projection of the micromirror array A normal pixel size of a display is around 280µm 280µm. It is possible that actual MEMS design and fabrication won t be optimized at this size. We can compensate this by projecting an array of small micromirror size to form a larger screen, or vice versa. In the following we just discuss magnifications, which are more likely needed. For a single lens projection of magnification factor M, the ray tracing equation is: ri M r = i 1/ f i. e. r = Mr i r = r i o o 0 1/ M o ro r o / f + r / M (2 2) where r o, r i are the ray positions at the object plane and the image plane, r' o, r' i are the ray directions at the object plane and the image plane, respectively. The basic geometry is [ ro, r'o ] Image Plane Object Plane [ ri, r'i ] Figure 2-3. Projection with scanning. shown in Figure 2-3. The projection also introduced a trade-off of weaker brightness. We can see that r i is magnified by M, and r' i is reduced by M and shifted by -r o /f. So we can project a small micromirror array to form a larger screen, and the scanning property at the image plane is proportional to that at the object plane. In other words, we can form a larger micromirror image with a smaller scanning range. As an example, we can enlarge the TI DMD 26

32 Chapter 2 Optical Design for MEMS 3-D display micromirror by 4 (equivalent to 64 µm 64 µm) and get ±5 scanning range. Also we need some way to compensate the -r o /f shift. 2.2 ADVANTAGES OF MEMS APPROACH Simpler optical design As discussed above, reflectivity of micromirror arrays does not depend on wavelength, thus we don't need to use partial pixels to deal with three prime colors individually. Also the different views of a picture are generated by scanning of micromirrors. For every pixel of the display, instead of designing diffractive properties of 3 16 partial pixels we now only need to design the reflective property of one scanning micromirror Smaller diffractive factor The size of partial pixels is about 50µm 50µm. Light coming out of this size aperture will have a large expansion due to diffraction. On the contrary, if the micromirrors are larger than 200µm (of course they should still be smaller that human eye resolution), the diffraction expansion will be much smaller, simplifying our system design Weaker requirement for image source For partial-pixel architecture, the pixel number of the original display must equal to the total partial pixel number (this is spatial multiplexing). This number is as large as 16 multiples of current display technologies, as discussed in Chapter 1. As for the MEMS approach, time multiplexing can be introduced on top of the spatial multiplexing. By using a fast display, 16 viewing zones can be easier to achieve. 27

33 Chapter 2 Optical Design for MEMS 3-D display Also since reflectivity is not influenced by the collimation or coherence of the incident light, we don t have special requirements for the image source. We can use a variety of image sources, including LCD, DMD, and CRT. 2.3 GENERIC OPTICAL DESIGN FOR A MEMS 3-D DISPLAY In this section we use an optical ray-tracing [48-50] program, BeamThree, to design and simulate the 3-D systems. Because the software doesn t provide ray-tracing options for individual micromirrors, we approximate these with a tilting mirror. If the deflection angle is display relaying lens x a) beam splitter y q vz z left viewing zone micromirrors at -q vz b) right viewing zone micromirrors at +q vz Figure 2-4. Initial system: a) left viewing zone and b) right viewing zone. 28

34 Chapter 2 Optical Design for MEMS 3-D display small, the approximation will be good enough for simulation purposes. We begin with a initial design, which is the most straightforward approach from an optical perspective. The MEMS design and fabrication were performed in parallel. While facing the MEMS limitations of our fabrication lab, we modified our optical design to accommodate the actual micromirror array that we could realize, as will be discussed in detail in Chapter Initial system The initial system is exact approach of Figure 2.2, where left- and right-eye views are relayed onto the micromirror array and then redirected to the viewing zones. To do so, the micromirrors must be actuated two-dimensionally (horizontally and vertically) and the actuations are different depending on the micromirror position. In Figure 2-4, a micromirror array is approximated by a titling concaved mirror, which shows the redirection for left and right viewing zones. The concave radius R is determined by the imaging equation S 1 RLMM + S 1 MMVZ 1 = R 2 (2 3) Where S RLMM is the distance between the relaying lens and the micromirrors, and S MMVZ is the distance between micromirrors and viewing zones. Depending on the micromirrors position in (m x, m y ), they should have their surface normal pointed to α = cos β = cos 1 ( m 1 x ( m y / / m m 2 x 2 x + m 2 y + m 2 y + R 2 + R 2 ) ) ± θ vz (2 4) 29

35 Chapter 2 Optical Design for MEMS 3-D display Where cos 2 α + cos 2 β+ cos 2 γ = 1, α, β, and γ are directional angles with respect to X, Y, and Z axes, respectively (as shown in Figure 2-5). θ vz is half of θ scan from Equation (2-1), because the scanning angle is twice the deflection angle of a mirror. X (x, y) a b R Z Y Figure 2-5. Directional angles a and b for a spherical contour. The first terms of Equation (2-4) make up a spherical contour, and θ vz is for scanning of horizontal viewing zones. It is quite simple to convert the directional angles to the rotation angles with respect to x-axis (θ mmx ) and y-axis (θ mmy ), because θ mmx = β π/2 and θ mmy =π/2 α. θ θ mmx mmy = sin 1 = sin ( m 1 y / ( m x / m 2 x m + m 2 x 2 y + m + R 2 y 2 + R ) ± θ 2 ) (2 5) These are the actuation requirements for micromirrors of our initial system. Although this actuation profile is possible, like the Lucent Technologies WaveStar TM LambdaRouter discussed in Chapter 1, it is very complicated for MEMS design, fabrication and control. In fact, it is almost impossible to realize this actuation profile using current fabrication facilities vz 30

36 Chapter 2 Optical Design for MEMS 3-D display at the University of Cincinnati. Fortunately, we can modify our optical system to simplify the actuation requirements First simplification: Uniform and horizontal actuation At first, another collecting lens is introduced, as shown in Figure 2-6, to lower the actuation requirements to be one-dimensional, horizontal. Because the collecting lens will collect light deflected from the micromirrors to form left and right viewing zones, it is functionally equivalent to the spherical contour as shown in Figure 2-5. The collecting lens can be either a positive lens or a concave mirror. Practically, a mirror is preferred because it can be made in larger diameter and it is achromatic. display a) relaying lens collecting lens x y q vz z left viewing zone beam splitter micromirrors at -q vz b) right viewing zone micromirrors at +q vz Figure 2-6. Optical ray-tracing layout for a) left viewing zone, and b) right viewing zone. After introducing the collecting lens, only the uniformly horizontal actuation is required. 31

37 Chapter 2 Optical Design for MEMS 3-D display Figure 2-6 shows the optical ray-tracing simulations of a micromirror array with a concave mirror. We can rewrite the actuation requirements as: θ θ mmx mmy = ± θ = 0 vz = ± 5.3 (2 6) Where θ vz = θ scan / 2. This is a big simplification compared with Equation (2-5) Second simplification: Using interleaved profile for left and right viewing zones The second simplification takes the advantage of the separation between left and right viewing zones as show in Figure 2-1. There is an unoccupied region between the left 8 and right 8 viewing zones. Instead of using one micromirror to scan all these 16 viewing zones, a) Left Eye Viewing Zones L1 L2 L3 L4 L5 L6 L7 L8 Right Eye Viewing Zones R1 R2 R3 R4 R5 R6 R7 R8 b) Side View: q Figure 2-7. a) Left and right viewing zones separation and b) Interleaved micromirror profile for left and right viewing zones. we can use 2 micromirror columns, one for the left viewing zones, and the other for the right ones. The whole micromirror array will be an interleaved actuation profile as shown in Figure

38 Chapter 2 Optical Design for MEMS 3-D display The introduction of interleaved actuation reduces requirement of the driving force one step further. As a similar calculation of Equation (2-1) shows, the actuation requirement of each micromirror is reduced to ±2.3 : θ mmx 8 6 = ± tan 1 ( ) / 2 = ± 2.3 (2 7) Because every single frame of the image source provides two views, this actuation profile also lowers the refreshing rate requirements as discussed in Section For a display with 16 viewing zones, instead of 16 times the standard frame rate, this actuation profile only needs 8 times the standard frame rate. Of course, twice the number of horizontal pixels is required Third simplification: double-opening pupil at relaying lens A ±2.3 horizontal actuation is quite practical for MEMS design and fabrication. However, we can actually lower the requirement further by introducing a double opening at the relaying lens. This simplification is partially motivated by the fact that the micromirror arrays we fabricated to date have small deflection angles, about ±0.8, as we will discuss in detail in Chapter 3. The double-opening pupil approach not only enlarges the viewing zone separation, but also has the ability to provide more viewing zones. As shown in Figure 2-8, for each tilting position of the micromirrors, there are two viewing zones introduced by the double-opening pupil. The viewing zones are also farther separated than for a single-opening pupil. Hence we can provide left-eye and right-eye viewing zones even though the physical deflection angles of micromirrors are smaller than those requirements of Equation (2-6) or (2-7). 33

39 Chapter 2 Optical Design for MEMS 3-D display display relaying lens collecting lens x beam Splitter y q vz z a) micromirrors at -q vz L1 R1 b) micromirrors at +q vz L2 R2 Figure 2-8. Optical system design with the double-opening pupil (inset): a) L1 and R1 viewing zones, b) L2 and R2 viewing zones. Furthermore, the double-opening pupil can be controlled to be alternately on and off, that is to say, left opening is on and right opening is off at one time, then left opening is off and right opening is on. We can actually provide four viewing zones with just two tilting positions of the micromirrors. 34

40 Chapter 2 Optical Design for MEMS 3-D display Because the double openings are actually off the optical axis, the alignment is very strict. The double-opening pupil should be perfectly perpendicular to the optical axis, so that the double openings are symmetric and the light is evenly distributed. Let s make a detailed analysis on the optical system with a double-opening pupil. Because this is the most complex system we have, the analysis will include all features of other systems introduced before. Figure 2-9 shows the top view of Figure 2-8 a). Only two rays are shown to clarify the drawing. collecting lens f c display relaying lens f r beam Splitter S 1 d dopen S 2 micromirrors S' 2 Figure 2-9. Optical system analysis. If the image relay from the display to the micromirror array has a magnification of 1, then the distance between the relaying lens and the micromirror array is 2f r. The collecting lens (f c ) has two functions: a) to form a virtual image of the micromirror array (possible magnification) and b) to form real images of the relaying lens pupil, which are the viewing zones: 35

41 Chapter 2 Optical Design for MEMS 3-D display = S1 S 1 fc = S 2 S2 fc (2 8) where S 1 is the distance from the micromirror array to the collecting lens, and S 2 is the distance from the relaying lens to the collecting lens, as shown in Figure 2-9. The physical scanning range θ p will be the summation of the angles introduced by the double-opening pupil θ dopen and that of the micromirror: θ p = θ dopen + 2θ mmx = 2 tan 1 d dopen 2 f where d dopen is the distance between the two openings of the pupil. The effective scanning angle θ e is given by r / 2 + 2θ mmx (2 9) θ e tan 2 = θ tan 2 M 2 p = θ p tan 2 S 2 S 2 (2 10) where M 2 is the magnification between S' 2 and S 2. If θ p is small, θ p θ p /M 2. Our goal is to make the viewing zone separation VZS θ ( S 1 + S 2) 2 tan (2 11) 2 = e to be the average human interocular distance, 65mm, where (-S' 1 +S' 2 ) is the effective viewing distance. The geometric arrangement of (-S' 1 +S' 2 ), θ e, and θ p is shown in Figure As we will see in chapter 4, we can achieve this goal even if θ mmx is as small as 0.8º. 36

42 Chapter 2 Optical Design for MEMS 3-D display virtual image of micromirrors q e S' 1 collecting lens of f c S 1 q p micromirrors Figure Geometric arrangement of (-S' 1 +S' 2 ), θ e, and θ p Optical design summary It is very obvious that the more modest the actuation requirements for micromirrors, the more complicated the optical system. A practical system design will be a careful balance mainly between the optics and the MEMS, although we might consider other system design issues, like the image source refreshing rate. As an example, if a large deflection angle of ±5.3 is easy to achieve, an optical system like that of Figure 2-6 will be sufficient. 37

43 Chapter 3 Design and Fabrication of Micromirror Arrays Chapter 3 Design and Fabrication of Micromirror Arrays We designed and fabricated, for the first time, micromirror arrays for the multiple-view autostereoscopic 3-D display systems as proposed in Chapter 2. We were limited to bulk micromachining because of our MEMS design and fabrication facilities. Most of the MEMS fabrication was accomplished by Mr. Hyoung J. Cho, a Ph.D. student of Dr. Chong H. Ahn. The cooperation between optics and MEMS people turned out to be very beneficial to both sides. 3.1 MASK DESIGN The fabrication idea was quite simple: first make a membrane, and then create <100> W 13 mm <111> Wmask 57.47º T Figure 3-1. Mask 1 for membrane and anisotropic etching of a membrane. 38

44 Chapter 3 Design and Fabrication of Micromirror Arrays micromirror structures on the membrane. We made three masks as described below Mask 1 for the membrane To make a membrane, we used potassium hydroxide (KOH) as a wet anisotropic etchant for a <100> silicon substrate. Because the etched sidewall of <111> forms an angle of with respect to the <100> surface, the mask square window should have dimension of W mask 2 T = W + = W + 2T (3 1) tan ( ) Where W is the width of the membrane, T is the thickness of the wafer. For our mask, W = 12mm and T 0.26mm, we chose W mask = 13 mm to ensure the membrane size Mask 2 for permalloy layer The second mask was for electroplating a array of 400µm 400µm permalloy (Ni/Fe alloy) squares. The permalloy layer can be magnetized by an external magnetic field to produce magnetic actuation. Before electroplating, another metal layer, Ti/Cu, was Legends: : Si : SiO 2 : Ti/Cu : Ni/Fe alloy Figure 3-2. Mask 2 for electroplating a permalloy layer and the layers structure when electroplating. 39

45 Chapter 3 Design and Fabrication of Micromirror Arrays evaporated to form an electrode for the electroplating process Mask 3 for micromirror hinges The third mask was for making micromirror hinges and forming a array of 480µm 480µm micromirrors. The center-to-center distance of two adjacent micromirrors was 570µm. As shown in Figure 3-3, the mask design already used the idea of interleaved profile for left and right viewing zones of Sec Because the micromirrors were actuated downwards, the odd column and even column would be rotated in opposite directions. Figure 3-3. Mask 3 for micromirror hinges and the structure Alignment of three masks All three masks should be aligned carefully during every fabrication step, with the help of alignment marks. As shown in Figure 3-4, the alignment ensures that every micromirror has a permalloy block. For a uniform external field, every micromirror will receive an equal actuation force. 40

46 Chapter 3 Design and Fabrication of Micromirror Arrays Figure 3-4. Alignment of three masks. 3.2 FABRICATION PROCESS Fabrication process for magnetic-actuated micromirrors The fabrication steps for magnetic-actuated micromirrors are shown in Figure 3-5. First, the silicon wafer was oxidized, patterned (using mask 1) and etched to make a 5 to 20µm silicon membrane. Second, the wafer was reoxidized, and then a metal layer of Ti(200Å)/Cu(2000Å) was deposited as a electroplating seed layer. Photoresist was then spun on the wafer and patterned (using mask 2) to build the electroplating mold for the permalloy array. The permalloy array was electroplated up to the thickness of 8µm. Third, the micromirrors were patterned (using mask 3) and released by reactive ion etching (RIE). Finally, the micromirrors were coated with either gold or aluminum to yield high reflectivity. 41

47 Chapter 3 Design and Fabrication of Micromirror Arrays Oxidation Ti/Cu deposition Oxide etching Electroplating of Ni/Fe Si etching and oxidation removal Ti/Cu etching and RIE Re-oxidation Oxide removal Figure 3-5. Fabrication process for magnetic-actuated micromirrors Fabrication process for electrostatic-actuated micromirrors The fabrication steps for electrostatic-actuated micromirrors are shown in Figure 3-6. Every step is the same as that of magnetic-actuated micromirrors, but we skipped the electroplating of permalloy. We also keep the metal layer for applying voltage. Oxidation Oxide etching Metal deposition Si etching and oxide removal RIE and oxide removal Re-oxidation Figure 3-6. Fabrication process for electrostatic-actuated micromirrors. 42

48 Chapter 3 Design and Fabrication of Micromirror Arrays 3.3 FINISHED MICROMIRROR ARRAYS AND ACTUATION METHODS Figure 3-7 shows SEM (scanning electron microscope) pictures of a finished micromirror array with permalloy layer. The actual dimensions are: 403.7µm 403.7µm permalloys, 463.1µm 463.1µm micromirrors with center-to-center distance of 568.5µm and thickness about 20µm. Electrostatic-actuated micromirror arrays have the same structure without the permalloy layer. We made several micromirror arrays, with slightly different dimensions. However, Figure 3-7 shows the only two SEM pictures we had taken because the SEM process itself turned out to destroy quite a few micromirrors. (a) Figure 3-7. Fabricated mirror arrays with electroplated permalloy films: (a) view from the top and (b) view from another angle. We had two actuation methods, magnetic and electrostatic, as shown in Figure 3-8. We used a strong permanent magnet, NdFeB, for the magnetic field. We also applied up to 180V for the electrostatic field. (b) 43

49 Chapter 3 Design and Fabrication of Micromirror Arrays Movement V Movement Magnetic field Electrostatic field Figure 3-8. Electrostatic and magnetic actuation methods. 3.4 INITIAL TESTING AND DISCUSSION The initial testing setup for actuation angles is shown in Figure 3-9, where we used a He-Ne laser as an input and measure the deflected distance at a screen. Although the movements of micromirrors are quite uniform across the array, the actuation angles, denoted by θ mmx in Chapter 2, are ±0.79 for magnetic-actuation, and ±0.84 for electrostaticactuation. Record Screen Mirror He-Ne Laser Beam Splitter Micromirror array Figure 3-9. Initial testing setup for the micromirror arrays. 44

50 Chapter 3 Design and Fabrication of Micromirror Arrays As mentioned in Chapter 2, to work with small deflection angles, we opted for a doubleopening pupil. Had we a micromirror array with actuation angle of ±2.3 as calculated in Equation 2-7, we could have avoided a complicated optical system as shown in Figure 2-8. The major reason for the inflexibility of micromirrors is that they are too thick. We tried to make a membrane down to 5 to 10µm but did not succeed. A modified fabrication process, like a carefully controlled etch-stop, or even surface micro-machining, will be necessary to achieve a larger actuation angle. Unfortunately, time and budget constraints made the author unable to pursue an improved fabrication process for his thesis research. 45

51 Chapter 4 Testing of the Optical System Chapter 4 Testing of the Optical System Using backlit transparencies as the image source, we first tested the optical system with a collecting lens (Figure 2-6), and then tested with a double-opening pupil (Figure 2-8). The reason for using transparencies is that they can be easily created (from a color inkjet printer) and exchanged (in and out of a slot). 4.1 IMAGE SOURCES PREPARATION For an image relay from the image source to the micromirror array with a magnification of 1, and odd (even) column of the micromirrors pointing to the right (left) viewing zone, the image source should be an interleaving between left and right views, with pixel size equal to the micromirror separation of 568.5µm 568.5µm. There are three steps in the preparation: 1) generating stereo pictures, 2) interleaving left and right pictures, and 3) printing out. These steps are discussed in details as following. First, a 3-D ray-tracing software, POV-Ray TM [51], was used to generate stereo pictures. While the full program is listed in Appendix A, Figure 4-1 shows the code for setting a //#declare camerax = -32; //left-eye image #declare camerax = +32; //right-eye image #declare cameray = 108; #declare cameraz = -280; Figure 4-1. Camera positioning for left- and right-eye pictures in POV-Ray. 46

52 Chapter 4 Testing of the Optical System camera for left- and right-eye positions. One can easily switch between these two positions by commenting out the appropriate line. Second, Matlab [52] was used to interleave left and right pictures. The full program is also listed in Appendix A, and Figure 4-2 shows the simplified code for the interleaving: odd columns for right-eye views and even columns for left-eye views. Several programs in POV- Ray and Matlab are listed in detail in Appendix A. for ix = 1:1:inputImageInfo.Width, for iy = 1:1:inputImageInfo.Height, interleavedimage(iy,ix*2-1,:) = rightimage(iy,ix,:); interleavedimage(iy,ix*2,:) = leftimage(iy,ix,:); end end Figure 4-2. Interleaving of left- and right-eye pictures in Matlab. Finally, the interleaved pictures were printed out with resolution of 45 dpi (dots per inch), which is the reciprocal of the micromirror size of 568.5µm. Figure 4-3 shows two interleaved pictures for color testing and stereo testing. Magenta and cyan were used because they are prime colors of color printers. Notice Figure 4-1 is just for illustration, a real image source should be flipped left side right and upside down because Left Eye View Right Eye View a) Interleaved image source b) Figure 4-3. Image source preparation: a) color testing b) stereo image testing. 47

53 Chapter 4 Testing of the Optical System we have a magnification factor of -1. We used an ordinary halogen lamp bulb for backlighting the transparencies. 4.2 OPTICAL SYSTEM SETUP AND ADJUSTMENTS All the parameters for our optical system are listed in Table 4-1. Table 4-1 Optical system parameters (units in mm, paraxial approximation). Sub function Lens Object distance Image distance Image relay Relaying lens f r = 65 2f r = 130 2f r = 130 Virtual image of micromirror array Collecting lens f c = 140 S 1 = 59 S' 1 = -102 Viewing zones Collecting lens f c = 140 S 2 = 2f r +S 1 = 189 S' 2 = 540 Substituting these numbers, with ±0.8 actuation angles (θ mmx ) of the micromirrors, into Equations (2-9) ~ (2-11) we obtain the viewing zone separation (VZS) = 21.8mm for single opening pupil (corresponding to Figure 2-6), and for a 15mm-apart double-opening pupil, the separation between L1 and R2 viewing zones as shown in Figure 2-8 is 64.7mm, which is collecting lens f c transparency relaying lens f r beam Splitter S 1 2f r 2f r micromirrors to viewing zones Figure 4-4. Optical system adjustment: accurate positioning of the relaying lens and the collecting lens. 48

54 Chapter 4 Testing of the Optical System close to the average human eye separation of 65mm. The adjustment of the optical system is quite subtle, especially for the double-opening pupil. First, we have to get a good image relay onto the micromirror array. This means accurate positioning of the image source (a transparency), relaying lens, and the micromirror array, as shown in Figure 4-4. One actual picture of an almost perfect image relay is shown in Figure 4-5, where every pixel of the image source is mapped to a micromirror. This ensures that left- and right-eye pictures will be redirected to the left and right viewing zones. Figure 4-5. Actual picture of an almost perfect image relay onto the micromirror array: one to one correspondence between the image source and the micromirror array. Second, the collecting lens should be carefully placed. This ensures the calculated viewing zone separation. 4.3 TESTING WITH SINGLE-OPENING PUPIL The color testing The first optical testing was for the separation of the left and right viewing zones with a single-opening pupil at the relaying lens. Interleaved cyan and magenta color stripes as 49

55 Chapter 4 Testing of the Optical System shown in Figure 4-3a) were used as the image source, and the system was set up in such a way that the magenta stripes be relayed on the micromirrors for the right viewing zones (odd columns), and the cyan stripes for the left (even columns). We verified those optical raytracing simulations experimentally: If the observer moves the eyes, or a camera, from left to right positions, a clear separation between cyan and magenta colors are observed as shown in Figure 4-6 b) and c), which are actual photographs. Given the geometry, individual micromirrors could not be distinguished. In this case of single-opening pupil, the separation between left and right viewing zones is 22 mm, which is less than our goal of 65 mm. a) b) c) Figure 4-6. The color test result for single-opening pupil: a) the image source; b) left image and c) right image. The left-right separation is 22 mm The stereo image testing The second optical test is for the stereo image. We used the interleaved stereo pictures as shown in Figure 4-7a). Those color stripes wrapping around the stereo image are used as references to ensure the right image is onto odd columns of the micromirrors and left image is onto the even columns. The left and right eye views were observed when we moved the eye or camera from left to right viewing zones. Figure 4-7 b) and c) shows the actual pictures taken by a camera. The stereoscopic disparity between left and right eye views is perceivable. 50

56 Chapter 4 Testing of the Optical System a) Interleaved image source Figure 4-7. The stereo image testing result for single-opening pupil: a) the image source, b) left image, and C) right image with 22mm separation. Please keep in mind that with a larger deflection angle, ±2.3 as calculated in Equation (2-7), our goal of 65mm viewing zone separation could have been achieved just using a single opening pupil. Even with the 22mm separation, these results proved the concept of using micromirrors to redirecting light for all colors, and were important intermediate steps. b) c) 4.4 TESTING WITH DOUBLE-OPENING PUPIL As mentioned in Chapter 2, we trade off a straightforward optical system against simplified actuation requirements for micromirrors. An optical system with the double opening pupil requires much more accurate adjustment. Because it has two off-axis pupils, every element should be precisely perpendicular to the optical axis Color testing The image source setup is the same as Sec We used interleaved cyan and magenta color stripes as shown in Figure 4-3a) as our image source, and set up the system so that the magenta stripes be relayed on the micromirrors for the right viewing zones (odd columns), and the cyan stripes for the left (even columns). 51

57 Chapter 4 Testing of the Optical System We also verified those optical ray-tracing simulations for the double-opening pupil experimentally: If we move our eyes or a camera from left to right, we can observe clearly separated cyan and magenta colors in four positions, which are corresponding to the viewing zones labeled as L1, L2, R1, and R2 in Figure 2-8. Figure 4-5 shows real pictures taken at these four positions. In this case picture of L1 is the same as R1, and L2 is the same as R2. The separation between L1 and R2 is 66mm, right on target for average human eyes separation. Because of this separation, if the observer positioned the head correctly (about 540mm from the collecting lens), he can see a cyan picture from his left eye and a magenta picture from his right eye simultaneously. This is exactly what we designed for an autostereoscopic 3-D display. L1 L2 R1 R2 Figure 4-8. The color testing result for double-opening pupil: pictures taken at L1, L2, R1 and R2 viewing zones. The separation between L1 and R2 is 66 mm The stereoscopic image testing The image source setup is the same as Sec We used the interleaved stereo pictures as shown in Figure 4-7a) with the reference color stripes, and set up the system so that the right image is onto odd columns of the micromirrors and left image is onto the even columns. 52

58 Chapter 4 Testing of the Optical System Four viewing zones, labeled as L1, L2, R1, and R2, were observed when we moved our eye or camera from left to right. Figure 4-9 shows real pictures taken at these four positions. Again the separation between L1 and R2 is 66mm, so that one can see the left image from the left eye and the right image from the right eye simultaneously. Several people have seen these images and successfully fused the images. This is the first demonstration of an autostereoscopic 3-D display system using a micromirror array to direct light to viewing zones. L1 L2 R1 R2 Figure 4-9. The stereo image testing result for double-opening pupil: pictures taken at L1, L2, R1 and R2 viewing zones. The separation between L1 and R2 is 66 mm. In summary, we used backlit transparencies as the image source, and verified the optical system simulation for a collecting lens (Figure 2-6), and then with a double-opening pupil (Figure 2-8). The optical testing proved the advantages of the micromirrors, which are: 1) They are reflective and hence work for all colors; 2) A variety of displays can be used as image sources. All image sources are in color, thus demonstrating the first advantage. The use of transparencies exemplified the second advantage, because the backlight is an ordinary halogen lamp bulb. Although the system is for still pictures only, stereopsis was perceivable by several observers. 53

59 Chapter 4 Testing of the Optical System It should be emphasized again that a practical system design should be a careful balance between the optics and the MEMS. Compared with the system with a single-opening pupil, one with a double-opening pupil is not only harder to adjust, but also tends to yield inferior quality pictures with lower illumination at the viewing zones, as we can compare Figure 4-8 and 4-9 with Figure 4-6 and 4-7. The advantage of double-opening pupil is that it not only enlarges the viewing zone separation, but also has the ability to provide more viewing zones. We believe the actuation requirement for using single-opening pupil (±2.3 ) as calculated in Equation (2-7), is achievable given the fact that several commercial products showed even larger deflection angles as discussed in Chapter 1. Unfortunately, the fabricated micromirrors are currently limited to ±0.8 deflection angles, and a double-opening pupil is necessary to get 65mm viewing zone separation. 54

60 Chapter 5 Prototype 3-D Display Systems Chapter 5 Prototype 3-D Display Systems To set up a prototype 3-D display system using a micromirror array, using an ordinary color cathode ray tube (CRT) as the image source so that we can display 3-D animations. Several computer-generated, full-color 3-D animations have been displayed, and viewed successfully by several people. 5.1 TWO-VIEW AUTOSTEREOSCOPIC 3-D DISPLAY System setup The system setup is the same as the testing system of the double-opening pupil (Sec. 4.4), except that we replaced the backlit transparency with a color CRT. This CRT has a 0.28mm dot pitch, with red, green, and blue phosphors. The image source preparation is also similar to that of the testing system. For an image relay from the image source to the micromirror array with a magnification of 1, and odd Left Eye View Interleaved image source Right Eye View Figure 5-1. Example Image source on the color CRT. 55

61 Chapter 5 Prototype 3-D Display Systems (even) column of the micromirrors pointing to the right (left) viewing zone, the image source should be an interleaving between left and right views. To match the micromirror separation of 568.5µm 568.5µm, we simply used a block of 2 2 CRT pixels to make up one image source pixel. An example image source is shown in Figure 5-1, in which the picture is flipped for 1 magnification, and wrapped with red and green stripes for reference. The system adjustment is the same as discussed for the testing system in Chapter 4. The only difference is that a CRT is usually not as bright as a backlit transparency, making the adjustment more difficult. Of course the big advantage of CRT is that changing images is almost effortless: just open another file at the same screen position. We also created several 3-D animations by POV-Ray. The full program is listed in Appendix A, and Figure 5-2 shows the code example for translating an object backwards according to the clock. //animation: translate backwards translate <0,0,dot_radius*15*clock> Figure 5-2. Translate an object according to the clock in POV-Ray Left and right viewing zones The observer can see the left image with his left eye and the right image with his right eye. Because in this case L1 is the same as R1, and L2 is the same as R2, only photographs taken at L1 and R2 are shown in Figure 5-3. Notice the white signals are actually combinations of red, green and blue phosphors, so the pictures again proved that our system worked for all colors. 56

62 Chapter 5 Prototype 3-D Display Systems L1 R2 Figure 5-3. Pictures taken from the left (L1) and right (R2) viewing zones, which are 66mm apart. We can now call our system a prototype 3-D display, because it is a full-color and realtime system. We have displayed several 3-D still pictures and animations on our prototype system, which are viewed successfully by members of our group. 5.2 MULTIPLE-VIEW AUTOSTEREOSCIPIC 3-D DISPLAY SYSTEM We already have four viewing zones, L1, L2, R1 and R2, only that what we see form L1 is the same as R1, and L2 is the same as R2. We proposed to use an alternative shutter for the y x z relaying lens Figure 5-4. Alternatively open and close the double-opening pupil at the relaying lens: a) pupil at +y position is open at one time 2mt 0 +1, and b) pupil at y position is open at time 2mt 0. 57

63 Chapter 5 Prototype 3-D Display Systems double-opening pupil, and with the shutter synchronized with the CRT, we can actually provide four different viewing zones System modification The system has two modifications to the two-view prototype (Sec. 5.1). The 1 st modification is an alternative shutter for the double-opening pupil, as shown in Figure 5-4. For the relaying lens, the pupil at +y position is opened and the other is closed at time 2mt 0 +1, and then the pupil at -y position is opened and the other is closed at time 2mt 0, where t 0 is the open duration of one pupil, and m = 0, 1, 2,. L1 View Interleaved image source for time 2mt 0 +1 L2 View R1 View Interleaved image source for time 2mt 0 R2 View Figure 5-5. Example Image source sequence on the color CRT. The 2 nd modification is a dynamically changing image source synchronized with the alternative shutter at the double-opening pupil, as shown in Figure 5-5. At time 2mt 0 +1, an interleaved image of L1 and L2 view is displayed, and then interleaved image of R1 and R2 view is displayed at time 2mt 0. At time 2mt 0 +1, view L1 will be relayed onto even column of the micromirrors and redirected to viewing zone L1, view L2 will be relayed onto odd column of the micromirrors and redirected to viewing zone L2. Similar relaying and redirecting for R1 and R2 will be performed at time 2mt 0. If t 0 is small enough so that L1, L2, R1 and R2 views seem to appear simultaneously, we can provide multi-view autostereoscopic 3-D scenes. 58

64 Chapter 5 Prototype 3-D Display Systems Fortunately, such a device for synchronizing the shutter with the CRT monitor is commercially available with a very affordable price tag. The device is for a similar purpose: stereoscopic 3-D glasses for a personal computer with a CRT. The idea is quite straightforward: if the CRT monitor is synchronized to a pair of liquid crystal glasses, which can switch light on and off by polarization, then the system can provide left view to the left eye at one time, and right view to the right eye at the other time. If the system refreshes fast enough, the observer will see the left and right views as if they were simultaneously displayed, thus fuse a stereo scene. Because the device depends on polarization, a LCD can t be used as the image source. The system installation is shown in Figure 5-6, which is taken from a manual by VR Standard Corporation [53]. The controller will detect the video signal from the video card, and send synchronization signals to the liquid crystal glasses. Figure 5-6. A pair of commercial 3-D glasses synchronized with a CRT [53]. 59

65 Chapter 5 Prototype 3-D Display Systems As we can see, this device does exactly what we wanted for the alternative shutter. We just need to assemble the liquid crystal glasses onto the double-opening pupil at the relaying lens Simulation CRT relaying lens collecting lens x y q z micromirrors at q a) L1 L2 micromirrors at q R1 R2 b) Figure 5-7. Optical ray tracing for the multiple-view (2 left and 2 right) 3-D display: a) at time 2mt 0 +1, interleaved L1 and L2 views are displayed, pupil at +y position is open, b) at time 2mt 0, interleaved R1 and R2 views are displayed, pupil at -y position is open. 60