BINOCULAR DISPARITY AND DEPTH CUE OF LUMINANCE CONTRAST. NAN-CHING TAI National Taipei University of Technology, Taipei, Taiwan

Transcription

1 N. Gu, S. Watanabe, H. Erhan, M. Hank Haeusler, W. Huang, R. Sosa (eds.), Rethinking Comprehensive Design: Speculative Counterculture, Proceedings of the 19th International Conference on Computer- Aided Architectural Design Research in Asia CAADRIA 2014, , The Association for Computer-Aided Architectural Design Research in Asia (CAADRIA), Hong Kong BINOCULAR DISPARITY AND DEPTH CUE OF LUMINANCE CONTRAST Investigation of Perceptual Influence of Binocular Disparity on Depth Effect of Luminance Contrast through Stereo Display NAN-CHING TAI National Taipei University of Technology, Taipei, Taiwan 1. Introduction Abstract. Luminance contrast has been identified as an effective depth cue through perceptual studies using digital images generated by the integrated technologies of physically based lighting simulation and perceptually based tone mapping. However, the prior established framework utilizes a single camera viewpoint, failing to address the binocular vision of the human visual system. In this study, the computational framework is extended to incorporate 3-dimensional (3D) stereo display technology. Psychophysical experiments were conducted to investigate the depth effect of luminance contrast on the experimental scenes presented on conventional and stereo displays. The objective of this study was twofold: first, to investigate the effect of luminance contrast on depth perception, considering binocular vision; second, to further advance the visual realism of the computergenerated environment to reflect the perceptual reality of static pictorial and binocular disparity cues. Keywords. High dynamic range imagery; luminance contrast; binocular disparity; stereo display; depth perception. Contrast has proven to be an effective cue for creating illusory depth on a planar surface (O Shea et al., 1994). To utilize luminance contrast as a design parameter to enrich the spatial experience, Tai (2012) developed a computer-generated pictorial environment that can reflect perceptual reality to investigate and envision the effect of luminance contrast in three-

2 638 N. TAI dimensional space. The effect of light, architectural configuration, and depth perception of a visual target in an architectural scene was investigated, and luminance contrast was identified as an effective depth cue. A visual target that has a higher contrast against a foreground than against a background will appear to be deeper in space than its actual location (Tai and Inanici, 2012; Tai, 2013). Experimental scenes in those studies were generated by the physically based lighting simulation program RADIANCE in High Dynamic Range (HDR) image format (Ward and Shakespeare, 1998). The accuracy of the images generated by RADIANCE has been validated (Mardaljevic, 2001; Ruppertsberg and Bloj, 2006). To display an HDR scene on a common display with a limited range, it can be tone-mapped into a low-dynamic range format such as JPG. The Photographic tone-mapping operator developed by Reinhard et al (2002) is one of the best performers in several perceptual aspects in many perceptual studies (Kuang et al., 2007; Cadík et al., 2008). The prior studies have established a computational framework to generate a digital pictorial environment that can reflect the perceptual reality in terms of how light is distributed in an architectural scene, and how it is perceived by the human visual system. The final image from the previously established computational framework is output by a single camera viewpoint. Therefore, the visual realism it offers can at best match monocular vision, failing to address the binocular vision of the human visual system. In this study, a stereo display technology is incorporated into the established framework. Perceptual studies on the depth effect of luminance contrast were conducted in a computer-generated environment. The objective is to investigate the effect of luminance contrast on depth perception considering binocular vision and to further advance the visual realism of the computer-generated environment to reflect the perceptual reality of static pictorial and binocular disparity cues. 2. Binocular Disparity and Depth Perception Depth perception relies on depth cues. Depth cues can be generally categorized as binocular cues, including convergence and binocular disparity, and monocular cues, including kinetic and pictorial cues (Palmer, 1999; Solso, 2003). Convergence, a physiological cue, refers to the angle of convergence of the feedback from the two eyes that reflect the distance of the object on which they are focused. A kinetic cue is dynamic visual information resulting from the relative change of object location in a spatial layout due to the motion of the observer. It can inform us of both the static and dynamic spatial relationships of a larger-scale context. Pictorial cues, on the other hand, refer to the collective depth cues such as occlusion, relative size, linear per-

3 BINOCULAR DISPARITY AND DEPTH CUE 639 spective, and aerial perspective that are the visual information that can be observed from a real scene and can to be applied to create illusory depth in a picture (Wanger et al., 1992; Palmer, 1999). Pictorial cues are thus essential for creating a perceptually realistic, pictorial environment on planar media. In binocular disparity, it is the difference of the two retinal images that provides the visual information of the spatial layout and creates the stereo visual experience (Wanger et al., 1992; Palmer, 1999). Binocular cues can be considered to be two sets of slightly different pictorial cues. Visual perception results from how the visual system responds to the light reflected from a three-dimensional environment. Therefore, the previously established computational framework that integrates the physically based lighting simulation and perceptually based tone mapping can provide an image that encompass the pictorial cues to match the real scene. To incorporate binocular disparity, the framework was expanded to generate a set of two images, one for the left eye and one for the right eye, and then display these specifically to each eye to create the stereo visual experience. Techniques and technologies for displaying stereo images on planar media have been developed for some time. In general, they can be categorized as stereoscopic and autostereoscopic displaying technologies. Stereoscopic display technology requires viewers to wear special glasses; the source image is distinguished by the glasses through different techniques such as shutters, circular polarization, or simply filtered color. Autostereoscopic displays rely on the display device to project separate images specifically to the left and right eye. They often utilize sensors to detect the viewer s position and use various techniques such as lenticular lenses or parallax barriers (Lueder, 2012). In this study, both methods are employed to create the stereo visual representation of the experiment scenes. Anaglyph 3D images are composed of two different color-filtered images and can be delivered to each eye using Anaglyph 3D glasses. For autostereoscopic display, a TOSHIBA Satellite P850 laptop was used. As the stereo viewing mode of the TOSHIBA Satellite P850 can be toggled on and off to display the JPEG Stereoscopic (JPS, extension.jps) images and Anaglyph 3D images in JPG format, it was used to display the experiment scenes for this study. 3. Experiments This study adopts a previously established experiment design (Tai, 2013). A hallway space is composed of four 6 m 6 m 4 m modules. At the center of the 6 m 6 m space, the ceiling has a 2 m 2 m skylight. Each skylight can be open, half open, or closed to control the luminance distribution of the interior. A camera (M) is placed at one end of the hallway, located 1.5 m

4 640 N. TAI above the ground, focusing on the center of the visual target. The visual target is a red sphere with a radius of 30 cm, floating 1.6 m above the ground. The initial location of the visual target is 15 m from the viewpoint. Two more cameras (L) and (R), each of which is shifted 3 cm left and right from the first camera (M), are set up to create the stereo image. The skylights are controlled in two different manners, as illustrated in Figure 1, to create two different luminance distributions for the experimental scenes. In the F=B, the skylights are all opened. The luminance contrast of the visual target against the foreground is thus equal to the luminance contrast against the background. In the F>B, the skylights are half open, open, closed, and half open, respectively, to cause the luminance contrast of the visual target against the foreground to be greater than that against the background. Figure 1. Configurations of skylights to create two different luminance distributions for experimental scenes. Experimental scenes were rendered using RADIANCE, with all parameters constant. The HDR scenes were further tone-mapped by Photographic tone-mapping operator to generate the images in JPG format. The JPG format image output from camera (M) was used as the experimental scene set of a single camera. The JPG format images output by cameras (L) and (R) were processed in an image-editing program to produce an experimental scene set of Anaglyph 3D and JPG Stereoscopic (JPS).

5 BINOCULAR DISPARITY AND DEPTH CUE EXPERIMENT DESIGN The Method of Constant Stimuli was employed to measure the perceived distance of the visual target in the experimental scenes. Each of the two test scenes (visual targets located at 15 m under F=B and F>B lighting s) was paired with one of the seven comparison scenes (visual targets located at seven different locations ranging from m under F=B lighting ) to present to the subjects. The subjects were required to judge what visual target appeared to be closer and verbally report this to the researcher. Each combination of test scene and comparison scene was presented ten times to the subject. That is, each subject was required to make a perceptual judgment of the 15 m visual target, in F>B and F=B s, against the same visual target located at 12, 13, 14, 15, 16, 17, and 18 m in the F=B, respectively, ten times in a random order. The procedure was repeated four times. In the first of the four tests, the subjects used one eye to view the experimental scenes of the single camera set (Monocular 2D); in the second, the subjects used both eyes to view the experimental scenes of the same single camera set (Binocular 2D); in the third trial, the subjects used both eyes to view the Anaglyph 3D set (Anaglyph 3D); and finally, the subjects used both eyes to view the JPG Stereoscopic set (JPS 3D). Ten subjects participated in the experiment. Subjects were 20 to 41 years old with normal or corrected to normal vision. Experiments were performed in a research lab that used electric lighting only to ensure a stable lighting environment from trial to trial. Experiment scenes were presented on the TOSHIBA Satellite P850 laptop that was capable of displaying all the sets of the experiment scenes RESULTS The Method of Constant Stimuli requires subjects to make only binary judgments, and allow more intuitive responses (Gescheider, 1984). Experiment results were analyzed using a Probit analysis model (Finney, 1971). Figures 2 illustrate the Probit analysis results for the same experiments performed with four different experiment scene sets. In the Probit analysis, the x-axis represents the actual locations of the visual target in the comparison scenes, the y-axis represents the probability that the subjects reported that the test target was perceived to be closer. The intersection point of the 0.5 line and the Probit analysis curve is the Point of Subjective Equality (PSE), representing when the test and comparison targets are perceived to be equal in depth. Thus, the PSE can be considered as the measured perceived distance of the test target under different luminance contrast s. In Fig-

6 642 N. TAI ures 2, A, B, C, and D represent the PSEs for the F=B, and A, B, C, and D represent the PSEs for the F>B, respectively, for experiments performed with the scene sets of Monocular 2D, Binocular 2D, Anaglyph 3D, and JPS 3D. Figure 2. (a) Probit analysis for experiment results performed with experiment scene set of Monocular 2D, A is PSE for F=B, A is PSE for F>B ; (b) Probit analysis for experiment results performed with experiment scene set of Binocular 2D, B is PSE for F=B, B is PSE for F>B ; (c) Probit analysis for experiment results performed with experiment scene set of Anaglyph 3D, C is PSE for F=B, C is PSE for F>B ; (d) Probit analysis for experiment results performed with experiment scene set of JPS 3D, D is PSE for F=B, D is PSE for F>B. 4. Discussion Figure 3(a) illustrates the PSEs for the F=B for four different sets of experiment scenes. The luminance distribution of the test scene is identical to the comparison scenes, thus the measured perceived distances of the

7 BINOCULAR DISPARITY AND DEPTH CUE 643 visual targets were all measured close to the actual location of 15 m, specifically, ± 0.077, ± 0.066, ± 0.069, and ± m, respectively, for the experiment scene sets Monocular 2D, Binocular 2D, Anaglyph 3D, and JPS 3D. Conversely, in the F>B as illustrated in Figure 3(b), when the luminance contrast of the test target against the foreground is greater than it is against the background, the measured perceived distances of the test visual targets all increased, specifically ± 0.081, ± 0.072, ± 0.069, and ± m, respectively, for the experiment scenes sets Monocular 2D, Binocular 2D, Anaglyph 3D, and JPS 3D. Figure 3. (a) PSEs of experiment results performed with four experiment scene sets in F=B. A is for Monocular 2D, B is for Binocular 2D, C is for Anaglyph 3D, and D is for JPS 3D set; (b) PSEs of experiment results performed with four experiment scene sets in F>B. A is for Monocular 2D, B is for Binocular 2D, C is for Anaglyph 3D, and D is for JPS 3D set. The primary question asked in this study is whether binocular disparity would affect the previously established results of luminance contrast as an effective depth cue. The experiment design adopted from the previously established studies, that is, the experiment performed with the Binocular 2D set, is considered as an identical experiment to one of the s in the previous study (Tai, 2013). Table 1 compares the experiment results. In both experiments, the measured perceived distance of the visual target in the F>B increased, 8.17% and 9.31%, against its measured perceived distance in the F=B. Therefore, this study validates the effect of luminance contrast in increasing the perceived distance of a visual target using a perceptually realistic, pictorial environment without the consideration of binocular disparity.

8 644 N. TAI Table 1. Comparison of experiment results of Binocular 2D with the same s from the previous study. Binocular 2D Previous study of the same Measured perceived distance of the visual target in F=B Measured perceived distance of the visual target in F>B ± m ± m 8.17 % ± m ± m 9.31 % % Increase of perceived distance Table 2. Comparison of increased percentage of measured perceived distance of visual targets under different displaying s. The first and second rounds of the experiment asked subjects to view the same set of experiment scenes output by the single camera setting using monocular and binocular vision. The perceived distances of the visual targets increased 9.34% and 8.17%, respectively. In both the Anaglyph 3D and JPS 3D sets, in which the scene being displayed incorporated the binocular disparity cue, the luminance contrast continued to influence the perceptual judgment of the visual target s location in the experiment scene. The perceived distances of visual targets increased 9.09% and 6.02%, respectively, for the Anaglyph 3D and JPS 3D sets. However, the comparison of the viewing s between the Binocular 2D and JPS 3D, suggests that the additional depth cue of the binocular disparity incorporated in the simulated three-dimensional scene may help viewers realize the true distance perception, thus reducing the effect of the luminance contrast on affecting the object s perceived distance in a scene. Table 2 illustrates the comparison of the increased percentage of the measured perceived distances of the visual targets in the F>B to the F=B for experiments performed under four different display s. The percentage is significantly smaller for the JPS 3D display. Measured perceived distance Measured perceived distance % Increase of of the visual target in F=B of the visual target in F>B perceived distance Monocular 2D ± m ± m 9.34 % Binocular 2D ± m ± m 8.17 % Anaglyph 3D ± m ± m 9.09 % JPS 3D ± m ± m 6.02 %

9 BINOCULAR DISPARITY AND DEPTH CUE 645 Table 3 compares the D-Thresholds for the experiments performed under the four display s. The D-Threshold is the average of the upper D- Threshold and the lower D-Threshold. Each represents the distance on the x- axis between the intersection of 0.75 and 0.25 proportion lines to the PSE in the Probit analysis function (Gescheider, 1984). The smaller the D- Threshold is, the steeper the Probit analysis curve is, meaning a smaller range of error for the PSE. As indicated in Table 3, the D-Threshold decreases from the viewing of Monocular 2D to Binocular 2D. This suggests that the subjects can make a more determined judgment on the same scene using two eyes rather than one eye. For the two types of stereo display, the D-Threshold for the Anaglyph 3D is also smaller than the other two non-stereo display s (except the D-Threshold for F=B for Binocular 2D). However, the D-Threshold for the JPS 3D is significantly the smallest among the four types of viewing s. Therefore, this study concludes that the autostereoscopic display of JPS 3D experimental scenes can offer a pictorial environment allowing a more determined perceptual judgment on the depth effect resulting from luminance contrast. Table 3. Comparison of D-Thresholds for four different viewing s. D-Threshold for F=B D-Threshold for F>B Monocular 2D ± ± Binocular 2D ± ± Anaglyph 3D ± ± JPS 3D ± ± Conclusion There were two objectives in this study. The first was to investigate the influence of binocular disparity on the depth effect of luminance contrast. The second was to advance the visual realism of the computer-generated pictorial environment for studying and envisioning the effect of luminance contrast on depth perception. Based on the results, it is concluded that incorporation of binocular disparity can advance the visual realism of the computergenerated pictorial environment. A computational framework that incorporates physically based lighting simulation, perceptually based tone mapping, and autostereoscopic display technology can generate a pictorial environment that allows a more pronounced perceptual judgment of the depth effect resulting from luminance contrast. In addition, although the effect decreases

10 646 N. TAI somewhat, luminance contrast remains an effective depth cue that can affect the perceptual judgment of the perceived distance of an object in this computer-generated stereo pictorial environment. Acknowledgements The author wishes to express his appreciation to the people who participated in the experiments. The author also wishes to extend his sincere appreciation to the National Science Council in Taiwan. This research was funded by the National Science Council under grant No: NSC E References Cadík, M.; Wimmer, M.; Neumann, L. and Artusi, A.: 2008, Evaluation of HDR tone mapping methods using essential perceptual attributes. Computers & graphics, 32(3), Finney, D.: 1971, Probit analysis, 3rd ed., University Press, Cambridge. Gescheider, G. A.: 1984, Psychophysics: method, theory, and application, 2nd ed., Lawrence Erlbaum. Kuang, J.; Yamaguchi, H.; Liu, C.; Johnson, G. M. and Fairchild, M. D.: 2007, Evaluating HDR rendering algorithms, ACM transactions on applied perception. 4(2), article No. 9. Lueder, E.: 2012, 3D displays, Wiley, Hoboken, N.J. Mardaljevic, J.: 2001, The BRE-IDMP dataset: a new benchmark for the validation of illuminance prediction techniques, Lighting research and technology, 33(2), O Shea, R. P.; Blackburn, S. G. and Ono, H.: 1994, Contrast as a depth cue, Vision research, 34(12), Palmer, S. E.: 1999, Vision science: photons to phenomenology, 1st ed., The MIT Press, Cambridge, Massachusetts. Reinhard, E.; Stark, M.; Shirley, P. and Ferwerda, J.: 2002, Photographic tone reproduction for digital images, ACM transactions on graphics, 21(3), Ruppertsberg, A. I. and Bloj, M.: 2006, Rendering complex scenes for psychophysics using RADIANCE: How accurate can you get? Journal of the optical society of America A, 23(4), Solso, R.: 2003, The psychology of art and the evolution of the conscious brain. MIT Press, Cambridge, Massachusetts. Tai, N.-C.: 2012, Space perception in real and pictorial spaces: investigation of size-related and tone-related pictorial depth cues through computer simulations, Computer-aided design and applications, 9(2), Tai, N.-C.: 2013, Application of luminance contrast in architectural design. Computer-aided design and applications, 10(6), Tai, N.-C. and Inanici, M.: 2012, Luminance contrast as depth cue: investigation and design applications. Computer-aided design and applications, 9(5), Wanger, L. R.; Ferwerda, J. A. and Greenberg, D. P.: 1992, Perceiving spatial relationships in computer-generated images, IEEE Computer Graphics and Applications, 12(3), Ward, G. and Shakespeare, R.: 1998, Rendering with RADIANCE: the art and science of lighting visualization, Morgan Kaufmann Publishers.