Spatial and Angular Resolution Measurement of a Tensor Light Field Display

Transcription

1 Spatial and Angular Resolution Measurement of a Tensor Light Field Display P. A. Surman 1, S. Wang 1, J. Yuan 1, Y. Zheng 1 1 Nanyang Technological University, Singapore Abstract - Dependent on the way in which the images are formed, some light field three-dimensional (3D) displays are less straightforward to characterize than two-dimensional (2D) displays. They do not have an effective screen position that physically exists, they do not have pixels in the conventional sense that can be simply measured with a microscope and, in common with all 3D displays, they have light emission that varies with angle, giving them the characteristic of angular resolution. In our previous work on light field characterisation, subjective evaluation was used to determine perceived spatial resolution and some other characteristics; however, the measurement of angular resolution eluded us. As this is the vital characteristic that distinguishes 3D displays from 2D displays, we set out to determine a method of obtaining this parameter from existing data collected. Crucial to this was the realisation that, first, the meanings of the characteristics must be defined. It was observed that the overall display system, including; capture, processing and display, behaves as a low-pass filter. Regarding the system as such enables the spatial resolution to be expressed in terms of spatial cut-off frequency and angular resolution in terms of angular cut-off frequency. This led us to determine the angular resolution indirectly from the variation of spatial resolution with depth. In this paper, we describe the rationale behind our method, which also dispenses with the need to conduct lengthy and unreliable user trials, and give the results of the measurements. 1. INTRODUCTION In our previous work on light field display characterisation [1], the measurements were principally subjective as at the time these appeared are the most appropriate in the first instance. They were useful for comparative evaluation of the difference in performance between two different algorithms, but not particularly useful for absolute measurements. For example, the most important measurement, that of spatial resolution, was based on the subject s ability to determine the correct orientation of Snellen E s [2] that are used on visual acuity charts. A more objective method, even though based on a visual assessment, has been developed to determine the absolute value of spatial resolution according to the definition for displays having pixels without defined physical boundaries. Treating the overall display system as a low-pass filter means that to determine its cut-off frequency, we must find the spatial frequency at which the amplitude is halved; hence the spatial resolution is expressed in terms of cycles per metre. This can be achieved by measuring its amplitude function with respect to spatial frequency, which is equivalent to the modulation transfer function in a purely optical system. A convenient method achieving this is with the use of a grating input image and measuring the contrast of its displayed image; this is described in detail in Section 2. Due to the way in which the images are formed, displays that do not have definite (quantized) pixels also do not have a quantized angular resolution. In this case the cut-off frequency is expressed conveniently in cycles per radian. Examples of this type of display are the multi-layer Tensor Display [3] [4] and the Holografika multi-projector display [5] [6]. After having determined the definitions of angular and spatial resolution, it became clear that the lengthy and inaccurate process of trials on subjects is not necessary and can be replaced by a faster and more accurate analysis of photographs of the image on the display. In this paper, the theory is described in Section 2, in Section 3 we the briefly describe methods we investigated that are not suitable for these measurements as this can inform future investigations, the measurement results are given Section 4 and in Section 5 we describe proposed ways of refining the technique to obtain greater accuracy in the future. 2. THEORY Previous measurements based on user tests on subjects were carried out on the image of the object in Fig. 1(a). This has depth increments of 5 mm and size increments of the E s increasing by a factor of 2 1/6 1

2 Grey level Surman PIERS 2017 v3 (1.122) in the vertical direction between each E, thus enabling a range of resolutions and depths to be measured. In Fig. 1 (b), it is seen that the display acts as a low-pass filter as the blurring on the Es appears to get worse gradually over several pattern size increments and with increasing distance from a virtual screen position. An interesting observation on the appearance of the Snellen E images is that they closely resemble a sinusoidal grating [7]. Another manifestation of low-pass filter-like properties is the appearance of ringing in the images [8] where a ripple effect is seen around step functions - edges in images. This is also referred to as the Gibbs Phenomenon. (a) Solidworks Depth Object (b) Image of Depth Object on Tensor Display Fig. 1. (a) Virtual object comprises stepped planes in 5 mm depth increments with Snellen E s in 2 1/6 size increments. (b) Image of depth object on multi-layer Tensor Display. The bars on the E s can be considered as small sections of a larger continuous bar grating and can be analyzed by treating the luminance along a line at right angles to the bars as a square wave. In this case, the same considerations as an electrical waveform apply where any waveform shape can be represented by a series of sinusoidal waveforms whose frequencies are integer multiples of the fundamental frequency, otherwise known as the first harmonic. The other waveforms are referred to as the second harmonic, the third harmonic and so on. A square wave is made up of a sine wave at the first harmonic, and odd harmonics with amplitude decreasing with increasing frequency [9] (Fig. 2(a)). The combined effect of diminishing coefficients with increasing frequency, and of low-pass filtering, means that the nearest harmonic (the third harmonic) to the fundamental frequency is attenuated by almost 10 db, and consequently can be neglected in this analysis; all other harmonics are attenuated to an even greater extent. There is also a so-called DC component of an electrical waveform, which in the case of an image is a constant grey level. In relation the amplitude of the square wave, the peak-to-peak amplitude of the first harmonic is I.272 times that of the square wave. We are interested in where the spatial resolution is halved, which is the spatial cut-off frequency of the filter. As this is where the amplitude of the first harmonic is halved; the cut-off frequency can be found from where the contrast is equal to 1.272/2 = In practice, the full black and the full white do not have greyscale values of 255, 255, 255 and 0, 0, 0 respectively and allowance is made for this when measurements are made. Sum of first 7 harmonics Mean grey level Bar grating function First harmonic Close-up of E 0 Distance across screen (a) Fourier Series Components (b) Reference Sinusoidal Grating Fig. 2. (a) A low-pass-filtered square wave function can be approximated by a sine wave with amplitude x that of the square wave and a constant value that is half the amplitude of the square wave. (b) This shows good matching between the reference grating and the image of the bars of the E s 2

3 Image depth (mm) Image depth Z I (mm) Pitch of 'E' bars (mm) Virtual screen position Surman PIERS 2017 v3 As the profile of the bars can approximated by only a greyscale value and the first harmonic sinusoidal function, a sinusoidal grating can be used as a reference (Fig 2(b)) to find the boundary where the halving of spatial frequency occurs. In the images analyzed, the black and white levels are not close to 0 and 255 greyscale so the maximum and minimum reference greyscales are set to be of the difference, but with the same mean value. Strictly speaking, system gamma should be accounted for; however, in this case, the contrast is sufficiently low for this not to be necessary. To analyse the image, we need to know the magnification of the image and the depth calibration. Magnification was found by measuring the width and height of the image and dividing these by the object dimensions. Depth calibration was achieved by measuring the lateral shift of the pattern positions at the front LCD screen from viewpoints 5 and 10 either side of the central axis at one metre from the screen. The shifts were measured by simply marking the front screen with the positions the pattern images appeared and then applying similar triangles. The straight trend line in Fig. 3(a) shows that the depth magnification is 1.50 and the image does not appear in front of the front LCD. As the type of display does not have pixels as such, we must determine where the screen is effectively located; this is a virtual screen and its resolution is referred to as the screen resolution. In the multi-layer, tensor display this virtual screen is located somewhere between the outer pair of LCD panels, whose resolution is referred to as the native resolution. When the depth object is displayed on the Tensor Display it is observed that highest resolution occurs at one depth, and decreases with increasing distance from this position. This is what happens in conventional quantized pixel 3D displays such as multiview and integral imaging. Therefore, the maximum resolution plane can be considered as the virtual screen, and in the case of Fig. 1(b) it is the fourth column from the left. Object depth Z O (mm) ± 5-30 ± (a) Image Depth 2.52 S 2.24 S 2 S Pitch of E bars 1.78 (mm) S 1.59 S 1.41 S 1.26 S S Z N Z S (c) Half-resolution Plot on Image Z F Half spatial resolution Z F Z S Z N Z S -Z F Z N -Z S S S PLAN VIEW 1/2 spatial resolution Virtual screen 1/2 spatial res. (b) Angular Resolution -30 Image depth Z-15 I (mm) 0 Maximum spatial resolution Z S Z N Front screen Object depth (mm) Z o (mm) T0 VIEWER (d) Plan View of Image Fig. 3. (a) Depth calibration averaged from two sets of measurements. (b) Angular resolutions are found from dividing the distances from the virtual to the planes where the spatial resolution is halved. (c) Plot of half spatial resolution boundary; the screen virtual screen position, spatial resolution and angular resolutions can be obtained from the positions of the 3 points. (d) Plan of image; it is completely behind the front screen. In the Holografika display, where multiple projectors produce multiple images on a vertically diffusing screen, clearly the effective screen position is at the diffusing screen as this where the projectors are focused, and also the slightly horizontally diffusing property of the screen is employed to blend the quantized angular input from the projector lenses to a low-pass filtered angular output. The screen resolution is obtained from the value of S in Fig. 3(c) where the blue half spatial frequency curve is a minimum. Its value in cycles per metre is obtained by dividing 1000 by S, the pitch of the E bars in millimetres. Z F 3

4 Black White Black White Reference sinusoidal grating Virtual screen Surman PIERS 2017 v3 g. Rads. d (mm) uare x 10 mm deep The angular cut-off frequency is determined by the distance from the screen where the spatial resolution is halved, as shown in Fig. 3 (b). Separate measurements for near and far are taken as it has been found that in practice that the value for the near cut-off frequency is lower than for far as the image does not appear closer to the viewer than the front LCD screen and the virtual screen is not far behind this. As S << (Z S - Z F ), the cut-off frequency A F in cycles per radian is given approximately by (Z S -Z F )/S where Z S and Z F are the depths of the virtual screen and far half-resolution plane respectively. If the near halfresolution plane is at or behind the front LCD, angular resolution A N is (Z N -Z S )/S cycles per radian where Z N is the depth of the near half-resolution plane. Once the maximum spatial resolution and virtual screen position have been determined, the depth at which the spatial resolution is halved is found by plotting the half spatial frequency curve and finding the depth at which the maximum spatial frequency is halved. As the pitch of the E bars is a geometric progression with a common ratio of 2 1/6, the near and far half-resolution distances are found by finding the depths the half-resolution plot crosses an E that is located in the sixth position above the maximum resolution E, as shown in Fig. (c). If the depth of field (DOF) is considered as the distance behind, or in front of the screen to where the resolution is halved, then this definition conforms with that in in SID s Information Display Measurements [10] where in our case, pixel size is replaced by the pitch corresponding to the cut-off frequency. 3. FINDING THE CORRECT METHOD Several methods were tested and evaluated, but it was determined that all of these would not work or would give unreliable results. The first approach was to build a virtual object in Solidworks that could provide an angularly varying output. The simplest object for this is a set of vertical apertures with a series of vertical black and white bars behind it, as shown in the plan view of Fig. 4(a). Different aperture pitches were used to give a varying angular output at rates of 0.5 to 2.5 in 0.5 increments. The image from this did not give any meaningful results and neither did the strategy of putting alternate black and white images on the columns of the 7 x 7 matrix of input images. After subsequent angular resolution measurement; and upon further consideration, the reason for this became apparent, as explained in Section mm z z d z z z z z zzz Image Apertures PLAN OF Fig. ANGULAR 4. (a) Plan FREQUENCY view of angular OBJECT test object the apertures and bars are intended to provide a controlled angularly varying image. (b) Proposed reference sinusoidal grating image alongside image of stepped bar grating object. Test pattern pitch (pixels) As the image of a bar grating is more-or-less sinusoidal at its spatial cut-off frequency, the use of a series of varying pitch gratings, as illustrated in Fig. 4(b), was considered. In this proposed method, an accurately produced sinusoidal grating would be shown to the side and displayed on only the front layer. A stepped model with the same shape as the one described in Section 2 would have a series of identical bar gratings with reducing pitch in the same way as the E s on the original model. Although this would have involved the need for lengthy user trials, initially, it appeared to be a good approach. However, careful examination of the results from our original work revealed that variations in Contrast Sensitivity Test Pattern Near Far 4

5 subject s contrast sensitivity [11] could affect results in some cases. Also, the luminance of the reference grating that is only displayed on one panel, would not match the luminance of main image that is attenuated by three screen layers. Even if the luminance of the reference grating did match the object, it would be difficult to match two contrasts accurately as shown in Fig. 2(b) where the two samples are in contact with each other. It was therefore decided to find some means of extracting the angular resolution from the existing Depth Object if possible. 4. RESULTS Measurements were carried out on images of the Depth Object produced by two different algorithms referred to as Algorithm A where there are no polarisers between the screen layers, and Algorithm B where the polarisers remain in place. Algorithm A uses additive control on the light rays as the polarisation rotations of the three layers are summed, and Algorithm B is multiplicative as the attenuations of each layer are multiplied together. This provides the opportunity to conduct comparative assessments. Figure 5 shows the results for Algorithm A where the red lower plot is for the highest resolution at each depth for the 100% correct identification of Snellen E orientation by 12 subjects who viewed the screen from one metre away. The blue upper plot is for the contrast that corresponds to a halving of the spatial resolution. For both sets of results, there is a spacing that is close to two E pitches (equivalent to 1 db), showing that at one metre, subjects can consistently identify the orientation at resolutions just outside of the display s bandwidth. 2S = 3.56 mm pitch Screen position = -7.5 mm Screen resolution = 1.78mm = 562 cs/m D N = = mm D F = (- 30) = 22.5 mm A F = 22.5 / 1.78 = 12.6 cs/rad S = 1.78 mm pitch Z N Z S Z F Image depth (mm) Fig. 5. Image of depth object processed with Algorithm B. The blue upper line is the half-resolution boundary and the lower red line is from subjective tests on subjects. The half-resolution plot is extrapolated to estimate the position for the 3.56 mm pitch E. Doing this for the near point would not produce a reliable result in this Once the reference sinusoidal grating has been produced in accordance with the black and white levels in the image, it is a simple matter to move the grating around over the photograph to plot the boundary. Some sample calculations are given in Fig. 5 and a summary of all the results is given in Table 1. For this display, it was found for both algorithms that the image does not extend out beyond the front LCD screen (Fig. 3(a)) and the virtual screen is only 7.5 mm behind it. Therefore, although we can deduce that near depth of field is 7.5 mm, the near angular resolution value is meaningless; it is zero cycles per radian for the three left columns. Table 1 is a summary of the results along with equations used and the equivalent values where pertinent; for example, it is generally easier to visualize the angular resolution in terms of degrees rather than cycles per radian. 5

6 Table 1 Summary of Results and Equations Parameter Algorithm Equation Measured Equivalent Value Units Value Units Linear magnification M A & B M= image width/object width 1.59 Dimensionless Same Same Depth function A & B Z I = 1.50 Z O 1.50 Dimensionless Same Same Screen position A & B mm Same Same Screen resolution A & B 562 cs/m 1.78 mm Depth of Near A D N = Z N - Z S 7.5 mm Same Same field D N B 7.5 mm Same Same Far A D F = Z S - Z F 15 mm Same Same D F B 22.5 mm Same Same Angular Far A A F = (Z S - Z F )/S 8.33 cs/rad 6.77 deg. resolution A F B 12.6 cs/rad 4.52 deg. Near A & B A N = (Z N - Z S )/S X cs/rad X deg. A N The measured angular resolutions of 6.77 and 4.52 seem fairly coarse; however, this is not unexpected considering the capture camera angular resolution is determined by the use of 7 cameras (3.5 cycles) over 10 (Fig. 6 (a)) and display angular resolution set by the pixel width of mm and screen separation of 15 mm Fig. 6(b)). The sum of these contributions is 3.9, which is in the same order as the measured results when the measurement error is taken into account. This would explain why the angular test object did not work. By putting all black and all white on alternate channels possibly did not work as the angular frequency of the input is actually higher than the passband of the overall system. Fig. 6. (a) Capture angular resolution for 7 cameras is 3.5 cycles over 10. (b) Display angular resolution is 1 cycle per 1.06 (atan((0.227*2)/30)). Fig. 7. Proposed logarithmic grating object with 2 1/6 geometric progression and continuous depth. The approximate ½ resolution boundary is indicated. 5. FUTURE WORK Regarding measurement errors of this method, in addition to the usual distance measurements that have a precision in the order of 0.25 to 1 mm with the type of measurements taken here, the principal sources of error are the quantization of the E pitch increments which is around ± 6%, and the depth increments of 7.5 mm which, in the worst case, represent an error of ± 25%. Although the total error is relatively large, the measurements have given us ball-park figures we can use to inform the second generation of measurement. The stepped model was not originally intended for measuring angular resolution and consequently, the depth increments are rather large for accurate results. This limitation can be overcome with the use of a sloping surface on the front of the object (Fig. 7) so that the depth is continuously variable. Similarly, the E s can be replaced by a grating with a continuously increasing pitch. If the bars of the grating are 6

7 vertical this will enable it to be also used on our proposed horizontal parallax only prototype. The range of the maximum grating pitch must also be extended. It would be possible to have a grating that covers 0.5 mm to 8 mm pitch (4 octaves) with a geometric progression of 2 1/16 (1.044) that is only around 170 mm wide. A new model to this specification will be made and evaluated. 6. CONCLUSIONS Although the accuracy of the angular measurements is low at an estimated worst case of ± 30%, we now have a ball-park figure that appears to be reasonable in relation to the capture and display hardware angular resolutions; whereas, previously we no idea at all what the value of the angular resolution was. It has also indicated why our initial attempts to measure it failed. It is interesting to note that an eye spacing of 64 mm subtends 3.7 at a viewing distance of one metre. This is very close to the estimated overall system resolution of 3.9 and it this estimate is correct then it is understandable why the depth of field is low. This exercise has proved particularly useful in determining future techniques for measuring angular resolution in true 3D displays, and has led us to a deeper understanding of image space requirements. ACKNOWLEDGEMENT This work is funded by the Singapore National Research Foundation under Grant No: NRF-CRP REFERENCES [1] Wang, S., K. S. Ong, P. Surman, J. Yuan, Y. Zheng, X. W. Sun, Quality of experience measurement for light field 3D displays on multilayer LCDs, J. SID, , December 2016 [2] Vimont. C., All About the Eye Chart, AM ACAD OPTH, Accessed 22 Aug 2017 [3] Wetzstein, G., D. Lanman, M. Hirsch, R. Raskar, Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting, ACM TOG, Vol. 31 Iss. 4, July 2012 [4] Wang, S., et al. "Maximizing the 2D Viewing Field of a Computational Two-layer Light Field 3D Display." SID 2015 DIGEST. [5] US Patent No. 6,201,565 B1 Method and apparatus for displaying three-dimensional Images, T. Baloch, 2001 [6] Kovaks, P, A. Boev, R. Bregovic, A. Gotchev, Quality Measurements of 3D Light-field Displays, VPQM-2014, Chandler, Arizona, January 30-31,2014. (pp. 1-6). Arizona: Arizona State University. [7] Sekuler, R. et al., Perception, Chapter 5, 3rd ed., 1993 [8] Carslaw, H. S., Introduction to the theory of Fourier's series and integrals, (3rd ed.), Ch. IX, [9] Chapter 4 Fourier Series and Integralshttp, math.mit.edu/~gs/cse/websections/cse41.pdf Accessed 22 Aug 2017 [10] Information Display Measurements, SID, Definitions and Standards Committee, ICDM, p374, Standard Version 1.03 June 1, 2012 [11] Pellia, D. G., P. Bex, Measuring contrast sensitivity, VIS RES, Vol. 90, 2013,