Measuring and modeling per-element angular visibility in multiview displays

Transcription

1 Measuring and modeling er-element angular visibility in multiview dislays Atanas Boev, Robert Bregovic, Atanas Gotchev Deartment of Signal Processing, Tamere University of Technology, Tamere, Finland ABSTRACT Multiview dislays emloy an otical layer, which distributes the light of an underlying TFT-LCD anel towards different directions. Certain roerties of the layer create secific artifacts, such as ghost images, moiré erns and masking. We model the layer as image rocessing channel and identify dislay arameters related with the model, which are imortant for design of image rocessing algorithms for artifact mitigation. The identified arameters are interleaving ern, angular visibility and frequency throughut of the dislay. We resent a methodology for deriving these arameters for an arbitrary LCD-based multiview dislay, which does not require recisely-ositioned measurement equiment. As a case study, we resent measurement and modeling results for a articular multiview dislay. 1. INTRODUCTION Multiview dislays are class of autostereoscoic dislays, which can be used without the need of secial glasses and can be watched by multile users simultaneously 1,, 3, 4, 5, 6, 7. Multiview dislays generate multile observations of a scene, each one seen from a different angle. Usually, the image is formed on a TFT-LCD. Additional directionally selective otical layer mounted on to of the LCD redirects the light of the sub-ixels toward different directions 1, 4, 7, 18. The layer is either a arallax barrier, which blocks the light in some directions 7 or lenticular sheet, which works by refracting the light 8. The aarent brightness of each sub-ixel is a function of the angle. The grou of sub-ixels which is visible from one direction forms an image, known as view 1, 7, 9. From a certain sot in front of the dislay, all subixels that belong to a view are seen with maximal brightness. Such sot is referred to as an otimal observation sot for the corresonding view. Outside of that sot, there is a larger view visibility zone, in which the view is still visible, albeit with diminished brightness. In LCD-based multiview dislays, views are satially multilexed 1,,9. The rocess of maing multile images to the views of one dislay is called interdigitation 9, view multilexing 1, interlacing 3, 5 or view interleaving 14. The latter term is adoted in this aer. The relation between the osition of a sub-ixel and the view it belongs to is given by an interleaving ma. Since both the LCD and the otical layer have a reetitive structure, the interleaving ma can be described by a eriodic interleaving ern 7, 9. The ern is satially indeendent angular visibility of a sub-ixel deends on its osition in resect to the ern, but not on its absolute osition in resect to the dislay. The design of a multiview dislay is a trade-off between observation convenience and visual quality. The added convenience in using multiview dislay comes at the exense of limited brightness, contrast and resolution 1,9,1. The otical layer is a source of secific visual artifacts 1. These are moiré erns caused by aliasing 9, 11,13, ghost images caused by crosstalk 7,1, 19, and masking artifacts, caused by the otical layer behaving like an u-samling block. The masking artifacts manifest themselves as a fine mesh suerimosed over the image, an effect that can be regarded as imaging, as discussed in Section 5. The influence of these artifacts on the visual quality of a 3D scene deends both on image content and otical arameters of the dislay. It is ossible to use image rocessing methods to mitigate these artifacts 5, 9, 13, 14, and the effectiveness of the image rocessing methods highly deends on the information about the otical characteristics of the dislay, which is however rarely available to the end user. There are various methods for assessing the otical quality of the dislay for examle, using directional scanning for D 15 and 3D 1 dislays. In 8 the authors roose an extensive list of otical arameters which can be measured for characterization of autostereoscoic 3D dislays. In this aer, we aim to identify and measure arameters that can be used for visual otimization. Thus, we are interested in the arameters that are imortant from image rocessing oint of view, rather than in ones, which describe the otical quality of a 3D dislay. The aer is organized as follows. In Section, we roose a model, which considers a multiview dislay as an image rocessing channel. The model is used to describe the tyical visual artifacts in signal rocessing terms. The arameters 1

2 in this model identify which characteristics of the dislay are needed in visual otimization algorithms. In Section 3 and Section 4 we resent a simle, yet effective methodology to measure and model these arameters. More secifically, in Section 3 we describe a way to derive the interleaving toology, and in Section 4 we roose method for finding the angular visibility of each dislay element by combining measurement data from several oints. Finally, in Section 5, we characterize the roerties of the dislay in the frequency domain. As a case study, this aer resents measurement results for 3" 3D Dislay AD built by X3D-Technologies GmbH, which is hereafter referred to as X3D dislay.. MULTIVIEW DISPLAY AS IMAGE PROCESSING CHANNEL Multiview 3D dislays aim to generate multile images, each one seen from a different observation angle. The otical layer mounted above the screen surface acts as directionally selective filter and alies angular luminance function to each sub-ixel of the dislay. The angle, at which the angular luminance has its eak value, determines the otimal observation direction of the sub-ixel. There are grous of sub-ixels with similar angular luminance functions which are simultaneously visible, thus creating the illusion of an image, which is visible from certain angles and invisible from others. The number of grous with a similar angular luminance function determines the number of views generated by the dislay. The interleaving ma can be reresented as a set of non-overlaing lattices, where each lattice contains sub-ixels from a single view only 9. On an image with the full resolution of the LCD, each of these lattices acts as a rectangular subsamling ern with a different offset. An examle is shown in Figure 1a, where the intersecting dotted lines mark the osition of LCD sub-ixels, one lattice is marked with circles and another is marked with crosses. The horizontal ste of the lattice equals to the width of the interleaving ern (denoted with equals to the height of the ern (denoted with n in Figure 1a) and the vertical ste m ot in the same figure). All sub-ixels that aear on n column k and row k m, where k is an integer, belong to a grou with equal angular visibility. The subixels that aear on column k n x (for x n ) and row k m y (for y m ) also belong to a grou with equal angular visibility. It is ossible that more than one of these grous belong to the same view. A model of a multiview dislay as an image rocessing chain is shown in Figure 1b. We assume that as inut we have v images with full resolution, which have to be maed to v views generated by the dislay. Out of each image, only sub-ixels that belong to the corresonding view are used. This is modeled by D downsamling oeration. Since the views are satially multilexed, each image gets samled with different offset, reresented by image shift z x, y, where x, y are the horizontal and vertical offset. On the dislay, the sub-samled image is reresented in its original size. The visible sub-ixels aear either surrounded by black stries by the arallax barrier or enlarged by the lenticular sheet. This effect is modeled as an usamling stage, where the introduced samles are either set to zero, or are reetition of the same samle value. Since several grous of sub-ixels can belong to the same view, downsamling and usamling, in most general case, is not erformed on a rectangular grid. The angular luminance is modeled as a function of the observation angle. We model the effect of the otical layer on brightness of underlying ixels as visibility the ratio between the relative brightness of a view and the maximum brightness of the dislay as seen from the same angle. The function f k ) gives the visibility of view v from ( v observation angle. We refer to such function as angular visibility. We assume that the function is the same for all views, with the eak visibility of each view occurring at different observation angle. With k v we denote the offset of the function for view v.

3 n y Image 1 z x 1, y 1 f k 1 m x Image Image 3 z x, y z x 3, y 3 f f k k 3 Σ Interleaved image Image v z x v, y v f k v a) b) Figure 1, Model of the otical layer effect as an image rocessing channel: a) interleaving ma as a set of non-overlaing lattices and b) interleaved image as a weighted sum of samled images Using the roosed model, tyical visual artifacts on multiview dislays can be exlained from signal rocessing oint of view. Aliasing occurs if the source images have not been suitably re-filtered with an anti-aliasing filter before downsamling. The design of an anti-aliasing filter relies on knowledge of interleaving toology 9, 13, 14. The toology can be derived by finding n, m and all combinations of x and y that belong to the same view. Ghost images occur when images with different offset are simultaneously visible this effect can also be regarded as crosstalk. Crosstalk mitigation algorithms need knowledge of the angular visibility function 13,19, which is denoted as f k ) in our ( v model. Due to the otical layer, visible arts of sub-ixels have non-rectangular shae 16, 17, 18 and gas between them are directionally oriented. The resence of gas creates effects similar to the ones caused by usamling in the absence of a ost-filter. In samling and interolation literature the effect is denoted as imaging and filters tackling it are known as anti-imaging filters. In the case of multiview dislays, this effect is best quantified by analyzing the erformance of the dislay in the frequency domain. 3. DERIVING THE INTERLEAVING PATTERN In some cases, the descrition of the interleaving ern is artially or fully missing in the documentation rovided along with the dislay. This romts roosing a methodology for deriving the ern. Our roosed methodology has four stes finding the minimal width and height of the ern ( n and m in Figure 1), obtaining the number of views generated by the dislay, and finally, deriving the comlete interleaving ern. Ideally, a view should be visible with maximum brightness from a limited range of observation angles and be invisible from anywhere else. Examle for angular visibility of a view is given in Figure a, with visible and invisible regions marked with V and N, resectively. We refer to the ratio between the width of N and V regions as visibility searation. A grou of sub-ixels with similar angular visibility has a higher overall N/V ratio than a grou where each sub-ixel has a different otimal observation oint. The otimal interleaving ern of a 3D dislay is the one that searates the sub-ixels into grous that yield the highest visibility searation. A B C N V N V N N V N V N θ 1 θ (c) Figure, Angular visibility of multiview dislay: a) visibility searation, b) observation angles when taking a close shot, and c) close observation of angular visibility 3

4 In order to study the angular visibility of a sub-ixel one can selectively activate grous of sub-ixels and erform an angular scan 5, or use Fourier otics to study a oint on the screen from many angles simultaneously. Our aroach is to observe the dislay from a distance shorter than the otimal on, utilizing the sace-invariance of the ern.. In this aroach, the visibility of multile sub-ixels as seen in one camera osition is related to the visibility of one sub-ixel as seen from multile camera ositions. As exemlified in Figure b, oint A observed from close distance is seen from the same angle as oint B observed from the otimal observation distance at angle. Similarly, oint C as seen from closely ositioned camera should have the same visibility as oint B from observation angle. We refer to images taken from closely ositioned camera as close shot images. In the close shot, horizontal line of the screen is exected to have visibility roortional to the angular visibility of a oint from the otimal observation angle, as seen in Figure c. The higher angular visibility ratio (as seen in Figure a) is, the higher the N/V ratio of the horizontal line is (as seen in Figure c). In our exeriments, the distance between dislay and camera was 8cm. 3.1 Finding ern width The otimal width of the interleaving ern is found by observing a row of sub-ixels, robe for different ern widths, and select the set with the highest visibility searation. First, one should take a close shot of a square with known size in sub-ixels and use it to estimate the camera orientation and to determine the ratio k between dislay ixels and ixels in the acquired hotograh. This allows to estimate where each hotograhed sub-ixel aears on the hoto, regardless of the exact camera lacement. The next ste is to activate every n -th sub-ixel in one row (as seen in Figure 3a), and take a close shot. As the line consists of discrete sub-ixels, the angular visibility of the view would aear samled at discrete angles. In order to evaluate the visibility searation, one should distinguish dark ixels in N regions masked by the otical layer, and dark gas in V regions caused by inactive sub-ixels. The width of the gas in the V zones is roortional to the currently robed ern width n. For making the distinction, we aly a maximum value filter in horizontal direction using window size of n k / 3 ( k is divided by 3 to get the dislay ratio er sub-ixel) (c) Figure 3, Deriving the width of interleaving en: a) test image containing lines with different ste size b) close observation of the test image, b) test image, rocessed with maximum value filter with coressonding window size. By robing for various values of n, one can obtain the visibility searation for interleaving ern with the corresonding widths. The maximum n is bounded by the exected number of views; we used 1< n 64 in our exeriments. A test image, which contains lines with different n can be seen in Figure 3a. The same image, as seen on a close shot of the X3D dislay is in Figure 3b. The rocessed image, where each line is filtered with the corresonding maximum value filter is shown in Figure 3c. By counting consequitive black ixels in each row, one can find the minimal n which has the highest visibility searation. In our exeriments this is n Finding ern height The minimal height m of the interleaving ern can be found by testing erns with otimal width and variable height m. The test image for ern n m has sub-ixels on every n -th sub-ixel column and every m -th ixel row 4

5 lit. Each row in that image has otimal visibility searation, but if the otical layer is mounted at a slant, the osition of the V zones might differ across the rows. For some value of m, the otimal observation directions for each row coincide, making all active sub-ixels simultaneously visible from certain observation directions. If one considers the mean brightness of all visible sub-ixels, the otimal m is the value which yields the highest visibility searation for the dislay as a whole. Unfortunately, from close osition the angle from which dislay rows are seen varies in vertical direction as well. Thus, from a single close shot it is imossible to redict if the otimal observation oints of each row would coincide at some distance. However, most multiview dislays aim to rovide views which sread in horizontal direction only 1,7, 18. In that case, the angular visibility of the dislay should change frequently in horizontal and less often in vertical direction. In the close shot, this corresonds to vertical lines with minimal slant. In order to distinguish non-visible from inactive sub-ixels, one can aly a two-dimensional maximum value filter with window size of ( n k /3) ( m k) where n, m are the size of the ern used in each test. The filtered image with lowest frequency in vertical direction corresonds to the otimal size of the interleaving ern. The to row of Figure 4 shows close shots of different test erns on X3D dislay, and the bottom row shows the corresonding rocessed images. In our measurements, we used the osition of the biggest eak in the D frequency resonse to determine slant of the lines, and found ern size of 8 1to be the otimal one. 8x6 8x11 8x1 8x13 Figure 4, Deriving the heigth of the interleaving ern: to row close observations of test erns with various heigths, bottom row the same, rocessed with maximum value filter with coressonding window size. 3.3 Number of views It is ossible that a dislay with interleaving ern of n m generates less than n m views, i.e. there are sub-ixels with different coordinates in the ern which belong to the same view. This case can be tested by building a test image, using the otimal ern size with only one sub-ixel in the ern lit, as shown in Figure 5a. We refer to these images as test ern, followed by the row and column where the active sub-ixel is ositioned. For examle, test ern R4C7 is a test image that consists of a eriodic ern with size n m, where sub-ixel on row 4, column 7 is lit. The total number of such test ern images is. If two test erns contain ixels that belong to the same view, they would have identical angular visibility and contribute light to the same regions in the close shot. The close shots need to be rocessed with a max value filter with window size of ( n k / 3) ( m k) like it is done in the revious section. Test erns with sub-ixels which belong to one view should roduce slightly different outut, since sub-ixels of those erns aear on different coordinates of the dislay. However, test erns with different angular visibility should roduce noticeably different results. The norm of difference between two outut images can be used to determine if two test erns belong to the same view or not. One can calculate the l -norm of the difference between each air of filtered images, build a histogram of all norms, and find the threshold between similar and different norms, as shown in Figure 5b. The first grou in the histogram reresents differences between similar erns that belong to the same view. Close shots of X3D dislay exhibiting some test erns and their filtered counterarts are given in Figure 6. In this examle, sub-ixels R4C7 and R7C5 were found to belong to the same view. Test erns with normalized difference lower than the threshold were groued together. The whole set of 96 test erns roduced 4 distinctive n m 5

6 Frequency grous of 4 erns each, which means that X3D dislay is able to generate 4 distinctive views. Sub-ixels, which belong to the same grou, are marked with x on Figure 5a. Sub-ixel (R) (G) (B) (R) (G) (B) (R) (G) 1 x 3 4 x x x 11 1 R3C8 R4C7 R7C5 R9C same view different view Normalized difference 4.5x1 4 x 1 4 Figure 5, Single-sub-ixel test erns: a) osition of sub-ixel in the ern (ixels marked with x are found to belong to the same view), b) normalized differences between close observations of all test erns R4C7 R7C5 R4C8 R9C8 Figure 6, Deriving number of views: to row close observations of various test erns, bottom row the same test erns after filtering. Sub-ixles, which belong to the same view, roduce similar close observations (similarity treshold is shown in Fig. 5b) 3.4 Interleaving toology The final ste is to assign a view number to each grou of sub-ixels with similar angular visibility. The order of the views is arbitrary, but for ractical reasons it is referred that views with neighboring observation zones have consecutive numbers. To enumerate the views in that order, one should find the visibility zone of each view. A set of vmax test images is reared, where vmax is the derived number of views. In each test image, all sub-ixels which belong to the same view are lit. We refer to these images as single-view ones. By observing a single-view image on the dislay, one can search for an observation osition, from which the dislay is seen with maximal mean brightness. This osition is the otimal observation sot of the corresonding view. In our exeriments we could identify visibility zones with fuzzy borders, aearing aroximately on a line, in front of the dislay, as shown in Figure 7a. As this result is consistent with measurements of other dislays 5,, we aroximated otimal observation sots to be equidistant on the line, as marked with X in Figure 7a. In our measurements, the distance between the dislay and the line was aroximately 14cm. 6

7 Pixel rows Pixel column 1 Pixel column Sub-ixels columns n 1 Otimal observation sots Measurement oints (R) (G) (B) (R) (G) (B) (R) (G) Figure 7, Deriving the interleaving toology: a) measurement oints in the aroximate centers of the otimal observation sots (view from the to); b) interleaving ern for X3D dislay In our exeriments, we enumerate the zones from left to right, so that middle zone numbers are aligned with the center of the dislay. By labeling the sub-ixels with corresonding numbers, we obtain the interleaving ern for X3D dislay as shown in Figure 7b. 4. ANGULAR VISIBILITY OF EACH SUB-PIXEL In general, doing an angular scan to obtain the visibility as a function of the angle requires recise ositioning otherwise angular visibility curve is samled at irregular intervals. In our method, we measure the visibility of each view at arbitrary oints along the otimal observation distance, and search for single function that gives the best fit for all measurements regardless of the observation oint. 4.1 Measurements As a first ste, one needs to reare all single-view images that corresond to the views generated by the dislay. Then, the measurement oints have to be selected as close as ossible to the centers of the visibility zones. In our exeriments, twenty-five measurement oints were selected. Twenty-four of them are in the visibility zones of each view, and the last one is in the visibility zone of the first relica of view 1, as marked with X in Figure 7a. Observation oint 13 aears in the center. The next ste is camera calibration. Camera sensitivity and aerture should be set to minimum to minimize CCD noise. Camera exosition is to be chosen such that the maximum ixel value in the shots is under the range in which camera resonse gets into saturation. Then, the camera resonse function should be linearized, in a fashion described in. In our exeriments, we used 16 gray-level test images where all ixels were set to values between and 55, with ste of 16. All test images were shot in a dark room from measurement oint 13. The last ste is measuring the angular visibility. One should reare v max single-view images, where vmax number of views generated by the dislay. Additionally, one white image with all sub-ixels set to maximum brightness and one black image with all sub-ixels set to zero are needed. From each observation osition the white, black and all single-view images should be hotograhed. The set of images shot from one oint is to be rectified using the corners of the dislay in the white image. For each single-view image, the mean brightness in the center of the dislay is to be measured, and normalized to range from zero to one, where zero is the mean brightness measured for the black test image, and one is the brightness measured for the white test image. In our measurements, from each measurement oint we took 6 shots of X3D dislay, showing the black, white and 4 singe-view test images. After normalization we obtained 5 sets of 4 measurements indicating the brightness of each view relative to the full brightness of the dislay, for each observation osition. Because of the normalization, the maximum measured brightness is aroximately the same (close to.5) for all angles. We used the normalized is the 7

8 S 5 ( ) S 1 ( ) S 1 ( ) S 6 ( ) measurement as visibility observation S (v) where oint v is the 5 view number, and is the observation measurement oint oint. 13 Results for four (otimal for view 1) (otimal for view 6) measurement ositions.5 are observation shown in Figure oint observation oint 13 (otimal for view 1) (otimal for view 6),4,4.5.5,3,4,3,4,,3,,3,1,,1,, , observation oint 1 observation oint observation oint 1 observation oint 5,4,4.5.5,3,4,3,4,,3,,3,1,,1,, , Figure 8, Measurement results for X3D dislay for measurement oints 1 (to-left), 6 (to-right), 5 (bottom-left) and 1 (bottom-right). 4. Modeling angular visibility If the measurement oint is dislaced from the center of a visibility zone, the visibility function gets samled with an offset, and the maximum value of that function falls in between two samles. However, judging by measurement results in other works 7, we assume that the visibility curve for all observation oints can be closely aroximated by the same function, which has its eak occurring in the otimal observation sot for the corresonding view. We use denote offset of the visibility curve. Integer values of the offset visibility sot, and non-integer values of to corresond to the angular visibility in an otimal corresond to the angular visibility between sots. Based on this assumtion, one can search for single function that closely fits measurements for all ositions regardless of ossible offset. We decided to fit a eriodized Gaussian function, where v 1,..., v and search for otimal a, max G a,, v) k ( v kv max ( ae, that will give the minimum fit error in: ) (1) arg min max arg min a, R 1 G a, R, S () where max is the total number of measurement oints, and v max is the total number of views. The resulting Gaussian function for X3D-dislay with.51, By samling this curve for integer values of v and a is shown in Figure 9a along with ( ) S. 1 v v one can get the visibility of each view for observation oint. By relacing the view numbers in the interleaving ma with their visibilities, one can get the visibility of each sub-ixel for observation oint. 8

9 visibility visibility Since the visibility curve is the same for all observation ositions, and the otimal observation oints are equisaced, we can assume that visibility of one view from different observation oints follows the same function as visibility or different views from one observation oint. In that case one can estimate visibility between two otimal observation ositions by choosing fractional. As an examle, G (v ) for 5. 5 is shown in Figure 9 along with actual measurements done from observation oint that lies between oints 5 and G S G S z Figure 9, Modeling angular visibility: a) shae of derived Gaussian curve (G), with measurements for observation oint 1 (S 1 ), b) redicted visibility of all elements (G) between observation oints 5 and 6, with measurements done in that oint (S z ). 5. PERFORMANCE ANALYSIS OF A MULTIVIEW DISPLAY IN THE FREQUENCY DOMAIN Based on the measurements described in the revious sections we can determine the visibility of every ixel on the LCD matrix from an arbitrary observation oint. At the same time, due to the structure of the dislay, only a fraction of the ixels will be visible from one observation oint (view). In addition, there are gas between the visible sub-ixels in one view. Thus, when visualizing full-size images (i.e. images with resolution equal to the LCD matrix resolution) there are two effects involved: aliasing due to icking u sub-ixels on non-rectangular grid and imaging, due to the resence of gas- Aliasing can be fully tackled by an anti-aliasing re-filter. Usually, imaging is tackled by an anti-imaging ostfilter. As the imaging is created by the hysical structure of the dislay, in is imossible to imose a ost-filter. However, the effect can be artially mitigated by a re-filter. In order to determine the roerties of the required D filter, and consequently to have the best ossible reresentation of images on the dislay (minimizing aliasing, imaging and ghosting), it is necessary to determine the erformance of the dislay in the frequency domain, that is, we have to know which frequency comonents in the image we can kee (ones that will be roerly reresented on the screen) and which ones we have to attenuate (remove) as otential causes of distortions. Determining the erformance of the dislay in the frequency domain is not trivial. Based on the interleaving ern determined in Section 3 and the dislay model given in Figure 1b, one could derive an analytical exression of the frequency domain behavior of the dislay. However, this theoretical aroach does not take into account several effects occurring on the dislay. First, the visible sub-ixels are not on a rectangular grid (See Section 3). Second, every visible sub-ixel is seen with a different intensity (See Section 4). Finally, third, due to the masking effects, the ixels have a non-rectangular shae 16, 17, 18. The combination of these effects creates an intensity distribution ma, which alters the brightness of each image element in a non-linear fashion. Assuming that we could somehow model all nonlinear effects, the analytical aroach would become mathematically very demanding. Moreover, due to the fact that in the general case we do not know the exact roerties of all arts of the screen (otical layer, slant of the barrier, thickness, etc.) in the theoretical aroach many tradeoffs would be made thereby ending with a frequency resonse that might or might not describe the dislay well enough. In order to achieve a ractical solution, it has turned out that for deriving the frequency resonse of the dislay it is much more convenient to use a measurement-based aroach as described in this section. In the derived frequency resonse we will denote as assband, the region in the frequency domain containing frequencies that are roerly reresented on the screen. All other regions will be denoted as stoband. The main idea in the roosed aroach is to generate several images containing signals with various known frequencies, visualize them on the dislay, and then comare the outut of the dislay with the inut images. We illustrate the rocedure for the X3D dislay, but the aroach itself is erfectly alicable for any other multiview dislay. 9

10 5.1.1 Prearing test images First ste in measuring the frequency resonse of the dislay is to generate aroriate test images. For this urose we reared several hundred images, each of them being a ern of a fixed known frequency. Two of those images for frequencies (f x, f y ) = (.1,.1) and (f x, f y ) = (.,.3) are shown in Figure 1 with the corresonding sectra shown as contour lots in Figure 11. Here, f x and f y refer to frequencies along the x and y axis, resectively. The frequencies are normalized to f s /=1 with f s being the samling frequency. We can see in Figure 11 that each of those signals has distinct eaks in the sectra (the eak at (f x, f y ) = (, ) is the DC comonent and should be ignored). The motivation for using such images lies in the fact that by knowing exactly what image we sent to the dislay, based on what we see on the dislay, we can determine the roerties of the dislay in the frequency domain. We assume no a riori knowledge about the dislay roerties. Therefore, the test images must be generated for all sets of frequencies (f x, f y ) with f x [, 1] and f y [ 1, 1]. The signals for f x [ 1, ) and f y [ 1, 1] can be easily reconstructed by taking into account symmetry roerties of sectra of real signals. In order to obtain a recise frequency resonse of the dislay, we have to use a very dense grid of frequencies. However, a very dense grid means a high number of images that, in turn, will require a lot of measurements. Therefore, we suggest that first a larger grid is used, e.g. f.1, to roughly determine the roerties of the screen and then reeat the measurement with a denser grid, e.g. f.1, in the regions around the edges of the assband. In this aer, for the X3D dislay, we used the ste f =.5. Figure 1, Examle of inut (test) images: a) (f x, f y ) = (.1,.1) and b) (f x, f y ) = (.,.3). Figure 11, Examle of inut images Sectra (contour): a) (f x, f y ) = (.1,.1) and b) (f x, f y ) = (.,.3) Measurements Second ste in measuring the frequency resonse of the dislay is to visualize the above described inut images on the dislay and take hotos of the screen by using a high resolution digital camera. The hotos were taken from a distance of aroximately 4cm from the screen. Although the distance is not critical, the camera should not be ut too close to the screen in order to avoid interference between multile views. The hotos taken by the camera will be hereafter referred to as outut images. As an examle, for the inut images shown in Figure 1 the outut images are shown in Figure 1 (images have been enhanced for clarity) with the corresonding sectra given as contour lots in Figure 13. 1

11 Please note that due to the fact that k, the ratio between ixels on the dislay and ixels in the outut images, is less than one, after evaluating the sectra, the frequencies have to be rescaled by factor 1/k. This scaling has been already included in Figure 13. Figure 1, Examle of outut images (hotos taken from the dislay): a) (f x, f y ) = (.1,.1) and b) (f x, f y ) = (.,.3). Figure 13, Examle of outut images Sectra (contour): a) (f x, f y ) = (.1,.1) and b) (f x, f y ) = (.,.3). Several observations can be made based on these measurements. First, although each of the inut images contains only a single frequency comonent, the outut images contain many distinct frequency comonents. This is mainly due to the aliasing and imaging effects of the dislay. This is modeled in Figure 1 through down-samling and u-samling blocks. Second, although, there are many high frequency distortions in the outut image due to imaging, which is in turn due to the hysical gas between visible sub-ixels, those imaging comonents are artially suressed by human visual system. Therefore we are still able to see roerly the inut signal on the dislay as long as the inut signal contains only low enough frequency comonents. This is illustrated in Figure 1 and Figure 1. In the first figure, the diagonal lines are still seen even if they are heavily broken, but in the second figure, beside the barely visible diagonal lines from Figure 1, many other lines are also seen and therefore we cannot roerly identify the inut signal. Imaging is an issue as it occurs at the dislay and we do not have any ossibilities to remove it. Third, the dislay introduces non-linear distortions as illustrated by Figure 14. This figure shows the sectra along the x-axis for the inut signal (f x, f y ) = (., ) and the corresonding outut signal. Although the inut signal has only one sectral comonent at f x =., the outut signal also contains harmonics at f x =.4 that are aroximately 6-8 db lower than the main sectral comonent. 11

12 Figure 14, Nonlinear distortions Sectra along x-axis for signal (f x, f y ) = (., ): a) Inut image and b) Outut image Evaluating the frequency resonse of the dislay Third and final ste in measuring the frequency resonse of the dislay consists of comaring the sectra of the above derived inut and outut images. In order to suress the noise in the outut image, we threshold the sectrum of each image to -3dB bellow the strongest frequency comonent. This threshold was exerimentally chosen for the X3D dislay, but we believe that it can be used for any other dislay having 8bit image deth. The criteria for determining if a given frequency comonent asses through the system roerly or it is distorted due to the aliasing and imaging errors was following: For every inut signal of frequency (f x, f y ) we checked if the contributing aliasing / imaging comonents contain frequency comonents that are inside a circle with radius f x f y r, (3) that is, if there are signals with a lower frequency than the one used at the inut. In all cases we ignored the DC comonent. The motivation behind this criterion is twofold. First, according to the samling theory, aliasing will occur once the inut signal is greater than half of the samling frequency. The frequency of the aliased comonent is lower than the frequency of the signal itself. Therefore, the above criteria effectively checks if aliasing occurred or not. Second motivation lies in the fact that the low frequency errors (like the ones caused by aliasing) are much more visually annoying when occurring on the screen then the high frequency ones (e.g. imaging). Consequently, we define the assband of the dislay as region of all test frequencies, which cause no additional frequency comonents inside radius r, and define the stoband everywhere else. As an examle, the magnified versions of Figure 13 and Figure 13 are shown in Figure 15 and Figure 15, resectively. In these figures only the art containing frequencies smaller than r (reresented by the circle) is shown. As seen from the figures, in the first examle, there are no sectral comonents that are of a lower frequency than the one used for generating the inut image and therefore signal of this frequency would be in the assband. In the second examle, the outut image considerably differs from the one sent to the dislay due to the aliasing errors. In the sectral domain this can be noticed by resence of several frequency comonents that are inside the circle with radius r. There is no oint in trying to reresent images containing such inut frequences on the dislay under consideration. Figure 15, Sectra of outut images: a) (f x, f y ) = (.1,.1), r =.14 and b) (f x, f y ) = (.,.3), r =.36 1

13 By alying the above criteria to all used inut/outut images, the assband and stoband of the dislay is classified as given in Figure 16. In this figure, the assband is reresented by dots. Due to measurement errors, the assband region is not continuous. It can be easily smoothened by alying a 5 by 5 median filter. The final frequency resonse for the X3D dislay is shown in Figure 16. As exected, the frequency resonse of the dislay given in Figure 16 shows that the dislay is able to reresent signals containing low frequencies. By following our methodology, one can obtain the assband region for an arbitrary dislay. This resonse can be used for deriving anti-aliasing filters to be alied on images before visualizing them on the dislay. Alying such filters will remove moiré artifacts and make masking artifacts less visible. An aroach for designing efficient anti-aliasing filters for multiview dislays is discussed in f y f y f x f x Figure 16, Frequency resonse of the dislay: a) Passband region estimation based on measurements and b) Passband region after filtering. 6. CONCLUSIONS In order to understand the reasons for artifacts in multiview dislay, we modeled the effect of otical layer on the underlying TFT-LCD image. Based on this model, we exlained the reasons for some common artifacts. We identified which visual roerties of a multiview dislay are needed to design image rocessing algorithms for artifact mitigation namely interleaving ern, angular visibility of sub-ixels and dislay erformance in frequency domain. We described a methodology for measuring and modeling these arameters, which does not require recisely ositioned laboratory equiment. The methodology is simle, yet effective, and allows the end user to derive the number of images needed, the algorithm for interleaving them, and the re-filter which would otimize the visual quality of these images for a given multiview dislay. To exemlify our methodology, we resented measurement results for one multiview dislay. The recision of the model what we derived from the measurements is of sufficient recession for using it in visual otimization algorithms 14,19. REFERENCES S. Pastoor, 3D dislays, in (Schreer, Kauff, Sikora, edts.) 3D Video Communication, Wiley, 5. J.-Y. Son and B. Javidi, "Three-Dimensional Imaging Methods Based on Multiview Images," J. Dislay Technol. 1,. 15- (5) J.-Y. Son, B. Javidi, S. Yano, K.-H. Choi, Recent develoments in 3-D imaging Technologies, J. Dislay Technol. 99,. 1-1 (1) N. Dodgson, Autostereoscoic 3D Dislays, Comuter, vol.38, no.8, , Aug. 5, IEEE (5) 13

14 M. Salmimaa, T. Jarvenaa, Otical characterization of autostereoscoic 3-D dislays, J. Soc. Inf. Dislay 16, 85 (8) C. Moller, A. Travis, Correcting Interersective Aliasing in Autostereoscoic Dislays. IEEE Transactions on Visualization and Comuter Grahics 11, (Mar. 5), A. Schmidt and A. Grasnick, "Multi-viewoint autostereoscoic dislays from 4D-vision", in Proc. SPIE Photonics West : Electronic Imaging, vol. 466,. 1-1,. V. Berkel and J. Clarke, Characterisation and otimisation of 3D-LCD module design, in Proc. SPIE Vol. 653, Stereoscoic Dislays and Virtual Reality Systems IV, (Fisher, Merritt, Bolas, edts.), , May 1997 J. Konrad and P. Agniel, "Subsamling models and anti-alias filters for 3-D automultiscoic dislays," IEEE Trans. Image Process., vol. 15, , Jan. 6 M. Salmimaa, T. Jarvenaa, 3-D crosstalk and luminance uniformity from angular luminance rofiles of multiview autostereoscoic 3-D dislays, J. Soc. Inf. Dislay 16, 133 (8) V. Saveljev, J.-Y. Son, B. Javidi, S.-K. Kim, and D.-S. Kim, "Moiré Minimization Condition in Three-Dimensional Image Dislays," J. Dislay Technol. 1, 347- (5) J. Hakkinen, J. Takatalo, M. Kilelainen, M. Salmimaa, and G. Nyman, Determining limits to avoid double vision in an autostereoscoic dislay: Disarity and image element width, J. Soc. Inf. Dislay 17, 433 (9) Zwicker, M., Matusik, W., Durand, F., Pfister, H., and Forlines, C. 6. Antialiasing for automultiscoic 3D dislays. In ACM SIGGRAPH 6 Sketches (Boston, Massachusetts, July 3 - August 3, 6). SIGGRAPH '6. ACM, New York, NY, 17. A. Boev, R. Bregovic, A. Gotchev, K. Egiazarian, Anti-aliasing filtering of D images for multi-view autostereoscoic dislays, in Proc. of The 9 International Worksho on Local and Non-Local Aroximation in Image Processing, LNLA 9, Helsinki, Finland, 9 M. Becker, Dislay reflectance: Basics, measurement, and rating, J. Soc. Inf. Dislay 14, 13 (6) G. Woodgate, J. Harrold, Efficiency analysis for multi-view satially multilexed autostereoscoic -D/3-D dislays, J. Soc. Inf. Dislay 15, 873 (7) W. Mheö, Yi-Pai Huang, and Han-Ping Shieh, Enhancing the Brightness of Parallax Barrier Based 3D Flat Panel Mobile Dislays Without Comromising Power Consumtion, Journal of Dislay Technology, Vol. 6, Issue, (1) M. Krijn, S. de Zwart, D. de Boer, O. Willemsen, and M. Sluijter, -D/3-D dislays based on switchable lenticulars, J. Soc. Inf. Dislay 16, 847 (8) A. Boev, K. Raunio, A. Gotchev and K. Egiazarian, GPU-based algorithms for otimized visualization and crosstalk mitigation on a multiview dislay, Proc. SPIE-IS&T Electronic Imaging 8, Stereoscoic Dislays and Alications XIX, Vol. 683, San Jose, USA, January 8 P. Boher, T. Leroux, T. Bignon, V. Collomb-Patton, A new way to characterize auto-stereoscoic 3D dislays using Fourier otics instrument, in Proc. of SPIE, Stereoscoic dislays and alications XX, 19-1 January 8, San Jose, California, USA R. Hartley, A. Zisserman, Multile View Geometry in Comuter Vision, nd edition, ISBN: , Cambridge University Press, March 6. P. Debevec, J. Malik, Recovering High Dynamic Range Radiance Mas from Photograhs, in Proc. ACM SIGGRAPH,