Aalborg Universitet. Published in: Proceedings of the Working Conference on Advanced Visual Interfaces

Transcription

1 Aalborg Univeritet Software-Baed Adjutment of Mobile Autotereocopic Graphic Uing Static Parallax Barrier Paprocki, Martin Marko; Krog, Kim Srirat; Kritofferen, Morten Bak; Krau, Martin Publihed in: Proceeding of the Working Conference on Advanced Viual Interface DOI (link to publication from Publiher): / Publication date: 2012 Document Verion Early verion, alo known a pre-print Link to publication from Aalborg Univerity Citation for publihed verion (APA): Paprocki, M. M., Krog, K. S., Kritofferen, M. B., & Krau, M. (2012). Software-Baed Adjutment of Mobile Autotereocopic Graphic Uing Static Parallax Barrier. In G. Tortora, S. Levialdi, & M. Tucci (Ed.), Proceeding of the Working Conference on Advanced Viual Interface: AVI 2012 (pp ). Aociation for Computing Machinery. Proceeding of the International Working Conference on Advanced Viual Interface, DOI: / General right Copyright and moral right for the publication made acceible in the public portal are retained by the author and/or other copyright owner and it i a condition of acceing publication that uer recognie and abide by the legal requirement aociated with thee right.? Uer may download and print one copy of any publication from the public portal for the purpoe of private tudy or reearch.? You may not further ditribute the material or ue it for any profit-making activity or commercial gain? You may freely ditribute the URL identifying the publication in the public portal? Take down policy If you believe that thi document breache copyright pleae contact u at vbn@aub.aau.dk providing detail, and we will remove acce to the work immediately and invetigate your claim. Downloaded from vbn.aau.dk on: augut 24, 2018

2 Software-Baed Adjutment of Mobile Autotereocopic Graphic Uing Static Parallax Barrier Martin Paprocki Kim Krog Morten Bak Kritofferen Martin Krau Department of Architecture, Deign and Media Technology, ABSTRACT We how that the autotereocopic diplay of tereocopic image uing a tatic parallax barrier can be improved by adapting the rendering to the angle under which the uer i looking at a mobile diplay; thu, ghoting artifact and depth reveral can often be avoided even if the uer tilt the mobile device. Intead of moving the barrier itelf to compenate for a miplacement of the viewing zone in relation to the uer, we employ dynamic pixel column hift to provide a imilar compenation in oftware. Thi require a parallax barrier where each ection cover two pixel column at a time intead of one. The propoed method ha been implemented uing OpenGL hader and a parallax barrier that wa deigned for a diplay of exactly half the reolution of the employed diplay. Technical tet howed a good eparation of the left and right image for viewing angle of up to ± 30. Preliminary uer tet indicate that an improvement in the tereocopic experience can be achieved. Categorie and Subject Decriptor I.3.7 [Three-Dimenional Graphic and Realim]: Virtual reality General Term Algorithm Keyword Autotereocopy, diplay, parallax barrier, adaptive interface, mobile device, ghoting, context aware interface, GPU Permiion to make digital or hard copie of all or part of thi work for peronal or claroom ue i granted without fee provided that copie are not made or ditributed for profit or commercial advantage and that copie bear thi notice and the full citation on the firt page. To copy otherwie, to republih, to pot on erver or to reditribute to lit, require prior pecific permiion and/or a fee. AVI 12, May 21-25, 2012, Capri Iland, Italy Copyright c 2012 ACM /12/05... $ INTRODUCTION In recent year mobile conumer device tarted to adopt autotereocopic diplay. Although autotereocopic diplay with tatic parallax barrier are a low-cot and competitive olution for diplaying tereocopic image, they are quite limited ince they have to be viewed from a pecific poition. Diplaying the tereo effect correctly require the viewer to remain within a relatively narrow viewing angle, a well a maintaining an optimal viewing ditance to the autotereocopic diplay [12]. If thee condition are not fulfilled, ghoting or revered tereo depth will deteriorate the tereocopic effect. The ghoting i caued by leakage from the left eye image to the right eye image and vice vera. The revered tereo depth occur if the left image i een by the right eye and vice vera [11]. In ome cae thi make it impoible to obtain the correct tereocopic effect, e.g., in game application where the device i tilted a a part of the gameplay. In uch a ituation, one would from time to time ee the correct, a mixed and the revered tereocopic image. Preumably, thi reult in an uncomfortable experience due to the varying depth perception. To overcome ome of the limitation of tatic parallax barrier, we preent a low-cot oftware-baed improvement of autotereocopic diplay uing dynamic pixel column hift (ee Figure 1), which employ tatic parallax barrier with two pixel column per ection of the barrier (ee Figure 2). A tet application wa implemented demontrating how an OpenGL application can alter the pixel arrangement baed on gyrocope and accelerometer data in order to compenate for varying viewing angle. In the next ection, related work i ummarized followed by Section 3 decribing our method along with an implementation of it. In Section 4, the technical tet i dicued and in Section 5, a calibration cene i decribed and Section 6 report reult of a preliminary uer tudy. Concluion and future work are preented in Section RELATED WORK Conumer mobile device with autotereocopic diplay include martphone [7] and game conole [8]. In addition, low-cot parallax barrier which can be added to a mobile de-

3 Figure 1: (a) Correct viewing angle for tereocopy with a parallax barrier. (b) Miplaced eye after tilting. (c) Corrected viewing zone by hifting the pixel column. vice without a built-in autotereocopic diplay are available [2][10]. Solution to ome of the limitation of tatic parallax barrier have been propoed, although mot of them are not purely oftware baed. Perlin et al.[9] have demontrated electrical parallax barrier with dynamic width, which adapt to the uer by mean of camera eye tracking. Recently, LG Electronic [6] ha releaed commercial product uing imilar technology. Lanman et al.[5] have demontrated content adaptive parallax barrier, which allow the barrier to how non-binary opacitie. Mot imilar to our work i a patent by Groman [3] for an autotereocopic diplay uing a tatic parallax barrier. However, thi method i limited to egment that are everal pixel wide and include redundant image information. 3. ADJUSTING THE RENDERING We aume that uer look directly at the mobile device and keep their head till while tilting the device. The rendering on the mobile device hould then be adjuted to the angle between the uer viewing direction and the mobile device; otherwie the uer eye move out of the viewing zone of the autotereocopic diplay. The angular miplacement can be meaured in everal way, for example by uing head tracking or gyrocope and/or accelerometer data. In thi work, we employ gyrocope and accelerometer data in order to keep the implementational and computational cot low. We ugget adapting the rendering to the meaured angle. Specifically, the parallax barrier [2] employed in thi work require an arrangement of alternating pixel column of the image for the left and the right eye. If poitioned correctly, the parallax barrier make ure that each eye ee only the pixel column of the correponding image (ee Figure 1a). If the device i tilted, however, pixel column become viible that are intended for the other eye (ee Figure 1b). A uggeted by Ichinoe et al. [4], we can olve thi problem by hifting pixel column-wie in relation to the meaured angle (Figure 1c). However, hifting pixel column in dicrete tep generate dicontinuou change of the rendering. To avoid thi, an interpolation of the pixel column colour i ued. A the interpolation take effect, the tereocopic image would uually blend together, which would reult in trong ghoting. However, we can avoid thi problem with a creen Figure 2: (a) Standard ue of a parallax barrier. (b) Our ue of a parallax barrier with a creen of twice the reolution. (c) The ytem of pixel column arrangement ued to blend and hift pixel column. of twice the reolution of what the parallax barrier i made for (compare with Figure 2a and 2b) ince thi offer the poibility of tarting with every econd column being black (ee Figure 2b). Thi way, the blending doe not occur within the image themelve, but with the black pixel column; thu, the image are kept eparated. In our implementation (ee Figure 2c), the baeline pixel arrangement i rendered with one column of pixel of the left image (red pixel labelled L in Figure 2c), one column of black pixel, one column of pixel of the right image (green pixel labelled R in Figure 2c) and one more column of black pixel. Thi baeline pixel arrangement i ued for the correct viewing poition, i.e., when the difference of the meaured viewing angle to the optimal viewing angle i 0. The baeline arrangement i alo ued for multiple of 14. Thi particular value wa determined through two type of tet for the particular parallax barrier [2] employed in thi work. For other angle, four different pixel arrangement are ued a illutrated in Figure 2c. The firt pixel arrangement i ued from 0 to 3.5 (= 14 /4). The angle within each pixel arrangement i normalized by a linear mapping to 0 to 1. The normalized angle control the interpolation between pair of pixel column. For example, at 1.75 bothpixel ofeach pair would have the ame colour. At the point where the hift by one pixel i completed, i.e. at 3.5, the firt pixel arrangement top, and the econd pixel arrangement take over. Thi hift the image one pixel to the right. The third and

4 Figure 3: The two erie with and without the column hifting. fourth pixel arrangement are applied in the ame manner. At 14 the firt arrangement tart over again. Due to the effect of the barrier and the interpolation of colour, the perceived intenity varie with the angle. To compenate for thi variation, the intenity i increaed in dependency of the normalied angle with the maximum increae at 0.5 and noincreae at 0 and 1. Moreover, the angle lag behind if the rotation rate i high; therefore, a correction baed on the rotation velocity i added to the meaured angle predicting the actual orientation of the device. 4. TECHNICAL TEST The etup for the technical tet conited of a tet application on a fourth generation ipod Touch [1] runningios and a camera (imulating an eye) directed at it at a ditance of 54.5 cm. To demontrate how the olution perform, the ipod wa rotated from 0 to 30 in tep of 5. In each tep a photograph wa taken. Uing a tet cene with one camera len reveal which one of the two left/right image or what mixture of thoe i viible and would be perceived by the uer. Moreover, a video equence wa filmed to determine how fat one can rotate the device while till obtaining the correct image. Thi wa done with the ame etup and the rotation wa performed manually. Figure 3 how the reult from the technical tet. At 5 the tatic tereo image (i.e. without pixel column hift) ha moved two-third acro the creen to the right. Continuing the rotation reult in variou mixture of the two image, which would reult in trong ghoting artefact. The erie with pixel column hifting how an approximately contant image from 0 to 30. Thereafter the column hifting loe it impact. The video tet focue on the ipod gyrocope rotation rate, where two roll velocitie are taken a reference point. The velocitie are 0.1 rad, which i rather low, and, which i a moderate velocity. At 0.1rad, the column 0.2 rad hifting had ome trouble following the rotation but howed the correct image with a bit of flickering. At 0.2 rad, the column hifting fell behind and one-third of the wrong image hifted to the right during the clockwie rotation and the flickering increaed a well. 5. CALIBRATION SCENE When utilizing parallax barrier, one way to enure highquality tereo image with little or no ghoting in the viewing zone i to enure that the pixel column are only viible from the intended eye. Thi condition can be obtained by etting up a calibration cene, which in our cae conit of red and green column, with every other column being black, ee Figure 4. Figure 4: The tructure of the calibration cene. Figure 5: Photo of the autotereocopic diplay howing the tet cene ued for calibration. The calibration wa et up uch that the correct tereocopic effect i retricted to the center of the creen (ee the red area in Figure 5) a firt tet howed that thi reulted in a clearer tereocopic effect at more comfortable ditance when uing the employed parallax barrier [2]. The calibration enure a uitable ratio between the individual uer eye ditance and the ditance to the creen. Before uing the actual application, the uer are preented with the calibration cene and are aked to cloe one eye while poitioning themelve o that the other eye ee the two color covering half of the creen each. In thi poition, the uer hould then adjut the device uch that one color i in the middle and the other i at the ide, ee Figure 5. The calibration i verified by letting tet ubject open only the previouly cloed eye and confirm that they ee the invere colour. After the calibration proce i concluded and a proper ditance between the uer and the creen i found, the uer can proceed to the actual application. 6. USER TEST A uer tet wa conducted to ee whether uer preferred the adaptive rendering over the non-adaptive rendering. A imple game wa implemented which required the uer to aim at a moving target with a mall crohair by tilting

5 Table 1: Number of tet ubject and their preference. 1t evaluation 2nd evaluation Adaptive hader 7 8 Non-adaptive hader 4 3 Undecided 4 4 Figure 6: Photo of the game ued for the uer tet. the mobile device. Behind the target, ome textured, lowpolygon 3D object were placed to have ome object that the uer could perceive in tereo (ee Figure 6). A bar in the game coniting of green and red column (imilar to the calibration image) wa included to implify the calibration proce. The tet conited of four part: 1. Uing a tereo viewer with perfect eparation, it wa teted whether the tet ubject could perceive tereocopic image at all. Three et of two image were uedwhere eachet conited ofapictureandtheame picture in tereo. The tet ubject wa required to correctly identify all three tereo image. 2. The tet ubject wa hown the calibration image and wa intructed how to calibrate. 3. The tet ubject tried two verion of the game, one with and one without adaptive content and wa aked to tell which one he or he preferred. The uer wa given a limited amount of time with each verion and the equence were randomized between uer. 4. After a re-calibration, the tet ubject wa aked to try the verion again and wa told to focu epecially on the tereocopic effect and then tell which one he or he preferred. The tet ubject could witch between verion and had unlimited amount of time. Table 1 how the reult of tet with 15 ubject. Half of the tet ubject preferred adaptive tereocopy, a fourth of the uer preferred non-adaptive tereo and the lat fourth could not ee a ignificant difference between the two. 7. CONCLUSION AND FUTURE WORK We have preented a novel method in autotereocopy uing oftware-baed adjutment of graphic for tatic parallax barrier on mobile device. We preented an adaptive diplay olution that feature pixel column hift with interpolation. A firt uer tet indicate an enhancement of the uer perception of the tereo effect. The employed GPU hader i only a prototype olution. It performance i very limited ince it ue dicard hader intruction. An improved verion could ue the ame pixel arrangement but employ rendering to texture intead of dicard intruction. Thi would ave ignificant proceing reource and could open the poibility of applying head tracking in combination with the current method. Thi way, the uer could move the head without loing the tereocopic effect a the head tracking would determine how to adapt the pixel arrangement. 8. REFERENCES [1] Apple Inc. Apple - ipod touch - view technical pecification for ipod touch. [2] GRilli3D. Grilli3d home. [3] C. Gromann. Method and device for autotereocopy. US Patent No , October [4] S. Ichinoe, N. Tetutani, and M. Ihibahi. Full-color tereocopic video pickup and diplay technique without pecial glae. Dig. Tech. Paper SID Symp., 3014(4): , [5] D. Lanman, M. Hirch, Y. Kim, and R. Rakar. Content-adaptive parallax barrier: Optimizing dual-layer 3d diplay uing low-rank light field factorization. ACM Tran. Graph., 29(6):10, December [6] LG Electronic. Lg expand it glae-free 3d monitor line-up. November [7] LG Electronic. Lg get connected - lg optimu 3d. 3d/, December [8] Nintendo of America Inc. Nintendo 3d - official webite at nintendo. [9] K. Perlin, S. Paxia, and J. S. Kollin. An autotereocopic diplay. In Proceeding of the 27th annual conference on Computer graphic and interactive technique, SIGGRAPH 00, page , [10] Spatial View Inc. 3deelide. [11] A. van der Enden and H. Spekreije. Binocular depth reveral depite familiarity cue. Science, 244(26): , May [12] Q.-H. Wang, W.-X. Zhao, Y.-H. Tao, D.-H. Li, and R.-L. Zhao. Stereo viewing zone in parallax-barrier-baed autotereocopic diplay. Optik: International Journal for Light and Electron Optic, 121(22): , November 2010.