COMS W4172 Perception, Displays, and Devices 4

Transcription

1 COMS W4172 Perception, s, and Devices 4 Steven Feiner Department of Computer Science Columbia University New York, NY February 22, Stereoscopic Viewing Active Temporal multiplexing Liquid crystal shutter eyewear Can support multiple users at decreased fps L1 L2 R1 R2 CrystalEyes LC shutter eyewear Two-user stereo workbench Left user s view Right user s view Right user s view seen from left M. Agrawala et al., The Two User Responsive Workbench, SIGGRAPH

2 Stereoscopic Viewing Active Temporal multiplexing Liquid crystal shutter eyewear Combined with spatial multiplexing to support multiple users at full fps Kitamura et al., IllusionHole Two-user stereo workbench Left user s view Right user s view Right user s view seen from left Y. Kitamura et al., 3 Stereoscopic Viewing Head-Worn Binocular head-worn displays (HWDs) AKA head-mounted displays (HMDs) Head-tracked Private Opaque or see-through Sutherland (worn by Vickers), ca Saab, ca Meta,

3 Classification of AR s R. Raskar (MIT) projector retinal display headworn display hand-held display spatial see-through display projector Head-attached Body-attached Spatial 5 Classification of AR s S. Feiner Optical see-through Projection Optics optics Video see-through environment Camera(s) computer can be Worn Held In environment 6

4 See-Through Head-Worn s Optical see-through Optics optics Video see-through Vuzix Star 1200XLD Camera(s) computer Vuzix Wrap 1200DXAR 7 Optical See-Through s Optics optics: beam splitter = combiner Real world is full resolution/fov, but virtual world cannot occlude real world virtual/real worlds accommodate at different distances Vuzix Star 1200XLD 8

5 Optical See-Through s History Victorian stage effect: Pepper s Ghost H. Dircks and J.H. Pepper, 1862 Earlier versions described in 16th C. 9 Optical See-Through s History Victorian stage effect: Pepper s Ghost Example at George C. Page Musum, La Brea Tar Pits 10

6 Video See-Through s Camera(s) Vuzix Wrap 1200DXAR Real world is lower resolution/fov, but virtual world can occlude real world virtual/real worlds accommodate at same distance Cameras need to have same view as user for zero parallax Note: view FOV 11 Video See-Through s Achieving a zero-parallax, orthoscopic display Simulator application models HWD optics A. State, K. Keller, and H. Fuchs, ISMAR 2005 Orthoscopic = no optical distortion 12

7 Video See-Through s Approaches to Combining Real & Chromakeying (originally done with analog video hdw) Use reserved background color for each pixel in output image { Using an RGB camera if pixel in graphics image background output pixel in graphics image else output pixel in camera image Or blend graphics and camera images in then clause Note: Swap graphics and camera for classic use in TV weather Alternatively, use a separate alpha channel value at each pixel determines amount to blend, with no need for a reserved color 13 Video See-Through s Approaches to Combining Real & Rendering over camera image Fill frame buffer f with camera image Set z-buffer z to farthest possible value for each pixel p in each graphics object { Using an RGB camera z(p) = z of pixel coords (x, y) if z(p) not farther than z[x, y] { // update z[x, y] and draw pixel p z[x, y] = z(p) f[x, y] = color of pixel coords (x, y) Or blend graphics and camera images in then clause Can also use an alpha buffer z-buffer of depth values (Brighter = closer) 14

8 Video See-Through s Approaches to Combining Real & Full visible-surface determination Fill frame buffer f with camera image Fill z-buffer z with depth camera image Using an RGBD (RGB + Depth) camera for each pixel p in each graphics object { z(p) = z of pixel coords (x, y) if z(p) not farther than z[x, y] { // update z[x, y] and draw pixel p z[x, y] = z(p) f[x, y] = color of pixel coords (x, y) Or blend graphics and camera images in then clause Can also use an alpha buffer Microsoft Kinect v2 pmd technologies CamBoard pico XS 15 See-Through Head-Worn s Binocular (stereoscopic) Different image to each eye Saab AddVisor 150 Monocular (monoscopic) One image to one eye Biocular (same image to both eyes) Liteye 16