! Head Mounted Display Optics I! Gordon Wetzstein! Stanford University! EE 267 Virtual Reality! Lecture 7! stanford.edu/class/ee267/!!
! Logistics! HW3 is probably the longest homework, so get started asap if you have not done so already! What is MAR (minimum angle of resolution)?! Conflicting definitions of whether it s the visual angle of 1 pixel or 1 cycle (=2 pixels) for consistency, we use the same definition as in Guenther et al., which is 2 pixels! all hardware for HW4 will be given out in lab/office hours on Friday, 10-11:30am, Packard 001!
Lecture Overview! 1. stereo rendering for HMDs! 2. lens distortion correction using GLSL!
1. Stereo Rendering for HMDs! All Current-generation VR HMDs are Simple Magnifiers!
Image Formation! HMD! micro display! lens! Side View!
Image Formation! HMD! micro display! h' lens! f d ' d eye Side View! eye relief!
virtual image! Image Formation! HMD! h world origin is in the center of the virtual image!! micro display! h' lens! f d ' d Side View! d eye eye relief!
virtual image! Image Formation! h Gaussian thin lens formula:! 1 d + 1 d ' = 1 f d = 1 1 f 1 d ' micro display! h' HMD! lens! f d ' d Side View! d eye eye relief!
virtual image! Image Formation! h Gaussian thin lens formula:! 1 d + 1 d ' = 1 f Magnification:! M = f f d ' d = 1 1 f 1 d ' h = Mh' micro display! h' HMD! d ' lens! f d Side View! d eye eye relief!
virtual image! Image Formation! view frustum symmetric! h y z world origin is in the center of the virtual image!! d Side View! d eye eye relief!
virtual image! Image Formation! view frustum symmetric! near clipping plane! h y z d Side View! d eye eye relief!
virtual image! Image Formation! h y z view frustum symmetric! top! bottom! near clipping plane! z near d Side View! d eye eye relief!
virtual image! Image Formation! h similar triangles:! top = z near bottom = z near h ( ) 2 d + d eye h ( ) 2 d + d eye view frustum symmetric! top! bottom! near clipping plane! z near d Side View! d eye eye relief!
Image Formation! HMD! ipd! w' d ' d eye Top View! eye relief!
Image Formation Left Eye! HMD! w' 2 ipd/2! d eye Top View! eye relief!
virtual image! Image Formation Left Eye! HMD! w 1 x z ipd/2! w 2 d Top View! d eye eye relief!
virtual image! Image Formation Left Eye! w 1 x w 1 = M ipd 2 HMD! z ipd/2! w 2 w 2 = M w' ipd 2 d d eye eye relief! Top View!
virtual image! Image Formation Left Eye! w 1 x view frustum asymmetric! near clipping plane! z near z w 2 d d eye eye relief! Top View!
virtual image! Image Formation Left Eye! w 1 x view frustum asymmetric! near clipping plane! z near z right! left! w 2 d d eye eye relief! Top View!
virtual image! Image Formation Left Eye! w 1 similar triangles:! right = z near w 1 d + d eye view frustum asymmetric! near clipping plane! z near right! w 2 left = z near w 2 d + d eye d left! d eye eye relief! Top View!
w 2 virtual image! x w 2 = M Image Formation Right Eye! w' ipd 2 view frustum asymmetric! w 1 z w 1 = M ipd 2 d Top View! d eye eye relief!
w 2 virtual image! similar triangles:! Image Formation Right Eye! right = z near w 2 d + d eye view frustum asymmetric! right! z near left! w 1 left = z near w 1 d + d eye d Top View! d eye eye relief!
View Matrix - Lookat! center point (right eye)! eye position (right eye)! h ipd! center point (right eye)! d Top View! eye position (left eye)! d eye eye relief!
Prototype Specs View-Master Deluxe VR Viewer!! roughly follows Google Cardboard 2.0:! lenses focal length: 40 mm! lenses diameter: 34 mm! inter-lens distance: 64 mm! screen to lens distance: 39 mm! eye relief: 18 mm! Topfoison 6 LCD: width 132.5 mm, height 74.5 mm; either 1920x1080 pixels (1080p) or 2650x1440 pixels (2K)!
! Image Formation! use these formulas to compute the perspective matrix in WebGL! you can use:! THREE.Matrix4().makePerspective(left,right,top,bottom,near,far) THREE.Matrix4().lookAt(eye,center,up) that s all you need to render stereo images on the HMD!
Image Formation for More Complex Optics! especially important in free-form optics, off-axis optical configurations & AR! use ray tracing some nonlinear mapping from view frustum to microdisplay pixels! much more computationally challenging & sensitive to precise calibration; our HMD and most magnifier-based designs will work with what we discussed so far!
2. Lens Distortion Correction! All lenses introduce image distortion, chromatic aberrations, and other artifacts we need to correct for them as best as we can in software!
Lens Distortion! grid seen through HMD lens! lateral (xy) distortion of the image! chromatic aberrations: distortion is wavelength dependent!! image from: https://www.slideshare.net/mark_kilgard/nvidia-opengl-in-2016
Lens Distortion! Pincussion Distortion! Barrel Distortion!
Lens Distortion! Pincussion Distortion! optical! Barrel Distortion! digital correction!
image from: https://www.slideshare.net/mark_kilgard/nvidia-opengl-in-2016 Lens Distortion!
Lens Distortion! x u, y u undistorted point!
Lens Distortion! x u, y u undistorted point! ( ) ( ) x d x u 1+ K 1 r 2 + K 2 r 4 y d y u 1+ K 1 r 2 + K 2 r 4 ( x c, y c ) x d, y d K 1,K 2 r x c, y c distorted point coordinates! distortion coefficients! normalized distance from center! center of optical axis! Barrel Distortion! digital correction!
Lens Distortion! x d, y d K 1,K 2 r x c, y c x u, y u undistorted point! ( ) ( ) x d x u 1+ K 1 r 2 + K 2 r 4 y d y u 1+ K 1 r 2 + K 2 r 4 distorted point coordinates! distortion coefficients! normalized distance from center! center of optical axis! NOTES:! center is assumed to be the center point (on optical axis) on screen! distortion is radially symmetric around center point! easy to get confused!! can implement in fragment shader (not super efficient, but easier for us)!
Normalizing r! x u, y u undistorted point! ( ) ( ) x d x u 1+ K 1 r 2 + K 2 r 4 y d y u 1+ K 1 r 2 + K 2 r 4 un-normalized radial distance from center:! ( ) 2 + ( y u y c ) 2 r! 2 = x u x c x c, y c center! r! Calculate in metric units, e.g. mm. Need physical size of the pixels of your screen for this!
virtual image! Normalizing r! r! r = r! d in mm mm view frustum symmetric! r! distance of a pixel from center point in mm! d distance to lens in mm!! r normalized, unit-less distance! that we use for distortion!! d Side View!
Lens Distortion Center Point! x c, y c right eye! h ipd! x c, y c left eye! d Top View! d eye eye relief!
Lens Distortion Correction Example! stereo rendering without lens distortion correction!
Lens Distortion Correction Example! stereo rendering with lens distortion correction!
How to Render into Different Parts of the Window?! WebGLRenderer.setViewport(x,y,width,height) x,y lower left corner; width, height viewport size! width2 height2 height1 (x2,y2) (x3,y3) height3 (x1,y1) width1 width3
Microdisplays!
Liquid Crystal Display (LCD) - Subpixels! IPS! wikipedia! S-IPS! IPS! TN subpixels!
wikipedia! LCD!
LCD Backlight! wikipedia! extremetech.com!
Liquid Crystal on Silicon (LCoS)! basically a reflective LCD! standard component in projectors and head mounted displays! used e.g. in google glass!
Organic Light Emitting Diodes (OLED)! lightrabbit.co.uk!
Digital Micromirror Device (DMD)! developed by Texas Instruments! MEMS device! binary states (e.g. +/- 10 degrees)! gray-level through pulse width modulation (PWM)! Texas Instruments!
B. T. Schowengerdt, R. Johnston, C.D. Melville, E.J. Seibel. 3D Displays Using Scanning Laser Projection. SID 2012.!
Next Lecture: HMD Displays Optics II! advanced VR & AR optics! microdisplays! drawing from Google Glass patent!