Virtual Reality ll. Visual Imaging in the Electronic Age. Donald P. Greenberg November 16, 2017 Lecture #22

Transcription

1 Virtual Reality ll Visual Imaging in the Electronic Age Donald P. Greenberg November 16, 2017 Lecture #22

2 Fundamentals of Human Perception Retina, Rods & Cones, Physiology Receptive Fields Field of View Visual Acuity of Resolution Opponent Color Theory Compression Bandwidth Limitations Saccades

3 The Optomotor Cycle

4 Extraocular Muscles Foundations of Sensation and Perception. Mather, George

5 Saccade Control Saccade control is the ability of the eye(s) to move quickly from one fixation point to another ( ms) To obtain a complete picture, normal adults perform 3-5 saccades ( snapshots ) per second Fixation restops are ms

6 Saccadic Motion The eye jumps, comes to rest momentarily (producing a small dot on the record), then jumps to a new locus of interest. - David H. Hubel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.

7 Saccadic Masking Visual saccadic suppression The brain selectively blocks visual processing during eye movements Neither the motion of the eye or subsequent motion blur of the image nor the time gap in visual perception is noticeable to the viewer

8 Saccadic Masking There are two major types of saccadic masking or suppression Flash suppression is the inability of the light to see a flash of light during a saccade Suppression of image displacement is characterized by the inability to perceive whether a target has moved during a saccade.

9 Peak Angular Velocity Wikipedia

10 Human Depth Perception Depth Perception Oculomotor Visual Binocular Monocular Binocular Monocular Convergence Accommodation Stereopsis Static Cues Motion Parallax Perspective Familiarity, Relative Size Motion, Position Occlusion Texture Gradient Shading, Shadows, Highlights Atmospheric Blur

11 Monocular Human Depth Perception Depth Perception Oculomotor Visual Binocular Monocular Binocular Monocular Convergence Accommodation Stereopsis Static Cues Motion Parallax Perspective Familiarity, Relative Size Motion, Position Occlusion Texture Gradient Shading, Shadows, Highlights Atmospheric Blur

12 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shadows and Specular Highlights Atmospheric Blur

13 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shadows and Specular Highlights Atmospheric Blur

14 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shadows and Specular Highlights Atmospheric Blur

15 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shadows and Specular Highlights Atmospheric Blur

16 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shading, Shadows, and Specular Highlights Atmospheric Blur Viewpoint A Viewpoint B Viewpoint C

17 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shading, Shadows, and Specular Highlights Atmospheric Blur

18 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shading, Shadows, and Specular Highlights Atmospheric Blur

19 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shadows and Specular Highlights Atmospheric Blur

20 Monoscopic Depth Cues Perspective Depth from Motion, Relative Size, Position, Familiarity Occlusion Texture Gradient Parallax from Motion Shadows and Specular Highlights Atmospheric Blur Accommodation Note change in lens shape

21 Accommodation This is the process by which the vertebrate eye changes optical power to maintain a clear image or focus on an object as its distance varies.

22 Accommodation The reflex can be controlled but cannot be felt Accommodation amplitude declines with age

23 Vergence The simultaneous movement of the pupils of the eyes toward or away from one another during focusing. This measure of the convergence or divergence of a pair of light rays is defined as vergence.

24 Vergence Accommodation Conflict

25 Human Depth Perception Depth Perception Oculomotor Visual Binocular Monocular Binocular Monocular Convergence Accommodation Stereopsis Static Cues Motion Parallax Perspective Familiarity, Relative Size Motion, Position Occlusion Texture Gradient Shading, Shadows, Highlights Atmospheric Blur

26 Binocular Vision Binocular Vision, which is the basis for stereopsis is important for depth perception and covers 114 degrees (horizontally) of the human visual field. The remaining sixty to seventy degrees have no binocular vision (because only one eye can see those portions of the visual field)

27 Stereoscopic Depth Cues Stereopsis Horizontal Parallax Occlusion Revelation Shape Change Convergence

28 Stereoscopic Depth Cues Stereopsis Shape Change Standard Stereo HypoStereo (Gigantism) HyperStereo (Dwarfism) Convergence Standard HypoStereo HyperStereo

29 Stereoscopic Depth Cues Stereopsis Shape Change Convergence Maintain single binocular vision Fusion Vision and Visual Disabilities An Introduction, by Gerd Waloszek, SAP User Experience

30 Moore s Law Chip density doubles every 18 months. Processing Power (P) in 15 years:

31 Exponential Laws of Computing Growth

32 War for the Planet of the Apes

33 Off-line/On-line RRRRRRRRRR 10 hoooooooo 10 mmmmmmmmmmmmmmmmmmmmmmmm = (10 hrrrr.)(60 mmmmmm hrr = /eye = ssssss )(60 mmmmmm )(1,000mmmm 10 mmmm ssssss )

34 Potential Improvements Eye tracking and foveal rendering Multi resolution displays Monoscopic vs. stereoscopic level of detail Asynchronous spacewarp

35 Increasing Densities (ppi) of OLED Displays

36 Field of View of the Human Eye Wikipedia

37 Foveal Eye Tracking Constraints 2016 Speed- needs to be fast enough to meet update requirements (currently 11 milliseconds, 90 Hz) Accuracy- Gaze direction is < 0.5 degree Foveal direction accuracy can be ~ 1.0 arc minutes (1/60 of a degree) Non-invasive measurements- still need to see entire visual field

38 Eye Tracking

39 Sensing Methods: Retinal Tracking Hard problem with current technologies Extremely difficult to illuminate Must bounce light off of retina Light comes back through iris Light must be extremely bright Too much exposure will damage retina Typically done in ophthalmological setting Presently can only detect faint images of blood vessels, companies working on it Very high angular resolution, but would presently require occlusion of vision

40 Purkinje Reflections

41 Purkinje Reflections Cornsweet and Crane 1973

42 Purkinje Reflections

43 1 st and 4 th Purkinje Reflections

44 No Foveated Rendering Roadtovr.com

45 Contemporary blur foveated rendering Roadtovr.com

46 NVIDIA s contrast preserving rendering Roadtovr.com

47 Research on Foveated Displays

48 Alaskan Moose Diorama

49 Dall Sheep Restoration

50 Alaska Brown Bear Diorama

51 Plan of Typical Diorama

52 LOD Image Based Primitives Layered Depth Images

53 Space & Time Warping When frame rates are not met, there are several types of solutions, but all have their deficiencies Visual artifacts appear because of loss of accuracy e.g. Imperfect extrapolation, Object Disocclusion trails

54 Asynchronous Spacewarp

55 Asynchronous Spacewarp

56 Asynchronous Spacewarp

57 Asynchronous Spacewarp

58 Asynchronous Spacewarp

59 End