Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática. Output Devices - I

Transcription

1 Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática Output Devices - I Realidade Virtual e Aumentada 2016/2017 Beatriz Sousa Santos

2 What is Virtual Reality? A high-end user interface that involves real-time simulation and interaction through multiple sensorial channels. (vision, sound, touch, smell, taste) (Burdea and Coiffet., 2003) 2

3 Output Devices: Graphics, 3-D Sound, and Haptic Displays (Burdea and Coiffet., 2003) 3

4 The human senses need specialized interfaces Graphics displays for visual feedback 3-D audio hardware for localized sound Haptic interfaces for force and touch feedback Only a few examples of experimental devices providing smell olfactory and taste feedback 4

5 The ultimate display? "The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. (Ivan Sutherland, 1965) 5

6 Visual Displays A graphics display is a computer interface that presents synthetic world images to one or several users interacting with the virtual world. (Burdea and Coiffet., 2003) Personal displays: Main technologies: - HMDs (VR/AR) - Binoculars LEDs/OLEDs - Monitor-based displays/active glasses LCDs - Autostereoscopic displays lenticular/barrier Large volume displays: Workbenches Caves projectors Walls, domes Surface 6

7 Human Visual System And depth perception Vision is the dominant sensorial channel (Burdea and Coiffet., 2003) Depth perception in mono images is based: - on occlusion (one object blocks another from view) - shadows - textures - motion parallax (closer images appear to move more than distant ones) 7

8 Depth perception in stereo is based on steropsis (when the brain registers and fuses two images) Image parallax means that the two eyes register different images (horizontal shift) The amount of shift depends on the inter-pupillary distance (IPD) (varies for each person in the range of mm) Works in the near field (to a few meters from the eye) Left eye, right eye images 8

9 Many of the perceptual cues we use to visualize 3D structures are available in 2D projections Such cues include: occlusion (one object partially covering another) perspective (point of view) familiar size (we know the real-world sizes of many objects) atmospheric haze (objects further away look more washed out) Four cues are missing from 2D media: stereo parallax seeing a different image with each eye movement parallax seeing different images when we move the head accommodation the eyes lenses focus on the object of interest convergence both eyes converge on the object of interest 9

10 All 3D display technologies (stereoscopic displays) provide at least stereo parallax Autostereoscopic displays provide the 3D image without needing any eyewear Volume displays provide a real 3D image Item is no longer available 10

11 Stereopsis object Projection plane Stereo ="solid" or "three-dimensional opsis = appearance or sight eyes 'binocular vision, 'binocular depth perception', 'stereoscopic depth perception' Stereopsis is the impression of depth that is perceived when a scene is viewed with both eyes by someone with normal binocular vision Binocular disparity is due to the different position of our two eyes 11

12 (Hearn and Baker, 2002) object Right eye image Left eye image Projection plane eyes 12

13 In 1838 Charles Wheatstone published an explanation of stereopsis due from differences in the horizontal positions of images in the two eyes Béla Julesz invented in the 1960s the Random Dot Stereograms This led to the invention of the autostereogram: single images to be viewed without the stereoscope Pocket stereoscope and stereogram ( Random dot stereogram (Rock, 1984) Random_dot_stereogram 13

14 Autostereograms Are designed to create the visual illusion of a 3D scene from a 2D image There are two ways of viewing: - wall-eyed - cross-eyed When viewed in the opposite way the shape perceived in 3D is reversed Appear to "pop up" (or "pop down") out of the printed paper 14

15 We should be able to see the word STEREO 15

16 Wiggle stereoscopy Simple stereogram viewing technique; no glasses are needed Alternates between the left and right images of a stereogram Most people can get a crude sense of dimensionality due to persistence and parallax 16

17 Wiggle stereoscopy 3D Effect? None Slight Strong 17

18 3D with moving mages: the Pulfrich effect 3D effects with nothing more than "normal" video To experience the Pulfrich Effect use a pair of sunglasses Look through one of the dark lenses with one eye and nothing through the other Place the dark glass on the leading side of the motion In this case: dark glass on left Be sure to look at the screen with both eyes 19

19 3D Effect? None Slight Strong 20

20 Pulfrich effect The photoreceptors in the retina respond more sluggishly when the scene is less bright The image viewed through the colored lens is slightly outdated A moving object, viewed with one eye brighter than the other, appears either in front of or behind the true track, depending upon direction of motion

21 Implications for Stereo Viewing devices Need to present two images of the same scene The two images can be presented: at the same time on two displays (HMD) time-sequenced on one display (active glasses) spatially-sequenced on one display (auto-stereoscopic displays) Left eye, right eye images (Burdea and Coiffet., 2003) 22

22 Common ways to produce a 3D sensation Anaglyphs: two colored images and color coded glasses (red/cyan(green)) Two images with different light polarization and polarizing glasses Linear and circular Double frame-rate displays combined with LCD shutter glasses Autostereoscopic displays Parallax barrier and lenticular lens Head Mounted Displays (HMDs) and exotic displays 23

23 All show the right image to the right eye and the left image to the left eye! All these technologies provide: stereo parallax When combined with head tracking, they can provide movement parallax for a single viewer 24

24 Anaglyph images Cyan/ red image pairs (or other opposite color pairs) that can be viewed through colored coded glasses Cyan right eye image Red left eye image 25

25 3D Movies? 26

26 Put on your glasses! 27

27 Advantages of using cyan/red glasses: are generally inexpensive don't require any power don't require synchronization with the display do not suffer from flicker Disadvantages: Inaccurate color rendering Headaches after a long exposure 28

28 Vertical polarizer Polarized light stereoscopy 3D movies have used polarized technology since the 1930s Verical polarization of light Two images are projected on the same screen through different polarizing filters Gray linear-polarizing filters are easily manufactured, thus correct color rendition is possible Two types of polarization can be used: Linear Circular Circular polarized light wiki/polarized_3d_glasses 29

29 Advantages of polarized glasses: are generally inexpensive don't require any power don't require synchronization with the display do not suffer from flicker Disadvantages: Technologies/zalman-trimon-range The images for polarized glasses may have to share the screen simultaneously, and therefore cannot have full resolution There are incompatible polarized systems (circular or linear polarized) The head should not be tilted to maintain the 3D effect with linear polarization 30

30 Shutter glasses for stereoscopic displays Active-shutter glasses are small LCD screens that alternately dim the left and right "lenses" in succession They are synchronized with the display usually through IR or radio signals Each eye can see the image intended for it Can be bought for < 100 Need a battery 31

31 Passive (polarized) versus active (shutter)? Passive Active cheaper lighter batteryless syncless glasses Better image quality 32

32 Visual Displays A graphics display is a computer interface that presents synthetic world images to one or several users interacting with the virtual world. (Burdea and Coiffet., 2003) Personal displays: Main technologies: - HMDs (VR/AR) - Binoculars LEDs/OLEDs - Monitor-based displays/active glasses LCDs - Autostereoscopic displays lenticular/barrier Large volume displays: Workbenches Caves projectors Walls, domes Surface 33

33 Main technologies of Visual Displays - LCD displays - OLED displays - Autostereoscopic displays: lenticular/barrier - Projectors 34

34 LCDs (Liquid Crystal Displays) Layers in a LCD States of a LCD Colour LCD 35

35 OLEDs advantages Robust Design Viewing Angles High Resolution Electronic Paper Production Advantages Video Capabilities Hardware Content Power Usage 37

36 Main Properties of Visual Displays Visual presentation properties: Color Spatial resolution Contrast Number of display channels Focal distance Opacity Masking Field of view (FOV) Field of regard (FOR) Head position information Graphics latency tolerance Temporal resolution Logistic Properties: User mobility Interface with tracking Environment requirements Associability with other sense displays Portability Throughput Encumbrance Safety Cost 39

37 Focal distance Focal distance is the measurement between the participant's eyes and the virtual image. (A) projection display - distance to the display screen. (B) HBD create a virtual image at some distance beyond the physical display 40

38 Field of view (FOV) and Field of regard (FOR) FOV is the amount of the viewer's visual field covered by a display. Head-based displays (A) tend to have smaller, fixed FOV angles compared with those possible in projectionbased displays (Sherman and Craig, 2003) FOR is a measure of the amount of coverage a given display provides when head motion and other factors are considered. (A) Head-based displays can easily provide a loo% FOR, (B) stationary displays are limited to the area of the screens 41

39 Visual Displays: a possible taxonomy (Burdea and Coiffet, 2003) Personal displays: - HMDs (VR/AR) - Binoculars - Monitor-based displays/active glasses - Autostereoscopic displays Large volume displays: Workbenches Caves Walls, domes Surface 43

40 Visual Displays: another taxonomy (Sherman and Craig, 2003) Head-based (occlusive) Non-occlusive head-based Handheld Monitor- based (Fishtank) Projection Displays Stationary displays 44

41 Personal Displays A Visual display that outputs a virtual scene destined to be viewed by a single user. Such image may be monoscopic or stereoscopic, monocular (for a single eye) or binocular (displayed on both eyes). Head Mounted Displays (HMDs) 3-D Binoculars (hand supported) Auto-stereoscopic displays (desk supported) 45

42 HMD integration in a VR system simple HMD More professional HMD (Burdea and Coiffet., 2003) 47

43 zsight OLED professional HMD (Sensics) - Bright, crisp, high-contrast OLED display - Stereo - High resolution full-color SXGA pixels - 60º field of view - Integrated yaw/pitch/roll tracker - Self contained, lightweight (450 g) - Integrated high-quality stereo audio and microphone - Price ~$

44 Vuzix 920AR MaxReality Twin high-resolution 640 x 480 LCD displays Equivalent to a 67-inch screen (at ~ 3 m) 60 Hz progressive scan update rate Ultra-low video distortion 31-degree diagonal field of view 24-bit true color (16 million colors) Independent +2 to -5 diopter focus adjustment Price: ~$1700 Two discrete VGA (640 x 480) video cameras 30 frames/s video capture 640 x 480 USB video camera no proprietary drivers required 6 DOFs tracker yaw, pitch, roll, and x, y, z 3-axes magneto-resistive sensors 3-axes accelerometers 3-axis gyros Auto re-centering to adjust from drift Includes tracker and gyro calibration software _wrap920ar.html 49

45 Personal 3D Viewer HMZ-T2 Display Device - OLED Panel 2 Display Resolution 1280 x 720 Horizontal viewing angle- 45º Aspect Ratio Gradation - RGB 24bit Price ~1000$ 50

46 Oculus Rift 2014, DK2: - low persistence OLED display to eliminate motion blur and judder (two of the biggest contributors to simulator sickness) - It also makes the scene appear more visually stable, increasing the potential for presence pixels per-eye display improves clarity, color, and contrast. Price: ~~$

47 Google Glass (2013) Display - reflects light into the wearer's eye Camera photos and video Touchpad on the side Micro voice control 1500 $ glass/develop/overview

48 Google glass (2015) 54

49 Meta Glasses 3D See Through Display - Resolution: 960 x 540 pixels per eye - FOV : 35 degree field of view Cameras - 3D Time-of-flight depth camera - Color (RGB) Camera Head Tracking degree tracking - Accelerometer, gyroscope and compass Audio - Dolby 3D audio - Two built-in microphones 55

50 Hololens Microsoft AR glasses announced for

51 Virtual Binoculars Handheld, interactive, immersive displays combine high-resolution microdisplays with adjustable, wide field-of-view optics Have a familiar form (pair of binoculars) May have a tracker as accessory Examples: Virtual Binocular SX Virtual Binocular SV (Burdea and Coiffet., 2003) 58

52 Virtual Binocular SX Price: $20,900 Virtual Binocular SV Price: $7,900 60

53 Auto-stereoscopic displays Two technologies: lenticular barrier Do not require use of special glasses Allow several vantage points Passive - do not track user s head (restrict user s position) Active - track the head motion (give more freedom) 62

54 Autostereoscopic displays found practical uses in applications: in which 3D depth perception is vital: e.g. - scientific/medical visualization of complex 3D structures - remote robot manipulation in dangerous environments where the novelty of stereo parallax is a selling point e.g. - computer games - advertisement 63

55 Lenticular: an array of cylindrical lenslets is placed in front of the pixel raster lenslets direct the light from adjacent pixel columns to different viewing slots at the ideal viewing distance Each of the viewer s eyes sees light from only every second pixel column 64

56 Parallax barrier: a barrier mask is placed in front of the pixel raster Each of the viewer s eyes sees light from only every second pixel column 65

57 Virtual Window 19 (2001) Key Features: - Large 19 diagonal screen - 3D mode without glasses - Digital (DVI-D) and analog (VGA) inputs x 1024 resolution - Super fast 8ms response time - Price ~$4000 Optional Video Expansion Box 66

58 Sharp autostereoscopic laptop - 15 diagonal display x768 resolution - 2D and 3 D mode - uses parallax barrier Item is no longer available 68

59 PHILIPS BDL5231VS 1080P 52'' LCD FLAT (2015) Autostereoscopic 3D Full HD LCD display, 1920 x 1080p High brightness for clearer images SmartPower for energy saving ~ 9000 euros 69

60 Holographic displays The image source is based on standard flat panel technology of which the image is seen upon a nine optical layer glass panel Objects appear to float in space For the maximum 3D effect, the background seen through the display should be several feet behind the display and dark in color 70

61 Large Volume Displays Allow several co-located users to view a monoscopic or stereoscopic view of the virtual world - monitor-based large volume displays - projector-based large volume displays Allow more freedom of motion vs. personal displays. 71

62 Monitor-based Large Volume Displays Use active or passive glasses Several users can look at a monitor Can have a single monitor, or multiple side-by-side monitors If side-by-side, image continuity becomes an issue 72

63 Projector based Large-volume displays Workbanch-type displays CAVE type displays Wall-type displays Domes 73

64 Tilted surface Viewing Cone Reflector mirror Floor CRT projector (not shown) The old Fakespace ImmersaDesk workbench 75

65 Workbench-type display geometries Baron Item is no longer available V-desk 76

66 CAVEs CRT Projector Screen Mirror CAVE 3-D large volume display (Fakespace Co.) (Plato s allegory of the cave, where a philosopher contemplates reality and illusion...) 77

67 Fakespace Systems Inc., a world leader in immersive visualization systems, announced the commercial availability of a digital version of the CAVE(R) at SIGGRAPH Digital projection provides: - extra clarity and - richer colours CAVE 3-D large volume display (Fakespace Co.) 78

68 Enter the CAVE /Cave/Cave%20Diagram% pdf

69 CAVE 2 TM near-seamless, 320-degree, panoramic 2D/3D virtual environment that matches human visual acuity... The CAVE2 is a revolutionary system that supports information-rich analysis with stunning immersive visuals and intuitive interaction tools /cave2.aspx

70 Wall-type displays Accommodate several users A single projector on a large wall means small image resolution Tiled displays place smaller images side-by-side so they need multiple projectors Images need to have overlap, to assure continuity However overlap from two projectors means intensity discontinuity (brighter images in the overlap areas) Projectors need to modulate intensities to dim their light for overlap pixels 81

71 FLEX TM - Reconfigurable immersive display Widely installed immersive reconfigurable visual environment Adequate for applications that will benefit from more than one display configuration (there is a mobile version) Walls can be moved independently to create new formats (7.5' x 10' ) Has three walls and one floor screen In < 5 min can become: - A flat wall display - An angled theater, - An L-shape - A CAVE-like immersive room

72 Wall-type displays Typical uses: Multi-disciplinary research Product design reviews Large audience presentations Computational fluid dynamics Geophysical exploration and training Terrain mapping and situational awareness Any application where multiple display modes would be beneficial 84

73 Dome-type display Immersadome - 3D immersive visual environment - Up to 20 people (depending on dome size) - Resolution: 1400 x horizontal x 135 vertical (distortion free projection) - Multiple control options (mouse, keyboard, joystick, tracker-ball, haptic) - Applications: - museums, - group training - flight or driver simulation 85

74 87

75 Tiled composite image from four projectors 88

76 Tiled composite image from four projectors after adjustment 89

77 Dome-type displays Advantages: Accommodate many users (tens to hundreds) Give users more freedom of motion immersive-domes.aspx Disadvantages: Large cost (up to millions of dollars) Even with several displays resolution is low (larger area) 90

78 Dome type displays VisionDome 5(m); 2 to 20 users; price > $85,

79 Visual Displays: another taxonomy (Sherman and Craig, 2003) Head-based (occlusive) Non-occlusive head-based Handheld Monitor- based (Fishtank) Projection Displays Stationary displays 93

80 Benefits of Stationary Displays (Fishtank and Projection) Higher resolution (than most HMDs) Wider field of view Longer user endurance (i.e., can stay immersed for longer periods) Higher tolerance for display latency Greater user mobility (fewer cables) Less encumbering Lower safety risk Better for group viewing Better throughput 94

81 Benefits of Head-based Displays (Occlusive and Non-occlusive) Lower cost (for lower resolution models) Complete field of regard Greater portability Can be used for augmenting reality Can occlude the real world Less physical space required Less concern for room lighting and other environmental factors Benefits of Hand-based Displays Greater user mobility Greater portability Can be combined with stationary VR displays 95

82 Main bibliography - G. Burdea and P. Coiffet, Virtual Reality Technology, 2 nd ed. Jonh Wiley and Sons, Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications: Foundations of Effective Design, Morgan Kaufmann, J. Vince, Introduction to Virtual Reality, Springer,

83 Aknowledgement The author of these slides is greteful to: G. Burdea and P. Coiffet for making available the slides supporting their book All colleagues and students that have contributed in any way 97