Dynamic Light Sculpting: Creating True 3D Holograms With GPUs

Size: px

Start display at page:

Download "Dynamic Light Sculpting: Creating True 3D Holograms With GPUs"

Bennett Goodwin
5 years ago
Views:

1 Dynamic Light Sculpting: Creating True 3D Holograms With GPUs TM Official partner Key innovator in a volumetric sector worth 2bn according to MPEG committee on Immersive Media contributor

2 From Augmented to Mixed Reality Mobile AR HMD AR Mixed Reality Increasing Immersion Information overlay Full virtual object integration

Stereo displays do not generate full 3D effects.

Stereoscopic displays put strain on the users by

leads to eye strain and fatigue As users eyes are

3 Stereo displays do not generate full 3D effects. Nintendo 3DS 3D cinema and TVs Mixed reality headsets Stereoscopic displays put strain on the users by displaying two 2D images and using glasses to create an image. Visual cue mismatch of convergence and accommodation leads to eye strain and fatigue As users eyes are constantly adopting to different depths, stereo displays are extremely uncomfortable to use for a longer period

4 HOLOGRAPHY

5 A hologram is an interference pattern, that is used to engineer and manipulate light to produce a desired projection. Display (Spatial Light Modulator = SLM) + = Source of light (for example RGB lasers) Desired 3D holographic projection Interference pattern

6 What is a hologram? Laser source Object beam Reference beam Recording Surface

7 What is a hologram? Holographic Image Laser source Reconstruction beam Modified wave-front Observer

8 Digital Holography Compute the diffraction from any given virtual object by simulating wave propagation Concept: 1. Assume each object point is an aperture allowing light to radiate out spherically. 2. Calculate sum of spherical waves to give a complete interference pattern.

9 Classical Calculation of Holograms y x Hologram Plane

10 Classical Calculation of Holograms y Point Cloud Vectors in (x,y,z) x Hologram Plane

11 Classical Calculation of Holograms y x

12 Classical Calculation of Holograms r H x, y = A i e ikr i r i r = (x x ) 2 + (y y ) 2 +z 2

13 NOT scalable to large point clouds or high-pixel density displays

14 Need to simplify this calculation! Enter the Layer-based FFT approach

15 Layer-based FFT approach y x

16 Layer-based FFT approach y z x

17 Layer-based FFT approach FT(z 1 ) y x

18 Layer-based FFT approach FT(z 2 ) y x

19 Layer-based FFT approach FT(z n ) y H x, y = FT z i e ik x 2 +y 2 z i i x

21 HOLOGRAPHIC DISPLAY HARDWARE

22 Holographic Display Components LCoS Spatial Light Modulators Multiple Types - FLCoS - Nematic - Twisted Nematic Pixel Pitches from 3 8μm Available in 1K, 2K and 4K resolutions

23 Example holographic projector Spatial Light Modulator Image SLM Driver Beam Splitter Image Enlarging Optics Beam Splitter ARM Cortex Microcontroller Dichroic Mirrors Laser Diodes

24 HOLOGRAPHY ON NVIDIA GPUs

25 Hologram generation

26 Hologram generation

27 Hologram generation

28 Hologram generation

29 Hologram generation

30 Hologram generation cufft

31 Hologram generation cufft

32 Hologram generation cufft

33 Hologram generation cufft

34 Unity Streamer plugin

35 Unity Streamer plugin

36 Unity Streamer plugin

37 Unity Streamer plugin C# Script Hook into C++ / CUDA Core DLLs Render texture

38 Unity Streamer plugin C# Script Shader Hook into C++ / CUDA Core DLLs Depth Colour Render texture

39 Unity Streamer plugin C# Script Shader Hook into C++ / CUDA Core DLLs Depth Colour Render texture Export via CUDA/OpenGL interop VIVIDQ Core Create hologram from texture Format as BMP

CUDA/OpenGL interop VIVIDQ Core Create hologram from

40 Unity Streamer plugin C# Script Shader Hook into C++ / CUDA Core DLLs Depth Colour Render texture Export via CUDA/OpenGL interop VIVIDQ Core Create hologram from texture VIVIDQ View Output via DisplayPort Format as BMP

41 Framerate (Hz) Generation time (ms) Hologram generation framerate Num non-empty layers Depth tolerance factor Titan V P Ti Ti Titan V P Ti Ti

42 End-to-end Performance VIVIDQ Capture VIVIDQ Core VIVIDQ View Game objects captured at SLM resolution Point Clouds loaded onto GPU Holograms computed on Nvidia Titan X > 200 depth layers Laser controller synchronises pulses with hologram frames Holograms streamed to 2K FLCoS SLM Total time per frame < 10ms giving >60 fps

Tim Wilkinson (Head of Photonic Engineering at University of Cambridge) VividQ is going to completely revolutionise the way we

43 VividQ created a commercially viable product which defines and catalyses Mixed Reality applications Eric Ries (Lean Startup methodology founder) w w w. v i v i d - q. c o m TM VividQ solutions allow holography to break out of academic labs Prof. Tim Wilkinson (Head of Photonic Engineering at University of Cambridge) VividQ is going to completely revolutionise the way we think about 3D computing and display Phil Green (CFO at Deliveroo) C a m b r i d g e a n d L o n d o n, U K i n f v i v i d - q. c o m

Devices displaying 3D image. RNDr. Róbert Bohdal, PhD.

Devices displaying 3D image. RNDr. Róbert Bohdal, PhD. Devices displaying 3D image RNDr. Róbert Bohdal, PhD. 1 Types of devices displaying 3D image Stereoscopic Re-imaging Volumetric Autostereoscopic Holograms mounted displays, optical head-worn displays Pseudo