Mali Demos: Behind the Pixels Stacy Smith
Mali Graphics: Behind the demos Mali Demo Team: Doug Day Stacy Smith (Me) Sylwester Bala Roberto Lopez Mendez PHOTOGRAPH UNAVAILABLE These days I spend more time explaining demos than making them Occasionally this includes explaining why we make demos at all
Demos in a Nutshell What are demos for? How are people using our GPU? How would we like them to use the GPU? Behind the Pixels: How did we get here? Where are we going? How are we getting there?
A Good Analogy is Like a Salmon Desktop GPU Mobile GPU
A brief history of graphics Why do we do things the way we do? How did we do them before? The evolution of graphics
Pre-history : Before GPUs Software Rendering: Rasterizing Depth sorting for max overdraw Early shading models Textures Perspective correct textures Depth Buffers Light/shadow maps Blending
Early GPUs : Fixed Function : GLES1.1 Parallelise repeated work: Transform Rasterize Switchable fragment processes Texture Lighting Culling Fog Management of Buffers Color Depth Stencil
Current GPUs : Programmable : Open GLES 2.0 Programmable Vertex Shaders Arbitrary vertex attributes Arbitrary uniform values Arbitrary data interpolated in rasterizer Programmable Fragment Shader Bespoke lighting equations Per fragment procedural coloring Textures as continuous input parameters
So... What s next? OpenGL ES 3.0 Deferred lighting solutions Compute shaders (OpenCL / RenderScript)
Open GL ES 3.0 New kinds of textures 3D textures Shadow textures Seamless cube maps New render targets RGB10A2 Depth buffers New ways of getting info to the render state Instancing Uniform Buffer Objects Sampler Objects
Timbuktu2
Timbuktu 2: Shadow Samplers Render depth from viewpoint of light Use framebuffer as texture map Compare projected texture with distance from light Fragments further from the light than the shadow map are in shadow
Timbuktu 2: Shadow samplers GLES 2.0 Texture Compare: Interpolate Compare GLES 3.0 Shadow Texture Compare: Compare Compare Compare Compare Interpolate
Timbuktu 2: HDR Framebuffers LDR output uses 8 bits per channel White paint Centre of the Sun Using RGB10A2 surfaces quadruples our range 0-255 A shifting window of exposure
Timbuktu 2: Texturing improvements Timbuktu terrain offsets & shading RGBA image updated with gltexsubimage2d Surface normal in RGB Surface offset in A Used twice: Used for vertex modification in foreground Normal mapping in background Offsets into 3D texture map Smooth transition to deeper earth texture
Seemore
Seemore: Seamless cube maps Seamless cube maps used as rotated screen space lookup Give appearance of looking through window Use window texcoords to apply additional effects Shift mip level to blur environment behind dirty glass Add distortion for warped glass Also used to project grime from swinging light bulbs onto surfaces
Seemore: Sampler objects Sampler objects Allow textures to be associated and dissociated......without affecting other aspects of the state
Seemore: Instanced Draw Instancing Renders the same object many times in a single call Reduces vertex bandwidth drawcall CPU load
Deferred Lighting G buffers Raw tile rendering Store per fragment info: Depth Diffuse Normal Combine pixels in post processing Lighting Fog Specular Environment map
Lights Demo In reality lights attenuate based on the inverse square law Surface area of a sphere A = 4πr 2 Light power is divided by this
Lights Demo In reality lights attenuate based on the inverse square law Surface area of a sphere A = 4πr 2 1 0.9 Light Intensity Light power is divided by this In LDR lighting, 1 / 256 0 The light has dropped off in 4.5 units This gives a bounding radius 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
Lights demo Screen space bounding spheres cover influence
Hauntheim
Hauntheim Hauntheim also uses conic hulls Diffuse buffer Depth buffer Normal buffer Lighting hulls Accumulation buffer Final pass
Compute Shaders: OpenCL Reduce bandwidth by keeping things on the GPU Physics modelling Geometry generation Post processing
Flag 2 A rough guide to cloth physics Points modelled with mass & momentum Friction (Entropy) Impulse Inertia Gravity
Flag 2 A rough guide to cloth physics Points modelled with mass & momentum Points joined into strings with springs A lattice of strings makes a cloth. Optionally wider spanning springs Simulate stiffness.
Flag 2
Trollheim
Trollheim Multi resolution grid Grid follows offsets in texture map CL generating random landscape to texture
Trollheim Multi resolution grid Grid follows offsets in texture map CL generating random landscape to texture Constant tree count Tress repositioned as needed Similar system for grass & flowers
Evolution Evolution doesn t tend towards one perfect design New environmental conditions favour different features Similarly, use cases drive technological innovation The only thing needed to change the direction of future GPUs is for someone to pick one up and run with it
Thank You Any Questions? malideveloper.arm.com
Stacy Smith Mali Developer Education Stacy.Smith@arm.com