Engine Development & Support Team Lead for Korea UE4 Mobile Team Lead

Transcription

1

2 Jack Porter Engine Development & Support Team Lead for Korea UE4 Mobile Team Lead I ve worked on Unreal Engine development since 1998! Contributed to Unreal Tournament & Gears of War series

3 Introduction The UE4 mobile rendering pipeline and features for AAA games High-end graphics with mobile using OpenGL ES 3.1, Vulkan and Metal Upcoming UE4 mobile features

4 State of the UE4 for Mobile Game Market AAA, console quality games released with many more in development Full open-world MMOs now running on Mobile Lineage II Revolution was top grossing mobile game worldwide in February 2017

5 Real open-world MMO on mobile devices Has raised the bar for what s expected for mobile titles! Huge success!

6 Netmarble developed this game using standard UE4 features Development, testing and debugging on PC UE4 Mobile Renderer MobileHDR, post-processing Landscape Level Streaming Materials UI made with Widget Blueprints Device Profiles for scalability Minumum Spec: Galaxy S4, iphone 5

7

8 Unreal Engine 4 has three different renders: Deferred Renderer The default renderer for the editor, PC and Console Feature levels SM4, SM5 Forward VR Renderer Used mainly for desktop VR eg Robo Recall and to support MSAA Similar feature set to the deferred renderer with a few exceptions (eg screen-space effects) Feature levels SM5 Mobile Renderer Used for mobile devices Forward renderer Reduced feature set especially for lighting, shadowing Feature levels ES2, ES3_1

9 UE4 can target one of several feature levels, which specifies the set of Material Editor features that will be supported in the shaders. The feature level also determines what UE4 Renderer can be used. The mobile feature level names are ES2 and ES3_1 but do not exactly correspond to the OpenGL ES 2 and OpenGL ES 3.1. The ES2 renderer uses ES 3.0 features when they are availble. Feature Level Switch switches can be used in materials Editor Preview Rendering Level can be used to preview

10 ES2 ES3_1 High End Mobile SM4 SM5 Renderer Platforms Textures Notes Mobile Mobile Deferred Deferred or Forward VR OpenGL ES x (Android) OpenGL ES 2.0 (ios) OpenGL ES 3.1 (Android) Vulkan (Android) Metal (ios) Direct3D 10 OpenGL 3.0 Metal (MacOS, ios A8+ devices ) Direct3D 11 OpenGL 4.0 Playstation 4 Xbox One Metal (MacOS) OpenGL ES 3.1+AEP (Nvidia K1, X1 with Android ) 8 16 On Android: Shaders are patched to support OpenGL ES 3.0 features and other GL extensions on some devices Fallback for 32-bit render target Floating point render targets Vertex textures srgb support 16 Minimum feature level for deferred 16 + (using shared samplers) Metal Deferred is experimental Minimum feature level for tessellation support Compute shaders used for advanced rendering features OpenGL ES 3.1+AEP Deferred will be deprecated in the future

11 Selecting one of these Build options causes UE4 to generates an extra set of shaders using the ES3_1 feature level Devices will automatically choose the best RHI and feature level Can be overridden by Device Profiles

12 What is Vulkan? Vulkan is a next generation graphics API intended to replace OpenGL and OpenGL ES Designed by the industry, sponsored by Khronos Designed with graphics hardware in mind UE4 Android with Vulkan Android 7.0 devices now shipping with working Vulkan drivers Light-weight API minimizes CPU usage Pipeline State Objects make state changes efficient More draw calls because each draw call is more light-weight Render Passes make efficient use of mobile tiling GPU hardware

13 Comparing PC Direct3D RHI and Vulkan RHI on PC and device

14 Unlike the deferred renderer using a Gbuffer, UE4 mobile renderer is a forward renderer which saves only the output color. Scene Color Ideally 16-bit floating point render target on modern devices When not available we use RGBE encoding or mosaic RGB Stores the HDR color values while rendering Alpha channel stores the depth for later use (soft particles, decals) Depth buffer Usually 24 bits of depth, 8 bits of stencil Scene Color (HDR) Backbuffer 32-bit RGBA Final result after post-processing and tonemapping Backbuffer

15 RGB + Exponent encoding method needed for Galaxy S6/Note 4 with Android 5.x Requires EXT_shader_framebuffer_fetch extension available on Galaxy S6 Supports full HDR, bloom and no resolution limitations Mosaic HDR Native HDR Encoding HDR

16 32-bit hardware backbuffer & depth buffer Allocated by the platform, eg EGL in the case of Android srgb not possible on many devices Scene is rendered directly to the backbuffer in Gamma Space followed by translucency and UI Fastest possible rendering mode for simple or VR games Many effects do not work No post processing No deferred passes like decals or modulated shadows because the depth buffer is not accessible Backbuffer (LDR mode)

17 1. View setup 2. GPU Particle simulation 3. Shadow Map rendering 4. Base Pass 5. Decals 6. Modulated shadows 7. Translucency 8. Post-processing & Tonemapping 9. HUD & UI

18 Find out all visible objects Frustum culling Distance culling Precomputed visibility View relevance Find out all visible shadows Gather dynamic mesh elements Update view uniform buffer

19 Requires device with ES3_1 feaure level, or ES2 with an Android device with OpenGL ES 3.1 Simulates particle physics on GPU Writes particle Position to 128bpp target Writes particle Velocity to 64bpp target

20 Setup depth render target Find out which objects need shadows Render them using main light view Shadow map used later During base pass for CSM shadows After, for modulated shadow projections

21 Choose appropriate shaders depending on shadow and lighting properties Unlit Distance field shadows + LM Distance field shadows + LM + CSM Dynamic lighting (CSM) etc For dynamic elements, this happens every frame For static elements this occurs once when the scene is created, and the mesh drawing state is added to a static draw list. Primitives are grouped by material, vertex factory to reduce OpenGL state changes

22 Renders all objects with opaque materials and combines lighting, to scene color in HDR / linear space Dynamic directional and point lights (per-pixel) Static distance field shadows Dynamic CSM shadows Lightmaps (from stationary lights) Skylight Reflections

23 Drawing order depends on device Draw without any reordering Default option for ImgTec, which has Hidden Surface Removal Reorder meshes front to back in each list Minimizes state change Reorder meshes front to back across all lists Minimizes overdraw Default option for non-imgtec GPUs Can be tweaked depending on your content r.forwardbasepasssort = x

24 Requires scene depth fetch Implementation depends on supported extensions GL_ARM_shader_framebuffer_fetch_depth_stencil GL_EXT_shader_framebuffer_fetch GL_OES_depth_texture depth buffer resolve Supports Receives Decals flag Stencil operations Does not support lighting

25 Similar to deferred decals Modulated Shadows do not blend well with static lighting Static + CSM is a better option (4.12+)

26 Draw primitives with non-opaque blend mode using the same shaders as the Base Pass Does not write to depth buffer Refraction supported Requires full copy of scene color

27 Only supported when HDR is enabled Requires several passes depending on what effects are used Depth of Field Custom PostProcess Materials Tonemapper pass at the end Maps HDR color to 8-bit per-channel RGB and writes it to backbuffer Bloom is also applied at this stage Filmic tonemapper (4.13+) r.tonemapperfilm=1 is the default for 4.15 r.mobile.tonemapperfilm=0 is the default for 4.16

28 Draw UI elements directly to backbuffer Slate / Widget Blueprints Canvas Swap backbuffer

29 The mobile render uses the same physically-based material system as UE4 on desktop and console platforms.

30 Material graph generates HLSL functions Combined with hand-written HLSL source code for our Physically Based lighting model We use this directly on PC and Xbox One half3 GetMaterialBaseColor(FMaterialPixelParameters Parameters) { MaterialFloat3 Local5 = ( Material.VectorExpressions[0].rgb); MaterialFloat4 Local6 = ProcessMaterialColorTextureLookup(Texture2DSample(Material.Texture2D_1, Material.Texture2D_1Sampler, Parameters.TexCoords[0].xy)); MaterialFloat3 Local7 = (Local6.rgb + MaterialFloat3( , , )); MaterialFloat Local8 = dot(local6.rgb, MaterialFloat3( , , )); MaterialFloat Local9 = (Local ); MaterialFloat Local10 = (Local9 * ); MaterialFloat Local11 = min(max(local10, ), ); MaterialFloat Local12 = (Local8 + dot(materialfloat3( , , ), MaterialFloat3( , , ))); MaterialFloat Local13 = (dot(materialfloat3( , , ), MaterialFloat3( , , )) / Local12); MaterialFloat Local14 = (Local ); MaterialFloat Local15 = (Local14 * ); MaterialFloat Local16 = (Local ); MaterialFloat Local17 = min(max(local16, ), ); MaterialFloat Local18 = (Local11 * Local17); MaterialFloat3 Local19 = lerp(local6.rgb, Local7, MaterialFloat(Local18)); MaterialFloat3 Local20 = (Local19 * Local5); return Local20;; }

31 For mobile, we cross-compile HLSL to GLSL Version and GLSL extensions are enabled using the preprocessor #version 300 es out mediump vec4 out_fragcolor; uniform highp samplercube ps4; uniform highp sampler2d ps0; uniform highp sampler2d ps1; uniform highp sampler2d ps2; varying highp vec4 var_texcoord0; varying highp vec4 var_texcoord1; varying highp vec4 var_texcoord2; varying vec4 var_texcoord7; void main() { vec4 t0; vec4 t1; t1.xyzw = vec4(gl_fragcoord); t0.xyzw = t1; vec4 t2; highp vec4 t3; t3.xy = var_texcoord4.zw; t3.zw = var_texcoord5.zw; vec3 t4; vec3 t5; t5.xyz = vec3(t3.xyz); t4.xyz = t5; highp vec4 t6;

32 Movable Directional Light + Cascaded Shadow Map shadowing Editor setup Directional Light Component, set Mobility to Movable (1 supported) Set Dynamic Shadow Distance Movable Light Set Num Dynamic Shadow Cascades Advantages Lighting is fully dynamic and can be changed at runtime Reduced memory usage and package size as there is no precomputed lighting data Disadvantages All objects in the scene, static and dynamic, must be rendered into the CSM Extra draw calls and CSM generation cost

33 Simplest lighting setup for mobile Direct and Specular lighting calculated per pixel Intensity and color can be changed at runtime, but not direction Editor setup Directional Light Component, set Mobility to Stationary (1 supported) Set Cast Dynamic Shadows to False Advantages Fastest to render High quality distance-field shadows Disadvantages Light direction cannot be changed at runtime Only supports modulated player shadows Memory and package size footprint for precomputed shadowmaps

34 Optimized in version 4.13 No longer necessary to flag each object that will receive CSM shadows from players Editor setup Directional Light Component, set Mobility to Stationary (1 supported) Set Dynamic Shadow Distance Stationary Light Set Num Dynamic Shadow Cascades Disable Inset Shadows For Movable Objects Advantages Static geometry shadowed statically with distance field shadows Dynamic player shadows blend nicely with static shadows Disadvantages CSM generation cost (but only for dynamic objects) Base pass shaders require an extra texture sampler

35 Advantages High resolution for player shadows Works with static lighting Disadvantages Each object renders its own shadow map and has its own frustum Shadows do not blend nicely with the scene s static shadowing

36 Independent of other lighting setup Up to 4 lights are supported per object / surface Set Max Movable Point Lights > 0 in Project Settings, Engine Rendering Advantages Rendered as part of the base pass so no extra draw calls Disadvantages Extra shader permutations are generated

37 Lighting channels now supported on mobile (4.13) Limitations: only one Stationary or Movable Directional light can affect each object

38 Much more flexible for artists to achieve the look they want Enable with console var, eg by device profile r.tonemapperfilm=1 ( ) r.mobile.tonemapperfilm=1 (4.16+) Warning: enabled by default in 4.15! Has extra cost so only recommended on high-end devices

39 Custom Post Process (new in 4.13) Shader built using the material system

40 Custom Depth (new in 4.14) Selected objects can be rendered into a separate depth buffer This can be sampled in Custom Post Process

41 Custom Stencil (new in 4.15) On PC we the material can sample the value from the stencil buffer Not possible on mobile GPUs Instead we render out to a separate color target

42

43 Minimal APK/IPA + DLC Packaging Wizard (4.13) Small download from App Store / Google Play Only UI to download content Download game content and patches from the cloud Force user content updates

44 Mobile Patch Utilities Blueprint Library (4.14) Control download and install process Progress bar & error handling Storage space Wi-Fi available etc

45 Smaller Android executable size changes to reduce the code generated by the compiler and allow the linker to remove unused code UI Widget Blueprint performance improvements Android runtime permissions

46 Faster iteration time on Android Improved in-editor mobile preview Improved ios development from PC Easier remote toolchain setup Log output output

47