Texture Caching Implementation for Hardware Accelerated 2D Rendering

Transcription

1 Texture Caching Implementation for Hardware Accelerated 2D Rendering 양정수 (Jeongsoo Yang) yangjeongsoo at gmail.com 4 Featured Sessions - Kandroid S/W Fundamentals Study Group 12 th Kandroid Conference, Terminology and History of Android Performance Features 2. Automated Historical Performance Analysis with kmemtracer 3. High Performance Apps Development with RenderScript 4. Texture Caching Implementation for Hardware Accelerated 2D Rendering

2 CONTENT 1. What is Open GL Texture? 2. Understanding GPU Memory 3. Case Study : a. Cocos2d-x Texture Cache b. Android HWUI Texture Cache c. Chrome Browser Texture Cache d. Android SurfaceFlinger Texture Cache 4. Conclusion 2

3 The true worth of a man is not to be found in man himself, but in the colours and textures that come alive in others. - Albert Schweitzer Source 3

4 1. Load Bitmap 2. Load Texture from Bitmap 3. Free Bitmap 4. Draw Target with Texture Bitmap mbitmaptexture = BitmapFactory.decodeResource(mContext.getResources(), R.drawable.texture1); nativeinit(mbitmaptexture); mbitmaptexture.recycle(); nativerender(); glgentextures(1, &texture); glbindtexture(gl_texture_2d, texture); gltexparameterx(gl_texture_2d, GL_TEXTURE_MIN_FILTER, GL_NEAREST); gltexparameterx(gl_texture_2d, GL_TEXTURE_MAG_FILTER, GL_LINEAR); AndroidBitmap_lockPixels(env, bitmap, &pixels); glteximage2d(gl_texture_2d, 0, GL_RGBA, bitmapinfo.width, bitmapinfo.height,0, GL_RGBA, GL_UNSIGNED_BYTE, pixels); AndroidBitmap_unlockPixels(env, bitmap); 4

5 CONTENT 1. What is Open GL Texture? 2. Understanding GPU Memory 3. Case Study : a. Cocos2d-x Texture Cache b. Android HWUI Texture Cache c. Chrome Browser Texture Cache d. Android SurfaceFlinger Texture Cache 4. Conclusion 5

6 High Performance Applications with RenderScript Google I/O GB/s 50GB/s ~12GB/s GDDR5 DDR3 ISP LPDDR3 DSP GPU PCIe ~12GB/s CPU GPU CPU 3+TFLOPs ~250GFLOPs 70GFLOPs ~20GFLOPs Desktop/Server System Architecture Mobile System Architecture 6

7 Source : 7

8 Source : 8

9 Source : 9

10 Geforce GPU Architecture in NVIDIA Tegra Source : 10

11 Source : 11

12 Source : 12

13 CONTENT 1. What is Open GL Texture? 2. Understanding GPU Memory 3. Case Study : a. Cocos2d-x Texture Cache b. Android HWUI Texture Cache c. Chrome Browser Texture Cache d. Android SurfaceFlinger Texture Cache 4. Conclusion 13

14 Beyond Android With the current market saturation, is it possible to create a new platform? B2G, Tizen, LG Web OS, Ubuntu Mobile The successful development of a new platform could lead to advances. Nevertheless, it would be just one more in an already saturated market. So, we must streamline the market and focus on cross-platform compatibility with the status quo. Web-based (e.g. PhoneGap) Native-based (e.g. Cocos2d-x) VM-based (e.g. Mono) 14

15 Relationship in Cocos2d Family Source : 15

16 Cocos2d-x Blank Screen Issue We had some reports that was blank screen problems, but we couldn t reproduce that error. 16

17 Some Checkpoints for Cocos2d-x Blank Screen Issue 1. EGL Context related Cocos2d-x Architecture 2. Process and Thread termination related Binder Handle Recovery Mechanism 3. Texture Cache related Cocos2d-x Android Director ActivityManager Scene Activity Layer Surface Sprite (Node) Widget (View) Animations Frame, Tweened, Property Texture Cache? 17

18 Some Checkpoints for Cocos2d-x Blank Screen Issue 4. Snapshot Analysis for Cocos2d-x Memory State 18

19 Some Checkpoints for Cocos2d-x Blank Screen Issue 4. Historical Analysis for Cocos2d-x Memory State D/cocos2d-x debug info(697): OpenGL error 0x ,000, ,000, ,000, ,000, ,000,000 80,000,000 60,000,000 40,000,000 Size Alloc Free Native Pss Other Pss 20,000,000-19

20 Some Checkpoints for Cocos2d-x Blank Screen Issue 4. Historical Analysis for Cocos2d-x Memory State D/cocos2d-x debug info(697): OpenGL error 0x0505 Director Scene Layer Sprite (Node) Animations Cocos2d-x Texture Cache TextureAtlas (sprite image based) TextureCache (file name based) Android ActivityManager Activity Surface Widget (View) Frame, Tweened, Property? 20

21 Some Checkpoints for Cocos2d-x Blank Screen Issue 4. Historical Analysis for Cocos2d-x Memory State Total Alloc Free NPss OPss 0 21

22 CONTENT 1. What is Open GL Texture? 2. Understanding GPU Memory 3. Case Study : a. Cocos2d-x Texture Cache b. Android HWUI Texture Cache c. Chrome Browser Texture Cache d. Android SurfaceFlinger Texture Cache 4. Conclusion 22

23 OpenGLRenderer (libhwui) Properties Property Type Default Value ro.hwui.disable_scissor_opt boolean false ro.hwui.texture_cache_size float 24 ro.hwui.layer_cache_size float 16 ro.hwui.gradient_cache_size Float 0.5 ro.hwui.path_cache_size float 4 ro.hwui.shape_cache_size float 1 ro.hwui.drop_shadow_cache_size float 2 ro.hwui.fbo_cache_size integer 16 ro.hwui.r_buffer_cache_size float 2 ro.hwui.texture_cache_flush_rate float 0.6 ro.hwui.text_small_cache_width integer 1024 ro.hwui.text_small_cache_height integer 256 ro.hwui.text_large_cache_width integer 2048 ro.hwui.text_large_cache_height integer 512 hwui.text_gamma_correction string lookup hwui.text_gamma float 1.4 hwui.text_gamma.black_threshold integer 64 hwui.text_gamma.white_threshold integer 192 hwui.use_gpu_pixel_buffers boolean true Source : 23

24 libhwui : frameworks/base/libs/hwui/texturecache.cpp hwui Cache mcache GenerationCache<SkBitmap*, Texture*> Texture* TextureCache::get(SkBitmap* bitmap) { TextureCache LayerCache GradientCache PathCache Texture* texture = mcache.get(bitmap); if (!texture) { generatetexture(bitmap, texture, false); mcache.put(bitmap, texture); } else if (bitmap->getgenerationid()!= texture->generation) { generatetexture(bitmap, texture, true); } } return texture; 24

25 Android Animations : Frame, Tweened, Property 25

26 Android Animations : Frame, Tweened, Property ImageView logoimage = (ImageView) findviewbyid(r.id.iv1); logoimage.setbackgroundresource(r.drawable.logo_animation); logoanimation = (AnimationDrawable) logoimage.getbackground(); logoanimation.start(); Caches: Current memory usage / total memory usage (bytes): TextureCache / LayerCache 0 / RenderBufferCache 0 / GradientCache 0 / PathCache 0 / TextDropShadowCache 0 / FontRenderer / Other: FboCache 0 / 16 PatchCache 1 / 512 Total memory usage: bytes, MB 26

27 CONTENT 1. What is Open GL Texture? 2. Understanding GPU Memory 3. Case Study : a. Cocos2d-x Texture Cache b. Android HWUI Texture Cache c. Chrome Browser Texture Cache d. Android SurfaceFlinger Texture Cache 4. Conclusion 27

28 Google I/O 2012 Keynote Android vs. Chrome 28

29 Google I/O 2013 Keynote Sundar Pichai (SVP, Android, Chrome & Apps) 29

30 Chrome s Multi-process Architecture on Android Browser Process Render Process Main Thread (UI) I/O Thread IPC Main Thread Render Thread android:process=":sandboxed_process0" android:isolatedprocess="true android:process=":privileged_process2" android:isolatedprocess="false" android:isolatedprocess If set to true, this service will run under a special process that is isolated from the rest of the system and has no permissions of its own. The only communication with it is through the Service API (binding and starting). 30

31 Chrome s Hardware Acceleration on Android Compositing with the GPU Thread Render Process WebKit / Skia Compositor Shared Memory Commands Bitmaps & Arrays IPC Browser Process GPU Thread (server) Compositor Context GL/D3D 31

32 Compositor Thread Architecture & Texture Upload One challenge with all these textures is that we rasterize them on the main thread of the renderer process, but need to actually get them into the GPU memory. This requires handing information about these textures (and their contents) to the impl thread, then to the GPU process, and once there, into the GL/D3D driver. Done naively, this causes us to copy a single texture over and over again, something we definitely don't want to do. We have two tricks that we use right now to make this a bit faster. To understand them, an aside on painting versus rasterization. Painting is the word we use for telling webkit to dump a part of its RenderObject tree to a GraphicsContext. We can pass the painting routine a GraphicsContext implementation that executes the commands as it receives them, or we can pass it a recording context that simply writes down the commands as it receives them. Rasterization is the word we use for actually executing graphics context commands. We typically execute the rasterization commands with the CPU (software rendering) but could also execute them directly with the GPU using Ganesh. Upload: this is us actually taking the contents of a rasterized bitmap in main memory and sending it to the GPU as a texture. With these definitions in mind, we deal with texture upload with the following tricks: Per-tile painting: we pass WebKit paint a recording context that simply records the GraphicsContext operations into an SkPicture data structure. We can then rasterize several texture tiles from that one picture. SHM upload: instead of rasterizing into a void* from the renderer heap, we allocate a shared memory buffer and upload into that instead. The GPU process then issues its gltex* operations using that shared memory, avoiding one texture copy. Source : 32

33 CONTENT 1. What is Open GL Texture? 2. Understanding GPU Memory 3. Case Study : a. Cocos2d-x Texture Cache b. Android HWUI Texture Cache c. Android Animation & Texture Cache d. Chrome Browser Texture Cache e. Android SurfaceFlinger Texture Cache 4. Conclusion 33

34 Beyond Android With the current market saturation, is it possible to create a new platform? B2G, Tizen, LG Web OS, Ubuntu Mobile The successful development of a new platform could lead to advances. Nevertheless, it would be just one more in an already saturated market. So, we must streamline the market and focus on cross-platform compatibility with the status quo. Web-based (e.g. PhoneGap) Native-based (e.g. Cocos2d-x) VM-based (e.g. Mono) 34

35 SurfaceFlinger and OpenGL Texture How Android is using Gralloc The magic part is that SurfaceFlinger does not need to do the equivalent of GrallocTextureHostOGL::Unlock. In our case, we have a separate OpenGL texture object for each TextureHost, which typically (at least in the case of a ContentHost) represent one buffer each (so a double-buffered ContentHost has two TextureHost's). So we have to unbind the GraphicBuffer from the OpenGL texture before we can hand it back to the content side --- otherwise it would remain locked for read and couldn't be locked for write for content drawing. By contrast, SurfaceFlinger does not need to worry about this because it uses only one OpenGL texture, so that when it binds a new GraphicBuffer to it for compositing, that automatically unbinds the previous one! Source : 35

36 Google I/O 2008 : Anatomy & Physiology of an Android APP APP Surface Surface SurfaceFlinger FrameBuffer Surface 36

37 Source : 4 th kandroid conference, 2009 APP SurfaceFlinger::instantiate() - AddSevice( Surface Flinger..) SurfaceFlinger::readyToRun() Surface - Gather EGL extensions - Create EGL Surface and Map Frame Buffer - Create our OpenGL ES context - Gather OpenGL ES extensions - Init Display Hardware for GPU APP Surface SurfaceFlinger::threadLoop() - Wait for Event - Check for tranaction - Post Surface (if needed) - Post FrameBuffer Surface Flinger Frame Buffer Surface 37

38 Google I/O 2012 : For Butter or Worse Draw DisplayList Swap Buffers Composite Windows Post Buffer Overlay GPU Overlay FB Buffer Overlay FB Overlay FB 38

39 CONTENT 1. What is Open GL Texture? 2. Understanding GPU Memory 3. Case Study : a. Cocos2d-x Texture Cache b. Android HWUI Texture Cache c. Chrome Browser Texture Cache d. Android SurfaceFlinger Texture Cache 4. Conclusion 39

40 IDC 2013 agenda Seoul, Korea - 12th November 2013 Developer Event Agenda 13:00-13:20 Registration and demos 13:20-13:40 13:40-14:40 Introduction to Imagination Bryce Johnstone, Senior Manager Ecosystems Marketing Imagination Hardware Architecture Kristof Beets, Senior Business Development Manager, Imagination 14:40-15:00 Coffee Break & Demos 15:00-16:00 16:00-17:00 17:00-17:15 PowerVR SDK and Utilities Bob Gardner, Developer Technology Engineer, Imagination PowerVR Tips and tricks : Maximise your performance Bob Gardner, Developer Technology Engineer, Imagination Q+A, wrap up and prize draw Bryce Johnstone, Senior Manager Ecosystems Marketing 17:15-19:15 Networking session - beer and hors d'oeuvres. 40

41 ARM Technology Symposia 2013 : 11 월 19 일 41

42 13 th Kandroid Conference, 2014 Graphics & GPU Morning Sessions : Open Sessions Afternoon Sessions : Featured Sessions 42