OpenGL ES for iphone Games Erik M. Buck
Topics The components of a game n Technology: Graphics, sound, input, physics (an engine) n Art: The content n Fun: That certain something (a mystery) 2
What is OpenGL ES? n OpenGL for Embedded Systems Low-level interface between software and graphics acceleration n a royalty-free, cross-platform API for fullfunction 2D and 3D graphics on embedded systems - KronosGroup A state machine A well-defined subset of desktop OpenGL Asynchronous memory buffer number crunching monster 3
What is OpenGL ES? (Continued) n OpenGL ES 1.X PowerVR MBX chip present in iphone prior to 3GS and ipod Touch Fixed function hardware (more later) n OpenGL ES 2.X PowerVR SGX chip present in iphone 3GS Full programmable 3D graphics shaders (more later) Emulates OpenGL ES 1.X fixed functions using shaders 4
Other related technology n Core Graphics/Quartz n Game Engines Oolong, SIO2, Unity, ShiVa, Torque, Cocos2D, others http://maniacdev.com/2009/08/the-open-sourceiphone-game-engine-comparison/ n Physics Engines Bullet, Rag Doll, Havok Physics, Chipmunk - 2D Physics Engine 5
Using an existing engine n Oolong Examples 6
Using an existing engine (continued) n SIO2 Examples 7
Other related technology (Continued) n Content Pipeline & Tools 3D Studio Max, Maya, Blender, Milkshape 3D, Photoshop, configuration files, world builders, content management systems, n Open AL for positional sound n Cocoa Touch GameKit, UIKit, Core Animation, Foundation, Core Data, Core Image, 8
Alternatives to OpenGL ES n Core Graphics/Quartz C or Objective-C High quality anti-aliasing and text rendering Well suited to text and sprite style games Uses the CPU almost exclusively Limited performance 2D focus draws into buffers associated with graphics contexts Quartz ultimately uses OpenGL ES to composite buffers together for display 9
Alternatives (continued) n WebKit/HTML/Canvas Javascript Full fledged drawing surface anywhere in your HTML page Images, vectors, arcs, fills, strokes, radial fills, gradients, alpha blending and more Limited performance WebKit uses Quartz and ultimately uses OpenGL ES to composite buffers together for display 10
OpenGL ES in Games n Best performance Dedicated hardware offloads the ARM CPU Low-level optimizations n You will need to understand OpenGL ES Even if you use an existing game engine Essential for performance optimization n Writing your own engine is half the fun Differentiates your game (for better or worse) 11
OpenGL has a sordid history n SGI defined a thin software interface to 1992 hardware and called it a standard Hardware of the time was constrained and sub-optimal from a modern perspective Textures and shaders have superseded materials as the primary way to calculate final pixel color values Asynchronous processing has superseded immediate mode and display lists 12
ES is a big improvement n Simpler n Consistent buffer management Textures, vertex coordinates, normal vectors, colors, texture coordinates, Render Buffers, Frame Buffers All allocated, updated, and freed in similar ways n Buffers are processed asynchronously n Buffers reside in GPU local memory 13
Getting started n The OpenGL ES Xcode project template Supports and auto-detects OpenGL ES 1.x or 2.x n Frame Buffers, Render Buffers, and CALayers The drawing has to go somewhere and then get on screen n Contexts A big bag of state information 14
Getting started (continued) n The OpenGL ES Xcode project template Provides application structure put you drawing code here http://www.visualnewt.com/opengl/ learning_iphones_opengl_es/part_iv_- _the_iphone_opengl.html Provides UIView with CAEAGLLayer Handles timing for animation 15
Getting started (continued) n Provide OpenGL ES content using the CAEAGLLayer class 16
Getting started (continued) Frame Buffer GL_DEPTH_ATTACHEMENT GL_STENCIL_ATTACHEMENT Render Buffer Depth optional Render Buffer Stencil optional GL_COLOR_ATTACHEMENT0 Render Buffer Color OpenGL Context glbindframebuffer() CALayer CALayer CALayer Quartz Compositor 17
Getting started (continued) n Frame Buffer - abstract buffer that the GLuint framebuffer; OpenGL hardware can read and write glgenframebuffersoes(1, &framebuffer); glbindframebufferoes(gl_framebuffer_oes, framebuffer); n Render Buffer - contiguous block of bytes; the actual GLuint colorrenderbuffer; storage for a frame buffer glgenrenderbuffersoes(1, &colorrenderbuffer); glbindrenderbufferoes(gl_renderbuffer_oes, colorrenderbuffer); glrenderbufferstorageoes(gl_renderbuffer_oes, GL_RGBA8_OES, width, height); Frame Buffer has one or more attached render buffers n CALayer - software interface that mediates shared myeagllayer access = (CAEAGLLayer*)self.layer; to a render buffer and integrates glbindrenderbufferoes(gl_renderbuffer_oes, colorrenderbuffer); with [mycontext the renderbufferstorage:gl_renderbuffer_oes Quartz Compositer which fromdrawable:myeagllayer]; layers content on screen. 18
Getting started (continued) Frame Buffer GL_DEPTH_ATTACHEMENT GL_STENCIL_ATTACHEMENT Render Buffer Depth optional Render Buffer Stencil optional GL_COLOR_ATTACHEMENT0 Render Buffer Color OpenGL Context glbindframebuffer() glbindtexture() OpenGL Texture 19
Getting started (continued) n Contexts The OpenGL state Each context has one frame buffer at a time Each context stores integer identifiers for resources that reside in GPU local memory or client side memory Contexts can share GPU local memory in order to minimize GPU memory consumption You set the current state through immediate mode functions (that should be avoided when possible) and by binding 20
More on buffers n Textures texture; 2D pixel data with power of 2 dimensions GLuint glgentextures(1, &texture); glbindtexture(gl_texture_2d, texture); glteximage2d(gl_texture_2d, 0, GL_RGBA8, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL); n Vertex Buffers GLuint glgenbuffersarb(1, & vertexbuffer); glbindbufferarb(gl_array_buffer, vertexbuffer); glbufferdataarb(gl_array_buffer, sizeof(data), data, GL_STATIC_DRAW); vertexbuffer; Vertex coordinate vectors - 3D spatial position {x,y,z} coordinates Vertex normal vectors - 3D vector {x,y,z} coordinates define perpendicular to plane Used for lighting, collision detection, back facing triangle culling, special effects 21
More on buffers (continued) n Vertex Buffers Color vectors 3D {r,g,b} or 4D {r,g,b,a} vectors Used to interpolate color of pixels between vertices Texture coordinate vectors 2D {u,v} coordinates Used to map textures to spatial positions 22
Asynchronous Panacea GPU local Memory - Data you supply n GPUs are number crunching monsters as long as you keep them fed GPU local memory - Data generated by hardware - May be shared with CALayer 23
Asynchronous Problem n Don t add unnecessary synchronization points Avoid changing data in use by current commands or dependant on completion of previous commands glflush() - gltexsubimage(), glcopyteximage(), glcopytexsubimage(), glreadpixels(), glbindframebufferoes(), glbuffersubdata() glfinish() Calls glflush() and blocks calling process until all pending commands complete 24
Art - Content Develop and use content n Develop 3D Models Textures, bump maps (per pixel normal vectors encoded in texture format) Levels or Scenes Sound/Music Configuration files n Version control and content management 25
Art - Content Develop and use content (continued) n Developing 3D Models 3D Studio Max, Maya, Blender, Milkshape 3D n Using 3D Models with OpenGL ES What Model data format? n COLLADA is a royalty-free XML schema that enables digital asset exchange within the interactive 3D industry n Wavefront.OBJ format n Max.3DS format n Quake.md2 format 26
Art - Content Develop and use content (continued) n Loading and using 3D Models with OpenGL ES http://iphonedevelopment.blogspot.com/ 2009/06/using-3d-models-from-blender-inopengl.html http://www.heikobehrens.net/2009/08/27/ obj2opengl/ http://nehe.gamedev.net/data/lessons/ lesson.asp?lesson=31 http://www.ddj.com/mobile/209600498 27
Art - Content Develop and use content (continued) 28
Art - Content Develop and use content (continued) Allistair:iPhone erik$./obj2opengl.pl toyplane.obj! Input file :./toyplane.obj! Output file :./toyplane.h! Object name : toyplane! Center : <10.5398597570699, 0.882485880041218, 0.448425590864673>! Scale by : 0.0242175690907872! ----------------! Vertices : 36846! Faces : 55534! Texture Coords : 29204! Normals : 31903 29
Art - Content Develop and use content (continued) /*created with obj2opengl.pl! source file :./toyplane.obj! vertices : 36846! faces : 55534! normals : 31903! texture coords : 29204!! // include generated arrays! #import "./toyplane.h! // set input data to arrays! glvertexpointer(3, GL_FLOAT, 0, toyplaneverts);! glnormalpointer(gl_float, 0, toyplanenormals);! // draw data! gldrawarrays(gl_triangles, 0, toyplanenumverts);! */! unsigned int toyplanenumverts = 166602;! float toyplaneverts [] = {! // f 1017/1/1 1012/2/1 1011/3/1! 0.10513789394066, 0.0902944342978993, 0.299815186264476,! 0.102783510308792, 0.0860670396549806, 0.299815186264476, 30