Embedded Media Processing Ecosystem

Similar documents
Copyright Khronos Group, Page 1

Copyright Khronos Group, Page 1

Copyright Khronos Group, Page 1

Streaming Media. Advanced Audio. Erik Noreke Standardization Consultant Chair, OpenSL ES. Copyright Khronos Group, Page 1

Completing the Multimedia Architecture

Creating the Embedded Media Processing Ecosystem

Dave Shreiner, ARM March 2009

Copyright Khronos Group Page 1

Next Generation OpenGL Neil Trevett Khronos President NVIDIA VP Mobile Copyright Khronos Group Page 1

WebGL Meetup GDC Copyright Khronos Group, Page 1

AR Standards Update Austin, March 2012

Mobile AR Hardware Futures

OpenMAX AL, OpenSL ES

SIGGRAPH Briefing August 2014

Copyright Khronos Group, Page 1

Press Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem. Copyright Khronos Group Page 1

Press Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem. Copyright Khronos Group Page 1

Copyright Khronos Group, Page 1. Khronos Overview. Taiwan, February 2012

Copyright Khronos Group Page 1. Vulkan Overview. June 2015

Khronos and the Mobile Ecosystem

POWERVR MBX & SGX OpenVG Support and Resources

Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group. Copyright Khronos Group Page 1

Bringing it all together: The challenge in delivering a complete graphics system architecture. Chris Porthouse

Applications and Implementations

Hardware Accelerated Graphics for High Performance JavaFX Mobile Applications

Mobile graphics API Overview

Copyright Khronos Group Page 1

Copyright Khronos Group, Page 1. OpenCL. GDC, March 2010

The Mobile Advantage. Erik Noreke Independent Standardization Consultant Chair, OpenSL ES. Copyright Khronos Group, Page 1

Streaming Media Portability

Adding Advanced Shader Features and Handling Fragmentation

Khronos Connects Software to Silicon

Open Standards for Building Virtual and Augmented Realities. Neil Trevett Khronos President NVIDIA VP Developer Ecosystems

Whiz-Bang Graphics and Media Performance for Java Platform, Micro Edition (JavaME)

Introduction to OpenGL ES 3.0

Open Standards for Vision and AI Peter McGuinness NNEF WG Chair CEO, Highwai, Inc May 2018

Mobile Graphics Ecosystem. Tom Olson OpenGL ES working group chair

Applications and Implementations

Standards for WebVR. Neil Trevett. Khronos President Vice President Mobile Content,

OpenCL Overview. Shanghai March Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group

SC24/WG9 Liaison Meeting

Overview and AR/VR Roadmap

Vulkan 1.1 March Copyright Khronos Group Page 1

OpenCL Press Conference

Our Technology Expertise for Software Engineering Services. AceThought Services Your Partner in Innovation

Copyright Khronos Group 2012 Page 1. OpenCL 1.2. August 2012

The OpenVX Computer Vision and Neural Network Inference

Accelerating Vision Processing

Open Standards for AR and VR Neil Trevett Khronos President NVIDIA VP Developer January 2018

Introduction to Shaders.

WebGL, WebCL and Beyond!

Visual HTML5. Human Information Interaction for Knowledge Extraction, Interaction, Utilization, Decision making HI-I-KEIUD

Open Standards for Today s Gaming Industry

PowerVR: Getting Great Graphics Performance with the PowerVR Insider SDK. PowerVR Developer Technology

Building scalable 3D applications. Ville Miettinen Hybrid Graphics

Open API Standards for Mobile Graphics, Compute and Vision Processing GTC, March 2014

Tools To Get Great Graphics Performance

Accelerating Realism with the (NVIDIA Scene Graph)

The PowerVR Insider SDK. PowerVR Developer Technology

Vulkan Launch Webinar 18 th February Copyright Khronos Group Page 1

Profiling and Debugging OpenCL Applications with ARM Development Tools. October 2014

Update on Khronos Open Standard APIs for Vision Processing Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem

Open Standard APIs for Augmented Reality

OpenGL ES 2.0 : Start Developing Now. Dan Ginsburg Advanced Micro Devices, Inc.

The State of Gaming APIs

WHAT IS BFA NEW MEDIA?

More performance options

Mali Developer Resources. Kevin Ho ARM Taiwan FAE

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY

Multimedia Platform. Mainstream wireless multimedia expands globally with the industry s first single-chipset solution

The Mobile Internet: The Potential of Handhelds to Bring Internet to the Masses. April 2008

gltf Briefing September 2016 Copyright Khronos Group Page 1

Next Generation Visual Computing

Craig Peeper Software Architect Windows Graphics & Gaming Technologies Microsoft Corporation

Khronos Updates GDC 2017 Neil Trevett Vice President Developer Ecosystem, NVIDIA President,

Developing the Roadmap - Director Next Survey

Real - Time Rendering. Graphics pipeline. Michal Červeňanský Juraj Starinský

WebGL (Web Graphics Library) is the new standard for 3D graphics on the Web, designed for rendering 2D graphics and interactive 3D graphics.

Grafica Computazionale: Lezione 30. Grafica Computazionale. Hiding complexity... ;) Introduction to OpenGL. lezione30 Introduction to OpenGL

Enabling a Richer Multimedia Experience with GPU Compute. Roberto Mijat Visual Computing Marketing Manager

What was removed? (1) OpenGL ES vs. OpenGL

Hands-On Workshop: 3D Automotive Graphics on Connected Radios Using Rayleigh and OpenGL ES 2.0

NVSG NVIDIA Scene Graph

Module Introduction. Content 15 pages 2 questions. Learning Time 25 minutes

Copyright Khronos Group Page 1

OpenGL ES update. Eisaku Ohbuchi Digital Media Professionals Inc.

3D Graphics in Future Mobile Devices. Steve Steele, ARM

Overview. Technology Details. D/AVE NX Preliminary Product Brief

Bringing AAA graphics to mobile platforms. Niklas Smedberg Senior Engine Programmer, Epic Games

Shaders : the sky is the limit Sébastien Dominé NVIDIA Richard Stenson SCEA

The Benefits of GPU Compute on ARM Mali GPUs

Using the PowerVR SDK to Optimize your Renderer

PowerVR Hardware. Architecture Overview for Developers

<Insert Picture Here> JavaFX 2.0

Introduction. What s New in This Edition

The Handheld Graphics. Market. Size, needs, and opportunities. Jon Peddie Research

The Ultimate Developers Toolkit. Jonathan Zarge Dan Ginsburg

Real-Time Rendering (Echtzeitgraphik) Michael Wimmer

PowerVR Series5. Architecture Guide for Developers

Acceleration Standards for Mobile Augmented Reality

Transcription:

Embedded Media Processing Ecosystem Neil Trevett Vice President Mobile Content, NVIDIA President, Khronos Group Copyright Khronos Group, 2007 - Page 1

Pervasive Mobile Media Computing Handsets are becoming personal computing devices - not just phones - Your most personal computer mobility, connectedness and numerous sensors Sophisticated media processing will be central to handheld revolution - Graphics and media will become as pervasive as it is on the PC Diverse applications will drive the need for handheld accelerated media - User Interface, 2D maps and 3D navigation, TV/Video, games 100Ms / year 100Ks / year 10Ms / year Media Platforms 80 s Workstations Copyright Khronos Group, 2007 - Page 2 90 s PCs 00 s Handhelds

Khronos Open Standards Khronos develops state-of-the-art open standards for the Authoring and Acceleration of graphics and media content - 3D and Vector 2D Graphics, Video, Imaging, Audio Khronos API standards provide Foundation-Level acceleration - Close-to-the-silicon access to media processors for efficiency and generality Khronos standards are royalty-free - For wide adoption on multiple platforms reducing application source fragmentation Khronos APIs APIs enable enable mobile mobile software to to be be PORTABLE and and to to EFFICIENTLY use use graphics and and media media acceleration Copyright Khronos Group, 2007 - Page 3

Around 100 companies creating media authoring and acceleration standards Copyright Khronos Group, 2007 - Page 4

Khronos Participation Model Board decides strategy approves working groups, controls budget, ratifies specifications. $25,000 annual membership dues Promoters Contributors Any company can join Khronos to participate in any number of working groups to produce specifications. $7,500 annual membership dues A Working Group for each API standard one company one vote Openly and publicly distributed free of charge, royalty free Ratified Specifications SDKs Conformance Tests Conformance Tests and Conformance Test Process. Typically $10K per API fee Free libraries, utilities, examples, open source Adopters Conforming products can use API trademark and logo Copyright Khronos Group, 2007 - Page 5

Khronos Standards Ecosystem Dynamic Media Authoring 3D Authoring Dynamic Media Authoring Standards Open Standards Effects Framework Bridging 3D Authoring and Acceleration for advanced visual effects Family of Market-focused 3D APIs Safety Critical 3D Cross platform desktop 3D Embedded 3D Vector 2D Application Acceleration APIs Streaming Media AL Enhanced Audio Embedded Media APIs DirectX-like API set with OS Abstraction and API Interoperability System Integration Standards CODEC and media component portability DL Streaming Media System Integration IL Open Standards for window systems Copyright Khronos Group, 2007 - Page 6

Adoption of Embedded Khronos APIs Market adoption in media-accelerated handsets We Are Here - OpenGL ES 1.1 is widespread - OpenVG is in adoption ramp - OpenMAX is in adoption ramp 100% 3D Graphics Vector 2D Streaming Media Enhanced Audio Mid-2004 OpenGL ES 1.1 Spec release Mid-2005 OpenVG 1.0 Spec release Beginning-2006 OpenMax IL 1.0 Spec release TODAY! OpenSL ES 1.0 and OpenMAX AL 1.0 Spec release News Flash! Copyright Khronos Group, 2007 - Page 7

Penetration of Native Media APIs 900 800 700 600 MM phones Non 3D MM phones Native API based phones Millions of units Jon Peddie Research Handheld Multimedia Devices report 500 400 300 200 100 0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Copyright Khronos Group, 2007 - Page 8

OpenGL ES Applications and Devices OpenGL ES 1.1 has become the most widely deployed 3D API - Used in diverse applications, devices and markets Advanced User Interfaces - iphone uses OpenGL ES 3D Navigation - Google Earth and 3D city data Mobile Games Nokia - $7.2Bn market in 2010 (Informa) Nokia N93, N93, iphone, iphone, Sony Sony PS3 PS3 All All use use OpenGL OpenGL ES ES 1.1 1.1 Copyright Khronos Group, 2007 - Page 9

OpenGL ES Central to Mobile 3D JSR 239 Direct Java Bindings to OpenGL ES Java Applications C/C++ Applications Usable directly by applications Usable by higherlevel graphics libraries Close to the hardware API provides portability AND flexibility Hardware OpenGL ES Engines Scenegraph APIs M3G (JSR 184) Games Engines Middleware Libraries Software OpenGL ES Engines Brings advanced 2D/3D graphics to a wide range of OS platforms Copyright Khronos Group, 2007 - Page 10

OpenGL ES Roadmap Stability and reducing fragmentation is currently the key concern - More important than new functionality in the current phase of market development - Industry is still absorbing and implementing current OpenGL ES specifications OpenGL ES 2.0 provisional specification August 2005 OpenGL ES 2.0 Conformance Tests Mid-2007 OpenGL ES 2.0 final specification March 2007 Halti only when NEEDED to address market needs or 2.0 shortcomings OpenGL ES 2.0 accelerated products begin to ship Platforms can ship either 1.X and 2.X libraries or both Cheaper, more flexible than one large driver with both fixed and programmable functions Halti products ship in volume OpenGL ES 1.1 with hardware acceleration remains the Sweet Spot at least through 2008 OpenGL ES 1.1 will continue to be used in lower-cost devices 2007 2008 2009-10? Copyright Khronos Group, 2007 - Page 11

Shaders and Next Generation 3D OpenGL ES 1.1 for fixed function 3D graphics hardware - OpenGL ES 1.1 drivers can be shipped on new generation programmable hardware OpenGL ES 2.0 for new generation programmable 3D hardware - Vertex and pixel shaders programmed through GLSL ES shading language 3D Tomorrow - Shader Programmability Doom 3 s Zombies Unreal s Rocks 3D Today Fixed Functionality Halo s Ice Far Cry s Water Copyright Khronos Group, 2007 - Page 12

OpenGL ES 2.0 Design OpenGL ES 2.0 eliminates all fixed function redundancy - Fixed function transform and lighting, texturing, fog, etc. that can be shader-programmed - Significant reduction in driver complexity but not 100% backwards compatible Adds full Shader Programmability - Vertex and Fragment Shaders - Shader Language (GLSL ES) VERY similar to desktop GLSL - Adds option for off-line compilation by loading binary shaders - Add more control over buffer precision to save memory 8, 10, 16 bit precisions Other features added - Cube Map, 3D and NPOT Textures - Frame Buffer Objects - Point Sprites - Float / Half Float Textures ETC Texture Compression - ROYALTY-FREE - Developed by Ericsson - Better quality than DXTC Copyright Khronos Group, 2007 - Page 13

OpenGL ES 2.0 Programmable Pipeline Triangles/Lines/Points Vertex shaders replace Fixed Function Transform and Lighting API API Processing Primitive Primitive Processing Vertices Transform Transform Shader Vertex Vertex and Lighting Shader and Lighting Assembly Primitive Primitive Assembly Rasterizer Rasterizer Vertex Buffer Vertex Objects Buffer Objects Fragment shaders replace Fixed Function Texture, Color and Fog Environment Texture Texture Environment Fragment Fragment Colour Shader Colour Shader Sum Sum Fog Fog Alpha Test Alpha Test Stencil Depth Depth Stencil Colour Buffer Colour Blend Buffer Blend Dither Dither Frame Frame Buffer Buffer Copyright Khronos Group, 2007 - Page 14

Visual Advantages of Shaders Vertex and pixel shaders lead to better image synthesis with same data - Significant increase is visual quality with NO extra CPU load Copyright Khronos Group, 2007 - Page 15

Safety Critical OpenGL SC 1.0 Targeted at safety-certified avionics and automotive displays - Minimum possible driver size and complexity for DO-178B certification Developed by avionics and automotive graphics vendors - Seaweed, 3Dlabs, ALT, Barco, Diehl, Esmertec, Quantum3D in working group - Positive specification review by major avionics companies and software tools vendors Removes functionality not needed in instrumentation displays: format conversions and color manipulations, blending combinations etc. Safety Critical Profile Restores functionality used by legacy and auto-generated applications: e.g. display lists Copyright Khronos Group, 2007 - Page 16

OpenVG - Accelerated Vector Graphics Vector graphics - basis of popular formats such as Flash and SVG - Bezier curves - scaled and positioned at full quality - not polygon based But 2D vector graphics historically run un-accelerated! - Not effective on low-powered handset CPUs OpenVG is the industry s first native Bezier acceleration API - Enables a new class of hardware acceleration avoiding tesselation to 3D polygons OpenVG is an OpenGL-style, low-level API - Can be used by players such as Flash and SVG or programmed directly Applications and UI SVG/Flash/ Font Packages etc.. 2D Hardware Acceleration Copyright Khronos Group, 2007 - Page 17

OpenVG Pipeline Definition of path, transformation, stroke and paint Stroked path generation Transformation Rasterization Clipping and Masking Paint Generation Image Interpolation Blending Copyright Khronos Group, 2007 - Page 18

OpenGL ES and OpenVG OpenGL ES Accurately represents PERSPECTIVE and LIGHTING OpenVG Accurately represents SHAPE and COLOR Compelling user interfaces require both OpenGL and OpenVG Copyright Khronos Group, 2007 - Page 19

OpenVG Deployment X20 OpenVG 1.0 beginning to be widely deployed - A range of software and hardware implementations are coming to market Vector Graphics Performance X1 Software Implementations X3-7 Layered Implementations over OpenGL ES hardware Dedicated OpenVG Cores low power Copyright Khronos Group, 2007 - Page 20

OpenMAX - Three Layer Solution Media Infrastructure Portability Open, royalty-free standard AL Application Layer Media Application Portability Applications programmed using cross-vendor interfaces IL Integration Layer Media Graph Portability Integrate media networks using standard interconnect protocols DL Development Layer Media Component Portability Develop portable media components using low-level media APIs More media applications available on more platforms for more end-user value! Portable and powerful media processing graphs can flexibly leverage available platform media components Component vendors can ship more advanced functionality across more processors. Media silicon vendors reduce costs and time to market OpenMAX defines three holistically designed media open standards to provide complete media infrastructure portability Copyright Khronos Group, 2007 - Page 21

OpenMAX IL Streaming Media Enables arbitrary multimedia pipelines by plugging blocks together - Componentized architecture abstracts multimedia functionality block interfaces Wide variety of building blocks for imaging, video and audio functions - Encode, decode, apply an effect, capture, render, split, mix, etc Enables blocks from different sources to work together - Blocks can be implemented in software or hardware Portable & Re-usable streaming media building blocks Copyright Khronos Group, 2007 - Page 22

OpenMAX IL Example Graph Standardized component interfaces enable flexible media graphs Includes multi-stream synchronization Allows for custom plug-ins OpenMAX Component Interfaces Enables components to be flexibly connected in any graph topology E.g. send dance beats to 3D application for visualization *.mp4 *.mp4 / / *.3gp File *.3gp File Reader Reader AAC Audio Clock for Clock for AV AV Sync Sync Audio Decoder Audio Decoder Time Data Plug-in Plug-in Audio Renderer Audio Renderer Speakers MPEG4/ H.264 Video Video Decoder Video Decoder Decompressed Video Video Scheduler Video Scheduler Video Renderer Video Renderer Display Example: MPEG-4 video synchronized with AAC audio decode Copyright Khronos Group, 2007 - Page 23

OpenMAX AL Object Oriented Media OpenMAX AL - simplified object-oriented streaming media - Built to enable common use cases but also extensible - Can be implemented over OpenMAX IL OpenMAX AL Media Objects enable PLAY and RECORD of media - Media Objects input and output to Devices - Perform some operation on an input and emit the result as output - Can handle audio, images, video with synchronized audio Objects have control interfaces - Play, Seek, Rate, Audio, Display Region, Metadata Extraction - Record, Camera, Video Encoder, Audio Encoder, Metadata Insertion, Radio, MIDI Camera Audio Input URI Memory Content pipe? OpenMAX AL? Media Object Audio Mix Display Window URI Memory Content pipe Copyright Khronos Group, 2007 - Page 24

OpenMAX Milestone Overview OpenMAX IL 1.0 Specification and Conformance Process Released ST releases Bellagio OpenMAX IL 1.0 Linux Sample Implementation on Source Forge OpenMAX AL 1.0 Provisional Release 1Q06 2Q06 3Q06 4Q06 1Q07 2Q07 3Q07 4Q07 1Q08 OpenMAX DL 1.0 Specification Released OpenMAX DL/IL 1.0 Conformance Tests released OpenMAX IL 1.1 Released with Conformance Tests Copyright Khronos Group, 2007 - Page 25

OpenSL ES Enhanced Audio Audio API space is fragmented - Many proprietary audio APIs of varying functionality native and Java - Even playing a simple sound on different platform requires different code No standard foundation-level API to expose audio hardware acceleration - Audio silicon developers have to support many high-level APIs directly on silicon Applications OpenSL ES is a royalty-free, open standard Native interface to audio acceleration Broad audio functionality for applications, high-level libraries or middleware Middleware, JSR 135, JSR 234 High-level Audio Libraries Media Silicon (Audio HW, CPUs, DSPs) Copyright Khronos Group, 2007 - Page 26

OpenSL ES Functionality Playback of audio files - Playback PCM and encoded content - Good for sound effects; device UI sounds SP-MIDI, Mobile DLS, Mobile XMF - For interactive music and ring-tones Effects & Controls - Music and media player effects - Advanced environmental effects for gaming 3D Audio - Provided for gaming as companion to OpenGL ES 3D position Doppler MIDI messages Preset Reverb Environmental Reverb Volume EQ Pitch Rate Buffer Queues Metadata extraction Virtualization Stereo widening LED & Vibra Copyright Khronos Group, 2007 - Page 27

Three OpenSL ES Profiles Phone, Music and Game - Sufficient to cover all the market segments of importance to OpenSL ES Some profile overlap - Mostly in basic audio functionality Game-centric mobile devices Advanced MIDI functionality, sophisticated audio capabilities such as 3D audio, audio effects, ability to handle buffers of audio, etc. Music-centric mobile devices High quality audio, ability to support multiple music audio codecs, audio streaming support Game Music Phone Basic mobile phones Ring tone and alert tone playback (basic MIDI functionality), basic audio playback and record functionality, simple 2D audio games Copyright Khronos Group, 2007 - Page 28

OpenSL ES Members and Milestones OpenSL ES 1.0 Provisional Release 4Q07 1Q08 2Q08 3Q08 4Q08 1Q09 Conformance Tests and OpenSL ES 1.0 Final Copyright Khronos Group, 2007 - Page 29

OpenSL ES and OpenMAX AL (Enhanced audio API) Audio Effects Advanced MIDI 3D Audio Buffer queues Audio Playback Audio Recording MIDI Video playback Video recording Analog Radio & RDS Camera Image capture & display (Streaming Media APIs) Overlap streaming media functionality uses consistent APIs Copyright Khronos Group, 2007 - Page 30

Complete Khronos Media Stack The Khronos API family provides a complete ROYALTY-FREE, cross-platform media acceleration platform Applications or middleware libraries (JSR 184 engines, Flash players, media players etc.) AL 3D Small footprint 3D for embedded systems Vector 2D Low-level vector acceleration API Platform Media Frameworks IL Playback and recording interfaces Component interfaces for codec integration SOUND Enhanced audio EGL Graphics surface management for efficient mixed mode 2D/3D/video rendering Image Libraries, Video Codecs, Sound Libraries DL Accelerated media primitives for codec development Media Engines CPUs, DSP, Hardware Accelerators etc. Khronos defines low-level, FOUNDATION-level APIs. Close to the hardware abstraction provides portability AND flexibility Copyright Khronos Group, 2007 - Page 31

A SET of native APIs to de-fragment mobile application development. Like DirectX but an open standard for mobile platforms Operating System Abstraction APIs Define APIs to isolate applications from operating system differences SOURCE PORTABILITY for rich media applications Media APIs with defined and tested Interoperability Include Khronos media APIs and define how they work together Copyright Khronos Group, 2007 - Page 32

OpenKODE API Set Software Platform (e.g. Brew, Symbian UIQ, S60, WIPI) Native Applications Java JVM Java Applications User Interface 3D Game Engines Flash/SVG Native Applications Players Players TV/Video/Audio Bindings OpenKODE Core OS Abstraction EGL Surface abstraction and Trans-API Communication AL 3D Vector 2D Streaming Media Enhanced Audio Media Acceleration GPU / DSPs / CPUs Kernel Operating System (e.g. Rex, Symbian OS, Windows Mobile, Linux, Nucleus) Copyright Khronos Group, 2007 - Page 33

OpenKODE Components A conformant OpenKODE must pass all individual conformance tests PLUS trans-api tests to ensure APIs can communicate and operate concurrently Trans-API Conformance Tests 3D Enhanced Audio EGL Trans-API Communication OpenKODE Core Events, IO, files, memory, threads, strings, math, locale, time, networking, windowing, crypto, debug Vector 2D Streaming Media Selected Media APIs PLUS any EGL necessary for those APIs to communicate. (OpenGL ES 2.0, OpenMAX AL and OpenSL ES will be in OpenKODE 1.1 added through extensions to OpenKODE 1.0) POSIX-like APIs - mandatory for a compliant OpenKODE. Standalone conformance tests OpenKODE provides RELIABLE foundation-level acceleration for for media media applications that that MIX MIX multiple multiple media media types types Copyright Khronos Group, 2007 - Page 34

SoftBank Mobile Announcement New-generation mobile platform will provide native access to OpenKODE - POP-I Portable Open Platform Initiative Mandated native API set for advanced UI and games Copyright Khronos Group, 2007 - Page 35

OpenKODE Milestones OpenKODE 1.0 Provisional released at 3GSM 2007 - Twelve months from kick-off meeting to specification on web-site - Encouraging developer feedback before spec finalization Full conformance tests being prepared in parallel with specification - Written by Futuremark tests OpenKODE Core AND trans-api operation OpenKODE 1.0 Provisional Release OpenKODE 1.0 Final Release Target with full conformance tests OpenKODE 1.1? Release add OpenMAX and OpenSL ES support 1Q07 2Q07 3Q07 4Q07 1Q08 2Q08 3Q08 Provisional Conformance Tests to be Released Copyright Khronos Group, 2007 - Page 36

OpenKODE 1.1-3D Video Textures EGL is a Khronos API for abstracting rendering surfaces - OpenKODE 1.1 working to define how surfaces shared between OpenGL ES and OpenMAX - Aiming to eliminate unnecessary copies for power and processing efficiency Without EGLImage the game must copy the OpenMAX buffer into an OpenGL ES texture Data copy wastes cycles and space MPEG Video OpenMAX MPEG-4 Decode OpenMAX Buffer glteximage2d Surface OpenGL ES 3D Engine An EGLImage surface can be used as both the destination of the decode and a texture without copying the data MPEG Video OpenMAX MPEG-4 Decode EGLImage Surface OpenGL ES 3D Engine Copyright Khronos Group, 2007 - Page 37

Raising 2D and 3D Visual Quality State-of-the-art APIs enable compelling consumer displays Advanced functionality, fast interactivity and extremely high quality High-quality 2D graphics using OpenVG Video processing with OpenMAX High-quality 3D displays using OpenGL ES Older generation APIs Provide rudimentary graphics functionality and quality Copyright Khronos Group, 2007 - Page 38

COLLADA = Powerful Authoring COLLADA is a XML database schema for 3D assets - Can hold geometry, animation, visual effects, physics everything to do with a scene COLLADA transports 3D assets between applications - Enables binding of diverse DCC and 3D processing tools into a production pipeline COLLADA can be lossless never lose information - Retains all information - even multiple versions of the same asset COLLADA is an open, archive-grade format that retains meta information - When your DCC tool upgrades, you keep your assets COLLADA is non-destructive and so supports roundtripping of tools to enable powerful authoring pipelines Custom Tools Copyright Khronos Group, 2007 - Page 39

COLLADA Momentum COLLADA 1.4 supported by all major tools and thousands of users - 3ds Max, Photoshop CS3, Blender, DAZ Studio, C4 Game Engine, Poser - NVIDIA FX Composer, Google Earth, Houdini, Maya, Sketchup, and XSI DAZ Studio 1.7 exports COLLADA - Complete export of mesh, skinning, rigging, lighting, camera, and animation data. Caligari truespace7.5 exports COLLADA Adobe Photoshop CS3 Extended imports COLLADA - For texture editing on 3D objects Copyright Khronos Group, 2007 - Page 40

COLLADA UI Bringing 2D vector graphics to COLLADA file format Enables deep intermixing of 2D and 3D graphics Support for 2D and 3D Font libraries Font glyphs to support effects and animation for advanced rendering Copyright Khronos Group, 2007 - Page 41

COLLADA Conditioning COLLADA is an interchange format - not a delivery format or a scene graph Conditioning pipelines take authored assets and: 1. Strips out authoring-only information 2. Re-sizes to suit the target platform 3. Compresses and formats binary data for the target platform Multiple tools create assets and scenes in a COLLADA Database Different target platforms can use the same asset database with the appropriate conditioning pipeline Conditioning Pipeline Conditioning Pipeline Copyright Khronos Group, 2007 - Page 42

glfx Effects Framework COLLADA File format for 3D asset interchange widely adopted by DCC tools vendors and Google, Adobe, Epic etc. COLLADA FX Textures, shaders programs, geometry, control and pass information Conditioning - Existing Khronos standards - New glfx standard glfx File Format - OPTIONAL Contains effects optimized for deployment Runtime file format Application glfx Run-time API Application traverses scene data, uses the glfx Runtime API to use effects information to setup the rendering pipeline Runtime API OpenGL 2.1 / OpenGL 3.0 / ES 2.0 (Also ES 1.1 / Mount Evans) Copyright Khronos Group, 2007 - Page 43

Benefits of One Organization OpenGL and OpenGL ES are now evolving under one IP framework - Design innovations can be freely shared between the APIs OpenGL and OpenGL ES can share same resources and outreach - Common Conformance tests, marketing and web-site, tool chains etc. OpenGL and OpenGL ES Working Groups will remain independent - Both groups will be able to make decisions that best serve their own markets Architectural design expertise Embedded Markets Desktop Markets Market feedback on streamlining functionality Momentum - hundreds of millions of OpenGL ES devices Copyright Khronos Group, 2007 - Page 44

OpenGL Roadmap Synergy OpenGL ES has leveraged desktop OpenGL architecture - OpenGL ES 1.1 streamlined OpenGL 1.5 - OpenGL ES 2.0 streamlined OpenGL 2.0 and the GLSL shading language Next Generation OpenGL 3 and Mount Evans leverages OpenGL ES 2.0 - Learning from the streamlining of OpenGL ES 2.0 core OpenGL ES will benefit from the architectural innovation of Mount Evans - Nexgen OpenGL ES Halti will use desktop innovations for performance and functionality OpenGL 1.5/2.0 Architectural Foundation OpenGL ES 1.1/2.0 Functional Streamlining OpenGL 3 / Mount Evans Architectural Streamlining and next generation functionality Halti Next generation functionality for embedded markets Copyright Khronos Group, 2007 - Page 45

OpenGL 3 and Mount Evans OpenGL 3 before end of 2007 - Architectural streamlining of OpenGL - Elimination of fixed functionality evolving work in OpenGL ES 2.0 - New object model for speed and simpler programming - GL allocates handles, Direct editing, Efficient state grouping, Efficient sharing OpenGL 3 provides backwards compatibility - OpenGL 3 requires a new context - Both OpenGL 2.1 and OpenGL 3 contexts may bind to same drawable - Legacy code and LP code may peacefully co-exist - developers migrate at their own pace Mount Evans early 2008 - Adds significant new features to OpenGL 3 - Geometry shaders - Greatly increased integer support in the OpenGL Shading Language - Stream out of vertex data to a buffer object - Texture arrays - Texture buffer objects - Numerous new texture formats - Instanced rendering Copyright Khronos Group, 2007 - Page 46

OpenWF Windowing Foundation APIs to enable window systems to be built entirely on open standards - System-level APIs not intended for application developers First API is 2D screen composition API - Other APIs display control? 3D Composition Framework APIs? Portable applications use OpenKODE Core window system abstraction calls to communicate with the Windowing System OpenKODE Core Advanced Window System Applications and Media Engines Khronos Media APIs Khronos media APIs can be used by applications - and the window system to render and compose UI elements e.g. with 3D effects Window System can use OpenWF 2D Screen Composition API to bring application and UI elements together on the screen Screen Composition GPU Multi-tasking GPU accelerates multiple applications, window system UI / transitions and screen composition Copyright Khronos Group, 2007 - Page 47

Khronos 2007 Milestones iphone launched uses OpenGL ES 1.1 for UI and display composition OpenMAX AL 1.0 Provisional Released OpenSL ES 1.0 Provisional Released OpenMAX IL 1.1 Public Release with Conformance Tests OpenKODE 1.0 Provisional Released COLLADA Adopted by Adobe for Photoshop CS3 OpenVG 1.0 Conformance Tests Released OpenVG 1.0 Reference Implementation open sourced glfx Working Group Announced OpenWF Working Group Announced COLLADA UI TSG Announced OpenGL 3 features revealed OpenVG 1.1 Public Release OpenKODE 1.0 Final Release COLLADA 1.4.2 Conformance Tests Release OpenGL 3 Release 1Q07 2Q07 3Q07 4Q07 Copyright Khronos Group, 2007 - Page 48

Why Are Khronos Standards Key? Foundation Level APIs to enable software to effectively use silicon - Close to the silicon fundamental functionality needed on every platform Synergistic mix of authoring and acceleration standards - Being designed together Hundreds of man years invested - Beyond any single company now to produce specifications of this breadth and depth Royalty-free - Khronos is committed to generating market opportunities for its members and the industry Copyright Khronos Group, 2007 - Page 49

More Details? Khronos Mobile and Embedded Graphics track Here at ARM Developer Conference 9AM Thursday Speakers from Khronos, Imagination, ARM Copyright Khronos Group, 2007 - Page 50

Help Khronos Help You! Please consider joining Khronos! - Developer feedback and participation is always very welcome Please review our specifications and provide feedback - Public forums and developer resources here - http://www.khronos.org/developers/ Slides and Khronos membership details at www.khronos.org Copyright Khronos Group, 2007 - Page 51