3RUWDEOH1DWLYH0RELOH 0HGLD$SSOLFDWLRQV 1HLO7UHYHWW 9LFH3UHVLGHQW(PEHGGHG&RQWHQW19,',$ 3UHVLGHQW.KURQRV*URXS Copyright Khronos Group, 2007 - Page 1
3HUYDVLYH0RELOH0HGLD&RPSXWLQJ Handsets are becoming personal computing devices - not just phones - Your most personal computer mobility, connectedness and numerous sensors Sophisticated media processing will be central to handheld revolution - Graphics and media will become as pervasive as it is on the PC Diverse applications will drive the need for handheld accelerated media - UI, navigation with 3D terrain and cities, TV/Video, augmented reality, games 100Ms / year 100Ks / year 10Ms / year Media Platforms 80 s Workstations Copyright Khronos Group, 2007 - Page 2 90 s PCs 00 s Handhelds
0RELOH0HGLD,VVXHV Accelerated 3D games were expected to drive handheld media revolution - Significantly increased revenue per title for 3D games over 2D games is possible BUT there are still significant difficulties and obstacles - The market for mobile games is not growing as quickly as expected What are the issues holding it back? - What we can do to make it happen faster? - What opportunities it will create? Spider-Man 2 Activision Spider-Man 2: The Hero Returns Sony Pictures Spider-Man 2 3D: NY Subway Sony Pictures 2D Software 3D Accelerated 3D Copyright Khronos Group, 2007 - Page 3
0RELOH0HGLD,VVXH)UDJPHQWDWLRQ Every handset is unique from the programmers perspective - Differences in operating system functions, Java implementations and media functionality Severe platform fragmentation exists today - ISVs need to port to and support 100s (even 1000s) of source variants of each title Symbian 7, 8, 9, UIQ, S60 PocketPC / Windows Mobile / WinCE Many Linux variants (and growing) Java MIDP-1, MIDP-2, JSR fragmentation Numerous RTOS with no consistently defined OS abstraction or media portability Nucleus, Synergy etc. Copyright Khronos Group, 2007 - Page 4
0RELOH0HGLD,VVXH/RZ3HUIRUPDQFH Java is most widely used programming language for mobile applications - Well developed over-the-air deployment infrastructure BUT - Java doesn t provide optimal media performance - Native acceleration can provide X5 higher performance for 3D graphics x2-20 Performance Accelerated Java Higher quality games x3-5 Performance Accelerated Native Seamlessly accelerated mixed-media Java Compelling native mixed-media applications Accelerated UI Un-accelerated Java Measured by Kishonti Java (fps) OpenGL ES (fps) Native Advantage Simple Games Nokia N93 18.6 77.8 x4.2 Sony Ericsson M600 6.0 30.6 X5.1 Copyright Khronos Group, 2007 - Page 5 GLBenchmark direct port of identical Java benchmark
$Q2SHQ6WDQGDUGV6ROXWLRQ Lack of media Performance Functionality Fragmentation C Native Programming Environment Familiar from PCs and consoles Consistent APIs on every Handset Reliable functionality Native Media Acceleration APIs Open standard, cross-platform, native APIs for media application development Advanced User Interfaces Rich media HTML/Flash/SVG Browsers Navigation and location-based services TV, video and music services Games Khronos develops Foundation-Level API API Standards Close-to-the-silicon access access to to media media acceleration to to create create a a robust robust foundation for for media media middleware and and applications Copyright Khronos Group, 2007 - Page 6
.KURQRV 2SHQ0HGLD6WDQGDUGV Open International Membership Any company is welcome Funded by membership dues - $6K / year Open Standards Publicly available on web-site To be used by any company in any products Open Standards for Media Authoring and Acceleration Creating Market Opportunities Bringing together the industry s leading media technology experts Investing hundreds of man years to create state-of-the-art media API standards Making those standards available to the industry ROYALTY-FREE Copyright Khronos Group, 2007 - Page 7
Around 100 companies creating media authoring and acceleration standards Copyright Khronos Group, 2007 - Page 8
.KURQRV6WDQGDUGV(FRV\VWHP Cross-platform graphics authoring/acceleration ecosystem Safety Critical 2D/3D Comprehensive Mobile Media Acceleration Solution Cross platform desktop 3D Effects Framework 3D Authoring Dynamic Media Authoring Dynamic Media Authoring Standards 2D/3D Vector 2D Streaming Media Enhanced Audio Embedded Media Acceleration APIs DirectX-like set of native APIs Includes mixed media acceleration and OS portability APIs Copyright Khronos Group, 2007 - Page 9 Composition Working Group Hardware acceleration for window systems
$GRSWLRQRI(PEHGGHG.KURQRV$3,V Market adoption in media-accelerated handsets We Are Here - OpenGL ES 1.1 is widespread - OpenVG is in rapid adoption - OpenMAX is being implemented - OpenSL ES is being designed 100% 3D Graphics Vector 2D Streaming Media Enhanced Audio Mid-2004 OpenGL ES 1.1 Spec release Mid-2005 OpenVG 1.0 Spec release Beginning-2006 OpenMax IL 1.0 Spec release Mid 2007 OpenSL ES 1.0 Spec release End-2007 Copyright Khronos Group, 2007 - Page 10
$GYDQWDJHVRI0HGLD$FFHOHUDWLRQ Faster Performance at Higher Quality Hardware delivers smoother interaction with much better looking graphics 18 16 14 12 10 8 6 4 2 0 ARM9 32:(5()),&,(1&< 32-Bit RISC at 400MHz DSP at 175MHz APA 512 MiMagic 6 Software 3D Accelerated 3D Less Power Hardware accelerators exploit media pipeline parallelism and caching for a x10 increase in power efficiency over software Better User Experience Smaller screens need more advanced graphics processing per pixel Copyright Khronos Group, 2007 - Page 11
3HQHWUDWLRQRI1DWLYH0HGLD$3,V 900 800 700 600 MM phones Non 3D MM phones Native API based phones Millions of units Jon Peddie Research Handheld Multimedia Devices report 500 400 300 200 100 0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Copyright Khronos Group, 2007 - Page 12
1DWLYH)UDJPHQWDWLRQ Software Platform (e.g. Brew, Symbian UIQ, S60, WIPI) Native Applications JAR File Java JVM Java Applications Carrier OTA Provisioning No Native OTA Deployment User Interface 3D Game Engines Flash/SVG Players TV/Video/Audio Players Bindings Different OS APIs on every handset Proprietary media APIs, single-threaded, not tested for mixed-media operation Media Accelerator Silicon GPU / DSPs / CPUs Kernel Operating System (e.g. Rex, Symbian OS, Windows Mobile, Linux, Nucleus, Synergy) CPU Copyright Khronos Group, 2007 - Page 13
OpenKODE is a SET of native APIs to provide SOURCE PORTABILITY Operating System Abstraction Isolate applications from operating system differences Source Portability Media Interoperability Define how media APIs work together to enable rich mixed-media applications Copyright Khronos Group, 2007 - Page 14
2SHQ.2'($3,6HW Native Applications OpenKODE Core OS Abstraction EGL Trans-API Communication AL 3D Vector 2D Streaming Media Enhanced Audio Media Acceleration GPU / DSPs / CPUs Kernel Operating System (e.g. Rex, Symbian OS, Windows Mobile, Linux, Nucleus) The open standard equivalent of DirectX Providing a consistent set of functionality for media applications Copyright Khronos Group, 2007 - Page 15
&RKHUHQW0HGLD$FFHOHUDWLRQ6WDFN Software Platform (e.g. Brew, Symbian UIQ, S60, WIPI) Native Applications Java JVM Java Applications User Interface 3D Game Engines Flash/SVG Players TV/Video/Audio Native Applications Players Bindings OpenKODE Core OS Abstraction EGL Trans-API Communication AL 3D Vector 2D Streaming Media Enhanced Audio Media Acceleration GPU / DSPs / CPUs Kernel Operating System (e.g. Rex, Symbian OS, Windows Mobile, Linux, Nucleus) Copyright Khronos Group, 2007 - Page 16
,QQRYDWLYH0L[HG0HGLD$SSOLFDWLRQV Breakthrough mobile applications will leverage the unique capabilities of mobile devices: Use of handset sensors - Cameras for video and images, sound Mobility - With GPS location awareness Connectedness - Unique information flow and social interactions Pictures We started at the Eiffel Tower Video Family fun at the fountain OpenKODE enables innovative applications that MIX diverse media types - Augmented Reality - 3D over live video - TV channel selector in 3D - 3D avatars linked to speech - Graphics visualizations linked to music Handheld GeoBlog Concept Navigation using 2D maps, satellite, terrain and 3D city data. GPS phone captures and uploads images, audio and video. Location tagging inserts media correctly into the map. Blog can be viewed on handheld or PC. Opens innovation in Blogs, guides, journalism etc etc etc.. Copyright Khronos Group, 2007 - Page 17
2SHQ.2'(&RPSRQHQWV Trans-API Conformance Tests A conformant OpenKODE must pass all individual conformance tests PLUS trans-api tests to ensure APIs can communicate and operate concurrently 3D Enhanced Audio EGL Trans-API Communication OpenKODE Core Events, IO, files, memory, threads, strings, math, locale, time, networking, windowing, crypto, debug Vector 2D Streaming Media Selected Media APIs PLUS any EGL necessary for those APIs to communicate. (OpenMAX and OpenSL ES will be in OpenKODE 1.1) Mandatory for a compliant OpenKODE. Standalone conformance tests OpenKODE provides foundation-level acceleration for for advanced user user interfaces and and media media applications that that mix mix multiple multiple media media types types Copyright Khronos Group, 2007 - Page 18
2SHQ.2'(&RUH² 26$EVWUDFWLRQ Based on POSIX as far as possible - POSIX ES also draws on C89, C99 - Some added functionality events and IO functions Threads - Thread creation and synchronization Event system - An event system which abstracts the event system of the platform s OS - e.g. quit, pause and resume, window resize, input change, timer, and socket ready to read or write - An OpenKODE application may be written as either loop-in-application, where it contains the top-level loop processing an event each iteration, or loop-in-framework, where the framework calls an event handler for each event Utility functions - Conversions from string to number and vice versa, random number generation, memory allocation, memory and string copying, comparison and scanning, and assertions and logging Math - 32-bit floats, and analogs of many of the C standard math library functions Copyright Khronos Group, 2007 - Page 19
2SHQ.2'(&RUH² 26$EVWUDFWLRQ² &RQW Time and timers - Analogs of C time functions, as well as OpenKODE-specific functions for more accurate timekeeping, and for timers which generate events File system - Abstraction of platform s file system to a virtual file system - Application accesses only certain well-known locations (such as the files that came with the application ) to be written portably - File functions are analogs of familiar C and [POSIX] functions Networking - An API similar to BSD/[POSIX] sockets, but with different API semantics such that the event system is used to notify when a socket is ready to send to or receive from Input/output - Access inputs (such as buttons) and outputs (such as vibrate) in an extensible way, while specifying a small range of inputs and outputs that are likely to be present, such as game keys Windowing - Support for multiple non-full-screen windows - Simple manipulation of such windows - resizing and maximizing Copyright Khronos Group, 2007 - Page 20
2SHQ.2'(0LOHVWRQHV OpenKODE 1.0 Provisional released at 3GSM 2007 - Twelve months from kick-off meeting to specification on web-site - Encouraging developer feedback before spec finalization in 3Q07 Full conformance tests will be released in fall 2007 - Written by Futuremark tests OpenKODE Core AND trans-api operation OpenKODE 1.0 Provisional Release OpenKODE 1.0 Final Release Target OpenKODE 1.1 Release add OpenMAX and OpenSL ES support 1Q07 2Q07 3Q07 4Q07 1Q08 2Q08 3Q08 Conformance Tests to be Released If you are a developer download the spec and give us your feedback! Spec at www.khronos.org/openkode/ Feedback forums at www.khronos.org/message_boards/ Copyright Khronos Group, 2007 - Page 21
2SHQ.2'( '9LGHR7H[WXUHV EGL is a Khronos API for abstracting rendering surfaces - OpenKODE 1.1 defining how EGLImage surfaces shared between OpenGL ES and OpenMAX - Eliminates unnecessary copies for power and processing efficiency Without EGLImage the game must copy the OpenMAX buffer into an OpenGL ES texture Data copy wastes cycles and space MPEG Video OpenMAX MPEG-4 Decode OpenMAX Buffer glteximage2d Surface OpenGL ES 3D Engine An EGLImage surface can be used as both the destination of the decode and a texture without copying the data MPEG Video OpenMAX MPEG-4 Decode EGLImage Surface OpenGL ES 3D Engine Copyright Khronos Group, 2007 - Page 22
2SHQ.2'(5ROO2XW 2XW OpenKODE 1.0 allows media API selection - To enable widespread early adoption Simple build out of existing OpenGL ES adoption - Expecting rapid industry momentum Can be deployed on almost any OS - Including RTOS for media on feature phones OpenKODE 1.0 OpenKODE Core API has no silicon dependency and takes <50KB. OpenKODE provides enhanced 2D/3D mixed mode operation OpenKODE 1.1 Full DirectX functional equivalence. Robust trans- API functionality and testing OpenGL ES is widely deployed today 2008 2007 Now Copyright Khronos Group, 2007 - Page 23 1XFOHXV
2SHQ.2'(0RPHQWXP Futuremark developing Conformance Tests for OpenKODE. Graphic Remedy developing OpenKODE profiling tools. Ardites offering consultancy services OpenKODE Core can drop-in alongside OpenKODE media stacks. Typically middleware solutions provide native OTA provisioning over Java networks sometimes with portable binary capability OpenKODE 1.0 announced in Feb07 Momentum is already building quickly Announced OpenKODE media stacks with OpenGL ES and increasingly OpenVG Softbank has adopted OpenKODE for its POP-i platform Tools and Services OpenKODE Core and Middleware Silicon Acceleration Operators FREE OpenKODE Core PC implementation from Acrodea http://www.acrodea.co.jp/en/openkode Copyright Khronos Group, 2007 - Page 24
6RIW%DQN0RELOH$QQRXQFHPHQW New mobile platform will provide native access to OpenKODE - POP-I Portable Open Platform Initiative Mandated native API set for advanced UI and games Copyright Khronos Group, 2007 - Page 25
2SHQ.2'(1DWLYH(FRV\VWHP Carriers Open standards to simply specify a native, cross-platform media API stack Compelling media applications increase revenue through existing provisioning, billing, DRM certification infrastructure Native OTA provisioning uses existing certification, security, billing, lifecycle management infrastructure Proven mixed media architecture. Multi-vendor openstandards reduces risk & cost Cross-platform native APIs reduce source fragmentation and increase performance Handset OEMs Native, cross-platform media API stack Content Providers Copyright Khronos Group, 2007 - Page 26
/HYHUDJLQJDQG$FFHOHUDWLQJ-DYD JAR File Java OTA Provisioning Native Installer Native Applications Software Platform Java JVM Java Applications JSR 184 / 239 Other JSRs Existing provisioning, billing and security systems can be used to OTA provision native OpenKODE applications. Native applications will need certification as Brew does today Media Accelerator Silicon Kernel Operating System CPU Copyright Khronos Group, 2007 - Page 27
&RPSRVLWLRQ8,DQG2SHQ.2'( Applications and Media Engines OpenKODE media APIs can be used both by applications - and the window system to render and compose UI elements e.g. with 3D effects Portable applications use OpenKODE Core window system abstraction calls to communicate with the Windowing System OpenKODE Core Media APIs Built using open standard APIs Advanced Window System Window System can use a new Khronos Screen Composition API to bring application and UI elements together on the screen Screen Composition API GPU Multi-tasking GPU accelerates multiple applications, window system UI / transitions and screen composition Enabling advanced composited window systems to be built entirely from open standards Copyright Khronos Group, 2007 - Page 28
:K\$UH.KURQRV6WDQGDUGV.H\" Foundation Level APIs to enable software to effectively use silicon - Close to the silicon fundamental functionality needed on every platform Hundreds of man years invested - Beyond any single company now to produce specifications of this breadth and depth Reduces development and deployment costs - Working proactively to prevent every platform creating a different set of media APIs Royalty-free - Khronos is committed to generating market opportunities for its members and the industry Copyright Khronos Group, 2007 - Page 29