Research Placement Technical Report Cross-Platform OpenGL Shader Authoring Matthew O'Loughlin 2 nd June, 2012 Supervisor: Dr. Christian Sandor Introduction During my time working with the Magic Vision Lab at the University of South Australia, I was primarily tasked with improving the process of developing OpenGL shaders for use across multiple platforms. OpenGL shaders are small programs which can be executed on a GPU to create various different effects by attaching them to an OpenGL scene. The programs are written in GLSL (GL Shading Language), a C-like language which takes gives developers control over the graphics pipeline in a more portable way than direct assembly programming. The OpenGL standard currently provides three processors which can be programmed using shaders (Vertex, Fragment and Geometry); the system proposed will provide support for developing on all programmable units. The vertex processor is responsible manipulating data which is passed through the OpenGL pipeline for each vertex in the scene. After being transformed, the vertices generate primitives (e.g. triangles) and passes this information to the fragment processor, which is primarily responsible for computing the colour of each fragment it is passed. OpenGL Graphics Pipeline [1]
Problem Statement Work done at the Magic Vision Lab is often run on two different platforms; OpenGL for desktop computers or head-worn displays (HWD), and OpenGL ES - a subset of OpenGL designed for mobile devices such as the iphone. The two platforms use slightly different languages for shader development, with subtle differences in function availability and variable declaration, as well as the ability to read certain values from the graphics pipeline. Writing shaders to which maintain equivalent semantic meaning on both can be a tedious job, often presenting a number of challenges to team members responsible for porting code between them. This presents a major bottleneck in graphics application development, with developers often being required to repeatedly move between platforms for testing and coding. Development cycle for creating iphone shaders. In the following project, I present a system which provides an efficient model for creating and deploying OpenGL shaders across multiple platforms without rewriting code. The system will also offer debugging functionality for discovering issues with a shader program. Related Work As the popularity and complexity of shaders grew, a number of development environments were created to provide a simpler way for developers to quickly create shaders and receive feedback on changes as they're made. glsldevil (Strengert et al) is a debugging tool for the OpenGL shader pipeline, intercepting calls to the OpenGL framework and allowing a user to follow transitions through the OpenGL state machine while watching variables being altered. While there existed many hardware-aware OpenGL state machine debuggers, glsldevil provided the first feature-rich shader debugger. Many ideas from this research into hardware-aware shader debugging can be incorporated into the system. RenderMonkey [2] is a discontinued product from AMD offering OpenGL ES viewing window with full compatibility with shaders for mobile devices, one of the major features of the proposed system. While this system provides a powerful development system, its licensing and platform restrictions make it unusable in many circumstances.
Approach Given the level of related work on the field of computer graphics development, it was decided that starting a new project would be inefficient, with multiple open source projects already in existence. After considering numerous projects, I began extending the functionality of a Shader Maker [3], developed by members of the Computer Graphics lab at the Technical University of Clausthal, Germany. This program had already been adopted by members of the Magic Vision Lab, who expressed a number of issues with the limitations of the current implementation. In the first phase of development, I began improving the usability of the program with several small improvements which would be expected in most development environments. This was a relatively simple set of tasks, with minimal testing carried out to ensure the program ran as expected. The next goal was to have an emulated OpenGL ES surface within the Qt window which could load OpenGL ES shaders and run them as they would on an iphone. This functionality would provide the program with an advantage over similar development tools by streamlining the process of developing shaders for mobile devices on a desktop computer. Developers will be given the opportunity to make changes to shader programs more frequently, allowing them to experiment with various effects without consuming large amounts of time. At the Magic Vision Lab where effects must be believable by the user, this room to experiment is highly important. The next phase of development would provide the system with the ability to translate code between the two languages. The desired functionality would provide developers with an export tool which could provide equivalent code for an alternate platform with a single click. Taking a write-once approach to deployment across multiple platforms would again allow more time to innovate and optimise shader effects. The two languages have a similar syntax though there are a number of differences to consider, both syntactically and semantically. An intelligent parser and translation module would be required to ensure translated code performs in the same way as the original code. The system would then be supplemented with a ray-tracing visualiser, providing fast and accurate information of the path light takes through the scene. Demonstration The Shader Maker program runs on the Qt UI framework to provide portability across all major platforms. The initial interface was lacking in a number of areas which were addressed in the first phase of development.
Many interface issues of the system caused the development process to be slow and frustrating to users, providing only a small viewing window and offering little assistance for writing and debugging shaders. The system was improved and supplemented with new functionality to produce a more usable development experience. It now provided automatic compilation and linking of shaders, allowing developers to gain instant feedback of the effect of changes in their code, as well as assistance with program debugging. Using EGL [4] (a library used to provide OpenGL ES on the desktop), the system is fitted with a switching mechanism to change the view widget to render either of the two
platforms which developers are writing for. This major feature provides a productivity boost to developers working on multiple platforms. However, there is an issue with frame rates being reduced on most systems using the OpenGL ES emulation, which could be analysed in future work to deliver speeds equal to those from devices with native support. Discussion It is apparent that the proposed changes to the Shader Maker program have the potential to drastically improve the development process for software relying heavily on OpenGL shaders, especially across different platforms. Aside from the obvious benefits of the implemented usability changes, a developer could benefit from the OpenGL ES emulation mode to make fast changes to a shader program before deploying it on an iphone. With the original process, developers would write the shader on the desktop and test it on the device, moving between the platforms a number of times while attempting to achieve the desired result. That process can be discarded to allow developers to streamline cross-platform development by utilising the features of the new system. New System: Develop entirely on desktop, then deploy
Conclusion and Future Work The system described and demonstrated in this project has the potential to save development time for anybody working on cross-platform graphics applications utilising the programmable shader pipeline. Developers will have more time for experimentation and creativity by removing the need to manually rewrite large amounts of code for another target platform. This ability to test a larger number of effects will be an important feature for augmented reality researchers seeking to create the most immersive effects possible, to ensure demonstrations provide a believable experience to users. Future work will include completion of new features proposed, including an export function to provide equivalent ESSL code from an input GLSL file or vice versa. An existing toolkit for visualising ray tracing within an OpenGL scene will also be integrated to provide real-time analysis of lighting effects generated by a shader program. This extension will provide an important level of feedback to the developer and will promote a deeper understanding of ray-tracing computations and ensure program correctness. (Gribble et al. 2012) Creating the exporter program would require extensive research into the grammar of the two languages, and the functions available to each. From this work, an algorithm would be devised to parse a GLSL program, locate any GLSL code which would be invalid ESSL syntax, then convert it to equivalent ESSL. This poses a number of problems: 1. Several built-in data types in the GL shading language are not available in the ES shading language. These would have to be replaced with code with equivalent semantic meaning. 2. Type qualifiers between the two languages have several differences, such as the declaration of type precision in ESSL, as shown in the grammar excerpt [6] below. declaration: function_prototype SEMICOLON init_declarator_list SEMICOLON PRECISION precision_qualifier type_specifier_no_prec SEMICOLON precision_qualifier: HIGH_PRECISION MEDIUM_PRECISION LOW_PRECISION
Much of this translation work has been solved in existing projects, especially ANGLE [5] which provides translation between several high-level shading languages, including GLSL and ESSL. The functionality from this project could be incorporated into the shader authoring environment to provide the ability to easily translate between languages. References C. Gribble, J. Fisher, D. Eby, E. Quigley, G. Ludwig, 2012. Ray Tracing Visualization Toolkit. In ACM SIGGRAPH Symposium on Interactive 3D Graphics, 2012. M. Strengert, T. Klein, T. Ertl, 2007. A Hardware-Aware Debugger for the OpenGL Shading Language. In ACM SIGGRAPH/Eurographics Workshop on Graphics Hardware 2007 [1] OpenGL Pipeline - http://cse.csusb.edu/tong/courses/cs520/images/glsl/shader-arch.jpg [2] RenderMonkey - http://developer.amd.com/archive/gpu/rendermonkey/pages/default.aspx [3] ShaderMaker - http://cg.in.tu-clausthal.de/teaching/shader_maker/index.shtml [4] OpenGL ES 2.0 Emulator - http://www.malideveloper.com/opengl-es-20-emulator.php [5] Almost Native Graphics Layer Engine - http://code.google.com/p/angleproject/ [6] OpenGL ES 2.0 Specification - http://www.khronos.org/registry/gles/specs/2.0/glsl_es_specification_1.0.17.pdf