A Low Cost Tile-based 3D Graphics Full Pipeline with Real-time Performance Monitoring Support for OpenGL ES in Consumer Electronics

Transcription

1 A Low Cost Tile-based 3 Graphics Full Pipeline with Real-time Performance Monitoring Support for OpenGL ES in Consumer Electronics Ruei-Ting Gu, Tse-Chen Yeh, Wei-Sheng Hunag, Ting-Yun Huang, Chung-Hua Tsai Chung-Nan Lee, Ming-Chao Chiang, Shen-Fu Hsiao, Yun-Nan Chang, Ing-Jer Huang Abstract This paper presents a 3 graphics engine which is specifically designed to minimize the hardware cost while providing sufficient computing capability for consumer electronics with small to medium screen sizes (up to 800x600) such as digital television. The presented 3 engine consists of a fixed full 3 graphics pipeline for both geometry and rendering operation. This engine provides a standard AHB interface that makes it easily to be integrated into an AMBA-based SoC. The development of the 3 engine has gone through a rigorous design process: starting from system modeling (using System-C), RTL implementation, hardware/software co-simulation and FPGA verification to test chip fabrication. This 3 engine provides 8.34M vertices/s and 278M pixels/s in maximum performance at 139 MHz using 0.18 silicon technology with 987K gates that is sufficient for most applications for digital television. At the same time, a complete OpenGL-ES 1.1 API, windowing system, Linux operating system, device driver and a 3 performance monitoring tool have been developed for our 3 engine. This performance monitoring tool provides run-time performance information include frame rate, triangle rate, pixel rate, involved OpenGL function list, function counts, memory utilization and etc. Moreover, a built-in real-time AHB bus tracer is also provided to monitor the bus activities of the 3 engine and other components on the system bus. The bus tracer captures on-chip bus signals at ether cycle accurate or transaction levels and applies real-time compression to both levels of signals. With the performance monitoring tool and the bus tracer, the 3 application developer can easily analyze the communication of the components and fine tune the 3 application to optimize the entire SoC system performance and to satisfy performance/cost constrains on consumer electronics. Both of the hardware and software have been carefully verified and demonstrated on FPGA using ARM versatile SoC develop board. Index Terms 3 graphics pipeline, OpenGL ES, geometry engine, rendering engine, 3 graphics performance monitoring. I. INTROUCTION Accompanying with the improvement of silicon process the single chip has a capacity for some computation exhaustive jobs such like 3 graphics calculations. Hardware acceleration for 3 graphics processing is no longer only proper to the desktop PCs or workstations but also to the embedded system. In the 3 graphics field, it is a mature technology in the PC world. But for the consumer electronics how to build a 3 graphics chip with low cost, low power but enough performance has became a new challenge. The famous graphic IP provider Imagination Technologies TM [1] develops the PowerVR IP cores for graphics named MBX [2][3]. The integrated PowerVR solutions are used in a wide range of applications, from the most performance-hungry 3-enabled VB set-top box to the most power-sensitive portable media units. The Bitboys TM [4] also provide 3 and Vector Graphics Acceleration, called Acceleon TM, for Mobile Phones and Embedded evices. The ATI s [5] Imageon TM products accelerate 2 & 3 graphics, gaming, video applications for many mobile phones. Another IP provider, Falanx Microsystems TM [6], provides the Mali TM Graphics Solution of 3 graphics accelerator IP Cores that support OpenGL ES [7] v1.1 and H.264 functionality and targets different performance, power and die size levels. The GSHARK-TAKUMI's [8] product family lineups graphics engine IP cores for embedded systems and personal information devices such as mobile phones that performing real time display of 3 graphics And also the academic field, there are several outstanding implementations of 3 graphics for consumer electronics [9-15]. [9, 13, 14] presented very high performance 3 graphics SoCs with programmability that can fully support OpenGL ES standard API. And another research team [12] also implemented a low power 3 graphics engine using RAM fabrication process that provides very low power consumption. Although several 3 graphics hardware accelerators have been proposed for consumer electronics, a careful balance between the application features and the

2 hardware cost is still a challenging problem. This paper presents a 3 graphics engine which is specifically designed to minimize the hardware cost while providing sufficient computing capability for consumer electronics with small to medium screen sizes (up to 800x600) such as digital television. On the other hand, it is hard to measure the run time 3 hardware performance. The AMBA-based SoC platform was selected to integrate the presented 3 graphics engine because the AMBA is the most popular system bus for SoC design. This engine provides a standard AHB interface that makes it easily to be integrated into an AMBA-based SoC. In addition, a built-in real-time AHB bus tracer is provided to monitor the bus activities of the 3 engine and other components on the system bus such that the 3 application developer can fine tune the 3 application to optimize the entire SoC system performance. Furthermore, the development of the 3 engine has gone through a rigorous design process: starting from system-c modeling to test chip fabrication. In addition, a complete OpenGL-ES 1.1 library, windowing system, Linux operating system, device drivers and a 3 performance monitoring tool have been developed for our 3 engine. The 3 engine provides 8.34M vertices/s and 278M pixels/s in maximum performance at 139 MHz which is targeting at applications of the digital television, and the area cost is about 987K logic gates including the geometry engine(295k), rendering engine(395k) and bus tracer(71k). II. SYSTEM MOELING AN PERFORMANCE ANALYSIS Before hardware implementation, we build the hardware models using system-c and integrate into the Coware TM Platform Architect [16] which is a System-C based simulation and analysis tool with graphical user interface. Figure 1 System model overview on Platform Architect 3 Chip There is a performance analysis after the system model was build. This performance can tell us that where the performance bottleneck is and can also verify the hardware functions and timing behavior. Figure 2 shows the performance analysis result example of system modeling. The benchmark is 100 cubes include 1200 vertices and render 4 frames with hardware acceleration. It shows the total execution time of each component. Thus we can find out the performance bottleneck. In this example, the Geometry Module (dark green color) and Rendering Module (purple color) are using hardware but Tile ivider is using software implementation. And the ratio of accelerator execution time to total execution time is very small. The green color is the CPU execution time that means the software performance. The result showed that most of the execution time is using for software that performs initial memory (clear frame buffer) and the software tile dividing. Thus we decide to add a hardware tile divider. After adding hardware tile divider model and compare to the software implementation so that we can know the performance effect and costs of the hardware tile divider and so as other hardware components. GM RM IAHB AHB Total time : 110,508,000cycle Platform : CPU + GM + RM in burst mode 501,486 GM Initial Memory 68,491,400 RM Tile ivider Benchmark : 12 triangle (Cube) *100 Four frame RM GM Initial Memory 36,591,000 Initial memory GM Tile ivider RM Memory Module : SRAM Priority GM > RM > CPU 1,430,370 Tile ivider Figure 2 Performance analysis on Platform Architect While building the system model and integrating into Platform Architect, the interface of each component is exactly the same with the real hardware. It is important because that the Platform Architect is not only for simulation and system modeling, but it can also do the hardware and software co-simulation for functional verification. We can reuse the system platform and have a very fast functional verification by only replacing the models to real hardware RTL designs. III. HARWARE/SOFTWARE IMPLEMENTATION Figure 3 is the proposed 3 graphics engine block diagram. This engine integrates geometry and rendering

3 engines for 3 computation, a bus tracer that provides the on-chip bus activity trace and a standard AHB interface that can connect to AMBA. The feature of this design is that the chip provides a complete AHB bus that makes itself a SoC platform so that users can add hardware into it and also provides the interface to connect to another platform as a component. 3 Graphics Engine Test Chip Geometry Engine Rendering Engine AMBA AHB Bus Tracer Master Interface External Master Input Next pixel Input M U X Pixel info. REGs REGs Interpolation parameter REGs REGs M U X S E L E C T O R PE1 PE2 PE3 PE4 M U X OUTPUT Arbiter ecoder Slave Interface Figure 3 3 graphics engine architecture External Slave A. Hardware evelopment According to the OpenGL ES, we divide the 3 pipeline into two operations. The first is geometry transform that transforms 3 vertices, normals, and texture coordinates to produce a primitive drawing passes to the second operation rendering. The rendering operation includes two sub functions called rasterization and pre-fragment. Rasterization converts the primitive drawing to a two-dimensional image and assigns a color and a depth. Pre-fragment modifies the pixel, produced by rasterization with window coordinates, in the framebuffer by series of test and then displays on screen. Thus the hardware has two main modules to provide geometry transform and rendering. The Geometry Module (GM) includes culling, clipping, lighting, model view transformation, view transformation, projection transformation, and vertex normal transformation. Because of such complex calculation process, we divide the geometry module into 3 pipeline stages and each stage runs 16. To reduce the area cost we try to reuse the hardware as more as possible. The Rendering Module adopts a tiled-based approach in order to reduce memory requirement. The tile-based RM consists of two sub modules. The rasterizers perform the scan conversion to fill the triangles with colors and then passes the image to the Pre-fragment Operation Units to draw the image into the frame buffer that will display to the monitor. There can be four rasterizers that perform efficient scan conversion operations to gain more performance. If the area cost is critical, the rasterizers can be reduced and there will be only a little performance drop. Actually in our final chip only uses two rasterizers Figure 4 Rendering engine block diagram An AMBA Multi-Resolution Trace Analyzer is built in this chip that provides different observe resolutions to trace the bus activities. As shown in figure 5, there are two abstraction levels, which are the timing and signal separately. For trace timing, the bus activities can be sampled in cycle level or transaction level. The cycle level means that the signals will be traced cycle by cycle. On contrast, the transaction level samples the signals only when the bus has a transaction. On the other hand, the complex bus signals are also abstract to three levels such as all signals level bus state level, and master operation level. On all signals level, the trace analyzer records all bus signals at cycle or transaction time. As the level arises to the bus level, it will record the bus behavior rather than every detail signals. Finally, on the master operation level, we will only record the signals while a master access data. After combine sample timing and signal abstraction, the trace analyzer supports five different trace modes. esigners can change trace mode dynamically during program execution at any time. For example, in mode 1 it records every bus signals cycle by cycle and it only samples partial bus signals like address, data and etc. at each transaction in mode 4. The bus tracer greatly reduces the trace size, ranging from 78% to 98% depending on the selected mode. Figure 5 Abstraction definition of bus tracer

4 B. Software evelopment This project plans to provide a 3 total solution for consumer electronics. The software can not be neglected. We need to provide the hardware device driver, OpenGL ES API and a real-time performance monitoring tool. The first thing for software implementation is porting the Linux kernel to the chosen platform, which is the ARM RealView Versatile [17] family of boards. The linux kernel has ported on the development board. And then the pure software implemented OpenGL ES API was developed as the base contrast for the performance analysis and hardware verification. There are 70 OpenGL ES functions to be implemented and after some main functions had finished we start to work with hardware team to implement the device driver simultaneously. While the device driver can successfully drive the 3 hardware, we start to develop the GPTT (Graphics Performance Tuning Tool). This tool can provide the performance information to the remote PC via network. The performance information includes frame rate, triangles/vertices per second, pixels per second, memory usage, CPU time for each functions, and etc. These will help the programmer to find the performance bottleneck of their 3 applications and then adjust it to fit the required performance. The device driver also provides the interface for GPTT that can communicate with hardware to provide hardware performance information to the GPTT and then it becomes a real-time hardware performance monitoring tool that could help to find the hardware performance bottleneck and optimize it to reach higher performance. Figure 6 shows the demonstration of the GPTT. A 3 application is running at the target platform (ARM versatile) using the provided OpenGL ES API and the GPTT can receive the performance from target versatile and display the result on the remote PC. Figure 6 Screen shot of GPTT demonstration The target platform performs 3 applications. The GPTT can display FPS, memory usage and etc. performance information to help programmer to refine their 3 software. C. Hardware/Software Integration To integrate this complex hardware and software system is not an easy thing. The first thing to integration is to define the hardware control flow and the interface between hardware and software before implementation. Then we build the SoC platform that includes CPU, AMBA and SRAM to integrate the 3 hardware components to work with OpenGL ES API. The hardware control flow has to operate in coordination with OpenGL ES API. Thus the hardware operation authority is controlled by API through device driver. The 3 hardware is mapped to the memory space and the API can freely set the hardware control registers to command the hardware operation. After defining the control and interface, we start to integrate the 3 chip into to the Easy platform [18] which is proposed by ARM that support standard AMBA AHB bus. Because the 3 chip already has the on-chip bus, we need an AHB-to-AHB bridge to connect to the Easy platform. Figure shows the 3 chip integration architecture. For chip verification and tape out, the 3 chip includes core and the I/O pads that make the simulation close the real case. The Easy platform provides standard AHB interface and protocol to our chip. With this platform we can actually run a real 3 program and verify our hardware and software. ARM Easy Platform Memory (3 Test Program) Arbiter MI Bridge Slave Figure 7 3 chip integration platform 3 Chip SI Bridge Master ecoder IV. VERIFICATION AN CHIP FABRICATION To verify this complex system is another struggle job. It is easily to verify the functions small components while the function is simpler, but it becomes very tough after integration because of the complex operations and functions. Even though the sub modules has been verified perfectly no one could ensure that it will work while putting them together.

5 A good test plan is very important. It will not only provide a good verification quality but also reduce the verification time for time to market. We propose a program level and cross verification method to increase the verification quality. There are three platforms that have been used during our design: the System-C model, the Easy platform and the versatile development board. We can reuse these platforms and cooperate to verify the chip. Firstly, we synchronize the 3 chip model, RTL design, gate-level and FPGA design using exactly the same interface with real chip so that we can change designs between these platforms. Secondly, we synchronize these three platforms that can use the same test programs. It will save a lot of time that we can reuse the test programs rather than different programs for different platforms. Another benefit for these synchronized platforms is that they can generate patterns to each other. For example, the System-C can easily dump the signals for the chip as cycle level test patterns. The Easy platform can dump the PA signals as the pattern for chip testing. And also the Versatile with FPGA can fast dump the frame buffer result as the golden pattern for System-C and Easy platform. By cross verification we can not only increase the verification quality but also reduce the verification time. 3 application example: 9 rotating objects based on OpenGL ES API 3 Graphics Engine ARM926EJ-S runs Linux Kernel, Application, OpenGL-ES API and Windowing System Figure 8 3 chip demonstration on ARM Versatile FPGA platform Figure 8 shows the demonstration result on ARM Versatile. This demonstration draws 9 colored cubes on the screen and then performs the rotate action and lighting. And these actions all worked perfectly. Finally a whole system demonstration is presented. The last thing for verification is the whole system demonstration which includes OS kernel, real 3 applications, OpenGL ES API, device driver and 3 engine. As shown in figure 8, this is a series of complex sceneries with a lot of complex objects. The screen resolution is , and over 330 thousand vertices in total are drawn. The draw effect includes fog, lighting and texture. Figure 8 Whole system demonstration There is a particular design of the I/O ports. This chip supports AMBA AHB I/O port to connect to ARM Versatile and makes us to test the system by easily replace the FPGA board. Thus we can reuse the software and do not need to build another test board that saves a lot of testing efforts. This 3 graphics SoC also supports single chip mode. The implemented 3 engine integrates about 3M transistors and occupies mm 2 die area using the TSMC 0.18 μm 1P6M CMOS process and consumes 400 mw. The core voltage is 1.8V and I/O cells voltage is 3.3V. The 3 graphics IP runs at 139MHz. V. CONCLUSIONS In this paper, we introduce a 3 graphics SoC for consumer electronics. We propose the total solution for 3 graphics from porting OS, device driver, OpenGL ES APT to 3 chip. A complete SoC design flow is presented. This flow includes specification definition, system modeling, performance analysis, hardware and software implementation and integration, chip verification and tape out. A cross verification method using different platforms is also presented to increase verification quality and reduce verification time by reuse test programs and test pattern generation. The 3 graphics engine provides three features to achieve more convenience for usage and integration. Firstly, a complete system bus, AMBA AHB, is included in the chip and also provides additional AHB interface to gain the best integration ability. Secondly, the 3 graphics chip is designed fully support OpenGL-ES. With this ability makes it easy to develop advanced 3 graphics application or to transplant games from other platform. Thirdly, a real-time bus

6 tracer and a performance monitor are also embedded in this chip. The performance monitor can work with the GPTT that provides real-time 3 hardware performance, and the bus tracer can record complete bus activity information for the advanced system debugging and monitoring. The whole system has been verified on the FPGA development board. The maximum performance of the 3 chip is 8.34M vertices/s and 278M pixels/s at 139 MHz. The chip area is 987K gates. This chip is now fabrication and will be tested in July when it is back. REFERENCES [1] Imagination Technologies, Ltd. [2] ARM MBX HR-S 3 Graphics Core - Technical Overview, ARM Ltd. and Imagination Technologies Ltd., [3] Ashley Stevens, ARM 3 Graphics Solutions, ARM Ltd. and Imagination Technologies Ltd. [4] Bitboys Ltd., [5] ATI Technologies Inc., [6] Falanx MALI series specification [Online]. Available: [7] avid Blythe and Aaftab Munshi, OpenGL ES Common/Common-Lite Profile Specification, Khronos Group, Inc., [Online] Available: [8] TAKUMI Corporation, [Online]. Available: [9] Yong-Ha Park et al., A 7.1-GB/s low-power rendering engine in 2- array-embedded memory logic CMOS for portable multimedia system, IEEE J. Solid-State Circuits, Vol 36, pp , June [10] R. Woo et al., A 210mW Graphics LSI Implementing Full 3 Pipeline with 264Mtexels/s Texturing for Mobile Multimedia Applications, ISSCC ig. Tech. Papers, pp , Feb [11] M. Imai et al., A 109.5mW 1.2V 600M texels/s 3 Graphics Engine, ISSCC ig. Tech. Papers, pp , Feb [12] R. Woo et al., A low-power 3 rendering engine with two texture units and 29-Mb embedded RAM for 3G multimedia terminals, IEEE J. Solid-State Circuits, Volume: 39, Issue: 7, pp: , July 2004 [13] J. H. Sohn et al., A 50 Mvertices/s graphics processor with fixed-point programmable vertex shader for mobile applications, IEEE ISSCC ig. Tech. Papers, pp , Feb [14]. Kim et al., An SoC with 1.3 Gtexels/sec 3- graphics full pipeline for consumer applications, IEEE ISSCC ig. Tech. Papers, pp , Feb [15] onghyun Kim et al., An SoC with 1.3 gtexels/s 3- graphics full pipeline for consumer applications, IEEE J. Solid-State Circuits, Vol. 41, pp , Jan [16] CoWare Inc., [17] Platform Baseboard for ARM926EJ-S User Guide, ARM Ltd. [18] AMBA esign Kit,