Accelerated Motion Estimation of H.264 on Imagine Stream Processor

Transcription

1 Accelerated Motion Estimation of H.264 on Imagine Stream Processor Haiyan Li, Mei Wen, Chunyuan Zhang, Nan Wu, Li Li, Changqing Xun School of Computer Science, National University of Defense Technology Chang Sha, Hu Nan, P. R. of China, Abstract. Imagine is a stream-based prototype processor designed for media processing. It uses a three-level bandwidth hierarchy to exploit parallelism and data locality. It has good performance in media processing. H.264 is the newest digital video coding standard. It can achieve high coding efficiency at the cost of complex computation. In addition, video pictures have natural stream features, such as good special locality and limited temporal dependency. This paper presents an accelerated implementation of motion estimation, which is the most time-consuming part in H.264 coding framework, on Imagine stream processor. Experimental results show that the coding efficiency for QCIF format can be up to 372fps and surpass real-time requirement. The acceleration of stream processing is significant. It proves that H.264 coding is suited for implementation on Imagine. 1 Introduction Imagine is a prototype processor of stream architecture developed by Stanford University in We have done much research on Imagine stream architecture [1,2]. The stream model decomposes applications into a series of computation kernels that operate on data streams. A kernel is a small program executed in arithmetic clusters that is repeated for each successive element of its input streams to produce output stream for the next kernel in the application. Streams are ordered finite-length sequences of data records. Each record in a stream is a collection of related data words of the same type [3]. Imagine can provide high performance in many domains such as media processing and signal processing. For example, Imagine is able to sustain performance of giga operations per second (GOPS) in MPEG-2 encoding application, corresponding to 287 frames per second (fps) on a 320*288- pixel, 24-bit color image [4]. H.264 [5], proposed by Joint Video Team (JVT), is a new digital video coding standard. It aims to high compression, high quality, and flexible network adaptability. Especially, it surpasses MPEG-4 in low-rate video coding, and is suited for the requirement of network video application with low bandwidth but high quality. H.264 is widely-used in video telephony, videoconferencing, television broadcasting, video surveillance, stream media applications and so on.

2 H.264 has high coding efficiency at the cost of complex computation. In addition, video pictures have natural stream features, such as good spatial locality and limited temporal dependency. Considering the high performance of MPEG-2 on stream architecture, it is inferred that H.264 can increase its coding efficiency by stream processing. If so, not only one video coding standard can be implemented efficiently on Imagine. Even for different algorithms in the same standard, stream architecture may satisfy them in virtue of its programmability. Analyzing each module in H.264 encoder, it can be seen that motion estimation may consume 60% (1 reference frame) to 80% (5 reference frames) of the total encoding time of the H.264 codec and much higher proportion can be obtained if RD optimization or some other tool is invalid and larger search range (such as 48 or 64) is used [6]. Thus, the key of H.264 encoding optimization is how to improve motion estimation algorithms and how to implement existing algorithms efficiently. This paper introduces H.264 motion estimation algorithm and maps it onto Imagine stream processor. The experimental results show that the coding efficiency for QCIF image format can be up to 372 fps, which exceeds real-time requirement greatly. Compared with V1304 H.264 encoder [7], the speed up is times and the acceleration of stream processing is significant. It proves that H.264 is suited for implementation on Imagine. 2 UMHexagonS Motion Estimation Algorithm [6] UMHexagonS algorithm proposed by Tsinghua University can solve localminimum problem well, and therefore it is adopted by H.264 formally. This algorithm uses the hybrid and hierarchical motion search strategies. It includes four steps with different kinds of search pattern: 1) Predictor selection and prediction mode reordering; 2) Unsymmetrical-cross search; 3) Uneven multi-hexagon-grid search; 4) Extended hexagon-based search. With the second and third step, the motion estimation accuracy can be nearly as high as that of full search. But the computation load and operations can be reduced even more. Unsymmetrical-cross search uses prediction vector as the search center and extends in the horizontal and vertical directions respectively. Uneven multi-hexagon-grid search includes two sub-steps: First a full search is carried out around the search center. And then a 16-HP multi-hexagon-grid search strategy is taken. Extended hexagon-based search is used as a center biased search algorithm, including hexagon search and diamond search in a small range. 3 Imagine stream processor [8] Imagine is a programmable coprocessor that directly executes applications mapped to streams and kernels. Fig.1 diagrams the Imagine stream architecture. Kernels typically loop through all input-stream elements, performing a compound stream operation on each element. A compound stream operation reads an element from its

3 input stream(s) in the stream register file (SRF). During computation, all temporary data are stored in the local register file (LRF) of each cluster. And the computed results in a kernel are sent back to the output stream in the SRF. Only the initial and final data streams need to be transferred to the off-chip SDRAM. This storage bandwidth hierarchy is able to meet the large instruction and data bandwidth demands of media processing well. Fig. 1. Imagine stream architecture Programming on Imagine is divided into two levels: stream level (using StreamC) and kernel level (using KernelC). These levels are corresponding to stream scheduling and stream processing in logic view respectively. Programmers can be absorbed in the stream framework definition and kernel partition at the stream level. While at the kernel level, programmers should pay attention to the implementation and optimization of the whole program. Imagine provides three kinds of parallelisms: instruction-level parallelism (ILP), data-level parallelism (DLP) and task-level parallelism (TLP). The choice of these three parallelism modes during the implementation depends on the characteristics of a practical application and the power of hardware resources. 4 Implementation of H.264 motion estimation H.264 supports a range of block sizes (from 16*16 down to 4*4). Here we take 8*8 block size as an example to describe our motion search kernel (called blocksearch). Current MB Motion Vectors row0 Kernel1: Kernel2 row1 blocksearch row2 Data Stream Fig. 2. Diagram of blocksearch kernel

4 The input and output streams of blocksearch are shown in Fig.2. The motion search window as reference frame is 24*24 and is loaded by way of three input streams. And the sequence of current blocks to be encoded is organized into one input stream. Motion vectors for each block produced by blocksearch will be the input for the next kernel. All these four input streams are of ubyte4 type (a basic data type in KernelC language [9]), which is composed of 4 packed 8-bit unsigned bytes. Thus, an ubyte4 stream element can contain four luminance components of horizontally-adjacent pixels in the same block. Fig.3 illustrates the distribution of input stream in eight clusters. It can be seen that each cluster processes a row of pixels in an 8*8 block. 24pixels Organization of motion search window 24rows row0 row1 row2 block0 block1 block2 Cluster0 Cluster1 Cluster Cluster Distribution of cluster Cluster4 Cluster Cluster Cluster Fig. 3. Organization of motion search window and distribution of cluster Unsymmetrical-cross search operates the blocks in row1 first, that means the search process begins in the horizontal direction. The sum of absolute difference (SAD) is chosen as our matching criterion. The black and dark gray points in Fig.4 form an 8*8 block. After computing SAD between this block and current block in Current MB, the black points in the left column are freed. And load the next column of pixels (gray points) to generate a new 8*8 block for the following search. When the left four columns of pixels are freed totally, eight corresponding stream elements are consumed, which are not reused in the following search. The search process in the horizontal direction of motion search window needs 17 times SAD computation. The vertical direction performs similar search process. The motion vector with the minimum SAD will be chosen as search center of next search step Search in the horizontal direction Fig. 4. Load and free pixels in unsymmetrical-cross search Fig. 5. Search process of a simple multi-hexagon-grid search

5 A simple multi-hexagon-grid search is shown in Fig.5 (a full search in a small range is not described here.). The white points are processed in the previous unsymmetrical-cross search; the gray point presents the result position of unsymmetrical-cross search; and the black points are indicators of reference block used in multi-hexagon-grid search. It can be seen that the reference block stream is irregular, not in the horizontal direction or in the vertical direction. Thus, the data rearrangement is needed to organize reference blocks in the order as Fig.5 shows. Index stream or communication kernel is feasible for complex stream rearrangement. There is large potential data-level parallelism in the block search process. We exploit the parallelism in two approaches, shown in Fig.6. One approach described above, uses four pixels as a stream element that we called pixel stream. SAD computation for different rows of a block can be processed simultaneously in eight clusters of Imagine. Data transfer is implemented through inter-cluster communication. While the second approach uses coarse-grain parallelism called macorblock stream, processing eight different blocks in its own motion search window to produce eight motion vectors simultaneously. The inter-cluster communication will be decreased. However, the number of stored reference block becomes larger. In addition, the motion search windows for each coding block may overlap and bring large redundancy. As a result, the requirement of SRF and LRF bandwidth will increase. Table 1 gives the comparison between pixel stream approach and macroblock stream approach. Table 1. Comparison between two approaches - pixel stream and macroblock stream Pixel stream Macroblock stream Stream organization Natural Complex Record Size 1 word 16 words Loading overhead Little Large Bandwidth requirement Low High Communication Large Little 5 Result Analysis For QCIF image format accepted by H.264 encoder, its definition is PAL: 176*144. At 500MHz on the simulator of Imagine (ISIM), Imagine stream processor can provide 897 cycles to execute unsymmetrical-cross search, and 1581 cycles for uneven multi-hexagon-grid search in pixel-stream approach. Processing 8*8 block needs 2478 cycles, namely 4.956ns. Thus, the total time for processing a frame of QCIF image, which has 396 invocations of kernel, is 1.96ms. Some techniques like loop unrolling or software pipelining may optimize kernel s implementation and improve system performance. For example after unrolling loop twice, 569 cycles is needed for unsymmetrical-cross search and 1165 cycles for uneven multi-hexagongrid search. In this instance, 1.37ms is enough for encoding one frame and the total performance improves nearly 30%. While in macroblock-stream approach, it needs 5646 cycles for eight different macroblocks. The run time of kernel is 2.5 times faster than that of pixel-stream

6 approach. However, it is only an ideal result. In practice, it will be limited by large overhead of stream loading. So we implemented our H.264 encoder by using pixelstream approach. in in xa>>z; CrossS(z); FS(z); MHexagonS(z); EHexagonS(z); DS(z); a<<mv(z); out pixel stream in in xa>>z; CrossS(z); FS(z); MHexagonS(z); EHexagonS(z); DS(z); a<<mv(z); out macroblock stream Macroblock stream Pixel stream Motion vector stream Computation kernel Fig.6. Kernel diagram of pixel stream and macroblock stream Fig. 7. Schedule diagram of blocksearch Using our H.264 encoder including basic parameter-i slices, P slices and CAVLC, the coding efficiency for QCIF image on Imagine stream processor can be up to 372fps. Table 2 illustrates the comparison of an average encoding time for each QCIF frame among JM50 reference program (written by JVT), ShowVideo encoder (with an improved algorithm) [10], V1304 (developed by DSP Research, Inc.) [7] and our implementation on Imagine. Though our experiment excludes extended hexagon based search, the motion estimation accuracy can satisfy the requirement of application. It can be seen that the speed of H.264 motion estimation on Imagine surpasses that of JM reference program greatly, and achieves more obvious performance improvement than that of improved algorithm on general PC. Compared with 25fps of V1304 H.264 encoder, the speed up is times. It profits from that stream architecture can support a great deal of ALUs for computation-intensive application, and provide enough instructions and data to fill ALUs in order to keep ALU utilization high (see the circle in Fig.7, where horizontal axis displays hardware resources and vertical axis displays time. A rectangle within the axes indicates using a resource over a period of time. ). The actual coding efficiency may be lower than theoretic value that is because the data stream has extra loading overhead and the functional units keep idle until all required data stream elements are loaded in each cluster. How to organize input/output stream efficiently and exploit parallelism to the utmost extent, is the key of improving the performance of application.

7 Table 2. Comparison of different implementation for H.264 motion estimation System Environment Chosen Algorithm Motion Estimation Range Average Encoding Time Imagine Imagine UMHexagonS ms/frame Implementation JM50 AMD 1.2G + UMHexagonS ms/frame Reference Program DDR256M ShowVideo AMD 1.2G + new algorithm ms/frame DDR256M in [10] V1304 H.264 Encoder V ms/frame (Data for JM50 and ShowVideo refer to [10]) 6 Conclusion UMHexagonS motion estimation algorithm of H.264 encoder is mapped onto Imagine stream processor in this paper. We have achieved good performance that the coding efficiency for QCIF image format is up to 372fps. We can infer that H.264 coding standard is suited for Imagine based on the implementation of H.264 core algorithm. Thus, Imagine can implement many video-coding standards (MPEG-2, H.264 etc.). The flexibility is comparable with DSP, but the performance can be increased significantly. Imagine has such a great advantage in video coding domain. Our future work is optimizing the H.264 encoder on Imagine. It can provide experience and reference for mapping other video standard on Imagine, and make a contribution to application extension of Imagine. Acknowledgements. We thank High Performance Group of School of Computer Science in National University of Defense Technology for helpful discussions and comments on this work. We thank Imagine project group of Stanford University for providing the Imagine simulator. We thank Xiang Zhong for providing necessary application support. We also thank the reviewers for their insightful comments. This work was sponsored by National Natural Science Foundation of China under Grant Reference 1. Mei Wen, Nan Wu, Haiyan Li, and Chunyuan Zhang, Research and Evaluation of Imagine Stream Architecture, Advances on Computer Architecture, ACA 04, August Mei Wen, Chunyuan Zhang, Nan Wu, Haiyan Li, and Li Li, A Parallel Reed-solomon Decoder on the Imagine Stream Processor, Second International Symposium on Parallel and Distributed Processing and Applications, ISPA 04, December 2004

8 3. Scott Rixner, William J. Dally, Ujval J. Kapasi, Brucek Khailany, Abelarbo Lopez- Lagunas, Peter R. Mattson, and John D. Owens, A Bandwidth-Efficient Architecuture for Media Processing, Appears in Micro-31, John D.Owens, Scott Rixner, Ujval J.Kapasi, Peter Mattson, Brian Towles, Ben Serebrin, and William J. Dally, Media Processing Application on the Imagine Stream Processor, Appears in the Proceedings of the 2002 International Conference on Computer Design, Thomas Wiegand, Draft Text of Final Draft International Standard (FDIS) of Joint Video Specification (ITU-T Rec. H.264 ISO/IEC AVC), 7th Meeting: Pattaya. March Zhibo Chen, Peng Zhou, and Yun He, Fast Integer Pel and Fractional Pel Motion Estimation for JVT, 6th Meeting: Awaji, December DSP Research, Inc. V1304:H.264/MPEG4-AVC Encoder, 8. Brucek Khailany, William J. Dally, Ujval J. Kapasi, Peter Mattson, Jinyung NamKoong, John D. Owens, Brian Towles, Andrew Chang, and Scott Rixner, Imagine: Media Processing with Streams, IEEE Micro, March-April Abhishek Das, Peter Mattson, Ujval Kapasi, John Owens, Scott Rixner, and Nuwan Jayasena, Imagine Programming System User s Guide 2.0, June Cao Wenfeng, Zhang Ying, Zhang Zhaoyang, and Zhang Yijun, An Integer Pixel Motion Estimation Algorithm Applicable to H.264, Journal of Shanghai University (Natural Science), August 2004