Hardware Architecture For Fast Intra Mode and Direction Prediction In Real-Time MPEG-2 to H.264/AVC Transcoder

Transcription

1 Hardware Architecture For Fast Intra Mode and Direction Prediction In Real-Time MPEG-2 to H.264/AVC Transcoder Tarek A Elarabi, Randa Ayoubi, Hanan Mahmoud University of Louisiana at Lafayette, CACS Lafayette, LA USA tae9963@cacs.louisiana.edu Magdy Bayoumi University of Louisiana at Lafayette, CACS Lafayette, LA USA mab@cacs.louisiana.edu Abstract In this article, we propose a hardware architecture for our fast Intra mode and direction prediction algorithm to accelerate the MPEG-2 to H.264/AVC transcoding devices. In order to eliminate the redundant operations in the transcoder, our implemented algorithm uses the DCT coefficients from the MPEG-2 decoder to predict the Intra mode and reconstruction direction for the H.264/AVC encoder. In addition, the Intra prediction process in the H.264/AVC part of the transcoder has been dramatically accelerated by using our full-search elimination technique. The empirical results show 92% reduction in the transcoding time while reducing the PSNR for less than 3.5%. The proposed architecture has achieved an operating frequency of 323MHz at a power consumption of 112 mw when implemented on Virtex-5 FPGA Development Board. Keywords-Video; Transcoder; H.264/AVC; Intra; MBEG-2; FPGA. I. INTRODUCTION The MPEG-2 compression standard has been widely deployed not only in video distribution infrastructures like DTV cable and satellite but also in several popular applications such as DVDs and DVRs. Also, for many years, endto-end systems have existed with millions of interoperable encoders and multiplexers deployed. On the other hand, H.264/AVC has been a break through in the area of video coding. It has been widely implemented in most of the mobile multimedia terminals because of its very high compression rate while the quality degradation cost is preserved to the minimum [1]. Also, for its high network adaption capability, H.264/AVC is promised to be the universal video codec for all network dependent multimedia applications, such as PMP, DVB, Internet video, etc. However, the need for MPEG-2 to H.264/AVC transcoder has been rapidly increased because of the fast spread of multimedia enable mobile devices and the large number of MPEG-2 video contents that already exist. To access such MPEG-2 contents from devices with different network capabilities, the need for an efficient transcoder has been arisen. Also, MPEG-2 bit stream requires almost double the transmission bit rate and larger storage capacity compared to what H.264/AVC consumes for the same picture quality [2]. The need for such transcoder presents many research challenges. The main problem in transcoding came from the differences between Figure 1. The cascaded MPEG-2 to H.264 transcoder [6]. H.264/AVC and MPEG-2 [3]. Moreover, the Conventional Cascaded Transcoder (CCT) is a direct well known yet naive MPEG-2 to H.264/AVC transcoder. It is achieved by cascading MPEG-2 decoder and H.264/AVC encoder. Although it represents an upper limit on computational complexity, it theoretically maximizes the video quality [2]. In section (2), the cascaded MPEG-2 to H.264/AVC transcoder architecture is discussed. A computationally efficient transcoder would take advantage of the information stored in MPEG-2 decoder to accelerate the recoding process in the H.264/AVC encoder [4]. Section (III) will summarize our fast Intra mode and direction prediction algorithm which first introduced in [5]. In Section (IV), we introduce the proposed Intra Prediction hardware architecture for real-time transcoding. Section (V) shows the empirical results which prove the efficiency of the implemented transcoding method. Finally the conclusion and our future work plan are drown in section (VI). II. THE CONVENTIONAL TRANSCODER The simplest transcoding method is the one that converts the MPEG-2 bit stream to the H.264/AVC bit stream by means of cascade transcoding. The idea behind such transcoder is based on the sequential process that first decodes the MPEG-2 bit stream completely in MPEG-2 decoder then re-encodes the reconstructed video in H.264/AVC encoder to H.264/AVC bit stream. As shown in Figure1, the MPEG-2 bit stream is decoded by the MPEG-2 decoder to produce the reconstructed video. Then the reconstructed video is used by the H.264/AVC encoder that generates the H.264/AVC bit stream. In terms of hardware and software implementation, the architecture of the cascaded transcoder is very implementation friendly structure. Also, it can re /12/$31.00 c 2012 IEEE

2 Figure 2. 16x16 Prediction Directions [8]. duce lab-to-market time and engineering cost [6]. It has been widely used in the market because it is conceptually straightforward and it can be easily implemented. On the other hand, the main problem of the conventional transcoder persists in the H.264/AVC encoder portion of the transcoder. H.264/AVC encoder, in contrast with the MPEG-2 encoder, permits not only Inter frame coding but also Intra frame coding to remove the spatial redundancy. The Intra Micro Block (MB) prediction mode decision is the most computational demanding stage in the Intra frame coding process [7]. It has two main prediction modes for Luminance (Luma) part and one for Chrominance (Chroma) part. The two Intra prediction modes for luminance are : Intra 16x16 and Intra 4x4 [1]. The Intra 16x16 mode performs direction prediction process on the whole 16x16 MB while the Intra 4x4 mode performs direction predictions process on all sixteen 4x4 sub-mb of each MB. Also, there are four prediction directions for the Intra 16x16 mode, as shown in Figure 2, and nine prediction directions for the Intra 4x4 type, as shown in Figure 3. On the other hand, there is only one Intra 8x8 prediction mode for Chroma MB. It has four prediction directions which are similar to Intra 16x16 Luma prediction directions [8]. In case of Figure 3. 4x4 Prediction Modes [8]. using The Conventional Cascaded Pixel domain Transcoding algorithm, all four Intra 16x16 predictions, all nine Intra 4x4 predictions for the Luma and the four predictions for the Chroma of each MB must be computed. In conclusion, the total calculation for each MB is 4 x (4 + 9 x 16) = 592 [7]. To conclude the previous analysis, the computational complexity of the H.264 encoder side is proved to be impractically huge. Beside requiring multiple Digital Signal Processing (DSP) cores, it also demands sophisticated parallel processing algorithms [6]. This implementation issue is very serious for real-time communication, for instance, real-time broadcasting to handheld devices using H.264 and video chat over mobile phones [6]. Moreover, it has been observed that the H.264/AVC video encoders in the mobile devices periodically use the Intra frame prediction to quickly recover from channel error propagation [7], at rate of one Intra frame per every five frames. Therefore, this research work is focusing on the Intra frames prediction of the H.264/AVC portion of the transcoder. There are a lot of efforts that have been introduced to reduce the computational complexity of such transcoders. However, only few have handled the Intra prediction process at the H.264/AVC encoder part. Those few efforts have recently produced several functional transcoding algorithms but inefficiently enough. The most interesting work was by Kim et al. They proposed a mode skipping rule for Intra prediction mode decision in the H.264/AVC encoder part of the transcoder. The idea based on an experimental analysis results shows that DCT energy trend in the decoded MB of the MPEG-2 bit stream has a strong correlation with the Intra prediction modes of H.264/AVC [6]. Yoo et al. proposed an Intra decision method for the MPEG-2 to H.264/AVC transcoder based on a spatial activity analysis for the DCT coefficients of the MPEG-2 decoder [7]. Inspired by both Kim s work and Yoo s observations, we introduced our fast efficient Intra prediction algorithm [5] which is summarized in section (III). III. FAST INTRA PREDICTION ALGORITHM As mentioned in [5], Kim s algorithm [6] performs an extensive calculations to decide both the mode and the direction for the Intra prediction. In the case of deciding the 4x4 mode, it performs a full search to decide the 4x4 direction in a non negligible percentage of the MBs. Motivating by such drawbacks, an efficient algorithm is proposed. It significantly reduces the computation complexity of the Intra 4x4 direction decision. It also completely avoids performing a full search to come up with a direction decision for the Intra 4x4 mode while preserving almost the same peck signal to noise ratio (PSNR). The flowchart in Figure 4 summarizes the proposed algorithm in details. The proposed algorithm processes the video frame in units of 16x16 MBs. It works on two phases, the first phase is to decide the Intra prediction mode. In other words, it decides if it could select the prediction direction for the whole 16x16 MB as one unit, or it has to divide the 16x16 MB into the sixteen 4x4 sub-mbs. Then, it selects a prediction direction for each sub- MB. This phase builds its decision based upon the optimized version of the smoothness factor of the 16x16 MB that was introduced in Kim s algorithm in [6]. If the optimized smoothness factor indicates a smooth enough MB then the mode decision is 16x16 mode, otherwise the mode decision is 4x4 mode. In such decision process, discrete cosine transform (DCT) coefficients of four non overlapping 8x8

3 Figure 4. Fast Full-Search eliminating Intra prediction algorithm. MBs from the MPEG-2 Decoder is used in the smoothness factor calculations as illustrated in (1) to (3). E n = DCT n (0, 0), n = 0, 1, 2and3 (1) { 3 } Ē = E n 4 (2) SF = n=0 3 En Ē (3) n=0 ;Where E n refers to the estimated energy of the n th 8x8 sub-block, DCT n (0, 0) is the DCT s DC value of the nth MB, Ē is the average estimated energy for the 4 8x8 MBs, and SF is the estimated smoothness factor for the 16x16 MB. The second phase of the algorithm concerns with the Intra direction decision. If the Intra mode decision is 16x16, the algorithm calculates the non-homogeneity factor for the 4 16x16 direction candidates vertical (V), Horizontal (H), Plane (P) and DC. Then, it uses these factors as an indicator for the Intra16x16 prediction direction. The decision is based on choosing the Intra direction that performs the least nonhomogeneity factor. In other words, the algorithm chooses the direction that achieves the highest homogeneity. Using the homogeneity measure as an indicator is an excellent logical indicator for smooth scenes. If a MB pixels are homogeneous in a specific direction, this will imply that the objective scene in the examined MB is extended through the neighboring MBs at the same directions, accordingly the values of the examined macro block pixels could be fairly predicted from the neighboring macro blocks at the direction that achieves the highest homogeneity (the lowest non- homogeneity). The calculation process of the nonhomogeneity factors is inherited from Kim s algorithm [6] with two enhancements. The first enhancement is using only the DCT DC coefficients for calculating the energy of the four 8x8 MBs and totally excluding the AC coefficients from the calculations as shown in (1). The empirical results show that the AC coefficients has very little effect that can be

4 neglected to reduce the computational complexity in the realtime applications. The second improvement, as shown in (7), is in the calculation of the non-homogeneity factor for the plane direction. The following set of equations show how the non-homogeneity factors are calculated for each of the Intra 16x16 direction candidate : H DC = SF 2 (4) H V = E 0 E 2 + E 1 E 3 (5) H H = E 0 E 1 + E 2 E 3 (6) H P = E 0 E 3 + E 1 E 2 (7) After calculating the non-homogeneity factors for each of the Intra16x16 candidate directions, the direction decision would be choosing the direction that achieves the minimum non-homogeneity factor. On the other hand, if the Intra mode decision is 4x4, the direction prediction would be built upon an edge orientation factor similar to the one being introduced by Yoo s algorithm in [7]. However, the calculating method used for the edge orientation factor is different than the one introduced by Yoo s algorithm. The angle is estimated from the DCT coefficients of the four 8x8 MBs as shown in (8), (9) and (10). Based on this angle, the 4x4 direction candidates are chosen from Table 3. The direction candidate that achieves the minimum loss is chosen as the 4x4 prediction direction for the examined 4x4 MB. The process of calculating the estimated angle is performed by (8) : 3 θ = arctan { Eng top (n)/eng left (n)}/4 (8) n=0 Eng top (n) = Eng left (n) = j=7,j=j+1 i=0,j=1 i=7,i=i+1 i=1,j=0 DCT n (i, j) (9) DCT n (i, j) (10) IV. THE PROPOSED HARDWARE ARCHITECTURE The proposed hardware architecture for the Intra prediction is shown in Figure 6. It consists of six functionally integrated blocks and two memory units. Initially, the DCT DC coefficients of the current frame is imported from the MPEG-2 decoder and stored in a register array. The stored DC coefficients of the current MB and the adjacent three MBs, the top MB, the top-left MB and the left MB; are then passed to the Smoothness Factor (SF) block to compute the value of the SF for the current MB. The first block to function is the SF block which consists of simple addition and shift units. The Mode Decision (MD) block contains a comparator to compare the current output of the SF block against a threshold. Also, it is a very critical block where determining the candidate operating mode of Intra prediction is done. Threshold (Th) is set ahead of time in order to fit the application requirements. For instance, some applications require high video quality and do not mind the decoding delay or the compression size, while others do not mind less video quality for the required high compression. Based on the output of the MD block, either the Direction Decision (DD) block or the Edge Orientation (EO) block will be activated. Such proposed split architecture achieved a dramatically reduction in the overall transcoding time compared to the conventional transcoder. In case of the Intra 16x16 mode has been selected, the DD block will make use of the current SF value which has been calculated by the SF block along with the DC coefficients of the current MB and the adjacent MBs to calculate the Non-Homogeneity Factor. The Non-Homogeneity (NH) block is a sub-block of the DD block. It also consists of simple addition and shift units. Moreover, the Non-Homogeneity factors which computed by the NH block are applied to the Minimum Cost (MC) block. Then, the MC block, which functions as a direction prediction generator, produces the candidate direction for the 16x16 MB which is only composed of a simple comparator unit. On the other hand, In case of the Intra 4x4 mode has been decided by the MD block, the Edge Orientation (EO) block will put into functional state. The EO block contains simple addition and shift units. The computed angle (θ ) is sent to the θ - Range block which is composed only from a simple look-up table as shown in Figure 6. Finally, Figure 5, the Chroma Direction Decision (CDD) block is composed of a simple comparator unit to decide the prediction direction for the 16x16 chroma MBs. It only depends on the 16x16 luma direction decision from DD block as a main input. V. SIMULATION AND IMPLEMENTATION RESULTS In our previous work [5], the proposed Intra prediction algorithm, Kim s and Yoo s techniques were all implemented using C++ and have been embedded in the H.264/AVC reference software (JM 17.2). Also, the comparison has been provided in terms of PSNR and Intra transcoding time between all the implemented algorithms along with the standard full search algorithm. The comparison showed that the proposed algorithm has achieved an enhancement to the transcoding time nearly with a factor of 75% compared to both Kim s and Yoo s algorithms and 92% compared to the full search algorithm. However, the cost was an acceptable drop in PSNR specially for real- time applications. The PSNR drop was almost 8% less than the standard full search algorithm. Moreover, it was very similar to the PSNR achieved by Kim s algorithm and 3% higher than Yoo s algorithm. Figure 7 and Figure 8 summarize the previously mentioned software result in [5]. On the other hand, the proposed architecture has been implemented and tested on Genesys Virtex-5 FPGA Development Board. The board is equipped with Virtex-5 LXT which is optimized for High-performance logic with advanced serial connectivity.

5 Figure 5. Chroma Direction Decision block Figure 6. Proposed Hardware Architecture

6 have a higher speed rate and lower power consumption than FPGA which encourages migrating to ASIC. However, the cost for FPGA designs compared to ASIC, if small production is required, is a better option to go for. For that reason, we used the Xilinx Virtex5 as a prototype to prove the functionality and efficiency of our proposed Intra prediction algorithm. We are currently working on the ASIC design and implementation for our proposed architecture. Figure 7. PSNR for different characteristic frames [5]. VI. CONCLUSION In this article, we introduce an efficient hardware architecture for our full search elimination algorithm. It directly uses the MPEG-2 s DCT coefficients to predict the Intra frame operation mode and reconstruction direction. Our transcoding algorithm achieves an average of 92 % reduction in the required transcoding time compared to the transcoding time in the conventional transcoder while resulting an acceptable PSNR degradation. The proposed architecture has been implemented on Virtex-5 FPGA Development Board to test its functionality and performance. The proposed implementation has operated correctly at frequency of 323MHz at a power consumption of 112 mw. Such results nominate the proposed architecture for real-time transcoders. REFERENCES [1] I. E. Richarson, The H.264 Advanced Video Compression Standard. West Sussex, UK: Wiley, [2] M. Shaaban and M. Bayoumi, A low complexity inter mode decision for mpeg-2 to h.264/avc video transcoding in mobile environments, in Multimedia, ISM, dec. 2009, pp Figure 8. Transcoding time for different characteristic frames [5]. The Virtex-5 LXT includes 7,200 slices, each containing four 6-input LUTs, eight flip-flops with 1.7M bits of fast block RAM and twelve digital clock managers. All these components make Virtex-5 LXT a perfect match for our work. Also, the board includes HDMI video port with resolution up to 1600x1200 and 24-bit color. Besides the VHDC connectors, an additional 4 HDMI terminal-d connectors were added with VmodMIB - VHDC Module Interface Board add-on board. Moreover, the board also includes 256 Mbyte DDR2 SODIMM with 64-bit wide data which makes it suitable for the experiment. The reprogrammable board provides post simulation accurate results until the design is ready for ASIC implementation. The reason behind using the Virtex-5 FPGA is for its mentioned built-in features such as high speed I/O, IP-core generator, automatic optimization routing for better performance, automatic clock trees and embedded CPU. The prototype hardware implementation for the proposed architecture was able to achieve an operating speed of 323MHz. However, the power consumption was 112 mw which is common for Virtex-5 FPGA due to the board leakage power. On the other hand, the ASIC will [3] Y. Shin, N. Son, N. D. Toan, and G. Lee, Low-complexity heterogeneous video transcoding by motion vector clustering, in Information Science and Applications (ICISA), International Conference on, april 2010, pp [4] H. Kalva and B. Petljanski, Exploiting the directional features in mpeg-2 for h.264 intra transcoding, Consumer Electronics, IEEE Transactions on, vol. 52, no. 2, pp , may [5] T. A. Elarabi, A. Ragab, H. Mahmoud, and M. Bayoumi, High speed intra mode and direction prediction for mpeg-2 to h.264/avc realtime transcoder, in Proc. IEEE International Workshop on Signal Processing Systems, SiPS 2011 IEEE, Beirut Lebanon, Oct. 2011, pp [6] X. Liu, K.-Y. Yoo, and S. W. Kim, Low complexity intra prediction algorithm for mpeg-2 to h.264/avc transcoder, Consumer Electronics, IEEE Transactions on, vol. 56, no. 2, pp , may [7] X. Liu and K.-Y. Yoo, A fast intra mb mode decision method for the mpeg-2 to h.264 transcoder, in Intelligent Pervasive Computing, IPC., oct. 2007, pp [8] vcodex dot com, H.264/mpeg-4 part 10 white paper, April 2003.