A GPU-Based DVC to H.264/AVC Transcoder

Transcription

1 A GPU-Based DVC to H.264/AVC Transcoder Alberto Corrales-García 1, Rafael Rodríguez-Sánchez 1, José Luis Martínez 1, Gerardo Fernández-Escribano 1, José M. Claver 2, and José Luis Sánchez 1 1 Instituto de Investigación en Informática de Albacete (I3A) Universidad de Castilla-La Mancha Albacete, Spain {albertocorrales,rrsanchez,joseluismm,gerardo, jsanchez}@dsi.uclm.es 2 Departamento de Informática Universidad de Valencia Burjassot, Valencia, Spain jclaver@uv.es Abstract. Mobile to mobile video conferencing is one of the services that the newest mobile network operators can offer to users. With the apparition of the distributed video coding paradigm which moves the majority of complexity from the encoder to the decoder, this offering can be achieved by introducing a transcoder. This device has to convert from the distributed video coding paradigm to traditional video coding such as H.264/AVC which is formed by simpler decoders and more complex encoders, and allows to the users to execute only the low complex algorithms. In order to deal with this high complex video transcoder, this paper introduces a graphics processing unit based transcoder as base station. The use of graphic accelerators in this framework has not been proposed before in the literature and offer a new field to explore with promising results. The proposed transcoder offers a time reduction of the whole process over 79% with negligible rate distortion penalty. Keywords: Distributed Video Coding, H.264/AVC, Graphic Processing Units, Transcoding, Heterogeneous Computing. 1 Introduction Multimedia communications between mobile devices are becoming an important area of interest in telecommunications because of the advance in mobile networks (such as 4G). Nowadays, one of the most requested mobile services is the video conferencing, where both the transmitter and receiver devices may not have necessary computing power, resources or complexity constraints to perform complex video algorithms (both coding and decoding). On the one hand, traditional video codecs such as H.264 Advanced Video Coding (AVC) [1] typically have highly complex encoders and less complex decoders. On the other hand, Distributed Video Coding (DVC) [2] has received great interest from multimedia research community because it offers low complexity encoders and more complex decoders. In other words, DVC framework offers E.S. Corchado Rodriguez et al. (Eds.): HAIS 2010, Part II, LNAI 6077, pp , Springer-Verlag Berlin Heidelberg 2010

2 234 A. Corrales-García et al. a reversal of the asymmetry in terms of complexity compared to traditional codecs like H.264/AVC. This mobile to mobile scenario is depicted in Figure 1. Fig. 1. Video communication system using a DVC to H.264/AVC video transcoder In order to achieve this low complexity communication between both paradigms, a DVC to H.264/AVC video transcoder needs to be included into the network which converts the bitstream. Basically, both sending and receiving devices shift their complexity to the base station resulting in less complex user devices. On the contrary, the transcoder has to handle two complex processes: DVC decoding and H.264/AVC encoding. It is worth to mention that this transcoder device does not have any computational restriction and it is designed to be a high processing unit with many resources. Recently, in high-performance computing are used accelerator or multi-core processor devices such as Graphics Processing Units (GPUs), Cell Broadband Engines (Cell BEs), and Field-Programmable Gate Arrays (FPGAs). These small devices consist of tens or hundreds of homogeneous processing cores which are designed and organized with the goal of achieving higher performance are being used. Therefore, these new hardware opportunities open a new door in the field of multimedia processing and computing; in particular, in the framework of DVC to H.264/AVC transcoders, which joint two of the most time consuming processes (such as H.264/AVC encoding and DVC decoding algorithms). At this point, this paper proposes a DVC to H.264/AVC GPU-based video transcoder in which the H.264/AVC encoding algorithm (the second half of the proposed transcoder) is accelerated by means of parallel processing. The Motion Vectors (MVs) generated in the DVC side information process (this process is the DVC motion estimation) are reused as MVs predictors in the H.264/AVC encoding stage and then, the H.264/AVC motion estimation is executed in a parallel way over a GPU. In other words, the center point of the H.264/AVC search area is adjusted based on the DVC incoming MVs. The proposed transcoder is a straight forward step because of the parallel processing has not been used before in the literature in the framework of DVC based transcoders; all the previous DVC-based transcoders (H.263 [3] and H.264/AVC [4]) are based on sequential execution. This paper is organized as follows: Section 2 presents the basics of DVC, H.264/AVC and GPUs. Section 3 presents the proposed GPU-based video transcoder which is evaluated in Section 4. Finally, the conclusions are presented in Section 5.

3 A GPU-Based DVC to H.264/AVC Transcoder Technical Background 2.1 Distributed Video Coding DVC provides a new video coding paradigm, where the architecture is characterized by encoders less complex than decoders. On the encoder side, frames are labeled as Key Frames (K) and Wyner-Ziv Frames (WZ). K frames are encoded as Intra frames in traditional codecs. However, WZ frames only store a few parity bits and temporal correlation is not exploited. For this reason, encoding procedure is much faster than traditional encoders. On the other hand, DVC decoder receives K frames firstly. From each two adjacent K frames is done an estimation of the middle WZ frame, which is called Side Information (SI). Figure 2 shows the first step in the SI generation for a MacroBlock (MB) using two frames with positions k and k+n in the sequence, then SI represents an approximation of the frame with position k+n/2. Every MB in the K frame k+n is matching with another MB in the K frame k. This matching is done by checking all the possibilities into the defined search area and choosing the lowest residual one. The displacement is quantified by a MV, and the middle of this MV represents the displacement for the MB interpolated. Afterwards, a channel decoding algorithm tries to refine this SI by using the parity information, as is specified in [2] [5]. Fig. 2. First step of SI generation process 2.2 Overview of H.264/AVC H.264/AVC [1] is the most recent predictive video compression standard that outperforms other previous existing video codecs. The H.264/AVC standard builds on those previous coding standards to achieve a compression gain of about 50%, largely at the cost of the increase in computational complexity of the encoder. These compression gains are mainly related to the variable and smaller block size motion compensation, improved entropy coding, multiple reference frames, smaller blocks transform and deblocking filter among others. The inter prediction in H.264/AVC supports motion compensation block sizes ranging from 16x16 to 4x4 with many options available between them. Then, the Motion Estimation (ME) process is carried out for each partition and sub-partition. This process is known as tree structured motion compensation algorithm. Therefore, the ME process is carried out many times per MB and, for this reason; this process spends most of the time of the encoding algorithm. Moreover, MVs neighboring partitions are often highly correlated and so each motion vector is

4 236 A. Corrales-García et al. predicted from vectors of nearby, previously coded partitions. The predicted MVs are computed as the average of candidate MVs. These MBs include the left MB, above MB, and above-right MB against the current MB. 2.3 Graphics Processing Units In the past few years new heterogeneous architectures are being currently used in high-performance computing [6]. An example of this architecture is the GPUs. GPUs are small accelerator devices with hundreds of cores which are organized in several Single Instruction Multiple Data (SIMD) blocks, and designed with the goal of achieve high performance in graphics applications. GPUs are characterized by a high parallelism level and they are usually used as a coprocessor to assist the Central Processing Unit (CPU) in computing massive data. It must be taken in account that current GPUs can offer 10x higher main memory bandwidth and use data parallelism to achieve up to 10x more floating point throughput than the CPUs. Although GPUs can be used for general purpose they come primarily from multimedia and gaming applications. To facilitate the programming tasks of these devices, GPU manufacturers provide diverse tools, function libraries, languages or extensions for the most common used high level programming languages. For example, NVIDIA proposes a powerful GPU architecture called Compute Unified Device Architecture (CUDA) [7]. This architecture allows a great number of threads running simultaneously the same code (kernel) taking advantage of the high computation capacity and main memory bandwidth. 3 Proposed DVC to H.264/AVC GPU-Based Video Transcoder This work proposes a DVC to H.264/AVC transcoding architecture from each DVC GOP to H.264/AVC I11P GOP (baseline profile). This procedure is done in an efficient way by the reusing of the MVs calculated during the DVC decoding phase in order to determine the MBs predictors of the H.264/AVC ME task. In consequence, the time spent in the overall transcoding process is largely reduced. 3.1 Allocation of Motion Vectors In the DVC decoding stage, MVs are calculated during the SI generation process, as it was explained in section 2.1. These MVs offer an approximation about the displacement between frames. For different DVC GOP lengths, decoding process changes but MVs are selected in a similar way. For example, Figure 3 shows the mapping of MVs from a DVC GOP length 4 to a H.264/AVC I11P GOP. On the step 1, DVC decodes frame WZ 2 using K 0 and K 4 frames as references. As a result, MVs V 0-4 are available, but these MVs are not considered because references with high distance do not provide a good accuracy. On the second step, frame WZ 1 is decoded using frames WZ 0 and WZ 2 as references, and likewise frame WZ 3 is decoded using as references frames WZ 2 and WZ 4. MVs generated in the second step (V 0-2 and V 2-4 ) provide a better accuracy, so they will be used by H.264/AVC, which will use them like predictors. However, as they are calculated for a distance of 2 and P frames have references

5 A GPU-Based DVC to H.264/AVC Transcoder 237 Fig. 3. Allocation of MVs from DVC GOP 4 to H.264/AVC I11P GOP with distance 1, they are split into two halves. Notice that each DVC GOP has the last DVC step in common and MVs used like predictors for H.264 are selected in this step. Consequently, MVs extraction process is generic for each DVC GOP. 3.2 H.264/AVC GPU-Based Execution The improved H.264/AVC encoding algorithm as part of the whole transcoding process is presented in the following lines. The idea behind of this approach is motivated by the fact that the ME is carried out many times in H.264/AVC encoding algorithm. As it has been explained in Section 2.2, the H.264/AVC encoding algorithm supports many MB partitions and for each of them, it calls to the ME algorithm. As the number of partitions increase, the time consumption also increases. Therefore, GPUs philosophy fits well in this framework because of they are based on a SIMD computing device. In fact, the ME algorithm is carried out over multiples data, which are the MB positions to be checked. Therefore the H.264/AVC inter prediction (which includes the ME process) is executed over the GPU by using CUDA. For this purpose, the proposed algorithm is divided into three steps; all of them need to be executed sequentially but each one is exploited following a highly parallel procedure by using the GPU. The goal of the first kernel is to obtain the Sum Absolute Differences (SAD) calculation between the current MB (split into sixteen 4x4 partitions) and all MB positions in the reference frame inside the search range. Then, by using the previous 4x4 block SAD calculations, it is able to obtain the SAD costs for the different sub-partitions. Finally, the last kernel reduces the SAD cost to one SAD cost for each one of the 41 MB partitions of each MB. More detail about the algorithm can be found in [8] In a nutshell, the main challenge of this approach is to support efficiently the tree structured motion compensation algorithm developed at the H.264/AVC encoding algorithm. In order to achieve that, H.264/AVC encoding algorithm uses the SAD calculation to determine the best MB partition, which is calculated in parallel over the GPU. The main problem of performing the ME in a parallel way is that the MVs predictors of neighboring MBs are not accessible for the current MB. As it has been

6 238 A. Corrales-García et al. explained in Section 2.2, the defined search area for each MB is determined based on the MVs of its neighboring but, this information is not accessible because they are being calculated at the same time that the current MB. In the present approach, the MVs generated in the DVC decoding algorithm, which are calculated as Section 3.1 explains, are used to determine the predicted search area. Figure 4 depicts this approach. Fig. 4. Predicted search area based on the incoming DVC motion vectors 4 Performance Evaluation In order to evaluate the performance of the proposed transcoder, four QCIF sequences were encoded at 30 fps by a DVC codec based on VISNET-II [9]. QCIF format was selected because it is the most suitable format for mobile-to-mobile video communications due to the reduced size of mobile displays and the low network bandwidth requirements. In the DVC encoding stage, 300 frames were encoded for each sequence using a QP matrix fixed to 7 [5] and GOP lengths 2, 4 and 8. During the DVC decoding stage, MVs are passed to H.264/AVC encoder as predictors. On the second stage of the transcoder, the H.264/AVC encoder converts each DVC GOP input into a H.264/AVC I11P GOP using QPs = 28, 32, 36 and 40 as specified in Bjøntegaard and Sullivan s common test rule [10]. In the simulations the H.264/AVC JM reference software, version 15.1 [11] was used. The baseline profile with the configuration by default was applied. In addition, RD-Optimization was turned off to make a suitable real-time encoding for low complexity mobile devices. In order to evaluate the performance, percentage of ME Time Reduction (%TR) reports the average of times reduction of ME displayed by H.264/AVC for the four QP points under study. Table 1 shows RD results for the proposed transcoder. As it is observed, the transcoder complexity is highly reduced reaching about a 79% of TR on average without significant RD penalties. This small RD drop is a consequence of the parallel execution which cannot use the sequential standard process to calculate the predictors, so it uses an approximation provided by the DVC MVs. Moreover, similar results are observed for different GOPs due to the generic MVs extraction procedure employed by the proposal. The last column of Table 1 shows the frame per second (fps) rate achieved for the whole encoding process which performs real time encoding. The present approach is a straight forward step in the framework of GPU-based transcoders and, thus, although RD results are similar than presented in [8] (without using

7 A GPU-Based DVC to H.264/AVC Transcoder 239 MVs), this approach provides a more accurate displacement of the search area. This could be extended as future work trying to reduce the search area by using the incoming MVs to reach better time reduction without large increasing of the RD penalty. In addition, Figure 5 displays RD results from a graphical point of view. As it is shown, all QP points are much close and the proposal presents a similar behavior for different GOPs. Table 1. Performance of the proposed transcoder for 30 fps QCIF sequences RD performance of the WZ/H.264/AVC video transcoder 30fps Sequence GOP PSNR (db) Bitrate (%) TR (%) fps ,12 Foreman , , ,96 Hall , , ,11 Coastguard , , ,57 Soccer , ,65 mean , Sequences QCIF (176x144) 30 fps GOP = Hall Soccer CoastGuard Foreman PSNR Reference Proposed Bit rate [kbit/s] Fig. 5. PSNR/bitrate results for sequences with GOP = 2. Reference symbols: Foreman Hall CoastGuard Soccer. 5 Conclusions This paper presents a GPU-based video transcoder to efficiently support the mobile to mobile communications. The incoming DVC MVs are used as candidate to define the predicted search area and then, the ME algorithm is executed in parallel over the

8 240 A. Corrales-García et al. GPU. The presented transcoder shows that the parallel computing in general, and GPUs as particular, is another efficient way to accelerate video coding algorithms. The improved transcoder depicted in this paper achieves a time reduction 79% on average with negligible rate distortion penalty. Ongoing work walks to improve the DVC decoding part of the transcoder using also parallel processing. Acknowledgments. This work was supported by the Spanish MEC and MICINN, as well as European Comission FEDER funds, under Grants CSD , TIN C04 and TIN E. It was also partly supported by The Council of Science and Technology of Castilla-La Mancha under Grants PEII , PII2I and PCC The work presented was developed by using the VISNET2-WZ-IST software developed in the framework of the VISNET II project. References 1. ITU-T and ISO/IEC JTC 1: Advanced Video Coding for Generic Audiovisual Services. ITU-T Rec. H.264/AVC and ISO/IEC Version 8 (2007) 2. Girod, B., Aaron, A., Rane, S., Monedero, D.R.: Distributed Video Coding. In: Proc. of IEEE Special Issue on Advances in Video Coding and Delivery, vol. 93(1), pp (2005) 3. Peixoto, E., Queiroz, R.L., Mukherjee, D.: A Wyner-Ziv Video Transcoder. IEEE Trans. Circuits and Systems for Video Technology (to appear, 2010) 4. Martínez, J.L., Kalva, H., Fernández-Escribano, G., Fernando, W.A.C., Cuenca, P.: Wyner-Ziv to H.264 video transcoder. In: 16th IEEE International Conference on Image Processing (ICIP), Cairo, Egypt, pp (2009) 5. Ascenso, J., Brites, C., Pereira, F.: Improving frame interpolation with spatial motion smoothing for pixel domain distributed video coding. In: 5th EURASIP Conference on Speech and Image Processing, Multimedia Communications and Services, Smolenice, Slovak Republic (2005) 6. Feng, W.-c., Manocha, D.: High-performance computing using accelerators. Parallel Computing 33(10-11), (2007) 7. NVIDA, NVIDIA CUDA Compute Unified Device Architecture-Programming Guide, Version 2.2 (February 2009) 8. Rodriguez, R., Martínez, J.L., Fernández-Escribano, G., Claver, J.M., Sánchez, J.L.: Accelerating H.264 Inter Prediction in a GPU by using CUDA. In: Proceedings of IEEE International Conference on Consumer Electronics, Las Vegas, NV, USA (2010) 9. VISNET II project, (last visited March 2010) 10. Sullivan, G., Bjøntegaard, G.: Recommended Simulation Common Conditions for H.26L Coding Efficiency Experiments on Low-Resolution Progressive-Scan Source Material. ITU-T VCEG, Doc. VCEG-N81 (2001) 11. Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, Reference Software to Committee Draft. JVT-F100 JM15.1 (2009)