Cluster Adaptated Signalling for Intra Prediction in HEVC

Transcription

1 2017 Data Compression Conference Cluster daptated Signalling for Intra Prediction in HEVC Kevin Reuzé, Pierrick Philippe, Wassim Hamidouche, Olivier Déforges Orange Labs IER/INS 4 rue du Clos Courtel UMR CNRS Cesson-Sevigné, rance Rennes, rance kevin.reuze@orange.com wassim.hamidouche@insa-rennes.fr pierrick.philippe@orange.com olivier.deforges@insa-rennes.fr bstract he High Efficiency Video Coding (HEVC) standard defines 35 Intra Prediction Modes (IPM) to provide an efficient compression of intra coded blocks. hose IPMs are signalled to the decoder through the use of three compression tools: prediction, clustering and coding. In this paper we provide improvements to these three tools through: new labels for the prediction, new tests for the clustering and new coding schemes for the coding. he most significant improvement consists in the provision of a cluster-dependent code: adapting the coding scheme to the available information enables the average symbol cost to get within close margin of the entropy of the data. he system providing the best compression efficiency based on these improvements is then computed, enabling significant reduction in the average cost required to code the IPMs. he proposed method builds a new coding system with the same complexity as HEVC with 0.41% bit-rates savings in ll Intra coding configuration. 1 Introduction High Efficiency Video Coding (HEVC) [8] is the latest video coding standard. inalised in January 2013, it enables a bit-rate savings of 50% [6, 9] with regards to the previous standard, H.264/dvanced Video Coding (VC). In HEVC a quadtree structure is used to split the video pictures into blocks of pixels of different sizes. or each block syntax elements, such as the Intra Prediction Modes (IPM)s, are used to convey the pixel information to the decoder. he contribution of this paper is to provide a method which decreases the number of bits required to code those syntax elements. his method is used to reduce the cost of IPM signalling and could be adapted for other syntax elements. IPMs are used for spatial prediction and their signalling represents more than 9% of the bitstream on average in ll Intra (I) [11]. s illustrated in igure 1, spatial prediction uses reference pixels left and above the current block to predict the pixels of the current block. In HEVC, 35 different spatial prediction modes are available including 33 angular and 2 non-angular modes. he signalling of the IPMs relies on predictive coding based on contextual information which represents information already encoded that is available for the decoder. or IPM prediction the modes of two intra coded blocks are used: the one at the left and the one above the current block. Let the IPMs used for the left and above blocks be and B, respectively. HEVC s IPM derivation process predicts the correct mode on average for 73% of the cases [10] /17 $ IEEE DOI /DCC

2 (a) (b) igure 1: Directions of Intra Prediction Modes (IPM) in HEVC (a) and an example of intra prediction with the direction 29 (b). he rest of the paper is organized as follows: Section 2 presents the 3 steps used for syntax element signalling: prediction, clustering and coding. Section 3 describes the proposed improvements to these three coding steps. he performance of the new coding system is assessed in Section 4. inally, Section 5 concludes this paper. 2 Compression ools used for IPM signalling he compression of the IPMs in HEVC is performed through three main steps: prediction, clustering and coding. hese steps are introduced in this section. 2.1 Prediction Prediction is an essential part in compression. Prediction reduces the average cost of a symbol by using the contextual information available to guess which symbols are more likely to be used in a given situation. In a non-continuous system, as for the IPMs, a label is used to predict the symbols to compress. his label can be based on the contextual information or directly on the symbols to predict. or IPM signalling in HEVC the available information consists in the IPMs used for the left () andand above (B) blocks. he labels used are and B, the angular neighbours of mode ( +1and 1), the two non-directional modes (0 and 1), which are the two most used modes on average [10], and the vertical mode (26). 2.2 Clustering Clustering is the step that distinguishes the probability distributions of the data. Better performance will usually be achieved if a prediction is flexible or robust in the sense of being able to change according to the local statistical behaviour of the input being compressed [4]. In other words a smart prediction should adapt to the source at hand. his enables better conditioning and reduces the global entropy of the system [12]. his is why, to improve the efficiency of the prediction, contextual knowledge is used. he tests cluster the data, each cluster regrouping cases that can 192

3 be predicted in a similar fashion (with the same labels). he more tests are available the more different clusters there can be, which allows better predictions, leading to lower costs. or IPM signalling in HEVC, 4 tests are used to separate the data in 5 different clusters, with a maximum of 3 combined tests; where 3 labels are predicted on each cluster. he resulting system is summarized by a decision tree in igure 2. he used tests focus mainly in finding if the two modes and B are equal and if they are angular (eg. >1). == B &&B >1 + B<2 B B 1 B 26 igure 2: he system used for the IPM prediction in HEVC, based on the left and above IPMs (resp. and B). 2.3 Coding coding scheme enables to signal the decoder which symbol has been selected for the current block. o decrease the coding cost, the coding scheme has to consider the probability distribution of each cluster. In HEVC, for the coding of the 35 IPMs, two groups of modes are used. he first group consists of 3 predicted symbols named the Most Probable Modes (MPM). hose MPM are the ones most likely to be selected by the encoder. hey are therefore coded on a smaller number of bits than the remaining IPMs. able 1 describes the coding scheme used in HEVC. he first of the MPM is coded on 2 bits, the next two on 3 bits and the 32 other IPMs are coded on 6 bits. or clarity this coding scheme will be referred as (6x32): the number of bits used for each of the MPM followed by the number of bits used for the remaining modes with the number of remaining modes. his notation will also be used for other coding schemes. In this coding scheme the first bit is coded using a Context daptative Binary rithmetic Coder (CBC) context [8] while the rest of bits are bypassed. his 193

4 Number of bits Code Coded Mode 2 10 MPM MPM MPM remaining IPMs able 1: Used coding scheme for IPMs in HEVC (2-3-3-(6x32)). he first bit is coded using a CBC context. reduces the cost of the IPM signalling as the first bit does not have an equal probability of being 0 or 1. training set is created to compare different coding systems. Each sample of this training set consists of the chosen IPM and the ones of blocks left and above for blocks of the 5 first frames from 59 heterogeneous sequences encoded at various Quantization Parameter (QP)s. o make sure that the existing coding algorithm does not influence the new system each IPM is equally coded on 6 bits without any prediction. he resulting data (ie. training set) consists of 15 million samples. In HEVC the average cost for IPM signalling, using the clustering and prediction system in igure 2 and the coding scheme in able 1, is found to be 3.87 bits/ipm on this training set. 3 IPM Coding Improvements s described in the previous section, the HEVC compression tools are efficient and enable the signalling of the 35 different symbols on less than 4 bits. However, this coding scheme is not optimal and can be further improved. his section proposes improvements for the different tools to reduce the average coding cost of the IPMs. 3.1 Improved Prediction o reduce the average number of bits required to code each IPM, new labels are proposed for better predictions. inding labels relies on the understanding of the data [5]. herefore the dataset is studied to find which are the most used IPMs depending on the available information. he first labels chosen are derived from the existing ones: as the labels +1 and 1 are used in HEVC the labels ± 2and ± 3 are added. hose same labels are added for B. he labels min(, B) andmax(, B) are also added, as well as their angular neighbours (min(, B) ± 1, 2, 3). Moreover a mean to compute non-angular neighbours is added: mode 1 meant for the modes 0 and 1. he most used modes are also used as labels: the vertical mode 26, the horizontal mode 10 and the two non-directional modes planar and dc (0 and 1). he diagonal modes 2, 34 and 18 are also used. his results in 30 different labels including the 7 labels used in HEVC. 194

5 3.2 Improved Clustering In HEVC there are 35 different IPMs and two available blocks for prediction. his means that a complete clustering would result in 1225 clusters, one for each possible couple of values of and B. his system, using 1225 clusters, is simulated to compute the theoretical limit of the method for the given data. o find the minimal amount of bits/ipm achievable the conditional entropy of the data H(i, B) is computed as follows: H(i, B) = p(, B) p(i, B)log 2 (p(i, B)) (1) =0 B=0 where p(, B) is the probability of the left IPM being equal to and the above IPM being equal to B. p(i, B) is the probability of the chosen IPM being equal to i in the case where the left IPM is equal to and the above IPM is equal to B. On the dataset this gives a cost of 3.45 bits/ipm while the HEVC signalling scheme for IPMs performs on average 3.87 bits/ipm on the same dataset. his means that, as expected, there is still room for improvements to be made. However, creating a system with 1225 different clusters would not be practical in terms of storage for an implementation in the HEVC software as a look-up table with 1225 entries would be required. herefore the systems are designed with the same order of magnitude as HEVC, which uses 5 clusters. he proposed systems use up to 8 different clusters. he proposed systems use new tests to enable a more efficient clustering. s with labels, finding new tests cannot be done automatically, requiring a fine study of the data. he goal of the tests is to be able to create clusters that can be predicted through the same labels. Similarly to the label case, the chosen tests first consist of the existing tests in HEVC. hese tests are then derived to improve the clustering by finding more similarities. or example, the test B < 2 is added as an extension to the test == B. his test, when and B are angular, allows to detect if the two modes are really close to each other. Other added tests check on the proximity of the vertical, horizontal and diagonal directions (resp. 26, 10 and 18). ests using the labels min(, B) andmax(, B) are also proposed: the tests min(, B) > 1and max(, B) < 2 are added to determine if the modes are both angular or non-angular. 3.3 Improved Coding In order to improve the prediction efficiency it is proposed to predict more IPMs by increasing the size of the MPM list. his is performed by using a different code than the one used in HEVC. list of codes is created, including all codes that use 3 to 7 MPM. hese codes represent all possible codes with 3 to 7 reduced code-length and a fixed number of bits for the rest of the symbols. his gives a total of 4 different codes using 3 MPM, 8 using 5 MPM and 43 using 7 MPM for a total of 55 different codes. Moreover, when clustering the data each cluster has a different probability distribution. Using one code to fit the distribution of all clusters is therefore suboptimal. i=0 195

6 o improve the efficiency of the coding, it is proposed to use a different code on each cluster. his enables using a different number of MPM on each cluster. s stated in Section 2.3, the first bit signalling the IPM in HEVC is coded using a CBC context. herefore, when computing the average number of bits required for coding the data on a cluster, the first bit of the used code is is always counted using its entropy cost instead of counting it as 1 bit. his new set of codes puts a new theoretical limit on the best achievable cost. he value of 3.45 bits/ipm is the entropy of the data. o achieve such performance the system would need to use an entropy coded bit for every bit of each signalling scheme. Moreover, it would require the use of all the possible coding schemes for 35 symbols (of which there are 108,861,148 possibilities [7]), and not only the 55 described above. hese list of codes motivates the computation of another theoretical limit which considers only the 55 selected codes. Equation 1 is modified to fit this description. he part log 2 (p(i, B)) is replaced with the code-length that would be required to code the IPM i knowing the IPM used left and above are and B. his means having a system with 1225 clusters, each cluster using the most efficient code of the 55 selected ones. his gives a result of 3.52 bits/ipm. his value asserts that the created systems can reduce the average cost of IPM signalling in HEVC if the chosen tests and labels allow to reach this limit for a small number of clusters. 3.4 System built methodology he system creation is done through two parallel-friendly steps: 1. Compute the possible clusters and select the best predictions. 2. Combine the clusters from the first step to evaluate all possible systems. In step 1 each cluster can be computed independently from the others, and in step 2 each system is independent from the other systems. In the first step the cost of a cluster is computed by choosing the best predicted values for a given coding scheme. he cost of each cluster is then stored for each coding scheme. Clusters are computed up to a depth of 4 tests (no more than 4 tests are combined to create a cluster) which results in 14,589 possible clusters, on each of which all the 55 codes are tested. he second step combines the clusters in order to form a consistent system: each combination of and B has to be predicted by one and only one cluster. In this step the systems are compared to each other by computing the average number of bits/ipm which would have been required to code the training data. his method allows an automatic selection of the best system once tests and labels have been chosen. 196

7 4.1 Experimental Procedure 4 Experiments and Results he main goal of the method is to reduce the average number of bits/ipm required to code the training data. s explained in Section 2, HEVC performs on average 3.87 bits/ipm on the training data and the best system with the chosen list of codes would perform 3.52 bits/ipm. wo different setups are tested. first setup consists of the systems requiring only one coding scheme. hese systems assert that the added tests, labels and codes are efficient to produce a reduction in the average cost of each symbol. nother setup is tested allowing the use of a different code for each cluster. ll systems are evaluated by computing the average number of bits/ipm used on the dataset. One system is then chosen for having a good compromise in complexity and bits/ipm reduction and is studied in bit-rate using Bjøntegaard delta (BD)-rate calculation [2] compared to HEVC using HEVC test sequences. he reference software in all experiments is the HEVC reference software (HM) [1] in ll Intra coding configuration. 4.2 Results igure 3 shows the achieved number of bits/ipm by the different generated systems for a number of clusters going from 3 to 8. his figure shows that the proposed method gives a system with a lower average cost than HEVC s IPM signalling scheme whatever the number of clusters. verage symbol cost (Bits/IPM) HEVC One Code Multi Codes Method Limit Entropy of Dataset Number of Clusters igure 3: verage bits/ipm for the two evaluated system setups for different number of clusters. he black lines represent the two limits described in Section 3. he curve showing the one-code-only systems shows that the method can improve HEVC through the use of new tests and new codes even with a smaller number of clusters. When using one different codes for each cluster the results are even 197

8 better. he best system created uses on average 3.57 bits/ipm, which is only 0.05 bits/ipm away from the best system achievable, given the used codes and the available contextual information. s HEVC uses 5 clusters for IPM prediction the chosen system also uses 5 clusters. his system performs interesting gains since it reduces the cost of IPM coding from 3.87 to 3.61 bits/ipm. It is presented as a decision tree in igure 4. his system uses 4 different coding schemes on the 5 different clusters where 3 of those 4 coding schemes use 7 MPM. he only cluster using a 5-MPM coding scheme is the one where both IPMs are angular and are not close to each other (min(, B) > 1 is true and B < 2 is false). nother main difference with HEVC is the constant use in the MPM list of the two non-angular modes, either through the use of the labels min(, B) and min(, B) 1 on the first two clusters or explicitly as 0 and 1 on the other ones. his system is chosen to be tested in the Common est Conditions (CC) in ll Intra on the HEVC test sequences [3]. max < 2 min > 1 B < 2 min max min 1 max 1 max +1 max +2 max (7x28) min min B (7x28) B 0 == B 1 max (6x30) B min 1 max max (7x28) (7x28) igure 4: Chosen system shown as a decision tree. or each cluster the used code is shown below it. or clarity min(, B) andmax(, B) are simply written min and max. he detailed bitrate savings on the HEVC test sequences are shown in able 2. hese results show the average bit-rate reduction for each sequence and each class. he last line corresponds to the average savings for all sequences. he average gain is 0.41% and the system provides savings for all considered video sequences. 198

9 Resolution Sequence bit-rate savings Nebutaestival % PeopleOnStreet % Class SteamLocomotiverain % ( ) raffic % verage % Class B ( ) Class C ( ) Class D ( ) Class E ( ) Class (various resolutions) BasketballDrive % BQerrace % Cactus % Kimono % ParkScene % verage % BasketballDrill % BQMall % PartyScene % RaceHorses % verage % BasketballPass % BlowingBubbles % BQSquare % RaceHorses % verage % ourpeople % Johnny % KristenndSara % verage % BasketballDrillext % ChinaSpeed % SlideEditing % SlideShow % verage % ll Sequences Overall % able 2: Bit-Rate reduction per class of the selected system using 5 clusters. Gains are against HEVC using the Common est Conditions (ll Intra). 5 Conclusion and Perspectives In this paper a systematic method is proposed to enhance the performance of a predicting system. he method is tested on the prediction of the IPMs in HEVC. By choosing a new set of tests and labels to improve both the predictability and the clustering some gains can be achieved. Moreover, the gains increase when each cluster uses a different code, allowing the final system to use the most efficient code for each cluster. metric is presented to measure the theoretical limit of the method, given the available codes and a system with a small number of clusters can be designed 199

10 that closes to that limit. his allows to evaluate beforehand how the method would increase the performance of the existing system. he final system proposes a bit-rate reduction of 0.41% in ll Intra in the CC for the same complexity as in HEVC. Other systems with cluster-adapted codes can be easily designed with this method, which can be used for other video coding syntax elements or state-predictive systems. References [1] HEVC Reference Software. [Online]. vailable: svn HEVCSoftware/ [2] G. Bjontegaard, Calculation of average psnr differences between rd-curves, Doc. VCEG-M33 IU- Q6/16, ustin, X, US, 2-4 pril 2001, [3]. Bossen, Common test conditions and software reference configurations, Document JC-VC-L1100, 12th Meeting: Geneva, [4]. Gersho and R. M. Gray, Vector Quantization and signal compression, [5] M. Mahoney, Data compression explained, mattmahoney.net/dc/dce.html, [6] J.-R. Ohm, G. J. Sullivan, H. Schwarz,. K. an, and. Wiegand, Comparison of the Coding Efficiency of Video Coding Standards - Including High Efficiency Video Coding (HEVC), IEEE ransactions on Circuits and Systems for Video echnology, vol. 22, no. 12, pp , Dec [7] N. J.. Sloane, Handbook of Integer Sequences,. Press, Ed., [Online]. vailable: oeis.org/ [8] G. J. Sullivan, J.-R. Ohm, W.-J. Han, and. Wiegand, Overview of the High Efficiency Video Coding (HEVC) Standard, IEEE ransactions on Circuits and Systems for Video echnology, vol. 22, no. 12, pp , Dec [9]. K. an, R. Weerakkody, M. Mrak, N. Ramzan, V. Baroncini, J.-R. Ohm, and G. J. Sullivan, Video Quality Evaluation Methodology and Verification esting of HEVC Compression Performance, IEEE ransactions on Circuits and Systems for Video echnology, vol. 26, no. 1, pp , Jan [10] D. K. Vo Nguyen, Video compression based on smart decoder, hesis, University Nice Sophia ntipolis, Dec [11] L.-L. Wang and W.-C. Siu, Novel daptive lgorithm for Intra Prediction With Compromised Modes Skipping and Signaling Processes in HEVC, IEEE ransactions on Circuits and Systems for Video echnology, vol. 23, no. 10, pp , Oct [Online]. vailable: htm?arnumber= [12] M. J. Weinberger, G. Seroussi, and G. Sapiro, he loco-i lossless image compression algorithm: principles and standardization into jpeg-ls, IEEE ransactions on Image processing, vol. 9, no. 8, pp ,