arxiv: v1 [cs.mm] 18 Jan 2016

Transcription

1 Lossless Intra Coding in with 3-ta Filters Saeed R. Alvar a, Fatih Kamisli a a Deartment of Electrical and Electronics Engineering, Middle East Technical University, Turkey arxiv: v1 [cs.mm] 18 Jan 2016 Abstract This aer resents a ixel-by-ixel satial rediction method for lossless intra coding within High Efficiency Video Coding (). A well-known revious ixel-by-ixel satial rediction method uses only two neighboring ixels for rediction, based on the angular rojection idea borrowed from block-based intra rediction in lossy coding. This aer exlores a method which uses three neighboring ixels for rediction according to a two-dimensional correlation model, and the used neighbor ixels and rediction weights change deending on intra mode. To find the best rediction weights for each intra mode, a two-stage offline otimization algorithm is used and a number of imlementation asects are discussed to simlify the roosed rediction method. The roosed method is imlemented in the reference software and exerimental results show that the exlored 3-ta filtering method can achieve an average 11.34% bitrate reduction over the default lossless intra coding in. The roosed method also decreases average decoding time by 12.7% while it increases average encoding time by 9.7%. Keywords: Image coding, Video coding, Lossless coding, Intra rediction 1. Introduction Image 1 and video comression is erformed either lossless or lossy. In lossless comression, the visual data is reserved erfectly. In lossy comression, some amount of degradation in the visual data is tolerated to achieve better comression. Lossless and lossy comression are used in different alications. For examle, in many multimedia alications such as TV broadcast or videoconferencing, lossy comression is used. In alications where erfect reservation of visual data is more imortant than bandwidth, such as film archiving or some medical alications, lossless comression is used. Recently develoed [1] or widely used H.264/AVC [2] video coding standards suort both lossy and lossless comression. Both lossy and lossless comression in these standards is achieved with a block-based aroach. In lossy comression, a block of ixels are first redicted using ixels from a reviously coded frame (inter rediction) or using ixels from reviously coded regions of the current frame (intra rediction). The rediction is in many cases not accurate and as the next ste, the block of rediction errors are comuted and transformed to remove any remaining satial redundancy. Finally, the transform coefficients are quantized and entroy coded together with other relevant side information such as rediction modes. In lossless comression, the transform and quantization stes are skied and the rediction error block is directly entroy coded. However, the same block-based rediction methods are used. In the case of intra rediction, the block-based rediction addresses: saeed.alvar@metu.edu.tr (Saeed R. Alvar), kamisli@eee.metu.edu.tr (Fatih Kamisli) 1 This research was suorted by Grant 113E516 of Tübitak. aroach becomes less efficient if the transform ste is skied. Therefore, a large number of research aers have roosed to modify the block-based intra rediction aroach used in lossless coding within H.264/AVC or. There are two major aroaches to imrove the block-based intra rediction in lossless intra coding. In the first grou of aroaches, first the block-based intra rediction is erformed and then the rediction error block is further rocessed with a second ixel-by-ixel rediction ste [3, 4, 5, 6, 7]. In the second grou of aroaches, the block-based rediction aroach is relaced directly with a ixel-by-ixel rediction aroach [8, 9, 10, 11]. These aroaches are discussed in more detail in the next section. This aer exlores an aroach for lossless intra coding within which falls into the second grou of aroaches. In this aroach, satial rediction is erformed in a ixel-byixel manner, however, the rediction equations are changed to imrove the rediction erformance and increase the comression efficiency. In articular, while many methods in the literature use two neighbor ixels for redicting the current ixel from a articular direction [8] based on the angular rojection idea borrowed from block-based intra rediction, this aer uses three ixels for rediction according to a two-dimensional correlation model. This can imrove coding gains while merely increasing comutational comlexity. The used neighbor ixels and weights change deending on intra mode. To find the best weights for each intra mode, a two-stage offline otimization algorithm is utilized and a number of imlementation asects are discussed to simlify the roosed rediction method. Exerimental results with the reference software show imroved coding gains with resect to the default block-based rediction aroach or other imroved aroaches in the literature. Prerint submitted to Signal Processing: Image Communication August 22, 2018

2 The remainder of the aer is organized as follows. Section 2 discusses related revious research, in articular, the default intra rediction method in and some major aroaches for imroving lossless intra coding. Section 3 resents the roosed satial rediction aroach for lossless intra coding in, and discusses a number of related issues. Next, Section 4 rovides exerimental results to comare the comression erformance and comutational comlexity of the roosed aroach with the default lossless intra coding in and other imroved aroaches. Finally Section 5 concludes the aer. a w 32-w b 2. Previous Research This section discusses revious research closely related to this aer. First, a brief overview the block-based lossless intra coding in is given. Next, major aroaches for imroving the default lossless intra coding in are discussed, by categorizing them into three grous Lossless intra coding in Lossless intra coding in is achieved by retaining rediction and entroy coding and skiing transform, quantization, and in-loo filters [1].To exloit satial redundancy efficiently, block-based intra rediction in suorts 35 modes for all Prediction Units (PU) with different sizes ranging from 4x4 to 64x64. Prediction modes 0 and 1 are called lanar and DC modes, resectively and the remaining 33 modes are angular modes. While the lanar and DC modes rovide methods for effectively redicting image blocks with smooth or gradually changing content, the angular modes can effectively redict image blocks with directional structure. In DC mode, each block ixel is redicted with the same value, which is obtained by averaging the neighbor ixels immediately to left and to the above of the block to be redicted. In lanar mode, the block is redicted according to a lane model, which can rovide a gradually changing rediction block (see [12] for details). In the angular modes, the location of a block ixel to be redicted is rojected to the reference samles (i.e. neighbor ixels of the block) along an angle, as shown in Figure 1. Each of the 33 angular rediction modes uses a different rojection angle, as shown in Figure 2. The two closest ixels to the rojected location in the reference samles are used to linearly interolate (at 1/32 ixel accuracy) a rediction value as follows : = ((32 w) a + w b + 16) >> 5. (1) Here >> indicates a bit shift oerator, a and b reresent the reference samles, and 32 w and w reresent 5-bit integer interolation weights, which are determined by the angle or rediction mode [12] (See Figure 1) Satial rediction methods based on residual differential ulse code modulation (RDPCM) Since the transform is skied in losless intra coding, the block-based satial rediction alone can not efficiently reduce 2 Figure 1: Block-based angular rediction in. Block ixel to be redicted is rojected on the block neighbor ixels (reference samles) along an angular direction. Prediction is obtained from linear interolation of two closest reference samles a and b. the satial correlation. Therefore, satial rediction methods based on residual differential ulse code modulation (RDPCM) first erform the default block-based intra rediction and then rocess the rediction error block further with a second ixelby-ixel rediction ste. There are many methods roosed in the literature based on this aroach. One of the earliest of such methods was roosed in [3] for lossless intra coding in H.264/AVC. Here, first the block-based satial rediction is erformed, and then a simle ixel-by-ixel differencing oeration is alied on the residual ixels in only horizontal and vertical intra rediction modes. In horizontal mode, from each residual ixel, its left neighbor is subtracted and the result is the RDPCM ixel of the block. Similar differencing is erformed along the vertical direction in the vertical intra mode. Note that the residuals of other angular modes are not rocessed in [3]. This method was later acceted into H.264/AVC extension rofile due to its simlicity, good interlay with block-based rediction and its good erformance [13]. Note also that this method is sometimes referred to as the RDPCM method in the lossless intra coding literature although it is one ossible RDPCM aroach. Another method based on RDPCM is roosed for in [4] and is termed cross RDPCM (CRDPCM). The first art of this method is the same as in the above RDPCM method. In other words, first block-based satial rediction is erformed, and then for only horizontal and vertical modes, the rediction residual block is further rocessed with vertical ixel-by-ixel differencing in the vertical mode and horizontal ixel-by-ixel differencing in the horizontal mode. In the second art of CRD- PCM method, another ixel-by-ixel differencing is alied on the obtained RDPCM ixels along the cross direction. In other words, in the horizontal intra mode, the RDPCM ixels are subtracted from their uer neighbors along the vertical direction, and in the vertical mode, the RDPCM ixels are subtracted from their left neighbors along the horizontal direction. Note however that this second art is not always alied, but only in the blocks where it rovides further comression, which is determined at the encoder with rate-distortion (RD) otimized decision and signaled to the decoder with a flag in the bitstream. Another roosed method similar to CRDPCM is secondary RDPCM (SRDPCM) [5]. This method alies the second and

3 V-32 V-26 V-21 V-17 V-13 V-9 V-5 V-2 V0 V+2 V+5 V+9 V+13 V+17 V+21 V+26 V+32 a w 32-w b Figure 3: Samle-based Angular Prediction () roosed for in [8]. Block ixel to be redicted is rojected on the immediately above row (or column, deending on mode) ixels (reference samles) along an angular direction. Prediction is obtained from linear interolation of two closest reference samles a and b. H-26 H-21 H-17 H-13 H-9 H-5 H-2 H0 H+2 H+5 H+9 H+13 H+17 H+21 H+26 H+32 Figure 2: angular intra rediction modes numbered from 2 to 34 and the associated dislacement arameters. H and V indicate the horizontal and vertical directionality, resectively, and the numeric art of the arameter refers to the ixels dislacement as 1/32 ixel fractions. (icture reroduced from [12]) otional differencing oeration on the RDPCM ixels not along the cross direction, but along the same direction. Another method similar to CRDPCM and SRDPCM is roosed in [6]. This method alies the second and otional differencing oeration on the RDPCM ixels not along the cross or same direction but along both directions by subtracting from the RDPCM ixel the average of its left and uer neighbors. Yet another similar method is roosed in [7]. Here, instead of rocessing the block-based rediction residual with a ixel-by-ixel differencing along horizontal or vertical directions, a general linear rediction is alied using three neighbor residual ixels. The method is alied to all intra modes of and the linear rediction weights are udated at the encoder/decoder during encoding/decoding from reviously encoded/decoded ixels. While the simle RDPCM method [3] rovides modest imrovements in coding gain with resect to only block-based rediction, more comlicated methods such as CRDPCM [4] can require additional RD-otimized decisions at the encoder and syntax change to signal decision Satial rediction methods based on ixel-by-ixel rediction When the transform ste is skied in lossless intra coding, the block-based satial rediction becomes less effective since some block ixels are redicted from distant reference samles and there is no transform ste that can comensate for this inefficient rediction. However, since the transform is skied in lossless coding, a ixel-by-ixel satial rediction aroach 3 can now be used instead of a block-based aroach for more efficient rediction. Many lossless intra coding methods based on ixel-by-ixel rediction aroach aeared in the literature [8, 9, 10, 11]. One such method alied in is rovided in [8] and is called Samle-based Angular Prediction (). In the method, the lanar and DC modes are not modified. In the angular modes, the same angular rojection directions and linear interolation equations of are used, and only the used reference samles are modified. Instead of the the block neighbor ixels, the immediate neighbor ixels are used as reference ixels, as shown in Figure 3, resulting in a ixel-by-ixel rediction version of the block-based satial rediction aroach. Another ixel-by-ixel rediction aroach is given in [9], where it is alied to only the DC rediction mode. In this method, called iecewise DC rediction (PWDC), each block ixel is redicted by the average of its left and uer neighbors in the DC mode. The authors of [9] also combine their aroach with the RDPCM and CRDPCM methods discussed in Section 2.2 and reort imroved coding gains. Another ixel-by-ixel rediction aroach is given in [10], where it is alied to only the lanar rediction mode. Each block ixel is redicted by a weighted average of its four neighbors (left, uer-left, uer, uer-right) and the weights are determined during encoding/decoding using a table look-u [10]. A more recent ixel-by-ixel rediction aroach is given in [11], and is called adative directional (AD-). This method is similar to the method discussed above, in the sense that the same rediction equations are used in a ixel-byixel manner. However, unlike the method which uses the same rediction direction inside a single rediction block, the AD- method may adatively change the rediction direction for each ixel inside a rediction block, by checking how well the neighbor ixels can be redicted from their resective neighbor ixels [11]. While the ixel-by-ixel rediction methods discussed here rovide imroved coding gains, [8] alies the method to only angular modes, [9] alies it to only DC mode, [10] alies it to only lanar mode with a comlex algorithm and [11] re-

4 quires comlex calculations to change rediction direction for each ixel. As will be discussed in Section 3, this aer rovides a unified treatment of ixel-by-ixel rediction alied to all modes with otimized rediction weights and can rovide better coding gains Methods based on modified entroy coding In lossy coding, transform coefficients of rediction residuals are entroy coded, while in lossless coding, the rediction residuals are entroy coded. Considering the difference of the statistics of quantized transform coefficients and rediction residuals, several modifications in entroy coding were roosed for lossless coding in [14, 15, 16, 17, 18, 19]. A brief overview is as follows. In [15], the scanning order is reversed and the binarization used in the entroy coder is modified. An intra mode deendent scanning order is roosed in [16]. Coding of the last osition flag is modified in [17, 18] and coding of the levels of the coefficients/residuals is modified in [19]. c a b Figure 4: Proosed 3-ta filtering for lossless intra coding in. Each block ixel is redicted from a two-dimensional neighborhood of ixels based on a two-dimensional correlation model. Prediction is obtained as a linear combination of neighbor ixels a, b and c. ρ 1 a ρ 3 c ρ 2 ρ 3 b c ρ 1 a ρ 2 ρ 1 b a ρ 3 c ρ 3 c ρ 1 ρ 2 a b 3. Lossless Intra Coding with 3-ta Filters This section resents the satial rediction method exlored in this aer for lossless intra coding in. The method is based on the ixel-by-ixel rediction aroach discussed in Section ta filtering aroach A simle way to obtain a ixel-by-ixel rediction method for lossless intra coding in is to slightly modify its block-based angular rediction method so that the reference samles used for rediction are not taken from the distant block neighbor ixels but from the immediate neighbors in the above row or left column, deending on mode. This is exactly what is done in the method roosed in [8], which was discussed in Section 2.3. A major reason why the angular rojection based satial rediction method is commonly used for block-based satial rediction is that it is a comutationally simle method and works well together with a transform and adative block-sizes in lossy intra coding. However, the angular rojection aroach assumes a strong directional and one-dimensional correlation among ixels, which is a limited model. In articular, in a ixel-by-ixel neighborhood used in lossless intra coding, a two-dimensional ixel correlation (although ossibly directional) can be a much better model and does not increase comutational comlexity of rediction significantly. As a result, this aer rooses a ixel-by-ixel rediction aroach where each block ixel is redicted from a two-dimensional neighborhood of ixels. An examle is shown in Figure 4 where three neighbor ixels (left, uer, uer-right) are used for redicting a block ixel. The rediction in Figure 4 can be comared to the rediction of [8] shown in Figure 3. Note that similar twodimensional correlation models were also used for lossy intra coding with imroved results over block-based angular rojection methods [20, 21]. 4 ρ 2 b (a) Modes 2-9 (b) Modes 0,1,10-18 (c) Modes (d) Modes Figure 5: Neighbor ixels used for rediction in the 3-ta filtering method according to intra modes of. Intra modes 2-9 use neighbor ixels shown in (a), lanar, DC modes (0,1) as well as intra modes use neighbor ixels shown in (b), intra modes use neighbor ixels shown in (c) and intra modes use neighbor ixels shown in (d). The ixels in the two-dimensional neighborhood to be used for rediction must be chosen carefully to reflect various ossible directional correlation in images. In this aer, the neighbor ixels shown in Figure 5 are used for rediction in different intra modes of. These neighbor ixels were chosen so that each intra mode in 3-ta filtering method can reflect similar directionality as the same mode in. The rediction equation in each mode is the same and is given in Equation (2) below, however, the location of the neighbor ixels a, b and c as well as the rediction weights ρ 1, ρ 2 and ρ 3 change according to intra mode as shown in Figure 5. = ρ 1 a + ρ 2 b + ρ 3 c. (2) Notice that unlike in the method [8], the roosed ixelby-ixel rediction aroach is used for all angular modes as well as the DC and lanar modes. Notice also that it is ossible to use four or even five neighbors for rediction but due to the large number of rediction modes in, the increasing comutations and diminishing coding gains, only three neighbors are used for rediction in all intra modes Prediction weights While Equation (2) rovides the rediction exression, a major question is how to find the weights ρ 1, ρ 2 and ρ 3 for each intra mode of lossless intra coding in. These weights

5 were determined from a training sequence 2 using a two-stage otimization aroach. In the first stage, an iterative minimumsquared-error (MSE) method is used, which finds the weights that minimize the mean squared rediction error over the training sequence. These arameters are then further refined in the second stage by minimizing the bitrate over the training sequence. These final weights will then be later used to code the test sequences of. In the first stage of the otimization aroach, the weights ρ 1, ρ 2 and ρ 3 are obtained offline from a training sequence by minimizing the squared-error, i.e. the sum of squares of rediction errors in the training sequence. A rediction error ixel r in a block is obtained by subtracting out, from the original block ixel o, the rediction given in Equation (2) : r =o =o (ρ 1 a + ρ 2 b + ρ 3 c). (3) The weights ρ 1, ρ 2 and ρ 3 are then found by minimizing the sum of the squares of rediction error r for all ixels from the training sequence that were coded with the same intra mode : argmin (o ρ 1 a ρ 2 b ρ 3 c) 2. (4) ρ 1, ρ 2, ρ 3 Note that the above sum is over the set which includes all ixels in the training sequence coded with the same intra mode. In other words, the above minimization roblem is solved for each intra mode searately. The solution can be easily obtained and is given by Equation (5): ρˆ 1 ρˆ 2 = b a ρˆ 3 c a a 2 a b b 2 c b a c b c c 2 1 o a o b o c Finally, to obtain the weights for all intra modes collectively, an iterative aroach is used. In each iteration, the training sequence are encoded using the weights from the revious iteration, and Equation (5) is solved at the end for each intra mode to obtain the new weights for the next iteration. The initial iterations uses the method [8], and the iterations are stoed when the estimated weights don t change significantly and the coding gain stabilizes, which tyically takes about five to ten iterations. In the second stage of the otimization aroach, the rediction weights from the first stage are further refined by minimizing the bitrate over the training sequence. While MSE and bitrate are in general coherent, the ultimate erformance metric of a lossless codec is the bitrate, and therefore a second-stage otimization is erformed to further fine-tune the weights to achieve minimum bitrate instead of MSE over the training sequence. The exact otimization method used here will be discussed in Section A training sequence was formed from several images in the JPEG-XR image test set [22]. (5) Imlementation asects While Sections 3.1 and 3.2 discuss the roosed ixel-byixel rediction aroach and the determination of the rediction weights, a number of other asects related to the imlementation and simlification of the roosed rediction method need to be carefully considered. These asects are the bit-deth of the weights ρ 1, ρ 2 and ρ 3, whether the weights in each intra mode should be changed deending on the block-sizes available in, and whether the same weights can be used for symmetric modes such as horizontal and vertical modes. Note that the analysis resented here uses only the MSE method to determine the rediction weights. The second-stage otimization method will be used in Section 4.3 based uon the results rovided here Weights in different block-sizes While the default block-based satial rediction in and the ixel-by-ixel method in [8] use the same interolation weights at all block-sizes in each intra mode, it needs to be examined how the comression erformance changes in the roosed rediction method with different weights in each block-size. To examine this, mode-deendent weights are obtained for each PU block-size available in (4x4 to 32x32) searately and together. To obtain searate weights in each block-size, the set of training sequence ixels in Equations (3) and (4) is formed searately for each intra mode and each block-size, resulting in a total of 35 4 = 140 different mode and block-size deendent weights. To use same weights in each block-size, the set of training sequence ixels in Equations (3) and (4) is formed for each intra mode using all available block-sizes, resulting in a total of 35 different and only mode-deendent weights. The exerimental results rovided in Section 4 show that only mode deendent weights are sufficient and allowing the weights to change across the block-size does not bring significant coding gains Bit-deth of weights While the default block-based satial rediction in and the ixel-by-ixel method in [8] use 5-bits for the rediction weights (in interolation at 1/32 ixel fractions), it needs to be examined how varying the bit-deth of the weights affects the comression erformance in the roosed rediction method. To examine this, the weights obtained as discussed in Section 3.2 are quantized to 10-bits and 5-bits, and obtainable coding gains with both bit-deths are comared in the exerimental results in Section 4. These results show that using 5-bits for the weights is sufficient and the coding gain dro from using 10-bit weights is insignificant Weights in symmetric modes While the default block-based satial rediction in and the ixel-by-ixel method in [8] use the same interolation weights for symmetric modes such as modes 17 and 19 (see Figure 2), it needs to be examined how using different or same weights in the symmetric modes affects the comression erformance in the roosed rediction method. To examine

6 this, 5-bit and only mode-deendent weights are obtained searately and together for each symmetric mode. To obtain searate weights in each mode, the set of training sequence ixels in Equations (3) and (4) is formed searately for each intra mode, resulting in a total of 35 different mode deendent weights. To use same weights in symmetric modes, the set of training sequence ixels in Equations (3) and (4) is formed for each air of symmetric intra modes using ixels coded from both modes, resulting in a total of = 19 different weights. (Note that DC, lanar and mode 18 do not have symmetric airs and only the remaining 16 modes have symmetric airs.) The exerimental results rovided in Section 4 show that symmetric weights are sufficient and allowing different weights for symmetric modes does not bring significant coding gains. 4. Exerimental Results This section rovides exerimental results to comare the roosed ixel-by-ixel rediction aroach based on 3-ta filtering against other aroaches, in terms of achievable comression efficiency and incurred comutational comlexity. First, Section 4.1 resents the exerimental settings used in all rovided exerimental results. Next, Section 4.2 resents exerimental results to analyze the imlementation asects of the roosed 3-ta filtering method discussed in Section 3.3. Then, Section 4.3 discusses fine-tuning the 3-ta filter weights by otimizing for bitrate and resents imroved results. Next, Section 4.4 comares the comression efficiency and some statistical analysis results of the 3-ta filtering method with default and methods. Finally, Section 4.5 rovides a comlexity analysis of the comared methods Exerimental settings The roosed ixel-by-ixel rediction aroach based on 3- ta filtering is imlemented into the reference software HM12.0 [23]. For the rovided exerimental results, the common test conditions and the AI-Main reference configuration (in which all frames are coded as intra frames) in [24] are followed. All test sequences in class A to F are used in the exeriments. Notice that in all the exerimental results resented below, the used rediction weights were obtained offline as discussed in Section 3.2 from a training sequence that does not include any of the exeriment sequences Analysis of imlementation asects of the roosed 3-ta filtering method This section resents exerimental results to analyze the imlementation asects of the roosed 3-ta filtering method discussed in Section 3.3. In articular, the analyzed asects are the bit-deth of the rediction weights, whether the weights in each intra mode should be changed deending on the blocksizes available in, and whether the same weights can be used for symmetric modes. First, rediction weights are obtained in the most general case from the training sequences. In this case, the weights were 6 allowed to change in different PU block-sizes, have a large bitdeth (10-bits), and were not required to be same in symmetric modes. In this most general case, a total of 35 4 = 140 different mode and block-size deendent 10-bit weights were obtained from the training sequence and used in the exeriments. The comression results are shown in the first column of Table 1 as ercentage bit-rate reduction over the default block-based lossless intra coding in. It can be seen that significant bitrate savings can be achieved with an average of 10.84%. Next, to analyze how varying the weights in different blocksizes affects the comression erformance in the roosed rediction method, the weights at different block-sizes were forced to be same in each intra mode, resulting in a total of 35 only mode-deendent 10-bit weights. The comression results of this case are shown in the second column of Table 1. It can be seen that the comression results hardly change. In fact, the average bitrate reduction goes u by 0.02% to 10.86%. Next, to analyze the effect of the bit-deth of the rediction weights, the obtained 35 mode-deendent weights were quantized to 5-bits to be congruent with interolation weights. The comression results of this case are shown in the third column of Table 1. It can be seen that the average bitrate reduction dros slightly to 10.70%. Finally, to analyze the effect of using same weights in symmetric modes, the weights for symmetric modes were forced to be the same, resulting in a total of 19 mode-deendent 5-bit weights. The comression results of this final case are shown in the final column of Table 1. It can be seen that the average bitrate reduction dros slightly to 10.54%. To summarize, the resented exerimental results indicate that the discussed imlementation asects do not affect the comression erformance in a significantly unfavorable way, and the simlest and most imlementation friendly case, where only 19 mode-deendent 5-bit weights are used, achieves a cometitive 10.54% bitrate reduction over block-based lossless intra coding. Table 1: Average ercentage bitrate reduction of roosed 3-ta filtering method with different imlementation settings over lossless intra coding. General Block-size + Bit-deth + Symmetry case deendency reduced to enforced removed 5-bits Class A Class B Class C Class D Class E Average Fine-tuning of weights of 3-ta filtering method by otimizing for bitrate While rediction MSE and coding bitrate are in general coherent, in the sense that smaller squared rediction error generally leads to smaller coding bitrate, obtaining rediction

7 weights by minimizing squared rediction error is mismatched with the ultimate bitrate that lossless coders otimize. In other words, the rediction weights which minimize squared rediction error may not necessarily roduce minimum bitrate. Hence, a second-stage otimization is erformed to further finetune the weights from the MSE otimization stage to achieve minimum bitrate instead of MSE over the training sequence. Based on the results resented in the revious section (Section 4.2), only the simlest and most imlementation friendly case with only 19 mode-deendent 5-bit weights is further refined by otimizing for bitrate. A simle otimization algorithm is used to fine-tune the rediction weights. First, the coder is run with the rediction weights obtained from first stage (MSE) otimization and the resulting bitrate is B ot. Let ρ 1,k, ρ 2,k and ρ 3,k be the rediction weights of mode k and its symmetric mode if it exists. Then, the otimization aroach consists of alying the following stes : 1. Let k = 0 and B best = B ot. 2. Generate 6 candidates for rediction weights of mode k (ρ 1,k,i, ρ 2,k,i and ρ 3,k,i ), run coder by relacing mode k s weights with the candidates and record the resulting bitrates B i. Find candidate with smallest bitrate, Table 2: Candidate rediction weights (Candidate) i ρ 1,k,i ρ 2,k,i ρ 3,k,i Bitrate B i 1 ρ 1,k + 1 ρ 2,k 1 ρ 3,k B 1 2 ρ 1,k 1 ρ 2,k + 1 ρ 3,k B 2 3 ρ 1,k + 1 ρ 2,k ρ 3,k 1 B 3 4 ρ 1,k 1 ρ 2,k ρ 3,k + 1 B 4 5 ρ 1,k ρ 2,k + 1 ρ 3,k 1 B 5 6 ρ 1,k ρ 2,k 1 ρ 3,k + 1 B 6 i = argmin i B i. If B i < B ot, udate bitrate B ot = B i and rediction weights for mode k and its symmetric mode ρ 1,k = ρ 1,k,i, ρ 2,k = ρ 2,k,i and ρ 3,k = ρ 3,k,i. 3. If k < 18, (i.e. not the last mode without a symmetric air), increment k by one and go to ste 2. If k = 18, check if this iteration over all intra modes imroved bitrate, i.e. B ot < B best. If so, go to ste 1, otherwise finish. The rediction weights obtained from this algorithm increase the bitrate reduction over further as shown in Table 3. Table 4 lists the 19 mode-deendent 5-bit weights of the final obtained 3-ta filters Comarison to, and other methods The comression efficiency of the roosed 3-ta filtering method is comared against, the method [8], and several other methods from the literature. Table 5 shows the results. It can be seen from the table that the roosed 3-ta filtering method achieves the best average coding gain result; an average 11.34% bitrate reduction over. Notice that we imlemented only the method in addition to our roosed 3-ta filtering method, while the results of the other methods are taken from the resective aers. 7 Table 3: Average ercentage bitrate reduction of the roosed 3-ta filtering method over lossless intra coding after first (for MSE) and second stage (for bitrate) of otimizations. After otimization for MSE After otimization for bitrate Class A Class B Class C Class D Class E Average Table 4: 3-ta filter weights Modes ρ 1 ρ 2 ρ , , , , , , , , , , , , , , , , The frequency of selection for each intra mode with different rediction methods (averaged over all exeriment sequences) are rovided in Figure 6 for, and 3-ta filtering methods. It can be seen that in, lanar, DC, horizontal and vertical (0,1,10,26) modes are most frequently selected. In the method, lanar and DC are not the most frequently selected modes anymore due to their block-based rediction. In the 3-ta filtering method, lanar and DC modes are again amongst the most frequently selected modes together with horizontal and vertical modes. The frequency of selection for horizontal and vertical modes are twice as large as those of lanar or DC modes. The frequency of selection for each PU block-size with different rediction methods (averaged over all exeriments in each class of sequences) are rovided in Figure 7. It can be seen that in, 4x4 is by far the most frequently selected blocksize with around 80%, and 8x8 is the next most frequently selected block-size taking almost all of the remaining ercentage, and 16x16 and 32x32 block-sizes are almost not selected. This is due to the block-based rediction in, which is most

8 Table 5: Average ercentage bitrate reduction of multile methods from the literature and the roosed 3-ta filtering method over lossless intra coding. RDPCM CRDPCM PWDC AD- Proosed [4] [4] [9] [8] [11] 3-ta filtering Class A Class B Class C Class D Class E Class F Average accurate in the smallest 4x4 block-size. In the [8] and 3- ta filtering methods, due to the more accurate ixel-by-ixel rediction methods, 16x16 and 32x32 block-sizes are also selected with significant frequency. Notice that the frequency of selection for the 16x16 and 32x32 block-sizes are largest with the 3-ta filtering method, which can be and indication of its more effective satial rediction erformance Comlexity analysis This section resents a comlexity analysis based on the encoding and decoding time of reference software with different satial rediction methods. The results are normalized with resect to the encoding/decoding times and are given in Table 6. These results were obtained by averaging encoding and decoding times of all frames in all exeriment sequences. Notice that, even though more comutations are required to redict each block ixel in the and 3-ta filtering methods than in methods, the average decoding times with and 3-ta filtering methods decrease to 94.4% and 87.3% of the average decoding time. This is due to the bitrate reductions in the and 3-ta filtering methods which allow the comutationally comlex entroy decoding rocess to finish faster. In the encoder, the faster entroy encoder is not sufficient to comensate for the increased comutations of the redictions due to the reetitive rate-distortion otimized intra mode selection rocess, where a large number of intra modes must be comuted in each block. Hence, the encoding times with and 3-ta filtering methods increase to 101.8% and 109.7% of the average encoding time. In addition to encoding and decoding times obtained from reference software, the following observations can be also beneficial for some imlementation latforms, such as hardware imlementations. First, decoder imlementation is discussed. In the block-based method, all block ixels can be redicted in arallel because all deend on only the block-neighbor ixels. In the method, however, there is a deendency between rows (in vertical modes) or columns (in horizontal modes) of ixels because each row/column of ixels are redicted from the reconstructed ixels in the above row/column. Hence to redict each row/column, the reconstruction (i.e. adding rediction and residual to get original ixel values) of ixels in the revious row/column must be comleted first. A stronger deendency exist in the 3-ta filtering 8 method. Each ixel is redicted from the reconstructed left and above ixels. Hence to redict each block ixel, the reconstruction of immediately revious ixel and ixels in the revious row/column must be comleted first. Thus it seems that each ixel must be rocessed sequentially, however, an aroach similar to the wavefront arallel rocessing [1] of coding tree units in can be used here also to reduce the sequential rocessing deendencies. In encoder imlementations, all of the deendency issues can be overcome since all original ixels in a block are readily available. Table 6: Normalized average encoding and decoding times Encoding Time Decoding Time method 100.0% 100.0% method [8] 101.8% 94.4% 3-ta filtering 109.7% 87.3% 5. Conclusions This aer roosed a ixel-by-ixel satial rediction method for lossless intra coding in. The roosed method uses three neighboring ixels for rediction according to a two-dimensional correlation model. The used neighbor ixels and weights change deending on intra mode. To find the best weights for each intra mode, a two-stage offline otimization algorithm was used and a number of imlementation asects were analyzed to simlify the roosed rediction method. Exerimental results within reference software showed that when all intra modes are relaced with the roosed 3-ta filtering method, an average 11.34% bitrate reduction was achieved over the default lossless intra coding in. The roosed method also decreases average decoding time by 12.7% while it increases average encoding time by 9.7%. Acknowledgement This research was suorted by Grant 113E516 of Tübitak.

9 Frequency (%) method [8] 3-ta filter Intra mode Figure 6: Average frequency of selection for each intra mode in with different satial rediction methods. 100 Frequency (%) ta filter 3-ta filter 3-ta filter 3-ta filter 3-ta filter 3-ta filter Class A Class B Class C Class D Class E Average 4x4 8x8 16x16 32x32 Figure 7: Average frequency (%) of selection for each PU block-size in with different satial rediction methods. [1] G. Sullivan, J. Ohm, W.-J. Han, T. Wiegand, Overview of the High Efficiency Video coding () standard, Circuits and Systems for Video Technology, IEEE Transactions on 22 (12) (2012) doi: /tcsvt [2] T. Wiegand, G. Sullivan, G. Bjontegaard, A. Luthra, Overview of the H.264/AVC video coding standard, Circuits and Systems for Video Technology, IEEE Transactions on 13 (7) (2003) doi: / TCSVT [3] Y.-L. Lee, K.-H. Han, G. Sullivan, Imroved lossless intra coding for H.264/MPEG-4 AVC, Image Processing, IEEE Transactions on 15 (9) (2006) doi: /tip [4] S.-W. Hong, J. H. Kwak, Y.-L. Lee, Cross residual transform for lossless intra-coding for, Signal Processing: Image Communication 28 (10) (2013) [5] J.-H. Kwak, Y.-L. Lee, Secondary residual transform for lossless intra coding in, Journal of Broadcast Engineering 17 (5) (2012) [6] G. Jeon, K. Kim, J. Jeong, Imroved residual DPCM for lossless coding, in: Grahics, Patterns and Images (SIBGRAPI), th SIB- GRAPI Conference on, 2014, doi: /sibgrapi [7] X. Cai, J. S. Lim, Adative residual DPCM for lossless intra coding, in: IS&T/SPIE Electronic Imaging, International Society for Otics and Photonics, 2015, A 94100A. [8] M. Zhou, W. Gao, M. Jiang, H. Yu, lossless coding and imrovements, Circuits and Systems for Video Technology, IEEE Transactions on 22 (12) (2012) doi: /tcsvt [9] K. Kim, G. Jeon, J. Jeong, Piecewise DC rediction in, Signal Processing: Image Communication 29 (9) (2014) doi:htt: //dx.doi.org/ /j.image [10] E. Wige, G. Yammine, P. Amon, A. Hutter, A. Kau, Pixel-based averaging redictor for lossless coding, in: Image Processing (ICIP), th IEEE International Conference on, 2013, doi: /icip [11] X.-P. Xia, E.-H. Liu, J.-J. Qin, Imroved {} based on adative directional rediction for {} lossless intra rediction, Journal of Visual Communication and Image Reresentation 33 (2015) doi: htt://dx.doi.org/ /j.jvcir [12] J. Lainema, F. Bossen, W.-J. Han, J. Min, K. Ugur, Intra coding of the standard, Circuits and Systems for Video Technology, IEEE Transactions on 22 (12) (2012) doi: /tcsvt [13] G. J. Sullivan, P. N. Toiwala, A. Luthra, The h. 264/avc advanced video coding standard: Overview and introduction to the fidelity range extensions, in: Otical Science and Technology, the SPIE 49th Annual Meeting, International Society for Otics and Photonics, 2004, [14] S.-H. Kim, J. Heo, Y.-S. Ho, Efficient entroy coding scheme for H.264/AVC lossless video coding, Signal Processing: Image Communi- 9

10 cation 25 (9) (2010) [15] J.-A. Choi, Y.-S. Ho, Differential ixel value coding for lossless comression, INTECH Oen Access Publisher, [16] W. Gao, M. Jiang, Y. He, J. Song, H. Yu, A lossless coding solution in, JCTVC-G664, Geneva, Switzerland (2011) [17] S. Kim, A. Segall, Simlified CABAC for lossless comression, JCTVC- H0499, San José, CA, USA (2012) [18] Y. Piao, J. Min, S. Lee, AHG7: Coefficient coding for lossless coding, JCTVC-L0114, Geneva, Switzerland (2013) [19] J.-A. Choi, Y.-S. Ho, Efficient residual data coding in CABAC for lossless video comression, Signal, Image and Video Processing 9 (5) (2015) doi: /s z. [20] F. Kamisli, Intra rediction based on Markov rocess modeling of images, Image Processing, IEEE Transactions on 22 (10) (2013) doi: /tip [21] Y. Chen, J. Han, K. Rose, A recursive extraolation aroach to intra rediction in video coding, in: Acoustics, Seech and Signal Processing (ICASSP), 2013 IEEE International Conference on, 2013, doi: /icassp [22] JPEG core exeriment for the evaluation of JPEG XR image coding. URL htt://documents.efl.ch/grous/g/gr/gr-eb-unit/ www/iqa/original.zi [23] HM 12.0 reference software. URL htts://hevc.hhi.fraunhofer.de/trac/hevc/browser/ tags/hm-12.0 [24] F. Bossen, Common test conditions and software reference configurations, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC- F