A deblocing filter with two separate modes in bloc-based video coding Sung Deu Kim Jaeyoun Yi and Jong Beom Ra Dept. of Electrical Engineering Korea Advanced Institute of Science and Technology 7- Kusongdong Yusonggu Taejon Republic of Korea Email: sdim@issserver.aist.ac.r jyyi@issserver.aist.ac.r jbra@ee.aist.ac.r ABSTRACT This paper presents a method to remove blocing artifacts in low bit-rate bloc-based video coding. The proposed algorithm has two separate filtering modes which are selected by pixel behavior around the bloc boundary. In each mode proper one-dimensional filtering operations are performed across the bloc boundary along horizontal and vertical directions respectively. In the first mode corresponding flat regions a strong filter is applied inside the bloc as well as on the bloc boundary because the flat regions are more sensitive to the human visual system (HVS) and the artifacts propagated from the previous frame due to motion compensation are distributed inside the bloc. In the second mode corresponding to other regions a sophisticated smoothing filter which is based on the frequency information around bloc boundaries is used to reduce blocing artifacts adaptively without introducing undesired blur. Even though the proposed deblocing filter is quite simple it improves both subjective and objective image quality for various image features. Keywords: blocing artifacts blocing noise post-processing deblocing filter.. INTRODUCTION Traditional bloc-based video coders such as H.6 MPEG- and MPEG- suffer from annoying blocing artifacts when they are applied in low bit-rate coding because inter-bloc correlation is lost by bloc-based prediction transform and quantization. In order to overcome the blocing artifact problem various non bloc-based coding schemes have been proposed. Among them lapped orthogonal transform and embedded zerotree wavelet coding have been proposed as a non bloc-based texture coding method and warping prediction and overlapped bloc motion compensation 4 (OBMC) have been proposed as a non bloc-based prediction method. Though these activities have drawn a lot of interest recent international standards regarding low bit-rate video coding tend to adopt the bloc-based coding scheme as the baseline method based on the overall consideration of performance complexity compatibility maret requirements and so on. Therefore as a post-processing method deblocing filtering is regarded important due to improvement of visual quality in low bit-rate video coding. Even though the OBMC technique is adopted for deblocing in the recent low bit-rate video coder 5 H.6 it can not prevent the blocing artifacts caused by bloc-based texture coding and the artifacts propagated from the previous reconstructed frame. Therefore additional post-processing operations are also required for better image quality. Many deblocing schemes have been proposed in still image coding such as JPEG under the assumption that blocing artifacts are always located at bloc boundaries. In video coding however blocing artifacts of the previous frame can be propagated to the current frame and can be located at any position in a bloc due to motion compensated prediction. Therefore simple bloc boundary smoothing is not good enough to remove blocing artifacts appearing in video coding so a more complicated scheme is needed. Iterative methods based on projection on convex set 67 may be a candidate algorithm. However it is not adequate for real time video coding due to its complexity. In contrast smoothing sills using edge
information 8 may be easier to adopt to video coding. Unlie still image however a video sequence consists of a set of still images having various image features. Hence these methods relying on edge detection performance suffer from various image features. This is because it is hard to find a proper threshold value for each frame to mae a good edge map in real time and inaccurate edge detection may incur undesirable blur or regard blocing artifacts as edges. In order to find an efficient deblocing filter we investigate smoothing features in a video sequence in terms of the characteristics of HVS. According to these features across the bloc boundary a deblocing filter with two separate modes is proposed. Each filter is a one-dimensional filter appropriate in real time application. For the mode decision independent of artifact position we examine the existence of the offset in the region rather than the existence of the edge around the bloc boundary. In a very smooth region with small dc-lie offset strong smoothing is applied inside the bloc as well as on the bloc boundary. In the other region a sophisticated smoothing operation is applied to reduce blocing artifacts without introducing undesirable blur by using the frequency information around the bloc boundary. It has been shown that the proposed deblocing filter improves not only subjective image quality but also objective image quality for various image features.. PROPOSED DEBLOCKING FILTER A major operation of the deblocing filter is smoothing. But smoothing may introduce different effects according to local image characteristics across bloc boundaries. When we consider a smoothing operation to remove blocing artifacts in video coding we find three interesting observations. First the HVS is more sensitive to blocing artifacts in flat regions than in complex regions 9. Therefore a strong smoothing filter is required on those regions. In complex regions however smoothing of a few pixels around the bloc boundary is enough to achieve a desired deblocing effect. Secondly smoothing operations tend to introduce more undesirable blur in complex regions than in flat regions. Hence adaptive smoothing to preserve image details is mandatory in complex regions. Thirdly because of motion compensation blocing artifacts are propagated and the propagated artifacts are more visible in flat regions than in complex regions. Therefore smoothing in flat regions must cover the inside of a bloc as well as bloc boundaries. From the observations above it is found that the use of two separate filters depending on local image characteristics is preferable for effective deblocing. The filter for flat regions should provide a strong smoothing effect inside a bloc as well as on bloc boundaries. On the other hand the filter for complex regions is required to wor only on bloc boundaries. In the following discussions filtering modes for flat regions and for complex regions will be called as the DC offset mode and the default mode respectively. The filtering process consists of three major functional blocs i.e. mode decision filtering for the DC offset mode and filtering for the default mode; and is based on one-dimensional filtering along the boundaries of a 8 x 8 bloc (see Fig. )... Mode decision To select a proper mode between the DC offset mode and the default mode local image characteristics in the region are to be examined. In the proposed scheme we examine the flatness of the region by using the following measurement. When we consider a bloc boundary pixel array v = [v v v 8 ] 8 F( v) = φ( vi vi+ ) i= where φ( ) = T otherwise () In Eq. () T is set to a small value so that F(v) may reflect the flatness of the local image across a bloc boundary. If v is dc-lie F(v) has a big value larger than a certain threshold T. In that case v is assigned to the DC offset mode and strong smoothing is applied; otherwise v is assigned to the default mode and accurate and adaptive filtering is applied. Noticeable blocing artifacts may result from the concatenation of two flat regions with a small offset. In this case the DC offset mode is selected because F(v) still has a big value due to the two flat regions. It should be noticed that the value of F(v) does not depend on the location of blocing artifacts since it is obtained from difference values between neighboring pixels. This
mode decision part is summarized in Fig.... Filtering in the DC offset mode In this mode we apply a 9-tab smoothing filter inside the bloc as well as on a bloc boundary. To prevent real edges in the filtering region from smoothing however filtering is not performed when the difference between the maximum value and minimum value of v is larger than a certain value QP. Here QP is the quantization parameter of the macrobloc to which pixel v 5 belongs. This is because the offset related with blocing artifacts is usually a small value and is highly related with the quantization parameter. The filtering process of the 9-tab smoothing filter is as follows. 4 v n = b p 6 where = 4 n+ n 8 p m < v v v < QP pm = vm m 8 p = 9 8 v otherwise p m > and { b : 4 4} = { 4}. p 9 v9 v9 v8 < QP = v8 otherwise () In Eq. () p and p 9 are used as padded pixel values for 9-tab smoothing filter. In actual implementation 9-tab filtering can be simplified by using shift and addition operations only... Filtering in the default mode As was mentioned the default mode is applied on complex regions. In this mode we limit the pixels to smooth to the two bloc boundary pixels v 4 and v 5 and use the 4-point DCT as a frequency analysis tool to get the feature information of the pixel array. As shown in Fig. if a 4-point pixel array S is located across the bloc boundary 4-point DCT basis vectors of S have symmetric and anti-symmetric properties around the center of 4 points or the bloc boundary (refer Fig. ). We define a a a and a as the 4-point DCT coefficients of S. Then it is noticed that the high frequency anti-symmetric component a is a major factor affecting the blocing artifact. This means that the proper adjustment of a is directly related to the reduction of bloc discontinuity in the spatial domain. Thus in this mode the magnitude of a is to be scaled down by using a scaling factor MIN( a a a ) / a whose value is between and. The flatter neighboring regions are the smaller the scaling factor is. By doing this the bloc boundary is smoothened in smooth regions and is not affected in complex regions so that undesirable blurring can be prevented. If the magnitude of a is greater than a certain value (which is related with the quantization parameter) however the filter is not applied to preserve the image details. The detailed fitering procedure in the default mode can be described as follows. c v + c v + + c v + c v + 4 a = where c () and v 4 = v4 d (4) v = v + d 5 5 where c ( v4 v5) d = CLIP( ( a a ) ) c (5) MIN( a a a ) a a a = a a = Here CLIP(x p q) clips x to a value between p and q. And [c -c c -c ]/ c denotes a DCT ernel corresponding to the highest frequency component a. If we describe 4-point DCT/IDCT of S as a matrix form
a a = a a v v4 = v 5 v6 v v4 v5 v6 a a a a (6) (7) where =.5 π c = cos = =.65 8 c π = cos =.5 4 π c and = cos = =.76. 8 c (8) It is noted that using Eqs. (4) and (7) new values of v 4 and v 5 due to the change of a can be easily obtained without performing a full 4-point IDCT. If a increases by we can find thatv 4 = v 4 and v 5 = v 5. We should also note that the scaling of a can be easily achieved without a dividing operation because the scaled value a is given as SIGN(a )MIN( a a a ). The clipping operation in Fig. is used to mae sure that the magnitude of the gradient at the boundary is reduced without change in direction. Then the values of v 4 and v 5 always reside between and 55.. EXPERIMENTAL RESULTS Simulation is performed by using the MPEG-4 VM 5. coder for low bit-rate DCT-based video compression. The advanced prediction mode with 8 x 8 bloc motion vectors and the OBMC mode are turned on and a fixed quantization parameter is used. H.6 quantization method is adopted and the motion search range is [-6. 5.5]. Various MPEG-4 test sequences are used for the bit rates of Kbps 4Kbps 48 Kbps Kbps and Mbps. Each test sequence has frames and only the first frame is coded as the intra frame. The components of the DCT ernel corresponding to a c c and c are approximated to 5 and 8 respectively so that the filtering operation may require only integer multiplication and shift operations. And the threshold values of T = and T = 6 are used. The proposed deblocing filter is applied for all the bloc boundaries along the horizontal edges first and then along the vertical edges. If a pixel value is changed by the previous filtering operation the updated pixel value is used for next filtering. Fig. 4 shows deblocing results for the Foreman sequence. Even though the OBMC technique is used in the video coder blocing artifacts produced in low bit-rate coding are still objectionable. In addition we can see the drift of the location of blocing artifacts especially in the flat region. As shown in Fig. 4 the proposed deblocing filter reduces blocing artifacts substantially and still preserves edge details quite well. PSNR is also improved for all the test sequences with.4 db maximum in the Hall monitor sequence (see Table ). These PSNR improvements demonstrate that the proposed filtering scheme is robust with respect to various image characteristics.
4. CONCLUSIONS A new deblocing filter has been proposed to remove the blocing artifacts in reconstructed video frames. In order to minimize the computational complexity the filter is based on one-dimensional filtering with two separate modes. The selection of a proper mode and a corresponding filtering process provide the improvements of the PSNR as well as the subjective quality by reducing blocing artifacts without sacrificing image details. Because of its simple architecture the proposed deblocing filter can be easily implemented to real time applications. 5. REFERENCES. H. S. Malvar and D. H. Staelin The LOT: transform coding without blocing effect IEEE Trans. Acoust. Speech Signal Processing vol. 7 no. 4 pp. 55-559 989.. J. M. Shapiro Embedded Image Coding Using Zerotrees of Wavelet Coefficients IEEE Trans. Signal Processing vol. 4 no. pp. 445-46 Dec. 99.. Y. Naaya and H. Harashima Motion compensation based on spatial transformations IEEE Trans. Circuits Syst. Video Technol. vol. 4 no. pp. 9-56 June 994. 4. M. T. Orchard and G. J. Sullivan Overlapped Bloc Motion Compensation: An Estimation-Theoretic Approach IEEE Trans. Image Processing vol. no. 5 pp. 69-699 Sep. 994. 5. ITU-T Recommendation H.6 Video coding for low bitrate communication Draft May 996. 6. R. Rosenholtz and A. Zahor Iterative procedures for reduction of blocing effects in transform image coding IEEE Trans. Circuits Syst. Video Technol. vol. no. pp. 9-95 Mar. 99. 7. Y. Yang N. P. Galatsanos and A. K. Katsaggelos Regularized reconstruction to reduce blocing artifacts of bloc discrete cosine transform compressed image IEEE Trans. Circuits Syst. Video Technol. vol. no. 6 pp. 4-4 Dec. 99. 8. H. C. Kim and H. W. Par Signal adaptive postprocessing for blocing effects reduction in JPEG image Proc. Int. Conf. on Image Processing vol. pp. 4-44 Lausanne Switzerland Sep. 996. 9. S. A. Karunaseera and N. G. Kingsbury A distortion measure for blocing artifacts in images based on human visual sensitivity IEEE Trans. Image Processing vol. 4 no. 6 pp. 7-74 June 995.. MPEG-4 video verification model V.5. ISO/IEC JTC /SC 9/WG /N469 Nov. 996.
Bloc boundary S S v v v v v 4 v 5 v 6 v 7 v 8 v 9 Bloc boundary Pixels for filtering on a vertical edge v S v v v S S v 4 v 5 v 6 v 7 S v 8 v 9 Pixels for filtering on a horizontal edge Figure : 8 x 8 bloc boundaries. YES max min < QP YES Filtering in DC offset mode v = [v v v 8 ] max = MAX(v) min = MIN(v) F(v) T NO v = v NO NO a < QP YES Filtering in default mode Figure : Basic structure of the proposed deblocing scheme.
Wave number Symmetric Anti-symmetric Symmetric Anti-symmetric Bloc boundary Figure :DCT basis vectors for 4 points of S. Table : Performance evaluation of the proposed deblocing filter in terms of PSNR. PSNR_Y [db] Condition Sequence QP Bits Before filtering After filtering Kps Hall monitor 7 9658.4.7 QCIF Container ship 7 9556 9. 9.4 7.5Hz Mother & daughter 5 95579..48 4Kbps Hall monitor 9 6.85 4. QCIF Container ship 748.6.54 Hz Mother & daughter 8 79 5. 5.7 48Kbps Foreman 4788.9.6 QCIF Coast guard 4 4468 9. 9.9 Hz Silent voice 7 484656 4. 4.49 48Kbps News 8 4797..4 CIF Mother & daughter 4687 6.6 6.8 7.5Hz Hall monitor 45886.59 4. Kbps News 9868 4. 4.6 CIF Foreman 8458 8.5 8.5 5Hz Coast guard 9 746 6.6 6.4 Mbps Stefan 979675 9. 9. SIF Hz Mobile & Calendar 4 594 6.5 6.9
(a) (b) (c) (d) (e) (f) Figure 4: Deblocing results for Foreman sequence (CIF bps 5Hz); (a) (c) and (e) are the th 8 th and 6 th reconstructed images respectively; (b) (d)and (f) are deblocing results of (a) (c) and (e) respectively.