MEL A MITSUBISHI ELECTIC ESEACH LABOATOY http://www.merl.com Codng Artfact educton Usng Edge Map Guded Adaptve and Fuzzy Flter Hao-Song Kong Yao Ne Anthony Vetro Hufang Sun Kenneth E. Barner T-2004-056 June 2004 Abstract Ths paper presents a new adaptve approach for blockng and rngng artfact reducton. In order to avod smearng of the mage detals, the proposed method frst performs vsual artfacts detecton and then apples adaptve flterng to the corrupted blocks. Both vsual artfacts detecton and flterng are guded by an edge map whch s constructed based on the local features. The fuzzy dentty flter s used for mage de-rngng. Snce t possesses a good edge preservng property and the flterng operaton s appled to the edge blocks only (smooth and textured blocks are unaltered), the proposed method shows great effectveness of both artfacts reducton and detal preservaton. Experments demonstrate better results compared wth the other methods. Ths work may not be coped or reproduced n whole or n part for any commercal purpose. Permsson to copy n whole or n part wthout payment of fee s granted for nonproft educatonal and research purposes provded that all such whole or partal copes nclude the followng: a notce that such copyng s by permsson of Mtsubsh Electrc esearch Laboratores, Inc.; an acknowledgment of the authors and ndvdual contrbutons to the work; and all applcable portons of the copyrght notce. Copyng, reproducton, or republshng for any other purpose shall requre a lcense wth payment of fee to Mtsubsh Electrc esearch Laboratores, Inc. All rghts reserved. Copyrght c Mtsubsh Electrc esearch Laboratores, Inc., 2004 201 Broadway, Cambrdge, Massachusetts 02139
Proc IEEE Int l Conference on Multmeda and Expo (ICME)
CODING ATIFACTS EDUCTION USING EDGE MAP GUIDED ADAPTIVE AND FUZZY FILTEING Hao-Song Kong 1, Yao Ne 2, Anthony Vetro 1, Hufang Sun 1 and Kenneth E. Barner 2 1 Mtsubsh Electrc esearch Labs 201 Broadway, Cambrdge, MA 02139 hkong@merl.com, avetro@merl.com, hsun@merl.com ABSTACT Ths paper presents a new adaptve approach for blockng and rngng artfact reducton. In order to avod smearng of the mage detals, the proposed method frst performs vsual artfacts detecton and then apples adaptve flterng to the corrupted blocks. Both vsual artfacts detecton and flterng are guded by an edge map whch s constructed based on the local features. The fuzzy dentty flter s used for mage de-rngng. Snce t possesses a good edge preservng property and the flterng operaton s appled to the edge blocks only (smooth and textured blocks are unaltered), the proposed method shows great effectveness of both artfacts reducton and detal preservaton. Experments demonstrate better results compared wth the other methods. 1. INTODUCTION Hgh compresson technques are requred n many magng and vdeo applcatons. Vsual artfacts are normally present n decompressed mages due to coarse quantzaton and coeffcent truncaton. Blockng and rngng artfacts are the two major codng artfacts caused by hgh compresson. Many post-processng approaches have been proposed to remove the vsual artfacts ether from the spatal doman or the frequency doman [1],[2],[9]. They attempt to adaptvely flter each pxel n the mage based on quantzaton parameter and neghborng nformaton. Snce these flterng methods are pxel-by-pxel operatons, they nevtably ntroduce undesred smoothng effects to non-artfacts pxels. ecent research has proposed classfcaton-based methods to detect the artfacts before applyng the post-flterng [3],[4]. However, these methods manly concentrate on blockng artfacts, and are less effectve n removng rngng artfacts. In ths paper, we propose a new adaptve approach for both blockng and rngng artfacts removal. The proposed 2 Department of Electrcal and Computer Engneerng Unversty of Delaware, Newark DE 19716 yne@ee.udel.edu, barner@ee.udel.edu method s based on edge nformaton extracton and edge preservng flterng. Snce codng artfacts are observed as certan patterns to the human vsual system (for example, blockng artfacts appear as block boundary dscontnutes and rngng artfacts always appear around sharp edges), we frst apply pattern classfcaton technques to dentfy dfferent type of artfacts and then perform the flterng accordngly. Our strategy s as follows: 1) form an edge (varance) map based on the local statstcs; 2) accordng to the edge map, detect the blockng artfacts and classfy the codng blocks nto three (smooth, edge and texture) categores [5]; 3) apply a 1-D low-pass flter to reduce the blockng artfacts and a 2-D fuzzy dentty flter [6] to reduce the rngng artfacts. Snce the fuzzy dentty flter s appled to the edge blocks only (other types of blocks are unaltered) and t possesses a good edge preservng property, the fltered mages look sharp and clean. 2. THE POPOSED APPOACH Fgure 1 shows the system dagram of the proposed post-flterng scheme. The system conssts of the followng modules: feature extracton, blockng artfacts detecton and de-blockng, pxel/block classfcaton and de-rngng. Decoded Image Feature Extracton Generate Varance/ Edge map Detect Blockng Artfacts emove Blockng Artfacts Classf y Pxels Classfy Blocks Class_0 Class_1 Class_2 Fgure 1. Proposed post-flterng scheme. Smooth Texture Edge emove ngng Artfacts
2.1 Local feature extracton Snce the local varance can effectvely characterze the local feature of the mage, the varance wthn a 3x3 wndow centered at each pxel s calculated. The local varance values of all the pxels form a varance (edge) map for the entre mage. 2.2 Blockng artfact detecton and de-blockng The blockng artfact detecton s performed on the nonoverlappng 8 8 blocks n the edge map obtaned n the prevous step. For each block, only the top and left boundares need to be checked as shown n Fgure 2. The crteron for decdng that blockng artfacts are present at the correspondng boundary s 6 = 1 sgn ( ) 5, where and (=1,2, 6) represent the local varance values of 6 boundary pxels and 6 nner pxels adjacent to the boundary, respectvely. If the above crteron s not met, no flterng s performed; otherwse, a 1-D low-pass flter s appled across the boundary as shown n Fgure 3; the flter sze s adaptve to the local gradents across the boundary so that flter wth smaller sze s appled to where lower gradent presents. 2.3 Pxel/Block classfcaton and de-rngng The pxel classfcaton s performed by applyng thresholds to the local varance values n terms of the followng crtera: pxel ( x, y ) class _0 class _1 class _2 f σ 2 ( x, y ) < thresh _1 f σ 2 ( x, y ) > thresh _2 2 f thresh _1 σ ( x, y ) thresh _ 2 Here, σ 2 ( x, y) s the local varance value at pxel poston (x, y), class_0 (low varance) represents pxels of smooth regons, class_1 (hgh varance) represents pxels on edges, and class_2 (medum varance) ndcates pxels of texture regons. Thresh_1 and Thresh_2 are emprcally set to 10 and 400, respectvely. They are found to have very robust performance n varous vdeo sequences. Based on the pxel classfcaton, each 8 8 block can be classfed as smooth, edge or texture block. In our scheme, a block s decded to be an edge block f at least one edge pxel present n the block. Snce rngng artfacts usually occur around the sharp edges, the de-rngng operaton skps all smooth and texture blocks and s appled to edge blocks only. In ths operaton, each pxel n the edge block s fltered by an edge preservng flter,.e., the fuzzy dentty flter, n a 5 5 wndow. In the followng, we ntroduce the fuzzy dentty flter n more detals. Fgure 2. Blockng artfact detecton based on varance dfferences wthn an 8 8 block. Fgure 3. Blockng artfact removal usng 1-D low-pass flter. Three possble flter szes are shown at three sample postons. 2.4 Fuzzy dentty flter The fuzzy dentty flter, whch s derved from the fuzzy transformaton theory [6], has been successfully appled to codng artfacts reducton recently [11]. Ths flterng technque drected by the classfed edge map provdes an optmal soluton for rngng artfacts reducton. To see ths, we brefly revew the prncple of fuzzy transformaton and the edge preservng propertes of the fuzzy dentty flter. In fuzzy transformaton, relatonshps between each spatal sample x ( s the spatal ndex) and each order statstc x ( j) (j s the rank ndex and x ) wthn an ( 1) x(2)... x( N ) observaton wndow are establshed through a real-valued fuzzy spatal-rank (S) matrx, whch s defned by ~ ~ 1,(1) L 1,( N ) ~ = M O M, ~ ~ N,(1) L N,( N ) ~ where = µ ~ ( x, x ) [0,1],, j 1,2,..., µ ~ ( a, s a,( j) ( j) = N membershp functon to compute the fuzzy relaton between a and b wth the followng restrctons:
1. lm ~ ( a, 1 a b 0 µ = 2. lm a b µ ~ ( a, = 0 3. a1 b1 a2 b2 µ ( a1, b1) µ ( a2, b % % 2) 2 ( a / 2ξ The Gaussan membershp functon µ s G ( a, = e used n ths paper, where the spread parameterξ = 20. Snce the element values are dependent to the dstance between each par of samples, the fuzzy S matrx contans the spread nformaton embedded n the observaton samples. The orgnal (crsp) spatal samples can be transformed nto fuzzy spatal samples by multplyng the crsp order statstcs vector wth the row normalzed fuzzy S matrx. The output of the fuzzy dentty flter s just the fuzzy counterpart of the center sample n the observaton (flterng) wndow, whch can be obtaned usng the followng smplfed formula: y N x 1 ( j) µ xc x j= % ( j) = x% c = N µ ( x, 1 c x( j) ) j = % (, ), where x and x% are crsp and fuzzy center sample, c c respectvely. It s known that fuzzy transformaton has the property that t clusters the smlarly valued samples around ther local mean and leave solated samples unchanged. Therefore, fuzzy dentty flter possesses a data-adaptve smoothng feature and thus can perfectly preserve the strong edges whle removng weak ones. By applyng fuzzy dentty flter to the edge blocks only, we are able to removes annoyng rngng artfacts, preserve edges and avod unnecessary smoothng as well. 3. EXPEIMENTAL ESULTS Sx vdeo sequences were used for the evaluatons. These sequences were encoded and decoded usng MPEG-2 TM5 codec wth dfferent quantzaton scale parameters. In order to compare the performance of the proposed algorthm wth the exstng methods, four typcal methods are selected as references: the MPEG-4 flter descrbed n [4], the wavelet-based flter n [7], the DCT-doman flter n [8] and the POCS approach descrbed n [10]. One of the test sequence mages s gven n Fgure 4 for the subjectve comparson. It can be seen from the fgures that the wavelet method s able to remove most blockng and rngng artfacts, but t also blurs the entre mage. The MPEG-4 flter s good at de-blockng, but s unable to remove rngng artfacts. The POCS based method suppresses more rngng artfacts than MPEG-4, but also blurs the mage lke the wavelet method. The DCT flter mantans the sharpness of the mage, however, lke the MPEG-4 flter, t cannot remove the rngng artfacts successfully. It s evdent from the subjectve results that the proposed edge map guded edge preservng flterng method s the only method that s able to retan edge sharpness, yet stll removes the rngng artfacts. In terms of complexty, our proposed technque has very low complexty and s close to that of the MPEG-4 technque. The complexty of both technques s much less than the other three technques that have been evaluated. 4. CONCLUSIONS In ths paper, we proposed a new adaptve post-flterng scheme, whch sgnfcantly reduces the blockng and rngng artfacts whle preservng the mage detals. Based on the edge map nformaton, an adaptve 1-D flter and a 2-D fuzzy dentty flter are appled to remove blockng and rngng artfacts, respectvely. The expermental results demonstrate the superor performance of the proposed algorthm compared wth the exstng methods that have been evaluated. EFEENCES [1] S. Yang, Y. H. Hu, T. Q. Nguyen, and D. L. Tull, Maxmum-lkelhood parameter estmaton for mage rngng-artfact removal, IEEE Trans. Crcuts Syst. Vdeo Technol., Vol. 11, No. 8, pp. 963-973, Aug. 2001. [2] S. Lu, and A. C. Bovk, Effcent DCT-doman blnd measurement and reducton of blockng artfacts, IEEE Trans. Crcuts Syst. Vdeo Technol., Vol. 12, No. 12, pp. 1139-1149, Dec. 2002. [3] C. B. Wu, B. D. Lu, and J. F. Yang, Adaptve postprocessors wth DCT-based block classfcatons, IEEE Trans. Crcuts Syst. Vdeo Technol., Vol. 13, No. 5, pp. 365-375, May. 2003. [4] ISO/IEC 14496-2:2001, Informaton Technology Generc Codng of Audo-Vsual Objects Part 2: Vsual, 2 nd Ed., Appendx F3 Post processng for codng nose reducton, 2001. [5] H. Kong, and L. Guan, A nose-exclusve adaptve flterng framework for removng mpulsve nose n dgtal mages, IEEE Trans. Crcuts and System, Vol. 45, No. 3, pp. 422-428, Mar. 1998. [6] Y. Ne, and K. E. Barner, Fuzzy transformaton and ts applcatons, IEEE ICIP, Barcelona, Span, Sept., 2003 [7] Z. Xong, M. T. Orchard, and Y. Q. Zhang, A deblockng algorthm for JPEG compressed mages usng overcomplete wavelet representatons, IEEE Trans. Crcuts Syst. Vdeo Technol., Vol. 7, No. 2, pp. 433-437, Aug. 1997. [8] T. Chen, H.. Wu and B. Qu, Adaptve post-flterng of transform coeffcents for the reducton of blockng artfacts, IEEE Trans. Crcuts Syst. Vdeo Technol., Vol. 11, No. 5, pp. 594-602, May 2001. [9] T. Meer, K. N. Ngan and G. Crebbn, educton of blockng artfacts n mage and vdeo codng, IEEE Trans. Crcuts Syst. Vdeo Technol., Vol. 5, No. 3, pp. 490-500, Apr. 1999. [10] A. Zakhor, Iteratve procedures for reducton of blockng effects n transform mage codng, IEEE Trans. Crcuts Syst. Vdeo Technol., Vol. 2, No. 1, pp. 91-95, Mar. 1992. [11] Y. Ne, and K. E. Barner, Optmzed fuzzy transformaton for mage deblockng, IEEE ICME, Baltmore, Maryland, July, 2003.
(a) Decoded mage; PSN = 30.69 db ( Wavelet result; PSN = 29.86 db (c) MPEG-4 result; PSN = 30.88 db (d) POCS result; PSN = 30.12 db (e) DCT result; PSN = 30.21 db (e) Proposed method result; PSN = 30.93 db Fgure 4. Objectve and subjectve results for the News sequence.