Progressve san onverson based on edge-dependent nterpolaton usng fuzzy log P. Brox brox@mse.nm.es I. Baturone lum@mse.nm.es Insttuto de Mroeletróna de Sevlla, Centro Naonal de Mroeletróna Avda. Rena Meredes S/N, Edfo CICA, 0 Sevlle (Span) S. Sánhez-Solano santago@mse.nm.es Abstrat De-nterlang algorthms realze the nterlaed to progressve onverson requred n many applatons. The most ost effent are ntra-feld tehnques, whh nterpolate pxels of the same feld. Some of these methods use the upper and lower lne pxels. Among them, the algorthm s wdely employed sne t reonstruts the edges of the de-nterlaed mage wth more auray elmnatng nondesred problems suh as blurrng and starase effets. However, the algorthm does not perform well when there are non lear edges or n presene of nose. In order to redue these drawbaks, a new algorthm s presented n ths paper. It s based on a smple fuzzy system whh models heurst rules to mprove the algorthm. Two enhanements of ths new algorthm are also presented n ths paper. Smulaton results of vdeo sequenes prove the advantageous of the new algorthms. Keywords: uzzy log, de-nterlang, edgedreton deteton,. Introduton The standard televson vdeo systems (NTSC, SECAM, PAL) are based on an nterlaed vdeo sgnal whh halves the sgnal bandwdth. However, the advent of hgh qualty montors, dsplays and HDTV systems have nreased the development of effetve tehnques whh provde an nterlaed to progressve onverson [-9]. They an be lassfed nto three ategores: spatal (or ntra-feld) tehnques, temporal (or nter-feld) tehnques, and hybrd methods. The most ost-effent are ntrafeld tehnques, whh nterpolate between pxels from the same feld. They are wdely used beause they requre less omputatonal power and only one delay lne buffer. Among them, the smplest one s lne doublng. However, jagged and flker effets appear n the oblque lnes and edges. The lne average method whh nterpolates wth the upper and lower pxels s also very usual. It redues onsderably flker and alas problem but the starase effet remans n the edges. Ths problem s avoded wth the dreton-dependent nterpolaton algorthms. The smplest one s edge-based lne average () algorthm whh uses the dretonal orrelatons between pxels of the adjaent san lnes to nterpolate the mssng lne lnearly []. Ths algorthm works well when the edge dretons are estmated orretly but, otherwse, t ntrodues errors and degrades the mage qualty. These problems appear when dretonal orrelatons are smlar (n horzontal edges, for nstane) and/or the mage s orrupted wth nose. To nrease the robustness and redue the senstvty to nose, a three-pont medan flter that uses nformaton from the prevous feld s proposed n [], but ths requres the use of an expensve feld memory. To mprove estmaton of edge dretons, the algorthm s ombned wth the lne doublng method n [] and an adaptve tehnque s proposed n [], whh are more ostly omputatonally. Other authors resort to the use of a larger neghbourhood for the deteton of edge dretons (5+5 taps n [5], 6+6 taps n [6], 7+7 taps n [7], up to + taps n [8], and taps n [9]), wth the resultng nrease of hardware resoures. Besdes, the use of threshold values s proposed n [] and [8] to ensure that estmated edges are domnant. A new dretonal edge nterpolaton tehnque s presented n ths paper whh mproves the robustness of the orgnal algorthm. A set of smple fuzzy rules ndate the edge dretons and a lnear nterpolaton s realzed between taps of the urrent feld. The mprovements of ths new method are obtaned where there are non lear edges and when nosy mages are proessed. Besdes, ths s aheved wth a low nrease n omputatonal ost. Two modfatons of ths new algorthm 699
workng wth a large neghbourhood of pxels of the same feld have been also analyzed. The frst one onsders 5+5 taps whle the seond one uses samples from the prevous de-nterlaed lne. The paper s organzed as follows. Seton desrbes the proposed fuzzy algorthm and ts modfaton. Several smulaton and omparson results are shown n Seton. nally, some onlusons are gven n Seton. uzzy Edge-based Lne Average Algorthms. uzzy_ algorthm ( taps) gure shows the pxels used by the algorthm to nterpolate the pxel value X=(x,y). The pseudo ode of ths algorthm s as follows: a= A- b= B-E = C-D () f mn{a,b,=a, X=(A+)/ elsef mn{ a,b,=, X=(C+D)/ else X=(B+E)/ The dreton a orresponds to an edge under 5º and b to an edge under 5º. The zero or mnmum orrelaton does not always ndate the dreton of an edge. or example n presene of nose or when there s an edge not lear. If we apply heursts, we wll say that an edge s lear n dreton a not only f a s small but also b and are large, and somethng smlar happens to an edge n dreton. If there s a strongly small orrelaton n dretons a and and a large orrelaton n dreton b, there s not an edge but vertal lnear nterpolaton does not perform well. The best opton s a lnear nterpolaton between the neghbours wth small orrelatons: A, C, D,. In other ase the best thng s to alulate a vertal lnear nterpolaton. Ths heurst knowledge s fuzzy sne the onepts of small and large are not understood as threshold values but as fuzzy ones. Hene, our proposal s to model ths knowledge by a fuzzy system. The rule base of our fuzzy system s desrbed n Table. Usng fuzzy log, the onepts of SMALL and re represented by membershp funtons that hange ontnuously nstead of abruptly between 0 and membershp values, as shown n gure (a), (b). The lngust hedge strongly atng upon the onept of SMALL modfes ts membershp funton as llustrated n gure () [0]. Ths fuzzness makes all the rules may be atvated smultaneously, ontrary to what happens n the algorthm. The mnmum operator s used as onnetve and of anteedents and the atvaton degrees of the rules, α, are alulated as follows: { SMALLa ( h), LARGE b( h), LARGE ( h) { ( h), LARGE b( h), SMALL( h) ( h), ( h), ( h) α = mn α = mn α = mn α = α α α { stronglysmall a LARGE b stronglysmall Sne the onsequents,, of the rules are not fuzzy, the global onluson provded by the system s alulated by applyng the uzzy Mean defuzzfaton method, as follows: ( x, y) = α Table : uzzy Rule Set f anteedents then onsequent ) (a s SMALL) and (b s LARGE) (A+)/ and ( s LARGE) ) (a s LARGE) and (b s LARGE) (C+D)/ and ( s SMALL) ) (a s strongly SMALL) and (b s LARGE) (A++C+D)/ and ( s strongly SMALL) ) otherwse (B+E)/ α () = () = α α = Substtutng the onsequents,, by ther values n y- y x- x x+ A B C x ORIGINAL LINE INTERPOLATED LINE LARGE SMALL strongly SMALL y+ ORIGINAL LINE D E gure : x wndow for the algorthm H h H h H/ h (a) (b) () gure : Membershp funtons SMALL and LARGE 700
Table, and applyng that α +α +α +α s equal to, the above expresson an be gven as: y- y y+ ( x, y) x- x- x x+ x+ A A B C C A + = α B + E C + D α A + + C + α D Hene, the fuzzy algorthm apples a lnear nterpolaton n the dretons a or b f there s a lear edge n that dreton (one α takes the value and the others are 0). Otherwse several rules are atve and the nterpolaton s non lnear.. uzzy_ algorthm (5+5 taps) X D D E The algorthm proposed n the prevous seton searhes for edges n only two dretons a and. A larger neghbourhood has been onsdered n order to nrease the edge dretons. gure shows the pxels used n ths new verson of fuzzy algorthm. The number of taps has been nreased from to 5+5 and two new edge dretons are onsdered a y. () a = A - = C -D (5) The added rules (frst and seond n Table ) Table : uzzy Rule Set Interpolated lne gure : 5x5 wndow for the uzzy_5+5 f anteedents then onsequent ) (a s SMALL) and (a s LARGE) and (b s LARGE) (A + )/ and ( s LARGE) and ( s LARGE)) ) (a s LARGE) and (a s SMALL) and (b s LARGE) (A+)/ and ( s LARGE) and ( s LARGE)) ) (a s LARGE) and (a s LARGE) and (b s LARGE) (C+D)/ and ( s SMALL) and ( s LARGE)) ) (a s LARGE) and (a s LARGE) and (b s LARGE) (C +D )/ and ( s LARGE) and ( s SMALL)) 5) (a s strongly SMALL) and (b s LARGE) (A++C+D)/ and ( s strongly SMALL) 6) otherwse (B+E)/ onsder the new dretons a and (orrespondng to 5,º and 6,6º). The onepts of SMALL and re represented by the membershp funtons prevously defned. The produt operator s now used to represent the onnetve and so that the atvaton degrees of the rules, α,, ould be alulated as follows: α = prod { SMALL a' LARGE b LARGE LARGE ' ( h) α = prod { ' LARGE b LARGE SMALL ' ( h) α = prod { SMALL a LARGE b LARGE ( h) ( 6) α = prod { LARGE b SMALL ( h) α = prod ( h) 5 6 { stronglysmall a LARGE b α = α α α α α 5 stronglysmall The onluson s also obtaned usng the uzzy Mean Defuzzaton method as follows: A' + ' C' + D' A + ( x, y) = α (7) C + D A + + C + D B + E 5 6. Reursve uzzy_ algorthm The orgnal uzzy algorthm ( taps) s modfed n order to nrease the edge-deteton onssteny. The domnant dreton s aheved usng further nformaton from the neghbourhood. gure shows the pxels evaluated. A, C are samples from the prevous de-nterlaed lne (the frst lne de-nterlaed s alulated wth the uzzy method desrbed prevously). The new orrelatonal dretons onsdered are gven by the expressons n (8). a = A -A a = A - = C -C = C -D (8) The new fuzzy rule set s desrbed n Table. The frst and seond rules are modfed to ensure the presene of an edge n one of the two orentatons y- y- y y+ x- x+ A C x- x x+ A B C D X E gure : Wndow for the uzzy-reursve algorthm Prevous Denterlang lne De-nterlang lne 70
Table : uzzy Rule Set f anteedents then onsequent )((a s SMALL) or (a s LARGE) or (a s LARGE)) (A +A+ )/ and (b s LARGE) and ( s LARGE) and ( s LARGE)) ) (a s LARGE) and (a s LARGE) and (b s LARGE) (C +C+D )/ and( ( s SMALL) or ( s SMALL) or ( s SMALL)) ) (a s strongly SMALL) and (b s LARGE) (A++C+D)/ and ( s strongly SMALL) ) otherwse (B+E)/ (angle 5º or 5º). The atvaton degree of the rules, showed n (0), are alulated usng the mnmum as onnetve and and the maxmum as onnetve or. max ( SMALL a SMALL a SMALL a( h) ), α = mn LARGE b LARGE LARGE ( h) (9) LARGE b( h), α = mn max ( SMALL a SMALL SMALL ( h) ) α = mn { stronglysmall a LARGE b stronglysmall ( h) α = α α α The fnal onluson s obtaned by applyng the same defuzzfaton method, as follows: ( x, y) = A + A + A + C + C + C + D α (0) A + + C + D B + E Smulaton results Extensve smulatons ondued wth vdeo sequenes and stat mages have been performed to analyze the proposed methods over other ntra-feld methods. The even/odd lnes of orgnal progressve frames have been elmnated from the frame of the sequene wth an even/odd number. These mssng lnes have been alulated by applyng a denterlang tehnque. The lne doublng, lne average,, the uzzy_ algorthm wth rsp nstead of fuzzy desrptons of labels SMALL and LARGE (rsp method), the method developed n [5], and our fuzzy algorthms have been programmed n Matlab. Table shows the average of PSNR values from 50 th to 70 th de-nterlaed frames of the Clare sequene. Table 5 shows the average of PSNR values from 6 th to 0 th frame of Suse gure 5: a) One progressve frame of the Clare sequene sequene. The robustness of the algorthms aganst nose has been analyzed by addng artfal nose to orgnal frames. Two types of nose (Gaussan and mpulsve) have been analyzed wth dfferent degree of degradaton. The results show that our proposed methods perform onsderably better than the orgnal algorthm. Comparng wth the lne average method the results are slghtly better. However, fuzzy algorthms reonstrut better the edges of the de-nterlaed frames. Ths s orroborated wth the zooms (gure 6) of the de-nterlaed frame obtaned from one of the progressve frames of the sequene (gure 5). uzzy algorthms perform better when artfal nose s added. The frst olumn of tables show the (a) () (b) (d) gure 6: Zoom of the de-nterlaed frame wth lne doublng (a), lne average (b), (), and the uzzy_ algorthms (d) 70
(a) (b) () (d) gure 7: Orgnal progressve frame orrupted by Gaussan nose (a) and mpulsve nose (b). The orrespondng de-nterlaed mages wth uzzy_ method () and (d) PSNR value obtaned from the omparson between the progressve orgnal frame and the nosy one. The seond olumn shows the best expeted result whh has been obtaned denterlang felds wth the orgnal lnes wthout nose. The rest of the olumns show the obtaned results applyng denterlang methods (gure 7). Analyzng the results, the uzzy_ 5+5 method slghtly works better than the uzzy_ n ases that there s no nose or wth mages orrupted by mpulsve nose, whereas the reursve fuzzy_ algorthm mproves the results gradually when the degree of degradaton of the nosy mage nreases. The threshold value H used n the desrptons of the fuzzy labels (gure ) has been fxed heurstally to a value that has been the same n all the analyss. It has been proven that the results are not very senstve to ths value. Conlusons The uzzy_ algorthm mproves onsderably the qualty of de-nterlaed mages n omparson wth results. It works as well as reonstrutng lear edges and enhanes the results n the rest of the frame. The mprovements nrease gradually wth the presene of nose n the frames of the sequenes. Besdes, t requres a low omputatonal ost so that ts hardware mplementaton s very smple. The two enhanements of the algorthm mprove slghtly the results and requre a rse of resoures for ther mplementaton. These results have been obtaned from the analyss of the extensve smulatons of vdeo sequenes. Referenes [] T. Doyle and M. Looymans, Progressve san onverson usng edge nformaton, Pro. rd Int. Workshop on HDTV, 989. [] T. Doyle, Interlaed to sequental onverson for EDTV applatons, Pro. nd Int. Workshop Sgnal Proessng of HDTV, pp. - 0, 988. [] M. H. Lee, J.H. Km, J.S. Lee, K.K.Ryu and D. Song, A new algorthm for nterlaed to progressve san onverson based on dretonal orrelatons and ts IC desgn, IEEE Trans. on Consumer Eletrons, vol. 0, n., pp. 9-9, 99. [] C. J. Kuo, C. L. and C. C. Ln, Adaptve nterpolaton tehnque for sannng rate 70
Table : PSNR (n dbs) values for Clare sequene (50 th - 70 th frame) Input nosy mage Best expeted mage Double Average Crsp Modfed [5] uzzy uzzy 5+5 Reursve uzzy No nose 5,75 0,95 0,55 0,9 0,96,,5, 5,5% 7,55 0,57 7,5 8,77 0, 9,59 9,86 0, 0,5 0, Impulse Nose (Densty Nose) Gaussan Nose (Varane) 6,5% 6,79 9,87 6,8 8,06 9,7 8,89 9, 9,58 9,59 9,6 7,5% 6, 9,6 6,9 7,5 8,8 8,9 8,5 8,95 8,96 8,99 %, 7,9,5 5,8 6,5 6, 6, 6,76 6,80 6,8 5%, 6,,0, 5, 5, 5, 5,68 5,75 5,77 0,00 5,65 8,66 5, 6,69 6,7 6,69 6,7 7, 7,0 7, 0,009,0,96 0,85,5,9,5,,76,59,79 0,05 8,8,8 8,75 0,0 0,07 0,0 0,8 0,6 0,5 0,69 0,0 7, 0,9 7,6 8,59 8,66 8,59 8,8 9, 9,5 9, 0,07 6,9 9,7 6, 7,56 7,6 7,57 7,8 8,5 8, 8,7 onverson, IEEE Trans. on Cruts and Systems for Vdeo Tehnology, vol. 6, n., 996. [5] H. Y. Lee, J. W. Park, T. M. Bae, S. U. Cho and Y. H. Ha, Adaptve san rate up-onverson system based on human vsual haratersts, IEEE Trans. on Consumer Eletrons, vol. 6, n., 000. [6] J. Salonen and S. Kall, Edge adaptve nterpolaton for sannng rate onverson, n Sgnal Proessng of HDTV IV, pp. 757-76, 99. [7] G. De Haan, R. Lodder, De-nterlang of vdeo data usng moton vetor and edge nformaton, Pro. IEEE Int. Conf. on Consumer Eletrons, 00. [8] Y. L. Chang, S.. Ln and L. G. Chen, Extended ntellgent edge-based lne average wth ts mplementaton and test method, Pro. ISCAS 00, 00. [9] H. Yoo and J. Jeong, Dreton-orented nterpolaton and ts applaton to denterlang, IEEE Trans. on Consumer Eletrons, vol. 8, n., pp. 95-96, 00. [0] M. Sugeno and T. Yasukawa, A fuzzy-logbased approah to qualtatve modelng, IEEE Trans. uzzy Systems, vol., n., pp. 7-, 99. Table 5: PSNR (n dbs) values for Suse sequene (6 th - 0 th frame) Input nosy mage Best expeted mage Double Average Crsp Modfed [5] uzzy uzzy 5+5 Reursve uzzy No nose,7 6,6 5,66 6, 6,5 6,9 6,7 6,8 5,5% 8,8, 7,76 9,5 0,8 0, 0,55 0,97,0 0,99 Impulse Nose (Densty Nose) Gaussan Nose (Varane) 6,5% 7,55 0,59 7,5 8,7 0,06 9, 9,79 0, 0,9 0,6 7,5% 6,9 9,96 6,86 8, 9,8 8,8 9,7 9,58 9,65 9,6 %,9 7,9,8 6,09 7, 6,78 7,07 7, 7,5 7,8 5%,9 6,9,88 5, 5,98 5,79 6,06 6, 6,6 6, 0,00 6,99 9,99 5,95 7,59 7, 7,6 7,65 8,08 7,9 8,09 0,009 0,8,9 0,,58,5,57,69,9,05,8 0,05 8,9, 8, 9,5 9, 9,5 9,6 0,0 9,98 0,8 0,0 6,87 9,89 6,76 8,06 7,99 8,06 8,8 8,6 8,6 8,79 0,07 5,85 8,85 5,75 7,0 6,96 7,0 7, 7,58 7,58 7,76 70