Noise filtering for television receivers with reduced memory

Transcription

1 Noise filtering for television receivers with reduced memory R. J. Schutten, G. de Haan and A. H. M. van Roermund. Philips Research Laboratories, Television Systems Group, Prof. Holstlaan 4, 5656 AA Eindhoven, The Netherlands. Abstract: Television receivers are often used in places with sub-optimal reception conditions, therefore noise reduction is becoming a more popular feature. To include noise reduction in low-end TV receivers, low-cost noise reduction filters are desired. The paper describes a concept to generally reduce the memory capacity required by a recursive noise filter. This is achieved by subsampling. The principle of subsampling is tested on a previously developed 2-D spatial noise filter.. Introduction The use of memory in TV applications has increased significantly in the past few years, mainly because of the introduction of more sophisticated forms of signal processing. Specially in applications that include scan rate conversion, e.g. from 5 to Hz field frequency, use of large amounts of memory is no exception. In many of these applications the available memory can also be used for other types of signal processing, e.g. for first order temporal recursive noise filtering. However, there is still a large market for 5/6 Hz sets that do not have these amounts of memory to begin with. If one wants to incorporate noise reduction in these types of sets the added value of the feature is insufficient to allow the use of large amounts of memory, e.g. a full field memory. Noise filters that do not require a full field memory have been reported. Recently a 2-D spatial noise filter for TV applications has been developed (see [, 2]). To realize the 2-D spatial filtering, this noise filter applies just one line memory. This way the cost is reduced considerably. However, this is not the only way to achieve memory reduction. One can further reduce the amount of memory, by applying subsampling to the signal that is to be stored in memory. The paper analyses the concept of memory reduction applied to the 2-D spatial noise filter of [, 2], where the original version of the noise filter will be used as a reference. If the spatial noise filter with reduced memory has a reasonable performance, an implementation of this noise filter in an analogue rather than a digital process is an attractive option, as an analogue implementation is expected to be cheaper than the digital variant. 2. Concept of memory reduction The use of recursive filters is widespread in image data processing, mainly because of their improved hardware efficiency. This also applies to noise filters. Delft University of Technology, Faculty of Electrical Engineering, Electronics Group.

2 Noise filters generally are (adaptive, non linear) low-pass filters. An implication of a multi-dimensional recursive noise filter could be that output of the filter is already (low-pass) filtered in a direction orthogonal to the direction of the recursion, i.e. the data at the input of the memory is already low-pass filtered. This leads to the expectation that subsampling of the memory element(s) required for the recursion may be possible without artifacts (see figure ). Subsampling of a field memory with a factor of two in a temporal noise filter has first been reported by [4]. However, here the output is not orthogonally filtered with respect to the direction of recursion. The subsampling would theoretically be possible if the spectrum at the input of the memory were sufficiently, i.e. below the visibility level, suppressed for those frequencies exceeding the Nyquist limit. However, an adaptive noise filter will not always filter at maximum strength as its behaviour depends on the image content, so the output signal will not be low-pass filtered in all cases. If further the subsampling factor is chosen relatively high, the Nyquist criterion will not be always met. However, even when the subsampling jeopardizes the Nyquist criterion, the subsampling artifacts are not always clearly visible. It is expected that the non linear characteristic of noise filters (applied to prevent filtering of data with a low correlation) will prevent the introduction of severe artifacts. When the adaptation of the noise filter to the image content results in a reduction of the filter bandwidth, the data from the subsampled memory in the feedback loop (see figure ) will be used with reduced weight factors. Thus implying that subsampling artifacts present in the interpolated subsampled signal will be less present in the output of the filter. In Adaptive Mixing Subsampling LPF N Memory N LPF Out Figure. General concept of memory reduction in a recursive noise filter.. Specific implementation of memory reduction To test the concept of memory reduction it will be implemented on the 2-D spatial filter of [, 2]. This filter uses one line memory in its filtering. Therefore, it may not seem the most relevant case. It serves, however, as a test case for the concept. In the implementation of the idea we will apply subsampling with a factor of four.. Original implementation The 2-D spatial noise filter of [, 2] is a variant of the sigma nearest neighbour filter of [] and the gradient inverse filter of [5]. More specific it is a 2-D sigma nearest neighbour filter with two thresholds that is recursive in the vertical direction and transversal in the horizontal direction.

3 For every pixel position x, and input luminance signal Fxt (, ), the proposed filter output F F ( x, is defined as: F F ( x, Gx (, K ( x, n, Fx ( + nt, ) n N n N 2 ( ) F F ( x + n, K 2 x, n, t Where N and N 2 are sets of vectors defining a neighbourhood. The recursiveness of the filter results from the second term in equation (). Furthermore, K ( x, n, and K 2 ( x, n, are filter weights, related to the absolute difference between the weighted pixel and the current input pixel: + () K ( x, n, f( F( x, Fx ( + nt, )) K 2 ( x, n, f( F( x, F F ( x + n, ) Function f is a monotonously decreasing function, function normalization factor defined as: G ( x, K ( x, n, t ) + n N The function f is selected such that: fa () n N 2, a Th --, Th 4 < a Th 2, a > Th 2 K 2 ( x, n, Gx (, (2) in equation () is a gain or () (4) The thresholds Th and Th 2 should be adopted to the noise level, while Th 2 4 Th is experimentally chosen. Using only one line memory, the neighbourhoods are chosen as: N 2 2,, N w 4 + w w 4 + w 8 + w,,,, (5) (6) Variable w varies cyclically on a line basis following the sequence {,,, }. For more background and a subjective evaluation of the spatial filter we refer to [] and [2]..2 Subsampled implementation The subsampled version of the filter differs in two aspects with the description above. First the line memory is subsampled with a factor of four and second the line-alternating offset of the line memory is absent, so w.

4 The decimation (filtering and subsampling) with a factor of four is achieved by only storing the average of four filtered samples in memory. The upconversion back to the original sample rate is achieved by a linear interpolation, i.e. two subsampled memory elements are linearly combined with factors i 8 and ( 8 i) 8, with i { 57,,, } depending on the position. (See figure 2 for a representation of the construction of one filtered, subsampled and interpolated sample.) original filtered data interpolated sample obtained by linear interpolation Σ/4 Σ/4 /8 5/8 subsampled line memory obtained by averaging of 4 samples Figure 2. Schematic representation of subsampling and interpolation. The line-alternating offset of the line memory is not used in combination with the subsampling. The interpolated data from the line memory does not contain any high frequencies, so shifting the data by one pixel (which equals only a small phase shift for low frequencies) has very little effects. 4. Mathematical evaluation The behaviour of both filters depends on the image data. The behaviour is approximated by its transfer function in low contrast areas, i.e. all differences are below the lowest threshold. This means that all coefficients of the samples in the neighbourhood are equal to one, so that the effective weight of every sample in the neighbourhood is equal to /8. The calculation of the transfer function is in the z-domain. A horizontal delay of one pixel is represented by z x, and a vertical delay of one line is represented by z y. The calculation of the vertical frequency response is on a frame basis, so a line memory is represented by a delay of z 2 y. It is assumed that if there are differences between both implementations of the noise filter, they will be most prominent in low contrast areas, i.e. when the filter is active. Clearly there will be no difference in high contrast (detailed) areas, i.e. when the filter is inactive. A more difficult part to analyse is the filter s behaviour in medium contrast areas, i.e. when the filter is partially active, since this will involve the non linear characteristic of the noise filter. This is not analysed in the mathematical evaluation. 4. Original implementation The original noise filter incorporates a line-alternating offset of the line memory, with a four-line cycle time. Because of this four-phase offset, there are four configurations of the filter. Each configuration corresponds with a phase ϕ and is valid during one line. During the next line the configuration corresponding with phase ( ϕ + ) is valid. This means that there are four different filter outputs repeated after four lines (modulo 4; ϕ equals ϕ 4 ).

5 The generic input-output relation for phase ϕ is: Out ϕ A In+ B W ϕ z 2 y Out ( ϕ ) (7) The direct transversal response is determined by A: A -- ( z 2 8 x + + z2 x ) (8) z y 2 The recursive response from the previous line Out ( ϕ ) is determined by B and W ϕ, where B represents the recursive neighbourhood and represents the line-alternating offset: W ϕ B -- ( z 8 8 x + z 4 z4 z8 x ) x x (9) W W z x W 2 W z x () The resulting transfer function is: Out ϕ In A( + z 2 y BW ϕ + z 4 y B 2 W ϕ W ( ϕ ) + z 6 y B W ϕ W ( ϕ ) W ( ϕ 2) ) z 8 y B 4 W W W 2 W () Since W W W 2 W, equation () becomes: Out ϕ In A( + z 2 y BW ϕ + z 4 y B 2 W ϕ W ( ϕ ) + z 6 y B W ϕ W ( ϕ ) W ( ϕ 2) ) z 8 y B 4 (2) For a plot of the frequency response see figure. 4.2 Subsampled implementation The subsampled noise filter does not incorporate the line-alternating offset. However, due to the subsampling with a factor of four there is a pixel-alternating output, so each horizontal pixel position p (modulo 4) has a different output. The effect of the filtering, subsampling and interpolation (the block subsampling in figure ) is represented by S ( p, p' ). Each recursive sample on position p (rec_sam p ) that is a result of the filtering, subsampling and interpolation, is calculated out of the four outputs from the previous line Out p' combined with the linear interpolation in (two samples), a total of eight samples (see figure 2): S ( p, p' ) rec_sam p ( S ( p, ) Out + S ( p, ) Out + S ( p, 2) Out 2 + S ( p, ) Out ) The distance between samples in the recursive neighbourhood B and the subsampling factor is equal, so all samples rec_sam p in B are from the same position p. ()

6 phase phase phase 2 phase Figure. Frequency response of phase,, 2 & of original filter. a a. The horizontal and vertical frequencies and are normalized to the sample frequency. The input-output relation for position p is (with all recursive samples on the same position p): Out p A In+ B z 2 y ( S ( p, ) Out + S ( p, ) Out + S ( p, 2) Out 2 + S ( p, ) Out ) (4) Neighbourhoods A and B are the same as in the original filter. The filtering, subsampling and interpolation is represented by : S (, ) S ( 2, ) -----z z z 2 5 S ( p, p' ) S (, ) S (, ) -----z z z z 5 (5) S (, ) S ( 2, ) -----z z z z 7 S (, ) S (, ) -----z z z 2 7 (6)

7 7 S ( 2, ) 7 S ( 22, ) -----z z S (, ) 5 S ( 2, ) z z z -----z z 7 S ( 2, ) 7 S ( 2, ) 5 S (, ) 5 S (, ) -----z z -----z z z z z 4 (7) (8) The resulting transfer function is too complex to include in this paper (for a plot of the frequency response see figure 4). position position position 2 position Figure 4. Frequency response of position,, 2 & of subsampled filter. 4. Comparison of both filters A comparison of the frequency responses of the original and the subsampled filters shows that the main difference lies in the higher horizontal frequency region. In an experimental setup (real-time evaluation of simulations of both filters), it is just possible to see a minor difference between both filtered outputs,

8 but it is not evident which of both filters performs best. The subsampled version shows a little more noise reduction at the cost of a little more blurring of the image content. (Both filters compared with same Th.) The difference can be explained with the additional low pass filtering due to subsampling, as is also shown in the frequency responses. The difference is so small that when Th of the subsampled version is lowered by one LSB (the video signal is sampled at eight bits precision) the situation is reversed, so that the subsampled version now shows a little less noise reduction and somewhat reduced blurring. 5. Conclusions and recommendations Subsampling of the noise filter as described in [, 2] with a factor of four does not degrade the performance of the filter significantly. The outcome of the mathematical evaluation is not in contradiction with the real-time evaluation of the simulations. This positive outcome of the research encourages further studies of a completely analogue implementation of the spatial sigma noise reduction filter for low-cost applications. The subsampling of the memory elements is not limited to line memories in spatial noise filters. The application of the principle of subsampling can be extended to temporal noise filters, which can also be attractive for slightly more expensive solutions than applications with spatial filtering only. A field memory can be subsampled in both horizontal and vertical direction, so the achievable memory reduction is higher. Research on severely subsampled temporal noise filters is started recently and will be reported upon in future papers. 6. References [] G. de Haan, T.G. Kwaaitaal-Spassova and O.A. Ojo, Automatic 2-D and -D noise filtering for high-quality television receivers. Proceedings of the 7 th International Workshop on HDTV, Torino, October 994. [2] G. de Haan, T.G. Kwaaitaal-Spassova, M. Larragy and O.A. Ojo, Single-chip TV noise reduction. ICCE Chicago, 995. [] J.S. Lee, Digital image smoothing and the sigma filter. Computer Vision Graphics and Image Processing, 24, 98, pp [4] R. Lüder, Method for noise reduction in television or video signals (original in german), European Patent Application no A2, priority date [5] D.C.C. Wang, A.H. Vagnucci and C.C. Li, Gradient inverse weigthed smoothing scheme and the evaluation of its performance. Computer Graphics and Image Processing, Vol 5, 98, pp