LOSSLESS COMPRESSION OF BAYER MASK IMAGES USING AN OPTIMAL VECTOR PREDICTION TECHNIQUE

Transcription

1 1th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, Septeber -8, 2006, copyright by EUASIP LOSSLESS COMPESSION OF AYE MASK IMAES USIN AN OPTIMAL VECTO PEDICTION TECHNIQUE Stefano Andriani, iancarlo Calvagno and Daniele Menon Dept of Inforation Engineering, University of Padova via radenigo 6/b, 100, Padova, Italy phone: , fax: , eail: {stefanoandriani,calvagno,danieleenon}@deiunipdit web: wwwdeiunipdit ASTACT In this paper a lossless copression technique for ayer pattern iages is presented The coon way to save these iages was to colour reconstruct the and then code the full resolution iages using one of the lossless or lossy ethods This solution is useful to show the captured iages at once, but it is not convenient for efficient source coding In fact, the resulting full colour iage is three ties greater than the ayer pattern iage and the copression algoriths are not able to reove the correlations introduced by the reconstruction algorith However, the ayer pattern iages present new probles for the coding step In fact, adjacent pixels belong to different colour bands ixing up different kinds of correlations In this paper we present a lossless copression procedure based on an optial vector predictor, where the ayer pattern is divided into non-overlapped 2 2 blocks, each of the predicted as a vector We show that this solution is able to exploit the existing correlation giving a good iproveent of the copression ratio with respect to other lossless copression techniques, eg, JPE-LS 1 INTODUCTION Most digital caeras produce colour iages using a single CCD sensor provided by a colour filter array (CFA) In this way, adjacent pixels capture the light intensity value of different colour bands, and a full colour iage is obtained by a colour interpolation step which reconstructs the issing values The ayer pattern, presented by ayer in [1], is the ost popular and used colour filter array (CFA) It uses a rectangular grid for the red and blue bands and a quincunx grid for the green band, as shown in Fig 1 The choice of capturing a nuber of green pixels twice as high as the red and the blue bands is justified by the fact that the Huan Visual Syste (HVS) places ore ephasis on the green rather then on the red and blue coponents The coon way to anage the ayer pattern iages is to colour reconstruct the, applying autoatic white balancing and other colour corrections, and then copress the resulting iages This workflow is effective for ost digital caera users, but the professional photographers deand for custo post-processing of the captured iages To achieve this requireent, the professional digital photo caeras allow for saving the raw sensor data without any postprocessing step This option perits the advanced user to apply high-coplexity high-quality deosaicing algoriths during a post-processing step Until now, each captured iage has been saved in a AW forat without any copres- Figure 1: ayer pattern used in the paper sion step, and only Nikon proposes a visually, but not nuerically, lossless coder called NEF (Nikon Electronic Forat) This solution satisfies the user requireent, but liits the nuber of pictures which could be saved into the flash eory card of the caera esides, if we consider the cost of this kind of eory, the need of a good lossless copression technique is gaining iportance day by day In literature, the first works on this issue were developed by S Lee and A Ortega in [2] and CC Koh and SK Mitra in [] The first paper proposes that the copression step is placed before the colour reconstruction algorith The authors propose an iage transforation algorith to reduce the existing redundancy in a CFA iage and then they code the transfored iage using JPE Mitra et al propose new iage transforations to be applied to the ayer iage before the JPE copression step They code the red and blue bands without any transforation because these bands have a rectangular array suitable for JPE The transforations are applied only on the green band where a quincunx sapling is used They propose two ethods: the first uses a diaond filter with a 2-D ipulse response and then the data is separated into odd and even coponents independently coded In the second ethod the coluns of the quincunx array are collapsed into a copact array This operation creates false high frequencies in both the horizontal and vertical directions To reduce this effect the quincunx data has been thought as two interlaced fraes of a scene, so through the process of deinterlacing a sooth iage is obtained Unfortunately, these approaches introduce soe loss A lossless copression algorith for the color osaic iages is proposed by N Zhang and X Wu in [] They use the lifting integer wavelet to decorrelate the osaic data both in spatial and spectral doains Then the integer transfor

2 1th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, Septeber -8, 2006, copyright by EUASIP y y y 2 y 1 y H r r r r b b b b r r r r b b b b r r r r b b b b r r r r b b b b L H/2 r r r r r r r r r r r r r r r r L/2 y+1 Figure : Prediction blocks x x x x 2 x 1 x x+1 x+2 x+ x+ Figure 2: Manhattan distance (MD) of the neighbour pixels to the current one coefficients are coded by a siple context-based olob- ice coding schee Our approach uses a DPCM echanis to decorrelate the osaic colour iages, where the optial prediction is estiated using the causal adjacent pixels given by a raster scan order However, the ayer pattern iages are not considered as grayscale iages, but as non-overlapped block iages where each block contains two green, one red and one blue saples For each prediction step we estiate an iage block (ie, four iage adjacent saples) using the optial vector prediction theory In this way, our predictor is able to exploit both the spatial and spectral correlation and uses the to iprove the prediction perforance (reducing the prediction error entropy) The paper is organized as follows Section 2 presents the proposed coding schee, and its experiental results are shown in Section In Section we report the conclusion of this work 2 POPOSED ALOITHMS In one-diension the -order one-step linear predictor of a signal x at tie n is given by ˆx(n) = i=1 w i x(n i) where x(n i) are past observations of the signal, and w i are the prediction coefficients The predictor is called optial if the prediction coefficients w i iniize the energy of the prediction error: e(n) = x(n) ˆx(n) = x(n) i=1 w i x(n i) Fro the orthogonality principle, this condition is verified if and only if the prediction error is orthogonal to the past signal observations 21 Optial Scalar Prediction (OSP) For iage prediction we ust address a two-diensional proble The first issue is to define the set of the neighbour (causal) pixels which define the previous saples of the optial predictor Using a raster scan order, one solution to this proble could be given by the set of pixels having Manhattan distance saller than a fixed threshold T s (see Fig 2) Let p(x,y) be the pixel value at position (x,y), then the prediction set P T s x,y of this pixel is given by: where P T s x,y = { p(x k,y j) : (k, j) A k, j k + j T s } A k, j = {k Z, j N 0 : (k Z, j > 0) (k > 0, j = 0)} A k, j, N 0 and Z are the sets of the previous causal pixel, of the natural and of the integer nubers, respectively The threshold value T s fixes the predictor order, so if it is high the prediction set P T s x,y has an high cardinality, and we have an high-order predictor This solution works well if we predict a natural iage where there are sooth changes of luinance value, but for synthetic iages this solution does not work well because the luinance could change in strange (and unpredictable) ways On the other hand, if the threshold is too low, we obtain a low order predictor which works well on the edges, but not so well on sooth region To predict the pixel p(x,y) we use the previous coded pixels according to the scan order introduced in Fig 2 The optial prediction coefficients w x,y,k are adaptively coputed solving the linear syste x,y w x,y = 0,x,y, (1) where x,y is an estiate of the autocorrelation atrix, and 0,x,y is a correlation vector At the end, the predicted pixel ˆp(x, y) is coputed according to ˆp(x,y) = k=1 w x,y,k p x,y,k (2) where p x,y,k P T s x,y To copute the optial predictor, the autocorrelation atrix is needed We estiate it using the covariance ethod taking into account only the actual coded pixels In this way, the estiate of the autocorrelation atrix for the pixel at position (x, y) is given by x,y = y 1 L 1 j=0 k=0 and the vector 0,x,y by 0,x,y = y 1 L 1 j=0 k=0 α x k y j x 1 A k, j + k=0 α x k y j x 1 A 0,k, j + k=0 α x k A k,y, () α x k A 0,k,y, ()

3 1th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, Septeber -8, 2006, copyright by EUASIP where L is the width of the iage, and α is an epirical weight introduced to decrease the influence of the farthest pixels fro the current position (x, y) To build up the atrix A x,y and the vector A 0,x,y we define the vector p x,y as p x,y = p x,y,1 p x,y,2 p x,y,, () where the coponents p x,y,1,, p x,y, belong to the prediction set P T s x,y, and is the order of the predictor, ie, the cardinality of P T s x,y Fro p x,y, the atrix A x,y and the vector A 0,x,y are coputed according to A x,y = p x,y p t x,y p 2 x,y,1 p x,y,1 p x,y,2 p x,y,1 p x,y, p x,y,1 p x,y,2 p 2 x,y,2 p x,y,2 p x,y, =, p x,y,1 p x,y, p x,y,2 p x,y, p 2 x,y, A 0,x,y = p(x,y)p x,y = p(x,y)p x,y,1 p(x,y)p x,y,2 p(x,y)p x,y,, where A x,y is a syetric positive definite atrix, but not a Toeplitz atrix 22 Optial Vector Prediction (OVP) To extend the linear prediction to the vector case we divide the ayer pattern iage into non-overlapping 2 2 blocks considering the pixels belonging to each block as the coponents of a single pixel vector The resulting block iage has a diension equal to L/2 H/2 where L and H are the width and the height of the original iage, respectively (see Fig ) Each block is coposed by two green, one red and one blue pixels To distinguish the two green pixels we denote with r and b the green saples which belong, respectively, to the odd and to the even rows of the ayer pattern reported in Fig 1 Let b (x,y) be the vector of the block at position (x,y) b (x,y) = [ ] x,y r,x,y b,x,y x,y, (6) where the coordinate (x, y) refers to the block and not to the pixel position To predict this vector we use the causal blocks having Manhattan distance saller or equal than a fixed threshold T v Accordingly, the prediction set is given by P T v x,y = { b (x k,y j) : (k, j) A k, j k + j T } where A k, j is the set of the causal blocks in raster scan order In this paper, we consider a single threshold T v = 2 having P x,y 2 as prediction sets with a block cardinality equal to six To predict the pixel vector p(x, y) we use the previous coded pixel vectors according to the scan order introduced in Fig 2 To estiate the autocorrelation atrix we write the vector of the causal block as p x,y = [ bx,y,1 ] t bx,y,2 bx,y,, (7) where is the cardinality of the prediction set Fro p x,y the two atrices A x,y and A 0,x,y are coputed according to A x,y = p x,y p t x,y (8) Ā x,y,1,1 Ā x,y,1,2 Ā x,y,1, Ā x,y,2,1 Ā x,y,2,2 Ā x,y,2, =, Ā x,y,,1 Ā x,y,,2 Ā x,y,, A 0,x,y = [ b (x,y)p x,y ] t (9) = [ x,y p x,y r,x,y p x,y b,x,y p x,y x,y p x,y ], where A x,y is a block syetric atrix, but not block Toeplitz Each block Ā x,y,i, j represents the existing spatial and spectral correlation between the two pixel vectors b (x,y,i) and b (x,y, j) which belong to the prediction set The estiate of the autocorrelation atrix x,y and of atrix 0,x,y is ade as in () and (), respectively In optial vector theory, the prediction block coefficients w x,y,k are atrices which are adaptively coputed solving the linear systes x,y W x,y = 0,x,y, (10) where W x,y = w x,y,1 w x,y,2 w x,y,, and the predicted pixel vectorpˆ b (x,y) is coputed according to ˆ p(x,y) = k=1 w x,y,k bx,y,k (11) To suarize, for each prediction the following steps have to be perfored: 1 update the vector p x,y (7), and the two atrices A x,y (??) and A 0,x,y (9); 2 estiate the autocorrelation atrix x,y (), and the correlation atrix 0,x,y (); invert the autocorrelation atrix x,y to obtain the prediction coefficient vector W x,y = ( x,y ) 1 0,x,y ; (12) calculate the predicted pixel vector pˆ b (x,y) by (11) To reduce the coplexity of the second step we use the atrix update ethod introduced by [], where the high coputational cost proble is turned to a high eory requireent proble To calculate the inverse atrix of x,y we use the Cholesky factorization obtaining a coputational cost order equal to O(() /6)

4 1th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, Septeber -8, 2006, copyright by EUASIP JPE LS JPE LS JPE LS Storage Figure : JPE-LS applied on the three colour coponents obtained by a ayer pattern splitting The pixel vector value p x,y is then coded through an arithetic binary coder in bit-plane ode starting fro the MS down to the LS, and the zero s probability is coputed odelling the prediction error e x,y = p x,y ˆ p x,y by a odified Student s distribution centered on the predicted pixel value [] Since each frae is extended to a constant value outside the iage boundaries, the decoder can replicate the coder operations without requiring that the predictor coefficients are sent as side-inforation ESULTS We tested our algorith using the reference Kodak set iages, and the three test iages called Woan, ike and Monarch These iages have a 2-bit colour representation, and were sapled according to the ayer pattern shown in Fig 1 We copare our algorith with respect to the standard lossless coder JPE-LS presented in [6] and ipleented by HP Laboratories 1 We tested JPE-LS on ayer pattern iages in two ways: 1 directly applying JPE-LS to the ayer pattern iages; 2 applying JPE-LS to the three colour coponents obtained splitting the ayer pattern iages, as show in Fig Fro the results reported in Table 1 we can conclude that applying the JPE-LS coder directly to the ayer pattern iages is not a good solution In fact, JPE-LS is provided with a very siple but effective predictor called MED (Median Edge Detector) This predictor checks for the presence of vertical or horizontal edges predicting along the when they occur, otherwise it estiates the current pixel as the ean value of the adjacent pixels In the ayer pattern iages (PI) adjacent pixels always present abrupt intensity changes due to the fact that they belong to different colour coponents For this reason, the MED predictor is not able to detect edges, becoing a very siple and ineffective obile average filter To obtain the results reported in the second colun of Table 1 we split the ayer pattern iages into three different iages representing the three colour coponents (see Fig ) In this way, we obtain a copression gain, with respect to the first solution, roughly equal to 07 bit per pixel The sae gain (refer to the third colun of Table 1) is obtained applying the lossless ode of JPE2000 directly to 1 theory/loco/ the ayer pattern iages As said in [], this result is due to the reversible - integer discrete wavelet transfor which de-correlates the osaic data both in the spatial and spectral doains For the optial scalar predictor we used two different thresholds T s = {2,} and the corresponding prediction sets P x,y {2,} have cardinality equal to 6 and 12, respectively On the other hand, for the optial vector predictor we fixed the threshold equal to 2 (T v = 2) obtaining a prediction set P x,y 2 having a block cardinality equal to 6 where each block consists of four pixels The thresholds for the OSP algorith are selected considering that with T s = 2 the scalar and the vectorial predictor have the sae threshold value, and with T s = the two coders have the sae coplexity The OVP coder works better than the OSP coders (T s = 2 and T s = ) because it organizes the pixels into a well-defined structure where both the spatial and the spectral correlations are exploited On the other hand, in the OSP coders, the raster scan over a ayer pattern iage iplies a repetitive exchange between the position of the spectral and the spatial correlation inside the prediction vector p x,y and then inside the estiate of the autocorrelation atrix used to calculate the prediction weights For this reason, the structure of the OVP coder leads to higher copression ratios However, it is worth to note that the OSP coders applied to the ayer pattern iages achieve better results with respect to appling independently the sae algoriths to the ayer splitted iages This results lead us to conclude that also the OSP coders exploits both spectral and spatial correlations, but not as efficiently as the OVP coder Finally, we copare the proposed optial vector prediction algorith with respect to other lossless coders presented in literature, naely: JPE-LS independently applied on the three colour coponents obtained splitting the ayer data, JPE2000 applied on the ayer pattern iage, and the coder presented by Zhang and Wu in [] The perforance iproveent of the proposed algorith is about 0 bpp with respect to the standard lossless coder JPE-LS, and 02 bpp with respect to the Zhang and Wu algorith However, the coplexity of the proposed algorith is too high to allow for a real-tie ipleentation This high coputational cost is due to two ain reasons: the first is that we have to invert a 2 2 autocorrelation atrix x,y for each block prediction; the second is due to the estiate of the zero probability by nuerical integration of the odified Student distribution CONCLUSION In this paper we propose a new lossless copression algorith based upon the optial vector prediction theory The ayer pattern is divided into 2 2 non-overlapping blocks containing two green, one red and one blue pixels Each block is then predicted exploiting the spatial and the spectral correlations existing between pixels which belong to the sae block and to the adjacent causal blocks The obtained results show that the proposed ethod is effective, but its coplexity is too high to allow for a real-tie on-board hardware ipleentation, so the captured iages have to be saved in AW forat on the flash eory of the caera and then copressed in a post-processing step

5 1th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, Septeber -8, 2006, copyright by EUASIP JPE-LS Optial Spatial Predictor Optial Iage ayer Pattern JPE2000 Zhang and Wu T s = 2 T s = Vector Iage (PI) Splitted [7] [] PI PI Predictor Woan ike Monarch KD KD KD KD KD KD Average Table 1: Coparison between different ayer pattern lossless coders EFEENCES [1] ryce E ayer, Color Iaging Array, US patent,,971,06, 1976 [2] S Lee and A Ortega, A novel approach of iage copression in digital caeras with a ayer color filter array, in Proc IEEE International Conference on Iage Processing (ICIP 2001), Thessaloniki, reece, Oct 2001, pp 82 8 [] CC Koh and SK Mitra, Copression of ayer color filter array data, in Proc IEEE International Conference on Iage Processing (ICIP 200), arcellona, Spain, Sep 200 [] N Zhang and X Wu, Lossless copression of color osaic iages, in Proc IEEE International Conference on Iage Processing (ICIP 200), Singapore, Oct 200, pp [] Meyer and PE Tischer, LICAWLS - rey Level Iage Copression y Adaptive Weighted Least Squares, in Proc Data Copression Conference (DCC 2001), Snowbird, Utah, USA, Mar 2001, p0 [6] MJ Weinberger and Seroussi and Sapiro, The LOCO-I Lossless Iage Copression Algorith: Principles and Standardization into JPE-LS, IEEE Trans Iage Processing, vol 9, no 8, pp , Aug 2000 [7] DS Tauban and MW Marcellin, JPE2000 Iage copression, fundaentals, standards and practice, Kluwer Acadeic Publishers, 2002