Multi-Directional Upsampling for Multiple Channel Images

Transcription

1 , pp Multi-Directional Upsampling for Multiple Channel Images wanggil Jeon Department of Embedded Systems Engineering, Incheon National University 119 Academy-ro, Yeonsu-gu, Incheon , Korea Abstract In this paper, we propose a multi direction color interpolation method for multiple channel images. The proposed gradient udgment method silences the mediocre color interpolation artifacts. We adopted various weights from different directional gradients for conducting adaptive directional interpolation. The proposed weights are computed using correlation and intensity differences. The obtained weights are used for determining final interpolated results. Simulation results prove that the proposed method gives good performance in test set. Keywords: ayer Color filter array, color image upsampling, multiple channels 1. Introduction In order to acquire full color images, at least three channels are requested at each pixel location. However, image sensors acquire only one intensity from color filter array (CFA) to reduce the cost [1]. Therefore it is requested to restore missing color information and this process is called demosaicking [2-12]. In this paper, we present a new multi direction color interpolation method for multiple channel images [13-21]. Figure 1 shows the structures color image acquiring system. First preceding one uses three image sensors while the later one uses single image sensor. Lens Lens CCD CCD (a) Figure 1. (a) asic Structure of Three Image Sensors for Color Image Acquisition, (b) asic Structure of Single Image Sensor for Color Image Acquisition (b) ISSN: IJSEIA Copyright c 2014 SESC

2 i-1 i-1 i i i+1 i+1 i+2 i Figure 2. Adopted ayer Color Filter Array for our Multi-Directional Upsampling System The ayer pattern CFA is shown in Figure 2. As one can see, each of sensors is arranged to represent a specific color such as red, green and blue. The reason each component shows particular color is that, CFA filters the light by wavelength range [22-25]. The ayer CFA is named for its inventor. As human visual system is more sensitive to green color, green components have two pixels out of four pixels in a pair. The rest of this paper is arranged as follows. Section 2 describes proposed color image upsampling method. Section 3 shows simulation results. Section 4 provides conclusion remarks. 2. Proposed Method The proposed algorithm performs the interpolation as follows. The first step is green channel is interpolated. The second step is green channel update. The third channel is red and blue channel interpolation. We assume P is original pixel at () position at ayer pattern CFA pattern. Then the color different δ θ is calculated as Eq. (1), P P P i, i, P i, i,, (1) where θ is interpolation direction which can be horizontal or vertical directions. The first row of Eq. (1) is for green pixel position P, and the second row is for red (or blue) pixel position. The color different is used for calculating,, (2) i, i, 1 i, Copyright c 2014 SESC

3 where ω θ represent gradient in horizontal direction. Similarly, for vertical direction, Eq. (2) can be modified as, (3) i, i 1, i 1, For the directional interpolation, we use two sets of weights. The first set has two weights which are horizontal and vertical weights. The second set is four gradients, and they are up, down, left, and right. The gradient of the same color plane can be calculated as: Z Z i, i, 1 i, 1, (4) In the same manner, vertical is calculated as Z Z i, i 1, i 1,. (5) Let us assume represents gradient of the same color plane in each direction θ. The directional weight in horizontal direction is calculated as, / i, i m, l i m, l m 2 l 2 m 1 l 1, (6) 2 where τ is a small number. In the same manner, vertical direction is calculated. The other four weights up, down, left, and right are computed as, w i, i m, l i m, l 1 / 1 m 2 l 1 m 2 l 1. (7) Note that, Eq. (7) is for up case. Other three cases such as down, left, and right are calculated accordingly. Now, the refined final green channel value is computed as, F, i, i, i, P Q (8) where Q is calculated as { up, dow n, left, right } { up, dow n, left, right } i, i, Q q k (1 k ). i, i, w w q (9) Copyright c 2014 SESC 161

4 Here, q is computed as, q { h o rizo n ta l, v e rtic a l } d { h o rizo n ta l, v e rtic a l } w i, i, w. (10) Now, the red and the blue channel interpolation is operated as F F ( Q ), i. i. i. i. F F ( Q ), i. i. i. i. (11) where is matrix multiplication and H is interpolation filter which is defined as i. M, ( 2 )( w w w w ) i. i. i. i. (12) where θ 1, θ 2, θ 3, and θ 4 stand for upper left, upper right, down left, and down right direction, respectively. Matrix M is defined as Eq. (13). M w 0 w 0 0 i. i w 0 w 0 w 0 w w 0 w 0 w 0 w w 0 w i. i i. i. i. i i. i. i. i.. (13) 162 Copyright c 2014 SESC

5 3. Simulation esults Figure 3. Employed Dataset: 24 Kodak Images In this section, we compared the presented method with recently proposed methods. We use 24 Kodak imageset for comparison (Figure 3). The proposed method is compared with some conventional methods such as directional linear minimum mean square-error estimation (DLMMSE), least-squares luma-chorma demultiplexing (LSLCD), effective demosaicking algorithm based on edge property (EDAEP), edge strength filter (ESF), multi-scale gradient (MS), and voting-based directional interpolation (VDI). We used three obective performance metrics, CPSN, S- CIELA ΔE *, and FSIM. Figures 4-6 show results of each metrics. Subective performance comparison is provided in Figsure 7-8. From all results provided above, it can be concluded that the proposed method gives the best performance. Figure 4. CPSN performance comparison Copyright c 2014 SESC 163

6 Figure 5. S-CIELA ΔE * performance comparison Figure 6. FSIM performance comparison 164 Copyright c 2014 SESC

7 (a) (b) (c) (d) (e) (f) Copyright c 2014 SESC 165

8 (g) Figure 7. FSIM performance comparison on #5 Kodak image: (a) original image, (b) DLMMSE, (c) LSLCD, (d) ESF, (e) EDAEP, (f) MS, (g) VDI, and (h) Proposed method (h) (a) (b) (c) (d) 166 Copyright c 2014 SESC

9 (e) (f) (g) Figure 8. FSIM performance comparison on #7 Kodak image: (a) original image, (b) DLMMSE, (c) LSLCD, (d) ESF, (e) EDAEP, (f) MS, (g) VDI, and (h) Proposed method 4. Conclusions This paper proposes a multi direction color interpolation method for multiple channel images. The proposed gradient evaluation method suppresses the mediocre color upsampling artifacts. Various weights were obtained during the interpolation. The proposed weights were calculated suing intensity differences and correlation. Experimental results confirm that the presented method yields better performance than conventional methods in various performance metrics. Acknowledgements This research was supported by asic Science esearch Program through the National esearch Foundation of Korea(NF) funded by the Ministry of Science, ICT and Future Planning( ). (h) Copyright c 2014 SESC 167

10 eferences [1]. ayer, Color imaging array, U.S. Patent , (1976) July. [2] Yong Fang, Analysis on crossover probability estimation using LDPC syndrome, Science China: Inf. Sci., vol. 54, no. 9, (2011), pp [3] Yong Fang, Joint source-channel estimation using accumulated LDPC syndrome, IEEE Commun. Lett., vol. 14, no. 11, (2010), pp [4] Yong Fang, EEC-based length coding of variable-length data blocks, IEEE Trans. Circ. Syst. Video Technol., vol. 20, no. 10, (2010), pp [5] Y. Fang, Distribution of distributed arithmetic codewords for equiprobable binary sources, IEEE Signal Process. Lett, vol. 16, no. 12, (2009), pp [6] Y. Fang, Crossover probability estimation using mean-intrinsic-ll of LDPC syndrome, IEEE Commun. Lett., vol. 13, no. 9, (2009), pp [7] Y. Fang,. Jeon, and J. Jeong, State-information-assisting EEC, IEEE Signal Process. Lett., vol. 16, no. 4, (2009), pp [8] Y. Fang and J. Jeong, Correlation parameter estimation for LDPC-based Slepian-Wolf coding, IEEE Commun. Lett., vol. 13, no. 1, (2009), pp [9] Y. Fang, J. Jeong, et al, Error detection based on M types, Science China: Inf. Sci., vol. 51, no. 4, (2008), pp [10] Y. Fang, J. Jeong, et al, obust video transmission using block-interleaved error resilient entropy coding, SPIE Optical Engineer., vol. 46, no. 12, (2007), pp [11] Y. Fang, C. Wu, et al, Video transmission using advanced partial backward decodable bit stream, Elsevier J. Visual Commun. & Image epresent., vol. 18, no. 2, (2007), pp [12] Y. Fang, C. Wu, et al, i-directional error resilient entropy coding (EEC), EUASIP Signal Process.: Image Commun., vol. 21, no. 8, (2006), pp [13]. Jeon, M. Anisett V. ellandi and J. Jeong, Fuzzy ule-ased Edge-estoration Algorithm in HDTV Interlaced Sequences, IEEE Transactions on Consumer Electronics, (2007). [14]. Jeon, M. Anisett V. elland E. Damiani and J. Jeong, ough Sets- Assisted Subfield Optimization for Alternating Current Plasma Display Panel, IEEE Transactions on Consumer Electronics, (2007). [15]. Jeon, M. Anisett V. elland E. Damian J. Jeong Fuzzy Weighted Approach to Improve Visual Quality of Edge-ased Filtering IEEE Transactions on Consumer Electronics, (2007). [16]. Jeon, M. Anisett D. Kim, V. elland E. Damian J. Jeong, Fuzzy rough sets hybrid scheme for motion and scene complexity adaptive deinterlacing, IMAVIS Image and Vision Computing Journal, vol. 27, no. 4, (2009), pp [17]. Jeon, M. Anisett V. elland E. Damiani and J. Jeong Designing of a type-2 fuzzy logic filter for improving edge-preserving restoration of interlaced-to-progressive conversion, Information Sciences, Elsevier, (2009). [18]. Jeon, M. Anisett J. Lee, V. elland E. Damian J. Jeong, Concept of Linguistic Variable-ased Fuzzy Ensemble Approach: Application to Interlaced HDTV Sequences, IEEE Transactions on Fuzzy Systems, IEEE, (2009). [19]. Jeon, S.J. Park, Y. Fang, M. Anisett V. elland E. Damian and J. Jeong, Specification of efficient block matching scheme for motion estimation in video compression, SPIE Optical Engineering, vol. 48, no. 12, (2009). [20]. Jeon, M. Y. Jung, M. Anisett V. elland E. Damian and J. Jeong, Specification of the eometric egularity Model for Fuzzy If-Then ule-ased Deinterlacing, IEEE Journal of Display Technology, vol. 6, no. 4, (2010). [21] wanggil Jeon, Marco Anisett Seokhoon Kang A ank-ordered Marginal Filter for Deinterlacing, (2013). [22] N. naneswara ao, T. Sravani and V. Viaya Kumar, OCM: Optimal Cost egion Matching Similarity Measure for egion ased Image etrieval, IJMUE vol.9, no.4, pp , (2014). [23] L. Shen and Y. Niu, lind Color Image Fusion ased on the Optimal Multi-obective Particle Swarm Optimization, IJMUE, vol.2, no.3, (2007), pp [24].-H. Kang, A eview on Image and Video processing, IJMUE, vol.2, no.2, (2007), pp [25] S.-S. Yoo, Y.-T. Kim, S.-J. Youk, J.-H. Kim And.-K. Lee, Adaptive-inning Color Histogram for Image Information etrieval, IJMUE, vol. 1, no.4, (2006), pp Copyright c 2014 SESC

11 Authors wanggil Jeon is currently an assistant professor with the Department of Embedded Systems Engineering, Incheon National University, Incheon, Korea. His research interests fall under the umbrella of image processing, particularly image compression, motion estimation, demosaicking, and image enhancement as well as computational intelligence such as fuzzy and rough sets theories. Copyright c 2014 SESC 169

12 170 Copyright c 2014 SESC