SoC Design of An Auto-Focus Driving Image Signal Processor for Mobile Camera Applications

Transcription

1 10 SoC Design of An Auto-Focus Driving Image Signal Processor for Mobile Camera Applications Sung-Min Sohn, Sung-Hyun Yang, Sang-Wook Kim, Kug-Hyun Baek and Woo-Hyun Paik Abstract This paper presents a camera System on a Chip (SoC) solution to capture focused images for mobile camera phones. In this work, the Image Signal Processor (ISP) includes both a built-in Auto Focus (AF) control block and AF lens driver to fully implement AF with fast and accurate performance. Additionally, AF lens module has been formed by Voice Coil Motor (VCM) type for compact size. The proposed ISP with AF lens driver, AF_ISP, can meet constraints such as small size, low cost, and low power consumption for mobile camera applications 1. Index Terms Image Signal Processor (ISP), Auto Focus (AF), Camera system, Voice Coil Motor (VCM), AF lens driver I. Introduction Recently, camera systems are widely used in mobile applications such as handheld phone, Personal Digital Assistants (PDA), and laptop. And the mobile camera phone market is continuing to increase explosively especially with integration of high-end camera functions. Although the mobile camera application is different from a Digital Still Camera (DSC) in many aspects, the mobile camera phones tend to have all the qualities of a DSC [1]-[4]. Conventional camera phones have used pan focus or manual focusing techniques in which users would adjust the location of lens with their own hand for producing a focused image. The pan focus technique allows for all scenes, background and foreground, to be focused instantly for a snap-shot. But these skills are limiting the image quality since it is hard to focus on the object precisely. Therefore, phone manufacturers are developing camera phones with AF, zoom, as well as adopting high resolution sensors to enhance image quality [1]. In image focusing part, AF function can solve above-mentioned problems and provide users high quality images as well as ease of usage [2]. In mobile applications, there are several limitations to implement AF function such as power consumption, camera module size, and the requirements of additional circuits like MCU and motor driver, etc. Until recently, the AF scheme of conventional camera system was organized by components on a PCB board. However, this scheme is inappropriate for mobile phone applications, especially when the goal is to create a compact, cost effective and multi-functional mobile phone. This paper describes a one-chip AF camera solution to overcome size and cost limitations of mobile phones. II. Auto Focus Camera System Basically, AF camera module consists of AF lens, image sensor, AF lens driver, and Image Signal Processor (ISP). Fig. 1 (a) shows a conventional configuration for AF operation in mobile phone. Optical signals passing through AF lens are reaching image sensor, which transforms optical signal to electrical. Focus values, frequency components, are generated and AF algorithm finds the state of the focused image in ISP. If there is no ISP or an ISP can t support to generate the focus value, either application processor (AP) or baseband chip generates focus values and performs AF algorithm to search the best state. AF (a) A conventional AF camera configuration 1 This work was supported in part by the LG Innotek, camera module manufacturer. LG Innotek made the 2 mega pixels AF camera module using AF_ISP which is the smallest module size in the world (to date, Nov. 2005). The authors are with System IC division, LG Electronics, Seoul, Korea. ( ssm0625@lge.com) (b) The proposed AF camera configuration Fig. 1. AF mobile phone camera system Manuscript received January 16, /06/$ IEEE

2 S.-M. Sohn et al.: SoC Design of An Auto-Focus Driving Image Signal Processor for Mobile Camera Applications 11 algorithm decides the proper lens position on the basis of the focus values and controls lens driver to move AF lens at relative position. AF solution with conventional configuration has difficulties to make compact camera module and to have good matching characteristics among AF components. The harmonies between an AF step in algorithm and a minimum stroke in lens driver are very important for AF performance, such as accuracy and speed. The proposed AF camera configuration using AF_ISP is shown in Fig. 1 (b). This AF_ISP provides camera module manufacturer with one-chip AF solution to meet constraints such as small size, low cost, and good performance. The AF_ISP contains AF statistics block to generate focus value, AF controller block to perform algorithm, and AF lens driver block to control the position of lens in itself. So AF function can be operated without additional off-chip components. A. Auto Focus (AF) Theory As shown in Fig. 2, where object is at infinity, all parallel rays will be focused to a point referred to as the principal focal point. The distance from the lens to that point is the principal focal length, f, of the lens. If the object is not at infinity but at the finite object position (p), then a lens of focal length (f) form an image at the lens position (q) determined by lens equation (1). Focal length is already known from lens makers and fixed to a specific value. If we can know an object position (p), then the lens position is determined by equation (1) and AF lens driver controls the lens to that position. 1 f p q = (1) Because the auto focusing technique is an essential function in camera, various auto focusing methods have been developed for a long time. Mainly, automatic focusing system has two operation types, active and passive focusing techniques. Active focusing utilizes infrared to measure the distance of object directly, while passive focusing uses image data to measure the distance of object indirectly. Most mobile phone cameras with auto focus function have adopted the passive focusing due to camera module size and cost. This work is based on passive focusing. Fig. 2 shows the focusing algorithm which is based on the fundamental theory, so-called hill climbing algorithm. The AF statistics block extracts frequency components from image signals and generate focus values made by frequency components. B. AF lens driver and Lens In Fig. 2, AF algorithm is searching the best position of lens for well-focused image which has a lot of high frequency components. Once AF algorithm decides the lens position in each image shot, relative outputs control the AF lens driver and moves the lens position. AF lens drivers are generally selected according to the types of motor. For example, a Voice Coil Motor (VCM) type should have a driver with both Digital-to-Analog Converter (DAC) and Current Amplifier. And a stepping motor type should have a driver with both H- bridge driver and Pulse Width Modulator (PWM). Table 1 presents the merits and demerits according to lens and driver types. The stepping motor and driver module have many disadvantages, such as power consumption, module size and cost for mobile application. Recently, piezo motor and driver module has been appeared as the candidate of AF solution of mobile system, because of its small volume size. Even thought it has many merits, low power consumption, fast AF and small size but it is not popular scheme yet due to instability and high voltage supply. Nowadays, mobile phone makers require AF camera unit to be more compact volume and low cost. Accordingly, this work has been based on the VCM type lens module for AF camera system. Table 1 Performance comparison of various lens and drivers Fig. 2. (a) The focusing mechanism (b) The focusing algorithm

3 12 C. Image Data Processing Mobile phone camera users demand more high quality and high resolution image. The image processing of raw data from image sensor enhances the quality of the displayed and printed images. The AF_ISP has the basic image signal processing functions; image sensor interfaces, camera image processing, image enhancements, and various image data formatter. In part of image sensor interface, it performs bayer pattern interpolation, defect pixel compensation, lens shading correction, and offset & knee control. Camera image processing also does the AF, AWB, AE, Gamma correction and color correction. Image enhancements process the edge enhancement, noise reduction, contrast enhancement, and so on. Finally image data formatter converts image data to the format to be optimized for display equipments or the following chip set, such as, application processor or baseband IC. III. SoC Implementation This paper has focused on the one-chip integration of the AF solution which was composed of AF statistics, AF control and AF lens driver for mobile applications. AF lens module has been moved by VCM, which meets constraints of a camera module volume for mobile application. A. AF Control Block (AF filter & MCU) Like most camera phones, this work has adopted the passive focusing method using image data to get the focus information. As shown in Fig. 3, the proposed AF system has AF statistics block, AF control block and AF driver block. The AF statistics block extracts focus values from image data carried by image sensor. The more it increases the high frequency components in scene, the more it has well-focused image. So if it makes use of high frequency components made by high pass filter (HPF), it has a good performance in AF system. In passive focus system using high frequency information, there are some problems. First, focus values are dependant on the characteristics of HPF. If the HPF with unique cut-off frequency generates focus values, the AF system has low performances, speed and accuracy. Second, it is hard to obtain an accurate focus value, in which several objects are located separately on the scene of the image. Third, the AF system has a low speed AF function because a passive focus system is a feed-back operation; it always considers the focus values according to lens position of each image. The proposed AF_ISP has the blocks to solve the abovementioned problems. First, it can reduce the dependency of particular filter characteristics by using multiple HPFs with different cut-off frequency concurrently. As shown in Fig. 5 (b), this system generates the focus values having particular characteristic curve and the proposed AF control uses these focus values to search a peak position. Second, it has specific focusing windows and generates the focus values of each window simultaneously. Because it is easy to get the focus values with each object s distance, this system can have an image with an object to be focused. Third, lens control is operated by the focus values from two HPFs. Because the AF_ISP can control the AF steps adaptively during the hill climbing algorithm, coarse step size for the region far from the peak and fine step size around the peak, it enables AF operation to be fast and accurate. Fig. 4 shows the AF control flow in AF control block which determines the direction and the step of lens by comparing present focus values with former focus values. Once the direction of lens position, to which the focus values increases, is determined, it moves the lens position forward to that direction until the maximal point of focus values has reached. In Fig. 5 (a), a detailed VCM lens module is presented. An image sensor and ISP are placed on bottom area and lens can Fig. 3. AF block diagram Fig. 4. AF control flow

4 S.-M. Sohn et al.: SoC Design of An Auto-Focus Driving Image Signal Processor for Mobile Camera Applications 13 Fig. 6. Architecture of AF lens driver Fig. 5. (a) AF camera module (b) Focus values (c) AF lens driver outputs move within the range of a search area of lens module. Once focus values in Fig 5 (b) are computed and the AF (VCM) driver s output in Fig 5 (c) is determined, the lens can move forward from the bottom area to the peak position of focus values. B. AF lens driver The lens location is determined according to the voltage drop on the VCM. That is, as the voltage across the VCM increases, the AF lens moves apart from the image sensor lengthening the focus distance. During the time interval for reading each frame data, the lens position should be kept at a given location. This means AF (VCM) driver must provide constant current to the VCM for each frame time. Note that the AF (VCM) shows maximum current consumption when the voltage drop across the VCM is Vmax. Several factors should be considered to integrate the AF (VCM) driver with ISP. It is desirable for the AF (VCM) driver to be as small as possible for tiny chip for mobile system. The VCM has an equivalent load resistance of about 25Ω, thus the AF (VCM) driver should be able to drive this low resistance and have the current source/sink capability as high as several hundreds of ma. The power consumption of the AF (VCM) driver dominates in the buffer amplifier. To achieve low power consumption, short-circuit current flowing through the buffer amplifier s output stage should be minimized. Thus, the careful design for the buffer amplifier is required. Fig 6 shows the AF lens driver architecture. It consists of 7-bit DAC and current buffer amplifier. Fig. 7. Buffer Amplifier. ) Digital-to-Analog Converter After considering the system requirements, a 7-bit resistorstring type DAC is applied to our system. Up to 8-bit resolution, the resistor-string type DAC can be easily implemented, and it has a good linearity if the resistor matching is well achieved through a careful layout. In addition to that, resistor-string DAC satisfies other requirements such as robustness at low-speed, low power consumption, and small layout area. Under the supply voltage of 2.8V, if we choose the base resistance of 2kΩ, the dc current of the resistor-string is 2.8V / (128 2kΩ) = µA. The output range of the DAC is set to nearly full VDD to cover the wide dynamic range of the lens and maintain additional operating margins. ) Buffer Amplifier Since the DAC output is applied to the buffer amplifier, the buffer amplifier also has to operate up to the full VDD range. Fig. 7 shows the architecture of the buffer amplifier which is a rail-to-rail input/output amplifier. The input amplifier is a commonly used wide-swing folded cascade operational transconductance amplifier (OTA) and provides the gain of about 70-dB [5]. The output stage is the modified version of [6]. The output amplifier plays two important roles, one is to provide additional gain and the other is to suppress the short circuit current of the post driver. In Fig. 7, vgp and vgn are optimized to achieve minimum short circuit current of the post

5 14 driver. The post driver, MP23 and MN25, is a common-source type class AB output stage exhibiting a large output swing. Miller compensation capacitors are inserted between the input and the output amplifiers [6]. C. Image Data Processing Blocks In addition to the auto focus feature, the AF_ISP also includes complete auto exposure (AE) and auto white balance (AWB) capabilities to assure optimal image quality in any lighting environment. Furthermore, this supports various features such as Bayer pattern interpolation with reduced color aliasing artifacts, defect pixel compensation, lens shading correction, edge-preserved noise reduction, edge enhancement, color space conversion, gamma correction, adaptive histogram equalization, brightness and contrast control, hue and saturation control, image effects, and many other features and modes to enhance image quality and convert the image data format. In Fig 8, some blocks, such as defect pixel compensation, lens shading correction, offset and gain, bayer interpolation and color correction matrix, are included to compensate for the degradation and adjust for the imperfection in image quality caused by characteristics of image sensors or optics. This AF_ISP has essential automatic functions such as AE and AWB as well as AF. The AE function in this system adjusts the integration time and the gains of color components in order to obtain the correct amount of exposure in different lighting conditions. The auto exposure statistics is evaluated under 6 different user-defined windows. The weighted average of the auto exposure statistics is obtained within the defined windows of an image processing pipeline, and the auto exposure algorithm for the integration time and gain is implemented on the built-in MCU. The AWB refers to applying gain to each RGB channel to compensate for lighting conditions with different color temperatures. Each RGB pixel is accumulated within the white pixel region and the AWB gain is derived from the ratio of accumulated values. The blocks of luminance and chrominance processing support the color, brightness and contrast control. Additionally, more advanced hue and saturation control is possible in the color enhancement block. Combined with the AWB block, the coefficients for chrominance generation are determined by using the interpolation method. Fig. 9. (a) The AF camera module using the AF_ISP (b) The die photo of the AF_ISP D. Chip Design Fig 9 (a) shows the manufactured AF camera module that consists of VCM type lens module, CMOS image sensor with 2 mega-pixels, and proposed AF_ISP. The proposed AF_ISP is located on the module board in the state of a stacked-die as shown in Fig 9 (b). The chip was fabricated in a commercial 0.18µm 1P 6M mixed CMOS technology. IV. Experimental Results We verified image quality and AF performance of the manufactured camera system. Fig 10 shows the experimental pictures and table 2 summarizes the measurement results of the on-chip AF (VCM) driver. The output range is 0.05 ~ 2.65V under 2.8V supply. The maximum current drive capability is 150mA when the equivalent load of the VCM is 17-Ω. The settling time for output current is less than 200µs. Table 2. Measurement results of AF lens driver Fig. 8. AF_ISP block diagram

6 S.-M. Sohn et al.: SoC Design of An Auto-Focus Driving Image Signal Processor for Mobile Camera Applications Table 3. Comparison of camera modules 15 describe focused image during AF function. Actually, an AF function is operated within 0.4 ~ 0.8 second and has a good performance. Table 3 shows the results of comparison against other camera modules which have same configurations except for ISP and AF lens driver and this work has contributed to manufacture making the smallest 2-Mega pixels AF camera module. V. Conclusions Fig 10 is the captured images of camera module and measured outputs. Fig 10 (a) shows defocused image before the AF function and (b) shows clearly focused image after AF function. The measured focus values that increase to the peak value as a function of time are plotted in Fig 10 (c). In Fig 10 (d), the wave form is the measured output voltage of the AF (VCM) driver. The wave form shows the trace from the bottom level to describe defocused image to the top level to This paper describes an auto-focus driving image signal processor and the AF solution with it. Phone manufacturers are now demanding mobile camera modules which are not only becoming smaller, but they are becoming more cost effective and low power consumption. To meet above demands, we developed the SoC of an AF driving ISP which has all-in-one AF solution, such as AF statistics, AF control and AF lens driver. In addition, this SoC solution also allows for fast and optimal AF performance of mobile camera applications. Fig. 10. Experimental results: (a) defocused image (b) focused image (c) the measured focus values (d) The measured output voltage of the AF (VCM) driver

7 16 REFERENCES [1] Zhang, X et al, A signal processing system on chip for digital cameras, IECON th Annual Conference of the IEEE vol. 2, pp , Oct [2] Jie He; Rongzhen Zhou; Zhiliang Hong. Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera, IEEE Trans. Consumer Electronics, vol. 49, pp , May [3] Kang-Sun Choi; Jun-Suk Lee; Sung-Jae Ko. New autofocusing technique using the frequency selective weighted median filter for video cameras, IEEE Trans. Consumer Electronics, vol. 45, pp , Aug [4] Lionel J. D Luna; Kenneth A. Parulski. A systems approach to custom VLSI for a digital color imaging system, IEEE J. Solid-State Circuit, vol. 26, no. 5, pp , May [5] R. J. Baker, H. W. Li, and D. E. Boyce, CMOS Circuit Design, Layout, and Simulation, Ch. 25, IEEE Press, New York, [6] M. D. Pardoen and M. G. Degrauwe, A rail-to-rail input/output CMOS power amplifier, in Proc. IEEE Conf. Custom Integrated Circuits, 1989, pp. 25.5/1-25.5/4. Sung-Min Sohn received the B.S degree and the M.S degree in school of electrical engineering from Korea University, Seoul, Korea in 2002 and 2004, respectively. In 2004, He joined LG Electronics Inc, Seoul, Korea, and he is currently involved in CMOS circuit design and system integration for mobile camera applications. His present research interests are in the field of imaging system, such as image sensor and image signal processing, mixed mode circuit and CMOS SoC design. Sang-Wook Kim received the B.S. and M.S. degrees in Electrical Engineering from Yonsei University, Seoul, Korea in 2001 and 2003, respectively. In 2003, he joined LG Electronics Inc, Seoul, Korea, and he is currently involved in SoC Design of camera system. His present research interests include the ASIC design of image processing algorithm. Kug-Hyun Baek received the B.S., and M.S degrees in Electrical and Electronic Engineering from POSTECH (Pohang Institute of Science & Technology) in 1992, and 1994, respectively. In 1994, he joined LG Electronics Inc, Seoul, Korea, and he is currently involved in SoC design of camera for mobile application. His present research interests include the ASIC design of video compression and image processing algorithm for camera system. Woo-Hyun Paik received his B.S. and M.S. degrees in Electrical Engineering from Korea University, Korea in1984 and 1986, respectively. And he received Ph.D. degree in Electronics Engineering from Korea University in In 1986 he joined VLSI Design Group, LG Semiconductor Co. Ltd. From 1999, he is with LG Electronics Inc., Korea. He is a Research Fellow and is in charge of SoC Design Group. His recent research interests include design of mixed mode IC, high speed and low power digital systems for video/image signal processing and SoC design methodologies. Sung-Hyun Yang (M 05) received the B.S., M.S., and Ph. D. degrees in Computer and Communication Engineering from Chungbuk National University, Cheongju, Korea in 1999, 2001, and 2004, respectively. From 2004, he is with the LG Electronics Inc., Seoul, Korea. He is a member of Institute of Electrical and Electronics Engineer (IEEE), Institute of Electronics, Information and Communication Engineers (IEICE), and Institute of Electronics Engineers of Korea (IEEK). His research interests include highspeed and low-power circuit, CMOS active pixel image sensor, and continuous-time filter designs, etc.