Iterative Single-Image Digital Super-Resolution Using Partial High-Resolution Data

Transcription

1 Iterative Sinle-Imae Diital Super-Resolution Usin Partial Hih-Resolution Data Eran Gur, Member, IAENG and Zeev Zalevsky Abstract The subject of extractin hih-resolution data from low-resolution imaes is one of the most important diital processin applications in recent years, attractin much research. In this work the authors show how to improve the resolution of an imae when a small part of the imae is iven in hih-resolution. To obtain this result the authors use an iterative procedure imposin the low frequencies complete data of the oriinal low-resolution imae and the hih-resolution data present only in a fraction of the imae. The result is a clearer imae, with hiher correlation to the required hih-resolution imae. The authors show the use of such a procedure on Rosetta imaes to demonstrate the hiher frequencies obtained and on a text sample to show improvement in textual understandin. Index Terms Imae Processin, Sinal reconstruction, Superresolution. I. INTRODUCTION In recent years many researchers tried to address the subject of super resolution (SR). SR refers to recoverin hih resolution data from imaes that due to mis-focus, compression or other forms of distortion have lost the hih frequencies data that were oriinally embedded in the imae, and hence are now iven as low resolution imaes. The methods to overcome this problem of data loss, and enerate SR, are quite versatile. In some cases the method is to obtain data about the blurrin function and use an inverse filter to reconstruct the hih-resolution imae [,]. Unfortunately, two main problems limit this approach. The first, usually it is impossible to identify the exact blurrin function since it is a result of stochastic noise and thus only its statistical properties are known. The second, even if the blurrin function is known makin an inverse filter miht not be practical (e.. if the oriinal blurrin filter has zeros the inverse filter must have sinular values to obtain exact restoratio. Other methods use lare databases; they are divided into two roups. In the first roup [3,4] one takes a lare amount of different test-imaes present both in low resolution and hih resolution and attempt to find Manuscript submitted February 0, 007 Eran Gur is with Shenkar Collee of Enineerin & Desin, Ramat Gan, 556, Israel (correspondin author, phone: ; fax: ; eranur@mail.shenkar.ac.il). Zeev Zalevsky is with Bar Ilan University, Ramat Gan, 5900, Israel ( zalevsz@macs.biu.ac.il). the blurrin procedure that will yield the best results with respect to all imaes. There are two problems with this approach, no two pictures are identical and therefore we cannot be sure that the inverse blurrin procedure found will be applicable for the required new imae, and, usually the blurrin procedure varies from one test-imae to another and thus the anti-blurrin filter will be an averae of many filters, and not an exact filter. The second roup of SR usin lare databases assumes one has many low-resolution pictures of the required subject [5,6]. Since in every picture a different portion of the hih-resolution data is missin it is possible to extract some hih-resolution data from these imaes to obtain a sinle hih-resolution imae. The main drawback of these methods is the lare database required in order to increase the resolution of a sinle imae. In our work we suest a novel approach assumin only one imae is iven the required imae. Suppose one has to scan an imae with hih resolution, it is time and memory consumin. However, if only a small portion will be scanned with hih accuracy and most of the data scanned with lower accuracy the process it much faster and the storae capacity required to store the imaes is much smaller (e.. with CT scannin if the patient has to spend less time on the scannin device his/her inconvenience is reduced). This is the exact principle used in this work. We assume only a small portion of the required imae is iven in hih-resolution and use this data (assumin it has similar statistical properties to the neihborin low-resolution portion of the imae) to increase the resolution of the entire imae. To do this we use an iterative procedure relyin on two initial assumptions: in one small portion of the imae we have all the hih-resolution data, and the entire imae contains all the low frequencies of the oriinal hih-resolution imae. In the followin sections we explain the procedure and show some test cases supportin this approach. II. ITERATIVE SINGLE IMAGE SUPER-RESOLUTION A. Concept When usin a sinle imae we need to know the limits of our data. In our case we know for certain that the low frequencies exist in the low-resolution imae as they would exist in the hih-resolution imae (we assume the blurrin function has a relatively sharp frequency response, and thus the lower frequencies are not distorted). Thus we can impose a ISBN: WCE 007

2 frequency-domain restriction on the imae. We also know that a certain portion of the imae is presented in hih-resolution so we can impose an object-domain restriction on the imae. These two restrictions allow us to bounce back and forth from the object-domain to the frequency-domain, with a procedure similar to the one used for phase retrieval, as shown in the followin subsection. B. Review: Iterative Phase Retrieval A well-known problem is to determine the phase of a phase only object plane filter that will produce a required intensity distribution in the Fourier domain. In there paper [7] Gerchber and Saxton suested an iterative manner to do just that. This method is proven to convere to a phase filter with a minimal MSE [8]. The concept is quite simple: We bein with an arbitrary phase-only filter in the object domain multiplyin the input object (the oriinal imae), after a Fourier transform we obtain a Fourier domain imae and we impose the require Fourier intensity (actually the manitude), leavin the phase unharmed. An inverse Fourier transform brins us back to the object domain. Since we demand a phase-only filter we impose the intensity of the input object in this plane. Next we calculate the Fourier transform and return to the Fourier domain, and so on. This procedure is required since usin only the phase of the complex filter that converts the input imae exactly to the Fourier imae ives poor results. As can be seen, if we impose half of the information (intensity or phase) in both the input and the output domains the procedure converes monotonically. In later work Gerchber [9] and Papoulis [0] suested the use of this method for super-resolution. However, both presented relatively simple test cases and assumed the properties of all iterations to be identical (accept when noise reduction was addressed). There are of course other methods for obtainin the phase filter, such as Simulated Annealin [], which ensures that the MSE has indeed a lobal minimum, but it is time and resources consumin. C. Super-Resolution by factor of Now we present the approach used to obtain a factor- Super-resolution (i.e. the oriinal imae had N by N pixels but we obtain only an N by N pixels imae). Let us indicate the required imae by where m, n N, and the low-resolution imae as LR where m, n N. Since the size of the imaes is not the same we bein by plantin zeros between each row and column element of LR, thus eneratin: m + n + LR, odd( m &, = () 0 otherwise ( m The new imae has the same number of pixels as the required hih-resolution imae. We assume that a certain portion of the imae is known completely, so we may impose it on the new imae: G ( m = m m m, n n, () n In the Fourier plane we obtain for the hih-resolution imae: (, l) G k = M N M N m= n= m k n l exp jπ + M N whereas for the low-resolution imae, the DFT is LR ( k, l) = M N M N m= n= m k n l exp jπ + M N Since the low-resolution imae contains all the low frequencies of the hih-resolution imae we may deduce that we can use the followin restriction: G ( k l) = 4 G ( k, l) k M, l N LR (3) (4), (5) Where is the DFT of, and the factor of 4 is required since the size of and is not the same. If m m + = M and n n + = N then we have only 5% of the data in the object plane and 5% of the data in the Fourier plane. This is less than the required amount stated by the phase retrieval alorithm shown before and therefore the converence to a minimal MSE is not assured. In fact, when usin the iterative procedure for this case we obtain the followin: at first, the MSE decreases monotonically, but after several iterations the requirements are not stron enouh to keep the results in the riht track, and the MSE starts to rise. For this reason we add a new condition to the iterative procedure: halt when MSE reaches local minima. In addition we noticed that a smaller (better) MSE minimum could be obtained by radually increasin the frequency domain requirements, i.e., imposin only very low frequencies at the beinnin and radually increasin the frequencies imposed, up to the maximum value iven in (5). This is due to the fact that primarily is quite different from (yieldin a lare MSE) and thus imposin a lare portion of the Fourier domain at an early stae sets the results way out of track. The followin summarizes the procedure steps:. Obtain low-resolution imae and set initial low frequencies rane to impose. ISBN: WCE 007

3 . Implant zeros between data points to increase imae size to. 3. Impose hih-resolution portion on imae. 4. Perform DFT. 5. Impose low frequencies on DFT. 6. Perform IDFT. 7. Impose hih-resolution portion on imae. 8. Calculate MSE. If local minimum is obtained then increase the rane of low frequencies to impose. 9. If the rane of low frequencies has reached the complete rane available in the oriinal low-resolution imae then stop, else o to step 4. When usin this method we can obtain a sharp imae at relatively short processin times. To demonstrate this lets assume that each M by N imae requires time unit to process. Thus the oriinal hih-resolution imae would require 4 time units (because its size is M by N), however, an imae containin only one quarter of the data in hih-resolution will require.75 time units: time unit for the N by M hih resolution and 0.5 time units per each remainin M/ by N/ quarters (the zero paddin is not relevant in this case). Thus, the processin time is less then half of the oriinal one required if all data were iven at hih resolution. Fi.. Oriinal low resolution rosette imae containin 64 by 64 pixels. Fi. 3. Low-resolution rosette imae after paddin with zeros and embeddin the hih-resolution data in top left corner. Fi.. Oriinal hih-resolution rosette imae containin 8 by 8 pixels. III. SIMULATION RESULTS We tested the method on two typical examples. The first, a test rosette containin a variety of frequencies, thus makin it easy to view how resolution is improved. The second, a text example showin how barely readable text can be sharpened. Fi. shows the required hih-resolution rosette imae, whereas Fi. shows the low-resolution rosette. Fi. 3 collaborates the low-resolution data, after paddin with zeros to obtain same imae size as the hih-resolution imae, with a portion of the hih-resolution imae in the first quarter of the imae. The first quarter was used since in most cases the hiher frequencies lie in the center of the imae and we wanted to avoid a biased result. As can be seen, the hih-resolution portion of the imae contains both hih and low frequencies. The MSE between the 6 first two imaes is calculated to be (normalized to ). In Fi. 4 one can see the result of the super resolution 5 iterative procedure. MSE is reduced to (normalized ) and the correlation coefficient between the hih-resolution imae and the one obtained is 97.4%. ISBN: WCE 007

4 As can be seen, in the oriinal hih-resolution imae (Fi. 5) the text is relatively sharp, whereas in the oriinal low-resolution imae the text is quite smeared (Fi. 6). The 5 oriinal MSE in this case is and the correlation coefficient is 78.%. Fi. 4. Reconstructed rosette imae The results of the second test case are iven in Fi. 5 throuh Fi.7. Fi. 7. Reconstructed text imae, accordin to super resolution iterative procedure. However, when applyin the novel iterative approach one 4 obtains the imae shown in Fi. 7. The MSE is (an improvement by a scale of over 4) and the correlation coefficient is 90.8%, which is much hiher than the one obtained by simple reconstruction. Fi. 5. Oriinal hih-resolution text imae containin 8 by 8 pixels. Fi. 6. Oriinal low-resolution text imae containin 64 by 64 pixels. IV. CONCLUSION In this paper the authors suested a novel iterative method for achievin super resolution usin a low-resolution imae accompanied by a small portion of the hih-resolution imae. The new method allows obtainin only a small part of the data with hih accuracy and thus savin time while obtainin the imaes and memory while savin the data before processin. This method may be applicable either as a simple and fast alorithm for slihtly improvin imae content, or as a preliminary process before applyin advanced diital techniques (e.., text oriented reconition methods). REFERENCES [] S. Bellini, Blind Deconvolution, S. Haykin (Editor), chapter, pp. 8-55, 994. [] N.K. Bose, Wavelet-Based Blind Superresolution from Video Sequence and in MRI, PENNSYLVANIA STATE UNIVERSITY Final Report, 005. [3] W. T. Freeman, T. R. Jones, and E. C. PasztorA, Example-Based Super-Resolution, IEEE - Computer Graphics and Applications: Vol., No., pp.56 65, 00. [4] H. Chan, D-Y Yeun, and Y. Xion, Super-Resolution Throuh Neihbor Embeddin, IEEE - Computer Vision and Pattern Reconition (CVPR004): pp.75-8, 004. [5] M. Elad and A. Feuer,. Restoration of a Sinle Superresolution Imae from Several Blurred, Noisy, and Undersampled Measured Imaes, IEEE Transactions On Imae Processin, Vol. 6, No., pp , 997. ISBN: WCE 007

5 [6] A. Zomet and S. Pele, Multi-sensor Super-Resolution, IEEE workshop on Applications of Computer Vision (WACV0), pp. 7-3, 00. [7] R.W. Gerchber and W. O. Saxton, A practical alorithm for determination of phase from imae and diffraction plane picture, Optik (Stuttart), Vol. 35, pp , 97. [8] J. R. Feinup, Phase retrieval alorithms a comparison, Applied Optics, Vol., No. 5, pp , 98. [9] R.W. Gerchber, Super-resolution throuh error enery reduction, Optica Acta, Vol., No. 9, pp , 974. [0] A. Papoulis, A new alorithm in spectral analysis and band-limited extrapolation, IEEE transactions on Circuits and Systems, Vol., No. 9, pp , 975. [] S. Kirkpatrick, C. D. Gelatt Jr., and M. P. Vecchi, Optimization by simulated annealin, Science, Vol. 0, No. 4598, pp , 983. ISBN: WCE 007