Learning Discriminative Data Fitting Functions for Blind Image Deblurring

Transcription

1 Learnng Dscrmnatve Data Fttng Functons for Blnd Image Deblurrng Jnshan Pan 1 Jangxn Dong 2 Yu-Wng Ta 3 Zhxun Su 2 Mng-Hsuan Yang 4 1 Nanng Unversty of Scence and Technology 2 Dalan Unversty of Technology 3 Tencent Youtu Lab 4 UC Merced Abstract Solvng blnd mage deblurrng usually requres defnng a data fttng functon and mage prors. Whle exstng algorthms manly focus on developng mage prors for blur kernel estmaton and non-blnd deconvoluton, only a few methods consder the effect of data fttng functons. In contrast to the state-of-the-art methods that use a sngle or a fxed data fttng term, we propose a data-drven approach to learn effectve data fttng functons from a large set of moton blurred mages wth the assocated ground truth blur kernels. The learned data fttng functon facltates estmatng accurate blur kernels for generc scenes and doman-specfc problems wth correspondng mage prors. In addton, we extend the learnng approach for data fttng functon to latent mage restoraton and nonunform deblurrng. Extensve experments on challengng moton blurred mages demonstrate the proposed algorthm performs favorably aganst the state-of-the-art methods. 1. Introducton The goal of blnd mage deblurrng s to recover a blur kernel and a sharp latent mage from a blurred nput. It s a classcal vson problem, and sgnfcant progress has been made n recent years [10, 11, 14]. When the blur s spatally nvarant, the blur process can be modeled by a convoluton operaton: B = I k +n, (1) where B, I, k, and n denote the blur mage, latent mage, blur kernel, and nose, respectvely; and s the convoluton operator. Blnd mage deblurrng s an ll-posed problem because there are nfnte pars ofi andk whch satsfy (1), and a trval soluton exsts,.e., orgnal blurred mage and delta blur kernel. Although the number of solutons s nfnte, the soluton space of natural mages can be constraned. Numerous methods [2, 4, 15, 16] have been developed based on sparsty of mage gradents for kernel estmaton. On the other hand, recent algorthms explot varous mage prors to recover sharp mages, e.g., normalzed sparsty pror [13], re- (a) (b) (c) (d) Fgure 1. Effect of data fttng functons on kernel estmaton. (a) Blurred mage. (b) Results of [16] by a data fttng functon based on ntensty. (c) Results of [16] by a data fttng functon based on mage gradent. (d) Our results. All the results are generated wth the same settngs for far comparsons. The parts enclosed n the red boxes n (b) and (c) contan sgnfcant blur resdual and artfacts (Best vewed on hgh-resoluton dsplays wth zoom-n). current nternal patch recurrence [18], text mage pror [20], and dark channel pror [23]. These mage prors are based on statstcal assumptons of clear mages and have been shown to be effectve n deblurrng. Dscrmnatve methods [32, 37] have been developed to learn effectve mage prors [25] for blur kernel estmaton. In contrast to statstcal prors, several methods use exemplars for kernel estmaton [1, 6, 19, 28]. In addton to mage prors, another group of methods focus on sharp edge predctons for blur kernel estmaton. However, these methods usually nvolve heurstc edge selecton steps [3, 33] to estmate blur kernels. We note that the aforementoned methods focus on developng effectve mage prors for deblurrng. Among the methods n the lterature, ntensty nformaton s commonly used to defne the data fttng term. Levn et al. [16] show that proper use of mage gradents n a data fttng functon helps mprove blur kernel estmaton. Some recent algorthms [3, 20, 22, 23, 33, 35] use ntensty n latent mage restoraton (e.g., mnmzng l 2 reconstructon errors) and gradent n the kernel estmaton (e.g., mnmzng l 2 errors). However, the effect of ntensty and gradent nformaton n blnd mage deblurrng has not been well analyzed. In ths paper, nstead of proposng mage prors, we study the effect of data fttng functons for kernel estmaton. We show that the data fttng functon also plays a crucal role n blnd mage deblurrng as t measures the goodness-of-ft to the moton blur model n (1). Fgure 1 demonstrates the mportance of dfferent data fttng functons on blur kernel estmaton

2 To address ths ssue, we propose a two-stage approach for blnd mage deblurrng. In the frst stage, an effectve data fttng functon s learned for blur kernel estmaton. In the second stage, the data fttng functon s optmzed for latent mage restoraton. We present an effcent numercal algorthm to learn the data fttng functons for both blur kernel estmaton and latent mage restoraton. In addton, we show that the proposed algorthm can be appled to other doman-specfc deblurrng tasks wth dfferent prors and non-unform deblurrng. 2. Proposed Algorthm In ths secton, we present an algorthm to learn effectve data fttng functons for blnd mage deblurrng. We frst consder the blur kernel estmaton problem, and extend t to the latent mage restoraton task. In ths work, the blnd deblurrng problem s formulated as E(I,k) = ω f I k f B 2 2+ϕ(I)+φ(k), (2) where ω denotes the -th weght, f denotes a lnear flter operator whch can be learned by felds of experts [24], and ϕ(i) as well as φ(k) are the prors of latent mage and blur kernel. As the weghts control the mportance of each term for blur kernel estmaton, the man goal s to estmate these values effectvely Learnng Dscrmnatve Data Functons To estmate ω = {ω }, we collect a set of ground truth blur kernels {k } as well as a set of clear mages {I }, and propose the followng obectve functon, 1 mn ω 2 s.t. ω 0, k (ω) k gt 2 2 ω = 1, where k gt denotes the -th ground truth blur kernel, and k (ω) s the -th estmated blur kernel, whch can be obtaned by arg mn k,i (3) ω f I k f B 2 2+ϕ(I )+φ(k ). (4) To derve the relatonshp between blur kernelk and weght ω n (2), we propose an effcent algorthm to solve (4) Optmzng (4) Smlar to the exstng methods [20, 22, 35, 30], we adopt ϕ(i ) = λ I 0 and φ(k ) = γ k 2 2 as the regularzaton for the latent mage and blur kernel of (4), whereλand γ are weght parameters. We use the half-quadratc splttng L 0 mnmzaton method [34] and ntroduce an auxlary varable g = (g v,gh ) correspondng to the mage gradent I. Thus, (4) can be rewrtten as mn ω f I k f B 2 2 k,i,g +β g I 2 2 +λ g 0 +γ k Intermedate Blur Kernel Estmaton GvenI, the optmzaton wth respect tok s mn ω f I k f B 2 2 +γ k 2 2. k For smplcty, we use the matrx-vector form to express (6) mn ω A k b 2 2 +γ k 2 2, k (7) where A s the matrx form of f I wth respect to blur kernel k, b s the vector form of f B wth respect to blur kernelk, andk s the vector form ofk. Based on (7), the soluton ofk s (5) (6) ( ) 1 ( ) k = ω A A +γ ω A b. (8) Intermedate Latent Image Estmaton The optmzaton problem (5) wth respect to ntermedate latent magei s mn ω f I k f B 2 2 +β g I 2 2 I,g +λ g 0. (9) Note that ths problem nvolves varablesi andg. It can be effcently solved through alternatvely mnmzng I and g. In each teraton, the soluton ofg s obtaned by solvng { I, I 2 λ g = β, (10) 0, otherwse. Gven g, the ntermedate latent mage I can be obtaned by solvng mn ω f I k f B 2 2 +β g I 2 I 2, (11) and the closed-form soluton for ths problem s ) I = F 1 ( ω F(f )F(k )F(f B )+βf g F k +β( {h,v} F( )F( )), (12) 1069

3 Algorthm 1 Solvng (9) Input: Blurred mageb and blur kernel k. I B,β 2λ. repeat solveg usng (10). solvei usng (12). β 2β. untlβ > β max Output: Intermedate latent magei. Algorthm 2 Learnng dscrmnatve features Input: Blurred mages {B }, ground truth blur kernels {k gt }. ω 0. ntalzek wth results from the coarser level. whlel max ter1 do whlet max ter2 do solvei usng Algorthm 1. solvek usng (8). end whle ω = ω α L ω. end whle Output: The weghtω. where F( ) and F 1 ( ) denote the Fourer transform and ts nverse transform, respectvely, F( ) s the complex conugate operator, F k = ω F(f )F(k )F(k )F(f ), and F g = F( h )F(g h ) + F( v)f(g v ), where h and v denote the horzontal and vertcal dfferental operators. In case all the values of ω are zeros n (12) (whch wll lead to unstable kernel estmaton), we set the terms ω 0 F(k )F(f 0 B ) and ω 0 F(k )F(f 0 B ) to be (ω 0 + 1)F(k )F(f 0 B ) and(ω 0 +1)F(f 0 )F(k )F(k )F(f 0 ). The man steps for ntermedate latent mage estmaton are summarzed n Algorthm Optmzng (3) After obtanng k wth respect to ω, we can solve (3) by a gradent descent method. The gradent wth respect to ω s L = L ( ) k 1 = (k k gt ω k ω ) ω A A +γ A A k ( ) 1 ( ) +(k k gt ) ω A A +γ A b, (13) wherel = 1 2 k (ω) k gt 2 2. The detaled dervatons are presented n the supplemental materal. The man steps for learnng dscrmnatve data fttng functons are summarzed n Algorthm 2. In ths work, the step of gradent decentαs set to be0.01. Tranng Data. We construct a tranng dataset to learn Fgure 2. Some generated blur kernels that are used for tranng. the weghts n (2) by usng 200 mages from the BSDS dataset [17]. To generate blurred mages{b } and blur kernels {k }, we synthesze realstc blur kernels by samplng random 3D traectores used n [25]. These traectores are then proected and rasterzed to random square kernel szes n the range from up to pxels. Some examples of the generated blur kernels are shown n Fgure 2. Wth the blur kernels, we synthetcally generate blurred mages by convolvng each clean mage wth 100 generated blur kernels. A set of 200,000 blurred mages s constructed for learnng the weghts of (3) Kernel Estmaton After learnng the weghts, we solve (2) to obtan the blur kernels. That s, we alternatvely solve the ntermedate latent mage and blur kernel. The optmzaton algorthms wth respect to the blur kernel and latent mage are the same to those descrbed n Secton and For the lnear flters{f }, we choose the commonly used zero-order operator correspondng to the ntensty nformaton, and gradent operators ncludng the frst (two drectons) and second order (three drectons) operators. The concrete forms of 6 lnear operators are presented n Table Dscrmnatve Non Blnd Deconvoluton Once blur kernels are obtaned, we can use a varety of non-blnd deconvoluton methods to recover latent mages. However, we note that the proposed method used n the k- ernel estmaton process can also be appled to non-blnd deconvoluton. We formulate the non-blnd deconvoluton problem as mn I ω f I k f B 2 2 +φ(i), (14) where φ(i) s the regularzaton on the mage I, e.g., hyper- Laplacan prors [12]. In ths work, we use the commonly used total varaton regularzaton,.e.,φ(i) = µ I 1, for non-blnd deconvoluton. The weghtω can be obtaned by solvng 1 mn ω 2 s.t. ω 0, I (ω) I gt 2 2 ω = 1, (15) 1070

4 Table 1. Concrete forms of the lnear flters used n the learnng process. Flters f 1 f 2 f 3 f 4 f 5 f 6 Type zero-order frst order frst order second order second order second order Forms I k B h I k h B vi k vb h h I k h h B v vi k v vb h vi k h vb (a) (b) (c) Fgure 3. Non-blnd deconvoluton examples. (a) Blurred mage and blur kernel. (b) Result wth only ntensty nformaton n the data fttng functon. (c) Our result. The part n the red box n (b) contans sgnfcant rngng artfacts (Best vewed on hghresoluton dsplay wth zoom-n). where I (ω) s the soluton of (14) and I gt s the clear mage. To determne the relatonshp between I (ω) and ω, we use the same alternatve mnmzaton method descrbed n Secton to obtan I (ω). The weght ω can be obtaned by ω = ω α I (I I gt ) W, (16) whereα I s the gradent descent step. In the above equaton, I,I gt, as well asw denote the vectorzaton ofi,i gt and W, respectvely. EachW s defned by ( W = F 1 b ) f n d 2, (17) d where d = F k + β( {h,v} F( )F( )), f = F(f )F(k )F(k )F(f ), b = F(f )F(k )F(f B), and n = ω F(k )F(f B )+βf g. We use the same tranng data as dscussed n Secton 2.3 to learn ω. The detals regardng the gradent of (15) wth respect to ω and the optmzaton method wth respect to I n (14) are ncluded n the supplemental materal. Fgure 3 shows an example of the non-blnd deconvoluton result usng (14). We note that the recovered mage by the conventonal data fttng functon contans some rngng artfacts (Fgure 3(b)) whle the one by the proposed method s sharper (Fgure 3(c)). 3. Extenson to Non-Unform Deblurrng Our method can be drectly extended to handle nonunform deblurrng where the blurred mages are acqured from movng cameras (e.g., wth rotatonal and translatonal movements) [5, 7, 26, 29, 31]. Based on the geometrc model of camera moton [29, 31], the non-unform blur process can be formulated as: B = KI+n = Ak+n, (18) where I, k, and n denote vector forms of I, k, n n (1). In ths model, A as well as K denote the mage matrx and blur kernel matrx wth respect to the latent mage I and blur kernel k. Based on (18), the non-unform deblurrng problem s solved by alternatvely mnmzng: and mn I mn k ω KF I F B 2 2 +λ I 0, (19) ω A k B 2 2 +γ k 2 2, (20) where F s the matrx of the flter operator f. We use the fast forward approxmaton methods [7, 21] to estmate latent mages and blur kernels. 4. Analyss of Proposed Algorthm In ths secton, we analyze how the proposed algorthm performs on mage deblurrng. We also demonstrate the mportance of the proposed learned data fttng functons and dscuss the connectons to other methods Effect of Dscrmnatve Data Fttng Functons The proposed method s able to automatcally learn the most relevant data fttng functons for both blur kernel estmaton and latent mage restoraton. We analyze ts effect on blur kernel estmaton and latent mage restoraton wth comparsons to the commonly used data fttng functons. Effect on Blur Kernel Estmaton. The data fttng functons used n exstng methods are based on ntensty or gradent for both latent mage restoraton and kernel estmaton [3, 20, 33, 35]. Fgure 4 shows one example where the deblurred results by the methods wth only ntensty or gradent contan sgnfcant rngng artfacts. The methods usng ntensty for the ntermedate latent mage estmaton and gradent for the kernel estmaton perform better. However, the deblurred results stll contan blur resdual and blurry characters. In contrast, the deblurred results generated by the proposed method contan clearer characters, whch ndcate that the learned data fttng functons facltate blur kernel estmaton. We quanttatvely evaluate the proposed method and present the results n Table 2. The proposed method wth learned data fttng functons performs well aganst other alternatves based on ntensty, gradent, or combnaton. 1071

5 Table 2. Quanttatve comparsons wth the commonly used data fttng functons for the mage deblurrng. Wth only ntensty Wth ntensty and gradent Wth only gradent Ours Average PSNRs Table 4. Learned weghts for latent mage estmaton. ω 1 ω 2 ω 3 ω 4 ω 5 ω (a) (b) (c) Average Kernel Smlarty (d) (e) (h) Fgure 4. Effectveness of the proposed learned data fttng functon for blur kernel estmaton. (a) Blurred mage. (b) Results by only ntensty n the data fttng functon. (c) Results by only gradent n the data fttng functon. (d) Results by ntensty n the ntermedate latent mage estmaton and gradents n the kernel estmaton. (e) Deblurred results by [35]. (f) Our results. Table 3. Learned weghts for blur kernel estmaton. ω 1 ω 2 ω 3 ω 4 ω 5 ω Learned Weghts for Data Fttng Terms. To llustrate the mportance of each data fttng term n blur kernel estmaton process, we show the learned weghts n Table 3. We note that the learned weght of the data fttng term for ntensty s 0, whch demonstrates that ntensty does not help the blur kernel estmaton. The results are smlar to the expermental analyss of the state-of-the-art methods [16, 3, 33, 35, 20]. In addton, we note that the weghts,.e., ω 4 and ω 5, for the data fttng terms wth the second order nformaton are much larger than those of the data fttng terms wth the frst order ntensty, whch ndcate that hgher order nformaton plays more mportant roles for blur kernel estmaton. Effect on Non-Blnd Deconvoluton. The learned weghts of data fttng terms for latent mage restoraton are shown n Table 4. We note that the weght of the data fttng term wth the zero-order flter s much hgher than the others. Ths ndcates that ntensty nformaton plays an mportant role n non-blnd deconvoluton. The learned weghts n Table 3 and 4 demonstrate that dfferent data fttng terms should be used as kernel estmaton and non-blnd deconvoluton are dfferent processes Iteratons Fgure 5. Fast convergence property of the proposed algorthm Fast Convergence Property Compared to the exstng methods wth fxed data fttng functons based on only ntensty, gradent or combnaton [20, 35], the proposed algorthm nvolves addtonal data fttng terms wth dfferent weghts. However, ths does not sgnfcantly ncrease the computatonal load. We evaluate the convergence rate of the proposed method on the dataset [15] and show the kernel smlarty [9] wth respect to teratons n Fgure 5. The results demonstrate that the proposed method exhbts fast convergence. Table 5 shows that the run tme of our method compares favorably aganst the competng methods. Table 5. Run tme (seconds) on the same computer wth an Intel Core MQ processor and 16 GB RAM. The run tme of Xu et al. [35] s based on our mplementaton. Method Xu et al. [35] Krshnan et al. [13] Levn et al. [16] Pan et al. [23] Ours Expermental Results We present expermental evaluatons of the proposed algorthm aganst the state-of-the-art deblurrng methods. All the experments are carred out on a machne wth an Intel Core MQ processor and 16 GB RAM. The run tme for a mage s 5 seconds on MATLAB. In all the experments, we set λ = and γ = 2. We emprcally set β max = 10 5 n Algorthm 1. We use the deblurrng datasets by Sun et al. [27] and Levn et al. [15] as the man test datasets. Thus, the tranng and test datasets are not overlapped. For far comparsons, we use the executable 1072

6 Fgure 6. Quanttatve evaluatons on the benchmark dataset by Sun et al. [27]. Our method performs favorably aganst the stateof-the-art methods. code provded by the authors and tune the parameters to generate the best possble results of other methods. The MATLAB code and datasets are publcly avalable on the authors webstes. More expermental results can be found n the supplemental materal Quanttatve Evaluaton We evaluate the proposed method on the synthetc dataset by Sun et al. [27] and compare t wth several state-of-the-art deblurrng methods [3, 13, 16, 18, 21, 27, 35]. Ths dataset contans 80 mages and 8 blur kernels from [15]. We use the orgnal codes of the state-of-theart methods [3, 13, 16, 18, 21, 27, 35] to estmate blur kernels and use the non-blnd deblurrng method [36] to generate the fnal deblurrng results for far comparsons. The error rato [15] s used for performance evaluaton. Fgure 6 shows the quanttatve results on the dataset [27]. Overall, the proposed algorthm performs favorably aganst the state-of-the-art deblurrng methods. We note that the method [35] uses ntensty n ntermedate latent mage estmaton and gradent n the kernel estmaton. Compared wth ths method, the proposed algorthm acheves hgher success rates, whch ndcates the effectveness of the learned data fttng functons Real Images Fgure 7 shows the deblurred results on a real mage by the proposed algorthm and the state-of-the-art methods. We use the orgnal source or bnary codes and tune the parameters to generate the best possble results for far comparsons. The deblurred mages by [3, 13] contan sgnfcant rngng artfacts. Whle the state-of-the-art methods [16, 20, 23] generate better kernel estmates than other methods, the deblurred mages contan sgnfcant rngng artfacts. We note that the man dfference between the proposed algorthm and the method by Xu et al. [35] s that [35] uses ntensty for latent mage restoraton and gradent for kernel estmaton. However, the results demonstrate that ths manually desgned data fttng functon s not effectve for blur kernel estmaton on the real mage. In contrast, the deblurred mage by the proposed algorthm contans fewer artfacts, whch shows that the learned functon wth dfferent weghted combnaton of data fttng terms s effectve for kernel estmaton. Fgure 8 shows the deblurred results by the proposed algorthm and the state-of-the-art methods [3, 13, 16, 20, 23, 35] on a real blurred document mage. The state-of-theart deblurrng methods desgned for natural mages [3, 13] do not generate clear mages. Although the method by Pan et al. [20] manly focuses on the text mage deblurrng, the deblurred results stll contan sgnfcant blur resdual and rngng artfacts. The recent method based on sparsty of dark channel pror for blur kernel estmaton [23] does not perform well on ths mage as the assumpton on zerontensty values of an mage does not hold. In contrast, the deblurred mage by the proposed algorthm s clearer wth sgnfcantly fewer artfacts. Furthermore, the deblurred results shown n Fgure 8(e) and (h) demonstrate the effectveness of the proposed algorthm that learns dfferent weghted functons of data fttng terms for latent mage restoraton and blur kernel estmaton Non unform Deblurrng As dscussed n Secton 3, the proposed algorthm can be extended to handle non-unform blur. We present results on an mage degraded by spatally varant moton blur provded by [5] n Fgure 9. We compare the proposed algorthm wth the state-of-the-art non-unform deblurrng methods [5, 7, 8, 21, 31, 35]. Fgure 9 shows the letters of (b)-(f) contan rngng artfacts. Compared to the deblurred results by the state-of-the-art non-unform methods, the restored mage by the proposed algorthm contans sharper contents wth fewer artfacts Extensons of Proposed Method In ths work, we focus on learnng effectve data fttng functons for blur kernel estmaton and use the L 0 norm of mage gradent [35] as the mage pror. However, our method can be appled to other deblurrng tasks wth specfc mage prors, e.g., normalzed sparsty pror [13], L 0 - regularzed ntensty and gradent pror [20], and dark channel pror [23], to name a few. To demonstrate the flexblty of the proposed method, we use the L 0 -regularzed ntensty and gradent pror [20] as an example and show the deblurred results n Fgure 10. We note that the orgnal text deblurrng method uses ntensty for latent mage and gradent for kernel estmaton. However, ths combnaton does not always help blur kernel estmaton (see Fgure 10(b)). In contrast, the method wth the learned data fttng functon generates deblurred mages wth clearer characters as shown n Fgure 10(c). 1073

7 (a) Blurred mage (b) Cho and Lee [3] (c) Krshnan et al. [13] (d) Levn et al. [16] (e) Xu et al. [35] (f) Pan et al. [20] (g) Pan et al. [23] (h) Ours Fgure 7. Comparsons on a real mage. The proposed method generates a better kernel estmate and deblurred result wth fewer rngng artfacts. The comparson results shown n (e) and (h) ndcate the mportance of the proposed automatcally learned data fttng functons. (a) Blurred mage (b) Cho and Lee [3] (c) Krshnan et al. [13] (d) Levn et al. [16] (e) Xu et al. [35] (f) Pan et al. [20] (g) Pan et al. [23] (h) Ours Fgure 8. Comparsons on a real mage wth lots of characters. The proposed method generates an mage wth clearer characters. The comparson results shown n (e) and (h) ndcate the mportance of the proposed learned data fttng functons. Comparsons wth [32]. The recent method [32] ams to learn hgh-order flters for mage deblurrng. The mage pror based on the learned hgh-order flters s especally effectve for text mages. However, our method focuses on learnng good data fttng functons nstead of mage prors for mage deblurrng. Fgure 11 shows a real blurred text mage from [20]. The proposed method wth the L0 -regularzed ntensty and gradent pror performs compettvely aganst the state-of-the-art text deblurrng methods [20, 32]. 6. Concludng Remarks In ths paper, we propose an effectve algorthm whch learns effectve data fttng functons for both blur kernel estmaton and latent mage restoraton. We dscuss the effect of the proposed data fttng functons and show that ntensty has less effect on blur kernel estmaton and has more effect on latent mage restoraton. We show that the performance of deblurrng algorthms usng the learned data fttng functons can be sgnfcantly mproved. The proposed algorthm s also extended for non-unform deblurrng. In addton, we show that the proposed method can be extended to other specfc deblurrng tasks wth correspondng mage prors. Extensve expermental evaluatons on benchmark datasets and real mages demonstrate that the proposed algorthm performs favorably aganst the state-of-the-art methods for unform as well as non-unform deblurrng. Whle we focus on learnng effectve data fttng functons for blnd mage deblurrng, the choce of lnear flters s fxed. In addton, the optmzaton method may also play an mportant role [4, 16] and some models may accmmo1074

8 (a) Blurred mage (b) Gupta et al. [5] (c) Hrsch et al. [7] (d) Hu and Yang [8] (e) Whyte et al. [31] (f) Xu et al. [35] (g) Pan et al. [21] (h) Ours Fgure 9. The proposed method drectly apples to mages wth non-unform blur. The mages shown n (b)-(g) are drectly obtaned from the reported results n [21] (Best vewed on hgh-resoluton dsplay wth zoom-n). (a) Blurred mages (b) Pan et al. [20] (c) Ours Fgure 10. Extenson of text mage deblurrng algorthm [20]. The results generated by the proposed method contan clearer characters and fewer rngng artfacts (Best vewed on hgh-resoluton dsplay wth zoom-n). (a)blurred mage (b) Pan et al. [20] (c) Xao et al. [32] (d) Ours Fgure 11. Comparsons wth [32] on a real text mage from [20]. The mages shown n (b)-(c) are drectly obtaned from the reported results (Best vewed on hgh-resoluton dsplay wth zoom-n). date better optmzaton strateges than others. Our future work wll focus on learnng effectve lnear flters and optmzaton methods for mage deblurrng. Acknowledgements. Ths work has been partally support- ed by NSFC (No , ), NSF CAREER (No ), 973 Program (No. 2014CB347600), NSF of Jangsu Provnce (No. BK ), the Natonal Key R&D Program of Chna (No. 2016YFB ), and gfts from Adobe and Nvda. 1075

9 References [1] S. Anwar, C. P. Huynh, and F. Porkl. Class-specfc mage deblurrng. In ICCV, pages , [2] T. Chan and C. Wong. Total varaton blnd deconvoluton. IEEE TIP, 7(3): , [3] S. Cho and S. Lee. Fast moton deblurrng. In SIGGRAPH Asa, volume 28, page 145, , 4, 5, 6, 7 [4] R. Fergus, B. Sngh, A. Hertzmann, S. T. Rowes, and W. T. Freeman. Removng camera shake from a sngle photograph. ACM SIGGRAPH, 25(3): , , 7 [5] A. Gupta, N. Josh, C. L. Ztnck, M. F. Cohen, and B. Curless. Sngle mage deblurrng usng moton densty functons. In ECCV, pages , , 6, 8 [6] Y. HaCohen, E. Shechtman, and D. Lschnsk. Deblurrng by example usng dense correspondence. In ICCV, pages , [7] M. Hrsch, C. J. Schuler, S. Harmelng, and B. Schölkopf. Fast removal of non-unform camera shake. In ICCV, pages , , 6, 8 [8] Z. Hu and M.-H. Yang. Fast non-unform deblurrng usng constraned camera pose subspace. In BMVC, pages 1 11, , 8 [9] Z. Hu and M.-H. Yang. Good regons to deblur. In ECCV, pages 59 72, [10] J. Ja. Mathematcal models and practcal solvers for unform moton deblurrng. Cambrdge Unversty Press, [11] R. Köhler, M. Hrsch, B. J. Mohler, B. Schölkopf, and S. Harmelng. Recordng and playback of camera shake: Benchmarkng blnd deconvoluton wth a real-world database. In ECCV, pages 27 40, [12] D. Krshnan and R. Fergus. Fast mage deconvoluton usng Hyper-Laplacan prors. In NIPS, pages , [13] D. Krshnan, T. Tay, and R. Fergus. Blnd deconvoluton usng a normalzed sparsty measure. In CVPR, pages , , 5, 6, 7 [14] W.-S. La, J.-B. Huang, Z. Hu, N. Ahua, and M.-H. Yang. A comparatve study for sngle-mage blnd deblurrng [15] A. Levn, Y. Wess, F. Durand, and W. T. Freeman. Understandng and evaluatng blnd deconvoluton algorthms. In CVPR, pages , , 5, 6 [16] A. Levn, Y. Wess, F. Durand, and W. T. Freeman. Effcent margnal lkelhood optmzaton n blnd deconvoluton. In CVPR, pages , , 5, 6, 7 [17] D. Martn, C. Fowlkes, D. Tal, and J. Malk. A database of human segmented natural mages and ts applcaton to e- valuatng segmentaton algorthms and measurng ecologcal statstcs. In ICCV, pages , [18] T. Mchael and M. Iran. Blnd deblurrng usng nternal patch recurrence. In ECCV, pages , , 6 [19] J. Pan, Z. Hu, Z. Su, and M.-H. Yang. Deblurrng face mages wth exemplars. In ECCV, pages 47 62, [20] J. Pan, Z. Hu, Z. Su, and M.-H. Yang. Deblurrng text mages va L 0-regularzed ntensty and gradent pror. In CVPR, pages , , 2, 4, 5, 6, 7, 8 [21] J. Pan, Z. Hu, Z. Su, and M.-H. Yang. L 0-regularzed ntensty and gradent pror for deblurrng text mages and beyond. IEEE TPAMI, 39(2): , , 6, 8 [22] J. Pan and Z. Su. Fast l 0 -regularzed kernel estmaton for robust moton deblurrng. IEEE Sgnal Processng Letters, 20(9): , , 2 [23] J. Pan, D. Sun, H. Pfster, and M.-H. Yang. Blnd mage deblurrng usng dark channel pror. In CVPR, pages , 5, 6, 7 [24] S. Roth and M. J. Black. Felds of experts: A framework for learnng mage prors. In CVPR, pages , [25] U. Schmdt, C. Rother, S. Nowozn, J. Jancsary, and S. Roth. Dscrmnatve non-blnd deblurrng. In CVPR, pages , , 3 [26] Q. Shan, W. Xong, and J. Ja. Rotatonal moton deblurrng of a rgd obect from a sngle mage. In ICCV, pages 1 8, [27] L. Sun, S. Cho, J. Wang, and J. Hays. Edge-based blur kernel estmaton usng patch prors. In ICCP, , 6 [28] L. Sun, S. Cho, J. Wang, and J. Hays. Good mage prors for non-blnd deconvoluton - generc vs. specfc. In ECCV, pages , [29] Y.-W. Ta, P. Tan, and M. S. Brown. Rchardson-lucy deblurrng for scenes under a proectve moton path. IEEE TPAMI, 33(8): , [30] J. Tang, X. Shu, G. Q, Z. L, M. Wang, S. Yan, and R. Jan. Tr-clustered tensor completon for socal-aware mage tag refnement. IEEE TPAMI, 39(8): , [31] O. Whyte, J. Svc, A. Zsserman, and J. Ponce. Non-unform deblurrng for shaken mages. IJCV, 98(2): , , 6, 8 [32] L. Xao, J. Wang, W. Hedrch, and M. Hrsch. Learnng hgh-order flters for effcent blnd deconvoluton of document photographs. In ECCV, , 7, 8 [33] L. Xu and J. Ja. Two-phase kernel estmaton for robust moton deblurrng. In ECCV, pages , , 4, 5 [34] L. Xu, C. Lu, Y. Xu, and J. Ja. Image smoothng va L 0 gradent mnmzaton. In SIGGRAPH Asa, volume 30, page 174, [35] L. Xu, S. Zheng, and J. Ja. Unnatural L 0 sparse representaton for natural mage deblurrng. In CVPR, pages , , 2, 4, 5, 6, 7, 8 [36] D. Zoran and Y. Wess. From learnng models of natural mage patches to whole mage restoraton. In ICCV, pages , [37] W. Zuo, D. Ren, S. Gu, L. Ln, and L. Zhang. Dscrmnatve learnng of teraton-wse prors for blnd deconvoluton. In CVPR, pages ,