Generalized Video Deblurring for Dynamic Scenes

Transcription

1 Generalzed Vdeo Deblurrng for Dynamc Scenes Tae Hyun Km and Kyoung Mu Lee Department of ECE, ASRI, Seoul Natonal Unversty, , Seoul, Korea {llger9, However, prevous approaches, whch assume that the scene s statc, suffer from general blurs not only from camera shake but also from movng objects and depth varatons n a dynamc scene. As parameterzng a spatally varyng blur kernel n the dynamc scene s dffcult wth smple homography, kernel estmaton to handle dynamc scene becomes more challengng. Therefore, several rearxv: v1 [cs.cv] 9 Jul 2015 (c) Fgure 1: Blurry frame of the bcycle sequence. Deblurrng result of Cho et al.[7]. (c) Our result. Abstract Several state-of-the-art vdeo deblurrng methods are based on a strong assumpton that the captured scenes are statc. These methods fal to deblur blurry vdeos n dynamc scenes. We propose a vdeo deblurrng method to deal wth general blurs nherent n dynamc scenes, contrary to other methods. To handle locally varyng and general blurs caused by varous sources, such as camera shake, movng objects, and depth varaton n a scene, we approxmate pxel-wse kernel wth bdrectonal optcal flows. Therefore, we propose a sngle energy model that smultaneously estmates optcal flows and latent frames to solve our deblurrng problem. We also provde a framework and effcent solvers to optmze the energy model. By mnmzng the proposed energy functon, we acheve sgnfcant mprovements n removng blurs and estmatng accurate optcal flows n blurry frames. Extensve expermental results demonstrate the superorty of the proposed method n real and challengng vdeos that state-of-the-art methods fal n ether deblurrng or optcal flow estmaton. 1. Introducton Moton blurs are the most common artfacts n vdeos recorded usng hand-held cameras. For decades, several researchers have studed deblurrng algorthms to remove moton blurs. Ther methodologes depend on whether the captured scenes are statc or non-statc. Early works on sngle mage deblurrng usually assumed that the scene s statc wth constant depth [5, 9, 10, 11, 25, 27]. The successful approaches were naturally extended to vdeo deblurrng. In the work of Ca et al. [2], a mult-mage deconvoluton method was proposed usng sparsty of blur kernels and clear mage to handle regstraton errors. However, ths method only enables two-dmensonal translatonal camera moton, whch generates unform blur. Therefore, the proposed approach cannot handle rotatonal camera shake, whch s the man cause of large moton blurs [27]. To overcome ths lmtaton, L et al. [21] used a method parameterzng spatally varyng motons wth 3x3 homographes, and could handle spatally varyng blurs by camera rotaton. In the work of Cho et al. [4], camera moton n threedmensonal space was estmated wthout the assstance of specalzed hardware. In addton, non-unform blurs by projectve camera moton could be removed. Spatally varyng blurs by depth varaton n a statc scene was handled recently n the works of Lee and Lee [19] and Paramanand et al. [23].

2 (c) Fgure 2: Blurry frame of vdeo contanng movng car. Our deblurrng result. (c) Our color coded optcal flow. searchers have focused on restorng dynamc scenes, whch s manly grouped nto two approaches: segmentaton-based approach, and exemplar-based approach. Segmentaton-based deblurrng approaches smultaneously estmate multple motons, multple kernels, and assocated mage segments. Cho et al. [6] proposed a method that segments mages nto multple regons of homogeneous motons and estmates the correspondng blur kernel as a one-dmensonal Gaussan kernel. Therefore, ths method cannot handle complex motons of objects and rotatonal motons of cameras that generate locally varyng blurs. Bar et al. [1] proposed a layered model and segmented mages nto two layers (foreground and background). In addton, they estmated a lnear blur kernel correspondng to a foreground layer. Although ths method can explctly handle occluded regons usng a layered model, the kernel s lmted to a one-dmensonal box flter only, and only a statc camera s allowed. Wulff and Black [28] extended the prevous work of Bar et al. They focused on estmatng the parameters for both foreground and background motons. However, the motons wthn each segment are only parameterzed usng the affne model, and extendng to multlayered scenes s dffcult because such task requres jont estmaton of depth orderng of the layers. In summary, segmentaton-based approaches have the advantage of handlng blurs by movng objects n dynamc scenes. However, parameterzng the motons n each segment remans an ssue [16]. That s, t fals to segment non-parametrcally varyng complex motons such as motons of people, because dong so wth the smple models used n [1, 28] s dffcult. The works of Matsushta et al. [22] and Cho et al. [7] are typcal exemplar-based approaches. These works estmate latent frames by nterpolatng sharp patches, that commonly exst n a long mage sequence. Therefore, these methods dsregard accurate segmentaton and deconvoluton, enablng the emergence of rngng artfacts. However, the former work cannot handle blurs by movng objects. Moreover, the latter one can only treat blurs by slghtly movng objects n dynamc scenes because t searches sharp patches of a blurry patch usng globally parameterzed kernel wth homography. Therefore, handlng fast-movng objects, whch have dstnct motons from backgrounds, s dffcult. Moreover, t degrades md-frequency textures, such as grasses and trees, because ths method does not use deconvoluton wth spatal prors but use nterpolaton to restore latent frames, whch renders smooth results. To allevate the problems n prevous works, we propose a new generalzed vdeo deblurrng method that estmates latent frames wthout usng global moton parametrzaton and segmentaton. We estmate bdrectonal optcal flows and use them to estmate pxel-wse varyng kernels. Therefore, we can naturally handle coexstng blurs by camera shake, movng objects wth complex motons, and depth varatons. However, sharp frames are requred to obtan accurate optcal flows because estmatng flow felds s dffcult between blurry mages. In addton, accurate optcal flows are necessary to restore sharp frames. Ths case s a typcal chcken-and-egg problem, and thus we smultaneously estmate both varables. Therefore, we propose a new sngle energy model to solve our jont problem. We also provde a framework and effcent technques to optmze the model. The result of our system s shown n Fg.2, n whch the movng car s successfully restored because accurate optcal flows are jontly estmated. By mnmzng the proposed energy functon, we acheve sgnfcant mprovements n numerous real challengng vdeos that other methods fal to do, as shown n Fg.1. Furthermore, we estmate more accurate optcal flows compared wth the state-of-the-art flow estmaton method, that handles blurry mages. The performances are demonstrated n our extensve experments.

3 L 1 L L +1 x u +1 exposure tme t 1 tme nterval t t +1 τ = exposure tme tme nterval τ (u 1, v 1 ) k,x (u, v) v τ (u +1, v +1 ) u Fgure 3: Bdrectonal optcal flows. Pece-wse lnear blur kernel at pxel locaton x. 2. Generalzed Vdeo Deblurrng Most conventonal vdeo deblurrng methods suffer from the coexstence of varous moton blurs from dynamc scenes because the motons cannot be parameterzed usng global or segment-wse parameterzaton. To handle general blurs, we propose a new energy model usng pxel-wse kernel estmaton rather than global or segment-wse parameterzaton. As blnd deblurrng s a well-known ll-posed problem, our energy model not only conssts of data and spatal regularzaton terms but also a temporal term. The model s expressed as follows: E = E data + E temporal + E spatal, (1) and the detals of each term n (1) are gven n the followng sectons Data Model based on Approxmated Blur In conventonal works, the moton blurs of each frame are approxmated usng parametrc models such as homographes and affne models [1, 7, 21, 28]. However, these kernel approxmatons are vald when moton blurs are parameterzable wthn an entre frame or segment. Therefore, pxel-wse moton and kernel estmaton are requred to cope wth general blurs. We approxmate the pxel-wse blur kernel usng bdrectonal optcal flows, n accordance wth prevous works [8, 16, 24]. Specfcally, under an assumpton that the velocty of the moton s constant between adjacent frames, our blur model s expressed as follows: B = 1 2τ τ 0 H(L, t u +1 )+H(L, t u 1 )dt, (2) (c) Fgure 4: Blurry frame of a vdeo n dynamc scene. Locally varyng kernel usng homography. (c) Our pxelwse varyng kernel usng bdrectonal optcal flows. where u +1 = (u +1, v +1 ), and u 1 = (u 1, v 1 ) denote bdrectonal optcal flows at frame. Blurry frame and latent frame are B and L, respectvely. Camera duty cycle of the frame s τ and denotes relatve exposure tme [21]. We defne the mage warpng, H(L, t u +1 ), whch transforms the frame L to L +t when 0 t 1, and H(L, t u 1 ) transforms the frame L to L t. Our b-drectonal optcal flows, duty cycle, and the correspondng pece-wse lnear kernel used n our blur model are llustrated n Fg. 3. Although our blur kernel model s smple, our model can be justfed because we treat vdeo that has short exposure tme to some extent. Therefore, we approxmate the kernel as pece-wse lnear usng bdrectonal optcal flows: k,x (u, v) = δ(uv +1 vu +1) 2τ u +1, f u [0, τ u +1 ], v [0, τ v +1 ] δ(uv 1 vu 1), 2τ u 1, f u (0, τ u 1 ], v (0, τ v 1 ] 0, otherwse. (3) where k,x (u, v) s the blur kernel usng bdrectonal optcal flows at pxel locaton x, and δ denotes Kronecker delta. Usng ths pxel-wse kernel approxmaton, we can easly manage multple dfferent blurs n a frame, unlke conventonal methods. The superorty of our kernel model s shown n Fg. 4. Our kernel model fts blurs from dfferently movng objects and camera shake much better than the conventonal homography-based model. Therefore, we cast pxel-wse kernel estmaton problem as an optcal flows estmaton problem. Dscretzng the

4 constrant (2) gves the followng data term: E data (L, u, B) = λ K (τ, u +1, u 1 )L B 2, (4) where the row vector of blur kernel matrx K, correspondng to the blur kernel at pxel x, s the vector form of k,x (.), and ts elements are non-negatve and ther sum s equal to one. Lnear operator denotes the Toepltz matrces correspondng to the partal (e.g., horzontal and vertcal) dervatve flters. Parameter λ controls the weght of the data term, and L, u, and B denote the set of latent frames, optcal flows, and blurry frames, respectvely Temporal Coherence wth Optcal Flow Constrant Here, we determne that optcal flows are requred to estmate the pxel-wse blur kernel. However, the proposed data term does not have conventonal optcal flow constrants such as brghtness constancy or gradent constancy n (4). In general, such constrants do not hold between two blurry frames. Thus, Portz et al. [24] proposed a method to apply flow constrants between blurry mages. Based on the commutatve law of shft nvarance of kernels [13], the authors of [24] convolved the approxmated blur of each observed mage to the other mage and assumed constant brghtness between them at matched ponts. However, the commutatvty property does not hold n theory when the kernel s not translaton nvarant. Therefore, ths approach only works when the moton s smooth enough. To address ths problem, we propose a new model that fnds correspondences between two latent sharp mages to enable abrupt changes n motons and the correspondng kernels. In usng ths model, we need not restrct our blur kernels to be shft nvarant. Our model s based on the conventonal optcal flow constrant between latent mages, that s, brghtness constancy. The formulaton s expressed as follows: E temporal (L, u) = µ n L (x) L +n (x + u +n ), (5) where n denotes the ndex of neghborng frames at. Constant parameter µ n controls the weght of each term n the summaton. We apply the robust L 1 norm to offer robustness aganst outlers and occlusons. Notably, a major dfference between the proposed model and the conventonal optcal flow estmaton methods s that our problem s a jont problem. That s, the brghtness of latent frames and optcal flows need to be smultaneously estmated. Therefore, our model smultaneously enforces the temporal coherence of latent frames and estmates the correspondences Spatal Coherence To allevate the dffcultes of hghly ll-posed deblurrng and optcal flow estmaton problems, several researchers have emphaszed the mportance of spatal regularzaton. Therefore, we also enforce spatal coherence to penalze spatal fluctuatons whle allowng dscontnutes n both latent frames and flow felds. We assume that spatal prors for latent frames and optcal flows are ndependent. They are expressed as follows: E spatal (L, u) = L + g (x) u +n. The frst term n (6) denotes the spatal regularzaton term for the latent frames. Although more sparse L p norms (e.g., p = 0.8) ft the gradent statstcs of natural sharp mages better [17, 18, 20], we use conventonal total varaton (TV) based regularzaton [12, 14, 16], as TV s computatonally less expensve. The second term denotes the spatal smoothness term for optcal flows. We adopt edge-map coupled TV-based regularzaton [15] to preserve dscontnutes n the flow felds at edges. Smlar to [16], the edge-map s expressed as follows: (6) g (x) = ν exp( ( L σ I ) 2 ), (7) where ν controls the scale of the edge-map, parameter σ I controls the weght, and L s an ntal latent mage n the teratve optmzaton framework. 3. Optmzaton Framework In the prevous sectons, we descrbed the E data, E temporal, and E spatal terms. When camera duty cycle τ s known, our fnal objectve functon becomes as follows: mn λ L,u K (u +1, u 1 )L B 2 + µ n L (x) L +n (x + u +n ) + L + g (x) u +n. Unlke the work of Cho et al. [7], whch sequentally performs mult-phase approaches, our model obtans a soluton by mnmzng a sngle objectve functon. However, because of ts non-convexty, our model s requred to adopt practcal optmzaton methods to obtan approxmated soluton. Therefore, we dvde the orgnal problem nto two (8)

5 sub-problems and use conventonal teratve and alternatng optmzaton technques [5, 28] to mnmze the non-convex objectve functon. In the followng sectons, we ntroduce effcent solvers and descrbe how to estmate unknowns L and u, wth one of them beng fxed Sharp Vdeo Restoraton Whle the optcal flows u are fxed, correspondng blur kernels are also fxed, and our objectve functon n (8) becomes convex wth respect to L, and s expressed as follows: mn L λ K L B 2 + µ n L (x) L +n (x + u +n ) + L. To obtan L, we adopt the conventonal convex optmzaton method n [3], and derve the prmal-dual update scheme as follows: s m+1 s = m +η LAL m max(1, abs(s m +η LAL m )) q m+1,n = L m+1 q m,n +η Lµ nd,n Lm L m +n max(1, abs(q m,n +η Lµ nd,n Lm L m )) +n = arg mn L λ ( K L B ) 2 + (L (L m ɛ L (A T s m+1 (9) + N µ nd,n T q m+1,n )))2 2ɛ L, (10) where m 0 ndcates the teraton number, and, s and q,n denote the dual varables. Parameters η L and ɛ L denote the update steps. A lnear operator A calculates the spatal dfference between neghborng pxels, and another operator D,n calculates the temporal dfferences between L (x) and L +n (x+u +n ). To update the prmal varable and obtan L m+1 method to optmze the quadratc functon Optcal Flows Estmaton n (10), we apply the conjugate gradent Whle the latent frames L are fxed, temporal coherence term E temporal becomes convex but the data term E data remans non-convex. Therefore, we defne a non-convex fdelty functon ρ(.) as follows: ρ(x, u) = λ K (u +1, u 1 )L B 2 + µ n L (x) L +n (x + u +n ). (11) x x + u +1 + u x + u +1 L L +1 L +2 Fgure 5: Temporally consstent optcal flows over three frames. To fnd the optmzed values of optcal flows u, we frst convexfy the non-convex functon ρ(.) by applyng the frstorder Taylor expanson. Smlar to [16], we lnearze the functon near an ntal u 0 n the teratve process as follows: ρ(x, u) ρ(x, u 0 ) + ρ(x, u 0 ) T (u u 0 ). (12) Therefore, our approxmated convex functon for optcal flows estmaton s expressed as follows: mn ρ(x, u 0 ) + ρ(x, u 0 ) T (u u 0 ) + u g (x) u +n. (13) Next, we apply the convex optmzaton technque n [3] to the approxmated convex functon (13), and the prmal-dual update process s expressed as follows: { p m+1,n = p m,n +ηu(ga)um +n max(1, abs(p m,n +ηu(ga)um +n )) u m+1 +n = (um +n ɛ u(g A) T p m+1,n ) ɛ u,n ρ(x, u 0 ), (14) where p,n denotes the dual varable of u +n on the vector space and the dagonal matrx G s the weghtng matrx denoted as G = dag(g (x)). Parameters η u and ɛ u denote the update steps and,n ρ(x, u 0 ) means ρ(x,u) u +n u Implementaton Detals To handle large blurs and gude fast convergence, we mplement our algorthm on the tradtonal coarse-to-fne framework wth emprcally determned parameters. We use λ = 250 for our most experments, and other parameters are determned as µ n = λ, ν = 0.08λ, σ I = , and N = 2. In the coarse-to-fne framework, we buld mage pyramd wth 17 levels for a hgh-defnton(1280x720) vdeo, the scale factor s 0.9, and use b-cubc nterpolaton to propagate both the optcal flows and latent frames to the next pyramd level. Moreover, to reduce the number of unknowns n optcal flows, we only estmate u +1 and u 1. We approxmate u +2 usng u +1 and u For example, t satsfes, u +2 = u +1 + u +1 +2, as llustrated n Fg. 5, and we can easly apply ths for n 1.

6 The overall process of our algorthm s n Algorthm 1. Further detals on estmatng the duty cycle τ and postprocessng step that reduces artfacts are gven below. Algorthm 1 Overvew of the proposed method Input: Blurry frames B Output: Latent frames L and optcal flows u 1: Intalze duty cycle τ and optcal flows u. (Sec. 4.1) 2: Buld mage pyramd. 3: Restore sharp vdeo wth fxed u. (Sec. 3.1) 4: Estmate optcal flows wth fxed L. (Sec. 3.2) 5: Detect occluson and perform post-processng. (Sec 4.2) 6: Propagate varables to the next pyramd level f exsts. 7: Repeat steps 3-6 from coarse to fne pyramd level Duty Cycle Estmaton In ths study, we assume that the camera duty cycle τ s known for every frame. We can obtan the duty cyle from publc SDK, when we use Knect to capture RGB vdeos. However, when we conduct deblurrng wth conventonal data sets, whch do not provde exposure nformaton, we apply the technque proposed n [7] to estmate the duty cycle. Contrary to the orgnal method n [7], we use optcal flows nstead of homographes to obtan ntally approxmated blur kernels. Therefore, we frst estmate flow felds from blurry mages wth [26], whch runs n near real-tme. We then use them as ntal flows and approxmate the kernels to estmate the duty cycle Occluson Detecton and Refnement Our pece-wse lnear kernel naturally results n approxmaton error and t causes problems such as rngng artfacts. Moreover, our data model n (4), and temporal coherence model n (5) are nvald at occluded regons. To reduce such artfacts from kernel errors and occlusons, we use spato-temporal flterng as a post-processng: L m+1 (x) = 1 Z(x) w,n (x, y) L m +n(y), (15) y where y denotes a pxel n the 3x3 neghborng patch at locaton (x + u +n ) and Z s the normalzaton factor (e.g. Z(x) = N y w,n(x, y)). Notably, we enable n = 0 n (15) for spatal flterng. Our occluson-aware weght w,n s defned as follows: w,n (x, y) = o,n (x, y) exp( P (x) P +n (y) 2 ), 2σ 2 w (16) where occluson state o,n (x, y) {0, 0.5, 1} s determned usng the method proposed n [15]. The 5x5 patch P (x) s centered at x n frame. The smlarty control parameter σ w s fxed as σ w = 25/ Expermental Results In what follows, we demonstrate the superorty of the proposed method. (For more results, see the supplementary vdeo.) Frst, we compare our deblurrng results wth those of the state-of-the art exemplar based method [7] wth the vdeos used n [7]. As shown n Fg. 6, the captured scenes are dynamc and contan multple movng objects. The method [7] fals n restorng the movng objects, because the object motons are large and dstnct from the backgrounds. By contrast, our results show better performances n deblurrng movng objects and backgrounds. Ths exemplar-based approach also fals n handlng large blurs, as shown n Fg. 7, as the ntally estmated homographes n the largely blurred mages are naccurate. Moreover, ths approach renders excessvely smooth results for md-frequency textures such as trees, as the method s based on nterpolaton wthout spatal pror for latent frames. Next, we compare our method wth the state-of-the-art segmentaton-based approach [28]. In Fg. 8, the captured scene s a b-layer and used n [28]. Although the b-layer scene s a good example to verfy the performance of the layered model, naccurate segmentaton near the boundares causes serous artfacts n the restored frame. By contrast, our method does not depend on accurate segmentaton and thus restores the boundares much better than the layered model. In Fg. 9, we quanttatvely compare the optcal flow accuraces wth [24] on synthetc blurry mages. Although [24] proposed to handle blurry mages n optcal flow estmaton, ts assumpton does not hold n moton boundares, whch are very mportant for deblurrng. Therefore, ther optcal flow s naccurate n the moton boundares of movng objects. However, our model enables abrupt changes of motons and thus performs better than the prevous model. Moreover, we show the deblurrng results wth and wthout usng the temporal coherence term n (5), and verfy that our temporal coherence model sgnfcantly reduces rngng artfacts near the edges n Fg. 10. Other deblurrng results from numerous real vdeos are shown n Fg. 11. Notably, our model successfully restores the face whch has hghly non-unform blurs because the person moves rotatonally (Fg. 11(e)). 6. Conclusons In ths study, we ntroduced a novel method that removes general blurs n dynamc scenes, whch conventonal methods fal to do. By estmatng a pxel-wse kernel usng optcal flows, we handled general blurs. Thus, we proposed a new energy model that estmates optcal flows and latent frames, jontly. We also provded a framework and effcent solvers to

7 #014 #014 #018 #022 #014 #018 #022 #014 #018 #022 #032 #032 #036 #040 #032 #036 #040 #032 #036 #040 Fgure 6: Left to rght: Blurry frames of dynamc scenes, deblurrng results of [7], and our results. Fgure 7: Left to rght: Blurry frame, deblurrng result of [7], and ours. Fgure 8: Comparson wth segmentaton-based approach. Left to rght: Blurry frame, result of [28], and ours.

8 EPE = 23.3 EPE = 2.32 (c) Fgure 9: EPE denotes average end pont error. Color coded ground truth optcal flow between blurry mages. Optcal flow estmaton result of [24]. (c) Our result. (c) (d) (c) Fgure 10: Real blurry frame of a vdeo. Our deblurrng result wthout usng E temporal. (c) Our deblurrng result wth E temporal. mnmze the energy functon and acheved sgnfcant mprovements n removng general blurs n dynamc scenes. (e) Fgure 11: Left to rght: Numerous real blurry frames and our deblurrng results. - Data sets used n [7]. (c)-(e) Captured RGB data sets usng knect.

9 Acknowledgments Ths research was supported n part by the MKE (The Mnstry of Knowledge Economy), Korea and Mcrosoft Research, under IT/SW Creatve research program supervsed by the NIPA (Natonal IT Industry Promoton Agency) (NIPA-2013-H ), and n part by the Natonal Research Foundaton of Korea (NRF) grant funded by the Mnstry of Scence, ICT & Future Plannng (MSIP) (No ) References [1] L. Bar, B. Berkels, M. Rumpf, and G. Sapro. A varatonal framework for smultaneous moton estmaton and restoraton of moton-blurred vdeo. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, , 3 [2] J.-F. Ca, H. J, C. Lu, and Z. Shen. Blnd moton deblurrng usng multple mages. Journal of computatonal physcs, 228(14): , [3] A. Chambolle and T. Pock. A frst-order prmal-dual algorthm for convex problems wth applcatons to magng. Journal of Mathematcal Imagng and Vson, 40(1): , May [4] S. Cho, H. Cho, Y.-W. Ta, and S. Lee. Regstraton based non-unform moton deblurrng. In Computer Graphcs Forum, volume 31, pages Wley Onlne Lbrary, [5] S. Cho and S. Lee. Fast moton deblurrng. In SIGGRAPH, , 5 [6] S. Cho, Y. Matsushta, and S. Lee. Removng non-unform moton blur from mages. In Computer Vson, ICCV IEEE 11th Internatonal Conference on, pages 1 8. IEEE, [7] S. Cho, J. Wang, and S. Lee. Vdeo deblurrng for hand-held cameras usng patch-based synthess. ACM Transactons on Graphcs, 31(4):64:1 64:9, , 2, 3, 4, 6, 7, 8 [8] S. Da and Y. Wu. Moton from blur. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, [9] R. Fergus, B. Sngh, A. Hertzmann, S. T. Rowes, and W. Freeman. Removng camera shake from a sngle photograph. In SIGGRAPH, [10] A. Gupta, N. Josh, L. Ztnck, M. Cohen, and B. Curless. Sngle mage deblurrng usng moton densty functons. In ECCV, [11] M. Hrsch, C. J. Schuler, S. Harmelng, and B. Scholkopf. Fast removal of non-unform camera shake. In Computer Vson (ICCV), 2011 IEEE Internatonal Conference on, pages IEEE, [12] Z. Hu, L. Xu, and M.-H. Yang. Jont depth estmaton and camera shake removal from sngle blurry mage. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, [13] H. Jn, P. Favaro, and R. Cpolla. Vsual trackng n the presence of moton blur. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, [14] T. H. Km, B. Ahn, and K. M. Lee. Dynamc scene deblurrng. In Computer Vson (ICCV), 2013 IEEE Internatonal Conference on, pages IEEE, [15] T. H. Km, H. S. Lee, and K. M. Lee. Optcal flow va locally adaptve fuson of complementary data costs. In Computer Vson (ICCV), 2013 IEEE Internatonal Conference on, pages IEEE, , 6 [16] T. H. Km and K. M. Lee. Segmentaton-free dynamc scene deblurrng. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, , 3, 4, 5 [17] D. Krshnan and R. Fergus. Fast mage deconvoluton usng hyper-laplacan prors. In NIPS, [18] D. Krshnan, T. Tay, and R. Fergus. Blnd deconvoluton usng a normalzed sparsty measure. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, [19] H. S. Lee and K. M. Lee. Dense 3d reconstructon from severely blurred mages usng a sngle movng camera. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, [20] A. Levn and Y. Wess. User asssted separaton of reflectons from a sngle mage usng a sparsty pror. IEEE Trans. Pattern Analyss Machne Intellgence, 29(9): , [21] Y. L, S. B. Kang, N. Josh, S. M. Setz, and D. P. Huttenlocher. Generatng sharp panoramas from moton-blurred vdeos. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, , 3 [22] Y. Matsushta, E. Ofek, W. Ge, X. Tang, and H.-Y. Shum. Full-frame vdeo stablzaton wth moton npantng. Pattern Analyss and Machne Intellgence, IEEE Transactons on, 28(7): , [23] C. Paramanand and A. N. Rajagopalan. Non-unform moton deblurrng for blayer scenes. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, [24] T. Portz, L. Zhang, and H. Jang. Optcal flow n the presence of spatally-varyng moton blur. In Proc. IEEE Internatonal Conference on Computer Vson and Pattern Recognton, , 4, 6, 8 [25] Q. Shan, J. Ja, and A. Agarwala. Hgh-qualty moton deblurrng from a sngle mage. In SIGGRAPH, [26] A. Wedel, T. Pock, C. Zach, H. Bschof, and D. Cremers. An mproved algorthm for tv-l 1 optcal flow. In Statstcal and Geometrcal Approaches to Vsual Moton Analyss, pages Sprnger, [27] O. Whyte, J. Svc, A. Zsserman, and J. Ponce. Non-unform deblurrng for shaken mages. Internatonal Journal of Computer Vson, 98(2): , [28] J. Wulff and M. J. Black. Modelng blurred vdeo wth layers. In ECCV, , 3, 5, 6, 7