How is project #1 going?

Transcription

1 How is project # going?

2 Last Lecture Edge Detection Filtering Pramid

3 Toda Motion Deblur Image Transformation

4 Removing Camera Shake from a Single Photograph Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis and William T. Freeman Massachusetts Institute of Technolog and Universit of Toronto

5 Overview Original Our algorithm

6 Close-up Original Naïve Sharpening Our algorithm

7 Let s take a photo Blurr result

8 Slow-motion repla

9 Slow-motion repla Motion of camera

10 Image formation process = Blurr image Sharp image Blur kernel Input to algorithm Model is approimation Desired output Convolution operator

11 Wh is this hard? Simple analog: is the product of two numbers. What are the? No unique solution: = = = etc.. Need more information!!!!

12 Multiple possible solutions Sharp image Blur kernel = Blurr image = =

13 Natural image statistics Characteristic distribution with heav tails Histogram of image gradients

14 Blur images have different statistics Histogram of image gradients

15 Parametric distribution Histogram of image gradients Use parametric model of sharp image statistics

16 Three sources of information. Reconstruction constraint: = Estimated sharp image Estimated blur kernel Input blurr image 2. Image prior: 3. Blur prior: Distribution of gradients Positive & Sparse

17 Variational Baesian method Based on work of Miskin & Macka 2 Keeps track of uncertaint in estimates of image and blur b using a distribution instead of a single estimate Helps avoid local maima and over-fitting

18 Variational Baesian method Objective function for a single variable Maimum a-posteriori (MAP) Score Variational Baes Piel intensit

19 Overview of algorithm. Pre-processing Input image 2. Kernel estimation - Multi-scale approach 3. Image reconstruction - Standard non-blind deconvolution routine

20 Preprocessing Input image Convert to grascale Remove gamma correction User selects patch from image Baesian inference too slow to run on whole image Infer kernel from this patch

21 Initialization Input image Convert to grascale Remove gamma correction User selects patch from image Initialize 33 blur kernel Blurr patch Initial image estimate Initial blur kernel

22 Inferring the kernel: multiscale method Input image Convert to grascale Remove gamma correction Loop over scales User selects patch from image Upsample estimates Variational Baes Initialize 33 blur kernel Use multi-scale approach to avoid local minima:

23 Image Reconstruction Input image Convert to grascale Remove gamma correction Loop over scales User selects patch from image Full resolution blur estimate Upsample estimates Variational Baes Initialize 33 blur kernel Non-blind deconvolution (Richardson-Luc) Deblurred image

24 Results on real images Submitted b people from their own photo collections Tpe of camera unknown Output does contain artifacts Increased noise Ringing Compares well to eisting methods

25 Original photograph

26 Blur kernel Our output

27 Original photograph Matlab s deconvblind

28 Original photograph

29 Matlab s deconvblind

30 Photoshop sharpen more

31 Our output Blur kernel

32

33 Original photograph

34 Our output Blur kernel

35 Original photograph

36 Our output Blur kernel

37 Matlab s deconvblind

38 Original photograph

39 Our output Blur kernel

40 Close-up of bird Original Our output

41 Original photograph

42 Blur kernel Our output

43 Image artifacts & estimated kernels Blur kernels Image patterns Note: blur kernels were inferred from large image patches, NOT the image patterns shown

44 Summar Method for removing camera shake from real photographs First method that can handle complicated blur kernels Uses natural image statistics Non-blind deconvolution currentl simplistic

45 Image Warping image filtering: change range of image g() = T(f()) f T f image warping: change domain of image g() = f(t()) f T f

46 Image Warping image filtering: change range of image g() = T(f()) T image warping: change domain of image g() = f(t()) T

47 Parametric (global) warping Eamples of parametric warps: translation rotation aspect affine perspective clindrical

48 Parametric (global) warping T p = (,) p = (, ) Transformation T is a coordinate-changing machine: p = T(p) What does it mean that T is global? Is the same for an point p can be described b just a few numbers (parameters) Let s represent T as a matri: p = Mp = M

49 Scaling Scaling a coordinate means multipling each of its components b a scalar Uniform scaling means this scalar is the same for all components: 2

50 Scaling Non-uniform scaling: different scalars per component: X 2, Y.5

51 Scaling Scaling operation: Or, in matri form: b a = = = b a scaling matri S What s inverse of S?

52 2-D Rotation (, ) (, ) θ = cos(θ) - sin(θ) = sin(θ) + cos(θ)

53 2-D Rotation This is eas to capture in matri form: = cos sin θ ( θ ) sin( θ ) ( ) ( ) R cos θ Even though sin(θ) and cos(θ) are nonlinear functions of θ, is a linear combination of and is a linear combination of and What is the inverse transformation? Rotation b θ For rotation matrices R = R T

54 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Identit? = = = 2D Scale around (,)? s s * * = = = s s

55 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Rotate around (,)? * cos * sin * sin * cos Θ + Θ = Θ Θ = Θ Θ Θ Θ = cos sin sin cos 2D Shear? sh sh + = + = * * = sh sh

56 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Mirror about Y ais? = = = 2D Mirror over (,)? = = =

57 22 Matrices What tpes of transformations can be represented with a 22 matri? 2D Translation? = + t = + t NO! Onl linear 2D transformations can be represented with a 22 matri

58 All 2D Linear Transformations Linear transformations are combinations of Scale, Rotation, Shear, and = Mirror Properties of linear transformations: Origin maps to origin Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition = a c be d g a c b d f i h k j l

59 Homogeneous Coordinates Q: How can we represent translation as a 33 matri? = + t = + t

60 Homogeneous Coordinates Homogeneous coordinates represent coordinates in 2 dimensions with a 3-vector

61 Homogeneous Coordinates Q: How can we represent translation as a 33 matri? A: Using the rightmost column: = t t Translation t t + = + =

62 Translation Eample of translation + + = = t t t t t = 2 t = Homogeneous Coordinates

63 Homogeneous Coordinates Add a 3rd coordinate to ever 2D point (,, w) represents a point at location (/w, /w) (,, ) represents a point at infinit (,, ) is not allowed 2 (2,,) or (4,2,2) or (6,3,3) Convenient coordinate sstem to represent man useful transformations 2

64 Basic 2D Transformations Basic 2D transformations as 33 matrices Θ Θ Θ Θ = cos sin sin cos = t t = sh sh Translate Rotate Shear = s s Scale

65 Affine Transformations Affine transformations are combinations of Linear transformations, and Translations Properties of affine transformations: Origin does not necessaril map to origin Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition = w f e d c b a w

66 Projective Transformations Projective transformations Affine transformations, and Projective warps Properties of projective transformations: Origin does not necessaril map to origin Lines map to lines Parallel lines do not necessaril remain parallel Ratios are not preserved Closed under composition = w i h g f e d c b a w

67 Matri Composition Transformations can be combined b matri multiplication Θ Θ Θ Θ = w s s t t w cos sin sin cos p = T(t,t ) R(Θ) S(s,s ) p

68 2D image transformations These transformations are a nested set of groups Closed under composition and inverse is a member

69 Recovering Transformations? T(,) f(,) g(, ) What if we know f and g and want to recover the transform T? Using correspondences How man do we need?

70 Translation: # correspondences? How man correspondences needed for translation? How man Degrees of Freedom? What is the transformation matri? T(,)? = p p p p M

71 Euclidian: # correspondences?? T(,) How man correspondences needed for translation+rotation? How man DOF?

72 Affine: # correspondences?? T(,) How man correspondences needed for affine? How man DOF?

73 Projective: # correspondences?? T(,) How man correspondences needed for projective? How man DOF?

74 Eample: warping triangles Given two triangles: ABC and A B C in 2D (2 numbers) Need to find transform T to transfer all piels from one to the other. What kind of transformation is T? How can we compute the transformation matri: T(,)? A B Source C A C B Destination = f e d c b a

75 warping triangles (Barcentric Coordinaes) (,) A B Source T C (,) (,) Inverse change of basis change of basis A T 2 B Destination Don t forget to move the origin too! C Ver useful in Graphics

76 Image warping T(,) f(,) g(, ) Given a coordinate transform (, ) = T(,) and a source image f(,), how do we compute a transformed image g(, ) = f(t(,))?

77 Forward warping T(,) f(,) g(, ) Send each piel f(,) to its corresponding location (, ) = T(,) in the second image Q: what if piel lands between two piels?

78 Forward warping T(,) f(,) g(, ) Send each piel f(,) to its corresponding location (, ) = T(,) in the second image Q: what if piel lands between two piels? A: distribute color among neighboring piels (, ) Known as splatting

79 Inverse warping T - (,) f(,) g(, ) Get each piel g(, ) from its corresponding location (,) =T - (, ) in the first image Q: what if piel comes from between two piels?

80 Inverse warping T - (,) f(,) g(, ) Get each piel g(, ) from its corresponding location (,) =T - (, ) in the first image Q: what if piel comes from between two piels? A: Interpolate color value from neighbors nearest neighbor, bilinear, Gaussian, bicubic