Image Analysis. Rasmus R. Paulsen DTU Compute. DTU Compute

Transcription

1 Rasmus R. Paulsen Plenty of slides adapted from Thomas Moeslunds lectures

2 Lecture 8 Geometric Transformation 2, Technical University of Denmark

3 What can you do after today? Compute the translation of a point Compute the rotation of a point Compute the scaling of a point Compute the shearing of a point Construct a translation, rotation, scaling, and shearing transformation matrix Use transformation matrices to perform point transformations Describe the difference between forward and backward mapping Use bilinear interpolation to compute the interpolated value at a point Transform an image using backward mapping and bilinear interpolation Compute a line profile using bilinear interpolation 3, Technical University of Denmark

4 Geometric transformation Moving and changing the dimensions of images Why do we need it? 4, Technical University of Denmark

5 Change detection Patient imaged before and after surgery What are the changes in the operated organ? Patient can not be placed in the exact same position in the scanner Before surgery After surgery 5, Technical University of Denmark Bachelor project: Image Guided Surgery Planning

6 Image Registration Change one of the images so it fits with the other Formally Template image Reference image Template is moved to fit the reference Compute the difference Template Reference 6, Technical University of Denmark

7 Geometric Transform The pixel intensities are not changed The pixel values just change positions Same value Different place 7, Technical University of Denmark

8 Different transformations Translation Rotation Scaling Shearing Advanced transformations 8, Technical University of Denmark

9 Translation The image is shifted both vertically and horizontally 9, Technical University of Denmark

10 Rotation The image is rotated around the centre or the upper left corner Remember to use degrees and radians correctly Matlab uses radians Degrees easier for us humans 10, Technical University of Denmark

11 Rotation coordinate system x y 11, Technical University of Denmark

12 Rigid body transformation Translation and rotation Rigid body = stift legeme Angles and distances are kept 12, Technical University of Denmark

13 Scaling The size of the image is changed Scale factors X-scale factor S x Y-scale factor S y Uniform scaling: S x = S y 13, Technical University of Denmark

14 Similarity transformation Translation, rotation, and uniform scaling Angles are kept Distances change 14, Technical University of Denmark

15 Shearing Pixel shifted horizontally or/and vertically Shearing factors X-shear factor Y-shear factor B x B y Is less used than translation, rotation, and scaling 15, Technical University of Denmark

16 Transformation Mathematics f (x,y) Transformation of positions Structure found at position (x,y) in the input image f Now at position (x,y ) in output image g A mapping function is needed g (x,y ) 16, Technical University of Denmark

17 Translation mathematics The image is shifted both vertically and horizontally (x,y) (x,y ) 17, Technical University of Denmark

18 Matrix notation Coordinates in column matrix format 18, Technical University of Denmark

19 Translation mathematics in matrix notation The image is shifted both vertically and horizontally 19, Technical University of Denmark

20 Scaling The size of the image is changed Scale factors X-scale factor S x Y-scale factor S y Uniform scaling: S x = S y SS xx = 1.2 SS yy = , Technical University of Denmark

21 Matrix multiplication details Is equal to: 21, Technical University of Denmark

22 Transformation matrix Can be written as Where is a transformation matrix 22, Technical University of Denmark

23 x Rotation A rotation matrix is used y 23, Technical University of Denmark

24 Shearing Pixel shifted horizontally or/and vertically New x value depends on x and y 24, Technical University of Denmark

25 Affine transformation The collinearity relation between points, i.e., three points which lie on a line continue to be collinear after the transformation 25, Technical University of Denmark

26 Combining transformations Scaling Suppose you first want to rotate by 5 degrees and then scale by 10% Rotation How do we combine the transformations? 26, Technical University of Denmark

27 Combining transformations Combination is done by matrix multiplication Scaling Rotation Combined 27, Technical University of Denmark

28 Combining transformations Compact notation Scaling Rotation Combined 28, Technical University of Denmark

29 Combining transforms A B C D E The point (x,y)=(5,6) is transformed. First with: and then with: The result is: A) 1 B) 2 C) 3 D) 4 E) 5 29, Technical University of Denmark

30 What do we have now? We can pick a position in the input image f and find it in the output image g We can transfer one pixel what about the whole image? f g 30, Technical University of Denmark

31 Solution 1 : Input-to-output Run through all pixel in input image Find position in output image and set output pixel value Scaling example x x Missing value! 3 y f 3 y g 31, Technical University of Denmark

32 Input-to-Output The input to output transform is not good! It creates holes and other nasty looking stuff What do we do now? 32, Technical University of Denmark

33 Some observations We want to fill all the pixels in the output image Not just the pixels that are hit by the pixels in the input image Run through all pixels in the output image? Pick the relevant pixels in the input image? We need to go backwards From the output to the input x 2 3 y g 33, Technical University of Denmark

34 Inverse transformation We want to go from the output to the input Scaling example inverse x? x y f 3 y g 34, Technical University of Denmark

35 Output-to-input transformation Backward mapping Run through all pixel in output image Find position in input image and get the value x x y f 35, Technical University of Denmark 3 y What is the value here? g

36 Bilinear Interpolation Value? The value is calculated from 4 neighbours The value is based on the distance to the neighbours Lots of 89! Not so much , Technical University of Denmark

37 Linear Interpolation vv = tttt + 1 tt aa 1 t V=? a b 37, Technical University of Denmark

38 Bilinear interpolation 38, Technical University of Denmark

39 Bilinear interpolation 12 Bilinear interpolation is used to create a line profile from an image. In a given point (x,y) = (173.1, 57.8), the four nearest pixels are: A) 1 B) 2 C) 3 D) 4 E) 5 What is the interpolated value in the point: A B C D E 39, Technical University of Denmark

40 Output-to-input transformation Backward mapping Run through all the pixel in the output image Use the inverse transformation to find the position in the input image Use bilinear interpolation to calculate the value Put the value in the output image 40, Technical University of Denmark

41 Inverse transformation Scaling We can calculate the inverse transformation for the scaling What about the others? Inverse 41, Technical University of Denmark

42 General inverse transformation Affine transformation The transformation is expressed as a transformation matrix A The matrix inverse of A gives the inverse transformation Inverse transformation Where 42, Technical University of Denmark

43 General inverse transformation You can create an arbitrary sequence of rotation, scaling, and shearing Scaling Rotation Scaling Combined: Inverse: 43, Technical University of Denmark

44 Transformation A B C D E A) 1 The point (x,y) = (45, 23) is transformed using: B) 2 C) 3 D) 4 E) 5 And the result is translated with (-15, 20). The result is: 44, Technical University of Denmark

45 Profile Analysis Sample the pixel values under the profile From start point to end point 45, Technical University of Denmark

46 Profile Analysis p s Length of profile Unit vector p e Point on profile 46, Technical University of Denmark

47 Profile Analysis p s Point on profile Sample all points on profile using bi-linear interpolation p e 47, Technical University of Denmark

48 Profile Analysis ps p e p s p e 48, Technical University of Denmark

49 DTU Sign Challenge exercise 8 49, Technical University of Denmark

50 Data 34 photos with ground truth (GT) 50, Technical University of Denmark

51 Goal To create a function that can locate signs Should create a label image Label 0: Background Label 1: Sign 1 Label 2: Sign 2 51, Technical University of Denmark

52 Supplied scripts and functions Some scripts and functions supplied MyDTUSignFinder.m Should be renamed into a new cool name You should also make a commercial (a powerpoint slide or similar) for your amazing method! 52, Technical University of Denmark

53 How do we measure performance? 53, Technical University of Denmark

54 DICE Scores DDDDDDDD = 2(AA BB) AA + BB A: area of ground truth B: area of your result AA BB: area of intersection/overlap Residuals is the difference between ground truth and your result 54, Technical University of Denmark

55 DICE Scores DDDDDDDD = 2(AA BB) AA + BB 1 is perfect No residuals 0 is really bad Only residuals 55, Technical University of Denmark

56 Multiple DICE scores Average DICE score over found matches Penalises Too many sign candidates Too few sign candidates 56, Technical University of Denmark

57 57, Technical University of Denmark

58 58, Technical University of Denmark

59 Epic_SpleenMasterOrz9000Extreme_supre me_premium_deluxe_edition 59, Technical University of Denmark

60 60, Technical University of Denmark

61 Next week Company presentations Visiopharm JLI Vision Dalux IH Food 61, Technical University of Denmark