INTRODUCTION TO IMAGE PROCESSING (COMPUTER VISION)

Transcription

1 INTRODUCTION TO IMAGE PROCESSING (COMPUTER VISION) Revision: 1.4, dated: November 10, 2005 Tomáš Svoboda Czech Technical University, Faculty of Electrical Engineering Center for Machine Perception, Prague, Czech Republic

2 Course E383ZS (image processing part): Admin Course homepage 2/49 Both lectures and computer excercises will be tought at Karlovo namesti. Lectures, Thu 8:30-10:45, G102a Computer excercises Wed 11:00-12:30, K132 Grades (image processing part 12/24): Final exam: 9/12 Assignments: 3/12 Contact:

3 Why digital image processing? Digital images are ubiquitous. 3/49 digital photographs digital video digital tv broadcasting on-line resources on the web Firmware for acquisition devices. Cleaning noise. SW for digilabs...

4 What will we learn few examples Intensity corrections 4/49 Poorly exposed snapshot. Correction by histogram manipulation.

5 What will we learn few examples Homomorphic filtering Original image. Filtered image. Note improved sharpness and balanced tonality. 5/49

6 What will we learn few examples Image restoration 6/49 Blurred image. Restored imaged by inverse filtering.

7 What will we learn few examples Correction of perspective distortion 7/49 Distorted image. Correction by collineation.

8 HUMAN VISION vs. COMPUTER VISION Vision allows humans to perceive and understand the world surrounding them. Computer vision aims to imitate the effect of human vision by electronically perceiving image and understanding its content using computers. 8/49 Digital image = the input (understood intuitively), e.g., on the retina or captured by a TV camera. Image function f(x, y), f(x, y, t), or a matrix (after digitization).

9 EXAMPLES OF INPUT IMAGES 9/49

10 SEVERAL DISCIPLINES INDUCED 10/49 Digital image processing 2D static world, no image interpretation involved (rather independent of an application domain), signal processing techniques. Image analysis 2D world, image interpretation involved, i.e. image interpretation constitutes the crucial step. Computer vision the most general problem formulations, 3D world, interpretations, potentially dynamic (i.e., image sequence needed), ill posed tasks very ambitious.

11 LOW LEVEL vs. HIGH LEVEL PROCESSING Low level = image processing 11/49 Image data are not interpreted, i.e. semantics is not explored. Signal processing methods, e.g., 2D Fourier transformation. Same methods for wide class of problems. Images Images High level = image understanding, computer vision Interpretation to a specific application domain. Complex, artificial intelligence techniques, feedback. A tough problem. Often needs to be simplified. Images Description

12 ROLE OF INTERPRETATION, SEMANTICS 12/49 Interpretation: Observation Model Syntax Semantics Examples: Looking out of the window {rains, does not rain}. An apple on the conveyer belt {class 1, class 2, class 3}. Traffic scene seeking number plate of a car. Theoretical background: mathematical logic, theory of formal languages.

13 WHY IS COMPUTER VISION HARD? 6 REASONS Loss of information in 3D 2D due to perspective transformation (mathematical abstraction = pinhole model). 13/49 Measured brightness is given by a complicated image formation physics. Radiance ( brightness) depends on light sources intensity and positions, observer position, surface local geometry, and albedo. Inverse task is ill-posed. Inherent presence of noise as each real world measurement is corrupted by noise. A lot of data Sheet A4, 300 dpi, 8 bit per pixel = 8.5 Mbytes. Non-interlaced video , RGB (24 bit) = 225 Mbits/second. Interpretation needed (as discussed above). Local window vs. need for global view

14 OBJECT RECOGNITION HIERARCHY OF REPRESENTATIONS 14/49 Object or scene 2D image Digital image from objects to images from images to features Regions Edgels Scale Orientation Texture Image with features from features to objects Objects understanding objects

15 IMAGE Image - understood intuitively; image on the retina, captured by a TV camera. 15/49 Image function f(x, y), f(x, y, t). Outcome of the perspective projection. Y point in the 3D scene P(x,y,z) X y' x' y x Z f image plane x = x f z, y = y f z.

16 IMAGE FUNCTION, 2-DIMENSIONAL SIGNAL 16/49 Monochromatic static image f(x, y), where (x, y) are coordinates in a plane with domain R = {(x, y), 1 x x m, 1 y y n } ; f is image function value ( brightness, density of a transparent object, distance to observer, etc.) (Natural) 2D Images: Thin specimen in optical microscope, image of a character on a paper, finger print, a single cut from a tomograph, etc.

17 IMAGE FUNCTION DIGITAL IMAGE From continuous to discrete space. 17/49 Sampling of the image domain. Selection of dicrete points. Quantization of the image range. selection of disrete values. Usual representation = matrix. f(x, y) f(r, c). Pixel = Picture element. Sampling 50x50, Quantization to 32 values rows columns

18 SAMPLING 18/49 Two involved problems: 1. Arrangement of the sampling points. (a) (b) 2. Distance between sampling points (Shannon sampling theorem).

19 dots per inch [dpi] Sampling vs. Quantization 19/49 frames per second [fps] 24-bit color 256 gray levels Sampling is usually described as frequency or (frame)rate spacing density Quantization is usually described by the number of bits (bytes) per sample number of discrete values

20 SAMPLING EXAMPLE 1 20/49 Original

21 SAMPLING EXAMPLE 2 21/49 Original

22 SAMPLING EXAMPLE 3 22/49 Original

23 QUANTIZATION EXAMPLE 1 23/49 Original 256 levels 64 levels

24 QUANTIZATION EXAMPLE 2 24/49 Original 256 levels 16 levels

25 QUANTIZATION EXAMPLE 3 25/49 Original 256 levels 4 levels

26 QUANTIZATION EXAMPLE 4 26/49 Original 256 levels 2 levels

27 Resolution is the ability to distinguish between details. It is not the number of pixels. 27/49 Both images have the same number of pixels but different resolution. The resolution is more related to what we can reconstruct from the image.

28 DISTANCE 28/49 Function D is called the distance iff D(p, q) 0, D(p, q) = D(q, p), D(p, r) D(p, q) + D(q, r), particularly D(p, p) = 0 (identity). (symmetry). (triangular inequality).

29 Several definitions of distance in a square raster 29/49 Euclidean distance (as the crow flies) D E ((x, y), (h, k)) = (x h) 2 + (y k) 2. City block distance (also called Manhattan distance) D 4 ((x, y), (h, k)) = x h + y k. Chessboard distance D 8 ((x, y), (h, k)) = max{ x h, y k } D E D 4 D 8

30 4-neighbourhood and 8-neighbourhood 30/49 A set consisting of the pixel itself and its neighbours of distance 1.

31 CROSSING LINES PARADOX 31/49

32 BINARY IMAGE & RELATION being contiguous 32/49 black objects & white background Neigbouring pixels are contiguous. Two pixels are contiguous if there is a path between them consisting of neigbouring pixels.

33 REGION = compact set 33/49 Relation being contiguous is reflexive, symmetric, and transitive, i.e. this is an equivalence relation. Thus it decomposes the set of object pixels into equivalence classes = regions.

34 A FEW CONCEPTS 34/49 Boundary (of a region) vs. edge vs. edgel. Inner and outer boundary. Convex hull, lakes, bays.

35 Histogram of Image Intensities aka Image Histogram 35/49 Histogram of image intensities is an estimate of probability density image histogram of intensities

36 Histogram Equalization 36/49 The Aim: Increase contrast for a human observer. Normalize intensities for e.g., automatic image comparison original histogram histogram after equalization

37 Increased contrast after histogram equalization 37/49 original image... after equalization

38 Histogram equalization derivation 38/49 Input: histogram H(p) of the image with gray leveles p = p 0, p k. Aim: find a monotonic pixel brightness transformation q = T (p), such that the desired output histogram G(q) is uniform over the whole output brightness scale q = q 0, q k. The monotonicity of the transformation implies: k G(q i ) = k H(p i ). i=0 i=0 Equalized histogram uniform density G(q) = N 2 q k q 0.

39 Histogram equalization derivation II The exactly uniform histogram may be obtained only in continuous space. q q q 0 G(s) ds = N 2 ds = q 0 q k q 0 N 2 (q q 0 ) q k q 0 = p p 0 H(s) ds. p p 0 H(s) ds. p p 0 H(s) ds. N 2 (q q 0 ) = (q k q 0 ) q = T (p) = q k q 0 N 2 p p p 0 H(s) ds. p 0 H(s) ds + q 0. 39/49

40 Histogram equalization derivation III Continous space distribution function 40/49 q = T (p) = q k q 0 N 2 p p 0 H(s) ds + q 0. Dicrete space cumulative histogram q = T (p) = q k q 0 N 2 p i=p 0 H(i) + q 0.

41 More intensity transformations original 41/

42 More intensity transformations brightness q = p + const 42/

43 More intensity transformations original 43/

44 More intensity transformations contrast q = p const 44/

45 More intensity transformations original 45/

46 More intensity transformations gamma corrected q = p γ 46/

47 More intensity transformations original 47/

48 More intensity transformations histogram equalization q = p i=p 0 H(i) 48/

49 Questions? 49/49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72 Object or scene 2D image Digital image from objects to images from images to features Regions Edgels Scale Orientation Texture Image with features from features to objects Objects understanding objects

73 Y point in the 3D scene P(x,y,z) X y' x' y x Z f image plane

74

75

76 Sampling 50x50, Quantization to 32 values rows columns

77 (a) (b)

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94 D E D 4 D 8

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122