Achim J. Lilienthal Mobile Robotics and Olfaction Lab, AASS, Örebro University

Transcription

1 Achim J. Lilienthal Mobile Robotics and Olfaction Lab, Room T1227, Mo, o'clock AASS, Örebro University (please drop me an in advance) 1

2 4. Admin Course Plan Rafael C. Gonzalez, Richard E. Woods (3rd edition, 2008) Digital Image Processing 2

3 4. Admin Lab Groups o Hesho Rashid + Rasha Zaki G1 o Benny Frost G2 o Amanda Boström G3 o Eric Lundberg + Tom Olsson G4 o Felice Sallustio + Paolo Cesana G5 o Björn Nyström G6 o Jordi Moragrega G7 3

4 4.!!!!!!!!! Pre-Class Reading!!!!!!!!! Pre-Class Reading Schedule o Class 1 "Course Introduction" (Nov 17, 2014) o Class 2 "Introduction" (Nov 18, 2014)» Gonzalez/Woods Chapter 1 "Introduction"» Gonzalez/Woods Chapter 2 "Fundamentals"» (Lecture Notes from 2012) o Class 3 "Spatial Filtering" (Nov 20, 2014)» Gonzalez/Woods Chapter 3 "Intensity Transformations and Spatial Filtering"» (Lecture Notes from 2012) o Class 4 "Bilateral Filtering/Fourier Domain" (Nov 25, 2014)» "A Gentle Introduction to Bilateral Filtering and its Applications", Sylvain Paris, Pierre Kornprobst, Jack Tumblin, and Frédo Durand, SIGGRAPH 2008» "Bilateral Filtering for Gray and Color Images", C. Tomasi, R. Manduchi, Proc. Int. Conf. Computer Vision» Gonzalez/Woods Chapter 4 "Filtering in the Frequency Domain"» (Lecture Notes from 2012) 4

5 Contents 1. Image Enhancement in the Spatial Domain 2. Grey Level Transformations 3. Histogram Processing 4. Operations Involving Multiple Images Applications People Tracking 5. Spatial Filtering 5

6 1 Image Enhancement in the Spatial Domain 6

7 1. Image Enhancement in the Spatial Domain Image Enhancement o image processing o the result is supposed to be "more suitable"» "more suitable" according to a certain application more suitable for visual interpretation 7

8 1. Image Enhancement in the Spatial Domain We want to create an image which is "better" in some sense o helps visual interpretation (brightening, sharpening ) subjective o pre-processing for a subsequent image analysis algorithm performance metric (performance of a task) o make the image more "specific" application dependent T f(x,y) g(x,y) original image (or set of images) new image 8

9 1. Image Enhancement in the Spatial Domain Spatial Domain versus Frequency Domain o spatial domain» direct manipulation of the pixels discussed in this lecture» two types of transformations in the spatial domain: pixel brightness transformations, point processing (depend only on the pixel value itself) spatial filters, local transformations or local processing (depend on a small neighbourhood around the pixel) o frequency domain: modifications of the Fourier transform» discussed in coming lectures 10

10 1. Image Enhancement in the Spatial Domain Transformations in the Spatial Domain g ( x, y) = T[ f ( x, y)] o standard approach: T is applied to a sub-image centred at (x,y) o sub-image is called mask (kernel, filter, template, window) o mask processing or filtering o T can operate on a set of images 11

11 1. Image Enhancement in the Spatial Domain Transformations in the Spatial Domain g ( x, y) = T[ f ( x, y)] o fill new array with weighted sum of pixel values from the locations surrounding the corresponding location in the image using the same set of weights each time 12

12 2 Gray Level Transformations 13

13 2. Grey Level Transformations Grey Level Transformations o simplest case: each pixel in the output image depends only on the corresponding pixel in the input image o 1x1 neighbourhood (point processing) o example: contrast stretching s = T (r) s = T (r) 14

14 2. Grey Level Transformations Grey Level Transformations contrast stretching thresholding 15

15 2. Grey Level Transformations Grey Level Transformations f = imread('bubbles.tif'); fp = imadjust(f, [ ], [ ], 0.5); imshow(fp); o imadjust» parameters always specified in [0,1]» values below 0.1 clipped to 1.0» values above 0.9 clipped to 0.0» image intensity reversed since 0.0 <

16 2. Grey Level Transformations Grey Level Transformations f = imread('bubbles.tif'); fp = imadjust(f, [ ], [ ], 0.5); imshow(fp); o imadjust» parameters always specified in [0,1]» values below 0.1 clipped to 1.0» values above 0.9 clipped to 0.0» image intensity reversed since 0.0 < 1.0» gamma function parameter < 1 g = f γ 17

17 2. Grey Level Transformations Grey Level Transformations f = imread('bubbles.tif'); fp = imadjust(f, [ ], [ ], 0.5); imshow(fp); fp = imadjust(f, [ ], [ ], 3); 18

18 2. Grey Level Transformations Grey Level Transformations f = imread('bubbles.tif'); fp = imadjust(f, [ ], [ ], 0.5); imshow(fp); fp = imadjust(f, [ ], [ ], 3); o imadjust» gamma function parameter > 1 g = f γ 19

19 2. Grey Level Transformations Contrast Stretching o piecewise linear function o power law transformation (gamma transformation) γ s = cr 20

20 2. Grey Level Transformations Common Grey Level Transformations (Single Image) o linear» identity» inverse (negative) o power law» n. power» n. root o logarithmic 21

21 2. Grey Level Transformations Common Grey Level Transformations (Single Image) o inverse transform 22

22 2. Grey Level Transformations Common Grey Level Transformations (Single Image) o linear» identity» inverse o piecewise linear o power law (gamma)» n. power» n. root o logarithmic... with more than one input image o sum, mean o transformation based on statistical operations (variance, t-test ) 24

23 3 Histogram Processing 25

24 3. Histogram Processing Grey Scale Histogram o shows the number of pixels per grey level f = imread('bubbles.tif'); imhist(f); % displays the histogram % histogram display default 27

25 3. Histogram Processing Grey Scale Histogram o shows the number of pixels per grey level f = imread('bubbles.tif'); h1 = imhist(f); % default number of bins = 256 imhist(f,8); % number of bins = 8 28

26 3. Histogram Processing Grey Scale Histogram o shows the number of pixels per grey level f = imread('bubbles.tif'); h1 = imhist(f); % default number of bins = 256 h = imhist(f,16); % number of bins = 16 hn = h/numel(f); % normalized histogram % numel num. of elements (pixels) bar(hn) % normalized histogram 29

27 3. Histogram Processing Grey Scale Histogram o neutral transform 31

28 3. Histogram Processing Grey Scale Histogram o neutral transform o inverse transform 32

29 3. Histogram Processing Grey Scale Histogram o neutral transform o inverse transform o logarithmic transform 33

30 3. Histogram Processing Histogram Equalization o contrast / brightness adjustments sometimes need to be automatised o "optimal" contrast for an image? flat histogram 37

31 3. Histogram Processing Histogram Equalization o consider the continuous case: s, r [0,1] o probability density functions (PDFs) of s and r are related by gray levels as random variables! s = T (r) p s ( s) = p r ( r) dr ds = p r ( r) 1 T ( r) o transformation function = cumulative density function (CDF) ds dr r T ( r) p r ( ω) dω 0 r d = T ( r) = pr ( ω) dω = pr ( r) p s ( s) = 1 dr 0 38

32 3. Histogram Processing Histogram Equalization o discrete case pr rk ) = nk n ( s = = = k T ( rk ) pr ( rj ) j= 0 o does not generally produce a uniform PDF o tends to spread the histogram o enables automatic contrast stretching k k j= 0 n j n 39

33 3. Histogram Processing Histogram Equalization CDF 40

34 3. Histogram Processing Histogram Equalization 41

35 3. Histogram Processing Histogram Equalization f = imread('bubbles.tif'); g = histeq(f, 256); imshow(g); f = imread('bubbles.tif'); g = histeq(f, 4); % 4 output levels imshow(g); 42

36 3. ConcepTest ImageEnhancement Image Enhancement CT4 o Transformation by using adaptive/localized histogram equalization over a rectangular region around pixels (adaptive/local Histogram Equalization) what happens in case of a smaller region? original image global histogram equalization 44

37 3. ConcepTest ImageEnhancement Image Enhancement CT4 o Transformation by using adaptive/localized histogram equalization over a rectangular region around pixels (adaptive/local Histogram Equalization) what happens in case of a smaller region? original image local histogram equalization (radius = 100) 45

38 3. ConcepTest ImageEnhancement Image Enhancement CT4 o Transformation by using histogram equalization over a rectangular region around pixels (adaptive/local Histogram Equalization) what happens in case of a smaller region? original image local histogram equalization (radius = 50) 46

39 3. ConcepTest ImageEnhancement Image Enhancement CT4 o Transformation by using histogram equalization over a rectangular region around pixels (adaptive/local Histogram Equalization) what happens in case of a smaller region? original image local histogram equalization (radius = 25) 47

40 3. ConcepTest ImageEnhancement Image Enhancement CT4 o Transformation by using histogram equalization over a rectangular region around pixels (adaptive/local Histogram Equalization) what happens in case of a smaller region? original image local histogram equalization (radius = 12) 48

41 4 Image Enhancement in the Spatial Domain Operations Involving Multiple Images 49

42 4. Operations Involving Multiple Images Operations Between Two or More Images o image subtraction» Arteriography» tracking 50

43 4. Operations Involving Multiple Images Image Subtraction o DSA (Digital Subtraction Arteriography) mask image live image DSA image 51

44 4. Operations Involving Multiple Images Operations Between Two or More Images o image subtraction» Arteriography» tracking o image averaging (GW 3.4.2)» noise reduction» background modeling image subtraction 52

45 4. Operations Involving Multiple Images Image Subtraction o tracking with a stationary camera background image live image difference image 53

46 4 Applications People Tracking 54

47 4. Introduction Applications Imaging in the Visible and Infrared Bands o person tracking in mobile robotics 55

48 4. 56

49 4. Example: Person Tracking in Mobile Robotics PeopleBoy (ActiveMedia PeopleBot) thermal cam: pixels 15 Hz colour camera pixels 15 Hz 57

50 4. Person Tracking in Mobile Robotics 4 Thermal Camera o humans have a distinctive thermal profile o not influenced by changing lighting conditions o works in darkness Thermo Tracer TH7302, NEC visible range: 24 C to 36 C 58

51 4. Person Tracking in Mobile Robotics 4 Thermal Camera o humans have a distinctive thermal profile o not influenced by changing lighting conditions o works in darkness Colour Camera o improves accuracy o helps to resolve occlusions o dynamical colour model 59

52 4. Person Tracking in Mobile Robotics Person Tracking o no occlusions 60

53 4. Person Tracking in Mobile Robotics 4 Person Tracking o distinguish persons using an elliptic contour model 61

54 4. Person Tracking in Mobile Robotics Person Tracking Measurement Model o elliptic contour model! applicable if the person is far away! applicable if side-view is visible 62

55 4. Person Tracking in Mobile Robotics Person Tracking o no occlusions 63

56 4. Person Tracking in Mobile Robotics Person Tracking o thermal and colour information o occlusions 64

57 4. Operations Involving Multiple Images Operations Between Two or More Images o image subtraction» Arteriography» tracking o image averaging (GW 3.4.2)» noise reduction» background modeling image subtraction o time constant of averaging? (stability plasticity dilemma)» recency weighted averaging» sample-based background modelling 65

58 4. Operations Involving Multiple Images Sample-Based Background Modelling o stationary camera o no assumptions about the distribution required o not sensitive to outliers (robust statistics) Dynamic Sample Set Representation o representation as a set of measurements (samples) o sample set S(t i ) evolves by replacing samples randomly» u n samples replaced between two time steps» probability to have been added n t timesteps before: p ln [( 1 u ) n ] ( t) u e t ln[ 2] = (update rate u) t / 2 = ln[ 1 u] 1 = ln[ 2] λ 66

59 4. Operations Involving Multiple Images Interpretation of a Dynamic Sample Set! dynamic sample sets correspond to a time scale t 1/ 2 p u = ln 2 = c t T [ ] n ct c t : time constant t : time interval since the last frame p u : sample set update probability Deriving Foreground Probability Images o estimate background distribution» calculate kernel estimator (Parzen window)» background probability according to intensity density estimate o foreground probability = 1 - background probability 67

60 4. Operations Involving Multiple Images Foreground Probability Images t 1/2 = 1.5 s, σ = 20 t 1/2 = 115 s, σ = 20 68

61 4. Person Tracking in Mobile Robotics Person Tracking o stationary webcam o sample-based background subtraction (motion heat) o occlusions 69