Histograms. h(r k ) = n k. p(r k )= n k /NM. Histogram: number of times intensity level rk appears in the image

Transcription

1 Histograms h(r k ) = n k Histogram: number of times intensity level rk appears in the image p(r k )= n k /NM normalized histogram also a probability of occurence 1

2 Histogram of Image Intensities Create bins of intensities and count number of pixels at each level Normalized (divide by total # pixels) Frequency Grey level value 2

3 Histograms and Noise What happens to the histogram if we add noise? g(x, y) = f(x, y) + n(x, y) 3

4 Noise reduction with pixelwise addition Many noisy images of the same scene Averaging helps remove noise Why? Under what conditions? R. C. Gonzalez & R. E. Woods

5 MATLAB example for averaging 5

6 Gamma correction s = cr γ R. C. Gonzalez & R. E. Woods

7 Gamma transformations R. C. Gonzalez & R. E. Woods

8 Gamma transformations What happens to the histogram? R. C. Gonzalez & R. E. Woods

9 1. MATLAB example for gamma correction 2. Can we fix skewed histograms automatically? 9

10 Automatic histogram equalization Cionsider image intensity as a continuous valued random variable r in the interval [ 0, L-1 ] with pdf p r (r) What kind of histograms do we want? Find an intensity transformation s=t(r) such that the pdf p s (s) is uniform in the interval [ 0, L-1 ] How are the pdf s of r and s related? s = T(r) p s ( s) = p (r) dr r ds 10

11 Automatic histogram equalization Consider the transformation ( range [0, 1] ) r s = p ( w)dw r 0 PDF of r s CDF of s Slope = 1 r Area given by integral above s What is the pdf of s? What to do for other intensity range? 11

12 Automatic histogram equalization Consider the transformation s = L 1 ( ) p r w r 0 ( )dw Compute ds/dr ds ( ) d dr dr = L 1 # $% r 0 p r ( & w)dw Leibniz s rule '( = ( L 1 ) p ( r) r Verify pdf for s is uniform p s ( s) = p (r) dr r ds = p r (r) 1 L 1 ( ) p r (r) = 1 L 1 0 s L 1 12

13 S 0 =1, S 1 =3, S 2 =5, S 3 =6, S 4 =6, S 5 =7, S 6 =7, S 7 =7 (Discrete) histogram equalization s k = L 1 Example with L=8 s 0 =7x0.19=1.33 > 1 s 1 =7x( ) = 3.08 > 3. k j =0 ( ) ( ) p r r j S k

14 MATLAB example histogram equalization 14

15 Histogram equalization examples

16 Histogram Equalization 16

17 Tuning Down Hist. Eq. Transformation is weighted combination of CDF and identity with parameter alpha α = 0.0 α = 0.2 α = 0.4 α = 0.6 α = 0.8 α17 = 1.0

18 Adaptive Histogram Equalization (AHE) 18

19 How to do adaptive histogram equalization Given a local image block, either Do histogram equalization for that block, OR Standardize the mean and variance of each block g(x, y) = f (x, y) m B for ( x, y) B σ B Repeat over all blocks (non-overlapping) Or, Sliding window approach g(x, y) = f (x, y) m B(x,y) σ B(x,y)

20 Histogram statistics Image f(x,y), histogram p(r i ) Mean intensity m = L 1 r p( r ) = 1 i i MN i=0 M 1 N 1 x=0 y=0 f (x, y) Second central moment (intensity variance) σ 2 = L 1 ( r m) 2 p( r ) = 1 i i MN i=0 M 1 x=0 N 1 y=0 ( f (x, y) m) 2

21 Local histogram statistics B xy a neighborhood centered around (x,y) We can compute the intensity histogram and its statistics (mean, variance, etc.) limited to this region L 1 m = r p ( r ) = 1 Bxy i Bxy i i=0 B xy ( x,y) B xy f ( x, y) σ 2 Bxy L 1 i=0 ( ) 2 p Bxy r i = r i m Bxy ( ) = 1 B xy (x,y) B xy ( f (x, y) m ) 2 Bxy

22 Local histogram statistics For instance, lets define S xy as a 11 x 11 neighborhood centered at (x,y). Then m Bxy = 1 σ 2 Bxy 121 = u= 5 v= 5 5 u= 5 5 v= 5 f (x + u, y + v) ( f (x + u, y + v) m ) 2 Bxy Have to take care at image boundaries

23 A faster alternative instead Compute statistics at every pixel Divide image into blocks. Then compute statistics Much more efficient Can create a blocky effect in the output image

24 AHE Gone Bad What can we do about this? 24

25 Effect of Window Size Orig 10x10 25x25 50x50 25

26 AHE Application: Microscopy Imaging Original AHE 26

27 Histogram matching Histogram equalization aims for an uniform histogram for the output image Sometimes we might want to specify different histograms as the target s = T (r) = L 1 ( ) p r w r ( )dw s = G(z) = L 1 0 z 0 ( ) p z t ( )dt r p r (r) Input image s histogram s p s (s) Uniform histogram z = G 1 ( T ( r) ) z = G 1 (s) z p z (z) Target histogram

28 Invertible (one-to-one) transformations We need to be able to take the inverse of G(z) G(z) has an inverse if it is strictly z 0 s = G(z) = ( L 1) p ( t)dt z monotonically increasing as a function of z This is true if p z (z)>0 for all z

29 Histogram matching (discrete) 1. Compute and store in a table G( z ) = L 1 q q i=0 ( ) p ( z ) z i 2. Compute s k = T(r k ) = L 1 ( ) p r r j k ( ) 3. Create mapping from s k to z q (inverse of G) by finding the closest value in the table stored from step 1 to all s k. 4. Apply mapping created in step 3 to histogram equalized input image i=0

30 Example 1 G Inverse of G 3 2 Did this one before S 0 =1, S 1 =3, S 2 =5, S 3 =6, S 4 =6, S 5 =7, S 6 =7, S 7 =7 4. r k -> s k -> z q

31 Colorization example Match histograms of the gray scale input image to histograms of the Red, Green and Blue channels of a reference image individually to create 3 new images. Use these new images as red, green blue channels to create the output color image. To get realistic results input and reference image contents should be similar.

32 MATLAB code Given a gray scale image I2gray and a reference image I1color Uses image processing toolbox command imhistmatch I2red = imhistmatch(uint8(i2gray),i1color(:,:,1)); I2blue = imhistmatch(uint8(i2gray),i1color(:,:,3)); I2green = imhistmatch(uint8(i2gray),i1color(:,:,2)); I2colorized(:,:,1)=I2red; I2colorized(:,:,2)=I2green; I2colorized(:,:,3)=I2blue;

33 Colorization example

34 Other uses of histograms So far we used histograms for image enhancement purposes Quantization Reducing the number of gray levels or colors in an image for efficient storage Segmentation Image segmentation is the process of subdividing an image into its constituent regions or objects.

35 What is image segmentation? Image segmentation is the process of subdividing an image into its constituent regions or objects. Example segmentation with two regions: Input image intensities Segmentation output 0 (background) 1 (foreground)

36 Formal definition of segmentation R: set of all pixels in given image Segmentation into n regions R1,R2,...Rn Two important properties Complete Mutually exclusive

37 Two general approaches to segmentation Discontinuity based Partition an image based on abrupt changes of intensity. Find pixels which correspond to these abrupt changes, these will be the boundaries between regions. Edge detection is an example of this type of approach to segmentation Similarity based Group pixels into regions of similar intensity Thresholding is an example of this type of approach to segmentation

38 Global thresholding Input image f(x,y) intensities Segmentation output g(x,y) 0 (background) 1 (foreground) How can we choose T? Trial and error Use the histogram of f(x,y)

39 Role of noise T=120

40 Low signal-to-noise ratio T=140

41 Low signal-to-noise ratio T=140 How can we choose T? Trial and error Use the histogram of f(x,y) Automatically Otsu s method

42 Otsu s method Define an quantitative measure of goodness of a threshold Maximize over all possible thresholds k

43 Otsu s method 2 classes Define an quantitative measure of goodness of a threshold Difference of 2 class means Difference of class means from global mean σ 2 b (k) = P ( 1 k) m 1 k = P 1 k ( )P 2 k P 1 m 1 k ( ( ) m ) 2 G + P ( 2 k) m ( 2 k) m G ( ) 2 ( ) m ( 1 k) m ( 2 k) ( k) = p(i), P 2 k i=0 ( k) = 1 k P 1 ( ) k L 1 i=k+1 ( ) = p(i) ip(i), m 2 k i=0 ( ) = ( ) 2 Maximize over all possible thresholds k P 2 1 k ( ) L 1 ip(i), m G = ip(i) i=k+1 L 1 i=0

44 Otsu s method multiple classes ( ) 2 σ b 2 (k 1,..., k n 1 ) = P i m i m G k 1,...,k n 1 n i=1 argmax σ b 2 (k 1,..., k n 1 )

45 Effect of illumination on image histogram Images Histograms f x g = h R. C. Gonzalez & R. E. Woods Original Illumination Final image image image 45

46 Some applications Microscopy image showing bright cells on a dark background Find number of cells Time-lapse microscopy images with lighting varying over time Track number of cells over time

47 Local thresholding One simple way Subdivide image into blocks Assumes every block has a portion of foreground and background R. C. Gonzalez & R. E. Woods

48 Moving mean At every pixel (x,y) we can choose a threshold based on the mean m(x,y) of a local window This is very useful for adapting to changes in illumination Can be problematic if the window at (x,y) contains only foreground or only background R. C. Gonzalez & R. E. Woods

49 Moving mean - another example At every pixel (x,y) we can choose a threshold based on the mean m(x,y) of a local window This is very useful for adapting to changes in illumination Can be problematic if the window at (x,y) contains only foreground or only background R. C. Gonzalez & R. E. Woods

50 Some Extra Things Gaussian/normal distribution Weighted means 50

51 Gaussian Distribution Normal or bell curve Two parameters µ = mean, σ = standard deviation 51

52 Gaussian Properties Best fitting Gaussian to some data is gotten by mean and standard deviation of the samples Occurrence Central limit theorem: mean of lots of independent & identicallydistributed RVs Nature (approximate) Measurement error, physical characteristic, physical phenomenon Diffusion of heat or chemicals 52

53 Weighted Mean from Samples Suppose We want to compute the sample mean of a class of things (or we want to reduce it s influence) We are not sure if the ith item belongs to this class or not - partially belongs probability w i, random variable r i Sample mean (no weights) Weighted sample mean 53

54 Gaussian Mixture Modeling of Image Histograms K classes, N samples Class 2 Class probability Class 1 Class 3 Intensity 54

55 Problem Statement Goal: assign pixels to classes based on intensities (output = label image) Problem: can we simultaneously learn the class structure and assign the class labels? Histogram of image samples Intensity 55

56 Crisp vs. Soft Class Assignment If we knew the pdfs (Gaussians) of the classes, we could assign class labels to each data point/pixel Assume equal overall probabilities of classes Crisp Assign Soft Assign Find class that has max probability for given intensity r at pixel I. Assign that class label to that pixel For each pixel and each class, assign a (conditional) probability that that pixel belongs to that class 56

57 Simultaneously Estimate Class PDFs and Pixel Labels Iterative Algorithm Start with initial estimate of class models Compute matrix of soft assignments Use soft assignments to compute new weighted mean and standard deviation for each class Use new mean and standard deviation to compute new soft assignments and repeat (until change in parameters is very small) 57

58 EM Algorithm Example 58

59 MRI Brain Tissue Segmentation Segment gray matter, white matter, CSF and non-brain regions Manual segmentation requires expertise and is very time consuming. It is also subjective.

60 MRI Brain Example 60

61 Effect of noise Overlap in conditional probabilities of different classes. Image Thresholding only With spatial filtering

62 Non-brain tissues complicate matters

63 Observed image histogram P(r k ) = l { csf, gm,wm,background} P( r l)p(l) k

64 Non-brain tissues Large amount of overlap brain vs. non-brain Ground truth Classification CSF Gray matter White matter When conditional probabilities overlap this significantly, simple thresholding techniques fail. There are several possible solutions

65 Multidimensional histograms Sometimes we have multiple images of the same object Different channels in multispectral data Magnetic resonance images with different pulse sequences Each channel provides new information If we have two channels, we can create a 2D histogram A 2D histogram of a brain image from 2 MRI channels

66 Clustering PD T1 T2 CSF WM GM Muscle Fat

67 K-means clustering in 2D Given two images f(x,y) and g(x,y) of the same scene but representing different channels. To segment into N regions 1. Randomly initialize N cluster centers (f k,g k ) 2. For each (x,y) Compute the distance from (f(x,y), g(x,y)) to all cluster centers (f k,g k ) Assign this pixel to cluster with the smallest distance 3. Recompute cluster centers based on pixels assigned 4. Stop if cluster centers didn t move; otherwise, go to step 2

68 Fitting multi-dimensional Gaussians Can use expectation maximization to fit mixture of multivariate Gaussian distributions to data Learn means and covariance matrices To simplify can assume covariance matrix is diagonal

69 Non-brain tissues CSF Gray matter White matter One solution is to use atlases to guide the segmentation in addition to thresholding 452 subjects scanned Manually segmented Images aligned into common coordinate system Atlas: tissue classification probability images Gives a prior probability for all tissue classes at each voxel When we have a new MRI to segment Register this atlas onto new subject s image Use atlas to clarify ambiguities

70 Effect of non-homogeneous intensity Magnetic resonance images typically have a multiplicative bias field This is similar to variable illumination Unfortunately, the bias field can depend on the object being imaged! Fortunately, it can still be estimated Image Thresholding only With bias field correction