Digital Image Fundamentals

Digital Image Fundamentals

Image Quality Objective/ subjective Machine/human beings Mathematical and Probabilistic/ human intuition and perception 6

Structure of the Human Eye photoreceptor cells 75~50 million Rod cell 6~7 million Cone cell 7 http://en.wikipedia.org/wiki/blind_spot_(vision)

Human Visual Perception Perceived brightness is NOT a simple function of intensity Perceived brightness Actual illumination 8

Human Visual Perception Optical Illusion http://architecturehell.wordpress.com/ 9

Image Sensing and Acquisition Illumination Source EM energy, ultrasound, synthesized, Scene Element Objects, human organs, buried mineral, Sensing Material Single sensor: photodiode Sensor strips: require extensive processing Sensor arrays: CCD & CMOS

Image Sensing and Acquisition Single sensor Sensor Strip Circular Sensor Strip 2

Image Sensing and Acquisition Illumination source Image sampling & quantization Scene element Imaging system (Internal) image plane Output (digitized) image pixel (pel, picture element, image element) 3

Image Sampling & Quantization f ( x, y) A/D F( j, sampling quantization Original image ( x, y) f sampling quantization F( j, coordinates amplitude 5

Image Sampling & Quantization j k M-,,,0,,,0 0, 0, 0,0 N M M M N N a a a a a a a a a A ), (,) (,0) ( ) (, (,) (,0) ) (0, (0,) (0,0) ), ( N M F M F M F N F F F N F F F k j F ), ( k j F ), ( y x f ), ( k j F 6

Downsampling 024x024 32x32 Downsampled by a factor of 2 8

Re-Sampling Zero-Order-Hold Method (ZOH) Row and column duplication 9

L=256,28,64,32,6,8,4,2 256 28 6 8 64 32 4 2 20

Digital Image Representation 8-bit image is commonly used Storage Human perception 32 steps (5 bits) in gray level 64 steps (6 bits) in gray level 2

Image Enhancement

Image Enhancement Goal of Image Enhancement make images more appealing no theory, ad-hoc rules, derived with insights Two Approaches Contrast Manipulation Histogram Modification 23

Contrast Manipulation Transfer Function Linear Nonlinear Piecewise Continuous Image Quantized Image 24

Contrast Manipulation Linear scaling and clipping G( j, T F( j, 0 F( j, 25

Contrast Manipulation Power-Law G ( j, F( j, p 0 F( j, 26

Contrast Manipulation Power-Law p G ( j, F( j, 0 F( j, 27

Contrast Manipulation Rubber Band Transfer Function Piecewise linear transformation Inflection point (control point) Can choose the area where we want to stretch or reduce the contrast 28

Contrast Manipulation Logarithmic Point Transformation G( j, log e af( j, log 2.0 e 0 F( j, Fourier Spectrum 0 ~.50 6 0~6.2 Useful for scaling image arrays with a very wide dynamic range 29

Contrast Manipulation Reverse Function G( j, F( j, 0 F( j, Able to see more details in dark areas of an image 30

Contrast Manipulation Inverse Function G( j, 0. F( j, 0 F( j, 0. 0. F( j, 3

Contrast Manipulation Amplitude-Level Slicing (Gray-Level Slicing) L- L- 32

Histogram Modification Goal Rescale the original image so that the histogram of the enhanced image follows some desired form 33

Histogram Modification Histogram Equalization make the output histogram to be uniformly distributed Transfer function Bucket filling 34

Histogram Equalization Transfer Function Input F( j, Desired (uniform) G( j, Histogram Probability Mass Function CDF Look-up Table Fi Gi F G? 35

Histogram Equalization Transfer Function Output histogram not really uniformly distributed Still keep the shape More flat than the original histogram 36

Histogram Equalization Bucket Filling F(j, # of pixels 0 2 2 5 arbitrary 255 3 G(j, # of pixels 0 N/256 N/256 2 N/256 uniform 255 N/256 Not - mapping N: # of total pixels Accumulated probability may not end exactly at the boundary of a bin split it out 37

Noise Cleaning

Noise Cleaning Noise electrical sensor noise photographic grain noise channel error etc. Characteristics of the noise discrete not spatially correlated higher spatial frequency 39

Noise Cleaning Two types of noise Uniform Noise Additive uniform noise, Gaussian noise Impulse Noise Salt and pepper noise Solutions Uniform Noise low-pass filtering Impulse Noise non-linear filtering 40

Basics of Spatial Filtering Mask filter, kernel, template m x n m=2a+, n=2b+, where a and b are nonnegative integers e.g. 3x3 mask W(0,0) Spatial Filtering/Convolution G( j, w(, ) F( j, k ) w(,0) F( j, w(0,0) F( j, w(,0) F( j, w(,) F( j, k ) 4

Basics of Spatial Filtering Q: Boundary pixels? ), ( (,) ), ( (,0) ), ( (0,0) ), (,0) ( ), ( ), ( ), ( k j F w k j F w k j F w k j F w k j F w k j G 42

Basics of Spatial Filtering Boundary Extension (3x3 mas copy e.g. 3x3 mask, w odd copy even Q: 5x5 mask? 43

Noise Cleaning Uniform noise Perform low-pass filtering General form 2 2 4 2 2 6 H 2 2 2 b b b b b b H 2 0 H 9 H e.g. 80 80 0 0 80 80 0 0 80 80 0 0 80 80 0 0 F 44

High Frequency Noise Removal Low-pass filtering Normalized to unit weighting Averaging Smaller/Larger filter size? 3x3 7x7 45

Noise Cleaning Impulse noise black: pixel value =0 dead sensor white: pixel value=255 saturated sensor Solutions Outlier detection Median filtering Pseudo-median filtering (PMED) 46

Impulse Noise Removal Outlier detection if x 8 8 O then i i 8 x 8 i O i How to choose? Larger window? 47

Impulse Noise Removal Median filtering a,, a N sort those values in order pick the middle one in the sorted list e.g. 3x3 mask: where N is odd 2 3 I 3 8 7 5 6 Median is 3 48

Impulse Noise Removal Median filtering Preserve sharp edges Effective in removing impulse noise D/2D (directional) e.g. 2D square cross 49

Impulse Noise Removal e.g. D (window size = 5) Step Ramp Single Pulse Double Pulse Triple Pulse Triangle 50

Impulse Noise Removal Median filtering Fast computation Approximation of median e.g. 5-element filter a, b, c, d, e MED(a, b, c, d, e) =max( min(a,b,c), min(a,b,d), ) =min( max(a,b,c), max(a,b,d), ) there are 0 possible choices could be narrowed down 5

Impulse Noise Removal Pseudomedian filtering (PMED) e.g. 5-element filter a, b, c, d, e spatially ordered MAXMIN = A (under estimated) = max( min(a,b,c), min(b,c,d), min(c,d,e) ) MINMAX = B (over estimated) = min( max(a,b,c), max(b,c,d), max(c,d,e) ) PMED( a, b, c, d, e ) = 0.5 * ( A + B ) = 0.5 * ( MAXMIN + MINMAX ) ~ MED( a, b, c, d, e ) 52

Impulse Noise Removal Pseudomedian filtering (PMED) 2D case PMED 2 PMED x PMED y PMED x PMED y PMED max 2 min 2 MAXMIN ( x MINMAX ( x c c ), MAXMIN ( y ), MINMAX ( y R R ) ) 53

Impulse Noise Removal Pseudomedian filtering (PMED) MAXMIN Remove salt noise MINMAX Remove pepper noise May cascade two operations Remove salt and pepper noise 54

Impulse Noise Removal Original noisy image MAXMIN MINMAX of MAXMIN Q: same results? MINMAX MAXMIN of MINMAX 55

Quality Measurement Peak signal-to-noise ratio (PSNR) Mean squared error (MSE) MSE w* h j, k F' j, k The PSNR is defined as j k F 2 PSNR 0log 0 2 255 MSE 56

Example Original image Gaussian noise (σ=0) PSNR : 28.8dB Gaussian noise (σ=30) PSNR : 8.8dB Q: Represent perceived visual quality? 57