Ulrik Söderström ulrik.soderstrom@tfe.umu.se 17 Jan 2017 Image Processing Introduction
Image Processsing Typical goals: Improve images for human interpretation Image processing Processing of images for machine perception Image analysis Dealing with images for storage and communication (compression) Image handling 2
Image processing steps Low-level Primitive operations (filtering, noise reduction) Both input and output are images Mid-level Segmentation, description, recognition Input. images, output. image attributes High-level Making sense of the recognized objects, (like vision) 3
Course outline Lectures Spatial and Frequency domain, Restoration, Compression, Morphological Image Processing, Representation, Description Practical work 3 Labs Project (extensive lab) Written exam 4
Grading 3,4,5 Lab exercises ~ 30% Project ~ 30% Written exam ~ 40% 5
Images An image is a 2-D function f(x, y) x and y - spatial coordinates f - amplitude (intensity, graylevel) at a point with coordinates (x, y) All values finite and discrete - digital image Digital image processing - computers involved Each value - pixel (picture element) 6
Image creation Observation of energy Electromagnetic (EM) radiation most common Human vision is limited to a narrow band Sensors have higher capacities The whole EM spectrum + other energies 7
Photons Photon - an amount of energy No mass Travelling at the speed of light Different frequencies v and wavelengths λ c= speed of light, 3x10 8 Energy of a photon h = plancks constant 8
The EM spectrum 9
Ultraviolet (UV) light Flourescence images of corn UV light from the same star as previous 10
Visible light By far the most common 11
Visible light Microscopic images of a CD, cholesterol, and a microprocessor 12
Infrared light America (north and south) 13
Combined spectra Different bands give totally different images of the same object Astronomic images of the same region but in different bands 14
Other energies Sound High frequency, ultrasound (1-5 MHz) Medical images 15
Other energies Electron microscopy 16
Computer generated images No need for a physical energy source Fractals 3-D computer models 17
Foundations All imaging systems replicates the human visual system 18
Unknown functionalities 19
Unknown functionalities 20
Image aquisition Sensor to measure energy In digital cameras - CCD arrays Integrate over the sensor, values proportional to the number of photons hitting the surfaces 21
Image aquisition 22
Image formation An image f(x, y) When generated from a physical process: 0 < f(x, y) < 0 L min f(x, y) L max < (monocromatic image) The interval [L min, L max ] - grayscale of the image 23
Image formation Two components - illumination and reflectance f(x, y) = i(x, y) r(x, y) 0 < i(x, y) <, illumination component Determined by illumination source 0 < r(x, y) < 1, reflectance component Determined by object charasteristics Transmissivity is used instead of reflectivity in the case of illumination passing through objects, eg X-rays 24
Sampling and quantization The output of a sensor is in most cases a continuous voltage waveform Needs to be digitized Sampling - digitizing the coordinate values Usually M = 2 m steps in x-direction and N = 2 n steps in y-direction Quantization. digitizing the amplitude values L = 2 k gray values Image (storage) size = M N k/8 bytes 25
Sampling and quantization 26
Sampling and quantization 27
Sampling and quantization A square grid is the most common (the only one in the book) 28
Image representation Most common convention f(x, y) = Matrix representation of image values 29
Image representation Surface color Intensities Gray-level, Color information 30
Resolution Spatial resolution - determined by the Sampling Tightness in pixels Sampling distance The human eyes cannot detect resolution higher/lower than a threshold A computer might see more information 31
Spatial resolution 32
Acceptable resolution? Isopreference subjectively perceived quality of the images Many details- few gray levels needed 33
Sampling Sampling theorem If the distance between sampling points is larger than the smallest objects we want to capture, we get problems with aliasing Sampling introduces new frequencies The sampling frequency must be at least twice the highest frequency in the image Blur the image before sampling 34
Moiré patterns Moiré pattern effects occurs when periodic patterns break up. E.g. scanned images from printed pages with periodic dots 35
Zooming and shrinking Resampling an already digital image Resize the image grid Simplest way to enlarge an image to twice its size - duplicate all pixels (nearest neighbor interpolation) Better results if more neighbors are taken into account (e.g. bilinear interpolation, using the four nearest neighbors) 36
Zooming and shrinking 37
Zooming 38
Pixel relationships Neighbors of a pixel p with coordinates (x, y) Four horizontal and vertical neighbors (x+1, y), (x-1, y), (x, y+1), and (x, y-1) N 4 (p), the 4-neighbors of p Four diagonal neighbors (x+1, y+1), (x+1, y-1), (x-1, y+1), and (x-1, y-1) N D (p) Combined, N 4 (p) N D (p) (union) N 8 (p), the 8-neighbors of p 39
Adjacency Two pixels that are neighbors are adjacent 4-adjacency, two pixels p and q with values V are 4-adjacent if q is in the set N 4 (p) 8-adjacency, two pixels p and q with values V are 8-adjacent if q is in the set N 8 (p) 40
Distance measures p has coordinates (x, y), q has (s, t) Distances between p and q Euclidean distance (the most natural in R 2 ) D e (p, q) = [(x-s) 2 + (y-t)2] ½ 2 1 2 1 x 1 2 1 2 D 4 distance, - city block distance D 4 (p, q) = x-s + y-t The 4-neighbors of (x, y) have D 4 = 1 2 1 2 1 x 1 2 1 2 41
Distance measures D 8 distance, - chessboard distance D 8 (p, q) = max( x-s, y-t ) The 8-neighbors of (x, y) have D 8 = 1 D m distance 2 2 2 2 2 2 1 1 1 2 2 1 x 1 2 2 1 1 1 2 2 2 2 2 2 The number of jumps between p and q along the path that connects them, depending on the values of the pixels on the path and their neighbors. 42
Operations on a pixel basis It is common to carry out arithmetic operations on images E.g. dividing one image by another - not a defined matrix operation Pixel wise operations Images must be of equal size 43
Linear and nonlinear operations An operator H whose input and output are images is linear if H(af + bg) = ah(f) + bh(g) for any images f and g and any scalars a and b E.g. summing K images Computing the absolute value of a function is an example of a nonlinear operation 44