Visualizing Images Recall two ways of visualizing an image Lecture : Intensity Surfaces and Gradients Intensity pattern d array of numbers We see it at this level Computer works at this level Bridging the Gap Images as Surfaces Motivation: we want to visualize images at a level high enough to retain human insight, but low enough to allow us to readily translate our insights into mathematical notation and, ultimately, computer algorithms that operate on arrays of numbers. Surface height proportional to pixel grey value (dark=low, light=high) Note: see demoimsurf.m in matlab examples directory on course web site if you want to generate plots like these. Mean = 164 Std = 1.8 1
Mean = 111 Std = 15.4 Terrain Concepts How does this visualization help us?
Terrain Concepts Terrain Concepts Basic notions: Uphill / downhill Contour lines (curves of constant elevation) Steepness of slope Peaks/Valleys (local extrema) More mathematical notions: Tangent Plane Normal vectors Curvature Gradient vectors (vectors of partial derivatives) will help us define/compute all of these. www.pianoladynancy.com/ wallpapers/1003/wp86.jpg Consider function f(x) = 100-0.5 * x^ Consider function f(x) = 100-0.5 * x^ Gradient is df(x)/dx = - * 0.5 * x = - x Geometric interpretation: gradient at x 0 is slope of tangent line to curve at point x 0 tangent line Δy slope = Δy / Δx x 0 Δx = df(x)/dx x0 f(x) f(x) = 100-0.5 * x^ df(x)/dx = - x f(x) = 100-0.5 * x^ df(x)/dx = - x grad = slope = 1 0-1 - x 0 = - 3 gradients -3 3
f(x) = 100-0.5 * x^ Gradients on this side of peak are positive 3 df(x)/dx = - x 1 0 gradients -1 - -3 Gradients on this side of peak are negative Math Example : D Gradient f(x,y) = 100-0.5 * x^ - 0.5 * y^ df(x,y)/dx = - x df(x,y)/dy = - y Note: Sign of gradient at point tells you what direction to go to travel uphill Gradient is vector of partial derivs wrt x and y axes Math Example : D Gradient f(x,y) = 100-0.5 * x^ - 0.5 * y^ Plotted as a vector field, the gradient vector at each pixel points uphill The gradient indicates the direction of steepest ascent. The gradient is 0 at the peak (also at any flat spots, and local minima, but there are none of those for this function) Math Example : D Gradient f(x,y) = 100-0.5 * x^ - 0.5 * y^ Let g=[g x,g y ] be the gradient vector at point/pixel (x 0,y 0 ) Vector g points uphill (direction of steepest ascent) Vector - g points downhill (direction of steepest descent) Vector [g y, -g x ] is perpendicular, and denotes direction of constant elevation. i.e. normal to contour line passing through point (x 0,y 0 ) Math Example : D Gradient f(x,y) = 100-0.5 * x^ - 0.5 * y^ And so on for all points Image Gradient The same is true of D image gradients. The underlying function is numerical (tabulated) rather than algebraic. So need numerical derivatives. 4
Taylor Series expansion See also T&V, Appendix A. Taylor Series expansion See also T&V, Appendix A. Manipulate: Manipulate: Finite forward difference Finite backward difference Taylor Series expansion See also T&V, Appendix A. Finite forward difference See also T&V, Appendix A. subtract Finite backward difference Finite central difference More accurate Finite central difference Example: Temporal Gradient A video is a sequence of image frames I(x,y,t). Example: Temporal Gradient Consider the sequence of intensity values observed at a single pixel over time. df(x)/dx derivative 1D function f(x) over time 1D gradient (deriv) of f(x) over time df(x)/dx derivative Each frame has two spatial indices x, y and one temporal (time) index t. 5
Temporal Gradient (cont) What does the temporal intensity gradient at each pixel look like over time? Example: Spatial Image Gradients I(x,y) I(x+1,y) - I(x-1,y) Partial derivative wrt x I x =di(x,y)/dx I(x,y+1) - I(x,y-1) Partial derivative wrt y I y =di(x,y)/dy Functions of Gradients Functions of Gradients I(x,y) I x I y I(x,y) I x I y Magnitude of gradient sqrt(ix.^ + Iy.^) Measures steepness of slope at each pixel Angle of gradient atan(iy, Ix) Denotes similarity of orientation of slope Functions of Gradients Next Time: Linear Operators I(x,y) atan(iy, Ix) Gradients are an example of linear operators, i.e. value at a pixel is computed as a linear combination of values of neighboring pixels. What else do we observe in this image? Enhanced detail in low contrast areas (e.g. folds in coat; imaging artifacts in sky) 6