Fig. 3.1: Interpolation schemes for forward mapping (left) and inverse mapping (right, Jähne, 1997).

Transcription

1 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes Spatial transorms 3.1. Geometric operations (Reading: Castleman, 1996, pp ) - a geometric operation is deined as g o (x, y) = g i ( x!, y!) = g i [ a(x,y),b(x, y) ] (3.1) where a() and b() are the transormation equations that map an input image g i onto an output image g o Interpolation - as the pixel coordinates resulting out o most transormations do not coincide with the grid coordinates, the grey value o transormed pixels has to be derived through pixel illing or interpolation; in orward mapping (see Fig. 3.1), the transormed pixel value in the output image is derived rom the our input image pixels mapping onto it, such that each pixel's grey value G o is given by the sum o the areal ractions G o (x, y) =! (3.) k k G k Fig. 3.1: Interpolation schemes or orward mapping (let) and inverse mapping (right, Jähne, 1997). - a more sophisticated approach consists o deriving the grey values o the pixels in the transormed image through appropriate interpolation in the original image (Fig. 3.1, inverse mapping) - the simplest interpolation technique, nearest-neighbour or zero-order interpolation, consists in assigning the pixel a grey value that corresponds to that o its nearest neighbour - bilinear (irst-order) interpolation involves itting a hyperbolic paraboloid such that G(x, y) = ax + by + cxy + d (3.3) - rom the our corner coordinate pixels (see Fig. 3.) any inlying point can be derived through sequential interpolation: G(x, 0) = G(0,0) + x[ G(1,0)! G(0,0) ] G(x,1) = G(0,1) + x[ G(1,1)! G(0,1) ] G(x, y) = G(x, 0) + y[ G(x,1)! G(x,0) ] (3.4) - applications which demand continuous interpolation (also o the derivatives) require spline-based interpolation or more sophisticated algorithms implemented in the requency domain (or details see, e.g., Jähne, 1997); another option is to represent object outlines in a classiied image by splines or other continuous unctions

2 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes Fig. 3.: Bilinear interpolation scheme (Castleman, 1996) Geometric transorms (Reading: Schowengerdt, 1997, pp ) - aine transorms are linear coordinate transorms that can be broken down into a sequence o basic operations: translation, rotation, scaling, stretching, and shearing; these can be expressed in matrix notation (using homogeneous coordinates in which the x-y plane is represented by z=1 in x-y-z space) - translation by x 0, y 0 such that a(x,y)=x+x 0, b(x,y)=y+y 0 :! a(x, y) $! 1 0 x o $! x$ # b(x, y) = # 0 1 y 0 # y 1 % % 1% - stretching by c and d such that a(x,y)=x/c, b(x,y)=y/d:! a(x, y) $! 1/ c 0 0$! x$ # b(x, y) = # 0 1/ d 0 # y 1 % 0 0 1% 1% - rotation by the angle θ, such that a(x,y)=xcosθ -ysinθ, b(x,y)=xsinθ +ycosθ:! a(x, y) $! cos' (sin' 0$! x$ # b(x, y) = # sin' cos' 0 # y 1 % 0 0 1% 1% - a ull aine transormation can then be described as the matrix product o its component operations, e.g., or a rotation about a point x 0,y 0 other than the origin is given by! a(x, y) $! 1 0 x 0 $! cos' (sin' 0$! 1 0 (x 0 $! x$ # b(x, y) = # 0 1 y 0 # sin' cos' 0 # 0 1 (y 0 # y 1 % % 0 0 1% % 1% (3.5) (3.6) (3.7) (3.8) - or some applications, such as the mapping o ground control points (GCPs) rom a distorted satellite image to a reerence image (or map projection), non-linear distortion precludes the application o aine transorms; in the case o map projections, the transormation equations are known (see, e.g., Snyder J. P., 198: Map projections used by the U.S. Geological Survey. Geol. Surv. Bull., nd ed., 153, 313pp.) and can be applied accordingly

3 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes rectiication o distorted images or which the transormation equations are not known (e.g., a problem that might be encountered in geolocation o satellite imagery or aerial photography based on GCPs), requires a dierent approach; one such approach consists o deriving a generic polynomial model to relate the image coordinates x',y' to that o a reerence, x! = N N"1 i j # # a ij i= 0 j =0 N N "1 i j y! = # # b ij i = 0 j = 0 - or most applications -order polynomials are suicient, such that x! = a 00 + a 10 + a 01 + a 11 + a 0 x r + a 0 y! = b 00 + b 10 + b 01 + b 11 + b 0 x r + b 0 (3.9) (3.10) - now, or M pairs o GCPs located in the image and the reerence, equation 3.10 can be written or the M sets o equations in matrix notation " x 1! % " 1,1,1,1,1,1 $ x! ' $ 1,,,,,, $... ' = $ $ # x! ' $ M # 1, M,M, M, M, M X = RA,1, M " % $ ' $ ' $ ' $ $ # a 00 a 01 a 10 a 11 a 0 a 0 % ' ' ' ' ' (3.11) and similarly or the y' coordinates - i M is equal to the number o polynomial coeicients K, A and B can be obtained by inverting R: A = R!1 X B = R!1 Y (3.1) - or cases where M>K, least-square techniques can be applied to solve or A and B (or details see Schowengerdt, 1997, p. 337.) Analysis o stereoscopic images (Reading: Castleman, 1996, pp ) - because the reconstruction o the 3-dimensional topography o image pairs requires geometric transormation o images, it will be shortly discussed at this point - or two bore-sighted cameras (i.e. with parallel optical axes) as shown in Fig. 3.3, the image o a given point in 3-D space X 0,Y 0, on the image plane o the let camera l with ocal length (such that Z = - in the image plane), is given by X l =!X 0 Y l =!Y 0 (3.13) the corresponding right camera image is then given by a shit d in the image position relative to the origin which coincides with the lens l:

4 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes X r =!(X 0 + d)! d Y r =!Y 0 (3.14) - deining a coordinate system in the image plane that is rotated by 180 relative to the world coordinate system such that x l = -X l, y l = -Y l, = -X r d, = -Y r, the image coordinates or the point X 0,Y 0, are: x l = X 0 y Z l = Y 0 0 = (X 0 + d) = Y 0 (3.15) rearranging and solving or yields: X 0 = x l d =! x l Z = x 0 Z r! d 0 (3.16) - this is reerred to as the normal-range equation and relates the Z-coordinate (i.e. topography)o any given point to the corresponding dislocation in x-direction in the image pair; as both and d are typically small whereas may be quite large, determination o -x l to a high degree o accuracy is crucial - the true-range equation is derived trigonometrically as R = + x l + y l R = d + x l + y l! x l (3.17) which or many systems (with X 0,Y 0 << and x l,y l << ) can be approximated by equation while the derivation o the Z-topography is airly straightorward (at least or bore-sighted systems), the determination o -x l to a suicient degree o accuracy can be quite a challenge; in most applications the magnitude o the dislocation cannot be derived explicitly but has to determined or an entire image pair based on correlation techniques; some o these aspects will be covered at a later stage; however, a technique explicitly developed or stereo pair matching has been presented by Sun (Digital Image Computing: Techniques and Applications, pp , Massey University, Auckland, New Zealand, 10-1 December 1997) and the algorithm has been implemented at Fig. 3.3: Camera arrangement or stereoscopic imaging (Castleman, 1996).

5 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes Spatial iltering (Reading: Gonzalez and Wintz, 1987, pp ; Castleman, 1996, pp ; Jähne, 1995, 100.) - while point operations discussed in Section can be thought o as the simplest orm o a spatial operation (operating on a neighbourhood o 1,1 or each pixel), in a more general context transormations or iltering in the spatial domain correspond to the transormation o an input image g i (x,y) to an output image g o (x.y) through convolution with a linear, position-invariant 1 operator (also reerred to as mask or convolution kernel) h, such that g o (x, y) = h(x,y)! g i (x, y) (3.18) (as outlined in Section 4, this spatial operation corresponds to a multiplication o the Fourier transorms H(xu,v) and G(u,v) o the image and the kernel in the requency domain) - in its discrete orm, the convolution can be represented by the summation over a kernel k o size m,n or the product k() g i () o every x,y o the image pixel coordinates (Fig. 3.4) such that g o (x, y) = 1 C i " " k(m,n) g i (x + m,y + n) (3.19) m =! i n=! j j - the scaling actor 1/C is typically given as ΣΣk(m,n), with kernels mostly odd-sized (e.g., 3x3) with the origin at the center coordinate Fig. 3.4: Discrete convolution (Castleman, 1996). - as will be outlined in more detail in Section 4, the three major types o linear ilters can be represented both in the spatial and in the requency-domain (Fig. 3.5); with lowpass ilters mostly applied to smooth images and reduce the impact o impulse noise on image processing and high-pass ilters playing an important role in edge and eature detection; in general terms, a convolution can be applied to remove the 1 An operator h is linear i its application to a combined set o images g k (multiplied by a scalar actor a k ) yields the same result as the operation on the individual component images: " h #! k $ a k g k % =! a kh(g k ) k which implies that linear operations can be perormed on images decomposed into subunits rather than the whole image. Position- or shit-invariance requires that the result o an operation is the same regardless o the position o the image or the operator, which also aids in breaking down more complex operations into simpler component steps.

6 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes - - eects o its inverse (determined or provided a priori) on the image, such as artiacts resulting out o errors in the imaging system Fig. 3.5: Cross-sections o requency-domain (top) and spatial-domain ilters (Gonzalez and Woods, 199) Smoothing ilters - as shown in Fig. 3.5, lowpass or smoothing ilters reduce the high-requency component o the image; the range o requencies impacted by the spatial convolution is given by the ilter size, with the magnitude o the scaling actor and the shape o kernel k(m,n) determining the amplitude o the transormation (Fig. 3.6 shows simple low-pass ilter kernels o dierent sizes); the impact o the ilter operation can be modulated by, e.g., sampling a Gaussian distribution unction (Fig. 3.5a) and modiying its statistical moments Fig. 3.6: Kernels or smoothing ilters (Gonzalez and Woods, 199). - linear convolutions are associated with a (or large kernels substantial) loss o image inormation; or image enhancement purposes, so-called rank-order ilters (non-linear) achieve better results; or rank order iltering, the set o pixels sampled by a mask o size m,n is ordered and the center pixel (or origin) is transormed to the n-th element o the sorted list (e.g., median, minimum, maximum); as discussed in more detail in section 3.3, these non-linear ilters also orm the basis or morphological iltering techniques Fi.g. 3.7: Schematic depiction o 3x3 median (rank-order) ilter (Jähne, 1997).

7 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes Fig. 3.8: Noisy image (let) and result o 3x3 median iltering (right) Sharpening ilters - as shown in Fig. 3.5b, the high-pass ilter selectively operates on the high-requency component o images; because the integral o the ilter unction over the entire interval is zero (corresponding to a zerosum o the discrete ilter kernel, see Fig. 3.9); no change occurs in image regions that are o uniorm grey value, whereas ine details are "sharpened", e.g., in applications where a remedy or the eects o blurry optics is required; since this operation also enhances the noise component, the weight o the central pixel in the kernel may be increased to values >8 in a 3x3 mask (Fig. 3.9) which is oten reerred to as high-boost iltering and represents a compromise between sharpening and maintaining image integrity Fig. 3.9: Sharpening kernel (Gonzalez and Woods, 199). - derivative ilters are a variant o sharpening ilters that respond to spatial grey-value gradients in an image (Fig. 3.10); the gradient vector o a unction (x,y) is given by #"! = %"x ( % " ( $ "y ' with derivative ilters based on the determination o the magnitude o this vector (3.0)! =! = #" % $ "x # + ' " % $ "y (3.1) - implementation o gradient operators on discrete images can be achieved in several ways (see exemplary pixel neighbourhood and operators in Fig. 3.11); dierencing o the absolute values is the most common approach, with, e.g., so-called Roberts cross-gradient operators (corresponding to a x kernel) based on the approximation! " z 5 # z 9 # z 6 # z 8 (3.) - or 3x3 kernels, this approximation can be implemented by so-called Prewitt operators (Fig. 3.11) which compute the sum o dierences

8 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes - 4 -! " (z 7 + z 8 + z 9 ) # (z 1 + z + z 3 ) + (z 3 + z 6 + z 9 ) # (z 1 + z 4 + z 7 ) (3.3) - or some applications, the second-order derivative (Laplacian) o an image needs to be computed and can be approximated as shown! = " "x + " "y! # 4z 5 $ (z + z 4 + z 6 + z 8 ) (3.4) Fig. 3.10: First and second derivative across a boundary in an image (Gonzalez and Woods, 199). Fig. 3.11: Image segment (a) and kernels or gradient operators (b to d, or details see text; Gonzalez and Woods, 199).

9 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes Morphological iltering (Reading: Castleman, 1996, pp , Jähne, 1995, pp , Serra, 198) - as outlined in the previous section, the convolution o an image with a linear ilter mask involves the summation over the invidual products o mask and image matrix elements (Fig. 3.4); or non-linear ilters this product summation is replaced by a ranking (in rank-order ilters such as the median ilter shown in Figs. 3.7 and 3.8) or a logical operation (or morphological ilters) as shown in Fig. 3.1 Fig. 3.1: Transormation o an input image with a non-linear, morphological operator (Castleman, 1996). - rank-order ilters can be very powerul operators or image enhancement and extraction o image subsets (two examples will be discussed in class, with the median ilter as one o them also shown in Fig. 3.8) - morphological ilters operate on binary images, with each pixel represented by either 0 or 1 as a result o a segmentation, e.g. through an algorithm that distinguishes between a class o relevant eatures and a background signal; while the examples discussed here and in the class are based on square masks (mostly 3x3, as in Fig. 3.1), morphological ilter masks (also called structuring elements) can be o any shape, including composite masks that are broken down into components sequentially operating on the image - the two basic operations in morphological image processing are the erosion and the dilation; the elimination o all pixels adjacent to the outer margin o a eature in an image is reerred to as erosion (assuming here and throughout the text, that eatures are characterized by 0 and the background is 1 or 55 in binary images); in the terms o set theory, the result E o an erosion is the set o all points x,y such that S translated by x,y is ully contained within the eature set B E = B! S = { x, y S xy " B} (3.5) - an example o an erosion and other morphological processes is shown in Fig. 3.13; note how connected eatures end up disjointed i the bridges between the eatures are smaller than one structuring element diameter - a dilation integrates all pixels into an object that border on its perimeter; this is achieved belecting the structuring element B about its origin and then translating this by x,y across the image, the result o the dilation D is the set o all x,y or which S' and B overlap by at least one element: D = B! S = { x, y ( S " xy # B) $ B} (3.6) - sequential erosion and dilation or vice versa are known as opening O(B) and closing C(B), respectively:

10 Eicken, GEOS 69 - Geoscience Image Processing Applications, Lecture Notes O(B) = B S = ( B! S) " S C(B) = B os = ( B " S)! S (3.6) Fig. 3.13: Opening through sequential erosion (b,c) and dilation (d,e) and closing through sequential dilation (,g) and erosion (h,i) o a eature (a) (Gonzalez and Woods, 199). - thus erosion results in the elimination o eatures smaller than the size o the structuring element, separation o eatures joined by thin bridges and a reduction in the roughness o eature contours; sequential erosion with increasing mask sizes and determination o the corresponding change in total eature/background area is an eicient and robust method o size analysis - morphological operators, in particular when combined with conditional criteria that, e.g., ensure connectivity between objects, are powerul instruments in the pre-processing and analysis o image data, such as in the case o skeletonization or Euclidean distance distance transorms (a variant o the binary morphological operation, see Fig. 3.14) Fig. 3.14: Binarized image (let), euclidean distance map (center, grey value denotes distance rom margin o eatures), skeletonized image (right).