Geometric Rectification of Remote Sensing Images Airborne TerrestriaL Applications Sensor (ATLAS) Nine flight paths were recorded over the city of Providence. 1
True color ATLAS image (bands 4, 2, 1 in RGB) Distortions? Color balancing? Mosaicking? Subsetting? QuickBird Data Acquisition Multiple date and path imaging April 8, 2004 April 16, 2004 May 4, 2004 April 8, 2004 May 6, 2004 Long Island 2
Geometric Rectification or Georectification Raw remotely sensed data gathered by satellite or aircraft are representations of the irregular surface of the Earth. Remotely sensed images are distorted by both the curvatures of the Earth and the sensor being used. The process of shifting pixel locations to remove distortion is known as rectification or georectification. Map Projection and Georeferencing A map projection system is any system designed to represent the surface of a sphere or spheroid on a plane (UTM, State Plane, Geographic Coordinates ). The process of assigning geographic coordinates to an image is known as georeferencing. Often the process of rectification includes georeferencing, because one can both shift the pixels to remove distortion and assign coordinates to those pixels at the same time. Geometric Correction It is usually necessary to preprocess remotely sensed data and remove geometric distortion so that individual pixels are in their proper planimetric (x, y) map locations. Geometrically corrected imagery can be used to extract accurate distance, polygon area, and direction information. This allows remote sensing derived information to be related to other thematic information in GIS or spatial decision support systems (SDSS). 3
Internal and External Geometric Error Remotely sensed imagery typically exhibits internal and external geometric error. It is important to recognize the source of the internal and external error and whether it is systematic (predictable) or nonsystematic (random). Systematic geometric error is generally easier to identify and correct than random geometric error. Internal Geometric Error Internal geometric errors are introduced by the remote sensing system itself or in combination with Earth rotation or curvature characteristics. These distortions are often systematic (predictable) and may be identified and corrected using prelaunch or in-flight platform ephemeris (i.e., information about the geometric characteristics of the sensor system and the Earth at the time of data acquisition). 4
Image Offset (skew) caused by Earth Rotation Effects Earth-observing Sun-synchronous satellites are normally in fixed orbits that collect a path (or swath) of imagery as the satellite makes its way from the north to the south in descending mode. Meanwhile, the Earth below rotates on its axis from west to east making one complete revolution every 24 hours. This interaction between the fixed orbital path of the remote sensing system and the Earth s rotation on its axis skews the geometry of the imagery collected. 5
Image Skew a) Landsat satellites are in a Sun-synchronous orbit. The Earth rotates on its axis from west to east as imagery is collected. b) While the matrix (raster) may look correct, it actually contains systematic geometric distortion caused by the angular velocity of the satellite in its descending orbital path in conjunction with the surface velocity of the Earth as it rotates on its axis while collecting a frame of imagery. c) The result of adjusting (deskewing) the original Landsat data to the west to compensate for Earth rotation effects. External Geometric Error External geometric errors are usually introduced by phenomena that vary in nature through space and time. The most important external variables that can cause geometric error in remote sensor data are random movements by the aircraft (or spacecraft) at the exact time of data collection, which usually involve: - altitude changes, and/or - attitude changes (roll, pitch, and yaw). 6
Ground Control Points Geometric distortions introduced by sensor system attitude (roll, pitch, and yaw) and/or altitude changes can be corrected using ground control points and appropriate mathematical models. A ground control point (GCP) is a location on the surface of the Earth (e.g., a road intersection) that can be identified on the imagery and located accurately on a map. QuickBird Satellite Image Ortho-rectified photograph Ground Control Points The image analyst must be able to obtain two distinct sets of coordinates associated with each GCP: image coordinates specified in i rows and j columns, and map coordinates (e.g., x, y measured in degrees of latitude and longitude, feet in a state plane coordinate system, or meters in a Universal Transverse Mercator (UTM) projection). The paired coordinates (i, j and x, y) from many GCPs (e.g., 20) can be modeled to derive geometric transformation coefficients. These coefficients may be used to geometrically rectify the remote sensor data to a standard datum and map projection. 7
Ground Control Points Obtaining accurate ground control point (GCP) map coordinate information for image-to-map rectification include: 1. hard-copy planimetric maps (e.g., USGS 7.5-minute 1:24,000-scale topographic maps); 2. digital planimetric maps (e.g., the USGS digital 7.5- minute topographic map series) where GCP coordinates are extracted directly from the digital map on the screen; 3. digital orthophotoquads that are already geometrically rectified (e.g., USGS digital orthophoto quarter quadrangles DOQQ); and/or 4. global positioning system (GPS) instruments that may be taken into the field to obtain the coordinates of objects. Image to Map Registration 8
Image to Image Registration Image-to-image registration is the translation and rotation alignment process by which two images of like geometry and of the same geographic area are positioned coincident with respect to one another so that corresponding elements of the same ground area appear in the same place on the registered images. Hybrid Approach to Image Rectification/Registration In image-to-map rectification the reference is a map in a standard map projection. In image-to-image registration the reference is another image. If a rectified image is used as the reference base (rather than a traditional map) any image registered to it will inherit the geometric errors existing in the reference image. When conducting rigorous change detection between two or more dates of remotely sensed data, it may be useful to select a hybrid approach involving both image-to-map rectification and image-to-image registration. 9
Image to Image Hybrid Rectification/Registration a) Previously rectified 1982 Landsat TM band 4 data, resampled to 30 30 m pixels using nearest-neighbor resampling and a UTM map projection. b) Unrectified 1987, Landsat TM band 4 data to be registered to the rectified 1982 Landsat scene. How to do it? Image to Map Geometric Rectification Logic Two basic operations must be performed to geometrically rectify a remotely sensed image to a map coordinate system: Spatial interpolation Intensity interpolation 10
Spatial Interpolation (find accurate spatial locations) The geometric relationship between the input pixel coordinates (column and row; referred to as x,y ) and the associated map coordinates of this same point (X, Y) must be identified. A number of GCP pairs are used to establish the nature of the geometric coordinate transformation that must be applied to rectify or fill every pixel in the output image (x, y) with a value from a pixel in the unrectified input image (x,y ). This process is called spatial interpolation. Order of Transformation Polynomial equations are used to convert source file coordinates into the referencing map coordinates. Depending upon the distortion in an image, the number of GCPs used, and their locations relative to one another, complex polynomial equations may be required to express the needed transformation. 11
Order of Transformation Concept of how different-order transformations fit a hypothetical surface illustrated in cross-section. a) Original observations. b) First-order linear transformation fits a plane to the data. c) Second-order quadratic fit. d) Third-order cubic fit. The degree of complexity of the polynomial is expressed as the order of the polynomial. The order is simply the highest exponent used in the polynomial. Transformation Matrix A transformation matrix is computed from the GCPs. The matrix consists of coefficients which are used in polynomial equations to convert the coordinates. The size of the matrix depends on the order of transformation. 12
Linear Transformation The transformation matrix for a 1 st -order transformation consists of six coefficients, three for each coordinate (X,Y) a 1 a 2 a 3 b 1 b 2 b 3 which are used in a 1 st -order polynomial as follows: x 0 = b 1 + b 2 x i + b 3 y i y 0 = a 1 + a 2 x i + a 3 y i where, x i and y i are source coordinates (input) x 0 and y 0 are rectified coordinates (output) RMS Error: Polynomial Curve vs. GCPs It is almost impossible to derive coefficients that produce no error. Every GCP influence the coefficients. The distance between the GCP reference coordinate and the curve is called RMS error. Reference X Coordinate GCP Polynomial curve Source X Coordinate 13
Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Source X Coordinate (input) Transformation Example: 1 st order x r = (25) + (-8)x i Source X Coordinate What if the second GCP changed from 9 to 7? Reference X coordinate (output) 1 17 2 9 3 1 Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Source X Coordinate (input) Transformation Example: 1 st order x r = (25) + (-8)x i Source X Coordinate What if the second GCP changed from 9 to 7? Reference X coordinate (output) 1 17 2 9 to 7? 3 1 14
Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Transformation Example: 1 st order x r = (25) + (-8)x i Source X Coordinate What if the second GCP changed from 9 to 7? A line cannot connect these points, which illustrates that they cannot be expressed by a 1 st -order polynomial, like the one above. In this case, a 2 nd -order polynomial equation will be necessary to express these points. Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Transformation Example: 2 nd order x r = (25) + (-8)x i x r = (31) + (-16) x i + (2) x 2 i Source X Coordinate A line cannot connect these points, which illustrates that they cannot be expressed by a 1 st -order polynomial, like the one above. In this case, a 2 nd -order polynomial equation will be necessary to express these points. 15
Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Transformation Example: 2 nd order x r = (31) + (-16) x i + (2) x i 2 Source X Coordinate Source X Coordinate (input) Reference X coordinate (output) 1 17 2 7 3 1 Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Transformation Example: 2 nd order x r = (31) + (-16) x i + (2) x 2 i What if one more GCP is added? Source X Coordinate Source X Coordinate (input) Reference X coordinate (output) 1 17 2 7 3 1 4 5? 16
Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Transformation Example: 3 rd order Source X Coordinate As illustrated, this 4 th GCP does not fit the curve of the 2 nd - order polynomial equation. In order to have all of the GCPs fit into a curve, a higher order polynomial equation will be needed. Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Transformation Example: 3 rd order x r = (25) + (-5) x i + (-4) x i 2 + (1) x i 3 Source X Coordinate As illustrated, this 4 th GCP does not fit the curve of the 2 nd - order polynomial equation. In order to have all of the GCPs fit into a curve, a higher order polynomial equation will be needed. 17
Reference X Coordinate 0 4 8 12 16 0 1 2 3 4 Transformation Example: 3 rd order x r = (25) + (-5) x i + (-4) x i 2 + (1) x i 3 Source X Coordinate (input) Reference X coordinate (output) 1 17 2 7 3 1 4 5? Although mathematically this could be done, it may increase the unnecessary complexity and cause distortion in image processing. Minimum Number of GCPs Higher orders of transformation can be used to correct more complicated types of distortion. However, to use a higher order of transformation, more GCP s are needed. The minimum number of points required to perform a transformation of order t equals: ( ( t + 1 ) ( t + 2 ) ) 2 Order of Transformation Minimum GCPs Required 1 3 2 6 3 10 4 15 5 21 6 28 7 36 18
Compute the Root-Mean-Squared Error of the Inverse Mapping Function Using the six coordinate transform coefficients that model distortions in the original scene, it is possible to use the output-to-input (inverse) mapping logic to transfer (relocate) pixel values from the original distorted image x, y to the grid of the rectified output image, x, y. However, before applying the coefficients to create the rectified output image, it is important to determine how well the six coefficients derived from the least-squares regression of the initial GCPs account for the geometric distortion in the input image. The method used most often involves the computation of the root-mean-square error (RMSerror) for each of the ground control points. RMS Error per GCP A way to measure the accuracy of a geometric rectification algorithm (actually, its coefficients) is to compute the Root Mean Squared Error (RMSerror) for each ground control point using the equation: Where: Ri = the RMS error for GCPi XRi = the X residual for GCPi YRi = the Y residual for GCPi Source GCP X residual Y residual RMS error Retransformed GCP 19
UTM / Datum Grid: NAD83 UTM Datum Grid: WGS 84 20
Since the grid of pixels in the source image rarely matches the grid for the reference image, the pixels are resampled so that new data file values for the output file can be calculated. Intensity Interpolation (obtain accurate pixel values) Pixel brightness values must be determined. Unfortunately, there is no direct one-to-one relationship between the movement of input pixel values to output pixel locations. It will be shown that a pixel in the rectified output image often requires a value from the input pixel grid that does not fall neatly on a row-and-column coordinate. When this occurs, there must be some mechanism for determining the brightness value (BV ) to be assigned to the output rectified pixel. This process is called intensity interpolation. 21
Intensity Interpolation (obtain accurate pixel values) Intensity interpolation involves the extraction of a brightness value from an x,y location in the original (distorted) input image and its relocation to the appropriate x, y coordinate location in the rectified output image. This pixel-filling logic is used to produce the output image line by line, column by column. When this occurs, there are several methods of brightness value (BV) intensity interpolation that can be applied. The practice is commonly referred to as resampling. Intensity Interpolation There are several methods of brightness value (BV) intensity interpolation that can be applied, including: 1. nearest neighbor, 2. bilinear interpolation, and 3. cubic convolution. 22
Nearest-Neighbor Resampling The brightness value closest to the predicted x, y coordinate is assigned to the output x, y coordinate. Advantage Transfers original data values without averaging them, as the other methods do, therefore the extremes and subtleties of the data values are not lost. Nearest Neighbor Resampling Disadvantage When this method is used to resample from a larger to a smaller grid size, there is usually a stair stepped effect around diagonal lines and curves. Suitable for use before classification. Data values may be dropped, while other values may be duplicated. The easiest of the three methods to compute and the fastest to use. Appropriate for thematic files, which can have data file values base on a qualitative (nominal or ordinal) or a quantitative (interval or ratio) system. The averaging interpolation and cubic convolution is not suited to a qualitative class value system. Using on linear thematic data (e.g., roads, streams) may result in breaks or gaps in a network of linear data. 23
Bilinear Interpolation Assigns output pixel values by interpolating brightness values in two orthogonal direction in the input image. It basically fits a plane to the 4 pixel values nearest to the desired position (x, y ) and then computes a new brightness value based on the weighted distances to these points. Advantage Results in output images that are smoother, without the stair stepped effect that is possible with nearest neighbor. More spatially accurate than nearest neighbor This method is often used when changing the cell size of the data, such as in resolution merge. Bilinear Interpolation Resampling Disadvantage Since pixels are averaged, bilinear interpolation has the effect of a low-frequency convolution. Edges are smoothed, and some extremes of the data file values are lost. 24
Cubic Convolution Assigns values to output pixels in much the same manner as bilinear interpolation, except that the weighted values of 16 pixels surrounding the location of the desired x, y pixel are used to determine the value of the output pixel. Rectification Is the process of transforming the data from one grid system into another grid system using the nth order polynomial. Since the pixels of the new grid may not align with the pixels of the original grid, the pixels must be resampled. Resampling is the process of extrapolating data values for the pixels on the new grid from the values of the source pixels. Georeferencing Refers to the process of assigning map coordinates to image data. The remote sensing data may already be projected onto the desired plane (resampled), but not yet referenced to the proper coordinate system. 25
Geometric Rectification Mosaicking Subsetting Seamless Working Data 26