Smooth Anomalous Values in a DEM Reduce Noise and Speckling in a Satellite Image Smooth Anomalous Values in Ordinal Data (Nearest Neighbour Interpolation) Smooth Anomalous Values in Nominal Data (Nearest Neighbour Interpolation) Operations What Do I Need? Scan Filter DEMs with Patterns, Stepping, and Anomalies It is common for DEMs (Digital Elevation Models) to have anomalous values. Most DEMs are generated by interpolation from spot height and/or contour elevation information. Interpolation algorithms often generate block patterns and data spikes and pits that should be smoothed out before the DEM can be used in terrain analysis. This is particularly important for drainage and viewshed modelling. Noisy or Speckled Remote Sensing Imagery Satellite imagery often contains anomalous values. Instrument noise, transmission errors, and electromagnetic interference frequently degrade the quality of remote sensing imagery. Data smoothing can remove or reduce anomalous cell values and despeckle degraded data. Smoothing Ordinal and Nominal Data Continuous Interval and Ratio type data map layers such as DEMs (Digital Elevation Models) and remote sensing imagery can be smoothed by a mean, or averaging, filter. Anomalous values are smoothed out by averaging the surrounding values and applying that average to the cell containing the anomalous value (this is called nearest neighbour averaging); however, the weighted mean filter is not suitable for Ordinal or Nominal data. If you have a map layer of cells to which you have applied a ranking scheme and there are some cells that have not been assigned a rank or have had a rank misassigned, these anomalous values can be smoothed by assigning a new value based on the median nearest neighbour value. HD-SMT-1
If you have a map layer of classified data, such as a remote sensing image to which you have applied a classification scheme, and there are some misclassified cells, these values can be interpolated based on the most common, nearest neighbour value. Example Smoothing Anomalous Values in a DEM Upon first inspection, the DEM (Digital Elevation Model) named Pine appears to be acceptable. However, after the generation of a shaded relief model, an anomalous pattern will be revealed in the top left quadrant. Smoothing the DEM will reduce the effects of the pattern that was generated by the USGS s (United States Geological Survey) elevation interpolation algorithm: For details on how to generate a shaded relief model, please refer to the document: How Do I Create a Shaded Relief Map from a DEM. The process will be summarized only here. Before generating the shaded relief model you must create a kernel for the Filter operation. Use the New Map command from the File menu to create HD-SMT-2
a 3 cell by 3 cell map layer. Assign the values as shown in the sample below and save the map layer using the name Shading Kernel: Once you have saved the kernel, choose Filter from the Operations menu. In the Filter dialog box specify the DEM in the Map drop-down list and the map layer Shading Kernel in the Kernel drop-down list. Select the Low Pass filter and click on OK: If you were to perform this operation from the Script window the statement would be: Shaded Unsmoothed = Filter Pine Kernel Shading Kernel LowPass; This operation produces a shaded relief model of the Pine DEM with an apparent light source from the Northwest. As the figure below shows, there HD-SMT-3
is a slight blockiness in the upper left quadrant caused by the USGS interpolation algorithm that was used to generate the DEM: To reduce or eliminate this problem, smooth the DEM before you use it for analysis or to generate a shaded relief model. The Scan operation is used to smooth anomalous values. If you wish to apply a weighted average method of smoothing, use the Filter operation. The Scan operation uses a moving window to reassign values. The larger the scanning window the greater the number of values that will contribute to the assignment of new values. The default window size is 3 cells in HD-SMT-4
diameter. A test on the DEM proved this to be too small to remove the grid pattern, so a larger 5 cell diameter window is specified: If you were to perform this operation from the Script window the statement would be: Pine Smoothed = Scan Pine Within 5 Average; HD-SMT-5
This operation smooths the values in the Pine DEM. Anomalous, high frequency changes are reduced while preserving the overall trends in elevation variation: This smoothed DEM is now ready for analysis or to generate a shaded relief map layer. Apply the Filter operation with the kernel Shading Kernel to the smoothed DEM: If you were to perform this operation from the Script window the statement would be: Shaded Smoothed = Filter Pine Smoothed Kernel Shading Kernel LowPass; HD-SMT-6
This operation produces a shaded relief model of the Pine DEM lit from the Northwest. Compare the results of the unsmoothed and the smoothed DEM: Smoothing an Extreme Case of Artifacting in a USGS DEM From time to time you may encounter a DEM (Digital Elevation Model) with extreme artifacting. This can include blocky segments, banding, and extremely anomalous values. Low quality source data and poor seaming are typically the causes of artifacting. HD-SMT-7
The DEM map layer named Hope Valley, RI appears to have some minor banding and artifacting: HD-SMT-8
However, using this DEM to generate a shaded relief model reveals a large number of artifacts over the whole map layer, including blockiness, banding, and spiky values: HD-SMT-9
To make an effective shaded relief model or to use this DEM in drainage and viewshed analysis, these artifacts must be reduced or removed. The following script will incrementally remove the anomalies and produce a higher quality DEM and shaded relief model (Note: the kernel named Shading Kernel is the same kernel as was used in the previous example): Smooth1 = Scan Hope Valley, RI Average Within 7; Smooth2 = Scan Smooth1 Average Within 5; Smooth3 = Scan Smooth2 Average Within 3; Hope Final Shaded Relief = Filter Smooth3 Kernel Shading Kernel LowPass; Save Hope Final Shaded Relief ; HD-SMT-10
Example Reduce Noise and Speckling in a Satellite Image There are many factors that can contribute to the degradation of a remote sensing image. Generally it is recommended that you reject images that are badly degraded; however, there are times when it is necessary to use whatever image is available, regardless of quality. For example, a noisy image may be the only image available for a time sensitive study. Given below are two SPOT images of the Highway 401 and Wellington Road interchange in London, Ontario, Canada. These images both depict green wavelengths. The image on the right is contaminated by signal degradation; however, it is the only image available for the particular day of interest. This image must be cleaned up as much as possible so that it can be used in a false colour composite of this area that will be generated using the Merge operation: You can use MFworks to smooth out noise in one of two ways, either by applying the Filter Low Pass operation or the Scan Average operation. Each operation has certain advantages. The Filter operation allows you to specify the default or a custom made convolving kernel. If you are less familiar with remote sensing image processing you may want to avoid the Filter operation and, instead, use the Scan Average operation. The Scan Average operation allows you to apply a standard mean filter to your data. You can specify different window sizes to increase or decrease the set of values from which the average value will be calculated. Both the Filter operation and the Scan operation will reduce, but not completely remove, the effects of degraded data. Below are three possible ways to reduce the noise in the XS1+P image. The first is the Filter Low Pass operation using the default kernel. Next is the Scan Average operation using the default 3 cell diameter scanning window. Last is the Scan Average operation using a 5 cell diameter scanning HD-SMT-11
window. For your own images, experiment with different diameter windows. Those users familiar with remote sensing image processing can create their own kernels for the Filter Low Pass operation. This Filter dialog box displays the default settings for the Filter Low Pass operation: If you were performing this operation from the Script window the statement would be: XS1+P Filter LowPass = Filter XS1+P (Noisy) LowPass; HD-SMT-12
This Scan dialog box displays the default settings for the Scan Average operation: If you were performing this operation from the Script window the statement would be: XS1+P Scan Avg 5 = Scan XS1+P (Noisy) Within 5 Average; This Scan dialog box displays the settings for the Scan Average operation with a 5 cell diameter Window size specified: HD-SMT-13
Each of these operations produces slightly different results. The larger the scanning window, the more smoothed and blurred the image becomes. You will have to decide for yourself which image best suits your purpose: Example Smoothing Anomalous Values in Ordinal Data by Nearest Neighbour Interpolation If you have anomalous or missing values in Ordinal (ranked) type data, filling them in by averaging does not make sense. They should be assigned actual values from the data set rather than the intermediate values that would result from averaging. To interpolate missing values or remove anomalous values in ordinal data, use the Scan Median operation. HD-SMT-14
The map layer Ranked Suitability for Agri was generated by combining remote sensing imagery, soil map layers, drainage map layers, and land use map layers. Due to anomalies in the remote sensing data, some cells were not ranked and remained as VOID : To assign a ranking to these cells, the following Scan Median operation will be used: If you were to perform this operation from the Script window the statement would be: HD-SMT-15
Smoothed Ranked Suitability = Scan Ranked Suitability for Agri Within 3 Median; This Scan operation assigns values to the unassigned cells based on the median value of the nearest neighbour cells: Example Smoothing Anomalous Values in Nominal Data by Nearest Neighbour Interpolation If you have anomalous or missing values in Nominal/Categorical (e.g., classified satellite data) data, filling them in by averaging will not yield desirable results. They should be assigned actual values from the data set rather than the intermediate values that would result from averaging. To interpolate missing values or remove anomalous values in Nominal data, use the Scan Majority operation. The map layer Chatham Classified was generated by performing an unsupervised classification on remote sensing imagery. Some Forest cells HD-SMT-16
within the forest patches were not properly classified and some Non-Forest cells were classified as Forest: HD-SMT-17
To reclassify these cells based on the most common value of the four nearest neighbours, the following Scan Majority operation will be used: If you were to perform this operation from the Script window the statement would be: Chatham Scanned 3 Maj = Scan Chatham Classified Within 3 Majority; HD-SMT-18
This Scan operation assigns values to cells based on the highest, most common value of the nearest neighbour cells: If you want to produce a map that shows you which cells were reassigned and which remained unchanged add the scanned map layer to the original map layer. You must do this in the Script window. Select New Script from the Windows menu and enter the following statement in the Script window: Chatham Changed and Unchanged = Chatham Scanned 3 Maj + Chatham Classified ; Adding these two map layers together produces a map layer that reveals those cells that changed as a result of the Scan operation. The majority of HD-SMT-19
changed cells, those with the value 1, were forest and non-forest cells that were isolated within patches of the other coverage type: HD-SMT-20