Single-Frame Image Processing Techniques for Low-SNR Infrared Imagery

Transcription

1 Single-Frame Image Processing Techniques for Low-SNR Infrared Imagery Richard Edmondson, Michael Rodgers, Michele Banish, Michelle Johnson Sensor Technologies Huntsville, AL Heggere Ranganath University of Alabama in Huntsville Huntsville, AL Sensor Technologies is a small, woman-owned business with expertise in the following areas: Custom Optical Sensor Prototypes Advanced Signal Processing Algorithms 3D Visualization Technology Field Test support and Data Analysis Diverse Expertise and Customer Base DoD and DHS Small Business Innovation Research Collaborating with Large and Small Businesses Multiple contract vehicles for Engineering Services (CIMTIC, AMCOM, SETAC, etc.) Founded in 2003 and located in Huntsville, Alabama Meet us in booth # 28! is marketing our Sensors, Tracker, and 3D Visualization System.

2 Problem Being Addressed Detect a signal of small spatial extent in infrared imagery at low Signal to Noise Ratio (SNR ~2 ) SNR defined as peak signal intensity divided by spatial standard deviation of background Initial Approach Detect target in uncorrected infrared imagery (no NUC) Pulse Coupled Neural Network (PCNN) Signal eliminates is blurred due to mechanical motion - DC data of no interest Unfortunately, there was no way to devise a set of optimal control parameters An acceptable solution was achieved with training IR HWIL Data: Difficult data set to process artifacts, low S/N PCNN offers a solution with significant improvements over thresholding Best performance to date offers >7000 fewer False Target Pixels Approach Segment signal pixels from background pixels as near the focal plane as possible Explore ways to optimize and automate PCNN solutions that were identified in prior research as candidates for signal segmentation Evaluate simple spatial filter techniques that exploit sensor and readout circuit properties to segment signal Compare and report the results of all viable approaches Sample input imagery at 2 contrast settings (SNR = 2.5), and sample PCNN binary output Fixed pattern noise reduced by background subtraction; readout noise still apparent 2

3 Pulse Coupled Neural Network A pixel feeds a neuron with feed-forward and feed-back input from a local neighborhood and the network is inspired by vision biology The neuron state is compared to a threshold that decays until the neuron fires, at which point the threshold is increased (pulse) Linking and Feeding kernels are used to weight the neighborhood around the neuron when evaluating its state S is input signal, U is neuron state, Y is activation state, Θ is the threshold " Variables are the a values, b, and V values, as well as the size and weighting of the Linking and Feeding kernels, W and M F ( n) = e L ( n) = e U ( n) = F ( n Q ( n) = e F ( n -) + S + V n L ( n -) + VL )[ + bl ( n) ] M ì, if U( n) > Q ( n -) Y ( n) -a F Dn -a LD -a QDn Q( n -) + V Y ( n) Q F W Y ( n -) Y ( n -) Equations are evaluated iteratively Each iteration is called a pulse PCNN For Image Processing Research has shown there is value in using PCNN for image processing Segmentation (Ranganath, Banish, Lindblad, Kinser) Edge detection (Lindblad, Kinser) Target recognition (Lindblad) Each input pixel feeds a single neuron in the PCNN Problems using PCNN for image processing Heuristics for selecting coefficients require human intervention Not previously possible to determine on which pulse the feature of interest will be identified, meaning that human evaluation of output after each pulse is necessary Ranganath and Banish explored method of using image statistical properties to set coefficients, and evaluated using only one pulse Single pulse does not fully allow power of decaying threshold to segment image Our solution is use of Genetic Algorithm (GA) to select coefficients which segment image on the pulse of our choosing Typical scene and signal used for algorithm development 3

4 Genetic Algorithm for PCNN Genetic Algorithm (GA) allows use of large data set to search for a solution to a nonlinear problem. It is an intelligent exploitation of a random search that finds acceptably good solutions. The values for each variables in the PCNN are encoded into bit fields in a chromosome that represent a vector of PCNN coefficients Each input vector of PCNN coefficients, called an individual is scored for fitness as a solution Individuals are selected for mating within and between populations and sub-populations The GA used Rank order selection Demes (sub-populations) Uniform crossover Scoring was based on Successfully identifying the signal Failure to identify the signal Identifying pixels not on signal in the output (false alarms) The PCNN was always scored after a specific pulse in our case pulse 4 but the pulse number could have been encoded as another variable in the chromosome Typical Results Point signal blurred across several pixels (jitter) SNR defined as peak signal counts divided by the spatial sigma of the background Input Image Indicates Program Goals Truth Image Output Image 4

5 Spatial Filtering Approach Evaluate methods that reject readout and other noise sources, but return signal pixels Perform statistical analysis of background imagery Devise methods to segment small signals from background using a variety of kernels Convolution kernels can be implemented using a shift register scheme in an FPGA Requires only multiply/accumulate operations and comparison to a threshold InSb and HgCdTe IR FPA imagery have distinct properties such as focal plane non-uniformity and column readout noise that make segmentation of point signals difficult. Kernels for Spatial Filtering Local Mean versus Global Threshold Local Mean versus Border Threshold, aka Box Filter Intensity versus Local Shaped Threshold, aka Cross Average Filter Intensity versus Local Shaped Threshold, aka Line Average Filter 5

6 Simple Thresholding Kernels Local Mean m = S 9 ì if m > mspatial + 2 s Y î0 spatial Local Mean Filtering simply compares the mean of a region against the global mean of the image plus two times the standard deviation Mean and standard deviation can be computed on previous image if background image properties change slowly Requires relatively uniform background Filtering with the Box Filter compares the mean of a region against mean of a border region plus two times the standard deviation Mean and standard deviation can be computed on previous image if background image properties change slowly Requires relatively uniform background m = 9 b = 24 mn mn ì if m > b + 2 s Y S S, where m and n specify the 24 border pixels in a 7x7 neighborhood about the pixel of interest spatial Box Filter Kernels with Shaped Thresholds Cross Average Filter m = 20 ì, if S > m + 3 s T ì, if T > 5 Y S, where k and l specify th e 20 pixels that form a cross pattern in a 9x9 neighborho od ± 4 pixels from the pixel of interest spatial where k and l specify a 5x5 kernel about the pixel of interest Line Average Filter m = 0 ì, if S > m + 3 s T ì, if T > 5 Y S, where k and l specify th e 0 pixels that reside in a 9 pixel line spatial ± 4 where k and l specify a 5x5 kernel about the pixel of interest pixels from the pixel of interest These two techniques work to identify signal in backgrounds which exhibit linear fixed pattern noise properties, such as column readout noise Both filters compute a temporary state indicating whether the signal intensity exceeds the mean of the linear regions in the kernel by three global standard deviations Algorithm output values are computed using a median filter; i.e. counting the number of on pixels in a region 6

7 Comparison of Algorithms Probability of Detection (Pd) PCNN provides best detection at very low SNR Shown are PCNNs trained with 50 and 250 false alarm pixels per frame allowed Using the GA, PCNN solutions can achieve Pd approaching 90% at SNR of 2 if the application can tolerate 500 to 000 false alarm pixels per frame GA scoring can be adjusted to create solutions that favor either Pd or low false alarms Single Frame Probability of Detection Results Signal to Noise Ratio (SNR) PCNN trained with 50 allowable False Alarms Local Mean Box Filter Cross Average Line Average Vertical Line Average PCNN trained with 250 allowable False Alarms Comparison of Algorithms Spatial filters provide the simplest implementation to low SNR detection, but are not effective below SNR of 3 Spatial filters excel at rejecting background, but do so at the expense of signal detection at low SNR More highly dependent on spatial extent than PCNN Single Frame Probability of Detection Results Probability of Detection (Pd) Signal to Noise Ratio (SNR) PCNN trained with 50 allowable False Alarms Local Mean Box Filter Cross Average Line Average Vertical Line Average PCNN trained with 250 allowable False Alarms 7

8 Comparison of Results at Low SNR for Simulated IR Backgrounds and Single Point Signals Input Truth Local Mean Box Filter Image Image SNR of 2.5 is below the performance ability of spatial filters PCNN identifies pixels on all signal patches Cross Line Matched PCNN Average Average Filter Input is Background Subtracted. Readout noise is apparent, but non-uniformity of background is reduced All non-zero signal pixels are shown Green pixels are on signal, red pixels are missed signals, and yellow are false alarms All non-zero pixels for each signal patch are shown, but only 2-3 pixels per patch are more than a few counts DSP/FPGA Processing Solutions Boards to host these and other image processing algorithms in DSP and FPGA nearing completion Applications include UAV, UGV, and remote sensing for homeland security and vision based guidance and control Flexible Programming Environment High Speed Operation Memory Onboard High Speed Bus Real Time Processing Digital and analog inputs and outputs Gen : one TI DSP and Gen 2: two TI DSPs. Both with an aggregate computational rate of 4.8 billion fixed point instructions per second or 3.6 billion floating point instructions per second. Gen : Lattice FPGA and Gen 2: Two Lattice FPGAs. Each DSP has access to 28 Mbytes of zero wait state synchronous dynamic RAM Zero wait state Dual Port RAM. Bi-directional communications FIFOs. Gen : implements one Gen 2: implements two 00 MHz external memory interface (EMIF) buses each capable of sustaining 400 MByte/second data transfers. Command/Status registers and interrupt logic provide for inter-processor data sharing and handshaking. 52 x 52 imagery at 400 frames/second. This estimate assumes 50 fixed point operations/pixel. EMIF interface port is attached to each bus to allow the use of custom or 3rd party daughter cards. 8