COMPARISON OF SENSOR NOISE EFFECTS ON FITTS AND PROJECTION BASED PHASE ONLY CROSS CORRELATION ALGORITHMS FOR HIGH SPEED VIDEO TRACKERS (POSTPRINT)

Transcription

1 AFRL-RD-PS- TP AFRL-RD-PS- TP COMPARISON OF SENSOR NOISE EFFECTS ON FITTS AND PROJECTION BASED PHASE ONLY CROSS CORRELATION ALGORITHMS FOR HIGH SPEED VIDEO TRACKERS (POSTPRINT) David C. Dayton, et al. A-Tech Corporation 1300 Britt Street NE Albuquerque, NM September 014 Technical Paper APPROVED FOR PUBLIC RELEASE; DISTRIBUTION IS UNLIMITED. AIR FORCE RESEARCH LABORATORY Directed Energy Directorate 3550 Aberdeen Ave SE AIR FORCE MATERIEL COMMAND KIRTLAND AIR FORCE BASE, NM

2 NOTICE AND SIGNATURE PAGE Using Government drawings, specifications, or other data included in this document for any purpose other than Government procurement does not in any way obligate the U.S. Government. The fact that the Government formulated or supplied the drawings, specifications, or other data does not license the holder or any other person or corporation; or convey any rights or permission to manufacture, use, or sell any patented invention that may relate to them. This report was cleared for public release by the Air Force Research Laboratory [insert TD site] Public Affairs Office and is available to the general public, including foreign nationals. Copies may be obtained from the Defense Technical Information Center (DTIC) ( AFRL-RD-PS-TP HAS BEEN REVIEWED AND IS APPROVED FOR PUBLICATION IN ACCORDANCE WITH ASSIGNED DISTRIBUTION STATEMENT. //Signed// ISSAC THORNTON Program Manager //Signed// KENTON T. WOOD, DR-IV, DAF Chief, Laser Division This report is published in the interest of scientific and technical information exchange, and its publication does not constitute the Government s approval or disapproval of its ideas or findings. *Disseminated copies will show //signature// stamped or typed above the signature blocks.

3 REPORT DOCUMENTATION PAGE Form Approved OMB No Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing this collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports ( ), 115 Jefferson Davis Highway, Suite 104, Arlington, VA Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT DATE (DD-MM-YYYY) REPORT TYPE Technical Paper 3. DATES COVERED (From - To) 30 June September TITLE AND SUBTITLE 5a. CONTRACT NUMBER Comparison of Sensor Noise Effects on Fitts and Projection Based Phase Only FA C-071 5b. GRANT NUMBER Cross Correlation Algorithms for High Speed Video Trackers (PostPrint) 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER David C. Dayton, Rudolph Nolasco, Mary Lou Robinson, James B. Lasche 5e. TASK NUMBER 5f. WORK UNIT NUMBER D PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER A-Tech Corporation 1300 Britt Street NE Albuquerque, NM SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR S ACRONYM(S) AFRL/RDLT Air Force Research Laboratory 3550 Aberdeen Ave SE Kirtland AFB, NM DISTRIBUTION / AVAILABILITY STATEMENT 11. SPONSOR/MONITOR S REPORT NUMBER(S) AFRL-RD-PS-TP (377-ABW ; 30 Sep 014. OPS ). 13. SUPPLEMENTARY NOTES Accepted for publication in the Proceedings of SPIE, Volume 97, Published 30 Sep 014, 1 papers (16 pages), ISBN: Government Purpose Rights 14. ABSTRACT The Fitts correlation algorithm has been widely used for over forty years in high speed video trackers. It has the advantage that it is very simply implemented in a digital computer with a small number of calculations. At each step the algorithm attempts to estimate the shift between an image of a moving target and a proto-type image. There are several well-known short comings of the Fitts algorithm. First the error in the shift estimate increases if the shift is greater than one pixel of the digital image. Second the Fitts algorithm is susceptible to errors from sensor noise if the video images have low signal to noise ratio. These errors can force a lower tracker closed loop bandwidth to maintain track loop stability. An alternative correlation tracker algorithm is known as Projection Based Phase Only Correlation. In this paper we compare the two algorithms with respect to the effect of sensor noise. 15. SUBJECT TERMS Fitts Algorithm, Fast Cross-Correlation, Phase Only Matched Filter, LADAR 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT a. REPORT Unclassified b. ABSTRACT Unclassified c. THIS PAGE Unclassified SAR 18. NUMBER OF PAGES 11 19a. NAME OF RESPONSIBLE PERSON Issac Thornton 19b. TELEPHONE NUMBER (include area code) Standard Form 98 (Rev. 8-98) Prescribed by ANSI Std i

4 Comparison of Sensor Noise Effects on Fitts and Projection Based Phase Only Cross Correlation Algorithms for High Speed Video Trackers David C. Dayton a, Rudolph Nolasco a, Mary Lou Robinson b, James B. Lasche b a Applied Technology Associates, 1300 Britt SE, Albuquerque, NM 8713 b Air Force Research Laboratory, RDLT, 3550 Aberdeen SE, Kirtland AFB,NM ABSTRACT The Fitts correlation algorithm has been widely used for over forty years in high speed video trackers. It has the advantage that it is very simply implemented in a digital computer with a small number of calculations. At each step the algorithm attempts to estimate the shift between an image of a moving target and a proto-type image. There are several well-known short comings of the Fitts algorithm. First the error in the shift estimate increases if the shift is greater than one pixel of the digital image. Second the Fitts algorithm is susceptible to errors from sensor noise if the video images have low signal to noise ratio. These errors can force a lower tracker closed loop bandwidth to maintain track loop stability. An alternative correlation tracker algorithm is known as Projection Based Phase Only Correlation. In this paper we compare the two algorithms with respect to the effect of sensor noise. Keywords: Fitts Algorithm, Fast Cross-Correlation, Phase Only Matched Filter, LADAR 1.0 INTRODUCTION Correlation based shift estimation algorithms are often used in tracking systems to estimate the change in position of an object in an image frame and a reference image. The image shift is estimated from the peak of the cross-correlation. The main issue with such an approach is that the cross-correlation operation is computationally intensive. Most tracking algorithms run at a high frame rate and the shift estimation algorithm is required to operate in real time. In addition a reference image is usually unknown when the algorithm is initiated or it may be changing with time and so must be estimated on the fly. The reference image must often be estimated via some sort of maximum a-postiori approach, for this reason it is often referred to as the MAP image. This is usually done with a straight forward recursive averaging algorithm. One of the most widely used fast correlation algorithm is known as the Fitts 1 algorithm. In use since the 70 s it is simple and fast. A second fast algorithm is the projection-based, which reduces a -D crosscorrelation into two 1-D cross-correlations. In this paper we introduce a projection-based phase only (PBPO) cross-correlation algorithm, which is a hybrid of the phase only matched filter and projectionbased cross-correlation. The purpose of this paper is to make some comparisons between the Fitts and PBPO algorithms..0 FITTS ALGORITHM The Fitts algorithm 1 has been widely used for correlation trackers over forty years, primarily due to its simplicity and corresponding speed. Consider Taylor series expansion of a proto-type image w shifted by some amount d, w( w ( x d x ) = w( d + higher order terms. (1) x Note that w is a two dimensional image, but for brevity we only use a single index x. Let our measurement be 1

5 w( δ ( = w( x δ) w( δ, () ( where we keep only the linear term of the expansion. Then 1 d = ( d( w( ), (3) w( ( for all x over the image. This results in a system of equations from which we can get a least squares estimate of the shift d. Fitts 1 puts this into the form of a least squares matched filter, ˆ 1 d = W ( ( d( w( )dx, (4) c image where w( c = dx x, (5) image and w( W ( =. (6) x where g represents measurement errors and any aspect change of the target. To implement equation (4) we must have an estimate wˆ ( of the image of the object we are trying to track. The usual approach is to attempt to obtain a Maximum a-posteriori estimate of the object, or MAP for short, from a recursive average of past measurements. The derivatives in equations (5) and (6) must be estimated from the pixilated image data. w ( w( x 1, w( x + 1, (7) x w( w( y 1) w( y + 1) y (8) Sub-pixel shifts can be obtained directly from the Fitts algorithm without any special added processing, however the algorithm starts to break down with shifts greater than one pixel 3,4, PROJECTION-BASED PHASE ONLY ALGORITHM Image shift is often estimated via a cross-correlation based approach. The idea is to estimate the shift of the object by the peak of the cross-correlation between the image measurement and the MAP estimate of the object. The cross-correlation between a measurement image y( and the MAP estimate w( can be given by :

6 r w, y ( z) = w( d( z + x ) dx. (9) The normalized cross-correlation estimate of the shift is then given by: r ( ) ˆ w, d z d = arg max z (10) r w, w ( z) r d, d ( z) where the denominator mitigates effects of the measurement and MAP image shape on the crosscorrelation. It is computationally more efficient to calculate the cross-correlation in the frequency domain using fast Fourier transforms. r ( ) = I 1 I( ( )) I( ) ( ) * w, d z d x w x (11) Unfortunately there is no direct Fourier transform analogue to efficiently calculate the normalized crosscorrelation shown in equation (11). There is an approximate relationship however known as the phase only matched filter 6,7. In the Fourier domain I I ( d( ) I( w( )* ( d( ) I( w( ) (1) In equation (1) we use the Fourier amplitudes to normalize for product of the complex Fourier transforms, essentially keeping only the phase information of the numerator. ( d( ) I( w( )* ( ) ( ) d( I w( ) ˆ d I = I 1 y arg max z y I y (13) Horner 8 has shown that equation (11) produces a much more peaked result on which to perform shift estimation by finding a maximum than equation (8). In equation (7)-(11) d and w represent two dimensional images. We carried only one index variable just to make the equations simpler. The Fourier transforms shown were two dimensional. The number of computations needed to produce the shift estimate shown in equation (11) can be significantly reduced by using a projection based algorithm. The projection based algorithm replaces the two-dimensional Fourier transform by two one-dimensional transforms, greatly reducing the total number of calculations.. 3

7 Sum over columns Sum over rows x 1,1 x 1, x,1 x 1,p x,p Column Projection x p,1 x p, Row Projection Fig. 1 Illustration of the row and column projections for the -D projection algorithm. The projection concept is illustrated in Fig. 1. Two projections are formed by summing along the rows of the image and then summing along the columns resulting to two 1 dimensional arrays. Shifts in an image measurement with respect to a reference image can then be estimated by the peaks in the 1-D crosscorrelation. Cain 9 has shown that the location of the two 1-D cross-correlation peaks is the same as the location of the -D cross-correlation peak. The PBPO cross-correlation algorithm uses only the phase to perform the 1-D Fourier domain crosscorrelations. It is an approximation to the normalized cross-correlation approach that has been widely used.. ( ) ( ) ( ) ( ) I I * ˆ = I 1 d( w( d x arg max z (14) x I d( I w( ( d( ) I( w( )* ( ) ( ) d( I w( ˆ I d = I 1 y arg max z (15) y I 4.0 ALGORITHM COMPARISONS The beauty of the Fitts algorithm is its simplicity and speed. The motivation for looking at the PBPO algorithm is to achieve a fast shift estimation algorithm that can be run in real time while overcoming some of the shortcomings of the Fitts algorithm 10. It is well known that the Fitts algorithm s shift estimation error increases for shifts greater than one pixel. The Fitts algorithm works best with a high contrast objects against a smooth background. If the MAP image contains a lot of high spatial frequency background clutter, the differencing operation shown in eqns. (7) & (8) can lead to additional errors. The projection operation in the PBPO algorithm actually helps to smooth out high frequency noise and background clutter in both the MAP image and the measurement image. 4

8 .5 Fitts Performance with Gaussian Target (sigma.0 pi Fitts Ideal.5 PBPO Performance with Gaussian Target(Sigma.0 Pixels) PBPO PBPO Idea Fitts Shift (pixels) PBPO Shift (Pixels) Gain 1.0 Linear Range 0.88 R Applied Shift (Pixels) Applied Shift (Pixels) a) Fitts Algorithm b) PBPO Algorithm Fig. Comparison of Shift Error Between Fitts Algorithm and PBPO. Figure shows a comparison of the shift errors in the two algorithms as a function of the applied shift input. It can be seen that Fitts error increases when the shift is greater than one pixel, while the PBPO shift estimate remains linear out to many pixels. The stair step error in the PBPO shift estimator (Fig -b) is due to the pixilation in the 1-D discrete Fourier transforms used in its calculation. Fitts Mults N PBPO Mults 4Nlog N + 8N Table 1 Number of multiplication operations for NxN images. In evaluating the performance of an algorithm, the most time consuming operation is multiplication, and so we consider the number of multiplication operations required to implement the cross-correlations. These are shown in Table 1. It is assumed that the measurement and the MAP are NxN images. The Fitts algorithm requires N multiplications. Since the PBPO uses 1-D Fourier transforms of the projections, the Fourier transforms each require Nlog N multiplications forward and backward. There are then N complex multiplications each requiring 4 real multiplications. Table 1 indicates that if N is larger than 3, the PBPO algorithm will require fewer multiplications than the Fitts. 5.0 NOISE The derivations of the Fitts algorithm in Section.0 ignored error in the measurements d(. In fact d '( = d( + n( (16) Where n( is the measurement noise and is assumed to be uncorrelated Gaussian. Inserting equation (16) into equation (4) gives the noise associated with the shift estimate. 1 dˆ ' = d + W ( ( n( )dx c image (17) The second term in equation (17) is the noise term. This term is the correlation of the noise with the gradient of the MAP image W(. The projection based PBPO algorithm, as outlined in Section 3.0, first calculates the two projections of the measurements by summing over the rows and columns. The 1-D correlations are then formed with the two 5

9 projections of the MAP image. In this case the noise term can be given by the 1-D correlation between the projections and the projection of the noise. dˆ ' = d + w( n( dx y y (18) The projection operation acts as a smoothing filter. Thus in most cases we would expect the second term of (18) to be less than the second term of (17). 6.0 CONCLUSIONS High speed video trackers rely on fast correlation algorithms. Direct correlation calculations are far too computationally intensive to operate in real time. The Fitts correlation algorithm is widely used because it is simple and straight forward to implement and can produce sub-pixel shift estimates directly from the calculations without the necessity of Fourier transforms. It starts off with an optimal matched filter. An approximation to the matched filter weights is made using only the first term of a Taylor series expansion. Because only the first term in the expansion is used, shift estimate error increases when the shift is greater than one pixel. The PBPO algorithm is conceptually more complicated, however if it is carefully implements there are actually fewer multiplication operations. It is a hybrid approach which combines the projection based crosscorrelation algorithm with phase only matched filtering. It works well with multi-pixel shifts and because the projections are a smoothing operation, it works well with high frequency noise and clutter in the background. The PBPO algorithm uses projections of the measured data in the x and y directions. Since the projection operation acts as a low pass filter, we expect that the noise associated with this algorithm to be less than that associated with the Fitts algorithm. 6.0 REFERENCES [1] J. Fits, Precision correlation tracking via optimal weighting functions, 18 th IEEE Conference on Decision and Control, pp 80-83, Ft. Lauderdale, Fl. (April 1973). [] R. Brunson, D. Boesen, G. Crockett, J. Riker, Precision trackpoint control via correlation track referenced to simulated imagery, SPIE Vol. 1697, Acquisition, Tracking, and Pointing VI pp , (199). [3] L. Stephan, J. Albus, J. Marcovici, Fitts correlation tracker fidelity in presence of target translation, rotation, and size change, Acquisition, Tracking, and Pointing XVI, SPIE Vol (00). [4] B. Fitzpatrick, S. McCanless, Y. Want, Bayesian statistical approaches to tracking through turbulence, Proceedings of the American Control Conference, Boston, Mass. pp (004). [5] M. Abdelkader, R. Chellappa, Q. Zhheng, Integrated motion detection and tracking for visual surveillance, Proceedings of the fourth IEEE International Conference on Computer Vision Systems, (006). [6] D. Barnea, H. Silverman, A class of algorithms for fast digital image registration, IEEE transactions on Computers, c-1,, Feb. (197). 6

10 [7] Q. Chen, M. Defrise, F. Deconinck, Symmetric phase-only matched filtering fo Fourier-Mellin transforms for image registration and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 1, (Dec. 1994). [8] J. Horner and P.Gianino, Phase only matched filtering, Appl.Opt., 3, 6, (Mar. 1984). [9] S. Cain, M. Hayat, Projection-based image registration in the presence of fixed-pattern noise, IEEE transactions on image processing, 10, 1, (001). [10] D. Dayton, J. Allen, R. Nolasco, Comparison of fast correlation algorithms for target tracking, SPIE Unconventional Imaging and Wave-Front Sensing, San Diego, Ca. (Aug. 01). 7

11 DISTRIBUTION LIST DTIC/OCP 875 John J. Kingman Rd, Suite 0944 Ft Belvoir, VA AFRL/RVIL Kirtland AFB, NM Issac Thornton Official Record Copy AFRL/RDLT 1 cy 1 cy 1 cy 8