Practical Testing of the Precision and Accuracy of Target Image Centring Algorithms. T. A. Clarke

Transcription

1 Practical Testing of the Precision and Accuracy of Target Image Centring Algorithms M. R. Shortis Department of Geomatics The University of Melbourne Parkville, Victoria 3052 AUSTRALIA Telephone : Facsimile : m.shortis@unimelb.edu.au T. A. Clarke Department of Electrical and Electronic Engineering City University Northampton Square London EC1V 0HB, ENGLAND Telephone : Facsimile : t.a.clarke@city.ac.uk ABSTRACT S. Robson Department of Civil Engineering City University Northampton Square London EC1V 0HB, ENGLAND Telephone : Facsimile : s.robson@city.ac.uk Close range photogrammetry and vision metrology often use signalised points in the form of active or passive targets. Many theoretical and some practical tests of different target image centring algorithms have been carried out. This paper will describe the empirical testing of several such algorithms using real data acquired for industrial measurement projects and camera calibrations. The precision and accuracy of the centring algorithms will be characterised by analysis of self calibrating network solutions using multiple camera stations and a target array. Particular emphasis will be placed on the comparison between centroiding and ellipse fitting to locate target image centres. Keywords: Target images, CCD cameras, Centring algorithms, Thresholds, Calibration, Precision, Accuracy 1.1 Background 1. INTRODUCTION CCD cameras and digital images are widely used for qualitative and quantitative applications of machine vision in the manufacturing, aerospace and automobile industries. CCD cameras are inexpensive, readily available and offer a wide variety of format, resolution, sensitivity and interface. Digital images are easily captured, stored, manipulated and analysed. Moreover, CCDs are geometrically stable sensors which allow the possibility of reliable and accurate measurement, and are relatively immune to environmental effects. Quantitative applications of these sensors has given rise to a relatively new science, variously known as vision metrology, videometrics or digital, close-range photogrammetry. Precise measurement of discrete targets is a fundamental process for videometric applications, especially in the aerospace and manufacturing sectors of industry. Although it is certainly possible to obtain precise measurements using image templates 14, the majority of industrial applications use circular, retro-reflective or passive targets to unambiguously signalise points of interest 10. The targets may be self-adhesive, or may be attached to tooling hole locators. Retro-reflective targets are illuminated using a flash or light source adjacent to the camera lens to obtain high contrast images of the targets. Passive targets require considerable attention to the ambient lighting conditions to achieve adequate contrast throughout the CCD images 1. Irrespective of the lighting techniques used, the intent is to capture images in which the discrete targets have a substantial difference in grey value compared to the general background. CCD cameras used for videometric applications are typically treated as a central, perspective projection system in the same way as traditional photogrammetric cameras. Knowledge of the internal geometry of the camera is essential if the principle of collinearity is to be correctly applied. Without this knowledge, derived measurements in the object space will be affected by systematic errors and therefore degraded in accuracy. It is generally accepted and has been widely demonstrated that selfcalibration is the most accurate method of modelling the systematic errors in the camera imaging system. Although selfcalibration has the substantial advantage that the calibration is carried out simultaneously with the measurement of the object of interest, a minimum geometric configuration for the network is necessary and cannot always be achieved. The physical environment or circumstances surrounding many videometric applications often precludes self-calibration, and the cameras must be pre- or post-calibrated. This is particularly true of measurement tasks which involve the tracking of objects in motion 22. Videometrics IV, SPIE Conference 2598 at Photonics East, Philadelphia, USA, October 25-26, 1995

2 Target image locations are used to compute the object space coordinates of the targets using a photogrammetric network solution based on collinearity. Depending on the application, this may be a simple resection and intersection process for multiple, synchronised CCD cameras 5, a bundle solution for multiple CCD cameras 19 or a bundle solution using inner constraints for a single, mobile, still video camera 12. In the former cases the CCD cameras are pre- or post-calibrated, whilst in the latter case the solution includes a self-calibration of the camera. 1.2 Target image location The measurement task necessary to precisely locate the target images can be broken into two phases. The first phase requires the target images to be identified within the CCD image. This can be accomplished using one of a number of techniques, such as manual pointing with a mouse and cursor, driveback using prior geometric knowledge 20, image subtraction or scanning for grey values above a global threshold 21 and filtering followed by scanning for specific patterns 27. Each discrete target image can then be isolated within a small, local window for further processing. The second phase is the subtraction of a local threshold to remove image noise, or the background image if the lighting conditions were less than optimal when the images were captured, followed by determination of the precise centre of the remaining target image. Part of the threshold and centre location computation process is necessarily the removal of extraneous image information, which may be intrusions of other targets into the local window or above-threshold portions of the background within the local window. Sometimes known as blob testing, this segmentation process aims to isolate the target image as a contiguous area of above-threshold pixels 21. Once isolated by the blob test, the detected image can be subjected to a number of geometric tests to ascertain whether it is a true target or a false alarm. For instance, a minimum size restriction can be tested to eliminate noise and circularity limits can be imposed to eliminate partly obscured targets 21. The critical components of the second phase are the algorithms used for determination of the grey value threshold and the target image centre. Whilst many different threshold algorithms have been suggested and implemented, including those by Wong and Wei-Hsin 29, Trinder 26 and Snow et al 24, there is little information published on comparative tests. Many theoretical and practical studies of centring algorithms have been carried out, including Trinder 26, West and Clarke 28 and Shortis et al 21. The trends predicted by such studies agree that the location precision should improve for larger targets, higher resolution sensors and more sophisticated algorithms. Unfortunately, relatively few reports of actual comparative tests outside of the laboratory are available. The majority of CCD cameras in use for industrial applications supply 8 bit radiometric information, which limits the grey scale range to 255. Whilst there is a finite limit on contrast between the target images and the background, lighting can generally be arranged so that the majority of target images are clearly separated from the background. Target size and sensor resolution are usually project-constrained because of physical restrictions on fields of view and camera station locations, as well as limits on the range of CCD cameras and lenses available to an organisation. As a consequence, a variety of target image sizes could be expected for a reasonable range of industrial applications. Clearly, there are limits on the radiometry and geometry of the target images which are likely to be acquired, but there is still likely to be significant variation. On the other hand, in theory there are no limits on the range of the algorithms which can be used for thresholds and centre locators for discrete target images. Every case of industrial metrology or camera calibration could be treated separately if there were some guidelines on which algorithms suited particular circumstances. In other words, the algorithms used could be selected, and perhaps optimised, to the radiometric and geometric characteristics of the CCD images. It is well beyond the scope of the paper to test all possibilities, so a few representative techniques will be assessed to determine whether there are significant variations in precision and accuracy resulting from the use of different algorithms. The assessment will be conducted by empirical testing of real data acquired for industrial measurement projects and camera calibrations. The precision will be estimated using the image residuals from the photogrammetric networks, which in most cases will be computed as self-calibrating, free network solutions. The a posteriori estimate of unit weight, which is an internal measure of image plane precision for the network, will be used. The accuracy will be estimated using root mean square (RMS) errors between the object space coordinates of the targets derived from the photogrammetric network solution and an independent, external reference set of coordinates. The latter were provided from a variety of sources.

3 2. PREVIOUS RESEARCH AND INVESTIGATIONS 2.1 Introduction A great deal of research into subpixel location techniques for discrete target images has been conducted in the areas of remote sensing, computer vision and, more recently, digital photogrammetry. Of the work that has been concerned with the analysis of target image location errors, few investigations have made an attempt to explicitly make the connection between location precision in image space and 3D measurement accuracy in object space, as determined from an independent test. Although this review is not completely comprehensive, it does indicate that, in general, this area has not been as closely analysed as it could be considering the importance of the issue to videometric applications in industry and manufacturing. 2.2 Image precision analyses Some of the earliest uses of CCD sensors for subpixel target location of bright objects were in the field of star tracking. Brook and Purll 3 discussed the use of the centroid method as a means of tracking the star Polaris for geosynchronous satellite orientation purposes. The use of the centroid method allowed a successful implementation of a relatively low resolution sensor for this purpose. Later work by Stanton et al 25 reported location precisions of 0.01 pixels for tracking star images on area CCD arrays. There have been a number of theoretical studies conducted in order to quantify the best performance possible for target image location algorithms. Trinder 26 investigated target location precision by means of simulation trials and concluded that a subpixel precision of 0.01 pixels or better was possible provided the signal to noise ratio was reasonable. West and Clarke 28 investigated the accuracy obtained using an optical triangulation system and compared it to a simulation that included both noise and quantisation effects. In both cases a cross-section of a target was considered. They concluded that given noise with a uniform distribution over eight pixels, a subpixel precision of approximately 0.07 of a pixel was possible. Clarke et al 6 developed an algorithm to estimate the error in target location of the centroid method. Various size targets were placed on a flat panel and lighting and viewpoints simulated to give the maximum range of target sizes and intensities in the images. It was concluded that the quantisation effects were small compared to other random noise sources. Shortis et al 21 analysed the best performance that can be expected from a range of algorithms including ellipse-fitting, binary centroiding, and Gaussian shape fitting. They concluded that errors of the order of 0.1 pixels were common for all the binary methods tested. The analysis indicated that the results would be largely unaffected by quantisation or reasonable threshold levels, and that a marginal increase in target location precision would be expected with larger targets. In contrast, any grey-scale method was likely to perform around an order of magnitude better. Many practical tests have been carried out to verify target location precision within the image space. Clarke 7 conducted tests to determine the location accuracy of white light and laser targets and found that a standard deviation of the error in location of the centroid method was about 0.03 pixels. Clarke and Wang 8 investigated the effects of noise from CCD cameras on target location where they measured the noise of a particular frame-grabber and camera combination and applied those noise levels to a simulation. The simulation results were checked, for example, by moving a target with a stepper motor controlled micrometer and then analysing the residuals of a best fit straight line. Typical target location errors obtained under stationary conditions had a standard deviation of about pixels whereas the standard deviation of the moving target was as high as pixels. Maalen-Johansen 16 used a movable target platform on which were placed several different size targets. He noted that the standard deviation of target locations was around 0.02 pixels in the best cases. Raynor and Seitz 17 tested the subpixel location accuracy of a CCD camera and their own design of frame-grabber. The tests were conducted with a moving target. They reported a RMS precision of better than 0.01 pixels. They concluded that this level of precision corresponded to a geometric resolution of better than 1:75,000 and that 1:100,000 was possible. 2.3 Object space precision and accuracy analyses El Hakim 9 reported the testing of CCD camera system together with a coordinate measuring machine (CMM) and metric photogrammetric system. The results for the CCD camera equated to a precision 0.1 of a pixel using either a binary centroid or a simple grey scale centroid algorithm. The results indicated a subpixel accuracy of 0.1 of a pixel (high contrast retro-reflective targets were not used). Bosemann et al 2 discussed experiments using a target test field with a number of cameras. For the

4 cameras used, image location precisions of 0.02 to 0.09 pixels were experienced in bundle adjustments. West and Clarke 28 compared their simulated precisions of 0.07 pixels to the random errors in a triangulation system under the same conditions, where a standard deviation of 0.06 of a pixel was measured. Chen and Clarke 4 compared the use of the squared centroid method with the centroid method and found small, insignificant differences between adjustments computed using either method. Robson et al 18 analysed the RMS errors in the location of straight lines used in plumb line calibration. After the lenses were calibrated, the global RMS error in the location of the cross-sections of lines was 0.04 pixels. However, in a photogrammetric evaluation the RMS error in image co-ordinates following the bundle adjustment were 0.06 pixels. Beyer 1 discusses the repeatability of measurements of a test field using least squares target matching techniques and through a considerable number of test results he concluded that the precision of target image location can be better than 0.02 pixels. Using a comparison between theodolite measurements and 48 images from an off-the-shelf CCD camera, a relative accuracy in the object space of 1:46,000 was verified. This work provides perhaps one of the most comprehensive studies of the performance of CCD camera systems used for 3D measurement because of the systematic analysis of the contributing error sources in the measurement system. However, Beyer acknowledges that it is difficult to compare these results to other systems because of the difficulty of comparing the size and type of targets, the depth dimension of test fields, the number of frames, and many other parameters. Gustafson and Handley 15 used the Videk Megaplus camera to measure a test field and reported RMS closures of triangulation that indicated precisions of 0.03 to 0.04 pixels. These results correspond to a relative accuracy of 1:50,000 in the object space, using an eight camera station network tested against data from a large format film camera. They suggested that there are considerable advantages in the measurement of multiple images, but the theoretical improvement was not matched in practice. However, dramatic improvements were expected when suspected unknown systematic errors were traced and properly modelled. 2.4 Summary The research conducted in this field during the last decade is reasonably consistent in the level of precision obtained in practical tests and the level of precision predicted within the image space. There is general agreement that precisions of 0.01 pixels are theoretically possible, and some practical tests have realised this level of precision. For direct triangulation systems there appears to be a good correlation between the prediction by simulations and the practical reality within the object space. However when videometric, multi-station networks are used this relationship no longer appears to hold true and image location accuracies of the order of 0.02 to 0.03 pixels are the most favourable results reported. The relative accuracies indicate that there are yet unmodelled systematic errors in the networks, or the internal precision estimates are overly optimistic. Potential systematic errors within the camera calibration include variation of distortion within the image field 11 and sensor unflatness 13. Until the reasons for the discrepancy are identified it is difficult to make direct comparisons between subpixel location methods as such errors are significantly smaller than those for the process as a whole. 3.1 Threshold computation 3. ALGORITHMS Four different types of algorithms for the computation of local grey level thresholds were selected for testing. The selections were made on the basis of published reports and experience with target images acquired for camera calibrations and videometric applications. All algorithms assume that the target is near the window centre, and the first two algorithms assume that pixels near the window edges are representative of the background. If these assumptions are not correct then a combination of iterative computation, blob testing and a shrinking window are used to centre on the target image and remove the influence of any extraneous intrusions into the window 21. Pixels above the computed threshold level are then infrequent and are generally eliminated by the blob testing Statistical This threshold algorithm has been adopted from experience with laboratory calibrations for CCD and still video cameras. Under these circumstances the lighting is usually tightly controlled and the background is virtually random noise. This noise, generated

5 from the minimal background illumination and the electronics of the video system, is assumed to approximate a normal distribution. The pixels at the edge of the local window at each target are used to compute a mean and a standard deviation of the grey values of the noise. The threshold is set at the mean plus a multiple of the standard deviation. Empirical testing has indicated that the addition of three standard deviations to the mean is the minimal threshold level required to remove the background. This algorithm may produce inappropriate threshold values for images which have non-random backgrounds, such as videometric applications where the ambient light cannot be completely suppressed Additive constant This straightforward and effective threshold has been suggested by a number of authors, including Snow et al 24. The maximum grey level at the edge zone of the local window is determined and a constant value, typically five grey values, is added. A fixed separation between the background and the threshold values has proved to be surprisingly robust for a range of applications Proportional This algorithm is based on the premise that the background level is linearly correlated to the overall brightness of the scene. The threshold value is computed as a proportion of the difference between the minimum and maximum grey values within the window. Fifteen percent of the grey value range has been adopted by experimental testing as the factor which realises reliable threshold values. For discrete targets which are clearly separable from the background, this algorithm tends to produce the highest threshold value of all the algorithms used Wong The Wong algorithm was one of the first threshold computation strategies to be published 29. The threshold value is computed as the average of the mean and minimum grey levels in the local window. This threshold is dependent on the size of the window, as a larger area tends to take in more background and lowers the threshold. The Wong algorithm is also very sensitive to the minimum value and very noisy images produce threshold levels below an optimal value for background removal. 3.2 Target location Four different types of algorithms for the computation of the target image centre were selected for testing. Again, the selections were made on the basis of published reports and experience with target images acquired for camera calibrations and videometric applications. All algorithms assume a target image which is a contiguous blob near the window centre, although only the ellipse fitting algorithm will be seriously compromised if this is not so. A further assumption is that a threshold level of grey value has been subtracted to remove the background, as described in the previous section Simple and square-weighted centroids The simple centroid algorithm is given in Trinder 26 and is a centre of gravity or first moment weighted by intensity, or in the case of digital images, grey value. The intensity weighted centroid is widely used for videometric applications of all types. This is essentially the base line algorithm against which all other algorithms are measured. The centroid computation may be trivially altered to use the square of the grey values. As suggested by Chen and Clarke 4, the use of squared intensity values emphasises the main body of the target blob where the grey values are highest. As a consequence, peripheral pixels at the edge of the blob are less influential, especially for small target images. The disadvantage of both types of centroid computation is that no measure of the relative precision of location is possible. Lacking any other information, all centroids must be assumed to be of equal precision Ellipse fitting Due to perspective distortion, circular targets are imaged as ellipses. The centre of the ellipse can be determined by a least squares estimation of the ellipse shape based on the edge of the target image blob 30. The edge of the blob is defined by the threshold crossing, in this case computed to sub-pixel accuracy from linear interpolation. Ellipse fitting is consequently sensitive to the adopted threshold. The advantage of the ellipse fitting is that a measure of precision of the centre location is provided by

6 the least squares estimation. The disadvantage, somewhat diminished by modern computer technology, is the increased computation load. Figure 1. Coal dredge Figure 2. Rudder tab FAJ Figure 3. Thomson calibration Figure 4. Art panel test 0.25mm Figure 5. Coal dredge discrepancy vectors 0.05mm Figure 6. Thomson calibration discrepancy vectors

7 3.2.3 Simple centroid with intensity range precision factor In order to take advantage of the simplicity of the centroid computation and provide a relative precision measure, a fourth algorithm was added. The simple centroid computation was used, but for the photogrammetric network computations a relative precision of the target image centroid locations was introduced, based on the intensity range. As it is accepted that a greater signal to noise ratio realises a more confident measurement of any variable quantity, the relative precision was adopted as the inverse of the grey level range between the peak and the background within the window. To avoid extremes in the range, the minimum precision value was limited to the a posteriori estimate of unit weight precision from network solutions for simple centroids, and the maximum value was capped at ten times the minimum value. For example, a target image with a grey value range of 250 might have an estimated precision of 0.03 pixels, whilst a target image with a grey level range of only 10 would have a corresponding estimated precision of 0.3 pixels. 4. DATA SETS Four data sets were chosen for the comparative tests. Two of the data sets are straightforward videometric applications, the third is a laboratory calibration and the fourth is a quality control test for a videometric application. The data sets utilise three different types of CCD camera to also provide a range of sensors and, as a consequence of the variation in geometry and image scale, a range of target image diameters. A pre-requisite for each data set was an independent set of object space coordinates for the targets, or at least some mechanism by which an accuracy test could be made. A summary of the data sets is given in Table 1. Data Set Coal Dredge Rudder Tab FAJ Thompson Calibration Art Panel Test Cameras Camera type Kodak DCS200 Kodak DCS200 Thompson TH31156 Pulnix TM6CN Sensor : readout type Still video : digital Still video : digital Scientific : slow scan 2/3 : RS-170 Resolution 1524H by 1012V 1524H by 1012V 1024H by 1024V 744H by 568V Pixel Size (µm) 9 by 9 9 by 9 19 by by 8.3 Focal length (mm) and 6.5 Network Free with self-calib. Free with self-calib. Free with self-calib. Constr. with pre-calib. Camera stations Images Targets Image scale 1:1800 1:340 1:55 1:140 Target precision (mm) Target image size (pixels) 4 to 6 6 to to 16 3 to 6 Target grey scale 20 to or to 250 Background grey scale 25 to to Accuracy check CRC-1 photography Theodolite CMM validation Plane fit to glass sheet intersection Targets All 18 All 160 Check precision (mm) (estimated) Table 1. Summary of the data sets used for the comparative tests ( predicted using an image space precision of 0.05 pixels) The coal dredge project and data set are described in full by Fraser and Shortis 12. The intent of the project was to periodically characterise the shape of a load bearing surface of a coal dredge. The surface was targeted with self-adhesive retro-reflectors and photographed with a Kodak DCS200 still video camera and a Geodetic Services CRC-1 large format film camera. A typical DCS200 image is shown in figure 1. Apparent from this image is the reduction in response from the retro-targets with increasing incidence angle of the flash illumination. The CRC-1 photographs were observed using a Geodetic Services Autoset-2 comparator as an initial verification of the DCS200 measurements. The rudder tab final assembly jig (FAJ) project and data set are also described by Fraser and Shortis 12. The measurement of the FAJ was a tool verification exercise, using three independent systems. A combination of self-adhesive and tooling hole locator

8 retro-targets was used to identify the critical points on the jig. The jig was photographed with a DCS200 camera and a Geodetic Services CRC-2 medium format film camera. Manual check observations using a Leica industrial theodolite measurement system were also made. A typical DCS200 image is shown in figure 2. Two sets of images were exposed, one set with the flash illumination set to saturate the target images (grey values of 250) and one set with the flash set to avoid saturation (grey values of 150). The flash failed for one of the saturated images, leading to 23 high grey level images and 24 lower grey level images. For the purpose of accuracy verification for the comparative tests described in this paper, the theodolite measurements will be used because the observations were made closer in time to the DCS200 image capture 12. The calibration of the Thompson scientific CCD camera was conducted as part of an investigation into calibration techniques at NASA Langley Research Center 23. Laboratory pre-calibration of CCD cameras is necessary for the cameras used within wind tunnels at Langley, as the geometry of the measurement networks precludes self-calibration. The cameras are calibrated by a combination of a plumb line field to determine lens distortions, and a targeted test range to determine the remaining calibration parameters using a constrained self-calibration 23. The two phases use passive lines and targets respectively, with carefully controlled illumination. The consistency of the target grey levels is evident in the image shown in figure 3. The accuracy test is provided in this instance by measuring the targeted test range with a CMM. There was, and still is, some doubt associated with the CMM coordinates due to the difficulty of manually measuring large diameter targets. The fine art panel glass sheet measurement was the base line test for a program of deformation studies. The art panels are composite wooden boards which have been supported by various cradle lattices. The program was to determine the magnitude and shape of bending due to variations in temperature and humidity for different wood and cradle types 19. The panels were to be placed in a controlled environment chamber and imaged using 5 off-the-shelf Pulnix CCD cameras. Virtually zero background images were obtained using a combination of controlled lighting and retro-targets. The cameras were pre-calibrated using a technique similar to that for the Thompson camera, however an initial test was carried out to verify the configuration and precision of measurement. For this test, the glass sheet was placed within the field of view and imaged. One of the images is shown in figure 4 and, like the coal dredge images, there is retro-target response variability due to the incidence angle of the flash illumination. The accuracy verification is provided here by computing the best fit plane to the glass sheet and comparing the out of plane errors. The measurement of the target images within the digital image files was carried out using a simple driveback process to place a local window 23 for the threshold and centre location computations. In addition to the blob test and geometric criteria described in the introduction, two additional tests were applied. For all images, a minimum of eight grey scale values between the minimum and maximum in the window was imposed to ensure that a target image was present, especially for the coal dredge and art panel data sets. For the ellipse fitting algorithm, if either the solution failed or the minimum number of edge points was reached due to rejections or the centre precision exceeded a preset maximum, the target image location was automatically eliminated. Due to these extra restrictions, the ellipse algorithm produced approximately 5% less targets for all data sets except the large target images of the Thompson calibration. 5. RESULTS AND ANALYSIS The results of the comparative tests are summarised in tables 1-4, for the coal dredge surveillance, rudder tab FAJ measurement, Thompson calibration and art panel glass sheet test respectively. The first three are based on free network solutions with selfcalibration of the single camera. Calibration parameters were limited to the primary physical parameters (principal point, principal distance, radial and decentring lens distortions, and in some cases, image coordinate orthogonality and affinity). Statistically insignificant parameters were suppressed. The art panel network was computed using a minimally constrained network solution with pre-determined calibration data for the five cameras. In each case image measurement outliers were rejected using a consistent confidence level. Although this leads to different numbers of rejected image measurements for the individual networks, this is typically standard practice for photogrammetric network computations. For each case of a centre location type and threshold type, three values are shown. The first is the internal precision measure. This quantity is the a posteriori image measurement precision from the network solution, which is closely related to the RMS image error. This value effectively indicates the level of internal misclosure within the network. Smaller magnitude values indicate better internal consistencies for the networks. The second value shown is the external accuracy measure. For the first three data sets, this quantity is the RMS difference between the independently derived target coordinates and the target coordinates determined by the network solution. In the first three cases, because free network solutions were utilised, a three

9 dimensional similarity transformation has been applied to remove overall datum, scale and orientation differences. For the art panel data set, the value is the RMS out of plane error for a best fit plane computation. Smaller magnitude values indicate a better level of agreement between the network solution and the independent data. The third value is a relative accuracy measure provided for the purpose of a simplified comparison between all data sets. For the first three data sets, these values are the ratio of the RMS differences and the combined precisions from the two separate determinations of the target coordinates. For the fourth data set, the value is the ratio of the out of plane RMS error and the single precision of the out of plane target coordinates from the network solution. Again, smaller magnitude values indicate a better level of agreement between the network solution and the independent accuracy test. Inspection of the results shown in the tables shows several clear trends. First and foremost, the variation of the results is more dependent on the centre location algorithm than the threshold algorithm. For this reason, the tables include average results for each centre location type. Results for the centroid algorithms are very consistent for each data set, and in many cases almost independent of the threshold algorithm, despite the fact that the thresholds vary by as much as a few tens of grey levels. The implication of this result is that the central, high grey values of the target images dominate the centroid computation. Changes at the periphery of the target image are relatively inconsequential if there is a significant separation between the grey levels of the target image and the grey levels of the background, as would be expected for retro-reflective targets. The ellipse fit algorithm appears to be more sensitive to the threshold type. This can be attributed to variation of the integrity of the edge of the blob, especially with smaller target image diameters. The wide variation in the results for the ellipse fits for the smallest target image diameters, the art panel data, is particularly noteworthy. In general, the lower thresholds (statistical, additive constant) produce better results for images with larger target image diameters (Thompson calibration), whilst the higher thresholds (proportional, Wong) produce better results for the smaller target diameters (coal dredge). The probable reason for this phenomenon is that larger targets are less affected by peripheral pixels at the edge of the target blob and therefore can afford a minimal threshold to obtain 100% of the signal. Higher thresholds for small target images guarantees background removal, but also results in some signal loss. The exception to this rule is the statistical threshold for the art panel. This is due to the near zero background, which is best modelled by the statistical threshold algorithm. Note that there is no data shown for the statistical threshold and ellipse fit algorithm combination because the edge interpolation process requires that a non-zero threshold value must be set. This led to so many rejections of data that the network was not comparable with those used for the other algorithms. Centre Location Type Threshold Type -> Statistical Constant Proportional Wong Average Simple centroid Image Precision Object RMS error Relative Accuracy Square-weighted Image Precision centroid Object RMS error Relative Accuracy Ellipse fit with Image Precision centre precision Object RMS error Relative Accuracy Simple centroid Image Precision with precision Object RMS error Relative Accuracy Table 2. Results for the coal dredge network (precision in pixels, RMS error in mm) The trend for the centre location algorithms is also apparent and consistent amongst the data sets. On average, the ellipse fit algorithm produces the poorest results, in terms of both internal precision and external accuracy, in all cases. It is no surprise that the best results for this algorithm are for the Thompson calibration. For this data there is the least disparity between the precision of the ellipse fit and the centroids, and the ellipse fit algorithm produces the best relative accuracy, but at the expense of absolute accuracy. The large target image diameters provide the largest number of edge points and therefore the most confident

10 estimates of the ellipse centres. Moreover, the discrete sampling of the CCD arrays has a commensurately lesser influence on the shapes of the target images, as compared to the other data sets with smaller target images. For all the data sets, the centre precision predicted by the ellipse fit algorithm was substantially over-estimated. Typically, values produced by the algorithm were of the order of 0.01 pixels, whereas the average a posteriori unit weight precision was 0.05 pixels or larger. Centre Location Type Threshold Type -> Statistical Constant Proportional Wong Average Simple centroid Image Precision Object RMS error Relative Accuracy Square-weighted Image Precision centroid Object RMS error Relative Accuracy Ellipse fit with Image Precision centre precision Object RMS error Relative Accuracy Simple centroid Image Precision with precision Object RMS error Relative Accuracy Table 3. Results for the rudder tab FAJ network (precision in pixels, RMS error in mm) Centre Location Type Threshold Type -> Statistical Constant Proportional Wong Average Simple centroid Image Precision Object RMS error Relative Accuracy Square-weighted Image Precision centroid Object RMS error Relative Accuracy Ellipse fit with Image Precision centre precision Object RMS error Relative Accuracy Simple centroid Image Precision with precision Object RMS error Relative Accuracy Table 4. Results for the Thompson calibration network (precision in pixels, RMS error in mm) Centre Location Type Threshold Type -> Statistical Constant Proportional Wong Average Simple centroid Image Precision Object RMS error Relative Accuracy Square-weighted Image Precision centroid Object RMS error Relative Accuracy Ellipse fit with Image Precision centre precision Object RMS error no data Relative Accuracy Simple centroid Image Precision with precision Object RMS error Relative Accuracy Table 5. Results for the art panel network (precision in pixels, RMS error in mm)

11 The three centroid algorithms produce very similar results in all cases. The centroid with squared intensity is generally the least precise and the least accurate. The exception to this rule is the relative accuracy in three cases, indicating that the a posteriori precision estimates from the networks are more realistic than the other algorithms. On the other hand, the simple centroid with and without the precision estimator for the network solution, produces the best precision, the best or comparable absolute accuracy, and comparable relative accuracy in all cases. On average, the simple centroid with the precision estimator produces marginally better results, especially in the case of the coal dredge where the target images are small and there is a variation in retro-target response. The results for the art panel suggest that the square-weighted centroid realises the best results, which is attributed to this data set having the smallest target diameters. As previously noted, there are often discrepancies between the internal precisions and external accuracies of videometric networks. The results obtained for these comparative tests confirm that this is the case for the first three data sets, as the relative accuracy measures are all near or above two, whereas the expectation is the near unity values of the fourth case. However, the discrepancies which form the basis for the accuracy testing are, on the whole, random in nature. Figures 5 and 6 show plots of typical vector discrepancies between the DCS200 and the CRC-1 networks for the coal dredge, and the Thompson network and the CMM coordinates for the calibration test field, respectively. Apart from some local groupings, there are no clearly visible systematic trends. Some of the vector patterns are consistent for all the Thompson networks, however given the doubt attached to the CMM measurements, it is unwise to draw any strong conclusions with regard to systematic errors in the photogrammetric process. 6. CONCLUSIONS This paper has investigated the effects of different threshold and centre location algorithms on the precision and accuracy of measurement of discrete target images. The precision and accuracy were determined from the analysis of four cases of videometric applications and camera calibrations in order to test a variety of sensors and environments. In general it can be concluded that the selection of the threshold and centre location algorithms will affect the outcome, and appropriate selections will improve the precision and accuracy. In general, centroid algorithms are very robust and relatively insensitive to threshold level changes. The ellipse fit algorithm for target location is more dependent on the threshold level and produces comparable results only for large target images. It can be concluded that, for the range of target diameters typical for videometric applications in industry and manufacturing, centroid algorithms will produce superior results. The threshold algorithm used in conjunction with the centroid algorithms realises only minor changes in the precision and accuracy. However, the results obtained here indicate that algorithms which produce minimal threshold values should be used with larger target image diameters, whilst algorithms which produce higher threshold values should be used with small target image diameters to ensure the removal of the background. Finally, the centroid algorithm with a precision estimator based on grey level range produced the best results overall for precision and accuracy. The precision estimator effectively weights the network solution according to the signal strength obtained for the target image. The arbitrary system of precision estimation adopted here is worthy of further investigation to ascertain whether additional improvements can be gained. Possible lines of research include a non-linear relationship between the signal and precision estimator, incorporation of a measure of target image diameter or area and the combination of the square-weighted centroid and a precision estimator. However, any further investigations should be conducted in conjunction with improvements in the photogrammetric model to resolve the inequalities between image space precision and object space accuracy. Refinements to the model will increase the significance and usefulness of target image precision estimations in videometric applications. 7. REFERENCES 1. Beyer, H.A., "Geometric and radiometric analysis of a CCD-camera based photogrammetric close-range system", Mitteilungen Nr. 51, ETH, Zurich, 186 pages, Bosemann, W., Godding, R. and Riechmann, W., "Photogrammetric investigation of CCD cameras", Int. Arch. of Photogrammetry and Remote Sensing, 28(5) : , Brook, R.A. and Purll, D.J., "Design of a Polaris star sensor using charged coupled device (CCD) imaging and LSI microprocessor-based signal processing", SPIE Vol. 178, pp , Chen, J. and Clarke, T.A., "The automatic location and identification of targets in digital photogrammetric engineering

12 measurement", Int. Arch. of Photogrammetry and Remote Sensing, 29(5): , Childers, B.A., Snow, W.L., Jones, S.B. and Shortis, M.R., "Support of wake vortex detection research in flight and wind tunnel testing using videometric techniques", Int. Arch. Photogrammetry and Remote Sensing, 30 (5) : 41-46, Clarke, T.A., Cooper, M.A.R. and Fryer, J.G., "An estimator for the random error in subpixel target location and its use in the bundle adjustment", Optical 3-D Measurements Techniques II, Wichmann, Karlsruhe, pp , Clarke, T.A., "An analysis of the properties of targets uses in digital close range photogrammetric measurement", Videometrics III, SPIE Vol. 2350, Boston, pp , Clarke, T.A. and Wang, X., "An analysis of subpixel target location accuracy using Fourier Transform based models", To be published in Videometrics IV, SPIE 2598, Philadelphia, El Hakim, S.F., "Real time image metrology with CCD cameras", Photogrammetric Engineering and Remote Sensing, 52(11) : , Fraser, C.S. and Brown, D.C., "Industrial photogrammetry : New developments and recent applications", Photogrammetric Record, 12(68) : , Fraser, C.S. and Shortis, M.R., "Variation of distortion within the photographic field", Photogrammetric Engineering and Remote Sensing, 58(6) : , Fraser, C.S. and Shortis, M.R., "Metric exploitation of still video imagery", The Photogrammetric Record, 15(85) : , Fraser, C.S., Shortis, M.R and Ganci, G., "Multi-sensor system self-calibration", Invited paper to be published in Videometrics IV, SPIE Vol. 2598, Philadelphia, Gruen, A. and Baltsavias, E., "Geometrically constrained multiphoto matching", Photogrammetric Engineering and Remote Sensing, 54(5) : , Gustafson, P.C. and Handley, H.B., "A video-based industrial measurement system", Int. Arch. of Photogrammetry and Remote Sensing, 29(B5) : , Maalen-Johansen, I., "On the precision of subpixel measurements in videometry", Optical 3-D Measurement Techniques II, Wichmann, Karlsruhe, pp , Raynor, J.M. and Seitz, P., "The technology and practical problems of pixel-synchronous CCD data acquisition for optical metrology applications", Int. Arch. of Photogrammetry and Remote Sensing, 28(5) : , Robson, S., Clarke, T.A. and Chen, J., "The suitability of the Pulnix TM6CN CCD camera for photogrammetric measurement", Videometrics II, SPIE Vol. 2067, pp 66-77, Robson, S., Brewer, A., Cooper, M.A.R., Clarke, T.A., Chen, J., Setan, H.B., and Short, T., "Seeing the wood from the trees - An example of optimised digital photogrammetric deformation detection", Int. Arch. of Photogrammetry and Remote Sensing, 30(5W1) : , Shortis, M.R., Burner, A.W., Snow, W.L., and Goad, W.K., "Calibration tests of industrial and scientific CCD cameras", Invited Paper (Paper 6, Volume 1), First Australian Photogrammetric Conference, Sydney, 12 pages, Shortis, M.R. Clarke, T.A., Short, T., "A comparison of some techniques for the subpixel location of discrete target images", Videometrics III, SPIE Vol. 2350, Boston, pp , Shortis, M.R. and Snow, W.L., "Calibration of CCD cameras for field and frame capture modes", In press for Conference on Digital Photogrammetry and Remote Sensing '95, SPIE, St. Petersburg-Great Lakes, Shortis, M. R., Snow, W. L. and Goad, W. K., "Comparative geometric tests of industrial and scientific CCD cameras using plumb line and test range calibrations", Int. Arch. of Photogrammetry and Remote Sensing, 30(5W1) : 53-59, Snow, W.L., Childers, B.A. and Shortis, M.R., "The calibration of video cameras for quantitative measurements", Presented paper, 39th International Instrumentation Symposium, Albuquerque, 28 pages, Stanton, R.H., Alexander, J.W., Dennison, E.W., Glavich, T.A. and Hovland, L.F., "Optical tracking using Charge-Coupled Devices", Optical Engineering, 26(9) : , Trinder, J.C., "Precision of digital target location", Photogrammetric Engineering and Remote Sensing, 55(6) : , van den Heuvel, F.A., Kroon, R.J.G.A. and Le Poole, R.S., "Digital close-range photogrammetry using artificial targets", Int. Arch. of Photogrammetry and Remote Sensing, 29(B5) : , West, G.A.W. and Clarke, T.A., "A survey and examination of subpixel measurement techniques", Int. Arch. of Photogrammetry and Remote Sensing, 28(5) : , Wong, K.W. and Wei-Hsin, H., "Close range mapping with a solid state camera", Photogrammetric Engineering and Remote Sensing, 52(1) : 67-74, Zhou, G., "Accurate determination of ellipse centres in digital imagery", ACSM-ASPRS Annual Convention, Vol. 4, Washington D.C., pp , 1986.