An overview of 3D measurement terms and how they assist in evaluating a 3D system by Raymond Boridy, Teledyne DALSA Introduction: In the world of 3D systems and in many instrumentation devices, performance is often evaluated by reviewing specifications to determine resolution, precision, accuracy, repeatability and reliability. When manufacturers advertise system performance, they will use one or many of these terms. This article will attempt to define these terms and explain their differences so that you can more effectively evaluate 3D technology and incorporate it into your applications Resolution, Precision, Accuracy and Repeatability: Most people evaluate or compare two systems by looking at resolution for first and precision or accuracy for the second system. If, for example resolution and precision are equal in both systems the evaluator will say these systems have the same performance or are comparable. While resolution and precision are related, they have different meanings. The same thing applies to accuracy and precision. However, there are circumstances where numbers used to describe both resolution and precision can result in the same number. Question: Is a device that is precise accurate? Answer: Not necessarily. Question: Is a device (or system) with good resolution precise or accurate? Answer: Not necessarily.
Question: Is a system that is repeatable accurate? Answer: Not necessarily. Question: Is a precise system repeatable? Answer: Yes. A definition of each term and their mathematical relationship follows. Weight Scale as an example A scale will measure weight in pounds and assume that the weight of a person does not change in a span of 10 minutes. The smallest unit read from this particular scale is 1 pounds so we can state that the resolution of this scale is 1 pound. Some scales can give us half pounds or even smaller, but let s stick to this 1 pound balance for ease of discussion. If you move on and off the scale let s say eight times within a few minutes, noting all eight results, and the result is the same, one can safely say this weighing scale is repeatable or precise. Question: But is this scale accurate? Answer: Not necessarily. Let s say the scale returned with the same weight of 176 pounds, 8 times, but when you visited your doctor, he or she recorded your weight as 178 pounds. This means your scale is precise but not accurate since your real weight is 178 pounds (assuming the doctor s scale is accurate). If your scale gives you several different readings around 176 pounds, (between 174 pounds and 178 pounds) then your scale is not precise, its findings are not repeatable and it, most probably, is not accurate. For this discussion, assuming the doctor s scale is precise and accurate and that your real weight was measured as 177 pounds, then we can establish the following things:
The resolution of your scale is 1 pound, the precision of your scale is 178 (highest value read) 174 (lowest value read) = 4 pounds. The precision is therefore 4 pounds. If the most statistically read value (not the middle one) was 175 pounds, then the accuracy is 177 175 = 2 pounds. So in summary: Resolution = 1 pounds Precision ~ repeatability = 4 pounds Accuracy = 2 pounds. We can define these measurement terms as follows: Resolution is the fineness (or smallest) value to which an instrument can read (or detect). Resolution is not a statistical value but a quality factor. Precision is the fineness (or smallest) value to which an instrument can read (or detect) repeatably and reliably. Precision is a statistical value based on several measurements. Precision is often compared, or similar to, the term repeatability. Accuracy is the difference between the most probable statistically read value from the instrument and the real value. Figure 1 below shows how these measurements terms are mathematically related.
DEFINITION OF TERMS IN INSTRUMENTATION 100 % Real value ACCURACY Most probable measured value RESOLUTION PRECISION Calibrated Values measured FIGURE # 1 The example of using a scale is only to facilitate the explanation. The definitions above apply very well to any measurement instrument including 3D systems. There is also another consideration to take into account. How is the resolution number obtained or defined? How resolution is defined is another thing to be careful of when comparing systems. In order to determine the noise amplitude of a signal, some manufacturers will often use 2.36 sigma, others will use 3 sigma and rarely manufacturers will use 6 sigma since 6 sigma relates to perfection and 6 sigma is not an advertising advantage.
When comparing 3D systems, we need to determine how resolution is defined or it becomes difficult to compare. 3 sigma is a safe statistical number. It signifies 99.7 % of the noise span (or peak to peak noise) to determine the smallest value read, the resolution number. In most cases, the smaller the resolution the better. It is usually the quality of the design of the system that will determine the resolution. The quality is largely dependent on optics, electronics, mechanics, software, firmware and calibration. The precision of a system is often determined by the resolution first and the quality of calibration. Calibration is another part of a system that can determine if the system is good or bad. A lot can be discussed about calibration, a subject we won t cover here. Using a 3D laser profiler to define its resolution easily: To qualify a 3D laser profiler as a good system, a good resolution, a good precision and a good accuracy are all required. In addition, customers will look for reliability. Reliability is often defined as a system that has solid hardware, stable software and a precise (repeatable) measurement results. Figure 2 below shows the profile of a part that has a step. The thickness of the profile is a direct indication of the resolution and rapidly gives us its value by measuring the peak to peak noise on the profile.
3 D LASER PROFILER RESOLUTION DETERMINATION Z axis (vertical calibrated values) 3 SIGMA PEAK TO PEAK NOISE = RESOLUTION X axis (Lateral calibrated values) FIGURE # 2 1024 POINTS IN PROFILE Conclusion In 3D measurement systems, users need to be careful about comparing manufacturer specifications. Often, manufacturers will specify the resolution of their system without defining how it was measured. This article provides some baseline advice on how to really determine the resolution of a laser profiler and differentiate the terms resolution, precision, accuracy and repeatability. It becomes difficult to compare all the 3D profilers between them without their definition of resolution and other terms. So, a few things to remember:
The resolution of a 3D system is often related to the performance of the design. Resolution is not a statistical value but more related to noise. The precision is dependent on resolution of course but also calibration. Precision is a statistical value. Accuracy is also a statistical value but also a quality concept. About the Author Raymond Boridy is a product manager at Teledyne DALSA. To learn more about this topic, please visit www.teledynedalsa.com.