Accuracy versus precision

Transcription

1 Accuracy versus precision Accuracy is a consistent error from the true value, but not necessarily a good or precise error Precision is a consistent result within a small error, but not necessarily anywhere near a true value

2 Representing Error All science and engineering has error associated with it and this error must be reported when reporting results Error due Normally seen in cutting off a Taylor Series expansion Some definitions of error (in modeling): = ' real world ' relative error: a True error : T =true value approximation true error Relative true error : t = true value current approximation previous approximation current approximation If we iterate until some value we can say a s If s = 0.5 x 10 2 n is met then correct to n significant figures

3 Types of Error in Numerical Methods Truncation Error ( Discretization Error ) Generically this is a quantization error in the model (function) Normally seen in cutting off a Taylor Series expansion Representation Error ( round-off error ) Generically this is a quantization error numerically Round-off or chopping a number due to the representation of a number (doesn't necessarily have to be a computer) Subtraction Error (Cancellation Error) Error due to significant figures (example further in lecture) Normalization Error Error due to significant figures (example further in lecture)

4 Types of Error in Numerical Methods Estimation Error (sampling error) Error due to viewing a sample instead of a whole population Small sample can lead to statistical biasing Systematic Error Error due to a problem with a measurement in a systematic way Types of systematic error Drift Random errors not due to sampling

5 Types of Error due to computers Binary representation of numbers Not all floating point numbers can be represented in binary Because we use base-10 if we used base-2 this would not be a problem (re: Ancient Egypt) Solution is to use a computer that uses base-10 (see later in lecture)

6 Binary Coded Decimal (BCD) A couple of calculators use floating point BCD. This will reduce the small error associated with some binary represented numbers. Error already exists in data from the nature of measurement so this advantage cannot be considered to large an advantage There is considerable waste in this design. Especially costly is the extra adder circuitry, etc. Slower especially on processors not designed with native BCD support Example:

7 Binary Coded Decimal (BCD) Are there any other advantages? Rounding, scaling are easier in the BCD format then the binary code usually used Clearly for display character conversion is easier. For binary it requires expensive circuitry. However due to the expensive of the addition hardware all computers are binary-based (including smart phones which will eventual replace the calculator)

8 Representing Number using Binary To represent number in binary we place represents a power of two This means it is possible to represent ANY integer as long as you have enough digits HOWEVER it is not possible to representing all decimal numbers exactly in a binary-base Only a base with the same factors can express all numbers exactly Binary, octal, and hexidecimal can all represent each other exactly A base-20 system could represent base-10 exactly (however I'm not sure why we would want to do that)

9 Floating Point Representation Decimal numbers are called floating point numbers in computers (these representations are standard but are not guaranteed) Single precision usually uses 32-bits to represent a decimal number 1 sign bit 8 exponent bits 23 significand bits (plus the one not explicitly stored) = 24 Double precision usually uses 64-bits to represent a decimal number 1 sign bit 11 exponent bits 52 significant bits (plus the one not explicity stored) = 53 Quadruple precision usually uses 128-bits to represent a decimal number (real*16 in Fortran) 1 sign bit 15 exponent bits 112 significand bits (plus the one not explicitly stored) = 113

10 Floating Point Numbers Many different floating point representations Most common is 32 bit (single precision) Mantissa is the same word for decimal portion of the fraction, so should avoid Exponent does not have a sign but uses excess-127 Equation is fp= 1 sb 1.f 2 excess 127 Where sb is the sign bit; f is the significand ; excess is what the 8 bit exponent is (0 would mean -127) When we make it the same exponent that is called normalization Apparently IEEE has a rule to sacrifice normalization for precision...there has to be a cost for this.

11 Floating Point Numbers (32 bit) Many different floating point representations In the 32 bit example the excess of the exponent is 255 therefore the maximum exponent that is possible is or Best significant we can have is Sign Exponent Significant sb excess f 0,

12 Floating Point Numbers Many different floating point representations For 8 bit example we would have 4-bits significant, 3-bit exponent and one sign bit In the 8 bit example the excess of the exponent is therefore the maximum exponent that is possible is or 2 4 or 16 Best significant we can have is ,0.8750,0.8125,0.7500,0.6875,0.6250,0.5625,0.5000, ,0.3750,0.3125,0.2500,0.1875,0.1250, How do we represent 1/3?, 2.7?, or ? Sign Exponent Significant sb excess f 0,

13 Floating Point Numbers The machine error is referred to as the machine epsilon and it is the difference between 1 and the next representable number greater than 1 In MATLAB and octave you can figure out this value by use the eps command octave:1> eps eps = e-16 machine =b 1 p If b is the base of the machine and p is the number of significant digits then the machine epsilon is

14 Two's Complement Method for representing negative by inverting the positive binary representation for integers; not for floating point (however given the standard representation of exponent that might be viewed as two's complement) This amounts to flipping the bits and adding a binary 1 Example

15 Normalization error Exponents are matched or normalized for addition and subtraction Leads to subtraction or normalization error Can be significant x x 10-3 = x x 10 1 Only a certain amount of significant digits are saved in any computational system (you should save more if doing by hand)...since the maximum significant digits in the above example is 4 we have x 10 1 as the answer. This leads to an error which is small, but if repeated millions of times will propagate

16 Normalization error Exponents are matched or normalized for addition and subtraction Leads to subtraction or normalization error Can be significant x x 10 3 = x 10 3 However we would need to convert back to x 10 0 which would loss us 3 significant figures... Multiplication and division don't have a problem here...but...

17 Normalization error Fixing the quadratic equation In certain instances you can change the way you do an operation to remove lessen this error An example is the quadratic equation There are two solutions, but one will be effected by subtraction error Produce a new equation to get fix the problem Will need to use both equations one for each of the two solutions x= b± b 2 4ac 2a If b 2 4ac then there will be a subtraction error for the plus portion of the equation Solution : Multiply by b b 2 4ac 2c Therefore: x= b b 2 4ac

18 Round-off error The interval between two numbers increases as the number grows in magnitude Chopping (easier to implement on computers) and rounding leads to different errors Chopping is when the digits after are chopped-off when representing the significant figures available Rounding is when the last digit in the significant figure moved up or left alone depending on the digits to be removed If is to be approximated by only two significant figures then it is 0.33 if rounded or chopped but for the number it is 0.34 if rounded and 0.33 if chopped Bounding of the relative error is as follows: =b 1 t where t is the number of significant digits x for chopping x x x for rounding 2

19 Fundamental Theorems of Calculus If f(x) is continuous on the closed interval [a,b] and if F is the anti-derivative we have b a f t dt=f b F a If f(x) is continuous on the open interval (a,b) then at any point θ, F is x F x = f t dt F ' x = f x

20 Mean-Value Theorem If f(x) is differentiable on the open interval (a,b) and continuous on the closed interval [a,b], then there must exist at least one point, say θ in (a,b) such that Then we can say f ' = f b f a b a F b F a =F ' b a b a b a f t dt= f b a where is anaveragevalue of f t on[a,b] f (t) g (t)dt= f (θ) a b g (t)dt where f (θ)is a g (t ) weighted average of f (t )on[a,b]

21 Taylor Series Any real function f(x) can be expanded at a point x=a by the Taylor series f x = f a f ' a x a f ' ' a 2! x a 2 f ' ' ' a 3! x a 3 f n a n! R n is a remainder known as the Lagrange Remainder (another form exists called the Cauchy Remainder) x a n R n x R n = a f n 1 t x t n n! dt Using mean value theorem previously stated we have R n = f n 1 x a n 1 n 1!

22 Taylor Series We could assume that the a is our present position and x is the future position to re-write the Taylor Series as f t i 1 = f t i f ' t i s f ' ' t i 2! s 2 f ' ' ' t i 3! s 3 f n t i n! s n R n where the step, s= t i 1 t i R n is now R n = f n 1 s n 1 n 1!

23 Taylor Series Truncation Error Using the Taylor Series to estimate our function we can also derive an estimate of our error (truncation error) f t i 1 = f t i f ' t i s R where s= t i 1 t i Therefore we have our well known estimate WITH an error f ' t i = f t i 1 f t i t i 1 t i R t i 1 t i So from the Taylor Series we have an estimate of error, which states that in this case our error is proportional to our step size R s = f ' ' 2! s= s

24 Taylor Series Truncation Error For an n th order polynomial an exact n th order Taylor series can be used so an approximation is not necessary But for sine, cosine, exponential, etc. only an infinite series can represent the function exactly...need to have an approximation Drawbacks to the error estimate Our theta (or zeta) lies somewhere between two points We need the n th derivative of the function in question, but since we don't know the function exactly how can we know the any derivatives...and if we did know the function what would be the point!

25 Error Analysis If we use the above estimates, each variable in a system could have a number of sources of error which would add An assumption of the error being uncorrelated is assumed Technically the total error is more like an upper bound total = T N R E b misc For one variable this is likely to be quite low, but due to successive operations in any calculation errors can increase rapidly f x, y, z = x y z f x, y, z = x total x y total y z total z f x, y, z =x yz ERROR f = total x y z z y y z That is the errors propagate (which we will discuss soon) This error gives a rather conservative error bound to any calculation

26 Forward Error Analysis Our previous error analysis with trunction or approximation of a function is a form of forward error analysis Forward error analysis usually is a conservative error that will be much larger than the true error Each error needs to be estimate which can be inaccurate Trying to determine each of the errors that might effect the result can be complex for very intricate calculations We can use Backward Error Analysis to try and get a better estimate Take the result and work backward to figure out the error of the initial data Works only for certain errors like round-off error Will need to combine different error techniques

27 Backward Error Analysis Requires backward stability which can be determined by sending a perturbation through the function to see if the result is stable If the system is stable that this analysis can be done The condition number can give a clue on how sensitive a system is and hence how stable it might be in a given region

28 Backward Error Analysis Example Round-off error for a d digit number will be ϵ round = d However this is worse case scenario with the error likely being less In a calculation differing directions are likely to lessen error as well