Physics 331 Introduction to Numerical Techniques in Physics

Physics 331 Introduction to Numerical Techniques in Physics Instructor: Joaquín Drut Lecture 2

Any logistics questions?

Today: Number representation Sources of error Note: typo in HW! Two parts c. Call them c1 and c2.

Representing numbers on a computer

Numbers Our understanding of nature has led us to a description in terms of real and complex numbers... What makes real numbers so special? What are their defining properties?

Numbers Our understanding of nature has led us to a description in terms of real and complex numbers... What makes real numbers so special? What are their defining properties? Reals form a field (6 axioms about + and. ) Reals are ordered (3 axioms allowing to define > and < ) Reals are complete (every convergent sequence has a real limit)

Numbers Our understanding of nature has led us to a description in terms of real and complex numbers...... but computers use a discrete and finite set of numbers. You can already imagine that the range and precision of numbers we can represent will somehow be limited. An engineering decision had to be made regarding how to manage such limitations. But there is more to consider

Digital computers In digital computers, everything is ultimately reduced to on and off, or 0 and 1. Numbers on such a computer are encoded in base 2: Base 10 Base 2 1 0 0 0 1 2 0 0 1 0 3 0 0 1 1 4 0 1 0 0 5 0 1 0 1 6 0 1 1 0 7 0 1 1 1 bits

Decimal and binary representations The decimal representation of numbers is so common that we often forget why we count the way we do. (Why?)

Decimal and binary representations The decimal representation of numbers is so common that we often forget why we count the way we do. Decimal Take the number 23,488.386 What does this mean in the decimal representation? 23,488.386 2 x 10 4 + 3 x 10 3 + 4 x 10 2 + 8 x 10 1 + 8 x 10 0 + 3 x 10-1 + 8 x 10-2 + 6 x 10-3

Decimal and binary representations Binary We only have 1 and 0, and then we move on to the next digit! You may have something like... 11100101001.0010 What does this mean?

Decimal and binary representations Binary We only have 1 and 0, and then we move on to the next digit! You may have something like... 11100101001.0010 What does this mean? Exercise: What number is this in decimal representation? Take 2 minutes and try to figure it out!

On a computer, numbers are represented in terms of bits, but that s not all there is to it

Floating point representations Decimal floating point representation (a.k.a. scientific notation) d.ddddddd x 10 p d: 0,1,..,9 mantissa think precision exponent think order of magnitude In general To find the first digit d (from the left), and thus the mantissa as well, divide by the highest power of 10 that is less than the number you want to represent. That power is the exponent p.

Floating point representations Binary floating point representation (a.k.a. most of you have never seen this before) 1.bbbbbbb x 2 bbb b: 0,1 mantissa exponent

Floating point representations IEEE standard: f = s M B e E There is also quadruple precision! SP sign exponent mantissa

Floating point representations IEEE standard: f = s M B e E Overflow 0 Underflow Overflow Note: - There is a largest positive number! Overflow - There is a smallest positive number! Underflow - Only a finite number of exact values! - Interval between numbers depends on exponent!

Examples f =+1.2546 10 135 f = s M B e E

Examples f =+1.2546 10 135 f = s M B e E f = 478.043

Examples f =+1.2546 10 135 f = s M B e E f = 478.043 f = 0.000465

Floating point representations IEEE standard: f = s M B e E There is also quadruple precision! SP sign exponent mantissa What range of values can we represent for the exponent with 8 bits?

The exponent e : true exponent f = s M B e E E : bias (127 in SP; 1023 in DP)

Sources of error

Sources of error when programming... Syntax errors (aka compile errors: code does not even compile) Runtime errors (e.g. seg-fault; code compiles but fails when running) Numerical errors (more on this soon!) Physics errors (not a programming error: the code does what you want and it does it right... you just have the physics wrong!) We will put aside discussions of the model/approach used to represent the physics. These could give... Systematic errors (controlled approximations; e.g. finite volume, finite mesh) Uncontrolled errors (the model may involve an approximation )

Sources of numerical error... Round-off errors (computers have finite precision) Round-off errors will propagate in iterative methods and potentially produce instabilities and give useless results.

Statistical error / signal-to-noise problem. Example 15 10 1 cosi loghxl x 0 x M x > 0.162177 5 0.2 0.4 0.6 0.8 1.0-5 -10-15

Sources of numerical error... Round-off errors (computers have finite precision) Round-off errors will propagate in iterative methods and potentially produce instabilities and give useless results. Statistical error (e.g. if your method uses random numbers, like in MC) You just need more statistics - it s almost like a systematic error... but it can be much harder to deal with! Truncation errors (caused by the numerical method) How do you estimate the value of complicated functions, series expansions, integrals, derivatives...? Should be a systematic error, i.e. something we can treat. Why?

Total error Total error = True solution - Numerical solution Total relative error = True solution - Numerical solution True solution Usually we do not know the true solution; that is what we are looking for! The numerical solution can be systematically improved. Ideally the only error left is just the round-off error. We will always be limited by this!