Scientific Programming in C VI. Common errors

Scientific Programming in C VI. Common errors Susi Lehtola 6 November 2012

Beginner errors If you re a beginning C programmer, you might often make off-by one errors when you use arrays: #i n c l u de < s t d l i b. h> i n t main ( void ) { i n t i ; i n t N=1000; double p=m a l l o c (N s i z e o f ( double ) ) ; f o r ( i =1; i<=n; i ++) p [ i ]= i ; f r e e ( p ) ; return 0 ; } Here the error is in the array indexing, which in C goes from 0 to N-1, whereas in Fortran (and Matlab) indexing goes from 1 to N. Valgrind is very useful in finding these kinds of errors. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 2/26

Beginner errors, cont d Actually, in Fortran you can make the indexing be whatever you like: PROGRAM TEST IMPLICIT NONE INTEGER : : I INTEGER, PARAMETER : : N=100 REAL ( 8 ), DIMENSION ( : ), ALLOCATABLE : : x! A l l o c a t e memory ALLOCATE( x( N: 1))! Set a r r a y e l e m e n t s DO i= N, 1 x ( i )= i END DO DEALLOCATE( x ) END PROGRAM In this program the indices of the x array range from -100 to -1. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 3/26

Beginner errors, cont d Of course, you can do the same in C as well just switch to using a pointer that is offset by the number you want: double q=p 1; Now if p ranges from p[0] to p[n-1], then q ranges from q[1] to q[n]. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 4/26

Beginner errors, cont d Of course, you can do the same in C as well just switch to using a pointer that is offset by the number you want: double q=p 1; Now if p ranges from p[0] to p[n-1], then q ranges from q[1] to q[n].... but it would just make things a lot more complicated. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 5/26

Type errors Another very common beginner s error are type errors: you are using an incorrect datatype for the purpose. integer division 1/2 = 0 1.0f/2 = 0.5f floating point division (single) 1/2.0f = 0.5f 1.0f/2.0f = 0.5f 1.0/2 = 0.5 1.0/2.0f = 0.5 floating point division (double) 1/2.0 = 0.5 1.0f/2.0 = 0.5 1.0/2.0 = 0.5 Division of two integers gives an integer here, a zero. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 6/26

Type conversions When there s an int and a float, the int is promoted to a float. When there s an int and a double, or a float and a double, the int (or float) is promoted to a double. You can write the division also as ( double ) 2/1 or 2/( double ) 1 which first casts 2 (or 1) into a double (2.0), and then performs the division. (You can also do both and use (double) 2 / (double) 1). Note that ( double ) (2/1) results in 0.0. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 7/26

Type conversions When you want to compute the floating-point division of two integers m and n, it s often easiest just to write m 1.0/ n which will perform the calculation in double precision. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 8/26

Loop indices The size t datatype is used whenever you need to handle large numbers (e.g. large indices). With the definitions why is const s i z e t N=1000; s i z e t i ; Scientific Programming in C, fall 2012 Susi Lehtola Common errors 9/26

Loop indices The size t datatype is used whenever you need to handle large numbers (e.g. large indices). With the definitions why is OK, const s i z e t N=1000; s i z e t i ; f o r ( i =0; i <N; i ++) a r r [ i ]= i ; Scientific Programming in C, fall 2012 Susi Lehtola Common errors 10/26

Loop indices The size t datatype is used whenever you need to handle large numbers (e.g. large indices). With the definitions why is const s i z e t N=1000; s i z e t i ; f o r ( i =0; i <N; i ++) a r r [ i ]= i ; OK, but f o r ( i=n; i >=0; i ) a r r [ i ]= i ; fails? size t is unsigned, so it can never be negative instead it just rolls over to UINT MAX or ULONG MAX. The breaking condition should read i<n instead. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 12/26

Shadowing declarations In ANSI C you can declare variables in a block-structured fashion within a function. For example i n t main ( void ) { i n t x =10; i n t i ; double r =0.0; f o r ( i =1; i <x ; i ++) { } i n t a=3; i n t x=a i ; r+=x x ; x+=r ; } return 0 ; Scientific Programming in C, fall 2012 Susi Lehtola Common errors 13/26

Shadowing declarations, cont d Here the original variable x is shadowed by the local variable x inside the for loop. The incrementation of x performed in the loop only affects the local variable. Once the incrementation has taken place, the program proceeds to the closing brace, and the local variables a and x vanish from the scope. After this, the program checks if i is less than the value x, which now is the original one. Since shadowing declarations can cause bugs that can be hard to solve, I strongly suggest using the -Wshadow flag in gcc. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 14/26

Truncation errors Since practically everything is done using floating-point arithmetic, the results depend on the order of summation. When you aim for high accuracy, then you need to adjust the summation order. As you remember, the machine epsilon ɛ is defined as the smallest number for which 1 + ɛ > 1 Thus, when you are computing sums such as n=1 1 n 2 = π2 6 1.644934066848226 if you use the simplest ordering, the first term is 1, meaning that the last term that has any significance is with N = ɛ 1/2 67 108 864 (double precision). Scientific Programming in C, fall 2012 Susi Lehtola Common errors 15/26

Truncation errors, cont d What happens if you reverse the summation order? Scientific Programming in C, fall 2012 Susi Lehtola Common errors 16/26

Truncation errors, cont d What happens if you reverse the summation order? In this case, the N+1:th term is unnegligible compared to the partial result of the sum. Thus, the sum can be extended further in n, and a more accurate result can be obtained even if the individual terms are small, their summed up contribution may be large. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 17/26

Truncation errors, cont d Sum up Sum down 100 1.6349839001848923 1000 1.6439345666815615 10000 1.6448340718480652 100000 1.6449240668982423 1000000 1.6449330668487701 10000000 1.6449339668472596 100000000 1.6449340578345750 1000000000 1.6449340578345750 100 1.6349839001848929 1000 1.6439345666815597 10000 1.6448340718480596 100000 1.6449240668982263 1000000 1.6449330668487263 10000000 1.6449339668482315 100000000 1.6449340568482265 1000000000 1.6449340658482263 10000000000 1.6449340667482264 100000000000 1.6449340668382264 Scientific Programming in C, fall 2012 Susi Lehtola Common errors 18/26

Truncation errors, cont d Note that if you use the trivial order of summation you will get a finite value for, i.e., the divergent sum n=1 1 n = since the logarithmic divergence of the upper limit is lost in the precision. The sum will always converge to a finite value, independent of the method you use to sum the elements the value of the sum will just depend on the details of the calculation. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 19/26

Loss of significance If you have alternating sums, you may also end up with loss of significance. For instance, when calculating the sum ( 1) n n n=1 = log 2 0.6931471805599453 you end up cancelling a lot of the significant digits, since you re substracting numbers of similar magnitude and then summing them all up together. Here the proper way is to compute the even and the odd series separately, since they have the opposite signs, and finally add up the two results together. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 20/26

Loss of significance If you have alternating sums, you may also end up with loss of significance. For instance, when calculating the sum ( 1) n n n=1 = log 2 0.6931471805599453 you end up cancelling a lot of the significant digits, since you re substracting numbers of similar magnitude and then summing them all up together. Here the proper way is to compute the even and the odd series separately, since they have the opposite signs, and finally add up the two results together. These examples are somewhat arbitrary, since often one simply doesn t know the form of the function one is integrating, and so ordering the integration correctly might be impossible. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 21/26

Loss of significance, cont d 10 terms Simple summation : 6.4563492063492067 e 01 S e p a r a t e d summation : 6.4563492063492056 e 01 100 terms Simple summation : 6.8817217931019503 e 01 S e p a r a t e d summation : 6.8817217931019536 e 01 1000 terms Simple summation : 6.9264743055982225 e 01 S e p a r a t e d summation : 6.9264743055981359 e 01 10000 terms Simple summation : 6.9309718305995827 e 01 S e p a r a t e d summation : 6.9309718305995371 e 01 100000 terms Simple summation : 6.9314218058498156 e 01 S e p a r a t e d summation : 6.9314218058493715 e 01 1000000 terms Simple summation : 6.9314668056025253 e 01 S e p a r a t e d summation : 6.9314668056035611 e 01 Scientific Programming in C, fall 2012 Susi Lehtola Common errors 22/26

Loss of significance, cont d There are much more drastic examples of loss of precision. For instance, the function f (x) = e x 1 has the limit f (x) 0 when x 0. However, when one computes double f ( double x ) { return exp ( x ) 1.0; } one will run into problems when x is small. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 23/26

Loss of significance, cont d Because e x 1 + x + 1 2 x 2 + 1 6 x 3 +... the function will always evaluate to 0 when x ɛ. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 24/26

Loss of significance, cont d Because e x 1 + x + 1 2 x 2 + 1 6 x 3 +... the function will always evaluate to 0 when x ɛ. The proper way to implement the function is to use the Taylor series around x = 0. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 25/26

Loss of significance, cont d Because e x 1 + x + 1 2 x 2 + 1 6 x 3 +... the function will always evaluate to 0 when x ɛ. The proper way to implement the function is to use the Taylor series around x = 0. In my experience, this is a skill you need quite often. Scientific Programming in C, fall 2012 Susi Lehtola Common errors 26/26