Numerics Motivation Modeling process (JLW) To construct a model we assemble and synthesize data and other information to formulate a conceptual model of the situation. The model is conditioned on the science or policy issue to be addressed, or the engineering problem to be solved, not just the hydrological setting. That is, the model can change with the issue or problem. The conceptual model is then translated into mathematical form, usually as a differential equation, but perhaps as an algebraic or integral equation, or in a statistical or probability framework. Analytical or numerical methods can be applied to solve the model, the choice of method depending on the complexity of the model and how it is to be used. Numerical methods provide greater flexibility and range of application, and these days they offer convenience as well. (Hybrid numerical-analytical approaches are also possible.) Assuming a numerical method is selected, the mathematics are transformed to numerical form, for example descretizing time and space. The numerical form must then be programmed, usually using a scientific/engineering language like Fortran, but more and more these days using a higher level language like Matlab. The numerical model, as programmed, must then be tested to ensure that it is performing as designed (model verification and validation). The program is then applied to the issue or problem at hand to produce a numerical solution. The numerical solution is post-processed to provide and present the desired information, and tested with the data. (At this stage the conceptual model is often revised, in order to do a better job representing reality. Here we assume that the conceptual model is accepted.) That information is then interpreted within the context of the issue or problem. For example, many engineering applications involve the design of something. The model is used to test and revise the design, in order to improve its performance. (JLW) In outline form the process (ignoring feedbacks, such as changing the conceptual model) is something like: 1. Data 2. Conceptualization 3. Mathematical Model 4. Numerical Solution Method(s) 5. Programing, including testing the program 6. Solution 7. Post-processing and interpretation Need for discretization When we adopt a numerical approach to a solution we replace the continuous function f(x) and its independent variable x, with the function f examined at only a few points, x i, where i =1, 2, 3, n is an index representing those points, and n is the number of points. In the continuous space of x, there are an infinity of points x and in the space of funciton f there could be an infinity of values f (if f and x are one-to-one). But computers can t handle an infinite number of values. Instead, they can deal only with finite numbers. So, along x we pick certain points x i that interest us, hopefully close enough to preserve a decide representation of f, and we perform numerical (not symbolic) calculations using those points. 112
Numerics Backgound (Most of this is material in Kreyszig, Section 19.1) Floating-Point Form of Numbers (text, section 19.1) Fixed point numbers = fixed number of decimals after the decimal point. 3D: 3= number of the digits after the decimal point e.g., 62.358, 0.014, 1.000, are all 3D Floating point numbers = number of significant digits are fixed, the decimal point is floating 4S: 4= number of significant digits e.g., 0.6247x10 3, 0.1735x10-13, -0.2000x10-1 or 6.247x10 2, 624.7 Any floating point number can be written as (1) a = ±m 10 n, 0.1 m 1, n integer Machine numbers On a computer, since m is limited to k digits (e.g., k=8) and n is limited, this gives the (machine number) representation as (2) a = ±m 10 n, m = 0.d 1 d 2 d 3 d k, d 1 >0. with a limited range of exponent n, depending on machine word length and whether the program is using machine single precision (e.g., -38 < n < 38) or software double precision. Values of n outside this range lead to underflow (number is too small) or overflow (number is too big and the computation crashes). Roundoff (text, section 19.1) Roundoff error is due to rounding a number to k decimals. Rules for rounding up and down leading to rounding error calculation: Let a = fl(a) in (2), where fl is the floating point approximation of a in (1). Then the roundoff rules give m- m ±½ 10 -k. Since m 0.1, this implies (when a 0) 113
Accuracy in tables Largely, look-up-tables (LUT) of function values, like sin x, have been replaced in computers with series, polynomial or other approximations. Yet LUTs are still used. In hydrology we find that we have a choice of using LUTs of certain functions or the data itself. Examples are equations of state (EOS) relating fluid density to pressure and temperature, and relative permeability relating porous media permeability to fluid saturation. If the typical functions don t fit the data well, the data itself, as a LUT, is often used. If the table shows k significant digits, it is assumed that any value a ~ in the table deviates from the exact value a by, at most ±½ unit in the kth digit. Algorithm (Numerical) Stability (text, section 19.1) Stable: Small changes in initial conditions should only cause small changes in results. Unstable: Small changes lead to significant deviation of results. Consider first theoretical stability, i.e., of the equations themselves. A system may be naturally unstable. Such conditions occur commonly in hydrology and related sciences, e.g., thermal convection cells; turbulence; and viscous, gravity, or dissolution fingering. The mathematics for these problems is ill conditioned. Instead, let s consider only problems which are well conditioned, with stable mathematical solutions. These are more common in hydrology. Even if the mathematics is well conditioned the numerical scheme used to solve it (approximately) may itself be unstable. Numerical instability can be avoided by changing algorithms (wholesale or just increase the order of the algorithm) or, with the same algorithm, increasing discretization resolution (e.g., more points x i ; i.e., make n bigger). Some algorithms are stable only within certain ranges, which must be respected. Errors of Numeric Results (text, section 19.1) Numerical results are approximations, that is, they are not exact but involve errors. - Experimental (data) errors - Roundoff errors - Truncation errors Truncation errors result from truncating approximations (e.g., reducing their order), not truncating digits (that is, instead, related to roundoff). Examples of truncation error are found in series solutions, Taylor series approximation of a derivative, and numerical integration. Error Let a ~ be the approximate value of a quantity whose exact value is a. (4) Error of a ~ : ε = a a~ or 114
(4 ) a = a~ + ε True value of a = Approximation + Error Examples: a ~ =10.5 and a= 10.2, then ε = - 0.3. Caution: there are other errors referred to the literature. Examples are and a ~ a. a a~ (absolute error), Relative error (5)(5 ) ε a a~ Error ε ε r = = = a a Relative Error a~ Error bounds. In practice one gets an error bound, that is a number β such that ε β hence Or for the relative error, a bound β r such that ε β r r hence a a~ a a~ a β r β Basic error principle Always accompany a numerical method with an error estimate, both in testing the method in general, and in every single application. Use a theoretical error estimate and/or error estimation by comparison. Loss of significant digits A calculation often has fewer (and never more) significant digits than the numbers from which it was obtained. Choose your steps carefully to minimize this issue. 115
Solution of equations by iteration (text, section 19.2) Consider the equation (1) f ( x) = 0 which could be an algebraic or transcendental equation, or a differential or integral equation. If (1) is non-linear then we often solve it by iteration. That is, we guess a solution (a value of x) and test it to see if it satisfies (1). If not we guess a new solution and repeat until we are satisfied, that is the solution converges, or we give up, perhaps because the solution diverges. Formal iterative methods use different methods to construct the next guess. Some are better than others, but the better ones are more expensive to use. Suppose a method converges but does so slowly. Another, higher order method may (often will) converge faster. The higher the order the method, the more information it uses, so that higher order methods are more expensive per step. There is a tradeoff between the number of steps and the expense of each step in determining optimal iterative method for a given problem. In general, with a more non-linear the problem you should use a higher order method. We visited one of these methods, Newton s method, during our review of calculus. We ll visit it and two other often used iteration methods below. We ll start with fixed point iteration, a lower order method, before revisiting Newton s method, and its cousin the secant method, which are of higher order. In general, these are likely the only sophisticated methods you will encounter in practice. In these three methods you make an initial guess of x, call it x 0, and use that to linearize (1). You then solve the remaining linear equation for x 2, your new estimate of x. That estimate x 2 is used to relinearize (1) and solve for a third estimate x 3, so on recursively, until you converge or give up. All three methods are used in hydrologic codes, primarily to solve non-linear differential equations. The unsaturated (and non-linear) code HYDRUS (http://www.ars.usda.gov/services/docs.htm?docid=8939), a popular code used in our vadose zone classes, research and application, uses fixed point iteration. The popular groundwater flow code MODFLOW is nonlinear when it deals with phreatic aquifers. It too uses fixed point iteration (http://water.usgs.gov/nrp/gwsoftware/modflow2000/modflow2000.html). The proprietary code MODFLOW- SURFAC (http://www.hglsoftware.com/modflow_surfact.cfm) was originally developed to tackle non-linear problems that MODFLOW can t handle, like mine dewatering and water table recovery. It uses Newton s method. The multiphysics code COMSOL Multiphysics code (see Hyd510 web links page) uses Newton iteration, even for linear problems. They figure that it takes just one iteration to converge for a linear problem, so why bother with all the extra baggage of special routines for those special (special to them, since they mainly attack non-linear problems) linear cases. 116
Fixed Point Iteration (text, section 19.2) Using algebra expend f as f ( x) = x g( x). Then rewrite (1) as (2) x = g(x) The exact solution, s, is where these two curves, for x and g(x), intersect. Then choose an x 0, and compute x 1 =g(x 0 ), x 2 =g(x 1 ), etc., or in general (3) x n+ 1 = g( xn ) n = 0, 1, 2. A solution x of (2) is called a fixed point of g. (1) can be converted to several different versions of (2). 117
Newton s Method (text, section 19.2) 118
Secant Method (text, section 19.2) 119