3.3 Function minimization

Transcription

1 3.3. Function minimization Function minimization Beneath the problem of root-finding, minimizing functions constitutes a major problem in computational economics. Let f () : X R a function that maps an n-dimensional vector X R n into the real line R. Then the corresponding minimization problem is defined as min f (). X Note that minimization and maimization do not have to be considered separately, as minimizing the function f () is eactly equal to its maimization. Obviously, minimization is very closely related to root-finding, as the derivative of a function becomes zero in a minimum. Consequently, if derivatives can be easily calculated, root-finding is a fair alternative to using a minimization method. However there are cases where using a minimization method is superior, e.g. if derivatives can t be calculated in a reasonable amount of time. In addition, non-differentiability of f eliminates the option of choosing a root-finding instead of a minimization algorithm. Last but not least, minimization methods often give us the possibility of constraining the set of possible values for. This is obviously not possible with root-finding algorithms. Figure 3.4 addresses this issue. Looking for the unconstraint minimum of f, we can see that we could use either a root f () * a b Figure 3.4: Constraint minimization finding or a minimization algorithm. However, if we would like to find the minimum of f only on the interval [a, b], root-finding will not help us, as f is positive on the whole interval. A constraint minimization method could however give us the actual solution a to this problem.

2 56 Chapter 3 Numerical Solution Methods Eample A household can consume two goods 1 and 2. He values the consumption of those goods with the joint utility function u( 1, 2 )= (1 + 2 ) 0.5. Here 2 acts as a luury good, i.e. the household will only consume 2, if his available resources W are large enough. 1 on the other hand is a normal good and will always be consumed. Naturally, we have to assume that 1, 2 0. With the prices for the goods being p 1 and p 2, the households has to solve the optimization problem ma 1, (1 + 2 ) 0.5 s.t. p p 2 2 = W. Note that there is no analytical solution to this problem. Due to marginal utility increasing to with 1 approaching zero, the optimal 1 will always be strictly larger than zero. However, we might result in a corner solution 2 = 0. As the set of allowed values for 2 is constraint, we will not be able to use a root-finding method to solve for the optimal choice of 1 and 2. Hence, we need a minimization routine. When using a minimization algorithm with an equality constraint, it is always useful to first plug in the constraint into the utility function and therefore reduce the dimension of the optimization problem. We therefore reformulate the above problem to ma 2 0 [ ] W p (1 + 2 ) 0.5. This problem can be solved with various minimization routines. p The Golden-Search method The Golden-Search method minimizes a one-dimensional function on the initially defined interval [a, b]. The ideas behind this method is quite similar to the one of bisection search. Golden-Search however divides the interval [a i, b i ] into two sub-intervals by using the points i,1 and i,2 with i,j = a i + α j (b i a i ) with α 1 = and α 2 = 1 α 1 = 5 1. (3.4) 2 The values α 1 and α 2 are chosen in a way, such that the interval [a i, b i ] is intersected according to the golden ratio. 13 We now compute the function values f ( i,j ) for j = 1, 2 and compare them. The net iteration s interval is then chosen according to a i+1 = a i and b i+1 = i,2 if f ( i,1 ) < f ( i,2 ), a i+1 = i,1 and b i+1 = b i otherwise.

3 3.3. Function minimization 57 a= a i,1 * = i i+1 i,2 b i+1 b i Figure 3.5: Golden search method for finding minima The idea behind this iteration rule is quite simple. If f ( i,1 ) < f ( i,2 ), the lower values of f will be located near i,1,not i,2. Consequently, one chooses the interval [a i, i,2 ] as new iteration interval and therefore rules out the greater values of f, see Figure 3.5. Program 3.9shows how to apply the Golden-Search method to the above problem. At the beginning of the program we choose values for the model parameter p 1, p 2 and W. Weassume the price of the luury good to be twice the price of the normal good and normalize the available resources W = 1. We then have to set the starting interval. Note that setting a = 0 restricts 2 to be non-negative. b is finally initialized in a way that guarantees the consumption of good 1 to be positive for any 2 [a, b]. In the iteration process we calculate i,1 and i,2 and the respective function values as shown in (3.4). Note that we always use the negative of the actual function value in order to assure that we maimize household s utility. In opposite to the previous section, we define our tolerance level as b i a i. This is because we now have two values i,1 and i,2 in every iteration and don t know which one to choose as approimation to our minimum. Consequently, taking the interval width of [a i, b i ] as tolerance criterion insures that we can take any value within this interval and meet our tolerance criterion. If our criterion is satisfied, we print the minimum and the respective function value on the console. If not, we set the new iteration s interval [a i+1, b i+1 ] according to the above iteration rule. Taking a look at the output of the program, we find that 1 = 1and 2 = 0istheoptimal solution to our optimization problem. This indicates that the constraint 2 > 0 is actually binding for this price and resource combination. We can test this by etending the initial interval [a, b] and setting a = -0.9d0 instead of 0. Doing this, Golden-Search return the unconstraint optimum and of the optimization problem in our eample. Note that, similar as in the bisection search method, convergence speed of Golden-Search is linear. 13 Note that α 2 eactly is the inverse of the golden ratio (1 + 5)/ 2.

4 58 Chapter 3 Numerical Solution Methods Program 3.9: Golden-Search method in one dimension program golden! variable declaration implicit none real*8, parameter :: p(2) = (/1d0, 2d0/) real*8, parameter :: W = 1d0 real*8 :: a, b, 1, 2, f1, f2 integer :: iter! initial interval and function values a = 0d0 b = (W-p(1)*0.01d0)/p(2)! start iteration process do iter = 1, 200! calculate 1 and 2 and function values 1 = a+(3d0-sqrt(5d0))/2d0*(b-a) 2 = a+(sqrt(5d0)-1d0)/2d0*(b-a) f1 = -(((W-p(2)*1)/p(1))**0.4d0+(1d0+1)**0.5d0) f2 = -(((W-p(2)*2)/p(1))**0.4d0+(1d0+2)**0.5d0) write(*, (i4,f12.7) )iter, abs(b-a)! check for convergence if(abs(b-a) < 1d-6)then write(*, (/a,f12.7) ) _1 =,(W-p(2)*1)/p(1) write(*, (a,f12.7) ) _2 =,1 stop endif write(*, (a,f12.7) ) u! get new values if(f1 < f2)then b = 2 else a = 1 endif enddo end program =,-f Brent s and Powell s algorithms The problem with Golden-Search is it s slow convergence. Therefore the function f is called quite often. For well-behaved functions a more sophisticated algorithm is based on Brent s method. It relies on parabolic approimations of the actual function f. Finding

5 3.3. Function minimization 59 the minimum of a parabola is quite easy. If the parabola is given by a + b + c 2,thenits minimum is located at = b 2c. Figure 3.6 shows how Brent s method proceeds in finding a minimum. We start with a * 2 1 b Figure 3.6: Brent smethodfor finding minima the initial interval [a, b] and compute the intersection point 1 = a+b 2.Wethencompute a parabola that eactly contains the three points (a, f (a)), (b, f (b)) and ( 1, f ( 1 )). The minimum of this parabola can be calculated and is denoted with 2. We then replace b with 2 and again compute a parabola through our new points. The method is repeated until we reach convergence. Note that in every iteration step, we now have to calculate only one function value, namely the value at the new point. For applying Golden-Search, we needed two function values in every iteration step. There also is a multidimensional etension of Brent s method called Powell s algorithm. This algorithm minimizes a multidimensional function by taking a set of n-dimensional, linearly independent direction vectors and minimizing f along the lines spanned by the n different vectors. Figure 3.7 demonstrates how Powell s algorithm works. We start at an initial guess 1. We then take our first optimization direction, represented by the arrow leaving 1 and minimize the function along this direction. Given this minimum, we take the net direction and minimize along this one. We then again step back to the first direction etc. We iterate until we finally reach convergence. For the minimization along the different directions, we can again use Brent s method. Both Brent s and Powell s method are included in the module minimization. It contains a subroutine fminsearch that is called with five arguments. Program 3.10 demonstrates its use taking the above eample. The program also incorporates a module globals that stores the parameters p 1, p 2 and W of the model. We also have to define a function that

6 60 Chapter 3 Numerical Solution Methods 1 Figure 3.7: Powell s algorithm for finding minima in multi dimensions we want fminsearch to minimize. The function utility eactly matches the declared interface and returns the negative of the utility function u( 1, 2 ). Note that, if we wanted to define a multidimensional optimization problem, we would have declared the input variable to the function as an assumed-size vector by using (:). We now set the optimization interval borders a and b and make an initial guess. fminsearch is then called with the initial guess, a scalar f in which the function value in the minimum is stored, the interval borders and the function to minimize. The solution of the minimization problem is returned in. If we had a multi-dimensional problem, a and b should be vectors ofthesamesizeas. The result if finally printed on the console The problem of local and global minima Unfortunately, there is no guarantee that a minimization method will find the global minimum of a function. It might well be that one ends in a local minimum. Figure 3.8 shows this problem with Golden-Search. We can see that f has a local minimum on the left and the global minimum on the right side of the optimization interval [a, b]. When we perform the first step of Golden-Search, we calculate the points 1 and 2 and the respective function values. For a net iteration s optimization interval, we would then choose [a, 1 ],asf ( 1 ) < f ( 2 ). Obviously, with this step, we have already eliminated the chance of finding the global minimum, as after the first step we concentrate on the left hand side of the interval [a, b]. Aneasilyimplementableapproach to overcome thisproblem istofirst divide the original interval [a, b] into n sub-intervals [ i, i+1 ] of equal size with 1 = a and n+1 = b. On every of these sub-intervals, one then performs a full minimization procedure to find the

7 3.3. Function minimization 61 Program 3.10: Brent and Powell for finding minima program brentmin! module use use globals use minimization! variable declaration and interface implicit none real*8 ::, f, a, b interface function utility() real*8, intent(in) :: real*8 :: utility end function end interface! initial interval and function values a = 0d0 b = (W-p(1)*0.01d0)/p(2)! set starting point = (a+b)/2d0! call minimizing routine call fminsearch(, f, a, b, utility)! output write(*, (/a,f12.7) ) _1 =,(W-p(2)*)/p(1) write(*, (a,f12.7) ) _2 =, write(*, (a,f12.7) ) u =,-f end program minimum i of f on the interval [ i, i+1 ]. Now we have a set of minima {i }n i=1.our global minimum finally is the value i with the smallest function value f (i ). Figure 3.9 demonstrates the approach. We used a division into five intervals in this case. The green dots mark the local minima on the sub-intervals. We can see that the global minimum is also contained in this set. Hence, with this procedure, we are able to find the global minimum of a function. Note, however, that the procedure is quite costly, as we have to perform 5 minimizations instead of only one. Consequently, it should only be used when one is sure that the problem of local optima may arise.

8 62 Chapter 3 Numerical Solution Methods a 1 2 * b Figure 3.8: The problem of local minima * 3 * 1 * 2 * 4 * 5 a * b Figure 3.9: Sub-interval division for solving the problem of local minima