6 Gradient Descent. 6.1 Functions

Transcription

1 6 Graient Descent In this topic we will iscuss optimizing over general functions f. Typically the function is efine f : R! R; that is its omain is multi-imensional (in this case -imensional) an output is a real scalar (R). This often arises to escribe the cost of a moel which has parameters which escribe the moel (e.g., egree ( 1)-polynomial regression) an the goal is to fin the parameters with minimum cost. Although there are special cases where we can solve for these optimal parameters exactly, there are many cases where we cannot. What remains in these cases is to analyze the function f, an try to fin its minimum point. The most common solution for this is graient escent where we try to walk in a irection so the function ecreases until we no-longer can. 6.1 Functions We review some basic properties of a function f : R! R. Again, the goal will be to unveil abstract tools that are often easy to imagine in low imensions, but automatically generalize to high-imensional ata. We will first provie efinitions without any calculous. Let B r (x) efine a Eucliean ball aroun a point x of raius r. That is, it inclues all points {y 2 R kx yk appler}, within a Eucliean istance of r from x. We will use B r (x) to efine a local neighborhoo aroun a point x. The iea of local is quite flexible, an we can use any value of r>0, basically it can be as small as we nee it to be, as long as it is strictly greater than 0. Minima an maxima. A local maximum of f is a point x 2 R so for some neighborhoo B r (x), all points y 2 B r (x) have smaller (or equal) function value than at x: f(y) apple f(x). Alocal maximum of f is a point x 2 R so for some neighborhoo B r (x), all points y 2 B r (x) have larger (or equal) function value than at x: f(y) f(x). If we remove the or equal conition for both efinitions for y 2 B r (x),y 6= x, we say the maximum or minimum points are strict. A point x 2 R is a global maximum of f if for all y 2 R, then f(y) apple f(x). Likewise, a point x 2 R is a global minimum if for all y 2 R, then f(y) f(x). There may be multiple global minimum an maximum. If there is exactly one point x 2 R that is a global minimum or global maximum, we again say it is strict. When we just use the term minimum or maximum (without local or global) it implies a local minimum or maximum. Focusing on a function restricte to a close an boune subset S R, if the function is continuous (we won t formally efine this, but it likely means what you think it oes), then the function must have a global minimum an a global maximum. It may occur on the bounary of S. A sale point is a type of point x 2 R so that within any neighborhoo B r (x), it has points y 2 B r (x) with f(y) <f(x) (the lower points) an y 0 2 B r (x) with f(y 0 ) >f(x) (the upper points). In particular, it is a sale if there are isconnecte regions of upper points (an of lower points). The notion of sale point is efine ifferently for =1. If these regions are connecte, an it is not a minimum or maximum, then it is a regular point. For an arbitrary (or ranomly) chosen point x, it is usually a regular point (except for examples you are unlikely to encounter, the set of minimum, maximum, an sale points are finite, while the set of regular points is infinite). 41

2 Example: Continuous Functions Here we show some example functions, where the x-axis represents a -imensional space. The first function has local minimum an maximum. The secon function has a constant value, so every point is a global minimum an a global maximum. The thir function f is convex, which is emonstrate with the line segment between points (p, f (p)) an (q, f (q)) is always above the function f. R R global maximum R 1 3 (p, f (p)) local maximum local minimum global minimum global minimum x 2 R + 23 (q, f (q)) x 2 R q x 2 R p In many cases we will assume (or at least esire) that our function is convex. To efine this it will be useful to efine a line ` R as follows with any two points p, q 2 R. Then for any scalar 2 R, a line ` is the set of points Convex functions. ` = {x = p + (1 )q 2 R an p, q 2 R }. When 2 [0, 1], then this efines the line segment between p an q. A function is convex if for any two points p, q 2 R, on the line segment between them has value less than (or equal) to the values at the weighte average of p an q. That is, it is convex if For all p, q 2 R an for all 2 [0, 1] f ( p + (1 )q) f (p) + (1 )f (q). Removing the (or equal) conition, the function becomes strictly convex. There are many very cool properties of convex functions. For instance, for two convex functions f an g, then h(x) = f (x) + g(x) is convex an so is h(x) = max{f (x), g(x)}. But one will be most important for us: Any local minimum of a convex function will also be a global minimum. A strictly convex function will have at most a single minimum: the global minimum. This means if we fin a minimum, then we must have also foun a global minimum (our goal). 6.2 Graients For a function f (x) = f (x1, x2,..., x ), an a unit vector u = (u1, u2,..., u ), then the irectional erivative is efine f (x + hu) f (x) ru f (x) = lim. h!0 h We are intereste in functions f which are ifferentiable; this implies that ru f (x) is well-efine for all x an u. The converse is not necessarily true. Let e1, e2,..., e 2 R be a specific set of unit vectors so that ei = (0, 0,..., 0, 1, 0,..., 0) where for ei the 1 is in the ith coorinate. Then efine ri f (x) = rei f (x) = f (x). xi

3 It is the erivative in the ith coorinate, treating all other coorinates as constants. We can now, for a ifferentiable function f, efine the graient of f as rf = f e 1 + f e f f e =, f,..., f. x 1 x 2 x x 1 x 2 x Note that rf is a function from R! R, which we can evaluate at any point x 2 R. Example: Graient Consier the function f(x, y, z) =3x 2 2y 3 2xe z. Then rf =(6x 2e z, 6y 2, 2xe z ) an rf(3, 2, 1) = (18 2e, 24, 6e). Linear approximation. From the graient we can easily recover the irectional erivative of f at point x, for any irection (unit vector) u as r u f(x) =hrf(x),ui. This implies the graient escribes the linear approximation of f at a point x. The slope of the tangent plane of f at x in any irection u is provie by r u f(x). Hence, the irection which f is increasing the most at a point x is the unit vector u where r u f(x) = hrf(x),ui is the largest. This occurs at rf(x) =rf(x)/krf(x)k, the normalize graient vector. To fin the minimum of a function f, we then typically want to move from any point x in the irection rf(x); this is the irection of steepest escent. 6.3 Graient Descent Graient escent is a family of techniques that, for a ifferentiable function f : R! R, try to ientify either min f(x) an/or x = arg min f(x). x2r x2r This is effective when f is convex an we o not have a close form solution x. The algorithm is iterative, in that it may never reach the completely optimal x, but it keeps getting closer an closer. Algorithm Graient Descent(f,x start ) initialize x (0) = x start 2 R. repeat x (k+1) := x (k) krf(x (k) ) until (krf(x (k) )kapple ) return x (k) Basically, for any starting point x (0) the algorithm moves to another point in the irection opposite to the graient in the irection that locally ecreases f the fastest. Stopping conition. The parameter is the tolerance of the algorithm. If we assume the function is ifferentiable, then at the minimum x, we must have that rf(x) =(0, 0,...,0). So close to the minimum, it shoul also have a small norm. The algorithm may never reach the true minimum (an we on t know what it is, so we cannot irectly compare against the function value). So we use krfk as a proxy. In other settings, we may run for a fixe number T steps.

4 6.3.1 Learning Rate The most critical parameter of graient escent is, the learning rate. In many cases the algorithm will keep k = fixe for all k. It controls how fast the algorithm works. But if it is too large, when we approach the minimum, then the algorithm may go too far, an overshoot it. How shoul we choose? Lipschitz boun. We say a function g : R! R k is L-Lipschitz if for all x, y 2 R that kg(x) g(y)k applelkx yk. If rf is L-Lipschitz, an we set apple 1 L, then graient escent will converge to a stationary point. Moreover, if f is convex with global minimum x, then after k = O(1/") steps we can guarantee that Strongly Convex Functions. x, y 2 R then f(x (k) ) f(x ) apple ". A function f : R! R is -strongly convex with parameter >0 if for all f(y) apple f(x)+hrf(x),y xi + 2 ky xk2. Intuitively, this implies that f is at least quaratic. That is, along any irection u = y x ky xk the function hf,ui is 1-imensional, an then its secon erivative 2 hf,ui is strictly positive; it is at least >0. Similarly, u 2 saying a function f has an L-Lipschitz graient is equivalent to the conition that 2 hf(x),uiapplelfor all u 2 x, y 2 R where u = y x ky xk. For an -strongly convex function f, that has an L-Lipschitz graient, with global minimum x, then graient escent with learning rate apple 2/( + L) after only k = O(log(1/")) steps will achieve f(x (k) ) f(x ) apple ". The constant in k = O(log(1/")) epens on the conition number L/. The conitions of this boun, imply that f is sanwiche between two quaratic functions; specifically for any x, y 2 R that we can boun f(x)+hrf(x),y xi + L 2 ky xk2 apple f(y) apple f(x)+hrf(x),y xi + 2 ky xk2. When an algorithm converges at such a rate (takes O(log(1/")) steps to obtain " error), it is known as linear convergence since the log-error log(f(x (k) ) f(x )) looks like a linear function of k. Line search. In other cases, we can search for k at each step. Once we have compute the graient rf(x (k) ) then we have reuce the high-imensional minimization problem to a one-imensional problem. Note if f is convex, then f restricte to this one-imensional search is also convex. We still nee to fin the minimum of an unknown function, but we can perform some proceure akin to binary search. We first fin a value 0 such that f x (k) 0 rf(x (k) ) >f(x (k) ) then we keep subiviing the region [0, 0 ] into pieces which must contain the minimum (for instance the golen section search keeps iviing by the golen ratio). In other situations, we can solve for the optimal k exactly at each step. This is the case if we can again analytically take the erivative f(x (k) ) rf(x (k) ) an solve for the where it is equal to 0.

5 Ajustable rate. In practice, line search is often slow. Also, we may not have a Lipschitz boun. It is often better to try a few fixe values, probably being a bit conservative. As long as f(x (k) ) keep ecreasing, it works well. This also may alert us if there there is more than one local minimum if the algorithm converges to ifferent locations. An algorithm calle backtracking line search automatically tunes the parameter. It uses a fixe parameter 2 (0, 1) (preferably in (0.1, 0.8); for instance use =3/4). Start with a large step size (e.g., =1). Then at each step of graient escent at location x, if f(x rf(x)) >f(x) 2 krf(x)k2 then upate =. This shrinks over the course of the algorithm, an if f is strongly convex, it will eventually ecrease until it satisfies the conition for linear convergence. Example: Graient Descent with Fixe Learning Rate Consier the function with graient f(x, y) =( 3 4 x 3 2 )2 +(y 2) xy rf(x, y) = 9 8 x y, 2y x We run graient escent for 10 iterations within initial position (5, 4), while varying the learning rate in the range = {0.01, 0.1, 0.2, 0.3, 0.5, 0.75}. We see that with very small, the algorithm oes not get close to the minimum. When is too large, then the algorithm jumps aroun a lot, an is in anger of not converging. But at a learning rate of = 0.5 it converges fairly smoothly an reaches a point where krf(x, y)k is very small. Using =0.5 almost overshoots in the first step; =0.3 is smoother, an its probably best to use a curve that looks smooth like that one, but with a few more iterations.

6 import matplotlib as mpl mpl.use( PDF ) import numpy as np import matplotlib.pyplot as plt from numpy import linalg as LA ef func(x,y): return (0.75*x-1.5)**2 + (y-2.0)** *x*y ef func_gra(vx,vy): fx = 1.125*vx *vy fy = 2.0*vy *vx return np.array([fx,fy]) #prepare for contour plot xlist = np.linspace(0, 5, 26) ylist = np.linspace(0, 5, 26) x, y = np.meshgri(xlist, ylist) z = func(x,y) lev = np.linspace(0,20,21) #iterate location v_init = np.array([5,4]) num_iter = 10 values = np.zeros([num_iter,2]) for gamma in [0.01, 0.1, 0.2, 0.3, 0.5, 0.75]: values[0,:] = v_init v = v_init # actual graient escent algorithm for i in range(1,num_iter): v = v - gamma * func_gra(v[0],v[1]) values[i,:] = v #plotting plt.contour(x,y,z,levels=lev) plt.plot(values[:,0],values[:,1], r- ) plt.plot(values[:,0],values[:,1], bo ) gra_norm = LA.norm(func_gra(v[0],v[1])) title = "gamma %0.2f final gra %0.3f" % (gamma,gra_norm) plt.title(title) file = "g-%2.0f.pf" % (gamma*100) plt.savefig(file, bbox_inches= tight ) plt.clf() plt.cla() 6.4 Fitting a Moel to Data For ata analysis, the most common use of graient escent is to fit a moel to ata. In this setting we have a ata set P = {p 1,p 2,...,p n } an a family of moels M so each possible moel M is efine by a -imensional vector = { 1, 2,..., } for p parameters. Next we efine a loss function L(P, M ) which measures the ifference between what the moel preicts an what the ata values are. To choose which parameters generate the best moel, we let f( ) :R! R be our function of interest, efine f( ) =L(P, M ). Then we can run graient escent to fin our moel

7 M. For instance we can set f( ) =L(P, M )=SSE(P, M )= X p2p(p y M (p x )) 2. (6.1) This is use for examples incluing maximum likelihoo (or maximum log-likelihoo) estimators from Bayesian inference. This inclues fining a single point estimator with Gaussian (where we ha a closeform solution), but also many other variants (often where there is no known close-form solution). It also inclues least squares regression an its many variants; we will see this in much more etail next. An will inclue other topics (incluing clustering, PCA, classification) we will see later in class Least Mean Squares Upates for Linear Regression Now we will work through how to use graient escent for simple quaratic regression. It shoul be straightforwar to generalize to linear regression, multiple-explanatory variable linear regression, or general polynomial regression from here. This will specify the function f( ) in equation (6.1) to where = 3, = ( 0, 1, 2 ), for each p 2 P we have p = (p x,p y ) 2 R 2, but we consier a vector q =(q 0 = p 0 x,q 1 = p 1 x,q 2 = p 2 x) as the set of explanatory variables. Finally, To specify the graient escent step: M (p x )= p x + 2 p 2 x = 0 q q q 2. := rf( ) we nee to efine rf( ). We will first show this for the case where n =1, that is when there is a single ata point p =(p x,p y )= (x 1,y 1 ). It shoul now be easy to verify that the cost function f 1 ( ) =( x x 2 1 y 1 ) 2 is convex. Next, erive j f( ) = j (M (x 1 ) y 1 ) 2 = 2(M (x 1 ) y 1 ) j (M (x 1 ) y 1 ) = 2(M (x 1 ) y 1 ) j ( = 2(M (x 1 ) y 1 )x j 1 2X j x j 1 y 1 ) j=0 Thus, we efine rf( ) = 0 f( ), 1 f( ), f( ) 2 =2 (M (x 1 ) y 1 ), (M (x 1 ) y 1 )x 1, (M (x 1 ) y 1 )x 2 1 Applying := rf( ) accoring to this specification is known as the LMS (least mean squares) upate rule or the Wirow-Huff learning rule. Quite intuitively, the magnitue the upate is proportional to the resiual norm (M (x 1 ) y 1 ). So if we have a lot of error in our guess of, then we take a large step; if we o not have a lot of error, we take a small step. To generalize this to multiple ata points (n >1), there are two stanar ways. Both of these take strong avantage of the cost function f( ) being ecomposable. That is, we can write nx f( ) = f i ( ), i=1

8 where each f i epens only on the ith ata point p i 2 P. In particular, where p i =(x i,y i ), then f i ( ) =(M (x i ) y i ) 2 =( x i + 2 x 2 i y i ) 2. First notice that since f is the sum of f i s, where each is convex, then f must also be convex; in fact the sum of these usually becomes strongly convex. Also two approaches towars graient escent will take avantage of this ecomposition in slightly ifferent ways. This ecomposable property hols for most loss functions for fitting a moel to ata. Batch graient escent. The first technique, calle batch graient escent, simply extens the efinition of rf( ) to the case with multiple ata points. Since f is ecomposable, then we use the linearity of the erivative to efine nx f( ) = 2(M (x i ) y i )x j i j i=1 an thus rf( ) = nx i=1 2(M (x i ) y i ), 2(M (x i ) y i )x i, 2(M (x i ) y i )x 2 i. That is, the step is now just the sum of the terms from each ata point. Since f is convex, then we can apply all of the nice convergence results iscusse about (strongly) convex f before. However, computing rf( ) each step takes O(n) time, which can be slow. Stochastic graient escent. The secon technique is calle incremental graient escent. It avois computing the full graient each step, an only computes it for f i for a single ata point p i 2 P ; see algorithm Algorithm Incremental Graient Descent(f,x start ) initialize x (0) = x start 2 R ; i =1. repeat x (k+1) := x (k) krf i (x (k) ) i := (i + 1) mo n until (krf(x (k) )kapple ) (or perhaps average of a few steps) return x (k) A more common variant of this is calle stochastic graient escent; see Algorithm Instea of choosing the ata points in orer, it selects a ata point p i at ranom each iteration (the term stochastic refers to this ranomness). Algorithm Stochastic Graient Descent(f,x start ) initialize x (0) = x start 2 R repeat Ranomly choose i 2{1, 2,...,n} x (k+1) := x (k) krf i (x (k) ) until (krf(x (k) )kapple ) (or perhaps average of a few steps) return x (k) These algorithms are often much faster than the batch version since each iteration now takes O(1) time. However, it oes not automatically inherent all of the nice convergence results from what is known about

9 (strongly) convex functions. Yet, since in most settings we are intereste in, there is an abunance of ata points from the same moel. This implies they shoul have a roughly similar effect. In practice when one is far from the optimal moel, these steps converge about as well as the batch version (but much much faster). When one is close to the optimal moel, then the incremental / stochastic variants may not exactly converge. However, if one is satisfie to reach a point that is close enough to optimal, there are some ranomize (PAC-style) guarantees possible for the stochastic variant. An in fact, for very large ata sets (i.e., n is very big) they typically converge before the algorithm even uses all (or even most) of the ata points.