Lagrange multipliers 14.8 14 October 2013
Example: Optimization with constraint. Example: Find the extreme values of f (x, y) = x + 2y on the ellipse 3x 2 + 4y 2 = 3. 3/2 1 1 3/2
Example: Optimization with constraint. Example: Find the extreme values of f (x, y) = x + 2y on the ellipse 3x 2 + 4y 2 = 3. 3/2 Maximum? 1 1 Minimum? 3/2
Example: Optimization with constraint. Example: Find the extreme values of f (x, y) = x + 2y on the ellipse 3x 2 + 4y 2 = 3. 3/2 Maximum? 1 1 Minimum? 3/2 Idea: f increases most rapidly when you move in the direction
Example: Optimization with constraint. Example: Find the extreme values of f (x, y) = x + 2y on the ellipse 3x 2 + 4y 2 = 3. 3/2 Maximum? 1 1 Minimum? 3/2 Idea: f increases most rapidly when you move in the direction 1, 2 which is f.
Example: Optimization with constraint. Example: Find the extreme values of f (x, y) = x + 2y on the ellipse 3x 2 + 4y 2 = 3. 3/2 Maximum? 1 1 Minimum? 3/2 Idea: f increases most rapidly when you move in the direction 1, 2 which is f. Constraint: 3x 2 + 4y 2 = 3, Objective function: f (x, y) = x + 2y
n f tangent line constraint f can be increased by moving right (clockwise) along curve
f n tangent line constraint f can be increased by moving left (counterclockwise) along curve
f n tangent line constraint f cannot be increased by moving in either direction along curve
constraint Idea: f has a critical point when f is parallel to the normal vector of the constraint.
Optimization with constraint: closer to rigorous. Same idea, closer to being rigorous: At a point on the constraint curve or surface, the possible directions of motion are given by the tangent plane. The rate of change of f in a given direction is the dot product of f with that tangent vector.
Optimization with constraint: closer to rigorous. Same idea, closer to being rigorous: At a point on the constraint curve or surface, the possible directions of motion are given by the tangent plane. The rate of change of f in a given direction is the dot product of f with that tangent vector. The direction in which f increases most rapidly is the projection of f onto the tangent plane.
Optimization with constraint: closer to rigorous. Same idea, closer to being rigorous: At a point on the constraint curve or surface, the possible directions of motion are given by the tangent plane. The rate of change of f in a given direction is the dot product of f with that tangent vector. The direction in which f increases most rapidly is the projection of f onto the tangent plane. The point is a critical point if and only if this projection is zero, i.e., f is orthogonal to the tangent plane.
Optimization with constraint: closer to rigorous. Same idea, closer to being rigorous: At a point on the constraint curve or surface, the possible directions of motion are given by the tangent plane. The rate of change of f in a given direction is the dot product of f with that tangent vector. The direction in which f increases most rapidly is the projection of f onto the tangent plane. The point is a critical point if and only if this projection is zero, i.e., f is orthogonal to the tangent plane. f is orthogonal to the tangent plane if and only if f is parallel to the tangent plane s normal vector. Recall that if the constraint is a level surface of a function g then the normal vector to the tangent plane is given by the gradient vector g.
Lagrange s Theorem. Theorem (Lagrange s Theorem) If P is a critical point of f (x, y) on the curve g(x, y) = k, and g P 0, then f P is parallel to g P : there is a scalar λ such that f P = λ g P.
Lagrange s Theorem. Theorem (Lagrange s Theorem) If P is a critical point of f (x, y) on the curve g(x, y) = k, and g P 0, then f P is parallel to g P : there is a scalar λ such that f P = λ g P. Theorem (Lagrange s Theorem, three variables) If P is a critical point of f (x, y, z) on the surface g(x, y, z) = k, and g P 0, then f P is parallel to g P : there is a scalar λ such that f P = λ g P.
Lagrange s Theorem. Theorem (Lagrange s Theorem) If P is a critical point of f (x, y) on the curve g(x, y) = k, and g P 0, then f P is parallel to g P : there is a scalar λ such that f P = λ g P. Theorem (Lagrange s Theorem, three variables) If P is a critical point of f (x, y, z) on the surface g(x, y, z) = k, and g P 0, then f P is parallel to g P : there is a scalar λ such that f P = λ g P. The scalar λ is called a Lagrange multiplier.
Lagrange s Theorem. Theorem (Lagrange s Theorem) If P is a critical point of f (x, y) on the curve g(x, y) = k, and g P 0, then f P is parallel to g P : there is a scalar λ such that f P = λ g P. Theorem (Lagrange s Theorem, three variables) If P is a critical point of f (x, y, z) on the surface g(x, y, z) = k, and g P 0, then f P is parallel to g P : there is a scalar λ such that f P = λ g P. The scalar λ is called a Lagrange multiplier. Usually looking for the point P or the value f (P), not λ.
Clicker Question: This plot shows the gradient vectors for a (hidden) function f (x, y) and a linear constraint. Which point is closest to the global min of f (x, y) on this constraint? A B C D receiver channel: 41 session ID: bsumath275
Example. f (x, y) = x + 2y g(x, y) = 3x 2 + 4y 2, g(x, y) = 3
Example. f (x, y) = x + 2y g(x, y) = 3x 2 + 4y 2, g(x, y) = 3 We have: f = 1, 2, g = 6x, 8y
Example. f (x, y) = x + 2y g(x, y) = 3x 2 + 4y 2, g(x, y) = 3 We have: f = 1, 2, g = 6x, 8y Solve 1 = 6xλ 2 = 8yλ 3x 2 + 4y 2 = 3
Example. f (x, y) = x + 2y g(x, y) = 3x 2 + 4y 2, g(x, y) = 3 We have: Solve f = 1, 2, g = 6x, 8y 1 = 6xλ 2 = 8yλ 3x 2 + 4y 2 = 3 = λ = 1 6x = 2 = 8y 6x = y = 3 2 x
Example. f (x, y) = x + 2y g(x, y) = 3x 2 + 4y 2, g(x, y) = 3 We have: Solve f = 1, 2, g = 6x, 8y 1 = 6xλ 2 = 8yλ 3x 2 + 4y 2 = 3 = λ = 1 6x = 2 = 8y 6x = y = 3 2 x = 3x 2 + 4( 3 2 x ) 2 = 3 = 12x 2 = 3
Example. f (x, y) = x + 2y g(x, y) = 3x 2 + 4y 2, g(x, y) = 3 We have: Solve f = 1, 2, g = 6x, 8y 1 = 6xλ 2 = 8yλ 3x 2 + 4y 2 = 3 = λ = 1 6x = 2 = 8y 6x = y = 3 2 x = 3x 2 + 4( 3 2 x ) 2 = 3 = 12x 2 = 3 = x = ± 1 2
Example. f (x, y) = x + 2y g(x, y) = 3x 2 + 4y 2, g(x, y) = 3 We have: Solve f = 1, 2, g = 6x, 8y 1 = 6xλ 2 = 8yλ 3x 2 + 4y 2 = 3 = λ = 1 6x = 2 = 8y 6x = y = 3 2 x = 3x 2 + 4( 3 2 x ) 2 = 3 = 12x 2 = 3 = x = ± 1 2 = y = ±3 4
Example, concluded. f = 1, 2, g = 6x, 8y Critical points: ( 1, ) ( 3 2 4, 1, ) 3 2 4. f (1/2, 3/4) = 2, maximum f ( 1/2, 3/4) = 2, minimum 3/2 1 1 3/2
Why is Lagrange s Theorem true? Proof (two-variable case). Let r(t) be a parametrization of the curve g(x, y) = k, with r(0) = P and r (0) 0. (There exists such a parametrization, by the Implicit Function Theorem from real analysis.)
Why is Lagrange s Theorem true? Proof (two-variable case). Let r(t) be a parametrization of the curve g(x, y) = k, with r(0) = P and r (0) 0. (There exists such a parametrization, by the Implicit Function Theorem from real analysis.) By assumption t = 0 is a critical point of f (r(t)).
Why is Lagrange s Theorem true? Proof (two-variable case). Let r(t) be a parametrization of the curve g(x, y) = k, with r(0) = P and r (0) 0. (There exists such a parametrization, by the Implicit Function Theorem from real analysis.) By assumption t = 0 is a critical point of f (r(t)). Thus d dt f (r(t)) t=0 = 0.
Why is Lagrange s Theorem true? Proof (two-variable case). Let r(t) be a parametrization of the curve g(x, y) = k, with r(0) = P and r (0) 0. (There exists such a parametrization, by the Implicit Function Theorem from real analysis.) By assumption t = 0 is a critical point of f (r(t)). Thus d dt f (r(t)) t=0 = 0. The Chain Rule for Paths gives f P r (0) = 0.
Why is Lagrange s Theorem true? Proof (two-variable case). Let r(t) be a parametrization of the curve g(x, y) = k, with r(0) = P and r (0) 0. (There exists such a parametrization, by the Implicit Function Theorem from real analysis.) By assumption t = 0 is a critical point of f (r(t)). Thus d dt f (r(t)) t=0 = 0. The Chain Rule for Paths gives f P r (0) = 0. That is, f P is perpendicular to r (0), the tangent vector to the curve at P. That is, f P is parallel to the normal vector at P, which is g P.
Example. Example: Find the extreme values of f (x, y, z) = xy + z 2 on the unit sphere x 2 + y 2 + z 2 = 1.
Example. Example: Find the extreme values of f (x, y, z) = xy + z 2 on the unit sphere x 2 + y 2 + z 2 = 1. f = y, x, 2z g = 2x, 2y, 2z Solve for (x, y, z) in f = λ g:
Example. Example: Find the extreme values of f (x, y, z) = xy + z 2 on the unit sphere x 2 + y 2 + z 2 = 1. f = y, x, 2z g = 2x, 2y, 2z Solve for (x, y, z) in f = λ g: y = λ2x x = λ2y 2z = λ2z x 2 + y 2 + z 2 = 1
Example. Example: Find the extreme values of f (x, y, z) = xy + z 2 on the unit sphere x 2 + y 2 + z 2 = 1. f = y, x, 2z g = 2x, 2y, 2z Solve for (x, y, z) in f = λ g: y = λ2x x = λ2y 2z = λ2z x 2 + y 2 + z 2 = 1 = y = 4λ 2 y = y = 0 or 4λ 2 = 1
Example, continued. f (x, y, z) = xy + z 2 g(x, y, z) = x 2 + y 2 + z 2 = 1 f = y, x, 2z g = 2x, 2y, 2z y = λ2x, x = λ2y, 2z = λ2z, x 2 + y 2 + z 2 = 1
Example, continued. f (x, y, z) = xy + z 2 g(x, y, z) = x 2 + y 2 + z 2 = 1 f = y, x, 2z g = 2x, 2y, 2z y = λ2x, x = λ2y, 2z = λ2z, x 2 + y 2 + z 2 = 1 y = 0: = x = 0 = z = ±1
Example, continued. f (x, y, z) = xy + z 2 g(x, y, z) = x 2 + y 2 + z 2 = 1 f = y, x, 2z g = 2x, 2y, 2z y = λ2x, x = λ2y, 2z = λ2z, x 2 + y 2 + z 2 = 1 y = 0: = x = 0 = z = ±1 4λ 2 = 1: { = λ = ± 1 2 = z = 0 = x = ±y x 2 + y 2 = 1
Example, continued. f (x, y, z) = xy + z 2 g(x, y, z) = x 2 + y 2 + z 2 = 1 f = y, x, 2z g = 2x, 2y, 2z y = λ2x, x = λ2y, 2z = λ2z, x 2 + y 2 + z 2 = 1 y = 0: = x = 0 = z = ±1 4λ 2 = 1: { = λ = ± 1 2 = z = 0 = x = ±y x 2 + y 2 = 1 Six critical points: (0, 0, ±1), (±1/ 2, ±1/ 2, 0).
Example, concluded. f (0, 0, ±1/2) = 1 f (1/ 2, 1/ 2, 0) = f ( 1/ 2, 1/ 2, 0) = 1 2 f (1/ 2, 1/ 2, 0) = f ( 1/ 2, 1/ 2, 0) = 1 2
Example, concluded. f (0, 0, ±1/2) = 1 f (1/ 2, 1/ 2, 0) = f ( 1/ 2, 1/ 2, 0) = 1 2 f (1/ 2, 1/ 2, 0) = f ( 1/ 2, 1/ 2, 0) = 1 2 The maximum is 1, at (0, 0, ±1). The minimum is 1/2, at ±(1/ 2, 1/ 2, 0).
Multiple constraints. Example: Find the maximum of f (x, y, z) = x on the curve C given as the intersection of g(x, y, z) = 2x 2 + 7y 2 + 5z 2 = 3, h(x, y, z) = x + y + z = 0
Multiple constraints. Example: Find the maximum of f (x, y, z) = x on the curve C given as the intersection of g(x, y, z) = 2x 2 + 7y 2 + 5z 2 = 3, h(x, y, z) = x + y + z = 0 Answer: At critical point P, for some scalars λ, µ. f P = λ g P + µ h P
Multiple constraints. Example: Find the maximum of f (x, y, z) = x on the curve C given as the intersection of g(x, y, z) = 2x 2 + 7y 2 + 5z 2 = 3, h(x, y, z) = x + y + z = 0 Answer: At critical point P, for some scalars λ, µ. f P = λ g P + µ h P f = 1, 0, 0, g = 4x, 14y, 10z, h = 1, 1, 1
Multiple constraints, continued. f = 1, 0, 0, g = 4x, 14y, 10z, h = 1, 1, 1 Solve for (x, y, z) in f P = λ g P + µ h P : 1 = 4λx + µ 0 = 14λy + µ 0 = 10λz + µ 2x 2 + 7y 2 + 5z 2 = 3 x + y + z = 0
Multiple constraints, continued. f = 1, 0, 0, g = 4x, 14y, 10z, h = 1, 1, 1 Solve for (x, y, z) in f P = λ g P + µ h P : 1 = 4λx + µ 0 = 14λy + µ 0 = 10λz + µ 2x 2 + 7y 2 + 5z 2 = 3 x + y + z = 0 ( ) 6 5 = = two critical points: ±, 59 2 59, 7 2 59
Multiple constraints, continued. f = 1, 0, 0, g = 4x, 14y, 10z, h = 1, 1, 1 Solve for (x, y, z) in f P = λ g P + µ h P : 1 = 4λx + µ 0 = 14λy + µ 0 = 10λz + µ 2x 2 + 7y 2 + 5z 2 = 3 x + y + z = 0 ( ) 6 5 = = two critical points: ±, 59 2 59, 7 2 59 6 So, the maximum value of f (x, y, z) = x is. 59
III 3x + y/2 = 12 Clicker (a) I < II, Question: < III (b) III < II < I (c) II < III < I (d) This II < contour I < III plot of f (x, y) also shows the circle of radius 2 (e) centered III < I < II at (0, 0). If you are restricted to being on the circle, how many local maxes and mins does f (x, y) have? 154. This contour plot of f(x, y) also shows the circle of radius 2 centered at (0,0). If you are restricted to being on the circle, how many local maxes and mins does f(x, y) have? A. 1 max, 1 min B. 2 maxes, 2 mins C. 3 maxes, 3 mins D. None receiver channel: 41 48 session ID: bsumath275
Clicker Question: Suppose (a, b, c) (0, 0, 0). How many local maxes and mins does the function f (x, y, z) = ax + by + cz have on the sphere x 2 + y 2 + z 2 = 1? A. 1 max, 1 min B. 2 maxes, 2 mins C. 3 maxes, 3 mins D. None E. Depends on a, b, c receiver channel: 41 session ID: bsumath275