Minima, Maxima, Saddle points

Similar documents
(1) Given the following system of linear equations, which depends on a parameter a R, 3x y + 5z = 2 4x + y + (a 2 14)z = a + 2

Tangent Planes/Critical Points

Unconstrained Optimization

Functions of Two variables.

3.3 Optimizing Functions of Several Variables 3.4 Lagrange Multipliers

Lecture 2 Optimization with equality constraints

Chapter 3 Numerical Methods

Math 213 Exam 2. Each question is followed by a space to write your answer. Please write your answer neatly in the space provided.

2 Second Derivatives. As we have seen, a function f (x, y) of two variables has four different partial derivatives: f xx. f yx. f x y.

= w. w u. u ; u + w. x x. z z. y y. v + w. . Remark. The formula stated above is very important in the theory of. surface integral.

5 Day 5: Maxima and minima for n variables.

Optimizations and Lagrange Multiplier Method

Multivariate Calculus Review Problems for Examination Two

LECTURE 18 - OPTIMIZATION

Date: 16 July 2016, Saturday Time: 14:00-16:00 STUDENT NO:... Math 102 Calculus II Midterm Exam II Solutions TOTAL. Please Read Carefully:

7/26/2018 SECOND HOURLY PRACTICE V Maths 21a, O.Knill, Summer 2018

. 1. Chain rules. Directional derivative. Gradient Vector Field. Most Rapid Increase. Implicit Function Theorem, Implicit Differentiation

22. LECTURE 22. I can define critical points. I know the difference between local and absolute minimums/maximums.

Daily WeBWorK, #1. This means the two planes normal vectors must be multiples of each other.

11/1/2017 Second Hourly Practice 11 Math 21a, Fall Name:

Convexity and Optimization

Affine function. suppose f : R n R m is affine (f(x) =Ax + b with A R m n, b R m ) the image of a convex set under f is convex

Constrained and Unconstrained Optimization

Convexity Theory and Gradient Methods

Convexity and Optimization

Multivariate Calculus: Review Problems for Examination Two

Lecture 6: Chain rule, Mean Value Theorem, Tangent Plane

14.5 Directional Derivatives and the Gradient Vector

Linear and quadratic Taylor polynomials for functions of several variables.

MTAEA Convexity and Quasiconvexity

2. Optimization problems 6

Answer sheet: Second Midterm for Math 2339

Local and Global Minimum

Lecture 10: SVM Lecture Overview Support Vector Machines The binary classification problem

NAME: Section # SSN: X X X X

Pre-Calculus Guided Notes: Chapter 10 Conics. A circle is

Section 4: Extreme Values & Lagrange Multipliers.

Tutorial on Convex Optimization for Engineers

Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 2. Convex Optimization

Winter 2012 Math 255 Section 006. Problem Set 7

11/1/2011 SECOND HOURLY PRACTICE IV Math 21a, Fall Name:

Lesson 4: Gradient Vectors, Level Curves, Maximums/Minimums/Saddle Points

= f (a, b) + (hf x + kf y ) (a,b) +

FAQs on Convex Optimization

Lecture 2: August 31

Convexity I: Sets and Functions

Math 213 Calculus III Practice Exam 2 Solutions Fall 2002

Chapter Multidimensional Gradient Method

Practice problems from old exams for math 233 William H. Meeks III December 21, 2009

302 CHAPTER 3. FUNCTIONS OF SEVERAL VARIABLES. 4. Function of several variables, their domain. 6. Limit of a function of several variables

f xx (x, y) = 6 + 6x f xy (x, y) = 0 f yy (x, y) = y In general, the quantity that we re interested in is

Optimization Methods: Optimization using Calculus-Stationary Points 1. Module - 2 Lecture Notes 1

Precomposing Equations

1. Gradient and Chain Rule. First a reminder about the gradient f of a smooth function f. In R n, the gradient operator or grad has the form

Worksheet 2.7: Critical Points, Local Extrema, and the Second Derivative Test

Optimization. Industrial AI Lab.

EC5555 Economics Masters Refresher Course in Mathematics September Lecture 6 Optimization with equality constraints Francesco Feri

Math 241, Final Exam. 12/11/12.

Edge and local feature detection - 2. Importance of edge detection in computer vision

MATH. 2153, Spring 16, MWF 12:40 p.m. QUIZ 1 January 25, 2016 PRINT NAME A. Derdzinski Show all work. No calculators. The problem is worth 10 points.

LINEAR ALGEBRA AND VECTOR ANALYSIS MATH 22A

COM Optimization for Communications Summary: Convex Sets and Convex Functions

Topic 6: Calculus Integration Volume of Revolution Paper 2

What you will learn today

Scale Space and PDE methods in image analysis and processing. Arjan Kuijper

Week 5: Geometry and Applications

MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question.

Hw 4 Due Feb 22. D(fg) x y z (

Scott Smith Advanced Image Processing March 15, Speeded-Up Robust Features SURF

Section 4.2 selected answers Math 131 Multivariate Calculus D Joyce, Spring 2014

Second Midterm Exam Math 212 Fall 2010

21-256: Lagrange multipliers

Convex Optimization - Chapter 1-2. Xiangru Lian August 28, 2015

Convex Sets (cont.) Convex Functions

Convex Optimization. 2. Convex Sets. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University. SJTU Ying Cui 1 / 33

Lecture 25 Nonlinear Programming. November 9, 2009

UNIT 3 EXPRESSIONS AND EQUATIONS Lesson 3: Creating Quadratic Equations in Two or More Variables

Bob Brown Math 251 Calculus 1 Chapter 4, Section 4 Completed 1 CCBC Dundalk

Catholic Central High School

CMU-Q Lecture 9: Optimization II: Constrained,Unconstrained Optimization Convex optimization. Teacher: Gianni A. Di Caro

Lecture 5: Properties of convex sets

A small review, Second Midterm, Calculus 3, Prof. Montero 3450: , Fall 2008

Finding local maxima and minima for functions z=f(x,y) [the cookbook]

Mathematical Programming and Research Methods (Part II)

Lecture 2: August 29, 2018

MEI Desmos Tasks for AS Pure

Linear Transformations

1. No calculators or other electronic devices are allowed during this exam.

EXTRA-CREDIT PROBLEMS ON SURFACES, MULTIVARIABLE FUNCTIONS AND PARTIAL DERIVATIVES

x 6 + λ 2 x 6 = for the curve y = 1 2 x3 gives f(1, 1 2 ) = λ actually has another solution besides λ = 1 2 = However, the equation λ

11/6/2012 SECOND HOURLY Math 21a, Fall Name:

Quadratic Equations. Learning Objectives. Quadratic Function 2. where a, b, and c are real numbers and a 0

Characterizing Improving Directions Unconstrained Optimization

Introduction to Modern Control Systems

7/28/2011 SECOND HOURLY PRACTICE V Maths 21a, O.Knill, Summer 2011

Lecture 4: Convexity

for Approximating the Analytic Center of a Polytope Abstract. The analytic center of a polytope P + = fx 0 : Ax = b e T x =1g log x j

AH Matrices.notebook November 28, 2016

Numerical Methods Lecture 1

Inverse and Implicit functions

Transcription:

Minima, Maxima, Saddle points Levent Kandiller Industrial Engineering Department Çankaya University, Turkey Minima, Maxima, Saddle points p./9

Scalar Functions Let us remember the properties for maxima, minima and saddle points when we have scalar functions with two variables with the help of the following examples. Minima, Maxima, Saddle points p./9

Scalar Functions Example. Let f(x,y) = x + y. Find the extreme points: Minima, Maxima, Saddle points p./9

Scalar Functions Example. Let f(x,y) = x + y. Find the extreme points:.5.5.5.5.5.5 Minima, Maxima, Saddle points p./9

Scalar Functions Example. Let f(x,y) = x + y. Find the extreme points:.5.5 f(x,y) x = x =. f(x,y) x =, y = y =. y =. Since we have only one critical point, it is either the maximum or the.5.5.5.5 minimum. We observe that f(x, y) takes only nonnegative values. Thus, we see that the origin is the minimum point. Minima, Maxima, Saddle points p./9

Scalar Functions Example. Find the extreme points of f(x,y) = xy x y x y + 4. Minima, Maxima, Saddle points p.3/9

Scalar Functions Example. Find the extreme points of f(x,y) = xy x y x y + 4. The function is differentiable and has no boundary points. Minima, Maxima, Saddle points p.3/9

Scalar Functions Example. Find the extreme points of f(x,y) = xy x y x y + 4. f x = f(x,y) x Thus, x = y = is the critical point. = y x, f y = f(x,y) y = x y. Minima, Maxima, Saddle points p.3/9

Scalar Functions Example. Find the extreme points of f(x,y) = xy x y x y + 4. f x = f(x,y) x Thus, x = y = is the critical point. f xx = f(x,y) x = y x, f y = f(x,y) y = x y. = = f(x,y) y = f yy, f xy = f(x,y) x y =. Minima, Maxima, Saddle points p.3/9

Scalar Functions Example. Find the extreme points of f(x,y) = xy x y x y + 4. f x = f(x,y) x Thus, x = y = is the critical point. f xx = f(x,y) x = y x, f y = f(x,y) y = x y. = = f(x,y) y = f yy, f xy = f(x,y) x y =. The discriminant (Jacobian) of f at (a, b) = (, ) is f xx f xy f xy f yy = f xx f yy f xy = 4 = 3. Since f xx <, f xx f yy f xy > f has a local maximum at (, ). Minima, Maxima, Saddle points p.3/9

Scalar Functions Theorem. The extreme values for f(x,y) can occur only at i. Boundary points of the domain of f. ii. Critical points (interior points where f x = f y =, or points where f x or f y fails to exist). Minima, Maxima, Saddle points p.4/9

Scalar Functions Theorem. If the first and second order partial derivatives of f are continuous throughout an open region containing a point (a, b) and f x (a,b) = f y (a,b) =, you may be able to classify (a,b) with the second derivative test: i. f xx <, f xx f yy fxy > at (a,b) local maximum; ii. f xx >, f xx f yy fxy > at (a,b) local minimum; iii. f xx f yy fxy < at (a,b) saddle point; iv. f xx f yy fxy = at (a,b) test is inconclusive (f is singular). Minima, Maxima, Saddle points p.4/9

Quadratic forms Definition. The quadratic term f(x, y) = ax + bxy + cy is positive definite (negative definite) if and only if a > (a < ) and ac b >. Minima, Maxima, Saddle points p.5/9

Quadratic forms Definition. The quadratic term f(x, y) = ax + bxy + cy is positive definite (negative definite) if and only if a > (a < ) and ac b >. f has a minimum (maximum) at x = y = if and only if f xx (, ) > (f xx (, ) < ) and f xx (, )f yy (, ) > f xy(, ). Minima, Maxima, Saddle points p.5/9

Quadratic forms Definition. The quadratic term f(x, y) = ax + bxy + cy is positive definite (negative definite) if and only if a > (a < ) and ac b >. f has a minimum (maximum) at x = y = if and only if f xx (, ) > (f xx (, ) < ) and f xx (, )f yy (, ) > f xy(, ). If f(, ) =, we term f as positive (negative) semi-definite provided the above conditions hold. Minima, Maxima, Saddle points p.5/9

Quadratic forms Now, we are able to introduce matrices to the quadratic forms: ax + bxy + cy = [x,y] a b x. b c y Minima, Maxima, Saddle points p.6/9

Quadratic forms Thus, for any symmetric A, the product f = x T Ax is a pure quadratic form: it has a stationary point at the origin and no higher terms. Minima, Maxima, Saddle points p.6/9

Quadratic forms Thus, for any symmetric A, the product f = x T Ax is a pure quadratic form: it has a stationary point at the origin and no higher terms. a a a n x xa T a a a n x x = [x,x,,x n ]....... a n a n a nn x n Minima, Maxima, Saddle points p.6/9

Quadratic forms Thus, for any symmetric A, the product f = x T Ax is a pure quadratic form: it has a stationary point at the origin and no higher terms. a a a n x xa T a a a n x x = [x,x,,x n ]....... a n a n a nn x n = a x + a x x + + a nn x n = n n i= j= a ijx i x j. Minima, Maxima, Saddle points p.6/9

Quadratic forms Definition. If A is such that a ij = f x i x j (hence symmetric), it is called the Hessian matrix. Minima, Maxima, Saddle points p.7/9

Quadratic forms Definition. If A is such that a ij = f x i x j (hence symmetric), it is called the Hessian matrix. If A is positive definite (x T Ax >, x θ) and if f has a stationary point at the origin (all first derivatives at the origin are zero), then f has a minimum. Minima, Maxima, Saddle points p.7/9

Quadratic forms Remark. Let f : R n R and x R n be the local minimum, f(x ) = θ and f(x ) is positive definite. Minima, Maxima, Saddle points p.8/9

Quadratic forms Remark. Let f : R n R and x R n be the local minimum, f(x ) = θ and f(x ) is positive definite. We are able to explore the neighborhood of x by means of x + x, where x is sufficiently small (such that the second order Taylor s approximation is pretty good) and positive. Minima, Maxima, Saddle points p.8/9

Quadratic forms Remark. Let f : R n R and x R n be the local minimum, f(x ) = θ and f(x ) is positive definite. We are able to explore the neighborhood of x by means of x + x, where x is sufficiently small (such that the second order Taylor s approximation is pretty good) and positive. Then, f(x + x) = f(x ) + x T f(x ) + xt f(x ) x. Minima, Maxima, Saddle points p.8/9

Quadratic forms Remark. Let f : R n R and x R n be the local minimum, f(x ) = θ and f(x ) is positive definite. We are able to explore the neighborhood of x by means of x + x, where x is sufficiently small (such that the second order Taylor s approximation is pretty good) and positive. Then, f(x + x) = f(x ) + x T f(x ) + xt f(x ) x. The second term is zero since x is a critical point and the third term is positive since the Hessian evaluated at x is positive definite. Thus, the left hand side is always strictly greater than the right hand side, indicating the local minimality of x. Minima, Maxima, Saddle points p.8/9

E Collaborative Work: Let f(x, x ) = 3 x3 + x + x x + x x + 9. Find the stationary and boundary points, then find the minimizer and the maximizer over 4 x x 3 33 3 3 3 9 8 7 6 5 4 3 9,,4,6,8,,4,6,8 C A,,4 B,6,8 3-4 -3,5 -,9 -,4 -,9 -,4 -.9 -.4 Minima, Maxima, Saddle points p.9/9

E Collaborative Work: Let f(x, x ) = 3 x3 + x + x x + x x + 9. Find the stationary and boundary points, then find the minimizer and the maximizer over 4 x x 3 33 3 3 3 9 8 7 6 5 4 3 9,,4,6,8,,4,6,8 C A, E,4 B,6,8 3-4 -3,5 -,9 -,4 -,9 -,4 -.9 -.4 33 3 3 3 9 8 7 6 5 4 3 9,,4,6,8,,4,6,8 C A B,,4,6,8 3-4 -.4 -.9 -,4 -,9 -,4 -,9-3,5 Minima, Maxima, Saddle points p.9/9

E Collaborative Work: Let f(x, x ) = 3 x3 + x + x x + x x + 9. Find the stationary and boundary points, then find the minimizer and the maximizer over 4 x x 3 f x f(x) = = x + x + x =. (x )(x ) = x + x x = x f x 33 3 3 3 9 8 7 6 5 4 3 9,,4,6,8,,4,6,8 C A,,4 B,6,8 3-4 -3,5 -,9 -,4 -,9 -,4 -.9 -.4 Minima, Maxima, Saddle points p.9/9

E 33 3 Collaborative Work: f(x) = f x f x = x + x + x x + x. = 3 3 9 8 7 6 5 4 3 9,,4,6 C A B,8,,4,6,8,,4,6,8 3 -.4 -.9 -,4 -,9 -,4 -,9-3,5-4 (x )(x ) = x = x Therefore, x A = ¾, xb = defined by 4 x x 3. ¾ 3 are stationary points inside the region Minima, Maxima, Saddle points p.9/9

E 33 3 Collaborative Work: f(x) = f x f x = x + x + x x + x. = 3 3 9 8 7 6 5 4 3 9,,4,6 C A B,8,,4,6,8,,4,6,8 3 -.4 -.9 -,4 -,9 -,4 -,9-3,5-4 (x )(x ) = x = x Therefore, x A = ¾, xb = defined by 4 x x 3. ¾ 3 are stationary points inside the region Moreover, we have the following boundaries ¾ defined by x I = ¾ x x C = ¾, xii =, xd = ¾ 3 x ¾ and xiii = 4, xe = ¾ x 4 3, xiv =, xf = ¾ x 3 4. ¾ Minima, Maxima, Saddle points p.9/9

E Collaborative Work: Let the Hessian matrix be f(x) = f x x f x x f x x f x x = x + 33 3 3 3 9 8 7 6 5 4 3 9,,4,6,8.,,4,6,8 C A,,4 B,6,8 3-4 -.4 -.9 -,4 -,9 -,4 -,9-3,5 Minima, Maxima, Saddle points p.9/9

E Collaborative Work: Let the Hessian matrix be f(x) = f x x f x x have f(x A ) = f x x f x x 3 = x + 33 3 3 3 9 8 7 6 5 4 3 9, and f(x B ) =,4,6,8,,4,6,8 C A,,4 B,6,8 3-4 -3,5. Then, we 5. -,9 -,4 -,9 -,4 -.9 -.4 Minima, Maxima, Saddle points p.9/9

E Collaborative Work: Let the Hessian matrix be f(x) = f x x f x x have f(x A ) = f x x f x x 3 = x + 33 3 3 3 9 8 7 6 5 4 3 9, and f(x B ) = Let us check the positive definiteness of f(x A ): v T f(x A )v = [v, v ] ¾ 3 ¾ v v,4,6,8,,4,6,8 C A,,4 B,6,8 3-4 -3,5. Then, we 5. = 3v + 4v v + v. If v =.5 and v =., we will have v T f(x A )v <. On the other hand, if v =.5 and v =., we will have v T f(x A )v >. Thus, f(x A ) is indefinite. -,9 -,4 -,9 -,4 -.9 -.4 Minima, Maxima, Saddle points p.9/9

E Collaborative Work: Let the Hessian matrix be f(x) = f x x f x x have f(x A ) = Let us check f(x B ): v T f(x B )v = [v, v ] f x x f x x 3 ¾ 5 = x + 33 3 3 3 9 8 7 6 5 4 3 9, and f(x B ) = ¾ v v Thus, f(x B ) is positive definite and x B = f(x B ) = 9.66667,4,6,8,,4,6,8 C A,,4 B,6,8 3-4 -3,5. Then, we 5. = 5v + 4v v + v = v + (v + v ) >. ¾ 3 is a local minimizer with -,9 -,4 -,9 -,4 -.9 -.4 Minima, Maxima, Saddle points p.9/9

E 33 3 3 Collaborative Work: 3 9 8 7 6 C A 5 4 3 9 B -.4 -.9 -,4 -,9 -,4 Let us check the boundary defined by x I :,,4,6,8,,4,6,8,,4,6,8 3-4 -,9-3,5 f(, x ) = x x + 9 df(, x ) dx = x. = x =. Since d f(,x ) = >, x dx = > is the local minimizer outside the feasible region. As the first derivative is negative for 4 x, we will check x = for minimizer and x = 4 for maximizer. Minima, Maxima, Saddle points p.9/9

E 33 3 3 Collaborative Work: 3 9 8 7 6 C A 5 4 3 9 B -.4 -.9 -,4 -,9 -,4 Let us check the boundary defined by x II :,,4,6,8,,4,6,8,,4,6,8 3-4 -,9-3,5 f(3, x ) = x + 5x + 65 df(3, x ) dx = x + 5. = x = 5. Since d f(,x ) = >, x dx = 5 < 4 is the local minimizer outside the feasible region. As the first derivative is positive for 4 x, we will check x = 4 for minimizer and x = for maximizer. Minima, Maxima, Saddle points p.9/9

E 33 3 3 Collaborative Work: 3 9 8 7 6 C A 5 4 3 9 B -.4 -.9 -,4 -,9 -,4 Let us check the boundary defined by x III :,,4,6,8,,4,6,8,,4,6,8 3-4 -,9-3,5 f(x,) = 3 x3 + x +9 df(x,) dx = x +x. = x =,. Since d f(x,) dx ( d f(,) dx = x +, x = is the local minimizer = > ) on the boundary, and x = is the local maximizer ( d f(,) = < ) outside the feasible region. As dx the first derivative is positive for x 3, we will check x = 3 for maximizer. Minima, Maxima, Saddle points p.9/9

E Collaborative Work: Let us check the boundary defined by x IV : f(x, 4) = 3 x3 + x 8x + 3 df(x, 4) dx = x + x 8 =. x = ± +3. Since d f(x, 4) dx root x = + 33 =.373 is the local minimizer ( d f(.373,) dx 33 3 3 3 9 8 7 6 5 4 3 9,,4,6,8, = x + again, the positive > ), and the negative root is the local maximizer but it is outside the feasible region. As the first derivative is positive for x 3, we will check x = 3 for maximizer again.,4,6,8 C A,,4 B,6,8 3-4 -3,5 -,9 -,4 -,9 -,4 -.9 -.4 Minima, Maxima, Saddle points p.9/9

E 33 3 3 Collaborative Work: 3 9 8 7 6 C A 5 4 3 9 B -.4 -.9 -,4 -,9 -,4,,4,6,8,,4,6,8,,4,6,8 3-4 -,9-3,5 To sum up, we have to consider (, 3), (,) and (.373, 4) for the minimizer; (3,) and (, 4) for the maximizer: f(, 3) = 9.6667, f(,) = 9, f(.373, 4) = 9.859 (,) is the minimizer! f(3,) = 3.5, f(, 4) = 3 (3,) is the maximizer! Minima, Maxima, Saddle points p.9/9

2024 © technodocbox.com Privacy Policy | Terms of Service | Feedback