Lecture 1: Introduction

Transcription

1 Lecture 1 1 Linear and Combinatorial Optimization Anders Heyden Centre for Mathematical Sciences Lecture 1: Introduction The course and its goals Basic concepts Optimization Combinatorial optimization Local minima Convex optimization Linear optimization

2 Lecture 1 2 Administrative matters Lecturer: Anders Heyden heyden@maths.lth.se Assistant: Petter Strandmark petter@maths.lth.se Litterature: Papadimitriou, C. H. and Steiglitz, K. Combinatorial Optimization: Algorithms and Complexity, Dover 1998, ISBN (More theoretical) Holmberg, K., Optimering, Liber 2010, ISBN (in Swedish) Examination: - 4 hand-in exercises (some programming, some theory) - Two computer laboratory sessions - Take-home exam + short oral exam Course home-page: education/lth/courses/linkombopt/2013

3 Lecture 1 3 The course and its goals Goal: After the course you should be able to formulate optimization problems. determine the degree of difficulty for some optimization problems. implement solution algorithms. Linear programming (easy) Convex programming (easy?) Everything else (difficult?) I expect you to actively read the text in the book, correct me when you find errors in the course material or on the black board. I will try to present parts of the material so you will find it easier to learn the content. I will present examples for improved understanding and motivation.

4 Lecture 1 4 Optimization Let D be a set and let f be a scalar valued function defined on D. Study the following questions: Is there a point x min D such that f min = f(x min ) f(y), y D? If so, then x min or sometimes (x min,f min ) a minimum. How can one find one or all such minima? The function f is called the objective function. Introduce notations min and argmin: f min = min x D f(x) x min = argmin x D f(x) If the domain D is discrete then the problem to find (x min,f min ) is called discrete or combinatorial optimization. This is in contrast to traditional optimization over a continuous set.

5 Lecture 1 5 Examples Linear programming: Good algorithms. Well studied (1940). Many applications (city planning, economy). Combinatorial optimization: Less rigorous (approximate) algorithms. Many interesting problems. Linear programming can be regarded as either a continuous or combinatorial problem. Kont Optimering Linjärprogrammering Kombinatorisk Optimering

6 Lecture 1 6 Local minima For optimization over a continuous domain we have the following definition. A point x loc is called a local minimum if there exists δ > 0 such that f(x loc ) f(y), y D, y x loc < δ If the gradient is non-zero at an interior point x D R n then one can find a point y with lower function value by searching along a descent direction, for example, Steepest Descent f.

7 Lecture 1 7 Discrete optimization For optimization over a discrete set, one can introduce the notion of neighbours. To each point x, one defines a set of neighbours G x. A point x loc is called a local minimum if f(x loc ) f(y), y G x. Optimization algorithm 1: Local search 1. Start in a point x. 2. If there exists a neighbour y G x with lower functional value than x, set x to this point and goto 1, else a local minimum has been found. End. Compare with graph theory (Domain - nodes, neighbour-nodes are connected with edges): NB: There may be many local minima.

8 Lecture 1 8 Convex Optimization Special case of optimization with continuous domain. Definition 1.1. A set C R n is called convex if x,y C,λ [0,1] λx+(1 λ)y C Definition 1.2. A function f is called convex if x,y C,λ [0,1] f(λx+(1 λ)y) λf(x)+(1 λ)f(y) Theorem 1.1. If f is a convex function then every local minimum is also a global minimum.

9 Lecture 1 9 Some properties of convex sets A closed half space is a convex set. A hyperplane is a convex set. The intersection of finitely many convex sets is convex. The solution set to Ax = b is a convex set (provided it is non-empty). Repeat terminolgy of sets in your favorite textbook on analysis in several variables: open and closed set interior, exterior and boundary point bounded, unbounded and compact set

10 Lecture 1 10 More on convex sets Definition 1.3. Let x j R n. The linear combination y = i λ ix i with λ i 0, i λ i = 1 is called a convex combination of the finite set {x 1,...,x n }. Theorem 1.2. The set of convex combinations of a finite set is convex. Such a set is called a convex polyhedron. A point u in a convex set S is called extreme point if it is not an interior point of some line segment in S. WARNING: This is an unfortunate notation since every local extreme point usually means local maximum or local minimum.

11 Lecture 1 11 Linear programming Definition 1.4. A general Linear programming problem: minimize or maximize z = c T x subject to A 1 x b 1 A 2 x b 2 A 3 x = b 3 Reformulations Change min to max. Change an inequality. Change an equality to an inequality. Change an inequality to an equality. Slack variable. Unconstrained variable.

12 Lecture 1 12 Reformulations Definition 1.5. Standard form subject to Ax b,x 0. max x z = ct x Definition 1.6. Canonical form subject to Ax = b,x 0. max x z = ct x Theorem 1.3. Linear programming (LP) is a convex optimization problem.

13 Lecture 1 13 Extreme Point Theorem Theorem 1.4. Let S be the domain to a general LP-problem. If S is non-empty and bounded, then there exists an optimal solution at an extreme point. If S is non-empty and unbounded and if there exists an optimal solution, then there exists an optimal solution at an extreme point. If there is no optimal solution, than either S is empty or unbounded.

14 Lecture 1 14 Repetition - Lecture 1 Objective function. Domain, min, argmin. Optimization (combinatorial, continuous). Local minimum. Convex function. Convex set. LP-problem.