CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 29
|
|
- Ashley Pitts
- 5 years ago
- Views:
Transcription
1 CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring / 29
2 CS 473: Algorithms, Spring 2018 Simplex and LP Duality Lecture 19 March 29, 2018 Some of the slides are courtesy Prof. Chekuri Ruta (UIUC) CS473 2 Spring / 29
3 Outline Simplex: Intuition and Implementation Details Computing starting vertex: equivalent to solving an LP! Infeasibility, Unboundedness, and Degeneracy. Duality: Bounding the objective value through weak-duality Strong Duality, Cone view. Ruta (UIUC) CS473 3 Spring / 29
4 Part I Recall Ruta (UIUC) CS473 4 Spring / 29
5 Feasible Region and Convexity Canonical Form Given A R n d, b R n 1 and c R 1 d, find x R d 1 max : s.t. c x Ax b Ruta (UIUC) CS473 5 Spring / 29
6 Feasible Region and Convexity Canonical Form Given A R n d, b R n 1 and c R 1 d, find x R d 1 max : s.t. c x Ax b 1 Each linear constraint defines a halfspace, a convex set. 2 Feasible region, which is an intersection of halfspaces, is a convex polyhedron. 3 Optimal value attained at a vertex of the polyhedron. Ruta (UIUC) CS473 5 Spring / 29
7 Ruta (UIUC) CS473 6 Spring / 29
8 Part II Simplex Ruta (UIUC) CS473 7 Spring / 29
9 Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Ruta (UIUC) CS473 8 Spring / 29
10 Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Questions Which neighbor to move to? When to stop? How much time does it take? Ruta (UIUC) CS473 8 Spring / 29
11 Observations For Simplex Suppose we are at a non-optimal vertex ˆx and optimal is x, then c x > c ˆx. Ruta (UIUC) CS473 9 Spring / 29
12 Observations For Simplex Suppose we are at a non-optimal vertex ˆx and optimal is x, then c x > c ˆx. How does (c x) change as we move from ˆx to x on the line joining the two? Ruta (UIUC) CS473 9 Spring / 29
13 Observations For Simplex Suppose we are at a non-optimal vertex ˆx and optimal is x, then c x > c ˆx. How does (c x) change as we move from ˆx to x on the line joining the two? Strictly increases! Ruta (UIUC) CS473 9 Spring / 29
14 Observations For Simplex Suppose we are at a non-optimal vertex ˆx and optimal is x, then c x > c ˆx. How does (c x) change as we move from ˆx to x on the line joining the two? Strictly increases! d = x ˆx is the direction from ˆx to x. (c d) = (c x ) (c ˆx) > 0. In x = ˆx + δd, as δ goes from 0 to 1, we move from ˆx to x. c x = c ˆx + δ(c d). Strictly increasing with δ! Due to convexity, all of these are feasible points. Ruta (UIUC) CS473 9 Spring / 29
15 Cone Definition Given a set of vectors D = {d 1,..., d k }, the cone spanned by them is just their positive linear combinations, i.e., cone(d) = {d d = k λ i d i, where λ i 0, i} i =1 Ruta (UIUC) CS Spring / 29
16 Cone at a Vertex Let z 1,..., z k be the neighboring vertices of ˆx. And let d i = z i ˆx be the direction from ˆx to z i. Lemma Any feasible direction of movement d from ˆx is in the cone({d 1,..., d k }). Ruta (UIUC) CS Spring / 29
17 Improving Direction Implies Improving Neighbor Lemma If d cone({d 1,..., d k }) and (c d) > 0, then there exists d i such that (c d i ) > 0. Ruta (UIUC) CS Spring / 29
18 Improving Direction Implies Improving Neighbor Lemma If d cone({d 1,..., d k }) and (c d) > 0, then there exists d i such that (c d i ) > 0. Proof. To the contrary suppose (c d i ) 0, i k. Since d is a positive linear combination of d i s, (c d) = (c k i =1 λ i d i ) = k i =1 λ i (c d i ) 0 A contradiction! Ruta (UIUC) CS Spring / 29
19 Improving Direction Implies Improving Neighbor Lemma If d cone({d 1,..., d k }) and (c d) > 0, then there exists d i such that (c d i ) > 0. Proof. To the contrary suppose (c d i ) 0, i k. Since d is a positive linear combination of d i s, (c d) = (c k i =1 λ i d i ) = k i =1 λ i (c d i ) 0 A contradiction! Theorem If vertex ˆx is not optimal then it has a neighbor where cost improves. Ruta (UIUC) CS Spring / 29
20 How Many Neighbors a Vertex Has? Geometric view... A R n d (n > d), b R n, the constraints are: Ax b Geometry of faces r linearly independent hyperplanes forms (d r) dimensional face. Ruta (UIUC) CS Spring / 29
21 How Many Neighbors a Vertex Has? Geometric view... A R n d (n > d), b R n, the constraints are: Ax b Geometry of faces r linearly independent hyperplanes forms (d r) dimensional face. Vertex: 0-D face. formed by d L.I. hyperplanes. Ruta (UIUC) CS Spring / 29
22 How Many Neighbors a Vertex Has? Geometric view... A R n d (n > d), b R n, the constraints are: Ax b Geometry of faces r linearly independent hyperplanes forms (d r) dimensional face. Vertex: 0-D face. formed by d L.I. hyperplanes. Edge: 1-D face. formed by (d 1) L.I. hyperlanes. Ruta (UIUC) CS Spring / 29
23 How Many Neighbors a Vertex Has? Geometric view... A R n d (n > d), b R n, the constraints are: Ax b Geometry of faces r linearly independent hyperplanes forms (d r) dimensional face. Vertex: 0-D face. formed by d L.I. hyperplanes. Edge: 1-D face. formed by (d 1) L.I. hyperlanes. In 2-dimension (d = 2) x x 1 Ruta (UIUC) CS Spring / 29
24 How Many Neighbors a Vertex Has? Geometric view... A R n d (n > d), b R n, the constraints are: Ax b In 3-dimension (d = 3) Geometry of faces r linearly independent hyperplanes forms (d r) dimensional face. Vertex: 0-dimensional face. formed by d L.I. hyperplanes Edge: 1-D face. formed by (d 1) L.I. hyperlanes. image source: webpage of Prof. Forbes W. Lewis Ruta (UIUC) CS Spring / 29
25 How Many Neighbors a Vertex Has? Geometry view... One neighbor per tight hyperplane. Therefore typically d. Suppose x is a neighbor of ˆx, then on the edge joining the two d 1 constraints are tight. These d 1 are also tight at both ˆx and x. One more constraints, say i, is tight at ˆx. Relaxing i at ˆx leads to x. 3 x Ruta (UIUC) CS Spring / 29
26 Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Questions + Answers Which neighbor to move to? One where objective value increases. Ruta (UIUC) CS Spring / 29
27 Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Questions + Answers Which neighbor to move to? One where objective value increases. When to stop? When no neighbor with better objective value. Ruta (UIUC) CS Spring / 29
28 Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Questions + Answers Which neighbor to move to? One where objective value increases. When to stop? When no neighbor with better objective value. How much time does it take? At most d neighbors to consider in each step. Ruta (UIUC) CS Spring / 29
29 Simplex in Higher Dimensions Simplex Algorithm 1 Start at a vertex of the polytope. 2 Compare value of objective function at each of the d neighbors. 3 Move to neighbor that improves objective function, and repeat step 2. 4 If no improving neighbor, then stop. Simplex is a greedy local-improvement algorithm! Works because a local optimum is also a global optimum convexity of polyhedra. Ruta (UIUC) CS Spring / 29
30 Part III Implementation of the Pivoting Step (Moving to an improving neighbor) Ruta (UIUC) CS Spring / 29
31 Moving to a Neighbor Fix a vertex ˆx. Let the d hyperplanes/constraints tight at ˆx be, d a ij x j = b i, 1 i d Equivalently, Âx = ˆb j=1 A neighbor vertex x is connected to ˆx by an edge. d 1 hyperplanes tight on this edge. To reach x, one hyperplane has to be relaxed, while maintaining other d 1 tight. 3 x Ruta (UIUC) CS Spring / 29
32 Moving to a Neighbor (Contd.)  1 =.. d 1... d d.. Lemma Moving in direction d i from ˆx, all except constraint i remain tight. Proof. For a small ɛ > 0, let y = ˆx + ɛ(d i ), then Ây = Â(ˆx + ɛd i ) = ˆx + ɛâ(  1 ) (.,i ) Ruta (UIUC) CS Spring / 29
33 Moving to a Neighbor (Contd.)  1 =.. d 1... d d.. Lemma Moving in direction d i from ˆx, all except constraint i remain tight. Proof. For a small ɛ > 0, let y = ˆx + ɛ(d i ), then Ây = Â(ˆx + ɛd i ) = ˆx + ɛâ(  1 ) (.,i ) = ˆb + ɛ[0,..., 1,..., 0] T Clearly, j a kjy j = b k, k i, and j a ijy j = b i ɛ < b i. Ruta (UIUC) CS Spring / 29
34 Computing the Neighbor Move in d i direction from ˆx, i.e., ˆx + ɛd i, and STOP when hit a new hyperplane! Need to ensure feasibility. Above lemma implies inequalities 1 through d will be satisfied. For any k > d, where A k is k th row of A, A k (ˆx + ɛd i ) b k (A k ˆx) + ɛ(a k d i ) b k ɛ(a k d i ) b k (A k ˆx) Ruta (UIUC) CS Spring / 29
35 Computing the Neighbor Move in d i direction from ˆx, i.e., ˆx + ɛd i, and STOP when hit a new hyperplane! Need to ensure feasibility. Above lemma implies inequalities 1 through d will be satisfied. For any k > d, where A k is k th row of A, A k (ˆx + ɛd i ) b k (A k ˆx) + ɛ(a k d i ) b k ɛ(a k d i ) b k (A k ˆx) (If (A k d i ) > 0) ɛ b k (A k ˆx ) A k d i Ruta (UIUC) CS Spring / 29
36 Computing the Neighbor Move in d i direction from ˆx, i.e., ˆx + ɛd i, and STOP when hit a new hyperplane! Need to ensure feasibility. Above lemma implies inequalities 1 through d will be satisfied. For any k > d, where A k is k th row of A, A k (ˆx + ɛd i ) b k (A k ˆx) + ɛ(a k d i ) b k ɛ(a k d i ) b k (A k ˆx) (If (A k d i ) > 0) ɛ b k (A k ˆx ) A k d i If moving towards hyperplane k (positive) Ruta (UIUC) CS Spring / 29
37 Computing the Neighbor Move in d i direction from ˆx, i.e., ˆx + ɛd i, and STOP when hit a new hyperplane! Need to ensure feasibility. Above lemma implies inequalities 1 through d will be satisfied. For any k > d, where A k is k th row of A, A k (ˆx + ɛd i ) b k (A k ˆx) + ɛ(a k d i ) b k ɛ(a k d i ) b k (A k ˆx) (If (A k d i ) > 0) ɛ b k (A k ˆx ) A k d i If moving towards hyperplane k (If (A k d i ) < 0) ɛ b k (A k ˆx ) A k d i If moving away from hyperplane k. No upper bound, and -ve lower bound! (positive) (negative) Ruta (UIUC) CS Spring / 29
38 Computing the Neighbor Algorithm NextVertex(ˆx, d i ) Set ɛ. For k = d n ɛ b k (A k ˆx ) A k d i If ɛ > 0 and ɛ < ɛ then set ɛ ɛ If ɛ < then return ˆx + ɛd i. Else return null. If (c d i ) > 0 then the algorithm returns an improving neighbor. Ruta (UIUC) CS Spring / 29
39 Factory Example max : x 1 + 6x 2 s.t. 0 x x x 1 + x ˆx = (0, 0) Ruta (UIUC) CS Spring / 29
40 Factory Example max : x 1 + 6x 2 s.t. 0 x x x 1 + x ˆx = (0, 0) [ ] 1 0 Â = 0 1 [ ] Â = = [d d 2 ] Ruta (UIUC) CS Spring / 29
41 Factory Example max : x 1 + 6x 2 s.t. 0 x x x 1 + x ˆx = (0, 0) [ ] 1 0 Â = 0 1 [ ] Â = = [d d 2 ] Moving in direction d 1 gives (200, 0) Ruta (UIUC) CS Spring / 29
42 Factory Example max : x 1 + 6x 2 s.t. 0 x x x 1 + x ˆx = (0, 0) [ ] 1 0 Â = 0 1 [ ] Â = = [d d 2 ] Moving in direction d 1 gives (200, 0) Moving in direction d 2 gives (0, 300). Ruta (UIUC) CS Spring / 29
43 Solving Linear Programming in Practice 1 Naïve implementation of Simplex algorithm can be very inefficient Exponential number of steps! Ruta (UIUC) CS Spring / 29
44 Solving Linear Programming in Practice 1 Naïve implementation of Simplex algorithm can be very inefficient 1 Choosing which neighbor to move to can significantly affect running time 2 Very efficient Simplex-based algorithms exist 3 Simplex algorithm takes exponential time in the worst case but works extremely well in practice with many improvements over the years 2 Non Simplex based methods like interior point methods work well for large problems. Ruta (UIUC) CS Spring / 29
45 Issues 1 Starting vertex 2 The linear program could be infeasible: No point satisfy the constraints. 3 The linear program could be unbounded: Polygon unbounded in the direction of the objective function. 4 More than d hyperplanes could be tight at a vertex, forming more than d neighbors. Ruta (UIUC) CS Spring / 29
46 Computing the Starting Vertex Equivalent to solving another LP! Find an x such that Ax b. If b 0 then trivial! Ruta (UIUC) CS Spring / 29
47 Computing the Starting Vertex Equivalent to solving another LP! Find an x such that Ax b. If b 0 then trivial! x = 0. Otherwise. Ruta (UIUC) CS Spring / 29
48 Computing the Starting Vertex Equivalent to solving another LP! Find an x such that Ax b. If b 0 then trivial! x = 0. Otherwise. min : s.t. Trivial feasible solution: s j a ijx j s b i, s 0 i Ruta (UIUC) CS Spring / 29
49 Computing the Starting Vertex Equivalent to solving another LP! Find an x such that Ax b. If b 0 then trivial! x = 0. Otherwise. min : s.t. s j a ijx j s b i, s 0 i Trivial feasible solution: x = 0, s = min i b i. Ruta (UIUC) CS Spring / 29
50 Computing the Starting Vertex Equivalent to solving another LP! Find an x such that Ax b. If b 0 then trivial! x = 0. Otherwise. min : s.t. s j a ijx j s b i, s 0 i Trivial feasible solution: x = 0, s = min i b i. If Ax b feasible then optimal value of the above LP is s = 0. Ruta (UIUC) CS Spring / 29
51 Computing the Starting Vertex Equivalent to solving another LP! Find an x such that Ax b. If b 0 then trivial! x = 0. Otherwise. min : s.t. s j a ijx j s b i, s 0 i Trivial feasible solution: x = 0, s = min i b i. If Ax b feasible then optimal value of the above LP is s = 0. Checks Feasibility! Ruta (UIUC) CS Spring / 29
52 Unboundedness: Example maximize x 2 x 1 + x 2 2 x 1, x 2 0 Unboundedness depends on both constraints and the objective function. Ruta (UIUC) CS Spring / 29
53 Unboundedness: Example maximize x 2 x 1 + x 2 2 x 1, x 2 0 Unboundedness depends on both constraints and the objective function. If unbounded in the direction of objective function, then the pivoting step in the simplex will detect it. Ruta (UIUC) CS Spring / 29
54 Degeneracy and Cycling More than d constraints are tight at vertex ˆx. Say d + 1. Suppose, we pick first d to form  such that ˆx = ˆb, and compute directions d 1,..., d d. Ruta (UIUC) CS Spring / 29
55 Degeneracy and Cycling More than d constraints are tight at vertex ˆx. Say d + 1. Suppose, we pick first d to form  such that ˆx = ˆb, and compute directions d 1,..., d d. Then NextVertex(ˆx, d i ) will encounter (d + 1) th constraint tight at ˆx and return the same vertex. Hence we are back to ˆx! Ruta (UIUC) CS Spring / 29
56 Degeneracy and Cycling More than d constraints are tight at vertex ˆx. Say d + 1. Suppose, we pick first d to form  such that ˆx = ˆb, and compute directions d 1,..., d d. Then NextVertex(ˆx, d i ) will encounter (d + 1) th constraint tight at ˆx and return the same vertex. Hence we are back to ˆx! Same phenomenon will repeat! Ruta (UIUC) CS Spring / 29
57 Degeneracy and Cycling More than d constraints are tight at vertex ˆx. Say d + 1. Suppose, we pick first d to form  such that ˆx = ˆb, and compute directions d 1,..., d d. Then NextVertex(ˆx, d i ) will encounter (d + 1) th constraint tight at ˆx and return the same vertex. Hence we are back to ˆx! Same phenomenon will repeat! This can be avoided by adding small random perturbation to b i s. Ruta (UIUC) CS Spring / 29
CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 50
CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 50 CS 473: Algorithms, Spring 2018 Introduction to Linear Programming Lecture 18 March
More informationCS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 36
CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 36 CS 473: Algorithms, Spring 2018 LP Duality Lecture 20 April 3, 2018 Some of the
More informationSection Notes 5. Review of Linear Programming. Applied Math / Engineering Sciences 121. Week of October 15, 2017
Section Notes 5 Review of Linear Programming Applied Math / Engineering Sciences 121 Week of October 15, 2017 The following list of topics is an overview of the material that was covered in the lectures
More informationLinear programming and duality theory
Linear programming and duality theory Complements of Operations Research Giovanni Righini Linear Programming (LP) A linear program is defined by linear constraints, a linear objective function. Its variables
More informationAdvanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs
Advanced Operations Research Techniques IE316 Quiz 1 Review Dr. Ted Ralphs IE316 Quiz 1 Review 1 Reading for The Quiz Material covered in detail in lecture. 1.1, 1.4, 2.1-2.6, 3.1-3.3, 3.5 Background material
More information6.854 Advanced Algorithms. Scribes: Jay Kumar Sundararajan. Duality
6.854 Advanced Algorithms Scribes: Jay Kumar Sundararajan Lecturer: David Karger Duality This lecture covers weak and strong duality, and also explains the rules for finding the dual of a linear program,
More informationMath 5593 Linear Programming Lecture Notes
Math 5593 Linear Programming Lecture Notes Unit II: Theory & Foundations (Convex Analysis) University of Colorado Denver, Fall 2013 Topics 1 Convex Sets 1 1.1 Basic Properties (Luenberger-Ye Appendix B.1).........................
More informationMathematical and Algorithmic Foundations Linear Programming and Matchings
Adavnced Algorithms Lectures Mathematical and Algorithmic Foundations Linear Programming and Matchings Paul G. Spirakis Department of Computer Science University of Patras and Liverpool Paul G. Spirakis
More informationORIE 6300 Mathematical Programming I September 2, Lecture 3
ORIE 6300 Mathematical Programming I September 2, 2014 Lecturer: David P. Williamson Lecture 3 Scribe: Divya Singhvi Last time we discussed how to take dual of an LP in two different ways. Today we will
More informationSubmodularity Reading Group. Matroid Polytopes, Polymatroid. M. Pawan Kumar
Submodularity Reading Group Matroid Polytopes, Polymatroid M. Pawan Kumar http://www.robots.ox.ac.uk/~oval/ Outline Linear Programming Matroid Polytopes Polymatroid Polyhedron Ax b A : m x n matrix b:
More informationLecture notes on the simplex method September We will present an algorithm to solve linear programs of the form. maximize.
Cornell University, Fall 2017 CS 6820: Algorithms Lecture notes on the simplex method September 2017 1 The Simplex Method We will present an algorithm to solve linear programs of the form maximize subject
More informationLecture Notes 2: The Simplex Algorithm
Algorithmic Methods 25/10/2010 Lecture Notes 2: The Simplex Algorithm Professor: Yossi Azar Scribe:Kiril Solovey 1 Introduction In this lecture we will present the Simplex algorithm, finish some unresolved
More informationLinear Programming and its Applications
Linear Programming and its Applications Outline for Today What is linear programming (LP)? Examples Formal definition Geometric intuition Why is LP useful? A first look at LP algorithms Duality Linear
More informationOutline. CS38 Introduction to Algorithms. Linear programming 5/21/2014. Linear programming. Lecture 15 May 20, 2014
5/2/24 Outline CS38 Introduction to Algorithms Lecture 5 May 2, 24 Linear programming simplex algorithm LP duality ellipsoid algorithm * slides from Kevin Wayne May 2, 24 CS38 Lecture 5 May 2, 24 CS38
More informationAdvanced Operations Research Techniques IE316. Quiz 2 Review. Dr. Ted Ralphs
Advanced Operations Research Techniques IE316 Quiz 2 Review Dr. Ted Ralphs IE316 Quiz 2 Review 1 Reading for The Quiz Material covered in detail in lecture Bertsimas 4.1-4.5, 4.8, 5.1-5.5, 6.1-6.3 Material
More informationLinear Programming in Small Dimensions
Linear Programming in Small Dimensions Lekcija 7 sergio.cabello@fmf.uni-lj.si FMF Univerza v Ljubljani Edited from slides by Antoine Vigneron Outline linear programming, motivation and definition one dimensional
More informationCollege of Computer & Information Science Fall 2007 Northeastern University 14 September 2007
College of Computer & Information Science Fall 2007 Northeastern University 14 September 2007 CS G399: Algorithmic Power Tools I Scribe: Eric Robinson Lecture Outline: Linear Programming: Vertex Definitions
More informationCS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension
CS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension Antoine Vigneron King Abdullah University of Science and Technology November 7, 2012 Antoine Vigneron (KAUST) CS 372 Lecture
More informationCS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm Instructor: Shaddin Dughmi Algorithms for Convex Optimization We will look at 2 algorithms in detail: Simplex and Ellipsoid.
More informationLinear Programming Duality and Algorithms
COMPSCI 330: Design and Analysis of Algorithms 4/5/2016 and 4/7/2016 Linear Programming Duality and Algorithms Lecturer: Debmalya Panigrahi Scribe: Tianqi Song 1 Overview In this lecture, we will cover
More informationLinear Programming Motivation: The Diet Problem
Agenda We ve done Greedy Method Divide and Conquer Dynamic Programming Network Flows & Applications NP-completeness Now Linear Programming and the Simplex Method Hung Q. Ngo (SUNY at Buffalo) CSE 531 1
More information16.410/413 Principles of Autonomy and Decision Making
16.410/413 Principles of Autonomy and Decision Making Lecture 17: The Simplex Method Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology November 10, 2010 Frazzoli (MIT)
More informationR n a T i x = b i} is a Hyperplane.
Geometry of LPs Consider the following LP : min {c T x a T i x b i The feasible region is i =1,...,m}. X := {x R n a T i x b i i =1,...,m} = m i=1 {x Rn a T i x b i} }{{} X i The set X i is a Half-space.
More informationSome Advanced Topics in Linear Programming
Some Advanced Topics in Linear Programming Matthew J. Saltzman July 2, 995 Connections with Algebra and Geometry In this section, we will explore how some of the ideas in linear programming, duality theory,
More informationLesson 17. Geometry and Algebra of Corner Points
SA305 Linear Programming Spring 2016 Asst. Prof. Nelson Uhan 0 Warm up Lesson 17. Geometry and Algebra of Corner Points Example 1. Consider the system of equations 3 + 7x 3 = 17 + 5 = 1 2 + 11x 3 = 24
More informationThe Simplex Algorithm
The Simplex Algorithm Uri Feige November 2011 1 The simplex algorithm The simplex algorithm was designed by Danzig in 1947. This write-up presents the main ideas involved. It is a slight update (mostly
More information/ Approximation Algorithms Lecturer: Michael Dinitz Topic: Linear Programming Date: 2/24/15 Scribe: Runze Tang
600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Linear Programming Date: 2/24/15 Scribe: Runze Tang 9.1 Linear Programming Suppose we are trying to approximate a minimization
More informationDM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini
DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 3. 4. Standard Form Basic Feasible Solutions
More informationLecture 2 - Introduction to Polytopes
Lecture 2 - Introduction to Polytopes Optimization and Approximation - ENS M1 Nicolas Bousquet 1 Reminder of Linear Algebra definitions Let x 1,..., x m be points in R n and λ 1,..., λ m be real numbers.
More informationmaximize c, x subject to Ax b,
Lecture 8 Linear programming is about problems of the form maximize c, x subject to Ax b, where A R m n, x R n, c R n, and b R m, and the inequality sign means inequality in each row. The feasible set
More informationCS522: Advanced Algorithms
Lecture 1 CS5: Advanced Algorithms October 4, 004 Lecturer: Kamal Jain Notes: Chris Re 1.1 Plan for the week Figure 1.1: Plan for the week The underlined tools, weak duality theorem and complimentary slackness,
More informationIntroduction to Mathematical Programming IE496. Final Review. Dr. Ted Ralphs
Introduction to Mathematical Programming IE496 Final Review Dr. Ted Ralphs IE496 Final Review 1 Course Wrap-up: Chapter 2 In the introduction, we discussed the general framework of mathematical modeling
More informationIntroduction to Mathematical Programming IE406. Lecture 4. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 4 Dr. Ted Ralphs IE406 Lecture 4 1 Reading for This Lecture Bertsimas 2.2-2.4 IE406 Lecture 4 2 The Two Crude Petroleum Example Revisited Recall the
More informationCS675: Convex and Combinatorial Optimization Spring 2018 Consequences of the Ellipsoid Algorithm. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Spring 2018 Consequences of the Ellipsoid Algorithm Instructor: Shaddin Dughmi Outline 1 Recapping the Ellipsoid Method 2 Complexity of Convex Optimization
More informationLP Geometry: outline. A general LP. minimize x c T x s.t. a T i. x b i, i 2 M 1 a T i x = b i, i 2 M 3 x j 0, j 2 N 1. where
LP Geometry: outline I Polyhedra I Extreme points, vertices, basic feasible solutions I Degeneracy I Existence of extreme points I Optimality of extreme points IOE 610: LP II, Fall 2013 Geometry of Linear
More informationEC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 2: Convex Sets
EC 51 MATHEMATICAL METHODS FOR ECONOMICS Lecture : Convex Sets Murat YILMAZ Boğaziçi University In this section, we focus on convex sets, separating hyperplane theorems and Farkas Lemma. And as an application
More informationLECTURE 6: INTERIOR POINT METHOD. 1. Motivation 2. Basic concepts 3. Primal affine scaling algorithm 4. Dual affine scaling algorithm
LECTURE 6: INTERIOR POINT METHOD 1. Motivation 2. Basic concepts 3. Primal affine scaling algorithm 4. Dual affine scaling algorithm Motivation Simplex method works well in general, but suffers from exponential-time
More information3. The Simplex algorithmn The Simplex algorithmn 3.1 Forms of linear programs
11 3.1 Forms of linear programs... 12 3.2 Basic feasible solutions... 13 3.3 The geometry of linear programs... 14 3.4 Local search among basic feasible solutions... 15 3.5 Organization in tableaus...
More informationChapter 4 Concepts from Geometry
Chapter 4 Concepts from Geometry An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Line Segments The line segment between two points and in R n is the set of points on the straight line joining
More informationAdvanced Linear Programming. Organisation. Lecturers: Leen Stougie, CWI and Vrije Universiteit in Amsterdam
Advanced Linear Programming Organisation Lecturers: Leen Stougie, CWI and Vrije Universiteit in Amsterdam E-mail: stougie@cwi.nl Marjan van den Akker Universiteit Utrecht marjan@cs.uu.nl Advanced Linear
More informationLinear Programming: Introduction
CSC 373 - Algorithm Design, Analysis, and Complexity Summer 2016 Lalla Mouatadid Linear Programming: Introduction A bit of a historical background about linear programming, that I stole from Jeff Erickson
More informationMA4254: Discrete Optimization. Defeng Sun. Department of Mathematics National University of Singapore Office: S Telephone:
MA4254: Discrete Optimization Defeng Sun Department of Mathematics National University of Singapore Office: S14-04-25 Telephone: 6516 3343 Aims/Objectives: Discrete optimization deals with problems of
More informationLecture 5: Duality Theory
Lecture 5: Duality Theory Rajat Mittal IIT Kanpur The objective of this lecture note will be to learn duality theory of linear programming. We are planning to answer following questions. What are hyperplane
More informationJ Linear Programming Algorithms
Simplicibus itaque verbis gaudet Mathematica Veritas, cum etiam per se simplex sit Veritatis oratio. [And thus Mathematical Truth prefers simple words, because the language of Truth is itself simple.]
More informationLecture 3. Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets. Tepper School of Business Carnegie Mellon University, Pittsburgh
Lecture 3 Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets Gérard Cornuéjols Tepper School of Business Carnegie Mellon University, Pittsburgh January 2016 Mixed Integer Linear Programming
More informationLecture 5: Properties of convex sets
Lecture 5: Properties of convex sets Rajat Mittal IIT Kanpur This week we will see properties of convex sets. These properties make convex sets special and are the reason why convex optimization problems
More informationCombinatorial Optimization
Combinatorial Optimization Frank de Zeeuw EPFL 2012 Today Introduction Graph problems - What combinatorial things will we be optimizing? Algorithms - What kind of solution are we looking for? Linear Programming
More informationCOMP331/557. Chapter 2: The Geometry of Linear Programming. (Bertsimas & Tsitsiklis, Chapter 2)
COMP331/557 Chapter 2: The Geometry of Linear Programming (Bertsimas & Tsitsiklis, Chapter 2) 49 Polyhedra and Polytopes Definition 2.1. Let A 2 R m n and b 2 R m. a set {x 2 R n A x b} is called polyhedron
More informationPolytopes Course Notes
Polytopes Course Notes Carl W. Lee Department of Mathematics University of Kentucky Lexington, KY 40506 lee@ms.uky.edu Fall 2013 i Contents 1 Polytopes 1 1.1 Convex Combinations and V-Polytopes.....................
More informationDiscrete Optimization 2010 Lecture 5 Min-Cost Flows & Total Unimodularity
Discrete Optimization 2010 Lecture 5 Min-Cost Flows & Total Unimodularity Marc Uetz University of Twente m.uetz@utwente.nl Lecture 5: sheet 1 / 26 Marc Uetz Discrete Optimization Outline 1 Min-Cost Flows
More informationLecture 4: Linear Programming
COMP36111: Advanced Algorithms I Lecture 4: Linear Programming Ian Pratt-Hartmann Room KB2.38: email: ipratt@cs.man.ac.uk 2017 18 Outline The Linear Programming Problem Geometrical analysis The Simplex
More informationLinear and Integer Programming :Algorithms in the Real World. Related Optimization Problems. How important is optimization?
Linear and Integer Programming 15-853:Algorithms in the Real World Linear and Integer Programming I Introduction Geometric Interpretation Simplex Method Linear or Integer programming maximize z = c T x
More informationAn iteration of the simplex method (a pivot )
Recap, and outline of Lecture 13 Previously Developed and justified all the steps in a typical iteration ( pivot ) of the Simplex Method (see next page). Today Simplex Method Initialization Start with
More informationDiscrete Optimization. Lecture Notes 2
Discrete Optimization. Lecture Notes 2 Disjunctive Constraints Defining variables and formulating linear constraints can be straightforward or more sophisticated, depending on the problem structure. The
More informationArtificial Intelligence
Artificial Intelligence Combinatorial Optimization G. Guérard Department of Nouvelles Energies Ecole Supérieur d Ingénieurs Léonard de Vinci Lecture 1 GG A.I. 1/34 Outline 1 Motivation 2 Geometric resolution
More informationPOLYHEDRAL GEOMETRY. Convex functions and sets. Mathematical Programming Niels Lauritzen Recall that a subset C R n is convex if
POLYHEDRAL GEOMETRY Mathematical Programming Niels Lauritzen 7.9.2007 Convex functions and sets Recall that a subset C R n is convex if {λx + (1 λ)y 0 λ 1} C for every x, y C and 0 λ 1. A function f :
More informationNumerical Optimization
Convex Sets Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Let x 1, x 2 R n, x 1 x 2. Line and line segment Line passing through x 1 and x 2 : {y
More informationLinear Programming. Larry Blume. Cornell University & The Santa Fe Institute & IHS
Linear Programming Larry Blume Cornell University & The Santa Fe Institute & IHS Linear Programs The general linear program is a constrained optimization problem where objectives and constraints are all
More informationIn this lecture, we ll look at applications of duality to three problems:
Lecture 7 Duality Applications (Part II) In this lecture, we ll look at applications of duality to three problems: 1. Finding maximum spanning trees (MST). We know that Kruskal s algorithm finds this,
More informationLinear Optimization. Andongwisye John. November 17, Linkoping University. Andongwisye John (Linkoping University) November 17, / 25
Linear Optimization Andongwisye John Linkoping University November 17, 2016 Andongwisye John (Linkoping University) November 17, 2016 1 / 25 Overview 1 Egdes, One-Dimensional Faces, Adjacency of Extreme
More informationCMPSCI611: The Simplex Algorithm Lecture 24
CMPSCI611: The Simplex Algorithm Lecture 24 Let s first review the general situation for linear programming problems. Our problem in standard form is to choose a vector x R n, such that x 0 and Ax = b,
More informationNotes taken by Mea Wang. February 11, 2005
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 5: Smoothed Analysis, Randomized Combinatorial Algorithms, and Linear Programming Duality Notes taken by Mea Wang February 11, 2005 Summary:
More informationCombinatorial Geometry & Topology arising in Game Theory and Optimization
Combinatorial Geometry & Topology arising in Game Theory and Optimization Jesús A. De Loera University of California, Davis LAST EPISODE... We discuss the content of the course... Convex Sets A set is
More informationLecture 6: Faces, Facets
IE 511: Integer Programming, Spring 2019 31 Jan, 2019 Lecturer: Karthik Chandrasekaran Lecture 6: Faces, Facets Scribe: Setareh Taki Disclaimer: These notes have not been subjected to the usual scrutiny
More information11 Linear Programming
11 Linear Programming 11.1 Definition and Importance The final topic in this course is Linear Programming. We say that a problem is an instance of linear programming when it can be effectively expressed
More informationIn this chapter we introduce some of the basic concepts that will be useful for the study of integer programming problems.
2 Basics In this chapter we introduce some of the basic concepts that will be useful for the study of integer programming problems. 2.1 Notation Let A R m n be a matrix with row index set M = {1,...,m}
More informationMATH 890 HOMEWORK 2 DAVID MEREDITH
MATH 890 HOMEWORK 2 DAVID MEREDITH (1) Suppose P and Q are polyhedra. Then P Q is a polyhedron. Moreover if P and Q are polytopes then P Q is a polytope. The facets of P Q are either F Q where F is a facet
More informationLecture 2 September 3
EE 381V: Large Scale Optimization Fall 2012 Lecture 2 September 3 Lecturer: Caramanis & Sanghavi Scribe: Hongbo Si, Qiaoyang Ye 2.1 Overview of the last Lecture The focus of the last lecture was to give
More information1 Linear programming relaxation
Cornell University, Fall 2010 CS 6820: Algorithms Lecture notes: Primal-dual min-cost bipartite matching August 27 30 1 Linear programming relaxation Recall that in the bipartite minimum-cost perfect matching
More informationInteger Programming Theory
Integer Programming Theory Laura Galli October 24, 2016 In the following we assume all functions are linear, hence we often drop the term linear. In discrete optimization, we seek to find a solution x
More information4 LINEAR PROGRAMMING (LP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1
4 LINEAR PROGRAMMING (LP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 Mathematical programming (optimization) problem: min f (x) s.t. x X R n set of feasible solutions with linear objective function
More informationPolar Duality and Farkas Lemma
Lecture 3 Polar Duality and Farkas Lemma October 8th, 2004 Lecturer: Kamal Jain Notes: Daniel Lowd 3.1 Polytope = bounded polyhedron Last lecture, we were attempting to prove the Minkowsky-Weyl Theorem:
More informationMathematical Programming and Research Methods (Part II)
Mathematical Programming and Research Methods (Part II) 4. Convexity and Optimization Massimiliano Pontil (based on previous lecture by Andreas Argyriou) 1 Today s Plan Convex sets and functions Types
More informationAM 121: Intro to Optimization Models and Methods Fall 2017
AM 121: Intro to Optimization Models and Methods Fall 2017 Lecture 10: Dual Simplex Yiling Chen SEAS Lesson Plan Interpret primal simplex in terms of pivots on the corresponding dual tableau Dictionaries
More informationMath 414 Lecture 2 Everyone have a laptop?
Math 44 Lecture 2 Everyone have a laptop? THEOREM. Let v,...,v k be k vectors in an n-dimensional space and A = [v ;...; v k ] v,..., v k independent v,..., v k span the space v,..., v k a basis v,...,
More informationDuality. Primal program P: Maximize n. Dual program D: Minimize m. j=1 c jx j subject to n. j=1. i=1 b iy i subject to m. i=1
Duality Primal program P: Maximize n j=1 c jx j subject to n a ij x j b i, i = 1, 2,..., m j=1 x j 0, j = 1, 2,..., n Dual program D: Minimize m i=1 b iy i subject to m a ij x j c j, j = 1, 2,..., n i=1
More informationLecture 16 October 23, 2014
CS 224: Advanced Algorithms Fall 2014 Prof. Jelani Nelson Lecture 16 October 23, 2014 Scribe: Colin Lu 1 Overview In the last lecture we explored the simplex algorithm for solving linear programs. While
More informationConvexity: an introduction
Convexity: an introduction Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 74 1. Introduction 1. Introduction what is convexity where does it arise main concepts and
More information15.082J and 6.855J. Lagrangian Relaxation 2 Algorithms Application to LPs
15.082J and 6.855J Lagrangian Relaxation 2 Algorithms Application to LPs 1 The Constrained Shortest Path Problem (1,10) 2 (1,1) 4 (2,3) (1,7) 1 (10,3) (1,2) (10,1) (5,7) 3 (12,3) 5 (2,2) 6 Find the shortest
More informationSection Notes 4. Duality, Sensitivity, and the Dual Simplex Algorithm. Applied Math / Engineering Sciences 121. Week of October 8, 2018
Section Notes 4 Duality, Sensitivity, and the Dual Simplex Algorithm Applied Math / Engineering Sciences 121 Week of October 8, 2018 Goals for the week understand the relationship between primal and dual
More informationRecap, and outline of Lecture 18
Recap, and outline of Lecture 18 Previously Applications of duality: Farkas lemma (example of theorems of alternative) A geometric view of duality Degeneracy and multiple solutions: a duality connection
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 3: Linear Programming, Continued Prof. John Gunnar Carlsson September 15, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, 2010
More informationFACES OF CONVEX SETS
FACES OF CONVEX SETS VERA ROSHCHINA Abstract. We remind the basic definitions of faces of convex sets and their basic properties. For more details see the classic references [1, 2] and [4] for polytopes.
More informationLecture 4: Rational IPs, Polyhedron, Decomposition Theorem
IE 5: Integer Programming, Spring 29 24 Jan, 29 Lecture 4: Rational IPs, Polyhedron, Decomposition Theorem Lecturer: Karthik Chandrasekaran Scribe: Setareh Taki Disclaimer: These notes have not been subjected
More informationNew Directions in Linear Programming
New Directions in Linear Programming Robert Vanderbei November 5, 2001 INFORMS Miami Beach NOTE: This is a talk mostly on pedagogy. There will be some new results. It is not a talk on state-of-the-art
More informationAMS : Combinatorial Optimization Homework Problems - Week V
AMS 553.766: Combinatorial Optimization Homework Problems - Week V For the following problems, A R m n will be m n matrices, and b R m. An affine subspace is the set of solutions to a a system of linear
More information15-451/651: Design & Analysis of Algorithms October 11, 2018 Lecture #13: Linear Programming I last changed: October 9, 2018
15-451/651: Design & Analysis of Algorithms October 11, 2018 Lecture #13: Linear Programming I last changed: October 9, 2018 In this lecture, we describe a very general problem called linear programming
More informationIntroduction to Modern Control Systems
Introduction to Modern Control Systems Convex Optimization, Duality and Linear Matrix Inequalities Kostas Margellos University of Oxford AIMS CDT 2016-17 Introduction to Modern Control Systems November
More informationConvex Geometry arising in Optimization
Convex Geometry arising in Optimization Jesús A. De Loera University of California, Davis Berlin Mathematical School Summer 2015 WHAT IS THIS COURSE ABOUT? Combinatorial Convexity and Optimization PLAN
More informationConic Duality. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 1 Conic Duality Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/
More informationChapter 15 Introduction to Linear Programming
Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2015 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of
More informationOPERATIONS RESEARCH. Linear Programming Problem
OPERATIONS RESEARCH Chapter 1 Linear Programming Problem Prof. Bibhas C. Giri Department of Mathematics Jadavpur University Kolkata, India Email: bcgiri.jumath@gmail.com 1.0 Introduction Linear programming
More informationPolyhedral Compilation Foundations
Polyhedral Compilation Foundations Louis-Noël Pouchet pouchet@cse.ohio-state.edu Dept. of Computer Science and Engineering, the Ohio State University Feb 15, 2010 888.11, Class #4 Introduction: Polyhedral
More informationWeek 5. Convex Optimization
Week 5. Convex Optimization Lecturer: Prof. Santosh Vempala Scribe: Xin Wang, Zihao Li Feb. 9 and, 206 Week 5. Convex Optimization. The convex optimization formulation A general optimization problem is
More informationDesign and Analysis of Algorithms (V)
Design and Analysis of Algorithms (V) An Introduction to Linear Programming Guoqiang Li School of Software, Shanghai Jiao Tong University Homework Assignment 2 is announced! (deadline Apr. 10) Linear Programming
More informationA linear program is an optimization problem of the form: minimize subject to
Lec11 Page 1 Lec11p1, ORF363/COS323 This lecture: Linear Programming (LP) Applications of linear programming History of linear programming Geometry of linear programming Geometric and algebraic definition
More informationLecture 9: Linear Programming
Lecture 9: Linear Programming A common optimization problem involves finding the maximum of a linear function of N variables N Z = a i x i i= 1 (the objective function ) where the x i are all non-negative
More informationLinear Programming. Linear Programming. Linear Programming. Example: Profit Maximization (1/4) Iris Hui-Ru Jiang Fall Linear programming
Linear Programming 3 describes a broad class of optimization tasks in which both the optimization criterion and the constraints are linear functions. Linear Programming consists of three parts: A set of
More informationMATH 310 : Degeneracy and Geometry in the Simplex Method
MATH 310 : Degeneracy and Geometry in the Simplex Method Fayadhoi Ibrahima December 11, 2013 1 Introduction This project is exploring a bit deeper the study of the simplex method introduced in 1947 by
More informationGraphs and Network Flows IE411. Lecture 20. Dr. Ted Ralphs
Graphs and Network Flows IE411 Lecture 20 Dr. Ted Ralphs IE411 Lecture 20 1 Network Simplex Algorithm Input: A network G = (N, A), a vector of capacities u Z A, a vector of costs c Z A, and a vector of
More information