Convergence of Nonconvex Douglas-Rachford Splitting and Nonconvex ADMM
|
|
- Jonathan Austin
- 5 years ago
- Views:
Transcription
1 Convergence of Nonconvex Douglas-Rachford Splitting and Nonconvex ADMM Shuvomoy Das Gupta Thales Canada 56th Allerton Conference on Communication, Control and Computing,
2 What is this talk about? this talk is about ADMM and Douglas-Rachford splitting for nonconvex problems the alternating direction method of multipliers (ADMM) originally designed to solve convex optimization problem Douglas-Rachford splitting algorithm ADMM is its special case in a convex setup both guaranteed to converge for convex problems 2
3 Motivation nonconvex ADMM (NC-ADMM) has become a popular heuristic to tackle nonconvex problems recently, NC-ADMM heuristic has been applied to [Erseghe, 2014] optimal power flow problem, [Takapoui et al., 2017] mixed integer quadratic optimization, [Iyer et al., 2014] submodular minimization with nonconvex constraints... [Diamond et al., 2018] Python package NCVX (extension of CVXPY) implements ADMM heuristic (NC-ADMM) often produces lower objective values compared with exact solvers within a time limit nonconvex Douglas-Rachford splitting (NC-DRS): analogous nonconvex heuristic based on Douglas-Rachford splitting not much has been done to improve the theoretical understanding of such heuristics
4 Summary of the results NC-DRS attacks the original problem directly optimal solutions can be characterized via the NC-DRS operator if deviation from a convex setup is bounded it will converge or oscillate in a compact connected set NC-ADMM works on a modified dual problem, not the original nonconvex problem not equivalent to NC-DRS, but there is a relationship between them likely to produce a lower objective value 4
5 Problem in consideration minimize a convex cost function with nonconvex constraint set minimize x subject to f (x) x C (OPT) f is closed, proper, and convex C is compact, but not necessarily convex
6 Reformulation through indicator function indicator function of set C: δ C (x) = { 0, if x C, if x / C δ C is closed and proper, but not necessarily convex we can write ( minimizex f (x) subject to x C ) = minimize x f(x) + δ C (x)
7 Proximal operator of f and projection onto C both NC-DRS and NC-ADMM have same subroutines: first prox γf, then Π C and finally proximal operator of f evaluated at point x with parameter γ > 0: ( 1 prox γf (x) = argmin y f(y) + y x 2) 2γ single-valued, continuous projection onto C: prox γδc (x) = Π C (x) = argmin y C ( y x 2 ) there can be multiple projections one such projection is denoted by Π C ( )
8 NC-ADMM and NC-DRS: NC-ADMM: x n+1 = prox γf (y n z n ) y n+1 = Π C (x n+1 + z n ) z n+1 = z n y n+1 + x n+1 NC-DRS: x n+1 = prox γf (z n ) y n+1 = Π C (2x n+1 z n ) z n+1 = z n + y n+1 x n+1 both have same subroutines, but different inputs 8
9 Pretend C is convex f is closed, proper, convex pretend C is convex x n,y n converge to an optimal solution for any initial condition but C is not necessarily convex in our setup convergence conditions are messy 9
10 Why are convergence conditions messy? the convergence conditions are messy because: subdifferential operator of δ C is monotone, but not maximally monotone Π C : is expansive i.e., not nonexpansive the underlying reflection operator is expansive Little bit of review...
11 Monotone and maximally monotone operators T is monotone if for every (x,u),(y,v) grat x y u v 0 T is maximally monotone if gra T is not properly contained by any other monotone operator s graph monotone, but not maximally monotone maximally monotone 11
12 Subdifferential operator for a nonconvex function g: closed, proper, but not necessarily convex g: subdifferential of g is monotone, but not maximally monotone g (x) = {u R n ( y R n ) g (y) g (x) + u y x } no subgradient at
13 Why are convergence conditions messy? our problem: minimize x f(x) + δ C (x) f: maximally monotone δ C : monotone, but not maximally monotone Π C is expansive (not nonexpansive) What is a nonexpansive operator?
14 What is a nonexpansive operator? T : single-valued operator on R n T is nonexpansive on R n if for every x,y we have T (x) T (y) x y prox γf is nonexpansive ) (2prox γf I n is nonexpansive 14
15 Operators that are expansive T is a single-valued operator on R n T is expansive if there exist x,y such that T (x) T (y) > x y ΠC is expansive 15
16 Characterization of minimizers: NC-DRS operator and its reflection T : NC-DRS operator T = Π C ( 2prox γf I n ) + I n prox γf. R: reflection operator of T NC-DRS in compact form: R = 2 T I n z n+1 = T z n = 1 2 ( R + In ) z n ΠC : expansive R: expansive root of all convergence issues 16
17 Characterization of minimizers argmin(f + δ C ) is the set of minimizers of min. x f(x) + δ C (x) prox γf ( fix T ) argmin(f + δc ) underlying assumptions: 1. zer ( f + δ C ) is nonempty 2. fix T is nonempty 3. fix {( 2Π C I n )( 2proxγf I n )} is nonempty
18 Convergence of NC-DRS: setup ε R xy : expansiveness of R at x,y ε R xy = { R(x) R(y) x y, if x y < R(x) R(y) 0, else σ R xy = ε R xy R(x) R(y) + x y
19 Convergence of NC-DRS: conditions (z n ) n N : sequence of vectors generated for some chosen initial point z 0 if the following holds: there exists a z fix T, such that n=0 (σ R znz) 2 is bounded then... above, and z 0 z 2 is finite z 0 z 2 + n=0 B(z;r): compact ball with center z and radius r define r := ( ) 2 σ R zn z
20 Convergence of NC-DRS then one of the following will happen: 1. convergence to a point: the sequence (z n ) n N converges to a point z B(z;r) 2. cluster points form a continuum: the set of cluster points of (z n ) n N forms a nonempty compact connected set in B(z; r) if situation 1 occurs and lim n (σ R znz ) 2 = 0, then x n = prox γf (z n 1 ) converges to an optimal solution
21 Some comments on convergence for convergence total deviation of R from being a nonexpansive operator over the sequence {(z n,z)} n N is bounded depends on the initial point in our case C is not necessarily convex sanity check: pretend C is convex total deviation of R from being a nonexpansive operator over the sequence {(z n,z)} n N is zero our convergence proof coincides with known convergence results for convex setup
22 Constructing NC-ADMM original problem: minimize x C f(x) take dual and apply NC-DRS to the dual resultant algorithm is relaxed NC-ADMM x n+1 = prox γf (y n z n ) y n+1 = Π convc (z n + x n+1 ) z n+1 = z n y n+1 + x n+1 relaxed NC-ADMM solves minimize x convc f(x)
23 Constructing NC-ADMM (continued) relaxed NC-ADMM x n+1 = prox γf (y n z n ) y n+1 = Π convc (z n + x n+1 ) z n+1 = z n y n+1 + x n+1 solves minimize x convc f(x) replace Π convc with Π C to solve minimize x C f(x) resultant algorithm is NC-ADMM: x n+1 = prox γf (y n z n ) y n+1 = Π C (x n+1 + z n ) z n+1 = z n y n+1 + x n+1
24 Constructing NC-ADMM
25 Pretend C is convex pretend C is convex strong duality holds Π convc = Π C Moreau s decomposition can be applied to δ C : prox δc + prox δ = I C n then: no information loss in constructing NC-ADMM from NC-DRS NC-ADMM and NC-DRS are equivalent to each other
26 However... C is nonconvex NC-ADMM and NC-DRS are not equivalent reasons for the loss of equivalency: 1. strict duality gap 2. Π conv C Π C 3. Moreau s decomposition does not hold NC-ADMM in NCVX-CVXPY works on a modified dual, hence produces a lower objective value relationship between minimizers of the original problem and the NC-ADMM operator breaks down
27 What if? is it possible to establish convergence NC-ADMM ignoring NC-DRS? our convergence analysis is established for nonempty, compact, but not necessarily convex constraint sets it is equally applicable to the smaller subclass of problems with convex constraint sets in this smaller subclass of convex problems NC-ADMM and NC-DRS have the same convergence properties 27
28 Summary NC-DRS and NC-ADMM are very similar looking, but very different heuristics the theoretical rationale to use NC-DRS could be stronger than NC-ADMM... 28
29 Limitations strong assumptions in minimizer characterization step size is 1, does not consider adaptive step sizes no numerical experiments in the paper
30 End of talk if you are working on nonconvex problems using ADMM/DRS, please talk to me! Thank you! Questions?
31 Diamond, S, Takapoui, R, & Boyd, S A general system for heuristic minimization of convex functions over non-convex sets. Optimization Methods and Software, 33(1), Erseghe, Tomaso Distributed optimal power flow using ADMM. IEEE transactions on power systems, 29(5), Iyer, Rishabh, Jegelka, Stefanie, & Bilmes, Jeff Monotone Closure of Relaxed Constraints in Submodular Optimization: Connections Between Minimization and Maximization. Pages of: Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence. UAI 14. Arlington, Virginia, United States: AUAI Press. 30
32 Takapoui, Reza, Moehle, Nicholas, Boyd, Stephen, & Bemporad, Alberto A simple effective heuristic for embedded mixed-integer quadratic programming. International Journal of Control,
Convex optimization algorithms for sparse and low-rank representations
Convex optimization algorithms for sparse and low-rank representations Lieven Vandenberghe, Hsiao-Han Chao (UCLA) ECC 2013 Tutorial Session Sparse and low-rank representation methods in control, estimation,
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms - C) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationSparse Optimization Lecture: Proximal Operator/Algorithm and Lagrange Dual
Sparse Optimization Lecture: Proximal Operator/Algorithm and Lagrange Dual Instructor: Wotao Yin July 2013 online discussions on piazza.com Those who complete this lecture will know learn the proximal
More informationConic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding
Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding B. O Donoghue E. Chu N. Parikh S. Boyd Convex Optimization and Beyond, Edinburgh, 11/6/2104 1 Outline Cone programming Homogeneous
More informationLecture 19 Subgradient Methods. November 5, 2008
Subgradient Methods November 5, 2008 Outline Lecture 19 Subgradients and Level Sets Subgradient Method Convergence and Convergence Rate Convex Optimization 1 Subgradients and Level Sets A vector s is a
More informationLagrangian Relaxation: An overview
Discrete Math for Bioinformatics WS 11/12:, by A. Bockmayr/K. Reinert, 22. Januar 2013, 13:27 4001 Lagrangian Relaxation: An overview Sources for this lecture: D. Bertsimas and J. Tsitsiklis: Introduction
More informationThe Alternating Direction Method of Multipliers
The Alternating Direction Method of Multipliers With Adaptive Step Size Selection Peter Sutor, Jr. Project Advisor: Professor Tom Goldstein October 8, 2015 1 / 30 Introduction Presentation Outline 1 Convex
More informationLecture 19: Convex Non-Smooth Optimization. April 2, 2007
: Convex Non-Smooth Optimization April 2, 2007 Outline Lecture 19 Convex non-smooth problems Examples Subgradients and subdifferentials Subgradient properties Operations with subgradients and subdifferentials
More informationParallel and Distributed Sparse Optimization Algorithms
Parallel and Distributed Sparse Optimization Algorithms Part I Ruoyu Li 1 1 Department of Computer Science and Engineering University of Texas at Arlington March 19, 2015 Ruoyu Li (UTA) Parallel and Distributed
More informationA More Efficient Approach to Large Scale Matrix Completion Problems
A More Efficient Approach to Large Scale Matrix Completion Problems Matthew Olson August 25, 2014 Abstract This paper investigates a scalable optimization procedure to the low-rank matrix completion problem
More informationConvex Optimization MLSS 2015
Convex Optimization MLSS 2015 Constantine Caramanis The University of Texas at Austin The Optimization Problem minimize : f (x) subject to : x X. The Optimization Problem minimize : f (x) subject to :
More informationMonotone Closure of Relaxed Constraints in Submodular Optimization: Connections Between Minimization and Maximization: Extended Version
Monotone Closure of Relaxed Constraints in Submodular Optimization: Connections Between Minimization and Maximization: Extended Version Rishabh Iyer Dept. of Electrical Engineering University of Washington
More informationBilinear Programming
Bilinear Programming Artyom G. Nahapetyan Center for Applied Optimization Industrial and Systems Engineering Department University of Florida Gainesville, Florida 32611-6595 Email address: artyom@ufl.edu
More informationCharacterizing Improving Directions Unconstrained Optimization
Final Review IE417 In the Beginning... In the beginning, Weierstrass's theorem said that a continuous function achieves a minimum on a compact set. Using this, we showed that for a convex set S and y not
More informationGeneralized Nash Equilibrium Problem: existence, uniqueness and
Generalized Nash Equilibrium Problem: existence, uniqueness and reformulations Univ. de Perpignan, France CIMPA-UNESCO school, Delhi November 25 - December 6, 2013 Outline of the 7 lectures Generalized
More informationAlternating Direction Method of Multipliers
Alternating Direction Method of Multipliers CS 584: Big Data Analytics Material adapted from Stephen Boyd (https://web.stanford.edu/~boyd/papers/pdf/admm_slides.pdf) & Ryan Tibshirani (http://stat.cmu.edu/~ryantibs/convexopt/lectures/21-dual-meth.pdf)
More informationAspects of Convex, Nonconvex, and Geometric Optimization (Lecture 1) Suvrit Sra Massachusetts Institute of Technology
Aspects of Convex, Nonconvex, and Geometric Optimization (Lecture 1) Suvrit Sra Massachusetts Institute of Technology Hausdorff Institute for Mathematics (HIM) Trimester: Mathematics of Signal Processing
More informationIDENTIFYING ACTIVE MANIFOLDS
Algorithmic Operations Research Vol.2 (2007) 75 82 IDENTIFYING ACTIVE MANIFOLDS W.L. Hare a a Department of Mathematics, Simon Fraser University, Burnaby, BC V5A 1S6, Canada. A.S. Lewis b b School of ORIE,
More informationThe Alternating Direction Method of Multipliers
The Alternating Direction Method of Multipliers Customizable software solver package Peter Sutor, Jr. Project Advisor: Professor Tom Goldstein April 27, 2016 1 / 28 Background The Dual Problem Consider
More informationDISTRIBUTED NETWORK RESOURCE ALLOCATION WITH INTEGER CONSTRAINTS. Yujiao Cheng, Houfeng Huang, Gang Wu, Qing Ling
DISTRIBUTED NETWORK RESOURCE ALLOCATION WITH INTEGER CONSTRAINTS Yuao Cheng, Houfeng Huang, Gang Wu, Qing Ling Department of Automation, University of Science and Technology of China, Hefei, China ABSTRACT
More informationConvex Optimization / Homework 2, due Oct 3
Convex Optimization 0-725/36-725 Homework 2, due Oct 3 Instructions: You must complete Problems 3 and either Problem 4 or Problem 5 (your choice between the two) When you submit the homework, upload a
More informationA general system for heuristic minimization of convex functions over non-convex sets
Optimization Methods & Software, 2017 http://dx.doi.org/10.1080/10556788.2017.1304548 A general system for heuristic minimization of convex functions over non-convex sets S. Diamond,R.TakapouiandS.Boyd
More informationA General System for Heuristic Solution of Convex Problems over Nonconvex Sets
arxiv:1601.07277v1 [math.oc] 27 Jan 2016 A General System for Heuristic Solution of Convex Problems over Nonconvex Sets Steven Diamond Reza Takapoui Stephen Boyd January 28, 2016 Abstract We describe general
More informationA proximal center-based decomposition method for multi-agent convex optimization
Proceedings of the 47th IEEE Conference on Decision and Control Cancun, Mexico, Dec. 9-11, 2008 A proximal center-based decomposition method for multi-agent convex optimization Ion Necoara and Johan A.K.
More informationApplied Lagrange Duality for Constrained Optimization
Applied Lagrange Duality for Constrained Optimization Robert M. Freund February 10, 2004 c 2004 Massachusetts Institute of Technology. 1 1 Overview The Practical Importance of Duality Review of Convexity
More informationStochastic Separable Mixed-Integer Nonlinear Programming via Nonconvex Generalized Benders Decomposition
Stochastic Separable Mixed-Integer Nonlinear Programming via Nonconvex Generalized Benders Decomposition Xiang Li Process Systems Engineering Laboratory Department of Chemical Engineering Massachusetts
More informationOptimality certificates for convex minimization and Helly numbers
Optimality certificates for convex minimization and Helly numbers Amitabh Basu Michele Conforti Gérard Cornuéjols Robert Weismantel Stefan Weltge October 20, 2016 Abstract We consider the problem of minimizing
More informationPenalty Alternating Direction Methods for Mixed- Integer Optimization: A New View on Feasibility Pumps
Penalty Alternating Direction Methods for Mixed- Integer Optimization: A New View on Feasibility Pumps Björn Geißler, Antonio Morsi, Lars Schewe, Martin Schmidt FAU Erlangen-Nürnberg, Discrete Optimization
More informationLecture 4 Duality and Decomposition Techniques
Lecture 4 Duality and Decomposition Techniques Jie Lu (jielu@kth.se) Richard Combes Alexandre Proutiere Automatic Control, KTH September 19, 2013 Consider the primal problem Lagrange Duality Lagrangian
More information1 Linear programming relaxation
Cornell University, Fall 2010 CS 6820: Algorithms Lecture notes: Primal-dual min-cost bipartite matching August 27 30 1 Linear programming relaxation Recall that in the bipartite minimum-cost perfect matching
More informationConvex Optimization Lecture 2
Convex Optimization Lecture 2 Today: Convex Analysis Center-of-mass Algorithm 1 Convex Analysis Convex Sets Definition: A set C R n is convex if for all x, y C and all 0 λ 1, λx + (1 λ)y C Operations that
More informationarxiv: v3 [math.oc] 3 Nov 2016
BILEVEL POLYNOMIAL PROGRAMS AND SEMIDEFINITE RELAXATION METHODS arxiv:1508.06985v3 [math.oc] 3 Nov 2016 JIAWANG NIE, LI WANG, AND JANE J. YE Abstract. A bilevel program is an optimization problem whose
More informationA parallel implementation of the ADMM algorithm for power network control
Aalto University School of Science Tuomas Rintamäki A parallel implementation of the ADMM algorithm for power network control The document can be stored and made available to the public on the open internet
More informationIteratively Re-weighted Least Squares for Sums of Convex Functions
Iteratively Re-weighted Least Squares for Sums of Convex Functions James Burke University of Washington Jiashan Wang LinkedIn Frank Curtis Lehigh University Hao Wang Shanghai Tech University Daiwei He
More informationLecture 2 September 3
EE 381V: Large Scale Optimization Fall 2012 Lecture 2 September 3 Lecturer: Caramanis & Sanghavi Scribe: Hongbo Si, Qiaoyang Ye 2.1 Overview of the last Lecture The focus of the last lecture was to give
More informationA Feasible Region Contraction Algorithm (Frca) for Solving Linear Programming Problems
A Feasible Region Contraction Algorithm (Frca) for Solving Linear Programming Problems E. O. Effanga Department of Mathematics/Statistics and Comp. Science University of Calabar P.M.B. 1115, Calabar, Cross
More informationImproving Dual Bound for Stochastic MILP Models Using Sensitivity Analysis
Improving Dual Bound for Stochastic MILP Models Using Sensitivity Analysis Vijay Gupta Ignacio E. Grossmann Department of Chemical Engineering Carnegie Mellon University, Pittsburgh Bora Tarhan ExxonMobil
More informationPoint-Set Topology II
Point-Set Topology II Charles Staats September 14, 2010 1 More on Quotients Universal Property of Quotients. Let X be a topological space with equivalence relation. Suppose that f : X Y is continuous and
More informationRevisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization. Author: Martin Jaggi Presenter: Zhongxing Peng
Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization Author: Martin Jaggi Presenter: Zhongxing Peng Outline 1. Theoretical Results 2. Applications Outline 1. Theoretical Results 2. Applications
More informationAM 221: Advanced Optimization Spring 2016
AM 221: Advanced Optimization Spring 2016 Prof. Yaron Singer Lecture 2 Wednesday, January 27th 1 Overview In our previous lecture we discussed several applications of optimization, introduced basic terminology,
More information11 Linear Programming
11 Linear Programming 11.1 Definition and Importance The final topic in this course is Linear Programming. We say that a problem is an instance of linear programming when it can be effectively expressed
More informationIntroduction to Optimization
Introduction to Optimization Second Order Optimization Methods Marc Toussaint U Stuttgart Planned Outline Gradient-based optimization (1st order methods) plain grad., steepest descent, conjugate grad.,
More informationCMU-Q Lecture 9: Optimization II: Constrained,Unconstrained Optimization Convex optimization. Teacher: Gianni A. Di Caro
CMU-Q 15-381 Lecture 9: Optimization II: Constrained,Unconstrained Optimization Convex optimization Teacher: Gianni A. Di Caro GLOBAL FUNCTION OPTIMIZATION Find the global maximum of the function f x (and
More informationCompact Sets. James K. Peterson. September 15, Department of Biological Sciences and Department of Mathematical Sciences Clemson University
Compact Sets James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University September 15, 2017 Outline 1 Closed Sets 2 Compactness 3 Homework Closed Sets
More informationOptimality certificates for convex minimization and Helly numbers
Optimality certificates for convex minimization and Helly numbers Amitabh Basu Michele Conforti Gérard Cornuéjols Robert Weismantel Stefan Weltge May 10, 2017 Abstract We consider the problem of minimizing
More informationA Little Point Set Topology
A Little Point Set Topology A topological space is a generalization of a metric space that allows one to talk about limits, convergence, continuity and so on without requiring the concept of a distance
More informationMVE165/MMG630, Applied Optimization Lecture 8 Integer linear programming algorithms. Ann-Brith Strömberg
MVE165/MMG630, Integer linear programming algorithms Ann-Brith Strömberg 2009 04 15 Methods for ILP: Overview (Ch. 14.1) Enumeration Implicit enumeration: Branch and bound Relaxations Decomposition methods:
More informationCONLIN & MMA solvers. Pierre DUYSINX LTAS Automotive Engineering Academic year
CONLIN & MMA solvers Pierre DUYSINX LTAS Automotive Engineering Academic year 2018-2019 1 CONLIN METHOD 2 LAY-OUT CONLIN SUBPROBLEMS DUAL METHOD APPROACH FOR CONLIN SUBPROBLEMS SEQUENTIAL QUADRATIC PROGRAMMING
More informationLaGO - A solver for mixed integer nonlinear programming
LaGO - A solver for mixed integer nonlinear programming Ivo Nowak June 1 2005 Problem formulation MINLP: min f(x, y) s.t. g(x, y) 0 h(x, y) = 0 x [x, x] y [y, y] integer MINLP: - n
More informationOn the Global Solution of Linear Programs with Linear Complementarity Constraints
On the Global Solution of Linear Programs with Linear Complementarity Constraints J. E. Mitchell 1 J. Hu 1 J.-S. Pang 2 K. P. Bennett 1 G. Kunapuli 1 1 Department of Mathematical Sciences RPI, Troy, NY
More informationInteger Programming ISE 418. Lecture 1. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 1 Dr. Ted Ralphs ISE 418 Lecture 1 1 Reading for This Lecture N&W Sections I.1.1-I.1.4 Wolsey Chapter 1 CCZ Chapter 2 ISE 418 Lecture 1 2 Mathematical Optimization Problems
More informationInteger Programming ISE 418. Lecture 7. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 7 Dr. Ted Ralphs ISE 418 Lecture 7 1 Reading for This Lecture Nemhauser and Wolsey Sections II.3.1, II.3.6, II.4.1, II.4.2, II.5.4 Wolsey Chapter 7 CCZ Chapter 1 Constraint
More information2. Convex sets. x 1. x 2. affine set: contains the line through any two distinct points in the set
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationThe Branch-and-Sandwich Algorithm for Mixed-Integer Nonlinear Bilevel Problems
The Branch-and-Sandwich Algorithm for Mixed-Integer Nonlinear Bilevel Problems Polyxeni-M. Kleniati and Claire S. Adjiman MINLP Workshop June 2, 204, CMU Funding Bodies EPSRC & LEVERHULME TRUST OUTLINE
More informationLECTURE 10 LECTURE OUTLINE
We now introduce a new concept with important theoretical and algorithmic implications: polyhedral convexity, extreme points, and related issues. LECTURE 1 LECTURE OUTLINE Polar cones and polar cone theorem
More information15.082J and 6.855J. Lagrangian Relaxation 2 Algorithms Application to LPs
15.082J and 6.855J Lagrangian Relaxation 2 Algorithms Application to LPs 1 The Constrained Shortest Path Problem (1,10) 2 (1,1) 4 (2,3) (1,7) 1 (10,3) (1,2) (10,1) (5,7) 3 (12,3) 5 (2,2) 6 Find the shortest
More informationCONVEX OPTIMIZATION: A SELECTIVE OVERVIEW
1! CONVEX OPTIMIZATION: A SELECTIVE OVERVIEW Dimitri Bertsekas! M.I.T.! Taiwan! May 2010! 2! OUTLINE! Convexity issues in optimization! Common geometrical framework for duality and minimax! Unifying framework
More informationElementary Topology. Note: This problem list was written primarily by Phil Bowers and John Bryant. It has been edited by a few others along the way.
Elementary Topology Note: This problem list was written primarily by Phil Bowers and John Bryant. It has been edited by a few others along the way. Definition. properties: (i) T and X T, A topology on
More informationGeneral properties of staircase and convex dual feasible functions
General properties of staircase and convex dual feasible functions JÜRGEN RIETZ, CLÁUDIO ALVES, J. M. VALÉRIO de CARVALHO Centro de Investigação Algoritmi da Universidade do Minho, Escola de Engenharia
More informationLecture Notes 2: The Simplex Algorithm
Algorithmic Methods 25/10/2010 Lecture Notes 2: The Simplex Algorithm Professor: Yossi Azar Scribe:Kiril Solovey 1 Introduction In this lecture we will present the Simplex algorithm, finish some unresolved
More informationLecture 5: Properties of convex sets
Lecture 5: Properties of convex sets Rajat Mittal IIT Kanpur This week we will see properties of convex sets. These properties make convex sets special and are the reason why convex optimization problems
More informationConvex Analysis and Minimization Algorithms I
Jean-Baptiste Hiriart-Urruty Claude Lemarechal Convex Analysis and Minimization Algorithms I Fundamentals With 113 Figures Springer-Verlag Berlin Heidelberg New York London Paris Tokyo Hong Kong Barcelona
More informationDC Programming: A brief tutorial. Andres Munoz Medina Courant Institute of Mathematical Sciences
DC Programming: A brief tutorial. Andres Munoz Medina Courant Institute of Mathematical Sciences munoz@cims.nyu.edu Difference of Convex Functions Definition: A function exists f is said to be DC if there
More informationTopology problem set Integration workshop 2010
Topology problem set Integration workshop 2010 July 28, 2010 1 Topological spaces and Continuous functions 1.1 If T 1 and T 2 are two topologies on X, show that (X, T 1 T 2 ) is also a topological space.
More informationA Computational Theory of Clustering
A Computational Theory of Clustering Avrim Blum Carnegie Mellon University Based on work joint with Nina Balcan, Anupam Gupta, and Santosh Vempala Point of this talk A new way to theoretically analyze
More informationM3P1/M4P1 (2005) Dr M Ruzhansky Metric and Topological Spaces Summary of the course: definitions, examples, statements.
M3P1/M4P1 (2005) Dr M Ruzhansky Metric and Topological Spaces Summary of the course: definitions, examples, statements. Chapter 1: Metric spaces and convergence. (1.1) Recall the standard distance function
More informationModified Augmented Lagrangian Coordination and Alternating Direction Method of Multipliers with
Modified Augmented Lagrangian Coordination and Alternating Direction Method of Multipliers with Parallelization in Non-hierarchical Analytical Target Cascading Yongsu Jung Department of Mechanical Engineering,
More informationThe Size Robust Multiple Knapsack Problem
MASTER THESIS ICA-3251535 The Size Robust Multiple Knapsack Problem Branch and Price for the Separate and Combined Recovery Decomposition Model Author: D.D. Tönissen, Supervisors: dr. ir. J.M. van den
More informationA Comparison of Mixed-Integer Programming Models for Non-Convex Piecewise Linear Cost Minimization Problems
A Comparison of Mixed-Integer Programming Models for Non-Convex Piecewise Linear Cost Minimization Problems Keely L. Croxton Fisher College of Business The Ohio State University Bernard Gendron Département
More informationTOPOLOGY, DR. BLOCK, FALL 2015, NOTES, PART 3.
TOPOLOGY, DR. BLOCK, FALL 2015, NOTES, PART 3. 301. Definition. Let m be a positive integer, and let X be a set. An m-tuple of elements of X is a function x : {1,..., m} X. We sometimes use x i instead
More information1. Introduction. performance of numerical methods. complexity bounds. structural convex optimization. course goals and topics
1. Introduction EE 546, Univ of Washington, Spring 2016 performance of numerical methods complexity bounds structural convex optimization course goals and topics 1 1 Some course info Welcome to EE 546!
More information2. Convex sets. affine and convex sets. some important examples. operations that preserve convexity. generalized inequalities
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationConvex Sets. CSCI5254: Convex Optimization & Its Applications. subspaces, affine sets, and convex sets. operations that preserve convexity
CSCI5254: Convex Optimization & Its Applications Convex Sets subspaces, affine sets, and convex sets operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationPROJECTION ONTO A POLYHEDRON THAT EXPLOITS SPARSITY
PROJECTION ONTO A POLYHEDRON THAT EXPLOITS SPARSITY WILLIAM W. HAGER AND HONGCHAO ZHANG Abstract. An algorithm is developed for projecting a point onto a polyhedron. The algorithm solves a dual version
More informationAlgorithms for Decision Support. Integer linear programming models
Algorithms for Decision Support Integer linear programming models 1 People with reduced mobility (PRM) require assistance when travelling through the airport http://www.schiphol.nl/travellers/atschiphol/informationforpassengerswithreducedmobility.htm
More informationConvex Optimization. Lijun Zhang Modification of
Convex Optimization Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Modification of http://stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf Outline Introduction Convex Sets & Functions Convex Optimization
More informationLinear Bilevel Programming With Upper Level Constraints Depending on the Lower Level Solution
Linear Bilevel Programming With Upper Level Constraints Depending on the Lower Level Solution Ayalew Getachew Mersha and Stephan Dempe October 17, 2005 Abstract Focus in the paper is on the definition
More informationConvex Optimization and Machine Learning
Convex Optimization and Machine Learning Mengliu Zhao Machine Learning Reading Group School of Computing Science Simon Fraser University March 12, 2014 Mengliu Zhao SFU-MLRG March 12, 2014 1 / 25 Introduction
More informationUnifying and extending hybrid tractable classes of CSPs
Journal of Experimental & Theoretical Artificial Intelligence Vol. 00, No. 00, Month-Month 200x, 1 16 Unifying and extending hybrid tractable classes of CSPs Wady Naanaa Faculty of sciences, University
More informationOpen problems in convex geometry
Open problems in convex geometry 10 March 2017, Monash University Seminar talk Vera Roshchina, RMIT University Based on joint work with Tian Sang (RMIT University), Levent Tunçel (University of Waterloo)
More informationIntroduction to Modern Control Systems
Introduction to Modern Control Systems Convex Optimization, Duality and Linear Matrix Inequalities Kostas Margellos University of Oxford AIMS CDT 2016-17 Introduction to Modern Control Systems November
More informationPoint-Set Topology 1. TOPOLOGICAL SPACES AND CONTINUOUS FUNCTIONS
Point-Set Topology 1. TOPOLOGICAL SPACES AND CONTINUOUS FUNCTIONS Definition 1.1. Let X be a set and T a subset of the power set P(X) of X. Then T is a topology on X if and only if all of the following
More informationDistributed Alternating Direction Method of Multipliers
Distributed Alternating Direction Method of Multipliers Ermin Wei and Asuman Ozdaglar Abstract We consider a network of agents that are cooperatively solving a global unconstrained optimization problem,
More informationLinear & Conic Programming Reformulations of Two-Stage Robust Linear Programs
1 / 34 Linear & Conic Programming Reformulations of Two-Stage Robust Linear Programs Erick Delage CRC in decision making under uncertainty Department of Decision Sciences HEC Montreal (joint work with
More informationBounded subsets of topological vector spaces
Chapter 2 Bounded subsets of topological vector spaces In this chapter we will study the notion of bounded set in any t.v.s. and analyzing some properties which will be useful in the following and especially
More informationA Class of Submodular Functions for Document Summarization
A Class of Submodular Functions for Document Summarization Hui Lin, Jeff Bilmes University of Washington, Seattle Dept. of Electrical Engineering June 20, 2011 Lin and Bilmes Submodular Summarization June
More informationTMA946/MAN280 APPLIED OPTIMIZATION. Exam instructions
Chalmers/GU Mathematics EXAM TMA946/MAN280 APPLIED OPTIMIZATION Date: 03 05 28 Time: House V, morning Aids: Text memory-less calculator Number of questions: 7; passed on one question requires 2 points
More informationLecture 1: Introduction
Lecture 1 1 Linear and Combinatorial Optimization Anders Heyden Centre for Mathematical Sciences Lecture 1: Introduction The course and its goals Basic concepts Optimization Combinatorial optimization
More informationTHREE LECTURES ON BASIC TOPOLOGY. 1. Basic notions.
THREE LECTURES ON BASIC TOPOLOGY PHILIP FOTH 1. Basic notions. Let X be a set. To make a topological space out of X, one must specify a collection T of subsets of X, which are said to be open subsets of
More informationSelected Topics in Column Generation
Selected Topics in Column Generation February 1, 2007 Choosing a solver for the Master Solve in the dual space(kelly s method) by applying a cutting plane algorithm In the bundle method(lemarechal), a
More informationConvex Optimization - Chapter 1-2. Xiangru Lian August 28, 2015
Convex Optimization - Chapter 1-2 Xiangru Lian August 28, 2015 1 Mathematical optimization minimize f 0 (x) s.t. f j (x) 0, j=1,,m, (1) x S x. (x 1,,x n ). optimization variable. f 0. R n R. objective
More informationRandomized Algorithms 2017A - Lecture 10 Metric Embeddings into Random Trees
Randomized Algorithms 2017A - Lecture 10 Metric Embeddings into Random Trees Lior Kamma 1 Introduction Embeddings and Distortion An embedding of a metric space (X, d X ) into a metric space (Y, d Y ) is
More informationConvexity Theory and Gradient Methods
Convexity Theory and Gradient Methods Angelia Nedić angelia@illinois.edu ISE Department and Coordinated Science Laboratory University of Illinois at Urbana-Champaign Outline Convex Functions Optimality
More informationMath 5593 Linear Programming Lecture Notes
Math 5593 Linear Programming Lecture Notes Unit II: Theory & Foundations (Convex Analysis) University of Colorado Denver, Fall 2013 Topics 1 Convex Sets 1 1.1 Basic Properties (Luenberger-Ye Appendix B.1).........................
More informationModule 1 Lecture Notes 2. Optimization Problem and Model Formulation
Optimization Methods: Introduction and Basic concepts 1 Module 1 Lecture Notes 2 Optimization Problem and Model Formulation Introduction In the previous lecture we studied the evolution of optimization
More informationProbabilistic embedding into trees: definitions and applications. Fall 2011 Lecture 4
Probabilistic embedding into trees: definitions and applications. Fall 2011 Lecture 4 Instructor: Mohammad T. Hajiaghayi Scribe: Anshul Sawant September 21, 2011 1 Overview Some problems which are hard
More informationLagrangean Methods bounding through penalty adjustment
Lagrangean Methods bounding through penalty adjustment thst@man.dtu.dk DTU-Management Technical University of Denmark 1 Outline Brief introduction How to perform Lagrangean relaxation Subgradient techniques
More informationTilburg University. Document version: Publisher final version (usually the publisher pdf) Publication date: Link to publication
Tilburg University Multi-stage Adjustable Robust Mixed-Integer Optimization via Iterative Splitting of the Uncertainty set (Revision of CentER Discussion Paper 2014-056) Postek, Krzysztof; den Hertog,
More informationEfficient Iterative LP Decoding of LDPC Codes with Alternating Direction Method of Multipliers
Efficient Iterative LP Decoding of LDPC Codes with Alternating Direction Method of Multipliers Xiaojie Zhang Samsung R&D America, Dallas, Texas 758 Email: eric.zhang@samsung.com Paul H. Siegel University
More informationONLY AVAILABLE IN ELECTRONIC FORM
MANAGEMENT SCIENCE doi 10.1287/mnsc.1070.0812ec pp. ec1 ec7 e-companion ONLY AVAILABLE IN ELECTRONIC FORM informs 2008 INFORMS Electronic Companion Customized Bundle Pricing for Information Goods: A Nonlinear
More information