Principles of Wireless Sensor Networks. Fast-Lipschitz Optimization

Size: px
Start display at page:

Download "Principles of Wireless Sensor Networks. Fast-Lipschitz Optimization"

Transcription

1 Lecture 5 Stockholm, October 14, 2011 Fast-Lipschitz Optimization Royal Institute of Technology - KTH Stockholm, Sweden carlofi@kth.se Previous lecture Non expansive mappings, agreement, consensus Today F-Lipschitz optimization Protocol stack Application Presentation Session Transport Routing MAC Phy

2 In lecture, you will learn some of the key aspects of Fast Lipschitz optmization When a problem is F-Lipschitz? Why should I care of F-Lipschitz? The theory is illustrated by distributed estimation applications Why F-Lipschitz is useful for distributed estimation? Other applications will be mentioned as well optimization centralized distributed/networked optimization network optimization Optimization over networks going for less and less base-stations but more and more distributed networks Optimization needs to be computed by fast algorithms of low complexity time-varying networks, little time to compute optimal solutions computations often must be distributed E.g., cross layer protocol design, distributed detection, estimation, content distribution, routing,... Parallel and distributed computation Fundamental theory for optimization over networks Drowback over energy-constrained wireless networks: the cost for communication not considered An alternative theory is needed In a number of cases, Fast-Lipschitz optimization

3 Outline Definition of Fast-Lipschitz optimization Computation of the optimal solution Problems in canonical form Examples of application Conclusions Network optimization We have seen that a class of distributed estimation problems needs a fast distributed computation of an optimization Is there a general formulation of optiomization problmes that can be solved quickly and efficiently in a distributed manner? In many cases, Fast-Lipschitz optimization C. Fischione, F-Lipschitz Optimization with Wireless Sensor Networks Applications, IEEE Transactions on Automatic Control, Accepted for Publication, to appear, 2011.

4 The Fast-Lipschitz optimization ex: x could be quality control action, radio power etc some box D nonempty compact set containing other constraints Computation of the solution Centralized optimization Problem solved by a central processor Network of n nodes Distributed optimization Decision variables and constraints are associated to nodes that cooperate to compute the solution in parallel

5 The F-Lipschitz optimization Non-Convex Optimization Convex optimization problems require heavier solvers to solve than F-Lipschitz Convex Optimization F-Lipschitz Optimization Interference Function Optimization Geometric Optimization F-Lipschitz optimization problems can be convex, geometric, quadratic, interference-function,... Pareto Optimal Solution

6 White board notes to the previous slide ExampleofParetoOptimal: 5!!! = 3 1 y!!! = Noneisbetterorworse,since notallelementsarethehighest, henceitisaparetooptimal solution Notation Gradient Norm infinity: sum along a row Norm 1: sum along a column

7 F-Lipschitz qualifying properties to be a F-Lipschitz optimization problem Criteria 1 must always be satisfied while only one of criteria 2-4 needs to be fulfilled Functions may be non-convex White board notes to the previous slide ExamplesofF=Lipschitzqualifyingproperties 1b) 3c) < 1! =!!!!!! < 1 < 1 2a)! = 3a)! = !!! =!!!!!!!!!!!!! = < 1!!!!!! < 1 < a)! = 0 0 < <!!!!!

8 Outline Definition of Fast-Lipschitz optimization Computation of the optimal solution Problems in canonical form Examples of application Conclusions Optimal Solution The Pareto optimal solution is just given by a set of in general nonlinear) equations. Solving a set of equations is much easier than solving an optimization problem by traditional Lagrangian methods.

9 Lagrangian methods Theorem: Consider a feasible F-Lipschitz problem. Then, the KKT conditions are necessary and sufficient. KKT conditions: with F-Lipschitz, we do not need the Lagrangian method to solve the problem scalar x, lambda is a vector of Lagrangian multipliers Lagrangian contractive iteration Lagrangian methods to compute the solution White board notes to the previous slide Lagrangianmethod 1) BuildtheLagrangian 2)!!!,! = 0!! Lagrangianmethods!!!,! = 0

10 Centralized optimization The optimal solution is given by iterative methods to solve systems of non-linear equations e.g., Newton methods) is a matrix to ensure and maximize convergence speed Many other methods are available, e.g., second-oder methods. Distributed optimization the value can not go outside the box D otherwise approx. around the max bound of the box)

11 F-Lipschitz optimization yes Inequality constraints satisfy the equality at the optimum? no Compute the solution by F- Lipschitz methods Compute the solution by Lagrangian methods F-Lipschitz optimization: a class of problems for which all the constraints are active at the optimum Optimum: the solution to the set of equations given by the constraints No Lagrangian methods, which are computationally expensive, particularly on wireless networks Outline Definition of Fast-Lipschitz optimization Computation of the optimal solution Problems in canonical form Examples of application Conclusions

12 Problems in canonical form Canonical form Bertsekas, Non Linear Programming, 2004 F-Lipschitz form possible by doing the following Problems in canonical form

13 Example 1: from canonical to F-Lipschitz The problem is convex, but is also F-Lipschitz: Off-diagonal monotonicity Diagonal dominance The solution is given by the constraints at the equality, trivially Example 2: a hidden F-Lipschitz Non F-Lipschitz Simple variable transformation,, F-Lipschitz

14 An F-Lipschitz Matlab Toolbox M. Leithe, Introducing a Matlab Toolbox for F-Lipschitz optimization, Master Thesis KTH, 2011 Outline Definition of Fast-Lipschitz optimization Computation of the optimal solution Problems in canonical form Examples of application Conclusions

15 Distributed estimation We now study F-Lipschitz optimization for distributed estimation distributed detection distributed radio power control Estimation phenomena phenomena sensors 1 2 N Fusion Center sensors 1 2 N Distributed estimation: no central coordination Centralized estimation: No/little intelligence on nodes A. Speranzon, C. Fischione, K. H. Johansson, A. Sangiovanni-Vincentelli, A Distributed Minimum Variance Estimator for Sensor Networks, IEEE Journal on Selected Areas in Communications, special issue on Control and Communications, Vol. 26, N. 4, pp , May A. Speranzon, C. Fischione, K. H. Johansson, Distributed and Collaborative Estimation over Wireless Sensor Networks, IEEE CDC 2006.

16 Network and signals dt) Nodes perform a noisy measurement of a common time-varying signal Communication subject to space-time varying packet losses Distributed estimator Nodes exchange local measurements and estimates Local estimate Global vector of the estimates Goal: find locally the coefficients and that minimize the variance of the estimation error

17 Estimation coefficients Estimation error Estimation coefficients minimizing the average estimation error, under stability constraints Small Bias Stable estimation error A centralized optimization problem How to distribute the computation of the optimal solution? 1. Cost function and first constraint easy to distribute 2. Second constraint is difficult to distribute How to distribute the second constraint Node j Node i The 1-norm and max-norm are easy to distribute, but give infeasibility

18 How to distribute the second constraint A global constraint is translated into a local one by using some thresholds Distributed estimator By using the thresholds: from centralized to distributed optimization

19 Estimation coefficients How to compute the thresholds? What is the performance of the estimator? How to compute the thresholds? The higher the thresholds the lower the estimation error A Lipschitz optimization problem See second part of the lecture

20 Performance The error variance of the estimator that makes a simple average of the received measurements is an upper bound to the error variance of the proposed distributed estimator. Simulation example Network with 30 nodes randomly deployed. Signal to track: Variance of the additive noise:

21 Simulation Example 2) Laplacian Estimator Instantaneous Average Estimator Proposed Distributed Estimator Packet loss probability Remarks A class of distributed estimators for WSNs Optimal distributed estimator Stability conditions Performance analysis Open issues Model-based estimator Estimator for signal with spatial and temporal correlation

22 Optimal thresholds Optimal solution: the fixed point of a contraction mapping Each node updates asynchronously its threshold after receiving those of neighboring nodes. Distributed binary detection measurements at node i hypothesis thesting with S measurements and threshold x i probability of false alarm probability of misdetection A threshold minimizing the prob. of false alarm maximizes the prob. of misdetection. How to choose optimally the thresholds when nodes exchange opinions?

23 Threshold optimization in distributed detection done in ex. cognitive radios How to solve the problem by parallel and distributed operations among the nodes? The problem is convex Lagrangian methods interior point methods) could be applied Drowback: too many message passing Lagrangian multipliers) among nodes to compute iteratively the optimal solution An alternative method: F-Lipschitz optimization Distributed detection: F-Lipschitz vs 231 Lagrangian methods can be used also in estimation or radio power control 10 nodes network 31 F-Lipschitz Lagrangian methods interior point) 36 5 Number of iterations Number of function evaluations

24 Interference function theory vector of radio powers e interference that the radio power has to overcome Tx Properties of the Type-I) interference function Rx nodes Foschini, Miljanic, A simple distributed autonomous power control algorithm and its convergence, IEEE Trans. Veh. Technol., Example: Radio Power Control with unreliable components Unreliable transceivers introduce intermodulation powers difficult to compensate SINR Tx Outages Rx A difficult optimization problem How to distribute the computation?

25 Power control as an F-Lipschitz problem A change of variables and a redefinition F-Lipschitz qualifying properties are much more general than the interference function ones. More radio power control problems can be solved than traditional ones based on the Interference Function. Conclusions Existing methods for optimization over networks are too expensive Studied the Fast-Lipschitz optimization Application to distributed estimation, and other cases F-Lipschitz optimization is a panacea for many cases, but still there is a lack of a theory for fast parallel and distributed computations

Fast-Lipschitz Optimization

Fast-Lipschitz Optimization Fast-Lipschitz Optimization DREAM Seminar Series University of California at Berkeley September 11, 2012 Carlo Fischione ACCESS Linnaeus Center, Electrical Engineering KTH Royal Institute of Technology

More information

Principles of Wireless Sensor Networks

Principles of Wireless Sensor Networks Principles of Wireless Sensor Networks www.kth.se/student/program-kurser/kurshemsidor/kurshemsidor/control/el2745 Lecture 6 Stockholm, February 6, 2012 Carlo Fischione Royal Institute of Technology - KTH

More information

Principles of Wireless Sensor Networks

Principles of Wireless Sensor Networks Principles of Wireless Sensor Networks https://www.kth.se/social/course/el2745/ Lecture 6 Routing Carlo Fischione Associate Professor of Sensor Networks e-mail:carlofi@kth.se http://www.ee.kth.se/ carlofi/

More information

Principles of Wireless Sensor Networks. Medium Access Control and IEEE

Principles of Wireless Sensor Networks. Medium Access Control and IEEE http://www.ee.kth.se/~carlofi/teaching/pwsn-2011/wsn_course.shtml Lecture 7 Stockholm, November 8, 2011 Medium Access Control and IEEE 802.15.4 Royal Institute of Technology - KTH Stockholm, Sweden e-mail:

More information

Principles of Wireless Sensor Networks. Routing, Zigbee, and RPL

Principles of Wireless Sensor Networks. Routing, Zigbee, and RPL http://www.ee.kth.se/~carlofi/teaching/pwsn-2011/wsn_course.shtml Lecture 8 Stockholm, November 11, 2011 Routing, Zigbee, and RPL Royal Institute of Technology - KTH Stockholm, Sweden e-mail: carlofi@kth.se

More information

Lecture 4 Duality and Decomposition Techniques

Lecture 4 Duality and Decomposition Techniques Lecture 4 Duality and Decomposition Techniques Jie Lu (jielu@kth.se) Richard Combes Alexandre Proutiere Automatic Control, KTH September 19, 2013 Consider the primal problem Lagrange Duality Lagrangian

More information

Principles of Wireless Sensor Networks

Principles of Wireless Sensor Networks Principles of Wireless Sensor Networks https://kth.instructure.com/courses/293 Lecture 1 Introduction to WSNs Carlo Fischione Associate Professor of Sensor Networks e-mail:carlofi@kth.se http://www.ee.kth.se/

More information

Programming, numerics and optimization

Programming, numerics and optimization Programming, numerics and optimization Lecture C-4: Constrained optimization Łukasz Jankowski ljank@ippt.pan.pl Institute of Fundamental Technological Research Room 4.32, Phone +22.8261281 ext. 428 June

More information

Principles of Wireless Sensor Networks

Principles of Wireless Sensor Networks Principles of Wireless Sensor Networks https://www.kth.se/social/course/el2745/ Lecture 1 Introduction to WSNs Carlo Fischione Associate Professor of Sensor Networks e-mail:carlofi@kth.se http://www.ee.kth.se/

More information

Adaptive Filters Algorithms (Part 2)

Adaptive Filters Algorithms (Part 2) Adaptive Filters Algorithms (Part 2) Gerhard Schmidt Christian-Albrechts-Universität zu Kiel Faculty of Engineering Electrical Engineering and Information Technology Digital Signal Processing and System

More information

Decentralized Low-Rank Matrix Completion

Decentralized Low-Rank Matrix Completion Decentralized Low-Rank Matrix Completion Qing Ling 1, Yangyang Xu 2, Wotao Yin 2, Zaiwen Wen 3 1. Department of Automation, University of Science and Technology of China 2. Department of Computational

More information

Lecture 12: Feasible direction methods

Lecture 12: Feasible direction methods Lecture 12 Lecture 12: Feasible direction methods Kin Cheong Sou December 2, 2013 TMA947 Lecture 12 Lecture 12: Feasible direction methods 1 / 1 Feasible-direction methods, I Intro Consider the problem

More information

Module 1 Lecture Notes 2. Optimization Problem and Model Formulation

Module 1 Lecture Notes 2. Optimization Problem and Model Formulation Optimization Methods: Introduction and Basic concepts 1 Module 1 Lecture Notes 2 Optimization Problem and Model Formulation Introduction In the previous lecture we studied the evolution of optimization

More information

EL2745 Principles of Wireless Sensor Networks

EL2745 Principles of Wireless Sensor Networks EL2745 Principles of Wireless Sensor Networks www.kth.se/student/program-kurser/kurshemsidor/kurshemsidor/control/el2745 Lecture 5 Stockholm, February 2, 2012 Carlo Fischione Royal Institute of Technology

More information

Principles of Wireless Sensor Networks

Principles of Wireless Sensor Networks Principles of Wireless Sensor Networks https://www.kth.se/social/course/el2745/ Lecture 5 January 31, 2013 Carlo Fischione Associate Professor of Sensor Networks e-mail: carlofi@kth.se http://www.ee.kth.se/~carlofi/

More information

CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 36

CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 36 CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 36 CS 473: Algorithms, Spring 2018 LP Duality Lecture 20 April 3, 2018 Some of the

More information

Lecture 18: March 23

Lecture 18: March 23 0-725/36-725: Convex Optimization Spring 205 Lecturer: Ryan Tibshirani Lecture 8: March 23 Scribes: James Duyck Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have not

More information

Advanced Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras

Advanced Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Advanced Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Lecture - 35 Quadratic Programming In this lecture, we continue our discussion on

More information

WE consider the gate-sizing problem, that is, the problem

WE consider the gate-sizing problem, that is, the problem 2760 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I: REGULAR PAPERS, VOL 55, NO 9, OCTOBER 2008 An Efficient Method for Large-Scale Gate Sizing Siddharth Joshi and Stephen Boyd, Fellow, IEEE Abstract We consider

More information

Luca Schenato Workshop on cooperative multi agent systems Pisa, 6/12/2007

Luca Schenato Workshop on cooperative multi agent systems Pisa, 6/12/2007 Distributed consensus protocols for clock synchronization in sensor networks Luca Schenato Workshop on cooperative multi agent systems Pisa, 6/12/2007 Outline Motivations Intro to consensus algorithms

More information

David G. Luenberger Yinyu Ye. Linear and Nonlinear. Programming. Fourth Edition. ö Springer

David G. Luenberger Yinyu Ye. Linear and Nonlinear. Programming. Fourth Edition. ö Springer David G. Luenberger Yinyu Ye Linear and Nonlinear Programming Fourth Edition ö Springer Contents 1 Introduction 1 1.1 Optimization 1 1.2 Types of Problems 2 1.3 Size of Problems 5 1.4 Iterative Algorithms

More information

Lecture 2. Topology of Sets in R n. August 27, 2008

Lecture 2. Topology of Sets in R n. August 27, 2008 Lecture 2 Topology of Sets in R n August 27, 2008 Outline Vectors, Matrices, Norms, Convergence Open and Closed Sets Special Sets: Subspace, Affine Set, Cone, Convex Set Special Convex Sets: Hyperplane,

More information

Simulation. Lecture O1 Optimization: Linear Programming. Saeed Bastani April 2016

Simulation. Lecture O1 Optimization: Linear Programming. Saeed Bastani April 2016 Simulation Lecture O Optimization: Linear Programming Saeed Bastani April 06 Outline of the course Linear Programming ( lecture) Integer Programming ( lecture) Heuristics and Metaheursitics (3 lectures)

More information

Computational Methods. Constrained Optimization

Computational Methods. Constrained Optimization Computational Methods Constrained Optimization Manfred Huber 2010 1 Constrained Optimization Unconstrained Optimization finds a minimum of a function under the assumption that the parameters can take on

More information

Contents. I Basics 1. Copyright by SIAM. Unauthorized reproduction of this article is prohibited.

Contents. I Basics 1. Copyright by SIAM. Unauthorized reproduction of this article is prohibited. page v Preface xiii I Basics 1 1 Optimization Models 3 1.1 Introduction... 3 1.2 Optimization: An Informal Introduction... 4 1.3 Linear Equations... 7 1.4 Linear Optimization... 10 Exercises... 12 1.5

More information

Data Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University

Data Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Data Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Search & Optimization Search and Optimization method deals with

More information

Optimization of Design. Lecturer:Dung-An Wang Lecture 8

Optimization of Design. Lecturer:Dung-An Wang Lecture 8 Optimization of Design Lecturer:Dung-An Wang Lecture 8 Lecture outline Reading: Ch8 of text Today s lecture 2 8.1 LINEAR FUNCTIONS Cost Function Constraints 3 8.2 The standard LP problem Only equality

More information

Gate Sizing by Lagrangian Relaxation Revisited

Gate Sizing by Lagrangian Relaxation Revisited Gate Sizing by Lagrangian Relaxation Revisited Jia Wang, Debasish Das, and Hai Zhou Electrical Engineering and Computer Science Northwestern University Evanston, Illinois, United States October 17, 2007

More information

LECTURE NOTES Non-Linear Programming

LECTURE NOTES Non-Linear Programming CEE 6110 David Rosenberg p. 1 Learning Objectives LECTURE NOTES Non-Linear Programming 1. Write out the non-linear model formulation 2. Describe the difficulties of solving a non-linear programming model

More information

Finite Math Linear Programming 1 May / 7

Finite Math Linear Programming 1 May / 7 Linear Programming Finite Math 1 May 2017 Finite Math Linear Programming 1 May 2017 1 / 7 General Description of Linear Programming Finite Math Linear Programming 1 May 2017 2 / 7 General Description of

More information

Distributed Optimization of Continuoustime Multi-agent Networks

Distributed Optimization of Continuoustime Multi-agent Networks University of Maryland, Dec 2016 Distributed Optimization of Continuoustime Multi-agent Networks Yiguang Hong Academy of Mathematics & Systems Science Chinese Academy of Sciences Outline 1. Background

More information

A Two-phase Distributed Training Algorithm for Linear SVM in WSN

A Two-phase Distributed Training Algorithm for Linear SVM in WSN Proceedings of the World Congress on Electrical Engineering and Computer Systems and Science (EECSS 015) Barcelona, Spain July 13-14, 015 Paper o. 30 A wo-phase Distributed raining Algorithm for Linear

More information

Delay-minimal Transmission for Energy Constrained Wireless Communications

Delay-minimal Transmission for Energy Constrained Wireless Communications Delay-minimal Transmission for Energy Constrained Wireless Communications Jing Yang Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland, College Park, M0742 yangjing@umd.edu

More information

PRIMAL-DUAL INTERIOR POINT METHOD FOR LINEAR PROGRAMMING. 1. Introduction

PRIMAL-DUAL INTERIOR POINT METHOD FOR LINEAR PROGRAMMING. 1. Introduction PRIMAL-DUAL INTERIOR POINT METHOD FOR LINEAR PROGRAMMING KELLER VANDEBOGERT AND CHARLES LANNING 1. Introduction Interior point methods are, put simply, a technique of optimization where, given a problem

More information

Optimization. Industrial AI Lab.

Optimization. Industrial AI Lab. Optimization Industrial AI Lab. Optimization An important tool in 1) Engineering problem solving and 2) Decision science People optimize Nature optimizes 2 Optimization People optimize (source: http://nautil.us/blog/to-save-drowning-people-ask-yourself-what-would-light-do)

More information

Introduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras

Introduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Introduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Module 03 Simplex Algorithm Lecture - 03 Tabular form (Minimization) In this

More information

Constrained Optimization COS 323

Constrained Optimization COS 323 Constrained Optimization COS 323 Last time Introduction to optimization objective function, variables, [constraints] 1-dimensional methods Golden section, discussion of error Newton s method Multi-dimensional

More information

Lecture 8 Wireless Sensor Networks: Overview

Lecture 8 Wireless Sensor Networks: Overview Lecture 8 Wireless Sensor Networks: Overview Reading: Wireless Sensor Networks, in Ad Hoc Wireless Networks: Architectures and Protocols, Chapter 12, sections 12.1-12.2. I. Akyildiz, W. Su, Y. Sankarasubramaniam

More information

Wireless Sensor Networks

Wireless Sensor Networks Wireless Sensor Networks c.buratti@unibo.it +39 051 20 93147 Office Hours: Tuesday 3 5 pm @ Main Building, second floor Credits: 6 Ouline 1. WS(A)Ns Introduction 2. Applications 3. Energy Efficiency Section

More information

Theoretical Concepts of Machine Learning

Theoretical Concepts of Machine Learning Theoretical Concepts of Machine Learning Part 2 Institute of Bioinformatics Johannes Kepler University, Linz, Austria Outline 1 Introduction 2 Generalization Error 3 Maximum Likelihood 4 Noise Models 5

More information

Local and Global Minimum

Local and Global Minimum Local and Global Minimum Stationary Point. From elementary calculus, a single variable function has a stationary point at if the derivative vanishes at, i.e., 0. Graphically, the slope of the function

More information

Fundamentals of Integer Programming

Fundamentals of Integer Programming Fundamentals of Integer Programming Di Yuan Department of Information Technology, Uppsala University January 2018 Outline Definition of integer programming Formulating some classical problems with integer

More information

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University September 20 2018 Review Solution for multiple linear regression can be computed in closed form

More information

DM6 Support Vector Machines

DM6 Support Vector Machines DM6 Support Vector Machines Outline Large margin linear classifier Linear separable Nonlinear separable Creating nonlinear classifiers: kernel trick Discussion on SVM Conclusion SVM: LARGE MARGIN LINEAR

More information

Recent Developments in Model-based Derivative-free Optimization

Recent Developments in Model-based Derivative-free Optimization Recent Developments in Model-based Derivative-free Optimization Seppo Pulkkinen April 23, 2010 Introduction Problem definition The problem we are considering is a nonlinear optimization problem with constraints:

More information

Outline. CS38 Introduction to Algorithms. Linear programming 5/21/2014. Linear programming. Lecture 15 May 20, 2014

Outline. CS38 Introduction to Algorithms. Linear programming 5/21/2014. Linear programming. Lecture 15 May 20, 2014 5/2/24 Outline CS38 Introduction to Algorithms Lecture 5 May 2, 24 Linear programming simplex algorithm LP duality ellipsoid algorithm * slides from Kevin Wayne May 2, 24 CS38 Lecture 5 May 2, 24 CS38

More information

Programs. Introduction

Programs. Introduction 16 Interior Point I: Linear Programs Lab Objective: For decades after its invention, the Simplex algorithm was the only competitive method for linear programming. The past 30 years, however, have seen

More information

Machine Learning. Topic 5: Linear Discriminants. Bryan Pardo, EECS 349 Machine Learning, 2013

Machine Learning. Topic 5: Linear Discriminants. Bryan Pardo, EECS 349 Machine Learning, 2013 Machine Learning Topic 5: Linear Discriminants Bryan Pardo, EECS 349 Machine Learning, 2013 Thanks to Mark Cartwright for his extensive contributions to these slides Thanks to Alpaydin, Bishop, and Duda/Hart/Stork

More information

Introduction to Optimization

Introduction to Optimization Introduction to Optimization Second Order Optimization Methods Marc Toussaint U Stuttgart Planned Outline Gradient-based optimization (1st order methods) plain grad., steepest descent, conjugate grad.,

More information

FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu

FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu FMA901F: Machine Learning Lecture 3: Linear Models for Regression Cristian Sminchisescu Machine Learning: Frequentist vs. Bayesian In the frequentist setting, we seek a fixed parameter (vector), with value(s)

More information

LECTURE 13: SOLUTION METHODS FOR CONSTRAINED OPTIMIZATION. 1. Primal approach 2. Penalty and barrier methods 3. Dual approach 4. Primal-dual approach

LECTURE 13: SOLUTION METHODS FOR CONSTRAINED OPTIMIZATION. 1. Primal approach 2. Penalty and barrier methods 3. Dual approach 4. Primal-dual approach LECTURE 13: SOLUTION METHODS FOR CONSTRAINED OPTIMIZATION 1. Primal approach 2. Penalty and barrier methods 3. Dual approach 4. Primal-dual approach Basic approaches I. Primal Approach - Feasible Direction

More information

European Network on New Sensing Technologies for Air Pollution Control and Environmental Sustainability - EuNetAir COST Action TD1105

European Network on New Sensing Technologies for Air Pollution Control and Environmental Sustainability - EuNetAir COST Action TD1105 European Network on New Sensing Technologies for Air Pollution Control and Environmental Sustainability - EuNetAir COST Action TD1105 A Holistic Approach in the Development and Deployment of WSN-based

More information

Multidimensional scaling

Multidimensional scaling Multidimensional scaling Lecture 5 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 Cinderella 2.0 2 If it doesn t fit,

More information

CONLIN & MMA solvers. Pierre DUYSINX LTAS Automotive Engineering Academic year

CONLIN & MMA solvers. Pierre DUYSINX LTAS Automotive Engineering Academic year CONLIN & MMA solvers Pierre DUYSINX LTAS Automotive Engineering Academic year 2018-2019 1 CONLIN METHOD 2 LAY-OUT CONLIN SUBPROBLEMS DUAL METHOD APPROACH FOR CONLIN SUBPROBLEMS SEQUENTIAL QUADRATIC PROGRAMMING

More information

10. Network dimensioning

10. Network dimensioning Partly based on slide material by Samuli Aalto and Jorma Virtamo ELEC-C7210 Modeling and analysis of communication networks 1 Contents Introduction Parameters: topology, routing and traffic Dimensioning

More information

DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini

DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 3. 4. Standard Form Basic Feasible Solutions

More information

Linear Programming Problems

Linear Programming Problems Linear Programming Problems Two common formulations of linear programming (LP) problems are: min Subject to: 1,,, 1,2,,;, max Subject to: 1,,, 1,2,,;, Linear Programming Problems The standard LP problem

More information

Advanced Topics in Digital Communications Spezielle Methoden der digitalen Datenübertragung

Advanced Topics in Digital Communications Spezielle Methoden der digitalen Datenübertragung Advanced Topics in Digital Communications Spezielle Methoden der digitalen Datenübertragung Dr.-Ing. Carsten Bockelmann Institute for Telecommunications and High-Frequency Techniques Department of Communications

More information

Unconstrained Optimization Principles of Unconstrained Optimization Search Methods

Unconstrained Optimization Principles of Unconstrained Optimization Search Methods 1 Nonlinear Programming Types of Nonlinear Programs (NLP) Convexity and Convex Programs NLP Solutions Unconstrained Optimization Principles of Unconstrained Optimization Search Methods Constrained Optimization

More information

Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects

Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects Intelligent Control Systems Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/

More information

An Industrial Employee Development Application Protocol Using Wireless Sensor Networks

An Industrial Employee Development Application Protocol Using Wireless Sensor Networks RESEARCH ARTICLE An Industrial Employee Development Application Protocol Using Wireless Sensor Networks 1 N.Roja Ramani, 2 A.Stenila 1,2 Asst.professor, Dept.of.Computer Application, Annai Vailankanni

More information

Lagrangian methods for the regularization of discrete ill-posed problems. G. Landi

Lagrangian methods for the regularization of discrete ill-posed problems. G. Landi Lagrangian methods for the regularization of discrete ill-posed problems G. Landi Abstract In many science and engineering applications, the discretization of linear illposed problems gives rise to large

More information

Chapter II. Linear Programming

Chapter II. Linear Programming 1 Chapter II Linear Programming 1. Introduction 2. Simplex Method 3. Duality Theory 4. Optimality Conditions 5. Applications (QP & SLP) 6. Sensitivity Analysis 7. Interior Point Methods 1 INTRODUCTION

More information

Multi-area Nonlinear State Estimation using Distributed Semidefinite Programming

Multi-area Nonlinear State Estimation using Distributed Semidefinite Programming October 15, 2012 Multi-area Nonlinear State Estimation using Distributed Semidefinite Programming Hao Zhu haozhu@illinois.edu Acknowledgements: Prof. G. B. Giannakis, U of MN Doctoral Dissertation Fellowship

More information

Degrees of Freedom in Cached Interference Networks with Limited Backhaul

Degrees of Freedom in Cached Interference Networks with Limited Backhaul Degrees of Freedom in Cached Interference Networks with Limited Backhaul Vincent LAU, Department of ECE, Hong Kong University of Science and Technology (A) Motivation Interference Channels 3 No side information

More information

Principles of Wireless Sensor Networks. Lecture 1 Stockholm, September 13, 2011

Principles of Wireless Sensor Networks.   Lecture 1 Stockholm, September 13, 2011 http://www.ee.kth.se/~carlofi/teaching/pwsn-2011/wsn_course.shtml Lecture 1 Stockholm, September 13, 2011 node = sensor (when Carlo talks about them) Royal Institute of Technology - KTH Stockholm, Sweden

More information

STRUCTURAL & MULTIDISCIPLINARY OPTIMIZATION

STRUCTURAL & MULTIDISCIPLINARY OPTIMIZATION STRUCTURAL & MULTIDISCIPLINARY OPTIMIZATION Pierre DUYSINX Patricia TOSSINGS Department of Aerospace and Mechanical Engineering Academic year 2018-2019 1 Course objectives To become familiar with the introduction

More information

THERE are a number of fundamental optimization problems

THERE are a number of fundamental optimization problems IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 15, NO. 6, DECEMBER 2007 1359 Modeling and Optimization of Transmission Schemes in Energy-Constrained Wireless Sensor Networks Ritesh Madan, Member, IEEE, Shuguang

More information

CS570: Introduction to Data Mining

CS570: Introduction to Data Mining CS570: Introduction to Data Mining Classification Advanced Reading: Chapter 8 & 9 Han, Chapters 4 & 5 Tan Anca Doloc-Mihu, Ph.D. Slides courtesy of Li Xiong, Ph.D., 2011 Han, Kamber & Pei. Data Mining.

More information

Lecture 2 September 3

Lecture 2 September 3 EE 381V: Large Scale Optimization Fall 2012 Lecture 2 September 3 Lecturer: Caramanis & Sanghavi Scribe: Hongbo Si, Qiaoyang Ye 2.1 Overview of the last Lecture The focus of the last lecture was to give

More information

Image Processing. Image Features

Image Processing. Image Features Image Processing Image Features Preliminaries 2 What are Image Features? Anything. What they are used for? Some statements about image fragments (patches) recognition Search for similar patches matching

More information

MP-DSM: A Distributed Cross Layer Network Control Protocol

MP-DSM: A Distributed Cross Layer Network Control Protocol MP-DSM: A Distributed Cross Layer Network Control Protocol Daniel C. O Neill, Yan Li, and Stephen Boyd Department of Electrical Engineering Stanford University dconeill, liyan, boyd@stanford.edu Abstract

More information

Introduction to Modern Control Systems

Introduction to Modern Control Systems Introduction to Modern Control Systems Convex Optimization, Duality and Linear Matrix Inequalities Kostas Margellos University of Oxford AIMS CDT 2016-17 Introduction to Modern Control Systems November

More information

Support Vector Machines. James McInerney Adapted from slides by Nakul Verma

Support Vector Machines. James McInerney Adapted from slides by Nakul Verma Support Vector Machines James McInerney Adapted from slides by Nakul Verma Last time Decision boundaries for classification Linear decision boundary (linear classification) The Perceptron algorithm Mistake

More information

Optimum Array Processing

Optimum Array Processing Optimum Array Processing Part IV of Detection, Estimation, and Modulation Theory Harry L. Van Trees WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Preface xix 1 Introduction 1 1.1 Array Processing

More information

Chapter 15 Introduction to Linear Programming

Chapter 15 Introduction to Linear Programming Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2015 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of

More information

LEARNING-BASED ADAPTIVE TRANSMISSION FOR LIMITED FEEDBACK MULTIUSER MIMO-OFDM. Alberto Rico-Alvariño and Robert W. Heath Jr.

LEARNING-BASED ADAPTIVE TRANSMISSION FOR LIMITED FEEDBACK MULTIUSER MIMO-OFDM. Alberto Rico-Alvariño and Robert W. Heath Jr. LEARNING-BASED ADAPTIVE TRANSMISSION FOR LIMITED FEEDBACK MULTIUSER MIMO-OFDM Alberto Rico-Alvariño and Robert W. Heath Jr. 2 Outline Introduction System model Link adaptation Precoding Interference estimation

More information

Chapter 3 Numerical Methods

Chapter 3 Numerical Methods Chapter 3 Numerical Methods Part 1 3.1 Linearization and Optimization of Functions of Vectors 1 Problem Notation 2 Outline 3.1.1 Linearization 3.1.2 Optimization of Objective Functions 3.1.3 Constrained

More information

Nonlinear Programming

Nonlinear Programming Nonlinear Programming SECOND EDITION Dimitri P. Bertsekas Massachusetts Institute of Technology WWW site for book Information and Orders http://world.std.com/~athenasc/index.html Athena Scientific, Belmont,

More information

Convex Optimization MLSS 2015

Convex Optimization MLSS 2015 Convex Optimization MLSS 2015 Constantine Caramanis The University of Texas at Austin The Optimization Problem minimize : f (x) subject to : x X. The Optimization Problem minimize : f (x) subject to :

More information

Part 4. Decomposition Algorithms Dantzig-Wolf Decomposition Algorithm

Part 4. Decomposition Algorithms Dantzig-Wolf Decomposition Algorithm In the name of God Part 4. 4.1. Dantzig-Wolf Decomposition Algorithm Spring 2010 Instructor: Dr. Masoud Yaghini Introduction Introduction Real world linear programs having thousands of rows and columns.

More information

QUANTUM BASED PSO TECHNIQUE FOR IMAGE SEGMENTATION

QUANTUM BASED PSO TECHNIQUE FOR IMAGE SEGMENTATION International Journal of Computer Engineering and Applications, Volume VIII, Issue I, Part I, October 14 QUANTUM BASED PSO TECHNIQUE FOR IMAGE SEGMENTATION Shradha Chawla 1, Vivek Panwar 2 1 Department

More information

PARALLEL OPTIMIZATION

PARALLEL OPTIMIZATION PARALLEL OPTIMIZATION Theory, Algorithms, and Applications YAIR CENSOR Department of Mathematics and Computer Science University of Haifa STAVROS A. ZENIOS Department of Public and Business Administration

More information

Lecture 19: November 5

Lecture 19: November 5 0-725/36-725: Convex Optimization Fall 205 Lecturer: Ryan Tibshirani Lecture 9: November 5 Scribes: Hyun Ah Song Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have not

More information

Simplex Algorithm in 1 Slide

Simplex Algorithm in 1 Slide Administrivia 1 Canonical form: Simplex Algorithm in 1 Slide If we do pivot in A r,s >0, where c s

More information

INTRODUCTION TO LINEAR AND NONLINEAR PROGRAMMING

INTRODUCTION TO LINEAR AND NONLINEAR PROGRAMMING INTRODUCTION TO LINEAR AND NONLINEAR PROGRAMMING DAVID G. LUENBERGER Stanford University TT ADDISON-WESLEY PUBLISHING COMPANY Reading, Massachusetts Menlo Park, California London Don Mills, Ontario CONTENTS

More information

Sequential Coordinate-wise Algorithm for Non-negative Least Squares Problem

Sequential Coordinate-wise Algorithm for Non-negative Least Squares Problem CENTER FOR MACHINE PERCEPTION CZECH TECHNICAL UNIVERSITY Sequential Coordinate-wise Algorithm for Non-negative Least Squares Problem Woring document of the EU project COSPAL IST-004176 Vojtěch Franc, Miro

More information

Performance Analysis of Adaptive Beamforming Algorithms for Smart Antennas

Performance Analysis of Adaptive Beamforming Algorithms for Smart Antennas Available online at www.sciencedirect.com ScienceDirect IERI Procedia 1 (214 ) 131 137 214 International Conference on Future Information Engineering Performance Analysis of Adaptive Beamforming Algorithms

More information

Rules for Identifying the Initial Design Points for Use in the Quick Convergent Inflow Algorithm

Rules for Identifying the Initial Design Points for Use in the Quick Convergent Inflow Algorithm International Journal of Statistics and Probability; Vol. 5, No. 1; 2016 ISSN 1927-7032 E-ISSN 1927-7040 Published by Canadian Center of Science and Education Rules for Identifying the Initial Design for

More information

Simplicial Global Optimization

Simplicial Global Optimization Simplicial Global Optimization Julius Žilinskas Vilnius University, Lithuania September, 7 http://web.vu.lt/mii/j.zilinskas Global optimization Find f = min x A f (x) and x A, f (x ) = f, where A R n.

More information

A Truncated Newton Method in an Augmented Lagrangian Framework for Nonlinear Programming

A Truncated Newton Method in an Augmented Lagrangian Framework for Nonlinear Programming A Truncated Newton Method in an Augmented Lagrangian Framework for Nonlinear Programming Gianni Di Pillo (dipillo@dis.uniroma1.it) Giampaolo Liuzzi (liuzzi@iasi.cnr.it) Stefano Lucidi (lucidi@dis.uniroma1.it)

More information

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 5, MAY

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 5, MAY IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 55, NO 5, MAY 2007 1911 Game Theoretic Cross-Layer Transmission Policies in Multipacket Reception Wireless Networks Minh Hanh Ngo, Student Member, IEEE, and

More information

A Tutorial on Decomposition Methods for Network Utility Maximization Daniel P. Palomar, Member, IEEE, and Mung Chiang, Member, IEEE.

A Tutorial on Decomposition Methods for Network Utility Maximization Daniel P. Palomar, Member, IEEE, and Mung Chiang, Member, IEEE. IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 24, NO. 8, AUGUST 2006 1439 A Tutorial on Decomposition Methods for Network Utility Maximization Daniel P. Palomar, Member, IEEE, and Mung Chiang,

More information

Algebraic Iterative Methods for Computed Tomography

Algebraic Iterative Methods for Computed Tomography Algebraic Iterative Methods for Computed Tomography Per Christian Hansen DTU Compute Department of Applied Mathematics and Computer Science Technical University of Denmark Per Christian Hansen Algebraic

More information

New Methods for Solving Large Scale Linear Programming Problems in the Windows and Linux computer operating systems

New Methods for Solving Large Scale Linear Programming Problems in the Windows and Linux computer operating systems arxiv:1209.4308v1 [math.oc] 19 Sep 2012 New Methods for Solving Large Scale Linear Programming Problems in the Windows and Linux computer operating systems Saeed Ketabchi, Hossein Moosaei, Hossein Sahleh

More information

Kernel Methods & Support Vector Machines

Kernel Methods & Support Vector Machines & Support Vector Machines & Support Vector Machines Arvind Visvanathan CSCE 970 Pattern Recognition 1 & Support Vector Machines Question? Draw a single line to separate two classes? 2 & Support Vector

More information

ME 555: Distributed Optimization

ME 555: Distributed Optimization ME 555: Distributed Optimization Duke University Spring 2015 1 Administrative Course: ME 555: Distributed Optimization (Spring 2015) Instructor: Time: Location: Office hours: Website: Soomin Lee (email:

More information

Ensemble methods in machine learning. Example. Neural networks. Neural networks

Ensemble methods in machine learning. Example. Neural networks. Neural networks Ensemble methods in machine learning Bootstrap aggregating (bagging) train an ensemble of models based on randomly resampled versions of the training set, then take a majority vote Example What if you

More information

Measurement Based Routing Strategies on Overlay Architectures

Measurement Based Routing Strategies on Overlay Architectures Measurement Based Routing Strategies on Overlay Architectures Student: Tuna Güven Faculty: Bobby Bhattacharjee, Richard J. La, and Mark A. Shayman LTS Review February 15 th, 2005 Outline Measurement-Based

More information

Lecture 12: Event based control

Lecture 12: Event based control Lecture 12: Event based Control HYCON EECI Graduate School on Control 2010 15 19 March 2010 Vijay Gupta University of Notre Dame U.S.A. Karl H. Johansson Royal Institute of Technology Sweden Lecture 12:

More information