ALGEBRAIC MODELS: Algorithms, software,
|
|
|
- Regina Miles
- 5 years ago
- Views:
Transcription
1 OPTIMIZATION WITHOUT ALGEBRAIC MODELS: Algorithms, software, and applications Nick Sahinidis Center for Computer-Aided Process Decision-making Department t of fchemical lengineering i Carnegie Mellon University sahinidis@cmu.edu Acknowledgments: Luis Miguel Rios NIH, NSF and DOE/NETL
2 HISTORICAL DEVELOPMENTS IN OPTIMIZATION 300 B.C: Shortest distance from a point to a line (Euclid) 1600s: Leibniz/Newton (Calculus) 1847: Gradient methods (Cauchy) 1875: Minimum free energy principle p (Gibbs) Late 40 s: Linear Optimization Army operations; Linear objective function and constraints Late 50 s: Nonlinear Optimization Chemical process industries; Nonlinear functions 60 s: Integer Optimization Discrete manufacturing; Integer variables to model discrete decisions and economies of scale
3 GLOBAL OPTIMIZATION OF MIXED-INTEGER INTEGER NONLINEAR CONVEXIFICATION OPTIMIZATION PROBLEMS f Cl i l ti i ti l ith id l l Classical optimization algorithms provide a local minimum closest to the starting point used x
4 BRANCH-AND-BOUND Search Tree Partition Discard
5 Convexification Range Reduction Finiteness x* BRANCH-AND-REDUCE
6 BARON SOFTWARE First commercial software to offer deterministic guarantee of global optimality for multi-extremal nonlinear optimization problems Two-pronged approach to technology transfer Commercial» Under the modeling languages GAMS and AIMMS Free» Under the NEOS server for optimization
7 BARON IN APPLICATIONS Development of new Runge-Kutta methods for partial differential equations Ruuth and Spiteri, SIAM J. Numerical Analysis, 2004 Energy policy making Manne and Barreto, Energy Economics, 2004 Model estimation and automatic control Bemporand and Ljung, Automatica, 2004 Agricultural l economics Cabrini et al., Manufacturing and Service Operations Management, 2005 Portfolio optimization for wealth-dependent risk preferences Rios and Sahinidis, idi Annals of Operations Research, 2010
8 THE ALGEBRAIC OPTIMIZATION PARADIGM Algebraic models Require optimization expertise Take a long time to develop Often require restrictive assumptions to increase solvability Practitioners do not have models when Proprietary software required for simulation Optimization is required in an experimental setting
9 MODEL CALIBRATION (Mugunthan and Shoemaker, 2005) Cl Cl Cl C C Cl Tetrachloroethene H H H Cl Cl C C Cl Trichloroethene Cl C C Cl cis-1,2-dichloroethene H H C H C H Ethene H C C H Cl H Vinyl Chloride Parameter estimation problem on top of PDEs Each function evaluation takes 2.5 hours
10 MICROREACTOR OPTIMIZATION McMullen and Jensen (2010)
11 AUTOMATIC TUNING OF OPTIMIZATION SOFTWARE Options Relaxation Reduction BARON Branching Optimize solver performance over A collection of test problems During run-time
12 DERIVATIVE-FREE OPTIMIZATION Optimization of a function for which derivative information is not symbolically available derivative information is not numerically computable Studied in a variety of areas under the terms: Black-box optimization Simulation-based optimization Design of experiments Response surface methods Active learning Optimization without an algebraic model
13 TIMELINE OF INNOVATION IN DERIVATIVE-FREE OPTIMIZATION
14 MOST CITED WORKS
15 NELDER-MEAD ALGORITHM (Nelder and Mead, 1965) x e Expansion x r x 1 x co Reflection x c Contraction outside x ci x 2 x3 Contraction inside
16 PATTERN SEARCH ALGORITHMS (Hooke and Jeeves, 1961; Torczon, 1997)
17 DIRECT ALGORITHM (Jones et al., 1993) start Identify potentially optimal Evaluate and divide Iteration 1 BIG partitions and/or LOW function values are preferable Iteration 2 Iteration 3
18 DERIVATIVE-FREE OPTIMIZATION ALGORITHMS LOCAL SEARCH METHODS Direct local search» Nelder-Mead simplex algorithm» Generalized pattern search and generating search set Based on surrogate models» Trust-region methods GLOBAL SEARCH METHODS Deterministic global search» Lipschitzian-based partitioning» Multilevel coordinate search Stochastic global optimization» Hit-and-run» Simulated annealing» Genetic algorithms» Particle swarm» Implicit filtering Based on surrogate models» Response surface methods» Surrogate management framework» Branch-and-fit
19 LOCAL SEARCH DERIVATIVE-FREE OPTIMIZATION SOFTWARE FMINSEARCH (Nelder-Mead) DAKOTA PATTERN (PPS) HOPSPACK (PPS) SID-PSM (Simplex gradient PPS) NOMAD (MADS) DFO (Trust region, quadratic model) IMFIL (Implicit it Filtering) i BOBYQA (Trust region, quadratic model) NEWUOA (Trust region, quadratic model) GLOBAL SEARCH DAKOTA SOLIS-WETS (Direct) DAKOTA DIRECT (DIRECT) TOMLAB GLBSOLVE (DIRECT) TOMLAB GLCSOLVE (DIRECT) MCS (Multilevel coordinate search) TOMLAB EGO (RSM using Kriging) g) TOMLAB RBF (RSM using RBF) SNOBFIT (Branch and Fit) TOMLAB LGO (LGO algorithm) STOCHASTIC ASA (Simulated annealing) CMA-ES (Evolutionary algorithm) DAKOTA EA (Evolutionary algorithm) GLOBAL (Clustering - Multistart) PSWARM (Particle swarm)
20 SOLVERS COMPARED
21 APPROACH Started seven years ago Collected over 500 NLP benchmarks Algebraic formulations; global solutions known (BARON) Developed unified interface to 25+ solvers Average-case comparisons Based on median objective function value of 10 runs from randomly generated starting points Solver solved problem if solution within 0.01 or 1% of optimal Tested all solvers with default options Communicated results with developers, who Revised software Revised algorithmic options
22 TEST PROBLEM CHARACTERISTICS Over 500 problems
23 QUESTIONS ADDRESSED Which solvers are most likely to find near- global optima? Which solvers are most likely to improve starting gpoints? Does quality drop significantly as problem size increases? Is there a minimal subset of existing solvers that would suffice to solve a large fraction of problems?
24 FRACTION OF PROBLEMS SOLVED: CONVEX SMOOTH
25 FRACTION OF PROBLEMS SOLVED: CONVEX NONSMOOTH OO
26 FRACTION OF PROBLEMS SOLVED: NONCONVEX SMOOTH
27 FRACTION OF PROBLEMS SOLVED: NONCONVEX NONSMOOTH OO
28 FRACTION OF PROBLEMS SOLVED AS A FUNCTION OF PROBLEM SIZE
29 FRACTION OF PROBLEMS SOLVER WAS BEST
30 MINIMAL SET OF SOLVERS MINIMAL SET OF SOLVERS ALL PROBLEMS
31 ERROR BARS
32 ERROR BARS NEWUOA
33 APPLICATION TO TRUE BLACK-BOX MODELS Portfolio optimization using the omega function R. Desai (MS thesis 2010) Pairs trading Y. Zheng (MS thesis 2011) Protein structural alignment S. Shah (PhD thesis 2011) Optimizing polymerase chain reaction (PCR) K.-F. Chang (MS thesis 2011) Relative solver performance on black-box Relative solver performance on black box models is similar to that presented for 500+ algebraic models
34 PAIRS TRADING
35 TRAINING AND TRADING
36 OPTIMAL TRADING OPTIMAL TRADING THRESHOLDS
37 CUMULATIVE RETURNS (9/81-4/97) Annual return 19.5% versus 11.8% for S&P 500
38 NEW ALGORITHMS Simulation/experimentation is expensive Solve auxiliary algebraic models to global optimality to expedite search Decide where to sample objective function» Guarantee geometry Construct surrogate models» Higher-quality surrogates Solve trust-region subproblems» Escape local minima; guarantee convergence BARON is highly efficient for problems below 100 variables
39 MODEL-AND-SEARCH LOCAL ALGORITHM Π x 3 x 1 x 5 x 2 r 1 r r 2 1 r 2 r 1 r 2 x 4 Collect points Scale and shift Add points: n around current origin to current linearly iteratet iteratet independent d points Check for positive Build and optimize i Evaluate points in basis. Add points if interpolating model specific order necessary
40 BRANCH-AND-MODEL AND GLOBAL ALGORITHM Partition the space into a collection H of hypercubes Reduce H to a potentially optimal set O For hypercubes in O, FIT models Sort O by size of hypercubes. Evaluate n 2 points Sort O by predicted optimal values. Evaluate n 1 points Optimize models in O
41 PROTEIN-LIGAND DOCKING Identify binding site and pose Conformation must minimize binding free energy Docking gpackages AutoDock, Gold, FlexX Most rely on genetic and other stochastic search algorithms
42 BINDING ENERGIES ASP PARTIC METALL LOPROT TEASES MISC CELLANE EOUS SERINE B&M outperformed AutoDock in 11 out of 12 cases, and found the best solution amongst all solvers for 3 complexes
43 CONCLUSIONS Deterministic i ti solvers perform better than stochastic solvers Commercial TOMLAB solvers Free MCS/SNOBFIT solvers Many opportunities New algorithms and theory are needed Applications abound Systematic treatment of noise
44 Noise Inputs BLACK BOX SYSTEM Outputs Noise
Surrogate models and the optimization of systems in the absence of algebraic models
Surrogate models and the optimization of systems in the absence of algebraic models Nick Sahinidis Acknowledgments: Alison Cozad, David Miller, Zach Wilson Atharv Bhosekar, Luis Miguel Rios, Hua Zheng
GLOBAL OPTIMIZATION WITH BRANCH-AND-REDUCE
GLOBAL OPTIMIZATION WITH BRANCH-AND-REDUCE Nick Sahinidis Department of Chemical Engineering Carnegie Mellon University sahinidis@cmu.edu EWO seminar, 23 October 27 THE MULTIPLE-MINIMA DIFFICULTY IN OPTIMIZATION
A Personal View on Global Optimisation
A Personal View on Global Optimisation Hermann Schichl March 20, 2014 2nd UK Workshop on Optimisation in Space Engineering Turing Gateway to Mathematics, Cambridge c Universität Wien. All rights reserved.
Parallel Direct Search in Structural Optimization
Parallel Direct Search in Structural Optimization J.B. Cardoso 1, P.G. Coelho 1 and A.L. Custódio 2, 1 Department of Mechanical and Industrial Engineering, New University of Lisbon, Portugal, 2 CMA and
Stochastic Separable Mixed-Integer Nonlinear Programming via Nonconvex Generalized Benders Decomposition
Stochastic Separable Mixed-Integer Nonlinear Programming via Nonconvex Generalized Benders Decomposition Xiang Li Process Systems Engineering Laboratory Department of Chemical Engineering Massachusetts
The ALAMO approach to machine learning: Best subset selection, adaptive sampling, and constrained regression
The ALAMO approach to machine learning: Best subset selection, adaptive sampling, and constrained regression Nick Sahinidis Acknowledgments: Alison Cozad, David Miller, Zach Wilson MACHINE LEARNING PROBLEM
VXQR: Derivative-free unconstrained optimization based on QR factorizations
VXQR: Derivative-free unconstrained optimization based on QR factorizations Arnold Neumaier, Hannes Fendl, Harald Schilly, Thomas Leitner Fakultät für Mathematik, Universität Wien Nordbergstr. 15, A-1090
Simplex of Nelder & Mead Algorithm
Simplex of N & M Simplex of Nelder & Mead Algorithm AKA the Amoeba algorithm In the class of direct search methods Unconstrained (although constraints can be added as part of error function) nonlinear
LECTURE NOTES Non-Linear Programming
CEE 6110 David Rosenberg p. 1 Learning Objectives LECTURE NOTES Non-Linear Programming 1. Write out the non-linear model formulation 2. Describe the difficulties of solving a non-linear programming model
CHAPTER 2 CONVENTIONAL AND NON-CONVENTIONAL TECHNIQUES TO SOLVE ORPD PROBLEM
20 CHAPTER 2 CONVENTIONAL AND NON-CONVENTIONAL TECHNIQUES TO SOLVE ORPD PROBLEM 2.1 CLASSIFICATION OF CONVENTIONAL TECHNIQUES Classical optimization methods can be classified into two distinct groups:
Reduced Order Models for Oxycombustion Boiler Optimization
Reduced Order Models for Oxycombustion Boiler Optimization John Eason Lorenz T. Biegler 9 March 2014 Project Objective Develop an equation oriented framework to optimize a coal oxycombustion flowsheet.
An Evolutionary Algorithm for Minimizing Multimodal Functions
An Evolutionary Algorithm for Minimizing Multimodal Functions D.G. Sotiropoulos, V.P. Plagianakos and M.N. Vrahatis University of Patras, Department of Mamatics, Division of Computational Mamatics & Informatics,
Module 1 Lecture Notes 2. Optimization Problem and Model Formulation
Optimization Methods: Introduction and Basic concepts 1 Module 1 Lecture Notes 2 Optimization Problem and Model Formulation Introduction In the previous lecture we studied the evolution of optimization
Delaunay-based Derivative-free Optimization via Global Surrogate. Pooriya Beyhaghi, Daniele Cavaglieri and Thomas Bewley
Delaunay-based Derivative-free Optimization via Global Surrogate Pooriya Beyhaghi, Daniele Cavaglieri and Thomas Bewley May 23, 2014 Delaunay-based Derivative-free Optimization via Global Surrogate Pooriya
Introduction to unconstrained optimization - derivative-free methods
Introduction to unconstrained optimization - derivative-free methods Jussi Hakanen Post-doctoral researcher Office: AgC426.3 jussi.hakanen@jyu.fi Learning outcomes To understand the basic principles of
Cover Page. The handle holds various files of this Leiden University dissertation.
Cover Page The handle http://hdl.handle.net/1887/22055 holds various files of this Leiden University dissertation. Author: Koch, Patrick Title: Efficient tuning in supervised machine learning Issue Date:
Machine Learning for Software Engineering
Machine Learning for Software Engineering Introduction and Motivation Prof. Dr.-Ing. Norbert Siegmund Intelligent Software Systems 1 2 Organizational Stuff Lectures: Tuesday 11:00 12:30 in room SR015 Cover
The Mesh Adaptive Direct Search Algorithm for Discrete Blackbox Optimization
The Mesh Adaptive Direct Search Algorithm for Discrete Blackbox Optimization Sébastien Le Digabel Charles Audet Christophe Tribes GERAD and École Polytechnique de Montréal ICCOPT, Tokyo 2016 08 08 BBO:
OPTIMIZATION METHODS
D. Nagesh Kumar Associate Professor Department of Civil Engineering, Indian Institute of Science, Bangalore - 50 0 Email : nagesh@civil.iisc.ernet.in URL: http://www.civil.iisc.ernet.in/~nagesh Brief Contents
Recent Developments in Model-based Derivative-free Optimization
Recent Developments in Model-based Derivative-free Optimization Seppo Pulkkinen April 23, 2010 Introduction Problem definition The problem we are considering is a nonlinear optimization problem with constraints:
Contents. I Basics 1. Copyright by SIAM. Unauthorized reproduction of this article is prohibited.
page v Preface xiii I Basics 1 1 Optimization Models 3 1.1 Introduction... 3 1.2 Optimization: An Informal Introduction... 4 1.3 Linear Equations... 7 1.4 Linear Optimization... 10 Exercises... 12 1.5
Introduction to Stochastic Optimization Methods (meta-heuristics) Modern optimization methods 1
Introduction to Stochastic Optimization Methods (meta-heuristics) Modern optimization methods 1 Efficiency of optimization methods Robust method Efficiency Specialized method Enumeration or MC kombinatorial
Embedding Formulations, Complexity and Representability for Unions of Convex Sets
, Complexity and Representability for Unions of Convex Sets Juan Pablo Vielma Massachusetts Institute of Technology CMO-BIRS Workshop: Modern Techniques in Discrete Optimization: Mathematics, Algorithms
Simplicial Global Optimization
Simplicial Global Optimization Julius Žilinskas Vilnius University, Lithuania September, 7 http://web.vu.lt/mii/j.zilinskas Global optimization Find f = min x A f (x) and x A, f (x ) = f, where A R n.
An evolutionary annealing-simplex algorithm for global optimisation of water resource systems
FIFTH INTERNATIONAL CONFERENCE ON HYDROINFORMATICS 1-5 July 2002, Cardiff, UK C05 - Evolutionary algorithms in hydroinformatics An evolutionary annealing-simplex algorithm for global optimisation of water
Introduction to Linear Programming. Algorithmic and Geometric Foundations of Optimization
Introduction to Linear Programming Algorithmic and Geometric Foundations of Optimization Optimization and Linear Programming Mathematical programming is a class of methods for solving problems which ask
Optimization in MATLAB Seth DeLand
Optimization in MATLAB Seth DeLand 4 The MathWorks, Inc. Topics Intro Using gradient-based solvers Optimization in Comp. Finance toolboxes Global optimization Speeding up your optimizations Optimization
Development of optimization methods for Volvo fuel economy simulation. Master s thesis in Signal and Systems RAFAEL KLÜPPEL SMIJTINK
Development of optimization methods for Volvo fuel economy simulation. Master s thesis in Signal and Systems RAFAEL KLÜPPEL SMIJTINK 1 Development of optimization methods for Volvo fuel economy simulation.
B553 Lecture 12: Global Optimization
B553 Lecture 12: Global Optimization Kris Hauser February 20, 2012 Most of the techniques we have examined in prior lectures only deal with local optimization, so that we can only guarantee convergence
A Nonlinear Presolve Algorithm in AIMMS
A Nonlinear Presolve Algorithm in AIMMS By Marcel Hunting marcel.hunting@aimms.com November 2011 This paper describes the AIMMS presolve algorithm for nonlinear problems. This presolve algorithm uses standard
March 19, Heuristics for Optimization. Outline. Problem formulation. Genetic algorithms
Olga Galinina olga.galinina@tut.fi ELT-53656 Network Analysis and Dimensioning II Department of Electronics and Communications Engineering Tampere University of Technology, Tampere, Finland March 19, 2014
Global Solution of Mixed-Integer Dynamic Optimization Problems
European Symposium on Computer Arded Aided Process Engineering 15 L. Puigjaner and A. Espuña (Editors) 25 Elsevier Science B.V. All rights reserved. Global Solution of Mixed-Integer Dynamic Optimization
MATLAB Based Optimization Techniques and Parallel Computing
MATLAB Based Optimization Techniques and Parallel Computing Bratislava June 4, 2009 2009 The MathWorks, Inc. Jörg-M. Sautter Application Engineer The MathWorks Agenda Introduction Local and Smooth Optimization
Solving Optimization Problems with MATLAB Loren Shure
Solving Optimization Problems with MATLAB Loren Shure 6 The MathWorks, Inc. Topics Introduction Least-squares minimization Nonlinear optimization Mied-integer programming Global optimization Optimization
OPTIMIZATION. joint course with. Ottimizzazione Discreta and Complementi di R.O. Edoardo Amaldi. DEIB Politecnico di Milano
OPTIMIZATION joint course with Ottimizzazione Discreta and Complementi di R.O. Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-16-17.shtml
Derivative Free Optimization Methods: A Brief, Opinionated, and Incomplete Look at a Few Recent Developments
Derivative Free Optimization Methods: A Brief, Opinionated, and Incomplete Look at a Few Recent Developments Margaret H. Wright Computer Science Department Courant Institute of Mathematical Sciences New
Workshop on Global Optimization: Methods and Applications
Workshop on Global Optimization: Methods and Applications May 11-12, 2007 Fields Institute, Toronto Organizers Thomas Coleman (University of Waterloo) Panos Pardalos (University of Florida) Stephen Vavasis
The AIMMS Outer Approximation Algorithm for MINLP
The AIMMS Outer Approximation Algorithm for MINLP (using GMP functionality) By Marcel Hunting marcel.hunting@aimms.com November 2011 This document describes how to use the GMP variant of the AIMMS Outer
25. NLP algorithms. ˆ Overview. ˆ Local methods. ˆ Constrained optimization. ˆ Global methods. ˆ Black-box methods.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2017 18 25. NLP algorithms ˆ Overview ˆ Local methods ˆ Constrained optimization ˆ Global methods ˆ Black-box methods ˆ Course wrap-up Laurent Lessard
MINLP applications, part II: Water Network Design and some applications of black-box optimization
MINLP applications, part II: Water Network Design and some applications of black-box optimization Claudia D Ambrosio CNRS & LIX, École Polytechnique dambrosio@lix.polytechnique.fr 5th Porto Meeting on
Metaheuristic Development Methodology. Fall 2009 Instructor: Dr. Masoud Yaghini
Metaheuristic Development Methodology Fall 2009 Instructor: Dr. Masoud Yaghini Phases and Steps Phases and Steps Phase 1: Understanding Problem Step 1: State the Problem Step 2: Review of Existing Solution
DETERMINISTIC OPERATIONS RESEARCH
DETERMINISTIC OPERATIONS RESEARCH Models and Methods in Optimization Linear DAVID J. RADER, JR. Rose-Hulman Institute of Technology Department of Mathematics Terre Haute, IN WILEY A JOHN WILEY & SONS,
Model Parameter Estimation
Model Parameter Estimation Shan He School for Computational Science University of Birmingham Module 06-23836: Computational Modelling with MATLAB Outline Outline of Topics Concepts about model parameter
IE598 Big Data Optimization Summary Nonconvex Optimization
IE598 Big Data Optimization Summary Nonconvex Optimization Instructor: Niao He April 16, 2018 1 This Course Big Data Optimization Explore modern optimization theories, algorithms, and big data applications
Index. affine dependency, 133 minimal, 133 affine hull, 392 affinely independent, 393 α-bb, 230, 258, 297 approximate solutions, 9 approximation, 86
Index affine dependency, 133 minimal, 133 affine hull, 392 affinely independent, 393 α-bb, 230, 258, 297 approximate solutions, 9 approximation, 86 AP X, 10 aspiration level, 86, 110 atomic clusters, 372
OPTIMIZATION METHODS. For more information visit: or send an to:
OPTIMIZATION METHODS modefrontier is a registered product of ESTECO srl Copyright ESTECO srl 1999-2007 For more information visit: www.esteco.com or send an e-mail to: modefrontier@esteco.com NEOS Optimization
DERIVATIVE-FREE OPTIMIZATION ENHANCED-SURROGATE MODEL DEVELOPMENT FOR OPTIMIZATION. Alison Cozad, Nick Sahinidis, David Miller
DERIVATIVE-FREE OPTIMIZATION ENHANCED-SURROGATE MODEL DEVELOPMENT FOR OPTIMIZATION Alison Cozad, Nick Sahinidis, David Miller Carbon Capture Challenge The traditional pathway from discovery to commercialization
Fast oriented bounding box optimization on the rotation group SO(3, R)
Fast oriented bounding box optimization on the rotation group SO(3, R) Chia-Tche Chang 1, Bastien Gorissen 2,3 and Samuel Melchior 1,2 chia-tche.chang@uclouvain.be bastien.gorissen@cenaero.be samuel.melchior@uclouvain.be
HAM: A HYBRID ALGORITHM PREDICTION BASED MODEL FOR ELECTRICITY DEMAND
HAM: A HYBRID ALGORITHM PREDICTION BASED MODEL FOR ELECTRICITY DEMAND 1 WAHAB MUSA, 2 SALMAWATY TANSA 1 Assoc. Prof., Department of Electrical Engineering, UNIVERSITAS NEGERI GORONTALO 2 Lektor., Department
Design of a control system model in SimulationX using calibration and optimization. Dynardo GmbH
Design of a control system model in SimulationX using calibration and optimization Dynardo GmbH 1 Notes Please let your microphone muted Use the chat window to ask questions During short breaks we will
M. Sc. (Artificial Intelligence and Machine Learning)
Course Name: Advanced Python Course Code: MSCAI 122 This course will introduce students to advanced python implementations and the latest Machine Learning and Deep learning libraries, Scikit-Learn and
The AIMMS Outer Approximation Algorithm for MINLP
The AIMMS Outer Approximation Algorithm for MINLP (using GMP functionality) By Marcel Hunting Paragon Decision Technology BV An AIMMS White Paper November, 2011 Abstract This document describes how to
Operations Research and Optimization: A Primer
Operations Research and Optimization: A Primer Ron Rardin, PhD NSF Program Director, Operations Research and Service Enterprise Engineering also Professor of Industrial Engineering, Purdue University Introduction
Theoretical Concepts of Machine Learning
Theoretical Concepts of Machine Learning Part 2 Institute of Bioinformatics Johannes Kepler University, Linz, Austria Outline 1 Introduction 2 Generalization Error 3 Maximum Likelihood 4 Noise Models 5
Methodologies and Software for Derivative-free Optimization
Methodologies and Software for Derivative-free Optimization A. L. Custódio 1 K. Scheinberg 2 L. N. Vicente 3 March 14, 2017 1 Department of Mathematics, FCT-UNL-CMA, Quinta da Torre, 2829-516 Caparica,
ALAMO: Automatic Learning of Algebraic Models for Optimization
ALAMO: Automatic Learning of Algebraic Models for Optimization Alison Cozad 1,2, Nick Sahinidis 1,2, David Miller 2 1 National Energy Technology Laboratory, Pittsburgh, PA,USA 2 Department of Chemical
Handling Multi Objectives of with Multi Objective Dynamic Particle Swarm Optimization
Handling Multi Objectives of with Multi Objective Dynamic Particle Swarm Optimization Richa Agnihotri #1, Dr. Shikha Agrawal #1, Dr. Rajeev Pandey #1 # Department of Computer Science Engineering, UIT,
An Initialization Strategy for High-Dimensional Surrogate-Based Expensive Black-Box Optimization
An Initialization Strategy for High-Dimensional Surrogate-Based Expensive Black-Box Optimization Rommel G. Regis Citation: Regis, R.G. 213. An Initialization Strategy for High-Dimensional Surrogate-Based
Convex Optimization. Lijun Zhang Modification of
Convex Optimization Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Modification of http://stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf Outline Introduction Convex Sets & Functions Convex Optimization
Today. Golden section, discussion of error Newton s method. Newton s method, steepest descent, conjugate gradient
Optimization Last time Root finding: definition, motivation Algorithms: Bisection, false position, secant, Newton-Raphson Convergence & tradeoffs Example applications of Newton s method Root finding in
Blackbox Optimization
Blackbox Optimization Algorithm and Applications Sébastien Le Digabel Charles Audet Christophe Tribes GERAD and École Polytechnique de Montréal Workshop on Nonlinear Optimization Algorithms and Industrial
Data Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Search & Optimization Search and Optimization method deals with
Bowie State University
Bowie State University Department of Mathematics Master of Science in Applied and Computational Mathematics Certificate in Applied and Computational Mathematics Department of Mathematics Crawford Science
Classification of Optimization Problems and the Place of Calculus of Variations in it
Lecture 1 Classification of Optimization Problems and the Place of Calculus of Variations in it ME256 Indian Institute of Science G. K. Ananthasuresh Professor, Mechanical Engineering, Indian Institute
LocalSolver A New Kind of Math Programming Solver
LocalSolver A New Kind of Math Programming Solver Thierry Benoist Julien Darlay Bertrand Estellon Frédéric Gardi Romain Megel jdarlay@localsolver.com www.localsolver.com 1/18 Who are we? Bouygues, one
The Branch-and-Sandwich Algorithm for Mixed-Integer Nonlinear Bilevel Problems
The Branch-and-Sandwich Algorithm for Mixed-Integer Nonlinear Bilevel Problems Polyxeni-M. Kleniati and Claire S. Adjiman MINLP Workshop June 2, 204, CMU Funding Bodies EPSRC & LEVERHULME TRUST OUTLINE
A Brief Overview of Optimization Problems. Steven G. Johnson MIT course , Fall 2008
A Brief Overview of Optimization Problems Steven G. Johnson MIT course 18.335, Fall 2008 Why optimization? In some sense, all engineering design is optimization: choosing design parameters to improve some
APPLICATION OF PATTERN SEARCH METHOD TO POWER SYSTEM ECONOMIC LOAD DISPATCH
APPLICATION OF PATTERN SEARCH METHOD TO POWER SYSTEM ECONOMIC LOAD DISPATCH J S Alsumait, J K Sykulski A K Alothman University of Southampton Electronics and Computer Sience School Electrical Power Engineering
computational Fluid Dynamics - Prof. V. Esfahanian
Three boards categories: Experimental Theoretical Computational Crucial to know all three: Each has their advantages and disadvantages. Require validation and verification. School of Mechanical Engineering
OR 674 DYNAMIC PROGRAMMING Rajesh Ganesan, Associate Professor Systems Engineering and Operations Research George Mason University
OR 674 DYNAMIC PROGRAMMING Rajesh Ganesan, Associate Professor Systems Engineering and Operations Research George Mason University Ankit Shah Ph.D. Candidate Analytics and Operations Research (OR) Descriptive
Constraint Satisfaction Problems
Constraint Satisfaction Problems Frank C. Langbein F.C.Langbein@cs.cf.ac.uk Department of Computer Science Cardiff University 13th February 2001 Constraint Satisfaction Problems (CSPs) A CSP is a high
Introduction to Optimization: Benchmarking
Introduction to Optimization: Benchmarking September 13, 2016 TC2 - Optimisation Université Paris-Saclay, Orsay, France Anne Auger Inria Saclay Ile-de-France Dimo Brockhoff Inria Saclay Ile-de-France Anne
Optimization of Chemical Processes Using Surrogate Models Based on a Kriging Interpolation
Krist V. Gernaey, Jakob K. Huusom and Rafiqul Gani (Eds.), 12th International Symposium on Process Systems Engineering and 25th European Symposium on Computer Aided Process Engineering. 31 May 4 June 2015,
ALAMO user manual and installation guide v
ALAMO user manual and installation guide v. 215.9.18 September 18, 215 For information about this software, contact Nick Sahinidis at niksah@minlp.com. Contents 1 Introduction.................................
Comparison of Some High-Performance MINLP Solvers
Comparison of Some High-Performance MINLP s Toni Lastusilta 1, Michael R. Bussieck 2 and Tapio Westerlund 1,* 1,* Process Design Laboratory, Åbo Akademi University Biskopsgatan 8, FIN-25 ÅBO, Finland 2
A Branch and Bound Algorithm for the Protein Folding Problem in the HP Lattice Model
Article A Branch and Bound Algorithm for the Protein Folding Problem in the HP Lattice Model Mao Chen* and Wen-Qi Huang School of Computer Science and Technology, Huazhong University of Science and Technology,
Data-Driven Modeling. Scientific Computation J. NATHAN KUTZ OXPORD. Methods for Complex Systems & Big Data
Data-Driven Modeling & Scientific Computation Methods for Complex Systems & Big Data J. NATHAN KUTZ Department ofapplied Mathematics University of Washington OXPORD UNIVERSITY PRESS Contents Prolegomenon
Optimization. Industrial AI Lab.
Optimization Industrial AI Lab. Optimization An important tool in 1) Engineering problem solving and 2) Decision science People optimize Nature optimizes 2 Optimization People optimize (source: http://nautil.us/blog/to-save-drowning-people-ask-yourself-what-would-light-do)
Origins of Operations Research: World War II
ESD.83 Historical Roots Assignment METHODOLOGICAL LINKS BETWEEN OPERATIONS RESEARCH AND STOCHASTIC OPTIMIZATION Chaiwoo Lee Jennifer Morris 11/10/2010 Origins of Operations Research: World War II Need
PORTFOLIO OPTIMISATION
PORTFOLIO OPTIMISATION N. STCHEDROFF Abstract. Portfolio optimisation is computationally intensive and has potential for performance improvement. This paper examines the effects of evaluating large numbers
Economic Plantwide Control for a Methanol Plant using Commercial Process Simulation Software
Economic Plantwide Control for a Methanol Plant using Commercial Process Simulation Software Adriana Reyes Lua Chemical Engineering Submission date: June 2014 Supervisor: Sigurd Skogestad, IKP Co-supervisor:
A NEW SEQUENTIAL CUTTING PLANE ALGORITHM FOR SOLVING MIXED INTEGER NONLINEAR PROGRAMMING PROBLEMS
EVOLUTIONARY METHODS FOR DESIGN, OPTIMIZATION AND CONTROL P. Neittaanmäki, J. Périaux and T. Tuovinen (Eds.) c CIMNE, Barcelona, Spain 2007 A NEW SEQUENTIAL CUTTING PLANE ALGORITHM FOR SOLVING MIXED INTEGER
Review of Mixed-Integer Nonlinear and Generalized Disjunctive Programming Methods
Carnegie Mellon University Research Showcase @ CMU Department of Chemical Engineering Carnegie Institute of Technology 2-2014 Review of Mixed-Integer Nonlinear and Generalized Disjunctive Programming Methods
Surrogate-enhanced evolutionary annealing simplex algorithm. for effective and efficient optimization of water resources. problems on a budget
Surrogate-enhanced evolutionary annealing simplex algorithm for effective and efficient optimization of water resources problems on a budget Ioannis Tsoukalas a, *, Panagiotis Kossieris a, Andreas Efstratiadis
Pre-requisite Material for Course Heuristics and Approximation Algorithms
Pre-requisite Material for Course Heuristics and Approximation Algorithms This document contains an overview of the basic concepts that are needed in preparation to participate in the course. In addition,
Optimization: beyond the normal
Optimization: beyond the normal Michael C. Ferris Joint work with: Michael Bussieck, Jan Jagla, Lutz Westermann and Roger Wets Supported partly by AFOSR, DOE and NSF University of Wisconsin, Madison Lunchtime
Learning a classification of Mixed-Integer Quadratic Programming problems
Learning a classification of Mixed-Integer Quadratic Programming problems CERMICS 2018 June 29, 2018, Fréjus Pierre Bonami 1, Andrea Lodi 2, Giulia Zarpellon 2 1 CPLEX Optimization, IBM Spain 2 Polytechnique
Stochastic Algorithms
Stochastic Algorithms Some of the fastest known algorithms for certain tasks rely on chance Stochastic/Randomized Algorithms Two common variations Monte Carlo Las Vegas We have already encountered some
A new Electromagnetism-like Algorithm with a Population Shrinking Strategy
6th WSEAS International Conference on SYSTEM SCIENCE and SIMULATION in ENGINEERING, Venice, Italy, November, 7 7 A new Electromagnetism-like Algorithm with a Population Shrinking Strategy ANA MARIA A.
Parallel Global Optimization with Deterministic Approaches
Parallel Global Optimization with Deterministic Approaches WU YONG School of Mechanical and Aerospace Engineering A thesis submitted to the Nanyang Technological University in fulfillment of the requirement
Blackbox optimization with the MADS algorithm and the NOMAD software
Blackbox optimization with the MADS algorithm and the NOMAD software Sébastien Le Digabel, Charles Audet, Viviane Rochon-Montplaisir, Christophe Tribes IREQ, 2018 06 04 NOMAD: Blackbox Optimization 1/41
Lecture 4. Convexity Robust cost functions Optimizing non-convex functions. 3B1B Optimization Michaelmas 2017 A. Zisserman
Lecture 4 3B1B Optimization Michaelmas 2017 A. Zisserman Convexity Robust cost functions Optimizing non-convex functions grid search branch and bound simulated annealing evolutionary optimization The Optimization
OptCE: A Counterexample-Guided Inductive Optimization Solver
Federal University of Amazonas (UFAM) Postgraduate Program in Electrical Engineering (PPGEE) OptCE: A Counterexample-Guided Inductive Optimization Solver Higo Albuquerque, Rodrigo Araújo, Iury Bessa, Lucas
Optimieren mit MATLAB jetzt auch gemischt-ganzzahlig Dr. Maka Karalashvili Application Engineer MathWorks
Optimieren mit MATLAB jetzt auch gemischt-ganzzahlig Dr. Maka Karalashvili Application Engineer MathWorks 2014 The MathWorks, Inc. 1 Let s consider the following modeling case study Requirements Item Nuts
Large-Scale Multi-Disciplinary Mass Optimization in the Auto Industry
Large-Scale Multi-Disciplinary Mass Optimization in the Auto Industry Don Jones Technical Fellow General Motors Product Development Presented at the MOPTA 2008 Conference, August 20, 2008 Outline The complexity
Lipschitzian Optimization, DIRECT Algorithm, and Applications
,, and April 1, 2008,, and Outline 1 2 3,, and Outline 1 2 3,, and Function Optimization Problem For a function f : D R d R, find min f(x). x D Simple Bounds Mostly we will assume l i x i u i for all i
PROBLEM SOLVING AND SEARCH IN ARTIFICIAL INTELLIGENCE
Artificial Intelligence, Computational Logic PROBLEM SOLVING AND SEARCH IN ARTIFICIAL INTELLIGENCE Lecture 2 Uninformed Search vs. Informed Search Sarah Gaggl Dresden, 28th April 2015 Agenda 1 Introduction
Rich Vehicle Routing Problems Challenges and Prospects in Exploring the Power of Parallelism. Andreas Reinholz. 1 st COLLAB Workshop
Collaborative Research Center SFB559 Modeling of Large Logistic Networks Project M8 - Optimization Rich Vehicle Routing Problems Challenges and Prospects in Exploring the Power of Parallelism Andreas Reinholz
YEAR 12 Core 1 & 2 Maths Curriculum (A Level Year 1)
YEAR 12 Core 1 & 2 Maths Curriculum (A Level Year 1) Algebra and Functions Quadratic Functions Equations & Inequalities Binomial Expansion Sketching Curves Coordinate Geometry Radian Measures Sine and
A Deterministic Global Optimization Method for Variational Inference
A Deterministic Global Optimization Method for Variational Inference Hachem Saddiki Mathematics and Statistics University of Massachusetts, Amherst saddiki@math.umass.edu Andrew C. Trapp Operations and