# Module 1 Lecture Notes 2. Optimization Problem and Model Formulation

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Optimization Methods: Introduction and Basic concepts 1 Module 1 Lecture Notes 2 Optimization Problem and Model Formulation Introduction In the previous lecture we studied the evolution of optimization methods and their engineering applications. A brief introduction was also given to the art of modeling. In this lecture we will study the Optimization problem, its various components and its formulation as a mathematical programming problem. Basic components of an optimization problem: An objective function expresses the main aim of the model which is either to be minimized or maximized. For example, in a manufacturing process, the aim may be to maximize the profit or minimize the cost. In comparing the data prescribed by a user-defined model with the observed data, the aim is minimizing the total deviation of the predictions based on the model from the observed data. In designing a bridge pier, the goal is to maximize the strength and minimize size. A set of unknowns or variables control the value of the objective function. In the manufacturing problem, the variables may include the amounts of different resources used or the time spent on each activity. In fitting-the-data problem, the unknowns are the parameters of the model. In the pier design problem, the variables are the shape and dimensions of the pier. A set of constraints are those which allow the unknowns to take on certain values but exclude others. In the manufacturing problem, one cannot spend negative amount of time on any activity, so one constraint is that the "time" variables are to be non-negative. In the pier design problem, one would probably want to limit the breadth of the base and to constrain its size. The optimization problem is then to find values of the variables that minimize or maximize the objective function while satisfying the constraints. Objective Function As already stated, the objective function is the mathematical function one wants to maximize or minimize, subject to certain constraints. Many optimization problems have a single

2 Optimization Methods: Introduction and Basic concepts 2 objective function. (When they don't they can often be reformulated so that they do) The two exceptions are: No objective function. In some cases (for example, design of integrated circuit layouts), the goal is to find a set of variables that satisfies the constraints of the model. The user does not particularly want to optimize anything and so there is no reason to define an objective function. This type of problems is usually called a feasibility problem. Multiple objective functions. In some cases, the user may like to optimize a number of different objectives concurrently. For instance, in the optimal design of panel of a door or window, it would be good to minimize weight and maximize strength simultaneously. Usually, the different objectives are not compatible; the variables that optimize one objective may be far from optimal for the others. In practice, problems with multiple objectives are reformulated as single-objective problems by either forming a weighted combination of the different objectives or by treating some of the objectives as constraints. Statement of an optimization problem An optimization or a mathematical programming problem can be stated as follows: Subject to the constraints x1 x2 To find X =. which minimizes f(x) (1.1). x n g i (X) 0, i = 1, 2,., m l j (X) = 0, j = 1, 2,., p where X is an n-dimensional vector called the design vector, f(x) is called the objective function, and g i (X) and l j (X) are known as inequality and equality constraints, respectively. The number of variables n and the number of constraints m and/or p need not be related in any way. This type problem is called a constrained optimization problem.

3 Optimization Methods: Introduction and Basic concepts 3 If the locus of all points satisfying f(x) = a constant c, is considered, it can form a family of surfaces in the design space called the objective function surfaces. When drawn with the constraint surfaces as shown in Fig 1 we can identify the optimum point (maxima). This is possible graphically only when the number of design variables is two. When we have three or more design variables because of complexity in the objective function surface, we have to solve the problem as a mathematical problem and this visualization is not possible. C 1 > C 2 > C 3 >C 4..> C n. f = C 1 f = C 2 f = C 3 f= C 4 f = C 5 Optimum point Fig 1 Optimization problems can be defined without any constraints as well. x1 x2 To find X =. which minimizes f(x) (1.2). x n Such problems are called unconstrained optimization problems. The field of unconstrained optimization is quite a large and prominent one, for which a lot of algorithms and software are available.

4 Optimization Methods: Introduction and Basic concepts 4 Variables These are essential. If there are no variables, we cannot define the objective function and the problem constraints. In many practical problems, one cannot choose the design variable arbitrarily. They have to satisfy certain specified functional and other requirements. Constraints Constraints are not essential. It's been argued that almost all problems really do have constraints. For example, any variable denoting the "number of objects" in a system can only be useful if it is less than the number of elementary particles in the known universe! In practice though, answers that make good sense in terms of the underlying physical or economic criteria can often be obtained without putting constraints on the variables. Design constraints are restrictions that must be satisfied to produce an acceptable design. Constraints can be broadly classified as: 1) Behavioral or Functional constraints: These represent limitations on the behavior performance of the system. 2) Geometric or Side constraints: These represent physical limitations on design variables such as availability, fabricability, and transportability. For example, for the retaining wall design shown in the Fig 2, the base width W cannot be taken smaller than a certain value due to stability requirements. The depth D below the ground level depends on the soil pressure coefficients K a and K p. Since these constraints depend on the performance of the retaining wall they are called behavioral constraints. The number of anchors provided along a cross section N i cannot be any real number but has to be a whole number. Similarly thickness of reinforcement used is controlled by supplies from the manufacturer. Hence this is a side constraint.

5 Optimization Methods: Introduction and Basic concepts 5 N i no. of anchors D Constraint Surfaces W Fig. 2 Consider the optimization problem presented in eq. 1.1 with only the inequality constraint g i (X) 0. The set of values of X that satisfy the equation gi(x) 0 forms a boundary surface in the design space called a constraint surface. This will be a (n-1) dimensional subspace where n is the number of design variables. The constraint surface divides the design space into two regions: one with g i (X) < 0 (feasible region) and the other in which gi(x) > 0 (infeasible region). The points lying on the hyper surface will satisfy g i (X) =0. The collection of all the constraint surfaces g i (X) = 0, j= 1, 2,, m, which separates the acceptable region is called the composite constraint surface. Fig 3 shows a hypothetical two-dimensional design space where the feasible region is denoted by hatched lines. The two-dimensional design space is bounded by straight lines as shown in the figure. This is the case when the constraints are linear. However, constraints may be nonlinear as well and the design space will be bounded by curves in that case. A design point that lies on more than one constraint surface is called a bound point, and the associated constraint is called an active constraint. Free points are those that do not lie on any constraint surface. The design points that lie in the acceptable or unacceptable regions can be classified as following: 1. Free and acceptable point 2. Free and unacceptable point

6 Optimization Methods: Introduction and Basic concepts 6 3. Bound and acceptable point 4. Bound and unacceptable point. Examples of each case are shown in Fig. 3. Infeasible region Behavior constraint g 2 0 Side constraint g 3 0. Feasible region Bound acceptable point.. Behavior constraint g 1 0 Free unacceptable point Bound unacceptable point. Free acceptable point Fig. 3 Formulation of design problems as mathematical programming problems In mathematics, the term optimization, or mathematical programming, refers to the study of problems in which one seeks to minimize or maximize a real function by systematically choosing the values of real or integer variables from within an allowed set. This problem can be represented in the following way Given: a function f : A R from some set A to the real numbers Sought: an element x 0 in A such that f(x 0 ) f(x) for all x in A ("minimization") or such that f(x 0 ) f(x) for all x in A ("maximization"). Such a formulation is called an optimization problem or a mathematical programming problem (a term not directly related to computer programming, but still in use for example,

7 Optimization Methods: Introduction and Basic concepts 7 in linear programming (see module 3)). Many real-world and theoretical problems may be modeled in this general framework. Typically, A is some subset of the Euclidean space R n, often specified by a set of constraints, equalities or inequalities that the members of A have to satisfy. The elements of A are called candidate solutions or feasible solutions. The function f is called an objective function, or cost function. A feasible solution that minimizes (or maximizes, if that is the goal) the objective function is called an optimal solution. The domain A of f is called the search space. Generally, when the feasible region or the objective function of the problem does not present convexity (refer module 2), there may be several local minima and maxima, where a local minimum x * is defined as a point for which there exists some δ > 0 so that for all x such that ; and that is to say, on some region around x * all the function values are greater than or equal to the value at that point. Local maxima are defined similarly. A large number of algorithms proposed for solving non-convex problems including the majority of commercially available solvers are not capable of making a distinction between local optimal solutions and rigorous optimal solutions, and will treat the former as the actual solutions to the original problem. The branch of applied mathematics and numerical analysis that is concerned with the development of deterministic algorithms that are capable of guaranteeing convergence in finite time to the actual optimal solution of a non-convex problem is called global optimization. Problem formulation Problem formulation is normally the most difficult part of the process. It is the selection of design variables, constraints, objective function(s), and models of the discipline/design. Selection of design variables A design variable, that takes a numeric or binary value, is controllable from the point of view of the designer. For instance, the thickness of a structural member can be considered a design variable. Design variables can be continuous (such as the length of a cantilever beam),

8 Optimization Methods: Introduction and Basic concepts 8 discrete (such as the number of reinforcement bars used in a beam), or Boolean. Design problems with continuous variables are normally solved more easily. Design variables are often bounded, that is, they have maximum and minimum values. Depending on the adopted method, these bounds can be treated as constraints or separately. Selection of constraints A constraint is a condition that must be satisfied to render the design to be feasible. An example of a constraint in beam design is that the resistance offered by the beam at points of loading must be equal to or greater than the weight of structural member and the load supported. In addition to physical laws, constraints can reflect resource limitations, user requirements, or bounds on the validity of the analysis models. Constraints can be used explicitly by the solution algorithm or can be incorporated into the objective, by using Lagrange multipliers. Objectives An objective is a numerical value that is to be maximized or minimized. For example, a designer may wish to maximize profit or minimize weight. Many solution methods work only with single objectives. When using these methods, the designer normally weights the various objectives and sums them to form a single objective. Other methods allow multi-objective optimization (module 8), such as the calculation of a Pareto front. Models The designer has to also choose models to relate the constraints and the objectives to the design variables. These models are dependent on the discipline involved. They may be empirical models, such as a regression analysis of aircraft prices, theoretical models, such as from computational fluid dynamics, or reduced-order models of either of these. In choosing the models the designer must trade-off fidelity with the time required for analysis. The multidisciplinary nature of most design problems complicates model choice and implementation. Often several iterations are necessary between the disciplines analyses in order to find the values of the objectives and constraints. As an example, the aerodynamic loads on a bridge affect the structural deformation of the supporting structure. The structural deformation in turn changes the shape of the bridge and hence the aerodynamic loads. Thus, it can be considered as a cyclic mechanism. Therefore, in analyzing a bridge, the

9 Optimization Methods: Introduction and Basic concepts 9 aerodynamic and structural analyses must be run a number of times in turn until the loads and deformation converge. Representation in standard form Once the design variables, constraints, objectives, and the relationships between them have been chosen, the problem can be expressed as shown in equation 1.1 Maximization problems can be converted to minimization problems by multiplying the objective by -1. Constraints can be reversed in a similar manner. Equality constraints can be replaced by two inequality constraints. Problem solution The problem is normally solved choosing the appropriate techniques from those available in the field of optimization. These include gradient-based algorithms, population-based algorithms, or others. Very simple problems can sometimes be expressed linearly; in that case the techniques of linear programming are applicable. Gradient-based methods Newton's method Steepest descent Conjugate gradient Sequential quadratic programming Population-based methods Genetic algorithms Particle swarm optimization Other methods Random search Grid search Simulated annealing Most of these techniques require large number of evaluations of the objectives and the constraints. The disciplinary models are often very complex and can take significant amount of time for a single evaluation. The solution can therefore be extremely time-consuming.

10 Optimization Methods: Introduction and Basic concepts 10 Many of the optimization techniques are adaptable to parallel computing. Much of the current research is focused on methods of decreasing the computation time. The following steps summarize the general procedure used to formulate and solve optimization problems. Some problems may not require that the engineer follow the steps in the exact order, but each of the steps should be considered in the process. 1) Analyze the process itself to identify the process variables and specific characteristics of interest, i.e., make a list of all the variables. 2) Determine the criterion for optimization and specify the objective function in terms of the above variables together with coefficients. 3) Develop via mathematical expressions a valid process model that relates the inputoutput variables of the process and associated coefficients. Include both equality and inequality constraints. Use well known physical principles such as mass balances, energy balance, empirical relations, implicit concepts and external restrictions. Identify the independent and dependent variables to get the number of degrees of freedom. 4) If the problem formulation is too large in scope: break it up into manageable parts, or simplify the objective function and the model 5) Apply a suitable optimization technique for mathematical statement of the problem. 6) Examine the sensitivity of the result, to changes in the values of the parameters in the problem and the assumptions.

### Guidelines for proper use of Plate elements

Guidelines for proper use of Plate elements In structural analysis using finite element method, the analysis model is created by dividing the entire structure into finite elements. This procedure is known

### LECTURE 13: SOLUTION METHODS FOR CONSTRAINED OPTIMIZATION. 1. Primal approach 2. Penalty and barrier methods 3. Dual approach 4. Primal-dual approach

LECTURE 13: SOLUTION METHODS FOR CONSTRAINED OPTIMIZATION 1. Primal approach 2. Penalty and barrier methods 3. Dual approach 4. Primal-dual approach Basic approaches I. Primal Approach - Feasible Direction

### Theoretical Concepts of Machine Learning

Theoretical Concepts of Machine Learning Part 2 Institute of Bioinformatics Johannes Kepler University, Linz, Austria Outline 1 Introduction 2 Generalization Error 3 Maximum Likelihood 4 Noise Models 5

### BCN Decision and Risk Analysis. Syed M. Ahmed, Ph.D.

Linear Programming Module Outline Introduction The Linear Programming Model Examples of Linear Programming Problems Developing Linear Programming Models Graphical Solution to LP Problems The Simplex Method

### White Paper. Scia Engineer Optimizer: Automatic Optimization of Civil Engineering Structures. Authors: Radim Blažek, Martin Novák, Pavel Roun

White Paper Scia Engineer Optimizer: Automatic Optimization of Civil Engineering Structures Nemetschek Scia nv Industrieweg 1007 3540 Herk-de-Stad (Belgium) Tel.: (+32) 013 55.17.75 Fax: (+32) 013 55.41.75

### Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers. Lagrange Multipliers

In this section we present Lagrange s method for maximizing or minimizing a general function f(x, y, z) subject to a constraint (or side condition) of the form g(x, y, z) = k. Figure 1 shows this curve

### MATH3016: OPTIMIZATION

MATH3016: OPTIMIZATION Lecturer: Dr Huifu Xu School of Mathematics University of Southampton Highfield SO17 1BJ Southampton Email: h.xu@soton.ac.uk 1 Introduction What is optimization? Optimization is

### Optimization Methods: Advanced Topics in Optimization - Multi-objective Optimization 1. Module 8 Lecture Notes 2. Multi-objective Optimization

Optimization Methods: Advanced Topics in Optimization - Multi-objective Optimization 1 Module 8 Lecture Notes 2 Multi-objective Optimization Introduction In a real world problem it is very unlikely that

### Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras. Lecture - 36

Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras Lecture - 36 In last class, we have derived element equations for two d elasticity problems

### A Deterministic Dynamic Programming Approach for Optimization Problem with Quadratic Objective Function and Linear Constraints

A Deterministic Dynamic Programming Approach for Optimization Problem with Quadratic Objective Function and Linear Constraints S. Kavitha, Nirmala P. Ratchagar International Science Index, Mathematical

### Analysis and optimization methods of graph based meta-models for data flow simulation

Rochester Institute of Technology RIT Scholar Works Theses Thesis/Dissertation Collections 8-1-2010 Analysis and optimization methods of graph based meta-models for data flow simulation Jeffrey Harrison

### Discrete Optimization. Lecture Notes 2

Discrete Optimization. Lecture Notes 2 Disjunctive Constraints Defining variables and formulating linear constraints can be straightforward or more sophisticated, depending on the problem structure. The

### Classification: Linear Discriminant Functions

Classification: Linear Discriminant Functions CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Discriminant functions Linear Discriminant functions

### Computational Methods. Constrained Optimization

Computational Methods Constrained Optimization Manfred Huber 2010 1 Constrained Optimization Unconstrained Optimization finds a minimum of a function under the assumption that the parameters can take on

### MULTIDISCIPLINARY OPTIMIZATION IN TURBOMACHINERY DESIGN

European Congress on Computational Methods in Applied Sciences and Engineering ECCOMAS 000 Barcelona, -4 September 000 ECCOMAS MULTIDISCIPLINARY OPTIMIZATION IN TURBOMACHINERY DESIGN Rolf Dornberger *,

### COMS 4771 Support Vector Machines. Nakul Verma

COMS 4771 Support Vector Machines Nakul Verma Last time Decision boundaries for classification Linear decision boundary (linear classification) The Perceptron algorithm Mistake bound for the perceptron

### Applied Lagrange Duality for Constrained Optimization

Applied Lagrange Duality for Constrained Optimization Robert M. Freund February 10, 2004 c 2004 Massachusetts Institute of Technology. 1 1 Overview The Practical Importance of Duality Review of Convexity

### Mathematical Programming and Research Methods (Part II)

Mathematical Programming and Research Methods (Part II) 4. Convexity and Optimization Massimiliano Pontil (based on previous lecture by Andreas Argyriou) 1 Today s Plan Convex sets and functions Types

### All lecture slides will be available at CSC2515_Winter15.html

CSC2515 Fall 2015 Introduc3on to Machine Learning Lecture 9: Support Vector Machines All lecture slides will be available at http://www.cs.toronto.edu/~urtasun/courses/csc2515/ CSC2515_Winter15.html Many

### Optimizations and Lagrange Multiplier Method

Introduction Applications Goal and Objectives Reflection Questions Once an objective of any real world application is well specified as a function of its control variables, which may subject to a certain

### Introduction to ANSYS DesignXplorer

Lecture 5 Goal Driven Optimization 14. 5 Release Introduction to ANSYS DesignXplorer 1 2013 ANSYS, Inc. September 27, 2013 Goal Driven Optimization (GDO) Goal Driven Optimization (GDO) is a multi objective

### PRIMAL-DUAL INTERIOR POINT METHOD FOR LINEAR PROGRAMMING. 1. Introduction

PRIMAL-DUAL INTERIOR POINT METHOD FOR LINEAR PROGRAMMING KELLER VANDEBOGERT AND CHARLES LANNING 1. Introduction Interior point methods are, put simply, a technique of optimization where, given a problem

### Machine Learning for Signal Processing Lecture 4: Optimization

Machine Learning for Signal Processing Lecture 4: Optimization 13 Sep 2015 Instructor: Bhiksha Raj (slides largely by Najim Dehak, JHU) 11-755/18-797 1 Index 1. The problem of optimization 2. Direct optimization

### Lecture 7: Support Vector Machine

Lecture 7: Support Vector Machine Hien Van Nguyen University of Houston 9/28/2017 Separating hyperplane Red and green dots can be separated by a separating hyperplane Two classes are separable, i.e., each

### Free-Form Shape Optimization using CAD Models

Free-Form Shape Optimization using CAD Models D. Baumgärtner 1, M. Breitenberger 1, K.-U. Bletzinger 1 1 Lehrstuhl für Statik, Technische Universität München (TUM), Arcisstraße 21, D-80333 München 1 Motivation

### A Short SVM (Support Vector Machine) Tutorial

A Short SVM (Support Vector Machine) Tutorial j.p.lewis CGIT Lab / IMSC U. Southern California version 0.zz dec 004 This tutorial assumes you are familiar with linear algebra and equality-constrained optimization/lagrange

### Introduction to Optimization Problems and Methods

Introduction to Optimization Problems and Methods wjch@umich.edu December 10, 2009 Outline 1 Linear Optimization Problem Simplex Method 2 3 Cutting Plane Method 4 Discrete Dynamic Programming Problem Simplex

### Convex Optimization CMU-10725

Convex Optimization CMU-10725 Conjugate Direction Methods Barnabás Póczos & Ryan Tibshirani Conjugate Direction Methods 2 Books to Read David G. Luenberger, Yinyu Ye: Linear and Nonlinear Programming Nesterov:

### Introduction to Design Optimization: Search Methods

Introduction to Design Optimization: Search Methods 1-D Optimization The Search We don t know the curve. Given α, we can calculate f(α). By inspecting some points, we try to find the approximated shape

### Support Vector Machines. James McInerney Adapted from slides by Nakul Verma

Support Vector Machines James McInerney Adapted from slides by Nakul Verma Last time Decision boundaries for classification Linear decision boundary (linear classification) The Perceptron algorithm Mistake

### Optimization. Using Analytic Solver Platform REVIEW BASED ON MANAGEMENT SCIENCE

Optimization Using Analytic Solver Platform REVIEW BASED ON MANAGEMENT SCIENCE What We ll Cover Today Introduction Frontline Systems Session Ι beta training program goals Overview of Analytic Solver Platform

### Simulation in Computer Graphics. Particles. Matthias Teschner. Computer Science Department University of Freiburg

Simulation in Computer Graphics Particles Matthias Teschner Computer Science Department University of Freiburg Outline introduction particle motion finite differences system of first order ODEs second

### LS-OPT : New Developments and Outlook

13 th International LS-DYNA Users Conference Session: Optimization LS-OPT : New Developments and Outlook Nielen Stander and Anirban Basudhar Livermore Software Technology Corporation Livermore, CA 94588

### Lecture 4. Convexity Robust cost functions Optimizing non-convex functions. 3B1B Optimization Michaelmas 2017 A. Zisserman

Lecture 4 3B1B Optimization Michaelmas 2017 A. Zisserman Convexity Robust cost functions Optimizing non-convex functions grid search branch and bound simulated annealing evolutionary optimization The Optimization

### The ALAMO approach to machine learning: Best subset selection, adaptive sampling, and constrained regression

The ALAMO approach to machine learning: Best subset selection, adaptive sampling, and constrained regression Nick Sahinidis Acknowledgments: Alison Cozad, David Miller, Zach Wilson MACHINE LEARNING PROBLEM

### Support Vector Machines

Support Vector Machines SVM Discussion Overview. Importance of SVMs. Overview of Mathematical Techniques Employed 3. Margin Geometry 4. SVM Training Methodology 5. Overlapping Distributions 6. Dealing

### Inverse and Implicit functions

CHAPTER 3 Inverse and Implicit functions. Inverse Functions and Coordinate Changes Let U R d be a domain. Theorem. (Inverse function theorem). If ϕ : U R d is differentiable at a and Dϕ a is invertible,

### Design Optimization - Structural Design Optimization -

16.810 Engineering Design and Rapid Prototyping Lecture 6 Design Optimization - Structural Design Optimization - Instructor(s) Prof. Olivier de Weck January 11, 2005 What Is Design Optimization? Selecting

### Operations Research and Optimization: A Primer

Operations Research and Optimization: A Primer Ron Rardin, PhD NSF Program Director, Operations Research and Service Enterprise Engineering also Professor of Industrial Engineering, Purdue University Introduction

### An Evolutionary Algorithm for Minimizing Multimodal Functions

An Evolutionary Algorithm for Minimizing Multimodal Functions D.G. Sotiropoulos, V.P. Plagianakos and M.N. Vrahatis University of Patras, Department of Mamatics, Division of Computational Mamatics & Informatics,

### WE consider the gate-sizing problem, that is, the problem

2760 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I: REGULAR PAPERS, VOL 55, NO 9, OCTOBER 2008 An Efficient Method for Large-Scale Gate Sizing Siddharth Joshi and Stephen Boyd, Fellow, IEEE Abstract We consider

### Fixture Layout Optimization Using Element Strain Energy and Genetic Algorithm

Fixture Layout Optimization Using Element Strain Energy and Genetic Algorithm Zeshan Ahmad, Matteo Zoppi, Rezia Molfino Abstract The stiffness of the workpiece is very important to reduce the errors in

### CT79 SOFT COMPUTING ALCCS-FEB 2014

Q.1 a. Define Union, Intersection and complement operations of Fuzzy sets. For fuzzy sets A and B Figure Fuzzy sets A & B The union of two fuzzy sets A and B is a fuzzy set C, written as C=AUB or C=A OR

### Learning and Generalization in Single Layer Perceptrons

Learning and Generalization in Single Layer Perceptrons Neural Computation : Lecture 4 John A. Bullinaria, 2015 1. What Can Perceptrons do? 2. Decision Boundaries The Two Dimensional Case 3. Decision Boundaries

### 10. Network dimensioning

Partly based on slide material by Samuli Aalto and Jorma Virtamo ELEC-C7210 Modeling and analysis of communication networks 1 Contents Introduction Parameters: topology, routing and traffic Dimensioning

### MS&E 213 / CS 269O : Chapter 1 Introduction to Introduction to Optimization Theory

MS&E 213 / CS 269O : Chapter 1 Introduction to Introduction to Optimization Theory By Aaron Sidford (sidford@stanford.edu) April 29, 2017 1 What is this course about? The central problem we consider throughout

### Development in Object Detection. Junyuan Lin May 4th

Development in Object Detection Junyuan Lin May 4th Line of Research [1] N. Dalal and B. Triggs. Histograms of oriented gradients for human detection, CVPR 2005. HOG Feature template [2] P. Felzenszwalb,

### Introduction. 1. Function and parameter optimization. 2. Problem Formulation. 3. Graphical Optimization. 4. Exercises. 5.

Introduction 1. Function and parameter optimization. Problem Formulation.1. Design variables.. Constraints.3. Objective function 3. Graphical Optimization 4. Exercises 5. References 1. Function and parameter

### 53:244 (58:254) Energy Principles in Structural Mechanics. virtual work, principles of stationary and minimum potential energy;

Introduction, course perspective, course organization. Material: Derivation of basic principles of mechanics; principle of virtual work, principles of stationary and minimum potential energy; Hamilton

### A Network Model to Optimise Cost in Underground Mine Design

A Network Model to Optimise Cost in Underground Mine Design M. Brazil, D. H. Lee, J. H. Rubinstein, D. A. Thomas, J. F. Weng, N. C. Wormald Department of Electrical & Electronic Engineering, The University

### Example 13 - Shock Tube

Example 13 - Shock Tube Summary This famous experiment is interesting for observing the shock-wave propagation. Moreover, this case uses the representation of perfect gas and compares the different formulations:

### Multi Layer Perceptron trained by Quasi Newton learning rule

Multi Layer Perceptron trained by Quasi Newton learning rule Feed-forward neural networks provide a general framework for representing nonlinear functional mappings between a set of input variables and

### Shape fitting and non convex data analysis

Shape fitting and non convex data analysis Petra Surynková, Zbyněk Šír Faculty of Mathematics and Physics, Charles University in Prague Sokolovská 83, 186 7 Praha 8, Czech Republic email: petra.surynkova@mff.cuni.cz,

### MLR Institute of Technology

Course Name : Engineering Optimization Course Code : 56021 Class : III Year Branch : Aeronautical Engineering Year : 2014-15 Course Faculty : Mr Vamsi Krishna Chowduru, Assistant Professor Course Objective

### Applications of Linear Programming

Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 1 Why LP? Linear programming (LP, also called linear

### 3 INTEGER LINEAR PROGRAMMING

3 INTEGER LINEAR PROGRAMMING PROBLEM DEFINITION Integer linear programming problem (ILP) of the decision variables x 1,..,x n : (ILP) subject to minimize c x j j n j= 1 a ij x j x j 0 x j integer n j=

### Energy Minimization -Non-Derivative Methods -First Derivative Methods. Background Image Courtesy: 3dciencia.com visual life sciences

Energy Minimization -Non-Derivative Methods -First Derivative Methods Background Image Courtesy: 3dciencia.com visual life sciences Introduction Contents Criteria to start minimization Energy Minimization

### APPLICATION OF VARIABLE-FIDELITY MODELS TO AERODYNAMIC OPTIMIZATION

Applied Mathematics and Mechanics (English Edition), 2006, 27(8):1089 1095 c Editorial Committee of Appl. Math. Mech., ISSN 0253-4827 APPLICATION OF VARIABLE-FIDELITY MODELS TO AERODYNAMIC OPTIMIZATION

### Engineering optimization. Chapter 7: Constrained direct search

Engineering optimization. Chapter 7: Constrained direct search Huy-Dung Han, Fabio E. Lapiccirella Department of Electrical and Computer Engineering University of California, Davis Davis, CA 95616, U.S.A.

### Artificial Intelligence

Artificial Intelligence Local Search Vibhav Gogate The University of Texas at Dallas Some material courtesy of Luke Zettlemoyer, Dan Klein, Dan Weld, Alex Ihler, Stuart Russell, Mausam Systematic Search:

### Formal Model. Figure 1: The target concept T is a subset of the concept S = [0, 1]. The search agent needs to search S for a point in T.

Although this paper analyzes shaping with respect to its benefits on search problems, the reader should recognize that shaping is often intimately related to reinforcement learning. The objective in reinforcement

### TV packaging optimization of the frontal drop impact using equivalent static loads

11 th World Congress on Structural and Multidisciplinary Optimisation 7-12, June 2015, Sydney Australia TV packaging optimization of the frontal drop impact using equivalent static loads Insik Han 1, Youngmyung

### KEYWORDS Non-parametric optimization, Parametric Optimization, Design of Experiments, Response Surface Modelling, Multidisciplinary Optimization

Session H2.5 OVERVIEW ON OPTIMIZATION METHODS Ioannis Nitsopoulos *, Boris Lauber FE-DESIGN GmbH, Germany KEYWORDS Non-parametric optimization, Parametric Optimization, Design of Experiments, Response

### 7 Fractions. Number Sense and Numeration Measurement Geometry and Spatial Sense Patterning and Algebra Data Management and Probability

7 Fractions GRADE 7 FRACTIONS continue to develop proficiency by using fractions in mental strategies and in selecting and justifying use; develop proficiency in adding and subtracting simple fractions;

### Constrained Functions of N Variables: Non-Gradient Based Methods

onstrained Functions of N Variables: Non-Gradient Based Methods Gerhard Venter Stellenbosch University Outline Outline onstrained Optimization Non-gradient based methods Genetic Algorithms (GA) Particle

### Linear Programming & Graphic Solution. Dr. Monther Tarawneh

Linear Programming & Graphic Solution Dr. Monther Tarawneh In this Lecture This topic concentrates on model formulation and computations in linear programming (LP). To illustrate the use of LP, real world

### Interaction of Fluid Simulation Based on PhysX Physics Engine. Huibai Wang, Jianfei Wan, Fengquan Zhang

4th International Conference on Sensors, Measurement and Intelligent Materials (ICSMIM 2015) Interaction of Fluid Simulation Based on PhysX Physics Engine Huibai Wang, Jianfei Wan, Fengquan Zhang College

### The Simplex Algorithm

The Simplex Algorithm Uri Feige November 2011 1 The simplex algorithm The simplex algorithm was designed by Danzig in 1947. This write-up presents the main ideas involved. It is a slight update (mostly

### Optimal Control Techniques for Dynamic Walking

Optimal Control Techniques for Dynamic Walking Optimization in Robotics & Biomechanics IWR, University of Heidelberg Presentation partly based on slides by Sebastian Sager, Moritz Diehl and Peter Riede

### Image Compression: An Artificial Neural Network Approach

Image Compression: An Artificial Neural Network Approach Anjana B 1, Mrs Shreeja R 2 1 Department of Computer Science and Engineering, Calicut University, Kuttippuram 2 Department of Computer Science and

### Economizing the stability of rubble-mound breakwaters using artificial neural network

Fluid Structure Interaction and Moving Boundary Problems IV 47 Economizing the stability of rubble-mound breakwaters using artificial neural network M. Shakeri Majd 1, M. A. Lashteh Neshaei 2 & M. Salehi

Optimization by Prof. Seungchul Lee Industrial AI Lab http://isystems.unist.ac.kr/ POSTECH Table of Contents I. 1. Optimization II. 2. Solving Optimization Problems III. 3. How do we Find x f(x) = 0 IV.

### Simplex of Nelder & Mead Algorithm

Simplex of N & M Simplex of Nelder & Mead Algorithm AKA the Amoeba algorithm In the class of direct search methods Unconstrained (although constraints can be added as part of error function) nonlinear

### Elliptic Curve Public Key Cryptography

Why? Elliptic Curve Public Key Cryptography ECC offers greater security for a given key size. Why? Elliptic Curve Public Key Cryptography ECC offers greater security for a given key size. The smaller key

### A Multiple Constraint Approach for Finite Element Analysis of Moment Frames with Radius-cut RBS Connections

A Multiple Constraint Approach for Finite Element Analysis of Moment Frames with Radius-cut RBS Connections Dawit Hailu +, Adil Zekaria ++, Samuel Kinde +++ ABSTRACT After the 1994 Northridge earthquake

### Stochastic Separable Mixed-Integer Nonlinear Programming via Nonconvex Generalized Benders Decomposition

Stochastic Separable Mixed-Integer Nonlinear Programming via Nonconvex Generalized Benders Decomposition Xiang Li Process Systems Engineering Laboratory Department of Chemical Engineering Massachusetts

### ANALYSIS OF BOX CULVERT - COST OPTIMIZATION FOR DIFFERENT ASPECT RATIOS OF CELL

ANALYSIS OF BOX CULVERT - COST OPTIMIZATION FOR DIFFERENT ASPECT RATIOS OF CELL M.G. Kalyanshetti 1, S.A. Gosavi 2 1 Assistant professor, Civil Engineering Department, Walchand Institute of Technology,

### IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 6, NOVEMBER Inverting Feedforward Neural Networks Using Linear and Nonlinear Programming

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 6, NOVEMBER 1999 1271 Inverting Feedforward Neural Networks Using Linear and Nonlinear Programming Bao-Liang Lu, Member, IEEE, Hajime Kita, and Yoshikazu

### Introduction to Linear Programming

Introduction to Linear Programming Linear Programming Applied mathematics is all about applying mathematical techniques to understand or do something practical. Optimization is all about making things

### Introduction to optimization methods and line search

Introduction to optimization methods and line search Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi How to find optimal solutions? Trial and error widely used in practice, not efficient and

### A Nonlinear Presolve Algorithm in AIMMS

A Nonlinear Presolve Algorithm in AIMMS By Marcel Hunting marcel.hunting@aimms.com November 2011 This paper describes the AIMMS presolve algorithm for nonlinear problems. This presolve algorithm uses standard

### Introduction. Chapter 15. Optimization Modeling: Applications. Integer Programming. Manufacturing Example. Three Types of ILP Models

Chapter 5 Optimization Modeling: Applications Integer Programming Introduction When one or more variables in an LP problem must assume an integer value we have an Integer Linear Programming (ILP) problem.

### Introduction to Optimization

Introduction to Optimization Second Order Optimization Methods Marc Toussaint U Stuttgart Planned Outline Gradient-based optimization (1st order methods) plain grad., steepest descent, conjugate grad.,

### SYSTEMS OF NONLINEAR EQUATIONS

SYSTEMS OF NONLINEAR EQUATIONS Widely used in the mathematical modeling of real world phenomena. We introduce some numerical methods for their solution. For better intuition, we examine systems of two

### Efficient Tuning of SVM Hyperparameters Using Radius/Margin Bound and Iterative Algorithms

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 13, NO. 5, SEPTEMBER 2002 1225 Efficient Tuning of SVM Hyperparameters Using Radius/Margin Bound and Iterative Algorithms S. Sathiya Keerthi Abstract This paper

### Conceptual Design and CFD

Conceptual Design and CFD W.H. Mason Department of and the Multidisciplinary Analysis and Design (MAD) Center for Advanced Vehicles Virginia Tech Blacksburg, VA Update from AIAA 98-2513 1 Things to think

### Introduction to Machine Learning. Xiaojin Zhu

Introduction to Machine Learning Xiaojin Zhu jerryzhu@cs.wisc.edu Read Chapter 1 of this book: Xiaojin Zhu and Andrew B. Goldberg. Introduction to Semi- Supervised Learning. http://www.morganclaypool.com/doi/abs/10.2200/s00196ed1v01y200906aim006

### ME 575: Two-bar Truss

Name ME 575: Two-bar Truss Consider the design of a simple tubular symmetric truss shown in Fig. 1.1 below (problem originally from Fox 1 ). A design of the truss is specified by a unique set of values

### / Approximation Algorithms Lecturer: Michael Dinitz Topic: Linear Programming Date: 2/24/15 Scribe: Runze Tang

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Linear Programming Date: 2/24/15 Scribe: Runze Tang 9.1 Linear Programming Suppose we are trying to approximate a minimization

### Chapter 7. Linear Programming Models: Graphical and Computer Methods

Chapter 7 Linear Programming Models: Graphical and Computer Methods To accompany Quantitative Analysis for Management, Eleventh Edition, by Render, Stair, and Hanna Power Point slides created by Brian

### Lecture 15: Segmentation (Edge Based, Hough Transform)

Lecture 15: Segmentation (Edge Based, Hough Transform) c Bryan S. Morse, Brigham Young University, 1998 000 Last modified on February 3, 000 at :00 PM Contents 15.1 Introduction..............................................

### Surrogate models for Rapid Sizing of Structures

ESAZO Stéphane Grihon Expert in Structure Optimization Surrogate models for Rapid Sizing of Structures An experience of surrogate models for structure optimization at AIRBUS Summary Introduction Various

### Overcompressing JPEG images with Evolution Algorithms

Author manuscript, published in "EvoIASP2007, Valencia : Spain (2007)" Overcompressing JPEG images with Evolution Algorithms Jacques Lévy Véhel 1, Franklin Mendivil 2 and Evelyne Lutton 1 1 Inria, Complex

### CLASSIFICATION OF CUSTOMER PURCHASE BEHAVIOR IN THE AIRLINE INDUSTRY USING SUPPORT VECTOR MACHINES

CLASSIFICATION OF CUSTOMER PURCHASE BEHAVIOR IN THE AIRLINE INDUSTRY USING SUPPORT VECTOR MACHINES Pravin V, Innovation and Development Team, Mu Sigma Business Solutions Pvt. Ltd, Bangalore. April 2012

### An Interactive Evolutionary Multi-Objective Optimization Method Based on Progressively Approximated Value Functions

An Interactive Evolutionary Multi-Objective Optimization Method Based on Progressively Approximated Value Functions Kalyanmoy Deb, Ankur Sinha, Pekka Korhonen, and Jyrki Wallenius KanGAL Report Number

### 9-1 GCSE Maths. GCSE Mathematics has a Foundation tier (Grades 1 5) and a Higher tier (Grades 4 9).

9-1 GCSE Maths GCSE Mathematics has a Foundation tier (Grades 1 5) and a Higher tier (Grades 4 9). In each tier, there are three exams taken at the end of Year 11. Any topic may be assessed on each of

### The Automation of the Feature Selection Process. Ronen Meiri & Jacob Zahavi

The Automation of the Feature Selection Process Ronen Meiri & Jacob Zahavi Automated Data Science http://www.kdnuggets.com/2016/03/automated-data-science.html Outline The feature selection problem Objective

### Lagrange Multipliers and Problem Formulation

Lagrange Multipliers and Problem Formulation Steven J. Miller Department of Mathematics and Statistics Williams College Williamstown, MA 01267 Abstract The method of Lagrange Multipliers (and its generalizations)