INTRODUCTION TO LINEAR AND NONLINEAR PROGRAMMING

Transcription

1 INTRODUCTION TO LINEAR AND NONLINEAR PROGRAMMING DAVID G. LUENBERGER Stanford University TT ADDISON-WESLEY PUBLISHING COMPANY Reading, Massachusetts Menlo Park, California London Don Mills, Ontario

2 CONTENTS Chapter 1 Introduction 1.1 Optimization Types of problems Size of problems Iterative algorithms and convergence 6 PART I Linear Programming Chapter 2 Basic Properties of Linear Programs 2.1 Introduction Examples of linear programming problems Basic Solutions The fundamental theorem of linear programming Relations to convexity Exercises 25 Chapter 3 The Simplex Method 3.1 Pivots Adjacent extreme points Determining a minimum feasible Solution Computational procedure simplex method Artiflcial variables 43 *3.6 Variables with upper bounds Matrix form of the simplex method The revised simplex method 55 *3.9 The simplex method and LU decomposition Summary Exercises 63 ix

3 x Chapter 4 Duality 4.1 Dual linear programs The duality theorem Relations to the simplex procedura Sensitivity and complementary slackness 76 *4.5 The dual simplex method 78 *4.6 The primal-dual algorithm Exercises 86 Chapter 5 Reduction of Linear Inequalities 5.1 Introduction Redundant equations Null variables Nonextremal variables Reduced problems Applications of reduction Exercises 105 PART II Unconstrained Problems Chapter 6 Basic Properties of Solutions and Algorithms 6.1 Necessary conditions for local minima Sufficient conditions for a relative minimum Convex and concave functions Minimization and maximization of convex functions Global convergence of descent algorithms Speed of convergence Summary Exercises 131 Chapter 7 Basic Descent Methods 7.1 Fibonacci and golden section search Line search by curve fitting Global convergence of curve fitting Closedness of line search algorithms Inaecurate line search The method of steepesj descent Newton's method Coordinate descent methods Spacer steps Summary Exercises 163

4 xi Chapter 8 Conjugate Direction Methods 8.1 Conjugate directions Descent properties of the conjugate direction method The conjugate gradient method The conjugate gradient method as an optimal process The partial conjugate gradient method Extension to nonquadratic problems Parallel tangents Exercises 186 Chapter 9 Quasi-Newton Methods 9.1 Modified Newton method Construction of the inverse Davidon-Fletcher-Powell method Convergence properties 197 *9.5 Scaling Combination of steepest descent and Newton's method Updating procedures Summary Exercises 213 PART III Constrained Minimization Chapter 10 Constrained Minimization Conditions 10.1 Constraints Tangent plane Necessary and sufficient conditions (equality constraints) Eigenvalues in tangent subspace Sensitivity Inequality constraints Summary Exercises 237 Chapter 11 Primat Methods 11.1 Advantage of primal methods Feasible direction methods Global convergence The gradient projection method Convergence rate of the gradient projection method The reduced gradient method Convergence rate of the reduced gradient method Variations Summary Exercises 273

5 xii Chapter 12 Penalty and Barrier Methods 12.1 Penalty methods Barrier methods Properties of penalty and barrier functions 283 *12.4 Extrapolation Newton's method and penalty functions Conjugate gradients and penalty methods Normalization of penalty functions Penalty functions and gradient projection Penalty functions and the reduced gradient method Summary Exercises 302 Chapter 13 Cutting Plane and Dual Methods 13.1 Cutting plane methods Kelley's convex cutting plane algorithm Modifications Local duality Dual canonical convergence rate Separable Problems Duality and penalty functions Exercises 322 Appendix A Mathematical Review A.l Sets 324 A.2 Matrix notation 325 A.3 Spaces 326 A.4 Eigenvalues and quadratic forms 327 A.5 Topological concepts 328 A.6 Functions 329 Appendix B Convex Sets B.l Basic definitions 332 B.2 Hyperplanes and polytopes 334 B.3 Separating and supporting hyperplanes 337 B.4 Extreme points 338 Appendix C Gaussian Elimination Bibliography 344 Index 353