David G. Luenberger Yinyu Ye. Linear and Nonlinear. Programming. Fourth Edition. ö Springer

Transcription

1 David G. Luenberger Yinyu Ye Linear and Nonlinear Programming Fourth Edition ö Springer

2 Contents 1 Introduction Optimization Types of Problems Size of Problems Iterative Algorithms and Convergence 6 Part I Linear Programming 2 Basic Properties of Linear Programs Introduction Examples of Linear Programming Problems Basic Solutions The Fundamental Theorem of Linear Programming Relations to Convexity Exercises 27 3 The Simplex Method Pivots Adjacent Extreme Points Determining a Minimum Feasible Solution Computational Procedure: Simplex Method Rinding a Basic Feasible Solution Matrix Form of the Simplex Method Simplex Method for Transportation Problems Decomposition Summary Exercises 73 4 Duality and Complementarity Dual Linear Programs The Duality Theorem 86 ix

3 x Contents 4.3 Relations to the Simplex Procedura Sensitivity and Complementary Slackness Max Flow-Min Cut Theorem The Dual Simplex Method *The Primal-Dual Algorithm Summary Exercises Inferior-Point Methods Elements of Complexity Theory *The Simplex Method Is Not Polynomial-Time *The Ellipsoid Method The Analytic Center The Central Path Solution Strategien Termination and Initialization Summary Exercises Conic Linear Programming ConvexCones Conic Linear Programming Problem Parkas' Lemma for Conic Linear Programming Conic Linear Programming Duality Complementarity and Solution Rank of SDP Interior-Point Algorithms for Conic Linear Programming Summary Exercises 174 Part II Unconstrained Problems 7 Basic Properties of Solutions and Algorithms First-Order Necessary Conditions Examples of Unconstrained Problems Second-Order Conditions Convex and Concave Functions Minimization and Maximization of Convex Functions *Zero-Order Conditions Global Convergence of Descent Algorithms Speed of Convergence Summary Exercises 209

4 Contents xi 8 Basic Descent Methods Line Search Algorithms The Method of Steepest Descent Applications of the Convergence Theory Accelerated Steepest Descent Newton's Method Coordinate Descent Methods Summary Exercises Conjugate Direction Methods Conjugate Directions Descent Properties of the Conjugate Direction Method The Conjugate Gradient Method The C-G Method as an Optimal Process The Partial Conjugate Gradient Method Extension to Nonquadratic Problems * Parallel Tangents Exercises Quasi-Newton Methods Modified Newton Method Construction of the Inverse Davidon-Fletcher-Powell Method The Broyden Family Convergence Properties Scaling Memoryless Quasi-Newton Methods *Combination of Steepest Descent and Newton's Method Summary Exercises 313 Part III Constrained Minimization 11 Constrained Minimization Conditions Constraints Tangent Plane First-Order Necessary Conditions (Equality Constraints) Examples Second-Order Conditions Eigenvalues in Tangent Subspace Sensitivity Inequality Constraints Zero-Order Conditions and Lagrangian Relaxation Summary Exercises 352

5 xii Contents 12 Primal Methods Advantage of Primal Methods Feasible Direction Methods Active Set Methods The Gradient Projection Method Convergence Rate of the Gradient Projection Method The Reduced Gradient Method Convergence Rate of the Reduced Gradient Method *Variations Summary Exercises Penalty and Barrier Methods Penalty Methods Barrier Methods Properties of Penalty and Barrier Functions Newton's Method and Penalty Functions Conjugate Gradients and Penalty Methods Normalization of Penalty Functions Penalty Functions and Gradient Projection *Exact Penalty Functions Summary Exercises Duality and Dual Methods Global Duality Local Duality Canonical Convergence Rate of Dual Steepest Ascent Separable Problems and Their Duals Augmented Lagrangian The Method of Multipliers The Alternating Direction Method of Multipliers *Cutting Plane Methods Exercises Primal-Dual Methods The Standard Problem A Simple Merit Function Basic Primal-Dual Methods Modified Newton Methods Descent Properties * Rate of Convergence Primal-Dual Interior Point Methods Summary Exercises 489

6 Contents xiii A Mathematical Review 495 A.l Sets 495 A.2 Matrix Notation 496 A.3 Spaces 497 A.4 Eigenvalues and Quadratic Forms 498 A.5 Topological Concepts 499 A.6 Functions 500 B Convex Sets 505 B.l Basic Definitions 505 B.2 Hyperplanes and Polytopes 507 B.3 Separating and Supporting Hyperplanes 509 B.4 Extreme Points 511 C Gaussian Elimination 513 D Basic Network Concepts 517 D.l Flows in Networks 519 D.2 Tree Procedure 519 D.3 Capacitated Networks 521 Bibliography 523 Index 539