w KLUWER ACADEMIC PUBLISHERS Global Optimization with Non-Convex Constraints Sequential and Parallel Algorithms Roman G. Strongin Yaroslav D.

Transcription

1 Global Optimization with Non-Convex Constraints Sequential and Parallel Algorithms by Roman G. Strongin Nizhni Novgorod State University, Nizhni Novgorod, Russia and Yaroslav D. Sergeyev Institute of Systems Analysis and Information Technology, University of Calabria, Rende, Italy and Nizhni Novgorod State University, Nizhni Novgorod, Russia w KLUWER ACADEMIC PUBLISHERS DORDRECHT / BOSTON / LONDON

2 CONTENTS PREFACE ACKNOWLEDGEMENTS xvii xxvii Part One GLOBAL OPTIMIZATION ALGORITHMS AS DECISION PROCEDURES. THEORETICAL BACKGROUND AND CORE UNIVARIATE CASE 1 1 INTRODUCTION Optimization Problems and Search Techniques A priori Information and Estimates for an Optimum 10 Role of a priori Information 10 Unimodality and Local Improvements 11 Multimodality and Adaptation of Local Techniques 13 A priori Information and Expansion in Standard Problems 17 Lipschitz Continuity Assumptions and Global Optimality 19 Objective Function as a Sample of some Random Function Decision Rules as Minimax Optimal Strategies 28 Minimax Approach 29 One-step Optimality Principle 33 vn

3 viii GLOBAL OPTIMIZATION WITH NON-CONVEX CONSTRAINTS 1.4 Information-Statistical Approach and Average Optimality Problem of Dimensionality and Reduction to One Dimension 40 Exponential Growth of the Grid Technique Complexity with the Increase of Dimensionality 40 Increasing Complexity of Building Effective Grids in Many Dimensions 41 Reduction to One Dimension Constraints and Reduction to Unconstrained Case without Penalties 47 Optimality and Constraints 47 Partial Computability of Problem Functionals 49 Indexes and Compatibility of Constraints 50 Reduction to the Unconstrained Case 50 GLOBAL OPTIMIZATION ALGORITHMS AS STATISTICAL DECISION PROCEDURES - THE INFORMATION APPROACH Estimates for the Global Optimum Based on a Stochastic Description of the Problem 53 A priori Description and Estimates 53 Model for Outcomes of Trials 55 A posteriori Estimates for the Global Optimizer 57 Estimates for the Case of Error-Free Observation Approximate Stochastic Estimators for the Global Optimizer 62 Simplified Estimators for the Global Optimizer 62 Sufficient Conditions for Approximation 64 Particular Stochastic Model 67 Bayesian Estimates for Error-Free Observations 70 Bayesian Estimates for Observations Corrupted by Gaussian Errors 74

4 Contents ix 2.3 Decision and Termination Rules for Error-Free Observations 78 Forecasting Outcomes of Trials 78 One-Step Optimal Decisions 82 Termination Rule and the Search Algorithm 84 Randomized Strategies Decision Rules for Observations Corrupted by additive Non-Biased Gaussian Errors 90 Forecasting Outcomes of Noise-Corrupted Trials 90 Decision Rule and Convergence Study 93 Numerical Simulation of Search in the Presence of Noise Estimations and Decisions in Problems of Equation Solving 106 Stochastic Model and Bayesian Estimates for a Root of an Equation 106 Decision Rule and Convergence Study 113 Root Search Algorithms 121 CORE GLOBAL SEARCH ALGORITHM AND CONVERGENCE STUDY Global Search Algorithm Convergence Conditions 133 Lipschitzian Case 133 Discontinuous Case 138 Smoothing Global Search Algorithm Rate of Convergence 149 Density of Trials 149 Sequence Structures in the Ranges of Function Linearity 151 Comparison with the Grid Technique in Ranges of Function Linearity Termination Criterion and Truncated Sequences Monotonous Convergence 170 Monotonous and Nearly Monotonous Convergence 170

5 x GLOBAL OPTIMIZATION WITH NON-CONVEX CONSTRAINTS Conditions for Nearly Monotonous Convergence 176 Monotonous Global Search Algorithm A priori Assumptions and Acceleration of Search 193 Local Refinement of the Best Current Estimate 193 Operating Characteristics and the Decision Rules Efficiency 202 Function Range Global Search Algorithm 212 Problems with Simple Geometrie Constraints Global Optimization over the Set of Open Intervals 216 Algorithm and Convergence Conditions 216 Case of Monotonically Unimodal Constraints 223 GLOBAL OPTIMIZATION METHODS AS BOUNDING PROCEDURES - THE GEOMETRIC APPROACH Introduction to the Geometrie Approach Local Tuning of Bounding Procedures for Lipschitz Problems 235 Convergence Conditions 237 Convergence Rate and Stability 241 Numerical Examples Algorithms Using Non-Smooth Bounding Procedures for Problems with Lipschitz Derivatives 249 Bounding Procedures with Simple Estimates of the Lipschitz Constant 249 Using Local Tuning for Obtaining More Precise Non-Smooth Auxiliary Functions 252 Convergence Conditions Methods Using Smooth Bounding Procedures 266 Smooth Bounding Procedures 266 A General Description of the Methods Using Smooth Bounding Procedures 270 Algorithm Using the Exact A Priori Given Global Lipschitz Constant 274

6 Contents xi Algorithm Adaptively Estimating the Global Lipschitz Constant During the Search 274 Algorithm Adaptively Estimating the Local Lipschitz Constants 277 Convergence Conditions 278 Numerical Examples Local Tuning and the Relationship between the Information and Geometrie Approaches 288 Convergence Conditions and Numerical Examples Fast Finding the First Root of an Equation by the Methods Using Smooth Bounding Procedures 295 Filters as an Example of Applications Where the Problem Arises 298 Description of the Algorithms 302 Convergence Analysis 309 Numerical Experiments 312 Part Two GENERALIZATIONS FOR PARALLEL COMPUTING, CONSTRAINED AND MULTIPLE CRITERIA PROBLEMS PARALLEL GLOBAL OPTIMIZATION ALGORITHMS AND EVALUATION OF THE EFFICIENCY OF PARALLELISM From Fast Sequential Methods towards Non-Redundant Parallel Algorithms Information Algorithm with Parallel Trials 324 Decision Rules of the Information Algorithm with Parallel Trials 325 Convergence Conditions 328 Estimates of the Efficiency of Parallelism Parallel Method for Solving Problems with the Objeetive Functions Satisfying a Generalized Lipschitz Condition 343 Decision Rules of the Method 344

7 xii GLOBAL OPTIMIZATION WITH NON-CONVEX CONSTRAINTS Convergence Conditions of the Parallel Algorithm 347 Efficiency Evaluation 352 Numerical Examples Parallel Algorithm for Solving Problems with Lipschitz Derivatives 360 Description of the Algorithm 361 Convergence Conditions 363 Efficiency of Parallelization 368 Numerical Examples 375 GLOBAL OPTIMIZATION UNDER NON-CONVEX CONSTRAINTS - THE INDEX APPROACH Problems with Partially Defined Objective Function and Constraints Reduction to Core Unconstrained Problem Index Method of Global Optimization Convergence Conditions e-reserved Solutions and Acceleration of Search 396 -Reserved Solutions and Convergence Properties 396 Reserves and the Rate of Convergence 403 Index Method with Adaptive Reserves Local Tuning for Solving Problems with Non-Convex Constraints 409 Description of the Algorithm 410 Sufficient Conditions of Global Convergence 413 Numerical Experiments 415 ALGORITHMS FOR MULTIPLE CRITERIA MULTIEXTREMAL PROBLEMS Multiobjective Optimization and Scalarization Techniques 419 Statement of the Problem 419 Scalarization Technique 421

8 Contents Xlll 7.2 Global Search Algorithm for Multicriteria Problems Multiple Criteria Problems with Non-Convex Constraints 431 Part Three GLOBAL OPTIMIZATION IN MANY DIMENSIONS. GENERALIZATIONS THROUGH PEANO CURVES PEANO-TYPE SPACE-FILLING CURVES AS MEANS FOR MULTIVARIATE PROBLEMS Peano Curves and Multidimensional Global Optimization 445 Space-Filling Curves and Reduction of Dimensionality 445 Algorithm for Unconstrained Global Search in Many Dimensions 453 Local Refinement of the Best Current Estimates 461 Optimization over the Cube with Cavities Built of Subcubes 463 Search for the Global Minimizer Yielding the Known Optimal Value Approximations to Peano Curves 467 Adjacent Subcubes 467 Numeration in the First Partition 468 Numeration in the Second Partition 471 Linking Numerations in Subsequent Partitions 475 Approximation by Centers of the Mth Partition Subcubes 484 Piecewise-Linear Approximations to Peano Curves 485 Peano Curves Versus Spirals and TV Evolvents 490 Non-Univalent Peano-like Evolvents 492 Standard Routines for Computing Approximations to Peano Curves Index Scheme for Multidimensional Constrained Problems 511

9 GLOBAL OPTIMIZATION WITH NON-CONVEX CONSTRAINTS Reduction to One Dimension 511 Multivariate Index Method 513 Convergence Conditions Multicriteria Scheme in Many Dimensions Peano Curves and Local Tuning for Solving Multidimensional Problems 541 MULTIDIMENSIONAL PARALLEL ALGORITHMS Parallel Multidimensional Information Algorithm Parallel Multidimensional Information Algorithm with Adaptive Local Tuning Parallel Characteristic Algorithms 566 Class of Parallel Characteristic Global Optimization Algorithms 566 Convergence of Parallel Characteristic Algorithms 569 Conditions of Non-Redundant Parallelization 580 Numerical Examples Parallel Asynchronous Global Search and the Nested Optimization Scheme 590 Nested Optimization Scheme and Parallel Computations 590 Asynchronous Parallel Algorithm for Univariate Global Optimization Problems 595 Convergence and Non-Redundancy Conditions 597 Numerical Examples 604 MULTIPLE PEANO SCANNINGS AND MULTIDIMENSIONAL PROBLEMS Metrie Properties in One and Many Dimensions: Multiple Shifted Scannings 611 Reduction to One Dimension and Retaining the Property of Nearness 611 Multiple Scanning 613

10 Contents xv Metrie Properties of Multiple Scannings Algorithm for Global Multidimensional Constrained Problems Employing Multiple Scannings 621 Index Method with Multiple Scannings 621 Convergence Properties Implementation of Global Optimization Schemes with Multiple Scannings on Multiprocessor Systems 633 Reduction to a Family of Linked Univariate Problems 633 Parallel Scheme and Search Algorithm 635 Convergence Conditions 642 REFERENCES 651 LIST OF ALGORITHMS 679 LIST OF FIGURES 683 LIST OF TABLES 693 INDEX 697