Evolutionary Algorithms

Size: px

Start display at page:

Download "Evolutionary Algorithms"

Neil Thompson
5 years ago
Views:

1 A Hybrid Optimization Algorithm With Search Vector Based Automatic Switching Eric Inclan and George S. Dulikravich Florida International University, Miami FL Presented at WCSMO10, Orlando, Florida, May 15-19, 19, November

2 Evolutionary Algorithms Main Characteristics Population of Design Vectors User-Defined Parameters Can be tailored to specific problems Rigid Formula Population update process introduces search bias Random number generation for increased flexibility Little or no adaptation 3 November

3 Single Problem Multiple Topologies 3 November

4 Hybrid Optimization Purpose Increase Performance Robustness Speed Accuracy Black-Box Optimization Software Approaches Merger [1] Teamwork [2] Competition (Automatic Switching) [2, 14] 3 November

5 No Free Lunch Limitations of Black-Box Optimizer No algorithm better than another on average [3] In practice, rule is relaxed [4] Limitations of Benchmarking Functions Can be taken advantage of [5] Why Try? Discounts are good! 3 November

6 Benchmarking Test Cases Schittkowski& Hock Collection [12, 13] 305 test cases 2-dimensional to 100-dimensional Linear and nonlinear objective functions Unconstrained and up to 50 constraints Linear and nonlinear Equality and inequality Constraints Enforcement Penalty Function Equality Constraints: g(x) > 1e-7 3 November

7 Search-Vector Based Hybrid Optimization Algorithm Selects Search Direction Competitively Selects a Constituent Algorithm Differential Evolution Modified best/2/bin (BST) [6] Modified Donor3 (DN3) [7] Particle Swarm Standard (PSO) [8] With Random Differencing (PRD) [9] Modified Quantum PSO (MQP) [10] Cuckoo Search (CKO) [11] 3 November

8 Search Vectors (1/4) Current Global Best (GB) 3 November

9 Search Vectors (2/4) Population Weighted Average (PWA) Individual Best Weighted Average (IBWA) 3 November

10 Search Directions (3/4) Negative of Current Global Worst (NGW) Upper bounds of domain Lower bounds of domain Design Vector 3 November

11 Search Vectors (4/4) Population Centroid (PC) Projected Global Best (PGB) Modeled after simple concepts, such as: Projectile Motion Equation (Constant Acceleration) Position of Particle at t=3 Given x 0, x 1, and x 2 Average of GB, PWA, and NGW (AV1) 3 November

12 Search Vector Selection Options Blind Approach Comparative Approach (Automatic Switching) VS. 3 November

13 Constituent Algorithm Prediction Create Temporary Population Without Evaluating Objective Function Constituent Algorithm 1 Constituent Algorithm 2 3 November

14 Basic Constituent Algorithm Selection Minimal Distance Maximum Dot Product Minimal Scaled Distance SELECT Constituent Algorithm 1 GB Constituent Algorithm 1 Constituent Algorithm 2 3 November

15 Alternative Constituent Algorithm Selection Method Calculate Centroid of Temporary Population Select Algorithm with Fittest Centroid CA1 CA2 3 November

16 Advanced Constituent Algorithm Selection Multi-Objective Approach (Automatic Switching) Minimal Distance Between Endpoints Fittest Temporary Population Centroid GB CA1 CA2 3 November

17 Blind Hybrid Optimization Method Results on 305 Test Cases of Schittkowski (1/3) Minimum Distance Maximum Dot Product Scaled Distance DE PSO GB PWA NGW IBWA GB PWA NGW IBWA GB PWA NGW IBWA Mean 15.3% 25.4% 33.9% 5.1% 4.1% 7.1% 17.6% 5.4% 4.1% 5.1% 22.0% 4.7% 3.7% 4.7% Std. Dev. 12.2% 44.7% 15.3% 8.5% 12.2% 8.1% 6.1% 10.2% 5.4% 6.8% 5.1% 9.8% 9.2% 6.8% Best 39.0% 18.6% 71.5% 8.8% 4.7% 12.5% 48.1% 4.4% 5.1% 8.8% 48.8% 4.4% 4.7% 8.8% Worst 42.4% 12.5% 21.7% 3.1% 3.7% 2.7% 13.6% 11.5% 2.4% 3.4% 15.9% 10.5% 3.7% 4.7% Table 1: Comparison of Algorithm Accuracy Minimum Distance Maximum Dot Product Scaled Distance DE PSO GB PWA NGW IBWA GB PWA NGW IBWA GB PWA NGW IBWA Mean 17.6% 4.1% 38.3% 1.4% 2.0% 2.7% 14.6% 2.0% 2.0% 3.7% 16.6% 1.0% 3.1% 1.4% Std. Dev. 6.1% 62.4% 5.8% 9.5% 2.0% 6.8% 1.4% 1.7% 2.0% 2.7% 0.7% 1.7% 1.0% 4.7% Best 10.2% 3.4% 19.0% 1.7% 8.8% 3.4% 16.6% 7.1% 6.1% 3.1% 13.6% 10.2% 6.8% 3.4% Worst 14.6% 13.9% 13.2% 2.4% 5.8% 2.7% 4.7% 10.2% 7.1% 4.7% 6.1% 11.9% 6.8% 6.4% Table 2: Comparison of Algorithm Convergence Rates 3 November

18 Blind Hybrid Optimization Method Results on 305 Test Cases of Schittkowski (2/3) GB PWA NGW IBWA Minimum Distance 73.22% 11.86% 6.10% 20.34% Maximum Dot Product 85.76% 9.83% 5.08% 9.83% Scaled Distance 84.41% 9.83% 5.76% 10.17% Table 3: Comparison of Accuracies of Hybrid Algorithms Min. Dist. Max. Dot Scaled Dist. GB 49.83% 30.51% 29.49% PWA 74.24% 18.64% 14.58% NGW 40.34% 35.25% 31.19% IBWA 60.34% 23.73% 24.41% Table 4: Comparison of Accuracies of Hybrid Algorithms 3 November

19 Blind Hybrid Optimization Method Results on 305 Test Cases of Schittkowski (3/3) Minimum Distance Maximum Dot Product Scaled Distance DE PSO GB AV1 PGB GB AV1 PGB GB AV1 PGB ALT Mean 15.3% 21.7% 28.8% 6.1% 19.0% 12.5% 7.5% 7.8% 13.2% 6.4% 9.2% 22.7% Std. Dev. 12.2% 43.7% 14.9% 15.3% 14.2% 5.8% 9.8% 5.8% 4.7% 10.5% 6.1% 18.0% Best 39.3% 18.6% 68.1% 7.8% 44.1% 45.8% 7.5% 24.1% 46.8% 7.8% 26.8% 41.0% Worst 43.1% 12.9% 19.7% 3.1% 12.9% 13.9% 4.1% 8.8% 14.2% 3.7% 7.8% 9.2% Table 5: Comparison of Algorithm Accuracy Minimum Distance Maximum Dot Product Scaled Distance DE PSO GB AV1 PGB GB AV1 PGB GB AV1 PGB ALT Mean 17.6% 3.7% 35.6% 2.0% 1.7% 15.3% 2.0% 1.4% 14.2% 2.0% 1.4% 4.4% Std. Dev. 6.1% 58.6% 4.1% 4.7% 4.4% 0.7% 5.1% 1.7% 0.7% 4.7% 2.0% 1.7% Best 9.5% 4.1% 23.7% 3.4% 3.1% 21.7% 2.7% 2.7% 19.3% 2.4% 2.0% 3.7% Worst 15.6% 15.3% 10.8% 3.1% 5.8% 5.8% 3.4% 3.4% 6.4% 3.7% 3.4% 5.8% Table 6: Comparison of Convergence Rates 3 November

20 Usage of Each Constituent Algorithm in Blind Hybrid Optimization Methods on 305 Schittkowski Test Cases GB NGW PWA IPWA 3 November

21 Special Case 2D Test Case GB No Constraints 150 F(X1,X2) X X November

22 Automatic Switching HOA Results (1/9) Blind Minimum Distance DE PSO GB AV1 PGB IBWA AUTO Mean 14.4% 19.4% 24.4% 5.7% 15.1% 12.0% 56.9% Std. Dev. 11.7% 46.2% 16.1% 25.8% 12.7% 17.7% 22.4% Best 37.8% 17.1% 66.9% 6.4% 44.1% 21.4% 77.3% Worst 50.8% 14.0% 23.4% 15.4% 17.4% 8.7% 8.0% Table 7: Comparison of Algorithm Accuracy Blind Minimum Distance DE PSO GB AV1 PGB IBWA AUTO Mean 20.1% 7.0% 18.7% 3.7% 1.7% 5.4% 45.8% Std. Dev. 9.4% 62.5% 3.3% 6.4% 8.0% 8.4% 3.7% Best 19.7% 4.7% 18.4% 9.4% 4.7% 6.0% 39.5% Worst 24.4% 17.1% 13.0% 17.1% 7.0% 10.0% 5.0% Table 8: Comparison of Algorithm Convergence Rates 3 November

23 Automatic Switching HOA Results (2/9) Highest Accuracy and Speed Highest Accuracy Only Blind Minimum Distance DE PSO GB AV1 PGB IBWA AUTO 4% 14% 15% 3% 5% 4% 54% 14% 19% 24% 6% 15% 12% 57% Table 9: Comparison of Algorithm Relative Accuracy and Speed 3 November

24 Automatic Switching HOA Results (3/9) AUTO 3 November

25 Automatic Switching HOA Results (4/9) 4D Test Case 3 November

26 Automatic Switching HOA Results (5/9) 4D Test Case 3 November

27 Automatic Switching HOA Results (6/9) 10D Test Case 35 Constraints 3 November

28 Automatic Switching HOA Results (7/9) 30D Test Case Unconstrained 3 November

29 Automatic Switching HOA Results (8/9) 50D Test Case 3 November

30 Automatic Switching HOA Results (9/9) 100D Test Case Unconstrained 3 November

31 Conclusions Blind HOA Useful tool for examining search directions Useful tool for examining optimization algorithm behavior Advanced HOA Solves a broader range of problems better than Blind HOA and Constituent Algorithms Better prediction of search direction with minimal additional function evaluations Most robust Better accuracy and speed in many cases 3 November

32 Future Work Include More Constituent Algorithms Develop Better Search Vectors Incorporate More Topology Information Undulation/Topology Fluctuation Extend to Multiple Objectives Develop Discrete Analogy 3 November

33 References 1/2 [1] Gong, W., Cai, Z., & Ling, C. X. (2010). DE/BBO: a hybrid differential evolution with biogeography-based optimization for global numerical optimization. Soft Computing, 15(4), [2] Moral, R. J., and Dulikravich, G. S. (2008, March). Multi-Objective Hybrid Evolutionary Optimization with Automatic Switching Among Constituent Algorithms. AIAA Journal, 46(3), [3] MacReady, W. G., & Wolpert, D. H. (1997, April). No Free Lunch Theorems for Optimization. IEEE Transactions On Evolutionary Computation, 1(1), [4] Droste, S., Jansen, T., & Wegener, I. (2002). Optimization with randomized search heuristics the (A)NFL theorem, realistic scenarios, and difficult functions. Theoretical Computer Science(187), [5] Ahrari, A., Saadatmand, M. R., Shariat-Panahi, M., & Atai, A. A. (2010). On the limitations of classical benchmark functions for evaluating robustness of evolutionary algorithms. Applied Mathematics and Computation(215), [6] Inclan, E. And Dulikravich, G. S. (2013) Effective Modifications to Differential Evolution Optimization Algorithm, International Conference on Computational Methods for Coupled Problem in Science and Engineering, (eds.: Idelsohn, S., Papadrakakis, M., Schrefler, B.), Santa Eulalia, Ibiza, Spain, June 17-19, [7] Fan, H.-Y., Lampinen, J., and Dulikravich, G. S. (2003). Improvements to Mutation Donor of Differential Evolution. EUROGEN International Congress on Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems, (eds: G. Bugeda, J. A-Désidéri, J. Periaux, M. Schoenauer and G. Winter), CIMNE, Barcelona, Spain, September November

34 References 2/2 [8] Kennedy, J., and Eberhart, R. (1995). Particle Swarm Optimization. Proceedings of IEEE International Conference on Neural Networks, pp [9] Inclan, E., Dulikravich, G. S., and Yang, X.-S. (2013). Modern Optimization Algorithms and Particle Swarm Variations, Proceedings of Symposium on Inverse Problems, Design and Optimization-IPDO2013, (eds.: Fudym, O., Battaglia, J.-L.), Albi, France, June 26-28, [10] Sun, J., Lai, C. H., Xu, W., and Chai, Z. (2007). A Novel and More Efficient Search Strategy of Quantum- Behaved Particle Swarm Optimization. ICANNGA '07 Proceedings of the 8th international conference on Adaptive and Natural Computing Algorithms, pp [11] Yang, X.-S., and Deb, S. (2010). Engineering Optimisation by Cuckoo Search. Int. J. Mathematical Modelling and Numerical Optimisation, 1(4), pp [12] Schittkowski, K. (1987). More Test Examples for Nonlinear Programming Codes. Berlin: Springer-Verlag. [13] Schittkowski, K., and Hock, W. (1981). Test Examples for Nonlinear Programming Codes - All Problems from the Hock-Schittkowski-Collection. Bayreuth: Springer. [14] Dulikravich, G. S., Martin, T. J., Dennis, B. H. and Foster, N. F.: Multidisciplinary Hybrid Constrained GA Optimization, Invited lecture, Chapter 12 in EUROGEN 99 -Evolutionary Algorithms in Engineering and Computer Science: Recent Advances and Industrial Applications, (eds: K. Miettinen, M. M. Makela, P.Neittaanmaki and J. Periaux), John Wiley & Sons, Jyvaskyla, Finland, May 30-June 3, 1999, pp November

35 Description of PWA/IBWA Equation Population Size Dimensionality of Design Vector Population Index Dimension Index Unit Vector in j th direction Resultant Vector (The Weighted Average) j th component of i th Design Vector from Population - PWA: the main design vector population - IBWA: the population of individual best vectors from PSO Weight Assigned to i th Vector Based on Fitness 3 November

36 Acknowledgements This work was partially sponsored by the US Air Force Office of Scientific Research under grant FA monitored by Dr. Ali Sayir. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the US Air Force Office of Scientific Research or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for government purposes notwithstanding any copyright notation thereon. 3 November

The Development of a Hybrid Optimization Algorithm for the Evaluation and Optimization of the Asynchronous Pulse Unit

Florida International University FIU Digital Commons FIU Electronic Theses and Dissertations University Graduate School 2014 The Development of a Hybrid Optimization Algorithm for the Evaluation and Optimization