Heatmap Visualisation of Population Based Multi Objective Algorithms

Transcription

1 Heatmap Visualisation of Population Based Multi Objective Algorithms Andy Pryke 1, Sanaz Mostaghim 2, Alireza Nazemi 3 1 Cercia, School of Computer Science / 3 Department of Civil Engineering University of Birmingham Birmingham, UK {A.N.Pryke,A.Nazemi@cs.bham.ac.uk} 2 AIFB Institute, University of Karlsruhe, Karlsruhe, Germany {mostaghim@aifb.uni-karlsruhe.de} Abstract. Understanding the results of a multi objective optimization process can be hard. Various visualization methods have been proposed previously, but the only consistently popular one is the 2D or 3D objective scatterplot, which cannot be extended to handle more than 3 objectives. Additionally, the visualization of high dimensional parameter spaces has traditionally been neglected. We propose a new method, based on heatmaps, for the simultaneous visualization of objective and parameter spaces. We demonstrate its application on a simple 3D test function and also apply heatmaps to the analysis of realworld optimization problems. Finally we use the technique to compare the performance of two different multi-objective algorithms. Keywords: Visualization; Multi-objective optimization; Multi-objective algorithms; Evolutionary algorithms; Real-world applications 1 Introduction Visualization of the optimal solutions plays a very important role in multiobjective optimization (MO). In MO with conflicting objectives there is no single optimum, and search methods return a set of solutions from which one must be selected. In order to select the solution, a decision maker usually needs to visualize the discovered solutions in the objective space. This can be done using scatterplots of the objective space if there are only 2 or 3 objectives. Visualization is also used to show the quality of the solutions. A good set of optimal solutions should contain well-distributed converged solutions along the Pareto front. Almost every new algorithm in MO is tested on several 2- and 3- objective problems, and beside numerical measurements, the obtained solutions are illustrated in objective space plots. This illustration is very valuable for many applications in science and industry as domain experts get information about the whole set of optimal solutions. Also, understanding the algorithm behavior is easier with this view.

2 Visualizing the objective space directly is possible for 2 and 3 objective spaces whereas for higher number of objectives, the solutions are only be evaluated by metrics. Metrics and numerical measures hide too much information and only consider the objective space. On the other hand, in almost all of the proposed methods and applications in MO, there is a high (>3) number of parameters which are not being visualized. However, showing the parameter values and visualizing them has a great impact on the decision making process. Here, we investigate several visualization methods in order to visualize parameters and objective values of a set of optimal solutions. We note previous application of objective plots, Self Organizing Maps (SOM) [2], and Distance and Distribution Charts[1]. We also introduce a new application of pre-existing visualization methods to population based algorithms: Heatmap visualization, which has previously been used mainly for the visualization of biological data. When evaluating visualization methods, it is wise to consider what features of the system we may wish to reveal. The possibilities for population based MOEAs include: the diversity and convergence of the solutions in both objective and parameter space; the relationship between parameter and objective values; A comparison between different runs or algorithms; Identification of clusters of solutions in either the parameter or objective spaces; Dynamically illustrating the progress of an algorithm towards its optimization goals. This introductory section gives some background on Multi-objective Optimization and previous visualization methods which have been applied to it. To familiarize the reader with our methods, we make use of a 3-objective test problem defined in Section 2. We then apply heatmaps to this problem in sections 3. Sections 4 applies heatmaps on a real world application in mineralogy. A second application in hydrological modeling shows the use of our methods for the comparison of the behavior of algorithms (Section 5). 1.1 Multi-objective Optimization Problems (MOPs) A Multi-objective Optimization Problem (MOP) contains several objective functions, which are to be optimized at the same time: minimize r f ( x r ) = ( f 1 ( x r ),L, f m ( x r )) r subject to e ( x r ) < 0 r x S involving m 2 (normally conflicting) objective functions f i : R n R m that we want to minimize simultaneously. The parameters x r = (x 1,L, x n ) r belong to the feasible region S. The feasible region is formed by constraint functions e ( x r ). We call the image of the feasible region feasible objective region. Its elements are called objective vectors and they consist of objective values. Many MOP have conflicting objectives, i.e. it is not possible to find a single solution that would be optimal for all the objectives simultaneously. In this case, we aim to find some optimal solutions where none of their objective values can be improved without deterioration of at least one of the other objective values.

3 These solutions are called Pareto-optimal solutions. A solution r x 1 is called Pareto-optimal, if there is no other solution that dominates 1 it. The Pareto-optimal set in the objective space is called Pareto-optimal front. The Pareto-rank of a solution is a count of the number of other solutions which dominate it. We note that many of the visualizations presented below can be applied either to a complete solution set, or to a subset of solutions selected using Pareto-rank. 1.2 Visualization of MOPs Objective space plots: The most commonly used visualization of the obtained solutions is to plot the objective values in the objective space plots. Figure 1 shows an example of the solutions of a 2-objective problem. The dotted line and small circles show the Pareto-front and the obtained solutions respectively. A good algorithm must obtain solutions with both good diversity and good convergence. Fig. 1 An example of a 2-objective plot This method is very useful, but cannot deal with more than 3 objectives. For a large set of optimal solutions, these plots are not accurate enough and numerical measures are required. Distance and distribution (DD) charts: Ang et al. [1] propose two separate charts which plot a set of non-dominated solutions using their distance to an approximate Pareto front and the distance between each other. This method requires the approximate Pareto front to be found, which is not always straightforward or even possible. These plots are based solely on the objective values and parameters values are not considered. 1 A parameter x r r 1 is said to dominate x 2 if x r 1 is not worse than x r 2 in all objectives and it is strictly better in at least one objective. Among a set of solutions, the non-dominated set of solutions contains those solutions that are not dominated by any member of the set.

4 Fig. 2. Distance and Distribution Charts for solution sets generated by two algorithms. Self-Organizing Map (SOM) method: Obayashi and Sasaki [2] use SOM to reduce the dimensionality of parameters and objectives for visualization. SOM is an unsupervised neural network method which generates a mapping of the high dimension data into cells in fewer, usually 2, dimensions. This has been used to visualize a set of relatively large set of non-dominated solutions. Figure 3 shows an example. To facilitate the analysis of SOM and the data, similar cells on the map are clustered into groups. In this approach, the parameter space itself is not visualized, but information about parameters is provided by examples of designs overlaid at appropriate points on the 2D map. Fig. 3 Self Organizing Map for aircraft part designs evolved using multiple objectives 2 An Example Multi-Objective Optimization Problem In order to introduce the visualization techniques, they are tested on a 3-objective version of the test function known as DTLZ2, the m objective sphere problem. This

5 function is defined in table 1. [3]. In order to observe the quality of solutions, we produce solutions with different diversity and convergence using a multi-objective optimization method from [4]. We produce multiple solution sets, known as good, bad-conv, bad-div-1 to bad-div-5. bad-conv refers to a set of solutions with bad convergence and bad-div-1 to bad-div-5 refer to sets with bad diversity and spread of solutions along the Pareto front. Figure 4 Shows the objective plot of the results of DTLZ2, in 3 dimensions. We have chosen this 3-objective function to allow us to illustrate the other visualization methods using consistent data, before applying the techniques to examples of real optimization problems. Table 1. Test functions Test function constraints r r r DLTZ2 π π f1( x) = (1 + g) cos( x1 2 ) cos( x2 2 ) x i [ 0,1] r r r π π f2 ( x) = (1 + g) cos( x1 2 ) sin( x2 2 ) n =10 r r π f ( x) = (1 + g)sin( x ) 3 g = n ( x i i= m r 2 0.5) 1 2 Fig. 4 Results of the DLTZ2 test problem with different diversities. Crosses indicate solutions from the good solution set while circles are solutions from the six bad sets.

6 3 Heatmaps Heatmap visualization is a technique which is most often applied to data gathered from microarrays. Microarray analysis [5] is a biological technique used to investigate the activation levels of large numbers of genes within cell samples. A typical dataset in this application consists of perhaps a dozen samples and hundreds if not thousands of genes. However, there is nothing intrinsically biological about heatmaps, and the technique can be applied to other data with a few modifications. The results of a MOEA are a population of solutions, with each solution consisting of values for a set of parameters and an associated scores on multiple objectives. In computing terms this is a two-dimensional array, the dimensions being solution ID and parameter/objective. In the heatmap in figure 6 each row is a solution and each column is a parameter or objective. The color of the cells represents the value of a parameter or objective for a particular solution. In figure 6 columns show parameters in numerical order, followed by objectives also in numerical order. The solutions have been ordered by using a hierarchical clustering method which keeps solutions with similar objective values together. The computational complexity of calculating a heatmap depends on the clustering method, this is typically O(N r 2 +N c 2 ) with N c and N r being the number of rows and columns respectively. One immediately noticeable feature is the high information density possible with heatmaps. Unlike most other visual representations, all the information from the original data is presented, the only loss being due to the limited number of shades/colors differentiable by the human eye. When taken to its extreme, a heatmap using 1 pixel per cell could represent nearly 2 million values simultaneously on a screen with a resolution of 1600x1200, though the limitations on a viewer s ability to perceive and successfully interpret such large heatmaps are untested. Figures 6a and 6b presents our exemplar solution sets in heatmap form. There are three types of column clearly visible in the plots. The parameters p1 and p2, which typically take on a wide range of values and correlate closely with the objectives; the remaining parameters p3..p10, which generally tend to middling values; and the three objectives which tend to lower values but can also be seen to conflict i.e. we do not get low values for all three objectives. We now discuss each of the plots in turn, indicating the features of interest. In the good solution set plot (Figure 6a) we note that the objective values tend to be low, but that they conflict. In the cases where two of the objectives are particularly low, the remaining one takes a higher value. A correlation between the first two parameters and the objectives can also be observed. The values for p3..p10 are generally middling and quite similar, as we d expect from examining the equations defining the objectives, which indicate the optimum value of p3..p10 is always 0.5, independent of other parameters.

7 Good Solution Set Key Highest Values Middle Values Lowest Values Fig. 5a Heatmap of Good Solution Set. The 1 st ten columns are parameter values, the final three are objective values. Data is normalized in such a way that colors are comparable with Fig 5b. Comparing Figure 5a to Figure 5b, the bad convergence solution set shows much less consistency in p3..p10, than the good set. However, it is interesting to note that the correlation between p1,p2 and the objectives is visible, indicating that the optimization process has begun to work on those parameters. As we d expect in a badly converged solution set, there are some quite high values for the objectives, particularly in cases where the other objectives are low. Bad Div 1 solutions can be seen to optimize two objectives at once, but the third objective always takes a high value. In comparison with the good set, there are more extreme values for p1 and p2, and p3..p10 show less convergence. The bad div 2 and bad div 3 heatmaps shows the only one of the objectives is optimized at a time. Where one objective has low values, the other two have high values. This behavior can also be seen in the 3d scatterplot in Figure 4 The bias towards o1 and o3 can be see for Bad Div 4. Another interesting feature is that the values of p4..p8 have a high variance in solutions where p2 is very high (bottom of diagram). In these case, o2 also takes on a high value. In bad div 5 we observe mainly middling values with little diversity for both parameters and objectives. Again, this can be seen to tally with Figure 4.

8 Bad-Conv Solution Set Bad Div-1 Bad Div-2 Bad Div-3 Bad Div-4 Bad Div-5 Fig. 5b Heatmaps of bad solution sets. The key can be seen in Figure 5a

9 Fig.6 Heatmap of good solution set. Trees indicating variable and solution clustering can be seen at the top and left of the diagram respectively. The key shown indicates the colors/shades associated with values in the cells. In order to make full use of the color information the data has been normalized across the good set. Colors cannot therefore be compared directly with those in Figure 5a and 5b The correlation between p1 and o3 and that between p2 and o2 is made plainer when they are placed side by side as in figure 6. The disadvantage of re-ordering like this is that the labels at the bottom of the diagram need to be consulted in order to determine which parameter or objective is represented by a particular column

10 5 Multi-component chemical systems in Mineralogy The proposed visualization methods have been applied to the results of a realworld optimization problem in quantifying the thermodynamic parameters of a multicomponent silicate melt system. This problem has been solved by a Bi-level optimization method in [6]. Here we visualize the last set of obtained non-dominated solutions and show the 13 parameters of the upper-level. In this problem the objectives are: 1) Minimize the difference between the free energy of solid and liquid. 2) Minimize the difference between obtained temperature at which solid and liquid can coexist and the absolute temperature T recorded in the experiments. Figure 7 and 8 show the plain-heatmaps-by-objective of two sets of solutions (set1 and set2) from different runs and Figure 9 illustrates the objective space plot. From Figure 9, we conclude that both of these sets have relatively close objective values. In the following, we analyze the heatmap plots. In these plots, parameters 1, 3, 9 and 2, 4, 10 refer to enthalpy and entropy of the components where parameters 5 to 8 indicate the uncertainty in measuring the free energy of solid in the experiments. The other parameters are related to the coefficients in the thermodynamic model. Figure 8 shows the heatmap of set2 and indicates if we select a certain constant value for uncertainties, almost all of the solutions on the front have the same enthalpy and entropy values. This can be observed by comparing the rows for the extreme solutions 12 and 6. This indicates that that the solutions, despite having good objective values, are located in a local optimum, where the uncertainty parameters are all equal and constant. But for lower values of uncertainties like shown in Figure 7 (heatmap of set1), there is a distinct difference between the parameters of the extreme solutions on the front (solutions 3 and 5). This is valuable knowledge if we want to select a solution from the middle of the front, we have to select the parameters in the middle of their ranges (solution 8). This analysis indicates, although the objective values of the two sets are very close to each other, parameters are located in different areas of the search space. This is very important for designing a proper thermodynamic model for mineralogists and for selecting an appropriate solution from the non-dominated set.

11 Fig. 7. Volcano Model. Solution Set 1. See Fig. 5a for key. Fig. 8. Volcano model. Solution Set 2. See Fig. 5a for key.

12 Fig. 9. Objective space plot for both sets of volcano model solutions. 6 Application in multi-objective calibration of hydrologic models The conversion of rain and snow to runoff has long been studied by engineers to design hydraulic systems and by scientist to develop an understanding of the process involved [7]. Most of the practical rainfall-runoff models contains some parameters that do not have any explicit physical interpretation and are set using an objective optimization problem. Although most of the previous attempts concentrated around single objective formulation of calibration [9], recent practical experiences suggest that a single objective functions are often inadequate to properly measure all of the characteristics of the observed data [10]. We are also aware that the behavior of the system being modeled has a number of different modes[11,12], depending on the recent history of precipitation. We would like to determine parameters which model the system well over all modes. By measuring model accuracy separately during these different modes, the calibration problem is converted to a multi-objective optimization problem. The result of this optimization problem will be a set of pareto parametric values, in which there is no solution better than the other in regarding to all performance across all modes.

13 In this study, a 5-parameter conceptual model was applied for modeling the rainfall-runoff process in Leaf River Basin, USA. 11 years of daily data ( ) was used for multi-objective calibration of the model. Four objective functions are used, measuring the Root Mean Squared Error (RMSE) of model predictions Driven High (FDH), Driven Low (FDL), Non-driven Quick (FNQ), and Non-driven Slow (FNS) flows. Two algorithms, NSGA-II [14] and Multi-Objective Shuffled Complex Evolution Metropolis algorithm (MOSCEM) [13], with function evaluations were applied for calibration of the model. In order to compare the results of these two algorithms, the heatmap visualization technique was applied. Figure 10 shows the heatmaps of archives related to MOSCEM and NSGA-II. As in previous examples, the objective function values are used to cluster the solutions. The first five columns from left are represent parametric values and the next four are represent objective functions. Comparing the number of solutions in the archives (941 solutions for NSGA-II and 88 for MOSCEM), initially it was supposed that NSGA-II would have a finer texture. However as it appears, the textures of both heatmaps are almost the same, showing that NSGA-II converged to limited number of distinct solutions for each model parameter. Additionally, we note that the two algorithms have converged to different regions of the parameter space. For the first parameter, MOSCEM generally converges to values which are smaller and more diverse than NSGA-II. For parameter 2 MOSCEM has again used a wider parametric region than NSGA-II. For parameter 3 on the other hand, NSGA-II has found some low values unused by MOSCEM, though MOSCEM covers other values which NSGA-II did not find. For parameter 4, it is quite obvious that NSGA-II converges almost completely to two extreme values, whereas MOSCEM produces a broad spectrum of results. For parameter 5, the differences are not so great, but NSGA-II has located some higher values. Looking to the columns related to objective functions, they reveal that for the first objective function both algorithms can find many good results, but NSGA-II allowed some bad results which trade off against objective 4. For second objective function, convergence is similar, however NSGA-II is again more tolerant of bad results. For third objective function the results of both algorithms are very similar. For the final objective function, it is quite obvious that NSGA-II can produce much better results than MOSCEM. However, the best results that NSGA-II could find for objective 4 are some of the worst for the other objectives. Looking at the four objectives together it is clear that the calibration process of the applied hydrologic model is inherently a multi-objective task, and that the applied model can not represent the whole hydrologic behavior of the catchment with a single parametric set or even a parametric region.

14 (a) MOSCEM (b) NSGA-II Fig. 10. The heatmaps of solution archives for hydrological model parameterization 7 Conclusion and Future Work The heatmap is a novel and interesting visualization method which can provide detailed insight into the multiple solutions generated by population based multi objective algorithms. The high information density of heatmaps allows whole populations of solutions to be visualized. One application for heatmaps is to gain greater insight into the behavior of particular MO algorithms, and to compare the performance of algorithms. This is of particular interest to the MO algorithm community. Another is the exploratory analysis of the parameter space and its relationship to the objective space. Domain experts and those applying MO algorithms to real-world problems will find this aspect particularly useful. Future work includes: the developments of color-scales and grayscales appropriate for colorblind users and grayscale printing; Ordering rows consistently between datasets to ease comparison; experimentation to understand the relationship between heatmaps representations and Pareto ranking; and development of an easy to use toolkit for MO researchers. References 1. Kiam Heong Ang, Gregory Chong and Yun Li. Visualization Technique for Analyzing Non-Dominated Set Comparison, in Lipo Wang, Kay Chen Tan, Takeshi Furuhashi, Jong-Hwan Kim and Xin Yao (editors), Proceedings of the 4th Asia-

15 Pacific Conference on Simulated Evolution and Learning (SEAL'02), pp , Vol. 1, Nanyang Technical University, Orchid Country Club, Singapore, November Shigeru Obayashi and Daisuke Sasaki, Visualization and Data Mining of Pareto Solutions Using Self-Organizing Map, Second International Conference on Evolutionary Multi-Criterion Optimization (EMO 2003), Faro, Portugal, LNCS 2632, Springer-Verlag Berlin Heidelberg 2003, pp , April K. Deb, L. Thiele, M. Laumanns, and E. Zitzler, Scalable multi-objective optimization test problems, in Proc. Congr. Evol. Comput., D. B. Fogel, M. A. El- Sharkawi, X. Yao, G. Greenwood, H. Iba, P. Marrow, and M. Shackleton, Eds., May 2002, vol. 1, pp S. Mostaghim and J. Teich., Strategies for finding good local guides in multiobjective particle swarm optimization. In IEEE Swarm Intelligence Symposium, pages 26 33, Indianapolis, USA, The Chipping Forecast II, Nature Genetics Special Issue, December 2002, Volume 32, No 4s 6. Halter, W.; Mostaghim, S., "Bilevel Optimization of Multi-Component Chemical Systems Using Particle Swarm Optimization," Evolutionary Computation, CEC IEEE Congress on, vol., no.pp , July O Loughlin, G., Huber, W., Chocat, B., Rainfall-runoff process and modeling, Journal of Hydraulic Research, VOL. 34, NO. 6, pp , Furundzic, D.,: Application example of neural networks for time series analysis: rainfall-runoff modeling, Signal Processing, VOL. 64, pp , Gan, T., Y., Biftu, G., F., Automatic calibration of conceptual rainfall-runoff models: optimization algorithms, catchment conditions, and model structure, Water resources research, VOL. 32, NO. 12, pp , Vrugt, J. A., Gupta H. V., Bouten, W., and Sorooshian, S., A shuffled complex evolution metropolis algorithm for optimization and uncertainty assessment of hydrologic model parameters, Water Resources Research, VOL. 39 (8), Boyle, D. P., Gupta, H. V., and Sorooshian S., Toward improved calibration of hydrological models: Combination the strengths of manual and automatic methods, Water Resources Research, VOL. 36(12), , Wagener, T., Wheater, H. S., On the evaluation of conceptual rainfall-runoff models using multiple-objectives and dynamic identifiability analysis, In Littlewood, I. (ed.), Continuous river flow simulation: methods, applications and uncertainty, British Hydrological Society, Occasional paper, No. 13, Wallingford, UK, pp.45-51, Vrugt, J. A., Gupta H. V., Bastidas, L. A., Bouten, W., and Sorooshian, S., Effective and efficient algorithm for multi-objective optimization of hydrologic models, Water Resources Research, VOL. 39 (8), Deb, K., Pratap, A., Agrawal, S. and Meyarivan, T. (2000). A fast and elitist multiobjective genetic algorithm: NSGA-II. Technical Report No Kanpur: Indian Institute of Technology Kanpur, India.