PREDICTION OF DESIGN PARAMETERS IN SHIP DESIGNING BASED ON DATA MINING TECHNIQUE BY COMBINING GENETIC PROGRAMMING WITH SELF ORGANIZING MAP

Transcription

1 PREDICTION OF DESIGN PARAMETERS IN SHIP DESIGNING BASED ON DATA MINING TECHNIQUE BY COMBINING GENETIC PROGRAMMING WITH SELF ORGANIZING MAP Kyungho Lee Department of Naval Architecture & Ocean Engineering INHA University South Korea Jonghoon Park Donggeun Kim Daesuk Kim Department of Naval Architecture & Ocean Engineering, Graduate School INHA University South Korea ABSTRACT Engineering data contains the eperiences and knowhow of eperts. Data mining technique is useful to etract knowledge or information from the accumulated eisting data. Although Korean shipyards have accumulated a great amount of data, they do not have appropriate tools to utilize the data in practical works. Most of shipyards utilize empirical formulas for the prediction of design parameters in preliminary design stage. But these formulas generated from past eisting ship data are old-fashioned, so it is not adequate to apply the formulas to new ship design for the prediction of parameters. This paper presents a machine learning method based on genetic programming (GP), which can be one of the components for the realization of data mining. Differing from neural network model which is black bo, user cannot perceive the predicted formula, GP can be seen. Users can use the generated formula in real design system. In practical cases, we don t have enough learning samples, but there are many design input parameters. Therefore, reducing the number of input parameters is essential. In order to reducing the number of input parameters, self organizing map (SOM) is adopted. By applying SOM to accumulated data, the influence of input parameters on design outputs can be found. The developed data mining system by combining GP with SOM can be powerful tool for the generation of empirical formulas to predict design parameters in ship design. KEYWORDS Data mining, Genetic Programming, Self Organizing Map (SOM), Ship Design, Machine 1. INTRODUCTION In engineering domain, the utilization of accumulated eisting data is very important problem to secure productivity. Because engineering data itself contains the eperiences and know-how of eperts. In viewpoint of the development of intelligent system, the approaches have been slowly changing from knowledge-based approach, in which knowledge is difficult to etract and represent, to data-driven approach, which is relatively easy to handle. As mentioned before, the reason why this kind of changing is that engineering data contains meaningful information such as the eperiences and know-how of eperts, so the development of data analysis and utilization method is very important. In order to support this concept, data mining or knowledge discovery in database (KDD) technique is one of the useful tools. The utilization of eisting data is very important concept in ship designing process, and most of ship design is performed by modifying the previous similar ship data. Especially, most of design parameters are determined by using empirical formulas, which are generated from eisting ship data, in preliminary ship design stage. But these empirical formulas had been made on the basis of traditional ships, such as bulk carrier, crude oil tanker and so on. So they are too old-fashioned to utilize in 67

2 building new typed high value added ships, such as LNG carrier, drill ship and so on. Generating new typed empirical formulas to adapt to high value added ships is essential issue. Unfortunately, although Korean shipyards have accumulated a great amount of data, they do not have appropriate tools to utilize the data in practical works. Data utilization tool to adapt to ship design has been developed by using genetic programming (GP) since last several years (Lee, K.H. et al., 006a; 006b). That is, polynomial GP and linear model GP were presented. They focus on how to adapt for ship designing, especially on the generation of empirical formulas to predict design parameters in preliminary ship design stage. Usually, artificial neural network (ANN) as a training system for the prediction is utilized in most engineering fields. But if the characteristics of the training data are nonlinear and discontinuous, the performance of the training result deteriorates. And also the trained results cannot be reflected in the artificial neural network; that is, ANN becomes a black-bo system. On the other hand, GP has ecellent ability to approimate with non-linear and discontinued data. Above all, GP can show the trained result as a function tree. Generally, a lot of accumulated data is needed for the training by using ANN or GP. But in a real situation in the ship designing field, we do not have enough data to utilize for the training procedure. Polynomial GP (PGP), Linear model GP (LM-GP) and combining GP (PLM-GP) were presented in previous papers. In this paper, the data mining system by integrating the three types of GP model to support predicting design parameters in ship designing process is presented. Figure 1 shows the typical concept of the developed system in this paper. In practical cases, we don t have enough learning samples, but there can be many design input parameters. Generally speaking, a lot of data is needed for generating empirical formulas to predict the relations of input parameters and output design parameters. Therefore, reducing the number of input parameters is essential issue. In order to reducing the number of input parameters, self organizing map (SOM) is adopted. By applying SOM to accumulated data, the influence of input parameters on design outputs can be found. The developed data mining system by combining GP with SOM can be powerful tool for the generation of empirical formulas to predict design parameters in ship design. Figure 1 Concept of data mining system in this paper In fore part of this paper, the developed GP programs will be introduced briefly. And then integrated data mining system and combining model with SOM are presented.. POLYNOMIAL GENETIC PROGRAMMING (PGP) Polynomials are widely used in many engineering applications such as response surface modeling (Simpson, T.W. et al., 1998; Malik, Z. et al., 1986; Alotto, P. et al., 1997; Ishikawa, T. et al., 1997; Myers, L.H. et al., 1995), since the mathematical form of a polynomial is simple, and very easy to handle. In the mechanical, electrical and electronic engineering field, the response surface method is adopted to reduce the computational cost required for analysis and simulation during the optimization design process. Thus, it is desirable to use the minimal size of samples to construct response surfaces. The classical method for attaining good polynomials is to use all-possible-regressions and stepwise regression methods (Ott, R.L., 199; Myers, L.H. et al., 1995), but there are limitations in obtaining polynomials with a desired accuracy. In the paper (Lee, K.H. et al., 006b), we tried to use Genetic Programming (Koza, J.R., 199) for generating optimal polynomials that approimate very highly nonlinear response surfaces using only minimal or very small size of learning samples. Major issues regarding finding such polynomials using GP are addressed below. First, the GP tree can easily represent the polynomial if a function set contains only +, -, * operators, and a terminal set includes only variables and constants, but it is difficult to epect for GP to generate polynomials enabling to model a nonlinear function using only such a function and terminal set. We tackled this problem by the use of low order Taylor series of various mathematical functions in a function set. That effectively makes GP produce very high order polynomials. But the generated 68

3 polynomial tends to become too comple, and it is necessary to control the size of polynomials. Our idea for easily generating a high order polynomial is to use Taylor series of mathematic functions in the function set. If high order series are taken, the GP tree produces a very comple polynomial. So, we determine to take only two or third order polynomials from whole Taylor series. We use the following function and terminal set. F i = { +,,*, g ( i = 1,...,18)} T = { one, rand, i ( i = 1,..., n)} Where, g i is a low order Taylor series as followings. g 1 : sin( ) =, g! : cos( ) = 1! g : tan( ) = +, g 4 : log(1 + ) = +! g 5 : ep( ) = , 1!!! g 6 : sinh( ) = +, g 7 : cosh( ) = 1+!! Second, the overfitting problem can be very serious, because we only have small learning samples, and there are no other kinds of additional samples. So, we introduced the EDS(Etended Data Set) method (Yeun, Y.S., et al., 1999) with the FNS(Function Node Stabilization) method. The detail description of the methods is shown at some references.. LINEAR MODEL IN GENETIC PROGRAMMING (LM-GP) The regression or function approimation finds the underlying model that can best eplain the given samples with consideration of the generalization capability. Perhaps, one of most important tasks is the selection of an appropriate functional form of a model. For instance, polynomials, sigmoid based neural networks, and radical based function networks can be considered. After the base functions are selected, a proper model form is constructed by combining these bases in a predefined way. Usually the model contains numerical coefficients or weights, which should be estimated in such a way that the learning error or other criterion is minimized by applying the optimization method. On the other hand, GP can offer a very different alternative for regression problems. GP deals with a tree-structured program, called a GP tree, whose structure evolves towards the minimization of its fitness value by using genetic operators. Unlike the traditional approimation methods, where the structure of an approimate model is fied, the structure of the GP tree itself is modified and optimized, and thus GP trees can be more appropriate or accurate approimate models. Much research has been done on GP through regression problems and system identifications (Lee, K.H., et al., 00). If the population includes several hundred trees, several hundred optimization processes are required for each generation. Computational cost is a heavy burden on the use of GP in many engineering applications such as the response surface method in optimization problems (Lee, K.H., et al., 00). Although the fast estimation technique based on linear associative memories has been proposed (Yeun, Y.S., et al., 000), this method also has difficulty in providing accurate values of the weights. Simply, the GP tree, that gives the minimum fitness value based on only the error measure of the learning samples, cannot be the one that generalizes best with the good description of the underlying regularity of samples. To select the best GP trees, statistical inference such as the Rissanen s modern minimum description length (MDL) principle (Barron, A., et al., 1998) may be required. As can be epected, it is nearly intractable to use the MDL criterion in GP because of the requirement of multiple integrations of the nonlinear GP tree. (Lee, K.H., et al., 006a) focuses on Rissanen s modern MDL for computing the fitness of the GP tree. MDL tries to find the model that is encoded with the shortest code length, and at the same time best describes all learning samples. So, the philosophical foundation behind MDL is closely related to Ockham s Razor, which insists that the simplest model with good fitting of samples is the best one. This paper investigates linear model GP (LM-GP) and MDL with a directional derivative based the smoothing (DDBS) method. The detail description can be referred from some references. 4. INTEGRATED DATA MINING SYSTEM The suggested prediction model, such as PGP, LM- GP and PLM-GP, were validated by adapting to mathematical models or real ship design problem in previous papers. 69

4 Figure Polynomial GP and its optional functions in the integrated data mining system Figure 4 Linear model GP and its optional mathematical functions in the integrated data mining system Figure Linear model GP and its optional mathematical functions in the integrated data mining system In this paper, the data mining system for a data analysis and utilization by using enhanced genetic programming with integrated model is developed. That is, the system is contrived to apply to ship design under the case that the accumulated data is not enough to make learning process. The integrated system can make fitting functions with types of GP, such as GP with high order polynomial (PGP), linear model GP with polynomial (PLM-GP) and linear model GP with math functions (LM-GP). Figure and Figure show the developed system for a data mining by using PGP and LM-GP respectively. Users can make the process of function approimation by selecting arbitrary functions that they want to use. According to selected type of GP, all sorts of options to eecute the system, such as the number of population, the number of initial tree, crossover Figure 5 Generated function tree can be convert to C code automatically in this system probability rate (Pc), mutation probability rate (Pm) and optional functions to be used for the generation of approimation function, are displayed. And the generated function tree by GP is very complicate to use in real design works as shown at Figure 4. This figure is an eample of generated GP tree. To reduce the effort of the utilization of this function, and occurrence of errors in the process of converting this function to computer program, the data mining system can convert the generated GP tree to C code in order to integrate with other program. Figure 5 shows an eample of converted C code for the generated GP tree. The system is implemented by using Microsoft Visual Studio.Net C# programming. This system is installed in DSME (Daewoo Shipbuilding & Marine Engineering) 70

5 company, one of world best shipyards in Korea, and utilized for real ship designing. 5. IMPROVED SYSTEM BY COMBINING GP WITH SELF ORGANIZING MAP In the previous chapters, data mining system to predict design parameters by generating empirical formulas with GP is presented. But if training data is not enough to eecute this system, fatal problem such like overfitting can be occurred. So the avoidance of overfitting problem is the most important issue in regression domain. Generally speaking, the number of training data is closely related to the number of input parameters. If the number of input parameters is increased, more and more training data is need. As mentioned before, we don t have enough data in ship designing process. So reducing input parameters prior to training process is required. In order to reduce input parameters, we have to know the influence of the input parameters. Self Organizing Map (SOM) is appropriate tool to find the influence of input parameters for output parameters. SOM is a well-known model of artificial neural network. It is devised by Kohonen, and is a typical method for competitive learning. In our approach, influence analysis for input parameters by SOM is performed firstly. According to the results, input parameters with low influence are removed in training data for GP. By combining GP with SOM, the performance and accuracy of training can be enhanced Eperiment of combining system for the prediction of design parameters for ship In order to validate the combining system, the prediction of block coefficient (Cb) for crude oil tanker, which is one of the most important design parameter to be estimated, is carried out. First of all, just 70 training data (not enough to perform training) are gathered, and 60 of them are used for training process, the remaining 10 data are used for test. Figure 6 is a part of training sample data. As shown at the figure, the learning sample consists of 7 input parameters, such as dead weight (DWT), length (LBP), speed, breadth, draft, depth and frude number (Fn), and output parameter, Cb. In this eperiment, the performance of prediction which can be compared by the value of RMSE (root Figure 6 A part of 70 learning samples to predict block coefficient (Cb) for crude oil tanker (7 input parameters, 1 output parameter) mean square error) is carried out for the following approaches. (1) Prediction by traditional empirical formula (Yamagata Formula) Cb = Fn Fn< 0. 4 = Fn Fn> 0. 4 = Fn> () Prediction by PLM-GP () Prediction by PLM-GP combining with SOM Although we have just 70 training data, but input parameters are 7. That is, input parameters are too many comparing with the number of training data. So we have to reduce input parameters by the evaluation of influence through SOM. The result of SOM is shown at Figure 7. By the evaluation of influence for the input parameters, Frude number (Fn) has the lowest influence value among input parameters. So Fn is removed from input parameters. Figure 8 represents the prediction result by Yamagata empirical formula. Figure 9 shows the prediction result by PLM-GP, and Figure 10 by PLM-GP combining with SOM. The cross dashed line is eact values of Cb (i.e estimated value is equal to real value). 71

6 Comparing Two Cb Values RMSE = Real Cb Values Estimated Cb Values by Yamagata's Formula Figure 7 Visualization of the influence value for each parameters in Self Organizing Map (SOM) 5.. Evaluation of the eperiments As we epect, the prediction result by Yamagata formula (Figure 8) is not good. But they have their own trend. The reason why the bad result is the formula was generated by using old-fashioned ship data. On the other hand, the result by GP trained by 60 learning data is considerably improved (RMSE is ). The developed PLM-GP can predict the design parameters very well with small learning samples. Finally, the result by PLM-GP combining with SOM is better than other results as we epect (RMSE is ). In this case, we use only 5 parameters as input (Fn is removed). 6. CONCLUSIONS In this paper, data mining system for the prediction of design parameters to assist the ship designing process with insufficient learning samples is developed. The integrated system is presented with PGP, LM-GP and PLM-GP. Designer can generate his/her empirical formulas by using this system easily with small learning samples. In order to improve the performance of prediction, SOM is combined with GP program. Through the evaluation of influence for the input parameters by SOM, the performance of prediction was considerably enhanced. The validation test and the adoption of the developed method in the ship designing process are presented. As a result, the system is good for nonlinear function approimation with limited amount of Figure 8 Prediction result by Yamagata empirical formula (RMSE is ) Real Values Real Values the Result of Data Mining before applying SOM RMSE = Estimated Values Figure 9 Prediction result by using PLM-GP (RMSE is ) the Result of Data Mining applying SOM RMSE = Estimated Values Figure 10 Prediction result by combining GP with SOM (RMSE is ) 7

7 learning data, without overfitting. The developed data mining system can be used in real preliminary ship design process to generate new empirical formulas for the net generation high value added ships, such as LNG carrier, FPSO, and so on. ACKNOWLEDGMENTS This work is supported by Advanced Ship Engineering Research Center (R ). REFERENCES Lee, K.H., Yeun, Y.S., Yang, Y.S., Lee, J.H. and Oh, J. (006 a), Data Analysis and Utilization Method based on Genetic Programming in Ship Design, Lecture Notes on Computer Science, 981, Lee, K.H., Oh, J. and Park, J.H. (006 b), Development of Data Miner for the Ship Design based on Polynomial Genetic Programming, Lecture Notes in Artificial Intelligence, 404, Simpson, T.W., Allen, J.K., and Mistree, F. (1998), Spatial Correlation and Metamodels for Global Approimation in Structural Design Optimization, Proc. Of DETC98, ASME. Malik, Z., Su, H. and Nelder, J. (1986), Informative Eperimental Design for Electronic Circuits, Quality and Reliability Engineering, vol.14, Alotto, P., Gaggero, M., Molinari, G. and Nervi, M. (1997), A Design of Eperiment and Statistical Approach to Enhance the Generalized Response Surface Method in the Optimization of Multi- Minimas, IEEE Transactions on Magnetics, Vol., No., Ishikawa, T. and Matsunami, M. (1997), An Optimization Method Based on Radial Basis Function, IEEE Transactions on Magnetics, Vol., No. /II, Ott, R.L. (199), An Introduction to Statistical Methods and Data Analysis, Wadsworth Inc. Myers, R.H. and Montgomery, D.C. (1995), Response Surface Methodology: Process and Product Optimization Using Designed Eperiments, John Wiley & Sons, Inc. Koza, J.R. (199), Genetic Programming: On the Programming of Computers by Means of Natural Selection, The MIT Press. Yeun, Y.S., Lee, K.H. and Yang, Y.S. (1999), Function Approimations by Coupling Neural Networks and Genetic Programming Trees with Oblique Design Trees, AI in Engineering, Vol.1, No. Lee, K.H., Yeun, Y.S., Ruy, W.S. and Yang, Y.S. (00), Polynomial genetic programming for response surface modeling, Proc. on 4th International Workshop on Frontiers in Evolutionary Algorithms(FEA00), In conjunction with Sith Joint Conference on Information Sciences. Yeun, Y.S., Suh, J.C. and Yang, Y.S. (000), Function approimation by superimposing genetic programming trees: with application to engineering problems, Information Sciences, vol.1, issue -4. Barron, A., Rissanen, J. and Yu, B. (1998), The minimum description length principle in coding and modeling, IEEE Trans. Information Theory, vol. 44, no. 6,