A Comparative Study of MLP and RBF Neural Nets in the Estimation of the Foetal Weight and Length

Transcription

1 A Comparative Study of MLP and RBF Neural Nets in the Estimation of the Foetal Weight and Length F. Sereno*, J. P. Marques de Sá*, A. Matos**, J. Bernardes** *FEUP Faculdade de Engenharia da Universidade do Porto, Portugal **FMUP Faculdade de Medicina da Universidade do Porto, Portugal INEB - Instituto de Engenharia Biomédica, Porto, Portugal fsereno@alf.fe.up.pt Abstract. Foetal weight estimation is a clinically relevant task for proper medical care in perinatal situations. Usually this estimation is based on features such as measurements derived from echographic examinations. Several formulas have been developed by other authors for performing this estimation with limited degree of success. Our approach is based on multilayer perceptrons (MLP) and radial basis functions (RBF) neural nets in order to achieve a clinically usable estimation of foetal weight. In this paper we report optimistic results by training the MLP using the fast Levenberg-Marquadt algorithm, and the RBF by using the EM (Expectation- Maximization) algorithm. The performance of the two architectures is compared and the results show significant improvements over the formulas. Introduction It is widely accepted to estimate the foetal weight with the traditional regression formulas proposed by Hadlock and Shepard [5]. These formulas take, in their own ways, as input variables echographic measures of the abdominal perimeter, the femur length and the biparietal diameter. For practical clinics purposes the obstetricians claim that these methods give poor results. Simple neural networks, although in principle should map the input data directly onto the required final output values, in practice improve the accuracy of the foetal weight approximations as we reported [13] using multilayer perceptrons neural networks. Multilayer perceptrons (MLP) and radial basis functions (RBF) networks, are the two most commonly used types of feedforward neural networks. They differ fundamentally in the way the hidden. To appear in the Proceedings of RECPAD 2000, 11 th Portuguese Conference on Pattern Recognition, Porto, May 11 12, 2000

2 2 units combine values coming from the inputs. The MLP use inner products, and the RBF Euclidean distance. Most methods for training MLP can also be applied to RBF networks. In the present work the results of both approaches are described and compared. The approximation problem The estimation problem that we have to solve can be viewed as an approximation problem, where one want to approximate the true foetal weight and length values using echographic measurements as input variables. There are three main ingredients of an approximation calculation, according to [11], which are as follows: (i) a function f, or some data, or more generally a member of a set, that is to be approximated; (ii) A set, A say, of approximations, and (iii) a means of selecting an approximation A. We have to approximate f by a member of A, and we require a criterion that selects suitable components. Moreover, in computer calculations of mathematical functions, the mathematical function is usually approximated by one that is easy to compute. In most approximation problems there exists a suitable metric space that contains both f and the set of approximations A. When the properties of metric spaces are not sufficiently strong we assume that A and f are contained in a normed linear space. The real multivariable approximation problem is formulated by Powell (1987) [12] in the following terms. Given m different points {x i ; i = 1,2,,m} in R n, and m real numbers {t i. ; i =1,2,,m} one has to calculate a function f from R n to R, that satisfies the approximation conditions: f(x i ) = t i, i = 1,2,...,m. (1.1) The function f can be obtained from a m-dimensional linear space, in which case, having chosen a basis of the space the conditions (1.1) provide a square system of linear equations in the coefficients of f. It is well known, however, that if the space is independent of {x i ; i = 1,2,,m}, and if the functions in the space are continuous, then it is possible for the different points {x i ; i = 1,2,,m} to have the property that the matrix of the system is singular, provided that n >= 2. The function f can be chose from a linear space that depends on the positions of the data points. This dependence is due to the use of radial basis functions of the form ( x ) φ, x R n, i = 1,2,,m (1.2) x i where φ is from R + to R, and where the norm of R n is Euclidean.

3 3 In the approximation method we are using f has the form provided that the matrix m i= 1 ( x ) f ( x) = λ φ, x R n (1.3) A ij i x i ( x x ) = φ, i, j = 1,2,...,m (1.4) i j is non-singular, the coefficients λ i ; i = 1,2,...,m are defined by the approximation conditions (1.1). We can write functions such as (1.3) in the form y k = y k (x; w) (1.5) where w denote a vector of parameters we call weights in the neural network models context. Many conventional approaches to statistical pattern recognition, as those of Hadlock and Shepard [5], can be viewed as specific choices for the functional forms used to represent the mapping (1.5), together with particular procedures for optimising the parameters w. Foetal weight (or length) estimation could be viewed as a classification, or as a regression problem. In the first case our task should be to assign new inputs to one of a number of discrete classes (or categories). In the second case, the outputs represent the values of continuous variables. Although our approach here described is more close to the second case, we remind Bishop s assertion (p.6) [1], that both regression and classification problems can be seen as particular cases of function approximation. Furthermore, many key issues which need to be addressed in tackling pattern recognition problems are common both to classification and regression. An overview of the mixture model The density function called mixture model provides a powerful technique for density estimation, and find important applications in the context of neural networks in configuring the basis functions in RBF neural networks. Among the three existing methods to train mixture models, according to [1], all being based on maximum likelihood, we use the re-estimation which leads to the EM algorithm for RBF training. Supposing we have a data set of N vectors X {x 1,... x N }. If these vectors are drawn independently from the distribution p(x θ), where θ is a set of parameters, then the joint probability density of the whole data set X is given by

4 4 N n p( X θ ) = p( x θ ) L( θ ) (2.1) n= 1 where L(θ) can be viewed as a function of θ for fixed X, in which case it is referred to as the likelihood of θ given X. The technique of maximum likelihood then sets the value of θ by maximising L(θ). This corresponds to the intuitively reasonable idea of choosing the θ which is most likely to give rise to the observed data [1,4,7,9,14]. To train the RBF neural nets we use a unsupervised procedure that takes the dimension of the input space, the number of centres in the mixture model and the type of the mixture model we want to use, as is described in the following sections. The RBF network training The radial basis function network mapping has the following form M y ( x) = φ ( x) (3.1) k w kj j = 0 j where the activation functions Φ j considered are Gaussian, and given by T 1 ( x µ ) Σ ( x ) 1 φ j ( x) = exp j j µ j (3.2) 2 and the activation of the extra bias function Φ 0 is set to 1. The basis functions can be interpreted in a way which allows the first-layers of weights (i.e. the parameters governing the basis functions) to be determined by unsupervised training techniques. To train the RBF we use a two stage training algorithm to set the weights. Firstly, the input data set {x n } alone is used to determine the parameters of the basis functions µ j and σ j. The basis functions are then kept fixed while the second-layer weights are found in a second phase of training. The centres are determined by fitting a Gaussian mixture model with circular covariances using the EM algorithm [1,2,7]. Secondly, the hidden to output weights that give rise to the least squares solution can then be determined using the pseudo-inverse : W T = Φ T (3.3) Although this procedure may not give solutions with an error as low as using general purpose nonlinear optimisers, it is much faster.

5 5 The MLP network training We use the MATLAB reduced memory Levenberg-Marquardt algorithm. It was designed to approach second-order training speed without having to compute the Hessian matrix. When the performance function has the form of a sum of squares (as is typical in training feedforward networks), then the Hessian matrix can be approximated as H = Z T Z (4.1) and the gradient can be computed as g = Z T e (4.2) where Z is the Jacobian matrix, which contains first derivatives of the network errors with respect to the weights and biases, and e is a vector of network errors. The Jacobian matrix can be computed through a standard backpropagation technique [3,6] that is much less complex than computing the Hessian matrix. The Levenberg-Marquardt algorithm uses this approximation to the Hessian matrix in the following Newton-like update: wnew = wold T ( Z Z + 1 T λi) Z ε ( wold ) (4.3) When the scalar l is zero, this is just Newton s method, using the approximate Hessian matrix. When l is large, this becomes gradient descent with a small step size. Newton s method is faster and more accurate near an error minimum, so the aim is to shift towards Newton s method as quickly as possible. Thus, l is decreased after each successful step (reduction in performance function) and is increased only when a tentative step would increase the performance function. In this way, the performance function will always be reduced at each iteration of the algorithm. Application to the foetal data case In the frame of a multicenter study comprising several Portuguese Hospitals a data set of 354 cases, each one comprising multivariate input features and the correct foetal weight and length measured at birth, were collected.

6 6 The input variables were, the echographic measured features : biparietal diameter (bpd), cephalic perimeter (cep), abdominal perimeter (bpd), femur length (fem), umbilical artery resistance index (uri). All the echographic features were measured using the same methodology based on an established protocol. We used the data in a cross-validation technique as the number of available cases was too small. The MLP neural nets we used have a two-layer network topology, with tan-sigmoid activation functions in the hidden layer, and a linear transfer function in the output layer. They were trained using the fast training Levenberg-Marquardt algorithm, with the early stopping option enabled to avoid overfiting MLP 54 MLP Weigth (grams) 2000 Length 44 (cm) # case (a) real estimated # case (b) real estimated Figure 1 Graphical representation of a sub-set of cases estimated using a two layers Multi-Layer Perceptron (MLP) neural net, with five tan-sigmoidal hidden units, two linear output units, using the Levenberg-Marquadt training algorithm with early stopping. (a) The real foetal weights are ordered increasingly and represented by dots. The corresponding estimated values are represented by circles. (a) The real foetal lengths are also ordered increasingly and represented by dots. The estimated values are represented by circles. The RBF neural nets were trained using a two stage training algorithm developed with the MATLAB Neural Networks Tools by Christopher M Bishop and Ian T Nabney (1996, 1997). The centres were determined by fitting a Gaussian mixture model with circular covariances using the EM

7 7 algorithm. The mixture model was initialised using a small number of iterations of the k-means algorithm. As the activation functions were Gaussians, then the basis function widths were then set to the maximum inter-centre squared distance. The hidden to output weights that give rise to the least squares solution can then be determined using the pseudo-inverse. All the MLP and RBF prediction models we used had the same five input variables (abp, fem, bpd, uri, cep) and were evaluated by training on 284 cases and using a cross-validation test on separate groups of 71 cases each. Figure 1 and 2 represent a some estimations of the MLP and RBF in one of the test samples RBF 54 RBF Weigth (grams) 2000 Length 44 (cm) # case (a) real estimated # case (b) real estimated Figure 2 Graphical representation of a sub-set of cases estimated using a Radial Basis Function (RBF) neural nets, with ten hidden units, Gaussians, two output linear units, using the Bishop and Nabney EM training algorithm. (a) The real foetal weightsare ordered increasinglyand e represented by dots, whereas the estimated values are represented by small circles. (a) The real foetal lengths are also ordered increasingly nad represented by dots, whereas the estimated values are represented by small circles.

8 8 We calculated the relative absolute errors in the weight and length estimation, and the percent of cases in which the estimation had a relative absolute error less than 5%. Table 1 shows the average results of 16 experiments using the same architecture for the MLP and RBF neural nets. MLP RBF Foetal Weight: Relative Absolute Error 7.52% 7.15% Percent of Errors < 5% 41.1% 42.7% Foetal Length: Relative Absolute Error 2.8% 2.6% Percent of Errors < 5% 85% 87% Table 1. Results of the foetal weight and length prediction. (a) The MLP neural net has 5 units in the hidden layer and 2 units in the output layer. (b) The RBF neural net has 10 units in the hidden layer and 2 units in the output layer. Conclusions Our present lowest relative absolute error for foetal weight is 7.15%, as can be seen in the Table 1 (using RBF neural nets), whereas the relative absolute error of the Hadlock and Shepard formulas was around 9%, as we reported in [13]. We cannot yet have a definite idea if the RBF, or the MLP, might be more appropriate to solve the problem of foetal weight prediction, as it is too small the number of cases we have now available. We are still investigating the reduction of the error ratios with alternative training strategies for these two architectures and doing further analysis in the input data to define a pre-processing scheme. We are trying to rescale the input variables in proportion to their relative importance in the output as suggested by Bishop [1], and to use prior knowledge, e.g. trying to know to what extent the sex of the foetus influence the weight of the new born, to initialise a hypothesis to perfectly fit the domain theory, then inductively refine this initial hypothesis as needed to fit the training data, as suggested by Mitchell [8] approach KBANN (Knowledge-Based Artificial Neural Network). Future projects aim also a reduction of variance by combining the outputs of several neural networks, MLP, RBF and other architectures not yet tested (e.g. Support Vector Machines and Self- Organising Maps) together to form committees that could effectively improve the accuracy of the foetal weight and length approximations.

9 9 References [1] Bishop, Christopher M., 1997, Neural Networks for Pattern Recognition, Oxford, Oxford University Press. [2] Dempster AP, Laird NM, Rubin DB, 1976, Maximum Likelihood from Incomplete Data via de EM Algorithm, in Journal of the Royal Statistical Society, B 39(1) [3] Demuth, Howard, Beale, Mark, 1998, Neural Network Toolbox For use with Matlab (version 3.0), Massachussets, The MathWorks, Inc. [4] Duda, S. A., Hart, P.E.; 1973, Pattern Classification and Scene Analysis, New York, John Wiley. [5] Farmer RM, Medearis AL, Hirata GI, Platt LD; 1992, The Use of a Neural Network for the Ultrasonographic Estimation of Fetal Weight in Macrosomic Fetus, Am J Obstet Gynecol May [6] Hagan, Martin, Demuth HB, Beale M, 1996, Neural Network Design, Boston, PWS Publishing Company. [7] Marques, Jorge S., 1999, Reconhecimento de Padrões, Métodos Estatísticos e Neuronais, Lisboa, IST Press [8] Mitchell T, 1997, Machine Learning, New York, McGraw Hill. [9] Murteira, Bento J F, 1988, Estatística: Inferência e Decisão, Lisboa, Imprensa Nacional Casa da Moeda [10] Nabney I T, 1999, Efficient Training of RBF Networks for Classification, Birmingham, Aston University. [11] Powell, M.J.D., 1981, Approximation Theory and Methods, Cambridge, Cambridge University Press. [12] Powell, M.J.D., 1987, Radial Basis Functions For Multivariable Interpolation : a review, in Mason JC, Cox, MG (eds.), 1987, Algorithms for Approximation, Oxford, Clarendon Press, Oxford University Press. [13] Sereno F., Marques de Sá, J.P., Matos, A., Bernardes J., 1999, Foetal weight estimation Using Neural Networks, in Medical & Biological Engineering & Computing Volume 37, Supplement 2, 1999 pp.( ) Proceedings of the The European Medical & Biological Engineering Congress EMBEC 99, Vienna, Austria. [14] Sivia, D.S., 1997, Data Analysis, A Bayesian Tutorial, Oxford, Oxford University Press.