1300 scree

Transcription

1 Determining the Number of Dimensions Underlying Customer-choices with a Competitive Neural Network Michiel C. van Wezel 1, Joost N. Kok 2, Kaisa Sere 3 1 Centre for Mathematics and Computer Science (CWI) P.O. Box 94079, 1090 GB Amsterdam, The Netherlands Phone: , Fax : michiel@cwi.nl 2 Leiden University, dept. of Mathematics and Computer Science P.O. Box 9512, 2 RA Leiden, The Netherlands 3 University of Kuopio, Department of Computer Science and Applied Mathematics P.O.Box 1627, FIN Kuopio, Finland ABSTRACT More and more data about consumer-behaviour is becoming available through the use of database systems and electronic-banking systems. In this paper we describe a NN-based technique to extract some useful information from a database holding data about customers of an insurance-company. Using this NN, we try to learn more about the needs and wishes that lead consumers towards specic behaviour, so that better service can be provided in the future. We formulate a more formal version of this problem, describe the NN used, and show results on articial and real data. 1. Intruction Thanks to computer systems, over the last decade there has been an enormous growth in the amount of data becoming available to analysts, managers, marketeers etc., describing consumer behaviour. It is obvious that these data contains potentially very valuable information, which could lead to interesting insights. In this paper we examine the application of a Neural Network based technique to extracting valuable information from a database holding data on customers from a big insurance company. The aim of the research presented here, was to develop a meth to determine the number of underlying stimuli inuencing the customers of the insurance company. The structure of the remainder of this paper is as follows: in section (2) a more accurate description of the problem is presented. In section (3) we discuss the NN used to solve the problem. In section (4) we show the results of our meth on articially generated data, and in section (5) we show the results on `real life' data. Finally, in section (6) we make some conclusions and point out some possible directions for further research. 2. Problem Description and Error Function The insurance company in question oers various categories of pructs to its customers, e.g. life insurances, car insurances, health insurances, travel insurances, etc. Within each category, a client has dierent alternatives to choose from. For example, someone with a health insurance can choose to have a dentist insurance included, a world-wide coverage of medical costs, an `own risk' of a certain amount of money, etc. Now, we ask ourselves the question if there are any `underlying' factors (or dimensions) that inuence the insurant in his decisions for taking (or not taking) a specic set of alternatives. Marketing literature

2 suggests that such underlying dimensions indeed exist ([4]). In the example of a health insurance, two important dimensions might be the price of the insurance and the quality of the medical care the insurant is entitled to. Somewhat more formally, we can formulate the following mel, describing the clients, the pruct categories, and the alternatives within each category. Mel Set of customers K = fk 1 ; : : : ; K n g. { A k-dimensional pattern v i = (v ij1 ; : : : ; v ijk ), expressing the demands/wishes needs of each customer, is associated with each customer i. Set of pruct categories S = fs 1 ; : : : ; S m g. { Within each pruct category S j in S, a number of alternatives is available. These alternatives are denoted O j1 ; : : : ; O jsj, where s j denotes the number of possible alternatives for pruct category S j. Each alternative O ab has an associated k-dimensional vector v Oab = (v Oabj1; : : : ; v Oabjk), expressing the value of alternative O ab on dimensions v 1 ; : : : ; v k. A client K chooses one alternative from each pruct category, namely the alternative O ji for which d(v Oji ; v K ) is minimal. Here, d(a; B) denotes the (Euclidean) distance between A and B. Now, we can formulate the problem to be solved in terms of this mel. Problem Suppose n clients have generated n m-dimensional vectors a i = (a ij1 ; : : : ; a ijm ), where m is the number of pruct categories, 1 i n and 1 a i jj s j for. A vector a i es no more than `indexing' the alternatives client i has chosen from the various pruct categories. The task is to nd the number of underlying dimensions k and estimates v 0 i = (v 0 i1 ; : : : ; v0 ik ) for the vectors v i = (v i1 ; : : : ; v ik ) (for all clients and alternatives) such that a suitable error-measure E(vi) 0 is acceptably low. The choice for E(v 0 i ) is relatively simple: we take it to be the number of erroneous alternatives chosen if the estimated v-coordinates for the customers and the alternatives are used to generate estimates (a 0 1 ; : : : ; m) for the vectors (a a0 1 ; : : : ; a m ), so E = X k2kx f(o k;s ; o k;s); (1) s2s where o k;s is the choice client k has made at pruct category s in reality, and o k;s is the choice client k has made at pruct category s according to our estimate. The function f is dened as follows: f(x 1 ; x 2 ) = 0 () x1 = x 2 1 () x 1 6= x 2 ; (2) Unfortunately, (1) is not dierentiable, so we cannot minimise it by applying gradient meths. 3. The Competitive Neural Network The neural network we used was inspired by simple competitive neural networks (see e.g. [2, 3]). Every customer and every alternative in our system was represented in this network by a competitive unit. The weight-vectors of the units corresponded with the v-coordinates of the customers and pruct-alternatives. The aim of the learning process was to nd values for the weight vectors such that error function (1) was minimised. In the beginning of the training process all weight vectors were initiated with ranm values in the interval [0,1]. Next, the training went on as follows.

3 In each iteration, all patterns (clients) in the database were presented to the network once, in a ranm order. Remember that one pattern in our database was no more than an integer vector a i = (a i j1; : : : ; a i jm) of which the components represent the choices the customer at hand (customer i) has made. The weight vectors of the units representing the choices were now pulled towards the weight vector of the unit representing customer i, and vice versa. This way, we hoped to increase the probability that the units representing the alternatives that customer i has chosen, are the closest units within each pruct-category in the next iteration. The parallel with a simple competitive neural network should be clear. In a simple competitive neural network the weight vectors of the units are pulled towards the input vectors. This way a voronoi tessellation of the input space is created. In our case, we know in advance that the alternatives within one set divide the input space up in a voronoi tessellation. We also know which client should lie in which voronoi cell for every pruct-category in the choice process. By means of the `competitive learning' algorithm, we move the positions of the clients and the alternatives around, and we hope to reach a state where most of the clients lie in the correct voronoi-cell for most of the pruct-categories in the choice-process. Unfortunately, there is a problem associated with this learning-scheme. If we start with ranm initial weights, the average direction of the weight updates will be inward. This will cause the neural network to imple. We can prevent this by re-normalising the weight vectors after each iteration. There is another potential pitfall for our system. Typically, we have data on several thousands of customers, but there are only a few pruct-category, with a few alternatives within them. This causes the alternative-coordinates to be updated many Much more frequent than the client coordinates. This problem can be solved by updating the weight vectors of the units representing the alternatives by a much smaller amount than the weight vectors of the units representing the customers. The basic algorithm is given in pseu-ce in gure (1). 1. for all clients and alternatives create units and initialise weights to ranm values 2. if is a `basic learning rate', calculate the appropriate learning rate for every unit by: - dividing by the number of pruct-categories for the clients - multiplying with teh number of alternatives in the option for every option, and dividing it by the number of clients afterwards 3. for cycle = 0 to number of cycles for client = 0 to number of clients - nd a client c not used in this cycle yet - for pruct = 1 to number of pructs calculate error normalize weight vectors 4. dump the weight vectors pull weight vectors client c and alternative client has chosen from for pruct together Fig. 1: Algorithm in pseu ce Determining the Number of Dimensions We can use the NN for nding estimates v 0 for the vectors with v-coordinates given the dimensionality D of those estimates, but we not know yet how to determine the appropriate number of underlying dimensions.

4 This problem can be solved by making a `t vs. number of dimensions'-plot. As t measure for this plot, we use (1). Now, we can construct a graph by running the NN with say, dimensionality D = 1; : : : ; N + 5, where N is a guess about the true number of underlying dimensions. This t vs. number of dimensions'-plot will show an elbow at the right number of dimensions. An example of a `t vs. number of dimensions'-plot clearly showing an elbow is given in gure (2). A very similar procedure for obtaining the number of underlying dimensions is often used in Multidimensional Scaling (MDS) (see e.g. [1]) and Principal Component Analysis (PCA) (see e.g. [5]). In the context of the latter technique, the `t vs. number of dimensions'-plot is often called `scree plot'. 1 scree Fig. 2: A `t vs. number of dimensions'-plot showing a clear elbow at four dimensions. 4. Results on Articial Data. In this section we present the results of our NN on articial data. For each experiment we generated ranm v-coordinates representing the demands of the n customers, and ranm v-coordinates representing the vectors v belonging to the various alternatives in one pruct category. Next, we generated the vectors a 1 ; : : : ; a n, which is now possible because the v-coordinates for both customers and pructs are known. The task was to nd the dimensionality of the vectors with the v-coordinates using only the vectors a 1 ; : : : ; a n. Thereto, `t vs. number of dimensions' plots were created using the NN. In total, we performed experiments on eight datasets this way, whereby the underlying number of dimensions was varied from two to ve. Four out of the eight datasets had ten pructs, and ten alternatives per pruct. The remaining four datasets had fteen pructs, and ve alternatives per pruct. The number of customers war set to 150 in all datasets. In gures (3) and (4) the resulting plots of t vs. number of dimensions are shown. Elbows are clearly visible for each problem instance. 850 scree scree scree scree Fig. 3: `t vs. number of dimensions'-plots for the datasets with 10 pructs and 10 alternatives per pruct.

5 1 1 1 scree scree scree scree Fig. 4: `t vs. number of dimensions'-plots for the datasets with 15 pructs and 5 alternatives per pruct. 5. Results on Real Data After a positive result was obtained with the articial data, we went on to the real data. From each original dataset available to us, we selected only the customers with all the elements of the a vectors specied. This left us with ca. 150 customers per dataset, approximately the same as with the articial data. The t vs. dimensionality plots resulting from these experiments are shown in gure (5). On rst look, it seems that the results of these experiments are not as go as the results with the articial data. This is probably due to the erroneous assumption that a customer always chooses the alternative within one pruct category, for which the v-coordinates are closest to his own v-coordinates. It is very probable that a lot of customers make their choices on a more ad hoc basis, thus intrucing noise in the data. Luckily, we can still see an elbow in many of the plots. Note that after the elbow, the error es not remain as constant as with the articial data. This is probably because the more dimensions we add, the easier it gets to mel the noise in the data, and thus the lower the error we obtain gets /runs_verlag_klein/scree_file.1../runs_verlag_klein2/scree_file.23../runs_verlag_klein2/scree_file.34../runs_verlag_klein2/scree_file.43../runs_verlag_klein/scree_file Fig. 5: `t vs. number of dimensions'-plots for the ve real datasets. 6. Conclusions & Further Research Looking back at the experiments we performed on both the articial and the real life data, we can conclude that we have succeeded in developing a meth for determining the number of underlying dimensions inuencing consumer behaviour. We have constructed a valuable `data-mining tool', with which we can see whether there are underlying dimensions in choice processes, and if so, how many. This was ne by a novell competitive neural network. Care had to be taken that the network did not imple, and thus arrived at a useless solution. Although the results on the real data are not as go as the results on the articial data, our NN is still able to nd a conguration with a low error. There are a number of interesting directions for further research in this eld. First of all, it may be useful to examine the `toughness' of the problem (i.e. can we prove that the problem is N P-complete?). Second, the current meth is based on intuition. We have not given an associated error function on which our network performs a gradient descent. It would be interesting to develop a continues-dierentiable error function, and derive a learning rule from it. Finally, it would be interesting to analyse other datasets with our meth. It is not impossible that data coming from other branches is less noisy, and thus leads to better results.

6 References [1] Mark L. Davidson Multidimensional Scaling, Wiley Series in Probability and Mathematical Statistics, Wiley, [2] Haykin, S. Neural Networks: a Comprehensive Foundation. New York, MacMillan College Publishing Company, [3] Hertz, J.A., A.S. Krogh & R.G. Palmer. Intruction to the theory of neural computation. Addison- Wesley, [4] Kotler, Philip. Marketing Management - Analysis, Planning, Implementation and Control 8th edition, Prentice Hall, [5] Krzanowski, W.J. Principles of Mulitvariate Analysis, Oxford Statistical Science Series, New York: Orford University Press, 1988.