A Novel Pruning Algorithm for Optimizing Feedforward Neural Network of Classification Problems
|
|
- Shannon Chambers
- 5 years ago
- Views:
Transcription
1 Chapter 5 A Novel Pruning Algorithm for Optimizing Feedforward Neural Network of Classification Problems 5.1 Introduction Many researchers have proposed pruning algorithms in numerous ways to optimize the network architecture (Castellano et al., 1997; Ahmmed et al., 2007; Henrique et al., 2000; Ponnapalli et al., 1999). Reed (1993) and Engelbrecht (2001) have given detailed surveys of pruning algorithms. Each algorithm has its own advantages and limitations. Some algorithms (Engelbrecht, 2001; Xing & Hu, 2009) prune both irrelevant input neurons and hidden neurons of the network and some algorithms (Zeng & Yeung, 2006) prune irrelevant hidden neurons only. Real-world applications prefer simpler and more efficient methods. But a significant drawback of most standard methods consist in their low efficiency. For example the main 94
2 weakness of the OBD and OBS techniques is their relative low computational efficiency. Magnitude based pruning (MBP) methods often remove important parts of the network as they assume that small weights are irrelevant. However small weights may be important compared to very large weights which cause saturation in hidden and output units (Engelbrecht, 2001). Some algorithms (Sietsma & Dow, 1988; Chung & Lee, 1992) require the user to specify the number of problem dependent threshold parameters or tuning parameters. More sophisticated methods (Engelbrecht, 2001; Xing & Hu, 2009; Lauret et al., 2006) reach better results, but the precision is usually compensated by unproportional increase in computation time (Reitermanova, 2008). Unfortunately, sensitivity analysis based pruning methods are not guaranteed to detect all redundant processing elements as they assume both the inputs of the network and the outputs of the hidden neurons are mutually independent (Reitermanova, 2008). When there are dependencies between inputs, the Sensitivity Analysis based method can be ineffective while the Mutual Information based methods can successfully avoid this limitation (Xing & Hu, 2009). This chapter focuses on developing a novel pruning algorithm which finds the optimal architecture of multilayer feedforward neural network (MLFNN) by removing both insignificant input nodes and hidden nodes based on a new significant measure that considers inputs of the network and outputs of the hidden neurons. The proposed work is concentrated primarily on removing nodes, since the node pruning algorithms are more efficient than weight elimination methods and the computational nodes are more important than individual connections as they rep- 95
3 resent the bottleneck through which information in a neural network is conveyed (Kruschke, 1998). This chapter is organized as follows: Section 5.2 explains the methodology of the proposed pruning, Section 5.3 describes the new pruning algorithm for optimizing the neural network architecture, Section 5.4 compares the results of the proposed method with other pruning methods in terms of pruning percentage, pruning speed and classification accuracy by implementing it on six different real datasets namely iris, Wisconsin breast cancer, hepatitis, diabetes, ionosphere and wave. 5.2 Proposed Pruning Method The basis of this pruning algorithm is to estimate the significance of each individual input node and hidden node of the trained neural network and prune all the nodes with significance value below the estimated threshold. This method is the posttraining pruning method. The proposed method, called as Neural Network Pruning by Significance (N2PS), prunes the insignificant neurons of a neural network based on its estimated significance value. This post-training pruning algorithm first uses the backpropagation training algorithm with momentum (Han & Kamber, 2001) to train a fully connected FNN and then prune the insignificant neurons Multilayer Network Model Let us consider a fully connected MLFNN as in Fig. 5.1 with an input layer, one or more hidden layers and an output layer. Let L be the total number of 96
4 Figure 5.1: MultiLayer feedforward neural network layers in a network being considered for pruning and let ml be the total number of neurons in each lth layer, 0 l L. Among L layers in a network, the first layer 0 is an input layer, the layers between 0 and L are hidden layers and the last layer L is an output layer. Let Ni0 be the ith input neuron of 0th layer where 0 i m0 and m0 th is the input neuron with bias value which is always equal to 1. Let np be the number of patterns considered for training and xip be the value of ith input neuron of pth pattern in a dataset. Let Njl be the j th neuron of lth hidden layer, 0 < l < L and 1 j ml. Let wij1 be the weight between input neuron Ni0 and neuron Nj1 in first hidden layer, where j {1, 2,..., m1 }, vjl kl+1 be the weight between a neuron Njl and neuron Nkl+1, where j {1, 2,..., ml }, k {1, 2,..., ml+1 } and initially both take random values between 1 to 1. Let hj1 be the activation value of the hidden neuron Nj1, hjl be the activation value of the hidden neuron Njl and ok be the output of the Nk th neuron in the outputl layer L, 1 k ml. The number of neurons in the output layer is equivalent to number of target classes in the dataset. Using the backpropagation algorithm, the 97
5 value of hj1 and hjl for pth pattern is calculated respectively by, hj1 = f ( m0 i=0 ml 1 (xip.wij1 )) and hjl = f ( (hjl 1.vjl 1 kl )) (5.1) jl 1 =1 where f (x) = 1,1 1+e x < l < L, and the output ok of network can be calculated by, ok = f ( ml 1 jl 1 =1 (hjl 1.vjL 1 kl )) (5.2) where f (x) = 1 1+e x The sigmoidal function f (x) = 1 1+e x is used to normalize any value within the bound 0 to 1 (Han & Kamber, 2001) Backpropagation Training The backpropagation algorithm learns iteratively by processing the np training patterns of a dataset, comparing the networks result ok for each pattern with the desired known target value dk for each target class k in a dataset. The target value is a known class label of the training pattern. Weights are modified for each pattern so as to minimize the mean squared error (mse). The value of mse is calculated according to the following equation 1 np n 1 mse =((dk ok )2 ) np p=1 k=1 2 (5.3) Weights update are made using momentum method (Setiono & Hui, 1995) in the backward direction i.e., from the output layer through hidden layer and to input layer. The purpose of inclusion of the momentum term with BP is to accelerate the convergence of the backpropagation training algorithm. The method involves 98
6 supplementing the current weight adjustments with a fraction of the most recent weight adjustment. The fraction is specified by a user selected positive momentum constant. Finally, the weights eventually converge and the learning process stops Pruning by Significance Initially, the network with the large number of hidden neurons is trained with the backpropagation with momentum algorithm to solve the classification and then the size of the trained network is optimized based on the significance of a neuron. The significance is based on its output. Equation (5.2) states that the output value of each neuron in a layer is corresponding to the sum of products of the activation value of all nodes in the previous layer and the weights of all its incoming connections. It shows that the activation value and all the outgoing weights of a node define the neuron s significance and hence the significance is computed by applying the sum-norm on those values. But before applying the sum-norm, the activation value has to be computed over the entire training set to reflect the aggregated effect of all patterns, since each pattern results in a different sensitivity matrix (Engelbrecht, 2001). To identify the aggregated activation value of a hidden neuron Njl where 0 < l < L and j {1, 2,..., ml } for all the patterns, first the total net value of each hidden neuron is identified, by computing the aggregated net values of all the patterns and then the sigmoidal function is applied on it. Applying the sigmoidal function on the input or output value of a neuron makes the neurons with larger value as closer to 1 and the neurons with smaller as closer to 0. Pruning the neurons 99
7 based only on its aggregated activation value may remove the important neurons as the significance of each node in a layer is not only based on the activation value over all the patterns but also the weights of its all outgoing connections. So the real significance of a neuron is evaluated by its aggregated activation value and also by considering the weights of its all outgoing connections. Let tnetjl be the total net value of the hidden neuron Njl and the computation of tnetjl is expressed as, tnetjl = npm0 p=1i=0 xip.wij1 ml 1 when l = 1 when 1 < l < L jl 1 =1 f (tnetjl 1 ).vjl 1 jl (5.4) After the value of tnetjl is identified, the function Sigmoidal is applied on it to compute the aggregated activation value and then the significance measure sjl of a hidden neuron Njl is computed by adding its aggregated activation value over all the patterns with all its outgoing connections as follows: sjl = ml+1 kl+1 =1 f (tnetjl ) + vjl kl+1 (5.5) where f (tnetjl ) = tnetjl1+e. 1 The status of each hidden neuron Njl, 1 l L of multilayer feedforward neural network is identified as insignificant if its significant measure sjl is smaller than the threshold value β i.e., Njl is insignif icant if sjl β, β = signif icantotherwise mljl =1 (sjl )/ml (5.6) Fig. 5.2 describes the process of identifying insignificant hidden neurons, where 100
8 all the hidden neurons with the significant measure sjl below the threshold limit are considered as insignificant neurons. Figure 5.2: Identifying insignificant hidden neurons based on threshold Similarly, the significance of an input neuron is determined by the normalized sum of all its input patterns and all its outgoing connections. The significance measure si of a node Ni0 in an input layer is computed as, si = m1 j1 =1 f (txip ) + wij1 (5.7) 1 1+e txip where 0 i < m0, f (xip ) = and txip = xip. The input neuron Ni0 is insignificant if its significant measure si is smaller than the threshold value np p=1 α i.e., Ni0 is insignif icant if si α, α = signif icantotherwise m0i=0 (si )/m0 (5.8) The threshold values α and β are calculated by finding the mean value of the significance of all nodes in that layer. After the status of each neuron is identified, 101
9 all the neurons with insignificant status are pruned from the trained neural network and then the pruned network is retrained to avoid the loss of performance due to pruning. The initial values of weights for retraining the pruned network are all inherited from the corresponding values of weights in the trained network in the previous step. After retraining, the classification accuracy of the pruned network is computed. If it falls below an acceptable level, the pruned network obtained in the previous step is retained and the process is terminated, otherwise the process is repeated. Eliminating the insignificant neurons using the threshold values α and β, calculated by the equations (5.6) and (5.8), require more pruning iterations when the number of neurons in a input layer or hidden layer is high. In this case the pruning process of that layer can be speeded up by modifying its threshold value α or β using α = q i=1 (si )/q and β = ri=1 (sjl )/r where q and r be the number of existing input neurons and the number of existing hidden neurons in lth layer, 1 < l < L in the pruned network after each pruning iteration. The main advantages of this proposed method are, (i) both insignificant input and hidden neurons are pruned, (ii) less number of retraining iterations are required since it removes insignificant neurons and it inherits the initial weights of the pruned network from the previous step, (iii) the computational cost is reduced since it doesn t require any complex calculations for pruning, (iv) the nodes are pruned directly instead of removing unwanted connections associated with those nodes, and hence the number of pruning steps is reduced, (v) No threshold or 102
10 tuning parameters are required and (vi) suitable for pruning the MLFNN with any topology. 5.3 N2PS Algorithm In this section, an algorithm which finds the optimal architecture by pruning the MLFNN based on the neuron s significance has been proposed. Input: A multilayer feedforward neural network consisting of L layers such as, an input layer with m0 input neurons, one or more hidden layers with ml hidden neurons, 0 < l < L and an output layer with ml output neurons equivalent to number of target classes, and a dataset with np patterns. Begin 1. Train the network T until a predetermined accuracy rate is achieved using the Backpropagation algorithm with momentum. 2. Consider a copy of the trained neural network T as a temporary pruned network P. 3. For each input neuron Ni0 of P in layer 0, 3.1. Compute si, the significance of the input neuron Ni0, using an equation (5.7) Frame the set I = {Ni0/(si α)}, the set of insignificant input 103
11 neurons where α = m0i=0 (si )/m0. 4. Update the temporary pruned network P by removing all the insignificant input neurons of I. 5. For each hidden neuron Njl of P in layer l, 5.1. Compute the total net value for all the patterns in a dataset using an equation (5.4) Compute sjl, the significance of the hidden neuron Njl, using an equation (5.5) Frame the set H = {Njl /(sjl β)}, the set of insignificant hidden neurons where β = mlj=1 (sjl )/ml. 6. Update the temporary pruned network P by removing all the insignificant hidden neurons of H. 7. Retrain the temporary pruned network and compute its classification accuracy on testing dataset. 8. If classification accuracy of the network P falls below an acceptable level then stop pruning otherwise consider this temporary pruned network P as trained pruned network T and goto step 2. Output: The pruned multilayer feedforward neural network T. 104
12 5.4 Experimental Results The proposed algorithm is implemented on six well known continuous and mixed mode WEKA s datasets and compared with other pruning methods such as VNP (Engelbrecht, 2001), Xing-Hu s method (Xing & Hu, 2009), MBP (Hagiwara, 1994), OBD (LeCun et al., 1990) and OBS (Hassibi et al., 1993). The datasets used to test the algorithm are, (i) Iris Plants dataset (iris), (ii) Wisconsin breast cancer dataset (cancer), (iii) Hepatitis Domain dataset (hepatitis), (iv) Pima Indians Diabetes dataset (diabetes), (v) Ionosphere dataset (ionosphere) and (vi) Wave form dataset (wave). The training and testing patterns are taken randomly from each class. For example, the iris dataset is having 3 classes with 50 patterns for each class. From each class 25 patterns are taken randomly for training and another 25 patterns are taken randomly for testing the network Result Analysis Experiments have been performed for the N2PS algorithm with the datasets namely iris, cancer, hepatitis, diabetes, ionosphere and wave. As a first step, the three layer feedforward neural network is trained with the training patterns of the dataset using the backpropagation algorithm. This algorithm uses momentum (µ) as 0.5 for all datasets and the learning rate (λ) as 0.1 for four datasets namely iris, cancer, hepatitis and diabetes and 0.9 for two datasets namely ionosphere and wave. Number of input neurons equals the number of attributes in the dataset, but one bias input is also given additionally to gain better training. The number of hid- 105
13 den neurons of the initial network are selected as completely the same with those used in (Engelbrecht, 2001) and (Xing & Hu, 2009) for comparing the performance of N2PS with the related works namely VNP and Xing-Hu s method. The network is trained until the error converges to predetermined mean squared error 0.01 or the prespecified maximum number of iterations 200 has expired, whichever is earlier. The proposed algorithm N2PS calculates the significance of each input neuron using an equation (5.5) and each hidden neuron using an equation (5.7) and it eliminates the insignificant input and hidden neurons using the equations (5.6) and (5.8) respectively. Then the pruned network is retrained as similar to training process but the prespecified maximum number of iterations is reduced to 50. Experiments were performed 10 times for each dataset by dividing the original dataset into training and testing using a different random seed every time. The average of the results of the 10 runs is calculated for each set. The performances of Table 5.1: Performance of N2PS algorithm the N2PS algorithm on six datasets are shown in Table 5.1. The results show that the algorithm doesn t require more iteration to prune the network and requires 106
14 maximum three pruning steps only. In each pruning step, the current architecture is pruned based on nodes significance and retrained. The results also show that the pruned network achieves higher accuracy than the initially selected network. The FNN with the architecture for iris dataset is trained by the Backpropagation training algorithm. It achieves the 96% classification accuracy in 120 iterations (mse=0.01). The proposed pruning algorithm N2PS removes the unwanted input neurons and hidden neurons from the trained neural network. Fig. Figure 5.3: architecture (excluding bias) pruned network of iris dataset with 98.7% accuracy 5.3 shows the pruned network of iris dataset with the classification accuracy of 98.7% for the architecture N2PS has pruned 7 hidden neurons, one input neuron and achieved higher classification accuracy. Also it finds the reduced architecture within two pruning steps and the first pruning step requires 27 iterations and the second pruning step requires only one iteration to retrain the network. On breast cancer dataset the Backpropagation training algorithm achieves the classification accuracy 95.4% in 123 iterations (mse=0.01) with the architecture feedforward neural network. The N2PS algorithm prunes the trained network 107
15 and achieves the reduced architecture with classification accuracy 97.1%. The pruned networks of this dataset are shown in Fig N2PS requires two Figure 5.4: architecture (excluding bias) pruned network of cancer dataset with 97.1% accuracy pruning steps to reduce the network and each pruning step requires only 50 iterations. The hepatitis dataset is initially trained with 25 hidden nodes. Table 5.2 shows the pruning results of the hepatitis dataset. At the first pruning step itself N2PS removes maximum number of hidden neurons i.e., out of 25 hidden neurons it removes 17. For pruning this hepatitis dataset with no reduction in accuracy, the N2PS method requires only three pruning steps. At the step of 3, N2PS has reduced the original network with architecture with accuracy 80.2% to the architecture with accuracy 86.4%. Since the performance of the network is greatly deteriorated at the step of 4, the pruning process was stopped and the current architecture is accepted. The pruning results show that the pruned network achieves higher accuracy and best generalization than the original network. The Pima Indian diabetes dataset is trained with 40 hidden nodes. In
16 Table 5.2: Pruning results of N2PS on Hepatitis dataset iterations (mse=0.14), the Backpropagation training algorithm achieves 68.6% accuracy on this dataset with the architecture feedforward neural network. The pruning procedure N2PS reduces the architecture of the trained network as with the classification accuracy 70.3%. N2PS requires only two pruning steps to reduce the network and each pruning step requires 50 iterations. The ionosphere dataset consists of 34 input attributes and 2 output classes and hence the initial architecture of the dataset is The Backpropagation algorithm trains this network up to 0.01mse and achieves 91.4% classification accuracy in 18 iterations. N2PS algorithm prunes the irrelevant input neurons and hidden neurons and finds the reduced network with the architecture for the ionosphere dataset. Also it requires only two pruning steps and the first pruning step requires 30 iterations and the second pruning step requires 41 iterations to retrain the network. The pruned network achieves the classification accuracy 94.9%. The wave-form dataset is trained with the architecture feedforward neural network. This dataset consists of 40 attributes and 3 output classes with
17 patterns. From each class 900 patterns are taken for training and the remaining patterns are taken for testing the network. In 200 iterations (mse=0.03), the Backpropagation training algorithm achieves 83.2% accuracy. The pruning procedure N2PS reduces the architecture of the trained network as Here 31 input neurons and 6 hidden neurons are removed by this algorithm and the pruned networks achieve the classification accuracy 85.5%. N2PS requires only two pruning steps to reduce the network and each pruning step requires only 50 iterations. The pruning results show that the pruned network achieves higher accuracy than the original network. The experimental results of the above examples show that the proposed method requires lesser number of pruning steps and requires lesser number of iterations for retraining the pruned network. Also the results clearly indicate that the N2PS achieves small networks with high classification accuracy and the generalization performance of the original network for all datasets are retained by the final architecture of the pruned network Comparison of Pruning Methods In this section, the performance of the proposed method is compared with other five pruning methods such as Variance Nullity Pruning (VNP), Magnitude Based Pruning (MBP), Optimal Brain Surgeon (OBS), Optimal Brain Damage (OBD) and Xing-Hu method. The pruning methods OBS and OBD require additional computation for calculating the Hessian matrix of the system but the proposed method N2PS doesn t require any complex computation to find the significant 110
18 measure of each node. The efficiency of the MBP method is also low, since it considers only the magnitude of weights to prune the network (Engelbrecht, 2001). The pruning methods OBD, OBS and MBP prune irrelevant hidden neurons only but the proposed method N2PS removes additionally the insignificant input neurons also. The sensitivity analysis based method VNP combines both the input units pruning and hidden units pruning of Multi Layer Perceptrons (MLPs) in a single formula and achieves satisfying results, but as discussed in section 1.2, VNP is not guaranteed to detect redundant neurons as it doesn t consider the mutual dependency between both the inputs of the network and outputs of the hidden neurons. The Xing-Hu s method overcomes this limitation by considering the mutual dependency between them but it performs pruning in two separate phases (Xing & Hu, 2009). The proposed method N2PS combines the advantages of both VNP and Xing-Hu. It performs pruning of the input units and hidden units of MLPs in a single formula as VNP and considers the mutual dependency between the inputs of the network and outputs of the hidden neurons like Xing-Hu s method. Xing-Hu achieves better results than VNP with two separate phases for pruning input units and hidden units respectively while N2PS achieves better results than Xing-Hu in just a single phase for pruning both units. Table 5.4 shows the comparison results of N2PS on four datasets namely iris, cancer, hepatitis and diabetes with the results of other five pruning methods in (Engelbrecht, 2001; Xing & Hu, 2009). For all pruning algorithms, a pruned network is only accepted if the deterioration in generalization is less than 1%. For all the classification problems, the proposed method resulted in better architecture with minimum number of nodes while hav- 111
19 Table 5.3: Result compariso n of N2PS with other five pruning methods 112
20 ing the accuracy similar to or better than that of other architectures obtained from other pruning methods. Regarding the classification accuracy, the N2PS algorithm achieves higher ac- curacy for all datasets except diabetes. Fig. 5.5 shows the comparison of the Figure 5.5: Comparing classification accuracies of N2PS algorithm with other pruning methods classification accuracies achieved by N2PS method and other pruning methods. It shows that the N2PS method achieves higher accuracy for all datasets than OBS, OBD and MBP and achieves maximum or equal accuracy for 3 datasets out of 4 than Xing-Hu and VNP. N2PS has also performed effectively in the removal of input neurons. Considering the removal of neurons, the N2PS method performs outstandingly while comparing with OBS, OBD and MBP and also comparable with VNP and XING-HU method. Fig. 5.6 compares the N2PS method with other pruning methods by its removal of hidden neurons. It shows that the N2PS method removes more hidden neurons for all 4 datasets than OBS, OBD and MBP and for 113
21 Figure 5.6: Comparing hidden nodes removal of N2PS with other five pruning methods 3 datasets than Xing-Hu and for 2 datasets than VNP. N2PS has also performed effectively in the removal of input neurons. Fig. 5.7 shows that the proposed algo- Figure 5.7: Comparing input nodes removal of N2PS with VNP and Xing-Hus methods rithm N2PS prunes maximum or equal input neurons for all datasets than Xing-Hu and for 3 out of 4 datasets than VNP. Considering the pruning speed of N2PS, when a network is pruned, VNP starts retraining of the reduced model on new initial random weights which may lead to the increase in number of iterations in each pruning step and decrease in classifi- 114
22 cation accuracy. But N2PS inherits the initial weights from previous step for the retraining process of the pruned network as Xing-HU (Xing & Hu, 2009). Unfortunately Xing-Hu requires more number of pruning and retraining steps for selecting the relevant input units in phase I and for removing the irrelevant hidden units in phase II. But the proposed method N2PS requires maximum 3 pruning steps only. Table 5.2 shows N2PS removes 17 hidden neurons and 10 input neurons of hepatitis dataset in a single pruning step itself. The maximum number of pruning steps required by N2PS for four data sets iris, cancer, hepatitis and diabetes respectively are 2, 2, 3 and 2 only while VNP requires 3, 7, 3 and 7. While comparing the maximum number of retraining iterations required by Xing-Hu and N2PS for the pruned network on four data sets iris, cancer, hepatitis and diabetes, Xing-Hu requires 1000, 100, 100 and 100 iterations respectively but N2PS requires only 27, 50, 50 and 50 iterations respectively. This reduction in number of pruning steps and number of retraining iterations even resulted in a better generalization than the original network and the pruned networks of the other pruning algorithms. In summary, the experimental results consistently indicate that the N2PS algorithm can reduce the neural network size significantly without reducing the network performance and hence the algorithm can be applied for the rapid removal of more irrelevant neurons from a network with large size. However when no neurons can be further removed, any weight elimination methods (Setiono & Liu 1995; Huynh & Setiono, 2005) can be used to remove single connections, for achieving more improvement in classification accuracy and optimization. 115
23 5.5 Conclusions A new pruning algorithm to determine the optimal architecture for feedforward neural network has been proposed based on new significance measure which is estimated using the Sigmoidal function and weights. Simulation results indicate that the proposed algorithm is very efficient in identifying insignificant input and hidden neurons and also confirm that the pruned neural network yields better accurate results than the original neural network used in the training phase. The main advantages of this algorithm are, no user defined parameters needs to be set, large decrease in number of nodes without affecting the classification accuracy, requires small number of pruning steps and requires small number of iterations for retraining the pruned network compared with other pruning methods and achieves better generalization ability on all datasets. The experimental results demonstrate that the proposed N2PS algorithm is very promising method for determining the optimal architecture of neural networks of arbitrary topology for classifying large datasets. 116
Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems
Chapter 6 Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems 6.1 Introduction The primary purpose of data mining is not simply classification, but to provide meaningful
More informationLECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS
LECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS Neural Networks Classifier Introduction INPUT: classification data, i.e. it contains an classification (class) attribute. WE also say that the class
More informationSupervised Learning in Neural Networks (Part 2)
Supervised Learning in Neural Networks (Part 2) Multilayer neural networks (back-propagation training algorithm) The input signals are propagated in a forward direction on a layer-bylayer basis. Learning
More informationOptimal Brain Damage. Yann Le Cun, John S. Denker and Sara A. Solla. presented by Chaitanya Polumetla
Optimal Brain Damage Yann Le Cun, John S. Denker and Sara A. Solla presented by Chaitanya Polumetla Overview Introduction Need for OBD The Idea Authors Proposal Why OBD could work? Experiments Results
More informationNotes on Multilayer, Feedforward Neural Networks
Notes on Multilayer, Feedforward Neural Networks CS425/528: Machine Learning Fall 2012 Prepared by: Lynne E. Parker [Material in these notes was gleaned from various sources, including E. Alpaydin s book
More informationComparative Study of Instance Based Learning and Back Propagation for Classification Problems
Comparative Study of Instance Based Learning and Back Propagation for Classification Problems 1 Nadia Kanwal, 2 Erkan Bostanci 1 Department of Computer Science, Lahore College for Women University, Lahore,
More informationAn Algorithm For Training Multilayer Perceptron (MLP) For Image Reconstruction Using Neural Network Without Overfitting.
An Algorithm For Training Multilayer Perceptron (MLP) For Image Reconstruction Using Neural Network Without Overfitting. Mohammad Mahmudul Alam Mia, Shovasis Kumar Biswas, Monalisa Chowdhury Urmi, Abubakar
More informationNeural Network Weight Selection Using Genetic Algorithms
Neural Network Weight Selection Using Genetic Algorithms David Montana presented by: Carl Fink, Hongyi Chen, Jack Cheng, Xinglong Li, Bruce Lin, Chongjie Zhang April 12, 2005 1 Neural Networks Neural networks
More informationClassification Lecture Notes cse352. Neural Networks. Professor Anita Wasilewska
Classification Lecture Notes cse352 Neural Networks Professor Anita Wasilewska Neural Networks Classification Introduction INPUT: classification data, i.e. it contains an classification (class) attribute
More informationAssignment 2. Classification and Regression using Linear Networks, Multilayer Perceptron Networks, and Radial Basis Functions
ENEE 739Q: STATISTICAL AND NEURAL PATTERN RECOGNITION Spring 2002 Assignment 2 Classification and Regression using Linear Networks, Multilayer Perceptron Networks, and Radial Basis Functions Aravind Sundaresan
More informationVisual object classification by sparse convolutional neural networks
Visual object classification by sparse convolutional neural networks Alexander Gepperth 1 1- Ruhr-Universität Bochum - Institute for Neural Dynamics Universitätsstraße 150, 44801 Bochum - Germany Abstract.
More informationA Network Intrusion Detection System Architecture Based on Snort and. Computational Intelligence
2nd International Conference on Electronics, Network and Computer Engineering (ICENCE 206) A Network Intrusion Detection System Architecture Based on Snort and Computational Intelligence Tao Liu, a, Da
More informationA Novel Technique for Optimizing the Hidden Layer Architecture in Artificial Neural Networks N. M. Wagarachchi 1, A. S.
American International Journal of Research in Science, Technology, Engineering & Mathematics Available online at http://www.iasir.net ISSN (Print): 2328-3491, ISSN (Online): 2328-3580, ISSN (CD-ROM): 2328-3629
More informationClassification and Regression using Linear Networks, Multilayer Perceptrons and Radial Basis Functions
ENEE 739Q SPRING 2002 COURSE ASSIGNMENT 2 REPORT 1 Classification and Regression using Linear Networks, Multilayer Perceptrons and Radial Basis Functions Vikas Chandrakant Raykar Abstract The aim of the
More informationNEURAL NETWORK-BASED SEGMENTATION OF TEXTURES USING GABOR FEATURES
NEURAL NETWORK-BASED SEGMENTATION OF TEXTURES USING GABOR FEATURES A. G. Ramakrishnan, S. Kumar Raja, and H. V. Raghu Ram Dept. of Electrical Engg., Indian Institute of Science Bangalore - 560 012, India
More informationSimulation of Zhang Suen Algorithm using Feed- Forward Neural Networks
Simulation of Zhang Suen Algorithm using Feed- Forward Neural Networks Ritika Luthra Research Scholar Chandigarh University Gulshan Goyal Associate Professor Chandigarh University ABSTRACT Image Skeletonization
More informationImage Compression: An Artificial Neural Network Approach
Image Compression: An Artificial Neural Network Approach Anjana B 1, Mrs Shreeja R 2 1 Department of Computer Science and Engineering, Calicut University, Kuttippuram 2 Department of Computer Science and
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou Sun yzsun@ccs.neu.edu November 19, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining
More informationWeek 3: Perceptron and Multi-layer Perceptron
Week 3: Perceptron and Multi-layer Perceptron Phong Le, Willem Zuidema November 12, 2013 Last week we studied two famous biological neuron models, Fitzhugh-Nagumo model and Izhikevich model. This week,
More informationCHAPTER 3 RESEARCH METHODOLOGY
CHAPTER 3 RESEARCH METHODOLOGY 3.1 Introduction This chapter discusses the methodology that is used in this study. The first section describes the steps involve, follows by dataset representation. The
More informationAkarsh Pokkunuru EECS Department Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Akarsh Pokkunuru EECS Department 03-16-2017 Contractive Auto-Encoders: Explicit Invariance During Feature Extraction 1 AGENDA Introduction to Auto-encoders Types of Auto-encoders Analysis of different
More informationCS 6501: Deep Learning for Computer Graphics. Training Neural Networks II. Connelly Barnes
CS 6501: Deep Learning for Computer Graphics Training Neural Networks II Connelly Barnes Overview Preprocessing Initialization Vanishing/exploding gradients problem Batch normalization Dropout Additional
More informationAssignment # 5. Farrukh Jabeen Due Date: November 2, Neural Networks: Backpropation
Farrukh Jabeen Due Date: November 2, 2009. Neural Networks: Backpropation Assignment # 5 The "Backpropagation" method is one of the most popular methods of "learning" by a neural network. Read the class
More informationEnsemble methods in machine learning. Example. Neural networks. Neural networks
Ensemble methods in machine learning Bootstrap aggregating (bagging) train an ensemble of models based on randomly resampled versions of the training set, then take a majority vote Example What if you
More informationTraffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers
Traffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers A. Salhi, B. Minaoui, M. Fakir, H. Chakib, H. Grimech Faculty of science and Technology Sultan Moulay Slimane
More information2. Neural network basics
2. Neural network basics Next commonalities among different neural networks are discussed in order to get started and show which structural parts or concepts appear in almost all networks. It is presented
More informationA Data Classification Algorithm of Internet of Things Based on Neural Network
A Data Classification Algorithm of Internet of Things Based on Neural Network https://doi.org/10.3991/ijoe.v13i09.7587 Zhenjun Li Hunan Radio and TV University, Hunan, China 278060389@qq.com Abstract To
More informationThe exam is closed book, closed notes except your one-page cheat sheet.
CS 189 Fall 2015 Introduction to Machine Learning Final Please do not turn over the page before you are instructed to do so. You have 2 hours and 50 minutes. Please write your initials on the top-right
More informationSNIWD: Simultaneous Weight Noise Injection With Weight Decay for MLP Training
SNIWD: Simultaneous Weight Noise Injection With Weight Decay for MLP Training John Sum and Kevin Ho Institute of Technology Management, National Chung Hsing University Taichung 4, Taiwan. pfsum@nchu.edu.tw
More informationCOMPUTATIONAL INTELLIGENCE
COMPUTATIONAL INTELLIGENCE Fundamentals Adrian Horzyk Preface Before we can proceed to discuss specific complex methods we have to introduce basic concepts, principles, and models of computational intelligence
More informationNeural Network Neurons
Neural Networks Neural Network Neurons 1 Receives n inputs (plus a bias term) Multiplies each input by its weight Applies activation function to the sum of results Outputs result Activation Functions Given
More information11/14/2010 Intelligent Systems and Soft Computing 1
Lecture 7 Artificial neural networks: Supervised learning Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in
More informationCse634 DATA MINING TEST REVIEW. Professor Anita Wasilewska Computer Science Department Stony Brook University
Cse634 DATA MINING TEST REVIEW Professor Anita Wasilewska Computer Science Department Stony Brook University Preprocessing stage Preprocessing: includes all the operations that have to be performed before
More informationAMOL MUKUND LONDHE, DR.CHELPA LINGAM
International Journal of Advances in Applied Science and Engineering (IJAEAS) ISSN (P): 2348-1811; ISSN (E): 2348-182X Vol. 2, Issue 4, Dec 2015, 53-58 IIST COMPARATIVE ANALYSIS OF ANN WITH TRADITIONAL
More informationArtificial Neural Networks Lecture Notes Part 5. Stephen Lucci, PhD. Part 5
Artificial Neural Networks Lecture Notes Part 5 About this file: If you have trouble reading the contents of this file, or in case of transcription errors, email gi0062@bcmail.brooklyn.cuny.edu Acknowledgments:
More informationArtificial Neuron Modelling Based on Wave Shape
Artificial Neuron Modelling Based on Wave Shape Kieran Greer, Distributed Computing Systems, Belfast, UK. http://distributedcomputingsystems.co.uk Version 1.2 Abstract This paper describes a new model
More informationIEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 6, NOVEMBER Inverting Feedforward Neural Networks Using Linear and Nonlinear Programming
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 6, NOVEMBER 1999 1271 Inverting Feedforward Neural Networks Using Linear and Nonlinear Programming Bao-Liang Lu, Member, IEEE, Hajime Kita, and Yoshikazu
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationPerformance Analysis of Data Mining Classification Techniques
Performance Analysis of Data Mining Classification Techniques Tejas Mehta 1, Dr. Dhaval Kathiriya 2 Ph.D. Student, School of Computer Science, Dr. Babasaheb Ambedkar Open University, Gujarat, India 1 Principal
More informationCharacter Recognition Using Convolutional Neural Networks
Character Recognition Using Convolutional Neural Networks David Bouchain Seminar Statistical Learning Theory University of Ulm, Germany Institute for Neural Information Processing Winter 2006/2007 Abstract
More informationA neural-networks associative classification method for association rule mining
Data Mining VII: Data, Text and Web Mining and their Business Applications 93 A neural-networks associative classification method for association rule mining P. Sermswatsri & C. Srisa-an Faculty of Information
More informationPerformance analysis of a MLP weight initialization algorithm
Performance analysis of a MLP weight initialization algorithm Mohamed Karouia (1,2), Régis Lengellé (1) and Thierry Denœux (1) (1) Université de Compiègne U.R.A. CNRS 817 Heudiasyc BP 49 - F-2 Compiègne
More informationFigure (5) Kohonen Self-Organized Map
2- KOHONEN SELF-ORGANIZING MAPS (SOM) - The self-organizing neural networks assume a topological structure among the cluster units. - There are m cluster units, arranged in a one- or two-dimensional array;
More informationNeural Networks Laboratory EE 329 A
Neural Networks Laboratory EE 329 A Introduction: Artificial Neural Networks (ANN) are widely used to approximate complex systems that are difficult to model using conventional modeling techniques such
More informationCGBoost: Conjugate Gradient in Function Space
CGBoost: Conjugate Gradient in Function Space Ling Li Yaser S. Abu-Mostafa Amrit Pratap Learning Systems Group, California Institute of Technology, Pasadena, CA 91125, USA {ling,yaser,amrit}@caltech.edu
More informationNeural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer
More informationReification of Boolean Logic
Chapter Reification of Boolean Logic Exercises. (a) Design a feedforward network to divide the black dots from other corners with fewest neurons and layers. Please specify the values of weights and thresholds.
More informationBack propagation Algorithm:
Network Neural: A neural network is a class of computing system. They are created from very simple processing nodes formed into a network. They are inspired by the way that biological systems such as the
More informationArgha Roy* Dept. of CSE Netaji Subhash Engg. College West Bengal, India.
Volume 3, Issue 3, March 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Training Artificial
More informationCS 4510/9010 Applied Machine Learning. Neural Nets. Paula Matuszek Fall copyright Paula Matuszek 2016
CS 4510/9010 Applied Machine Learning 1 Neural Nets Paula Matuszek Fall 2016 Neural Nets, the very short version 2 A neural net consists of layers of nodes, or neurons, each of which has an activation
More informationPublication A Institute of Electrical and Electronics Engineers (IEEE)
Publication A Yoan Miche, Antti Sorjamaa, Patrick Bas, Olli Simula, Christian Jutten, and Amaury Lendasse. 2010. OP ELM: Optimally Pruned Extreme Learning Machine. IEEE Transactions on Neural Networks,
More informationRegularization and model selection
CS229 Lecture notes Andrew Ng Part VI Regularization and model selection Suppose we are trying select among several different models for a learning problem. For instance, we might be using a polynomial
More informationBest First and Greedy Search Based CFS and Naïve Bayes Algorithms for Hepatitis Diagnosis
Best First and Greedy Search Based CFS and Naïve Bayes Algorithms for Hepatitis Diagnosis CHAPTER 3 BEST FIRST AND GREEDY SEARCH BASED CFS AND NAÏVE BAYES ALGORITHMS FOR HEPATITIS DIAGNOSIS 3.1 Introduction
More informationOverlapping Swarm Intelligence for Training Artificial Neural Networks
Overlapping Swarm Intelligence for Training Artificial Neural Networks Karthik Ganesan Pillai Department of Computer Science Montana State University EPS 357, PO Box 173880 Bozeman, MT 59717-3880 k.ganesanpillai@cs.montana.edu
More informationOMBP: Optic Modified BackPropagation training algorithm for fast convergence of Feedforward Neural Network
2011 International Conference on Telecommunication Technology and Applications Proc.of CSIT vol.5 (2011) (2011) IACSIT Press, Singapore OMBP: Optic Modified BackPropagation training algorithm for fast
More information5 Learning hypothesis classes (16 points)
5 Learning hypothesis classes (16 points) Consider a classification problem with two real valued inputs. For each of the following algorithms, specify all of the separators below that it could have generated
More informationDeep Learning for Visual Computing Prof. Debdoot Sheet Department of Electrical Engineering Indian Institute of Technology, Kharagpur
Deep Learning for Visual Computing Prof. Debdoot Sheet Department of Electrical Engineering Indian Institute of Technology, Kharagpur Lecture - 05 Classification with Perceptron Model So, welcome to today
More informationNeural Network Classifier for Isolated Character Recognition
Neural Network Classifier for Isolated Character Recognition 1 Ruby Mehta, 2 Ravneet Kaur 1 M.Tech (CSE), Guru Nanak Dev University, Amritsar (Punjab), India 2 M.Tech Scholar, Computer Science & Engineering
More informationMachine Learning (CSE 446): Unsupervised Learning
Machine Learning (CSE 446): Unsupervised Learning Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu 1 / 19 Announcements HW2 posted. Due Feb 1. It is long. Start this week! Today:
More informationEffect of the PSO Topologies on the Performance of the PSO-ELM
2012 Brazilian Symposium on Neural Networks Effect of the PSO Topologies on the Performance of the PSO-ELM Elliackin M. N. Figueiredo and Teresa B. Ludermir Center of Informatics Federal University of
More informationCHAPTER 6 MODIFIED FUZZY TECHNIQUES BASED IMAGE SEGMENTATION
CHAPTER 6 MODIFIED FUZZY TECHNIQUES BASED IMAGE SEGMENTATION 6.1 INTRODUCTION Fuzzy logic based computational techniques are becoming increasingly important in the medical image analysis arena. The significant
More informationThis leads to our algorithm which is outlined in Section III, along with a tabular summary of it's performance on several benchmarks. The last section
An Algorithm for Incremental Construction of Feedforward Networks of Threshold Units with Real Valued Inputs Dhananjay S. Phatak Electrical Engineering Department State University of New York, Binghamton,
More informationDr. Qadri Hamarsheh Supervised Learning in Neural Networks (Part 1) learning algorithm Δwkj wkj Theoretically practically
Supervised Learning in Neural Networks (Part 1) A prescribed set of well-defined rules for the solution of a learning problem is called a learning algorithm. Variety of learning algorithms are existing,
More informationLecture on Modeling Tools for Clustering & Regression
Lecture on Modeling Tools for Clustering & Regression CS 590.21 Analysis and Modeling of Brain Networks Department of Computer Science University of Crete Data Clustering Overview Organizing data into
More informationEffect of Hidden Layer Neurons on the Classification of Optical Character Recognition Typed Arabic Numerals
Journal of Computer Science (7): 578-58, 008 ISSN 59-66 008 Science Publications Effect of Hidden Layer Neurons on the Classification of Optical Character Recognition Typed Arabic Numerals Nidal F. Shilbayeh
More informationCOMPARISION OF REGRESSION WITH NEURAL NETWORK MODEL FOR THE VARIATION OF VANISHING POINT WITH VIEW ANGLE IN DEPTH ESTIMATION WITH VARYING BRIGHTNESS
International Journal of Advanced Trends in Computer Science and Engineering, Vol.2, No.1, Pages : 171-177 (2013) COMPARISION OF REGRESSION WITH NEURAL NETWORK MODEL FOR THE VARIATION OF VANISHING POINT
More informationarxiv: v1 [cs.lg] 25 Jan 2018
A New Backpropagation Algorithm without Gradient Descent arxiv:1802.00027v1 [cs.lg] 25 Jan 2018 Varun Ranganathan Student at PES University varunranga1997@hotmail.com January 2018 S. Natarajan Professor
More informationInversion of Fracture Parameters by Using the Artificial Neural Network
IC -AI 99 International Conference 57 Inversion of Fracture Parameters by Using the Artificial Neural Network Hengchang Dai, Xiang-Yang Li, and Colin MacBeth British Geological Survey, Murchison House,
More informationLinear Separability. Linear Separability. Capabilities of Threshold Neurons. Capabilities of Threshold Neurons. Capabilities of Threshold Neurons
Linear Separability Input space in the two-dimensional case (n = ): - - - - - - w =, w =, = - - - - - - w = -, w =, = - - - - - - w = -, w =, = Linear Separability So by varying the weights and the threshold,
More informationCS 4510/9010 Applied Machine Learning
CS 4510/9010 Applied Machine Learning Neural Nets Paula Matuszek Spring, 2015 1 Neural Nets, the very short version A neural net consists of layers of nodes, or neurons, each of which has an activation
More informationA new multilayer perceptron pruning algorithm for classification and regression applications
A new multilayer perceptron pruning algorithm for classification and regression applications Philippe Thomas, Marie-Christine Suhner To cite this version: Philippe Thomas, Marie-Christine Suhner. A new
More informationPattern Recognition 41 (2008) Contents lists available at ScienceDirect. Pattern Recognition
Pattern Recognition 41 (2008) 3706 -- 3719 Contents lists available at ScienceDirect Pattern Recognition journal homepage: www.elsevier.com/locate/pr Feature selection using localized generalization error
More informationLogical Rhythm - Class 3. August 27, 2018
Logical Rhythm - Class 3 August 27, 2018 In this Class Neural Networks (Intro To Deep Learning) Decision Trees Ensemble Methods(Random Forest) Hyperparameter Optimisation and Bias Variance Tradeoff Biological
More informationSimulation of Back Propagation Neural Network for Iris Flower Classification
American Journal of Engineering Research (AJER) e-issn: 2320-0847 p-issn : 2320-0936 Volume-6, Issue-1, pp-200-205 www.ajer.org Research Paper Open Access Simulation of Back Propagation Neural Network
More informationIN recent years, neural networks have attracted considerable attention
Multilayer Perceptron: Architecture Optimization and Training Hassan Ramchoun, Mohammed Amine Janati Idrissi, Youssef Ghanou, Mohamed Ettaouil Modeling and Scientific Computing Laboratory, Faculty of Science
More informationEfficient Training of RBF Neural Networks for Pattern Recognition
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 12, NO. 5, SEPTEMBER 2001 1235 Efficient Training of RBF Neural Networks for Pattern Recognition Francesco Lampariello and Marco Sciandrone Abstract The problem
More informationA Class of Instantaneously Trained Neural Networks
A Class of Instantaneously Trained Neural Networks Subhash Kak Department of Electrical & Computer Engineering, Louisiana State University, Baton Rouge, LA 70803-5901 May 7, 2002 Abstract This paper presents
More informationImproving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah
Improving the way neural networks learn Srikumar Ramalingam School of Computing University of Utah Reference Most of the slides are taken from the third chapter of the online book by Michael Nielson: neuralnetworksanddeeplearning.com
More informationThe Application Research of Neural Network in Embedded Intelligent Detection
The Application Research of Neural Network in Embedded Intelligent Detection Xiaodong Liu 1, Dongzhou Ning 1, Hubin Deng 2, and Jinhua Wang 1 1 Compute Center of Nanchang University, 330039, Nanchang,
More informationNetwork Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Sources Hastie, Tibshirani, Friedman: The Elements of Statistical Learning James, Witten, Hastie, Tibshirani: An Introduction to Statistical Learning Andrew Ng:
More informationForward Feature Selection Using Residual Mutual Information
Forward Feature Selection Using Residual Mutual Information Erik Schaffernicht, Christoph Möller, Klaus Debes and Horst-Michael Gross Ilmenau University of Technology - Neuroinformatics and Cognitive Robotics
More informationMATLAB representation of neural network Outline Neural network with single-layer of neurons. Neural network with multiple-layer of neurons.
MATLAB representation of neural network Outline Neural network with single-layer of neurons. Neural network with multiple-layer of neurons. Introduction: Neural Network topologies (Typical Architectures)
More informationEstablishing Virtual Private Network Bandwidth Requirement at the University of Wisconsin Foundation
Establishing Virtual Private Network Bandwidth Requirement at the University of Wisconsin Foundation by Joe Madden In conjunction with ECE 39 Introduction to Artificial Neural Networks and Fuzzy Systems
More informationAn Efficient Learning Scheme for Extreme Learning Machine and Its Application
An Efficient Learning Scheme for Extreme Learning Machine and Its Application Kheon-Hee Lee, Miso Jang, Keun Park, Dong-Chul Park, Yong-Mu Jeong and Soo-Young Min Abstract An efficient learning scheme
More informationUnit V. Neural Fuzzy System
Unit V Neural Fuzzy System 1 Fuzzy Set In the classical set, its characteristic function assigns a value of either 1 or 0 to each individual in the universal set, There by discriminating between members
More informationEE 589 INTRODUCTION TO ARTIFICIAL NETWORK REPORT OF THE TERM PROJECT REAL TIME ODOR RECOGNATION SYSTEM FATMA ÖZYURT SANCAR
EE 589 INTRODUCTION TO ARTIFICIAL NETWORK REPORT OF THE TERM PROJECT REAL TIME ODOR RECOGNATION SYSTEM FATMA ÖZYURT SANCAR 1.Introductıon. 2.Multi Layer Perception.. 3.Fuzzy C-Means Clustering.. 4.Real
More information6. Backpropagation training 6.1 Background
6. Backpropagation training 6.1 Background To understand well how a feedforward neural network is built and it functions, we consider its basic first steps. We return to its history for a while. In 1949
More informationIn this assignment, we investigated the use of neural networks for supervised classification
Paul Couchman Fabien Imbault Ronan Tigreat Gorka Urchegui Tellechea Classification assignment (group 6) Image processing MSc Embedded Systems March 2003 Classification includes a broad range of decision-theoric
More informationData Mining. Neural Networks
Data Mining Neural Networks Goals for this Unit Basic understanding of Neural Networks and how they work Ability to use Neural Networks to solve real problems Understand when neural networks may be most
More informationDeep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group
Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group 1D Input, 1D Output target input 2 2D Input, 1D Output: Data Distribution Complexity Imagine many dimensions (data occupies
More informationCHAPTER 7 MASS LOSS PREDICTION USING ARTIFICIAL NEURAL NETWORK (ANN)
128 CHAPTER 7 MASS LOSS PREDICTION USING ARTIFICIAL NEURAL NETWORK (ANN) Various mathematical techniques like regression analysis and software tools have helped to develop a model using equation, which
More informationUnivariate and Multivariate Decision Trees
Univariate and Multivariate Decision Trees Olcay Taner Yıldız and Ethem Alpaydın Department of Computer Engineering Boğaziçi University İstanbul 80815 Turkey Abstract. Univariate decision trees at each
More informationThe task of inductive learning from examples is to nd an approximate definition
1 Initializing Neural Networks using Decision Trees Arunava Banerjee 1.1 Introduction The task of inductive learning from examples is to nd an approximate definition for an unknown function f(x), given
More informationNeural Networks (Overview) Prof. Richard Zanibbi
Neural Networks (Overview) Prof. Richard Zanibbi Inspired by Biology Introduction But as used in pattern recognition research, have little relation with real neural systems (studied in neurology and neuroscience)
More informationPerceptrons and Backpropagation. Fabio Zachert Cognitive Modelling WiSe 2014/15
Perceptrons and Backpropagation Fabio Zachert Cognitive Modelling WiSe 2014/15 Content History Mathematical View of Perceptrons Network Structures Gradient Descent Backpropagation (Single-Layer-, Multilayer-Networks)
More informationClassification using Weka (Brain, Computation, and Neural Learning)
LOGO Classification using Weka (Brain, Computation, and Neural Learning) Jung-Woo Ha Agenda Classification General Concept Terminology Introduction to Weka Classification practice with Weka Problems: Pima
More informationGenerating the Reduced Set by Systematic Sampling
Generating the Reduced Set by Systematic Sampling Chien-Chung Chang and Yuh-Jye Lee Email: {D9115009, yuh-jye}@mail.ntust.edu.tw Department of Computer Science and Information Engineering National Taiwan
More informationRough Set Approach to Unsupervised Neural Network based Pattern Classifier
Rough Set Approach to Unsupervised Neural based Pattern Classifier Ashwin Kothari, Member IAENG, Avinash Keskar, Shreesha Srinath, and Rakesh Chalsani Abstract Early Convergence, input feature space with
More informationMLPQNA-LEMON Multi Layer Perceptron neural network trained by Quasi Newton or Levenberg-Marquardt optimization algorithms
MLPQNA-LEMON Multi Layer Perceptron neural network trained by Quasi Newton or Levenberg-Marquardt optimization algorithms 1 Introduction In supervised Machine Learning (ML) we have a set of data points
More informationMini-project 2 CMPSCI 689 Spring 2015 Due: Tuesday, April 07, in class
Mini-project 2 CMPSCI 689 Spring 2015 Due: Tuesday, April 07, in class Guidelines Submission. Submit a hardcopy of the report containing all the figures and printouts of code in class. For readability
More information