Backpropagation: In Search of Performance Parameters
|
|
- Abigail Montgomery
- 5 years ago
- Views:
Transcription
1 Bacpropagaton: In Search of Performance Parameters ANIL KUMAR ENUMULAPALLY, LINGGUO BU, and KHOSROW KAIKHAH, Ph.D. Computer Scence Department Texas State Unversty-San Marcos San Marcos, TX USA Abstract: - Ths wor s an extensve study of the bacpropagaton networ based on a new vsual tool, Equal Opportunty for Recognton (EOR) for all nputs to be recalled, whch s used to evaluate the overall networ performance, n partcular, ts generalzaton capabltes. The new procedure, EOR, s used as a means to assess the effect of other system parameters. Keywords: bacpropagaton, Networ evaluaton, generalzaton, EOR, Processng Elements (PEs) and parameters. Introducton A Bacpropagaton Networ s a multplayer, assocatve, and feed forward neural networ that features supervsed learnng usng gradent descent tranng procedure. Bac Propagaton s wdely used n applcatons nvolvng pattern recognton because of ts powerful capablty of generalzaton. Whle ts system structure and learnng algorthm are well documented, there exst no mathematcal crtera to assess the performance, partcularly the generalzaton capabltes, of the networ wth respect to such networ parameters as number of PEs on the hdden layer, the mean squared error, learnng rates, ntalzaton of weghts and thresholds. Searchng for a measure of system performance, we proposed a vsual method, the EOR plot, whch can be used as an ndcator of the overall system performance. Wth the ad of EOR plottng, we further studed the varous parameters of the system as they relate to the overall system behavor, ncludng MSE, hdden layer sze, learnng rates, weght and threshold ntalzaton, and threshold updatng. Descrpton of Experments To study the varous propertes of a bacpropagaton networ, we started wth 6 captal letters of the Englsh alphabet, each of whch s represented on a 4 by 4 grd as follows. Each grd was converted to a bnary vector of 576 elements. Fg. : Input patterns Each bnary vector s assocated wth a 8-bt ASCII code correspondng to the Englsh letter. Snce one hdden layer s generally suffcent for most applcatons [4], we have desgned a bacpropagaton networ of three layers, an nput layer wth 576 PEs, an output layer wth 8 PEs, and a hdden layer wth a varyng number of PEs. In the lght of the EOR (Equal Opportunty for Recognton) plot as presented below, we studed all parameters of a bacpropagaton networ based on our mplementaton of the networ on Mathematca 4. 3 EOR Bacpropagaton s a smple and powerful algorthm, yeldng satsfactory results f properly mplemented. Mathematcal crtera, however, are stll to be found that can be employed to evaluate system performance wth respect to such a networ parameters as the MSE, hdden layer sze, ntal weghts, and learnng rates. Many rules for choosng hdden
2 layer sze have been proposed, however none of them seem to be superor and all are result of some emprcal conjure. To guarantee the applcablty of a networ, however, some measures have to be taen to assess system performance. To avod overtranng, for example, constant montorng on system performance s necessary, ncludng the ncorporaton of test data n the process of tranng Gven a specfc applcaton, such as the recognton of the 6 captal Englsh letters, nose reducton and generalzaton capabltes n the presence of random nose are among essental requrements of the networ. In other words, we need to prove the probablstc performance of a networ so that, frst, all nput patterns can be recovered successfully wth an equal opportunty, and second, the probablty that an ndvdual nput can be recovered should meet the requrements of the applcaton. Both factors are related to all the parameters of a networ. In the absence of a mathematcal descrpton, we propose the EOR plot (Equal Opportunty for Recognton) as a vsual, probablstc method to evaluate system performance. Gven a set of system parameters, ncludng MSE, ntal weghts, thresholds, learnng rates, and hdden layer sze, we tran the networ and estmate the probablty of each ndvdual nput pattern correctly recognzed at a specfed rate of random nose. The latter could be done by repeatng the recall process on a suffcently large number of randomly corrupted nputs and montorng the behavor of the networ. After all ndvdual nputs have been processed, the performance of the networ can be analyzed usng EOR plots. Usng 9 hdden layer neurons wth a range of 00 to +00 for weght and threshold random ntalzaton, a learnng rate of, an MSE of 005, and random nose rates of 0% and 5%, respectvely, we obtan the followng EOR plots as an estmaton of the networ performance. p r o b a bnumber of teratons n hundreds l t y p r o b a bnumber of teratons n hundreds l t y P p Number of teratons n hundreds Fg. (a) Number of teratons n hundreds Fg. (b) Fg. : EOR plots for 0% & 5% nose respectvely Accordng to the two EOR plots, wth 0% random nose, each nput pattern can be correctly recognzed wth a probablty of over 90% n spte of the slght varatons; wth 5% random nose, all patterns can be recognzed. Fg 3 depcts sample letters wth 0% random nose. Fg. 3: Specmen wth 0% random nose All corrupted patterns can be correctly recovered wth a probablty of more than 90%. As shown by our experments, EOR plots can be used as an objectve descrpton of system performance. EOR plots can be utlzed n analyzng other networ parameters. 4 Results and Analyss 4. Mean Squared Error (MSE) MSE s generally used as an ndcator of networ convergence. However, MSE s not a suffcent factor and other networ characterstcs need to be consdered. Frst, we wll show that MSE s not always a suffcent descrptor of system performance. Usng 8 hdden layer PEs and an MSE of 005 wth a dfferent range for random ntalzaton of weghts and thresholds, we obtaned the followng 5% random nose EOR plots. In fgure
3 4(a), the range of weght and threshold ntalzaton s -.0 to +.0; n fgure 4(b), the range of weghts and threshold ntalzaton s - 05 to +05. p p Fg. 4(a) Fg. 4( b) Fg. 4: EOR plot for dfferent weght and threshold ntalzatons Second, gven a specfc topology of a networ, a small MSE does not always yeld better system performance. As shown by our experment, after a certan pont, the EOR plot remans vrtually the same wthout evdence of over-fttng. The followng results are obtaned usng 8 hdden layer nodes, a learnng rate of, and a range of 05 to +05 for weght and threshold random ntalzaton, at an MSE of 5, 05, and Fgure 5(c): M SE of 00 Fg 5: EOR plots for dffer ent MSE values Therefore, whle MSE s an mportant factor of a bacpropagaton networ, t s not suffcent for drawng conclusons about system performance. Other factors, ncludng weght ntalzaton and sze of the hdden layer also play an mportant role. 4. Weght Intalzaton In a three-layer networ, there are two weght sets. As a general rule, the weghts should be randomly ntalzed to small values to avod system oscllaton and as justfed by the dervatve of the actvaton functon. We started wth a range of.0 to +.0 and gradually reduced the range. We observed that smaller random ntalzaton yelds a better performance. For the followng graphs, a networ wth 8 hdden layer nodes used, together wth an MSE of 05, learnng rate of and dfferent ranges for weght and threshold ntalzaton. 0 5 Error Fg. 6(a) weght and threshold ntalzaton to + Fg. 5(a): MSE of 5 Fg. 5(b): MSE of 05 Fg. 6(b) weght and threshold ntalzaton -5 to +5
4 Fg. 6(c) weght and threshold ntalzaton to + the same MSE yelds smlar system performance regardless of the range of ntalzaton, and thus can be used to compare the effect of number of PEs on the hdden layer. Wth a small number of PEs on the hdden layer, compared to the nput and output layers the learnng curve exhbts a great deal of fluctuatons and does not converge to the specfed MSE. Ths mples the networ does not have enough learnng capacty.e. memory wth 3 hdden layer PEs, we observed the followng results: Fg. 6(d) weght and threshold ntalzato n 00 to Fg. 6(e) weght and threshold ntalzaton to Fg. 6: EOR plots usng varous ranges for weght and threshold random ntalzaton to +, -5 to +5, - to +, -00 to +00, and to +0000, respectvely. Although the weghts could all be ntalzed to zero, ths would result n a hghly symmetrcal networ and s thus created therefore; t s not a good choce for networ desgn. Ths emphaszes the statement made by Ramelhart et al.[6] Intal weghts of exactly 0 cannot be used, snce symmetres n the envronment are not suffcent to brea symmetres n ntal weghts. p Fg. 7(a) Learnng curve Fg. 7: Learnng curve and EO R plot for a networ wth 3 PEs on hdden layer Wth more PEs on the hdden layer, more nput patterns can be correctly recovered. Once the number of hdden layer PEs reaches an deal range the system performance stablzes and shows very lttle mprovement wth an addton of new PEs. The followng EOR plots are based on 5, 7, 9,, 4, 36, 48, 00 PEs, respectvely. p Fg. 7(b) EOR plot 4.3 Number of PEs on the Hdden Layer To study the effect of the number of PEs n the hdden layer on system performance, we performed a seres of experments where all weghts and thresholds were randomly ntalzed between 05 and +05, wth a fxed MSE of 005 and a learnng rate of. When the weghts are ntalzed to very small values, Fg. 8(a) E OR plot for 5 hdden layer PEs
5 p Fg. 8(b) EOR plot for 7 hdden layer PEs p Hdden layer PE s are the feature extractors. As the hdden layer sze ncreases, for a fxed error, the number of teratons to tran the networ converges to a value and wll not oscllate. Ths tells us after certan lmt the hdden layer sze does not have any effect on the number of teratons. Although the ncreasng the hdden layer sze brngs down the number of teratons there may not be much mprovement n the total tranng tme. Fg. 8(c) EOR plot for 4 hdden layer PEs p Fg. 8(d) EOR plot for 48 hdden layer PEs p Fg. 8(e) EOR plot for 00 hdden layer PEs Fg. 8: EOR plot for a networ wth 5, 7, 4, 48, 00 PEs on hdden layer, respectvely. We observed that usng a fxed MSE, the number of teratons s related to the number of PEs on the hdden layer by the followng curve. ter Fg. 9: Relatonshp between hdden layer sze and number of tranng teratons 4.4 Learnng Rates: Whle learnng rates are generally taen to be small numbers between 0 and, there s no crteron governng the selecton of a learnng rate. If t s too small, the error correcton s trval and the networ does not learn well, wth lttle chance of gettng out of a local mnmum; f t s too large, the learnng process s one of oscllaton, wth lttle chance of convergence to the necessary MSE. The tranng of a networ s amed at ts generalzaton performance, whch s acheved by system convergence, the speed of whch s adjusted by the learnng rates. To apprecate the effects of large learnng rates, consder the learnng curve of a networ wth 9 hdden layer PEs, a weght and threshold ntalzaton range of 05 to +05, and a learnng rate of 5, as depcted n fg e Fg. 0: Learnng curve at a hgh learnng rate (5). To assess the effect of learnng rate on system performance, we used a networ wth 6 hdden layer PEs, a range of 05 to +05 for weght and threshold ntalzaton, an MSE of 005, and varous learnng rates. Wth a learnng rate of 00 and 0% random nose, the EOR plots are as follows, correspondng to the learnng rate of 0,,, and. Networ dd not converge wth a Learnng rate of.0
6 Fg. (a) EOR plot at learnng rate of 0 Fg. (b) EOR plot at learnng rate of Fg. (d) EOR plot at learnng rate of Fg. : EOR plots at the learnng rates of 0,, and, respectvely. 4.5 Thresholds Thresholds, or bas, can be used on both the hdden layer and the output layer PEs, to fne-tune the system convergence. Each PE on the hdden and output layer can a threshold value, whch s updated drectly based on the delta value computed for that PE. The threshold updatng not only speeds up system convergence, but also t s potentally helpful n smoothng out system fluctuatons that mght be hard to deal wth usng weght updatng alone. n o = f ( a w θ ), () = Where O s the output of the th node on the hdden or output layer and θ s the correspondng threshold and f s the sgmod functon. If δ s the delta value for the node, θ should be updated as follows: θ ( t) = θ ( t ) εδ () where ε s the threshold learnng rate, δ s the delta value, and θ s the threshold value. 5 Conclusons As there are no formulas that can be readly used to evaluate the performance of a bacpropagaton networ, the Equal Opportunty for Recognton (EOR) plots represent a practcal tool for system assessment wth respect to the applcaton condtons. As a probablstc method, not only can t be used to descrbe system performance, t can also be ncorporated nto the recall process for demandng pattern recogntons. The EOR, has shown a great promse n fndng the optmal ntal condtons for our Neural Networ. The future wor can be n the drecton of fndng general prncples, to desgn a bacpropagaton networ wth near optmal ntal condtons, usng EOR. References: [] Freeman, James A. Smulatng Neural Networs. Addson-Wesley, 994. [] McAuley, Devn. The bacpropagaton networ: learnng by example, 997. [3] Mehrotra, Krshan, et al. Elements of artfcal neural networs, Cambrdge, MIT Press, 997. [4] Sureerattanan, Songyot, et al, New developments on bacpropagaton networ tranng, IEICE Trans., vol. E83-A, No. 6, pp , June, 000 [5] Bac Propagaton s Senstve to Intal Condtons (990) -John F. Kolen, Jordan B. Pollac [6] Learnng Representaton by Bac- Propagatng Errors. Nature 33: D. E. Rumelhart, G. E. Hnton, and R. J. Wllams [7] Sarle, Warren S. ftp://ftp.sas.com/pub/neural, 00.
Learning the Kernel Parameters in Kernel Minimum Distance Classifier
Learnng the Kernel Parameters n Kernel Mnmum Dstance Classfer Daoqang Zhang 1,, Songcan Chen and Zh-Hua Zhou 1* 1 Natonal Laboratory for Novel Software Technology Nanjng Unversty, Nanjng 193, Chna Department
More informationS1 Note. Basis functions.
S1 Note. Bass functons. Contents Types of bass functons...1 The Fourer bass...2 B-splne bass...3 Power and type I error rates wth dfferent numbers of bass functons...4 Table S1. Smulaton results of type
More informationCluster Analysis of Electrical Behavior
Journal of Computer and Communcatons, 205, 3, 88-93 Publshed Onlne May 205 n ScRes. http://www.scrp.org/ournal/cc http://dx.do.org/0.4236/cc.205.350 Cluster Analyss of Electrcal Behavor Ln Lu Ln Lu, School
More informationOutline. Type of Machine Learning. Examples of Application. Unsupervised Learning
Outlne Artfcal Intellgence and ts applcatons Lecture 8 Unsupervsed Learnng Professor Danel Yeung danyeung@eee.org Dr. Patrck Chan patrckchan@eee.org South Chna Unversty of Technology, Chna Introducton
More informationGeneral Vector Machine. Hong Zhao Department of Physics, Xiamen University
General Vector Machne Hong Zhao (zhaoh@xmu.edu.cn) Department of Physcs, Xamen Unversty The support vector machne (SVM) s an mportant class of learnng machnes for functon approach, pattern recognton, and
More informationSubspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;
Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features
More informationUnsupervised Learning
Pattern Recognton Lecture 8 Outlne Introducton Unsupervsed Learnng Parametrc VS Non-Parametrc Approach Mxture of Denstes Maxmum-Lkelhood Estmates Clusterng Prof. Danel Yeung School of Computer Scence and
More informationSkew Angle Estimation and Correction of Hand Written, Textual and Large areas of Non-Textual Document Images: A Novel Approach
Angle Estmaton and Correcton of Hand Wrtten, Textual and Large areas of Non-Textual Document Images: A Novel Approach D.R.Ramesh Babu Pyush M Kumat Mahesh D Dhannawat PES Insttute of Technology Research
More informationA Binarization Algorithm specialized on Document Images and Photos
A Bnarzaton Algorthm specalzed on Document mages and Photos Ergna Kavalleratou Dept. of nformaton and Communcaton Systems Engneerng Unversty of the Aegean kavalleratou@aegean.gr Abstract n ths paper, a
More informationHermite Splines in Lie Groups as Products of Geodesics
Hermte Splnes n Le Groups as Products of Geodescs Ethan Eade Updated May 28, 2017 1 Introducton 1.1 Goal Ths document defnes a curve n the Le group G parametrzed by tme and by structural parameters n the
More informationFEATURE EXTRACTION. Dr. K.Vijayarekha. Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur
FEATURE EXTRACTION Dr. K.Vjayarekha Assocate Dean School of Electrcal and Electroncs Engneerng SASTRA Unversty, Thanjavur613 41 Jont Intatve of IITs and IISc Funded by MHRD Page 1 of 8 Table of Contents
More informationSimulation Based Analysis of FAST TCP using OMNET++
Smulaton Based Analyss of FAST TCP usng OMNET++ Umar ul Hassan 04030038@lums.edu.pk Md Term Report CS678 Topcs n Internet Research Sprng, 2006 Introducton Internet traffc s doublng roughly every 3 months
More informationSmoothing Spline ANOVA for variable screening
Smoothng Splne ANOVA for varable screenng a useful tool for metamodels tranng and mult-objectve optmzaton L. Rcco, E. Rgon, A. Turco Outlne RSM Introducton Possble couplng Test case MOO MOO wth Game Theory
More informationCourse Introduction. Algorithm 8/31/2017. COSC 320 Advanced Data Structures and Algorithms. COSC 320 Advanced Data Structures and Algorithms
Course Introducton Course Topcs Exams, abs, Proects A quc loo at a few algorthms 1 Advanced Data Structures and Algorthms Descrpton: We are gong to dscuss algorthm complexty analyss, algorthm desgn technques
More informationOutline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:
Self-Organzng Maps (SOM) Turgay İBRİKÇİ, PhD. Outlne Introducton Structures of SOM SOM Archtecture Neghborhoods SOM Algorthm Examples Summary 1 2 Unsupervsed Hebban Learnng US Hebban Learnng, Cntd 3 A
More informationFeature Reduction and Selection
Feature Reducton and Selecton Dr. Shuang LIANG School of Software Engneerng TongJ Unversty Fall, 2012 Today s Topcs Introducton Problems of Dmensonalty Feature Reducton Statstc methods Prncpal Components
More informationNUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS
ARPN Journal of Engneerng and Appled Scences 006-017 Asan Research Publshng Network (ARPN). All rghts reserved. NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS Igor Grgoryev, Svetlana
More informationA Fast Visual Tracking Algorithm Based on Circle Pixels Matching
A Fast Vsual Trackng Algorthm Based on Crcle Pxels Matchng Zhqang Hou hou_zhq@sohu.com Chongzhao Han czhan@mal.xjtu.edu.cn Ln Zheng Abstract: A fast vsual trackng algorthm based on crcle pxels matchng
More informationSLAM Summer School 2006 Practical 2: SLAM using Monocular Vision
SLAM Summer School 2006 Practcal 2: SLAM usng Monocular Vson Javer Cvera, Unversty of Zaragoza Andrew J. Davson, Imperal College London J.M.M Montel, Unversty of Zaragoza. josemar@unzar.es, jcvera@unzar.es,
More informationMeta-heuristics for Multidimensional Knapsack Problems
2012 4th Internatonal Conference on Computer Research and Development IPCSIT vol.39 (2012) (2012) IACSIT Press, Sngapore Meta-heurstcs for Multdmensonal Knapsack Problems Zhbao Man + Computer Scence Department,
More informationMachine Learning 9. week
Machne Learnng 9. week Mappng Concept Radal Bass Functons (RBF) RBF Networks 1 Mappng It s probably the best scenaro for the classfcaton of two dataset s to separate them lnearly. As you see n the below
More informationTECHNIQUE OF FORMATION HOMOGENEOUS SAMPLE SAME OBJECTS. Muradaliyev A.Z.
TECHNIQUE OF FORMATION HOMOGENEOUS SAMPLE SAME OBJECTS Muradalyev AZ Azerbajan Scentfc-Research and Desgn-Prospectng Insttute of Energetc AZ1012, Ave HZardab-94 E-mal:aydn_murad@yahoocom Importance of
More informationEdge Detection in Noisy Images Using the Support Vector Machines
Edge Detecton n Nosy Images Usng the Support Vector Machnes Hlaro Gómez-Moreno, Saturnno Maldonado-Bascón, Francsco López-Ferreras Sgnal Theory and Communcatons Department. Unversty of Alcalá Crta. Madrd-Barcelona
More informationCompiler Design. Spring Register Allocation. Sample Exercises and Solutions. Prof. Pedro C. Diniz
Compler Desgn Sprng 2014 Regster Allocaton Sample Exercses and Solutons Prof. Pedro C. Dnz USC / Informaton Scences Insttute 4676 Admralty Way, Sute 1001 Marna del Rey, Calforna 90292 pedro@s.edu Regster
More informationLecture 5: Multilayer Perceptrons
Lecture 5: Multlayer Perceptrons Roger Grosse 1 Introducton So far, we ve only talked about lnear models: lnear regresson and lnear bnary classfers. We noted that there are functons that can t be represented
More informationSHAPE RECOGNITION METHOD BASED ON THE k-nearest NEIGHBOR RULE
SHAPE RECOGNITION METHOD BASED ON THE k-nearest NEIGHBOR RULE Dorna Purcaru Faculty of Automaton, Computers and Electroncs Unersty of Craoa 13 Al. I. Cuza Street, Craoa RO-1100 ROMANIA E-mal: dpurcaru@electroncs.uc.ro
More informationDetermining the Optimal Bandwidth Based on Multi-criterion Fusion
Proceedngs of 01 4th Internatonal Conference on Machne Learnng and Computng IPCSIT vol. 5 (01) (01) IACSIT Press, Sngapore Determnng the Optmal Bandwdth Based on Mult-crteron Fuson Ha-L Lang 1+, Xan-Mn
More informationSupport Vector Machines
/9/207 MIST.6060 Busness Intellgence and Data Mnng What are Support Vector Machnes? Support Vector Machnes Support Vector Machnes (SVMs) are supervsed learnng technques that analyze data and recognze patterns.
More informationClassifying Acoustic Transient Signals Using Artificial Intelligence
Classfyng Acoustc Transent Sgnals Usng Artfcal Intellgence Steve Sutton, Unversty of North Carolna At Wlmngton (suttons@charter.net) Greg Huff, Unversty of North Carolna At Wlmngton (jgh7476@uncwl.edu)
More informationA Modified Median Filter for the Removal of Impulse Noise Based on the Support Vector Machines
A Modfed Medan Flter for the Removal of Impulse Nose Based on the Support Vector Machnes H. GOMEZ-MORENO, S. MALDONADO-BASCON, F. LOPEZ-FERRERAS, M. UTRILLA- MANSO AND P. GIL-JIMENEZ Departamento de Teoría
More informationAn Optimal Algorithm for Prufer Codes *
J. Software Engneerng & Applcatons, 2009, 2: 111-115 do:10.4236/jsea.2009.22016 Publshed Onlne July 2009 (www.scrp.org/journal/jsea) An Optmal Algorthm for Prufer Codes * Xaodong Wang 1, 2, Le Wang 3,
More informationBOOSTING CLASSIFICATION ACCURACY WITH SAMPLES CHOSEN FROM A VALIDATION SET
1 BOOSTING CLASSIFICATION ACCURACY WITH SAMPLES CHOSEN FROM A VALIDATION SET TZU-CHENG CHUANG School of Electrcal and Computer Engneerng, Purdue Unversty, West Lafayette, Indana 47907 SAUL B. GELFAND School
More informationNeural Network Control for TCP Network Congestion
5 Amercan Control Conference June 8-, 5. Portland, OR, USA FrA3. Neural Network Control for TCP Network Congeston Hyun C. Cho, M. Sam Fadal, Hyunjeong Lee Electrcal Engneerng/6, Unversty of Nevada, Reno,
More informationA Statistical Model Selection Strategy Applied to Neural Networks
A Statstcal Model Selecton Strategy Appled to Neural Networks Joaquín Pzarro Elsa Guerrero Pedro L. Galndo joaqun.pzarro@uca.es elsa.guerrero@uca.es pedro.galndo@uca.es Dpto Lenguajes y Sstemas Informátcos
More informationTerm Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task
Proceedngs of NTCIR-6 Workshop Meetng, May 15-18, 2007, Tokyo, Japan Term Weghtng Classfcaton System Usng the Ch-square Statstc for the Classfcaton Subtask at NTCIR-6 Patent Retreval Task Kotaro Hashmoto
More informationAn Ensemble Learning algorithm for Blind Signal Separation Problem
An Ensemble Learnng algorthm for Blnd Sgnal Separaton Problem Yan L 1 and Peng Wen 1 Department of Mathematcs and Computng, Faculty of Engneerng and Surveyng The Unversty of Southern Queensland, Queensland,
More informationBiological Sequence Mining Using Plausible Neural Network and its Application to Exon/intron Boundaries Prediction
Bologcal Sequence Mnng Usng Plausble Neural Networ and ts Applcaton to Exon/ntron Boundares Predcton Kuochen L, Dar-en Chang, and Erc Roucha CECS, Unversty of Lousvlle, Lousvlle, KY 40292, USA Yuan Yan
More informationCS 534: Computer Vision Model Fitting
CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust
More informationAn Iterative Solution Approach to Process Plant Layout using Mixed Integer Optimisation
17 th European Symposum on Computer Aded Process Engneerng ESCAPE17 V. Plesu and P.S. Agach (Edtors) 2007 Elsever B.V. All rghts reserved. 1 An Iteratve Soluton Approach to Process Plant Layout usng Mxed
More informationComparison Study of Textural Descriptors for Training Neural Network Classifiers
Comparson Study of Textural Descrptors for Tranng Neural Network Classfers G.D. MAGOULAS (1) S.A. KARKANIS (1) D.A. KARRAS () and M.N. VRAHATIS (3) (1) Department of Informatcs Unversty of Athens GR-157.84
More informationSimulation: Solving Dynamic Models ABE 5646 Week 11 Chapter 2, Spring 2010
Smulaton: Solvng Dynamc Models ABE 5646 Week Chapter 2, Sprng 200 Week Descrpton Readng Materal Mar 5- Mar 9 Evaluatng [Crop] Models Comparng a model wth data - Graphcal, errors - Measures of agreement
More informationSolving two-person zero-sum game by Matlab
Appled Mechancs and Materals Onlne: 2011-02-02 ISSN: 1662-7482, Vols. 50-51, pp 262-265 do:10.4028/www.scentfc.net/amm.50-51.262 2011 Trans Tech Publcatons, Swtzerland Solvng two-person zero-sum game by
More informationSome material adapted from Mohamed Younis, UMBC CMSC 611 Spr 2003 course slides Some material adapted from Hennessy & Patterson / 2003 Elsevier
Some materal adapted from Mohamed Youns, UMBC CMSC 611 Spr 2003 course sldes Some materal adapted from Hennessy & Patterson / 2003 Elsever Scence Performance = 1 Executon tme Speedup = Performance (B)
More informationWishing you all a Total Quality New Year!
Total Qualty Management and Sx Sgma Post Graduate Program 214-15 Sesson 4 Vnay Kumar Kalakband Assstant Professor Operatons & Systems Area 1 Wshng you all a Total Qualty New Year! Hope you acheve Sx sgma
More informationThe Man-hour Estimation Models & Its Comparison of Interim Products Assembly for Shipbuilding
Internatonal Journal of Operatons Research Internatonal Journal of Operatons Research Vol., No., 9 4 (005) The Man-hour Estmaton Models & Its Comparson of Interm Products Assembly for Shpbuldng Bn Lu and
More informationBootstrapping Color Constancy
Bootstrappng Color Constancy Bran Funt and Vlad C. Carde * Smon Fraser Unversty Vancouver, Canada ABSTRACT Bootstrappng provdes a novel approach to tranng a neural network to estmate the chromatcty of
More informationSVM-based Learning for Multiple Model Estimation
SVM-based Learnng for Multple Model Estmaton Vladmr Cherkassky and Yunqan Ma Department of Electrcal and Computer Engneerng Unversty of Mnnesota Mnneapols, MN 55455 {cherkass,myq}@ece.umn.edu Abstract:
More informationParallelism for Nested Loops with Non-uniform and Flow Dependences
Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr
More informationPRÉSENTATIONS DE PROJETS
PRÉSENTATIONS DE PROJETS Rex Onlne (V. Atanasu) What s Rex? Rex s an onlne browser for collectons of wrtten documents [1]. Asde ths core functon t has however many other applcatons that make t nterestng
More informationMULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION
MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION Paulo Quntlano 1 & Antono Santa-Rosa 1 Federal Polce Department, Brasla, Brazl. E-mals: quntlano.pqs@dpf.gov.br and
More informationUnsupervised Learning and Clustering
Unsupervsed Learnng and Clusterng Why consder unlabeled samples?. Collectng and labelng large set of samples s costly Gettng recorded speech s free, labelng s tme consumng 2. Classfer could be desgned
More informationApplication of Maximum Entropy Markov Models on the Protein Secondary Structure Predictions
Applcaton of Maxmum Entropy Markov Models on the Proten Secondary Structure Predctons Yohan Km Department of Chemstry and Bochemstry Unversty of Calforna, San Dego La Jolla, CA 92093 ykm@ucsd.edu Abstract
More informationImproving anti-spam filtering, based on Naive Bayesian and neural networks in multi-agent filters
J. Appl. Envron. Bol. Sc., 5(7S)381-386, 2015 2015, TextRoad Publcaton ISSN: 2090-4274 Journal of Appled Envronmental and Bologcal Scences www.textroad.com Improvng ant-spam flterng, based on Nave Bayesan
More information12. Segmentation. Computer Engineering, i Sejong University. Dongil Han
Computer Vson 1. Segmentaton Computer Engneerng, Sejong Unversty Dongl Han Image Segmentaton t Image segmentaton Subdvdes an mage nto ts consttuent regons or objects - After an mage has been segmented,
More informationThe Comparison of Calibration Method of Binocular Stereo Vision System Ke Zhang a *, Zhao Gao b
3rd Internatonal Conference on Materal, Mechancal and Manufacturng Engneerng (IC3ME 2015) The Comparson of Calbraton Method of Bnocular Stereo Vson System Ke Zhang a *, Zhao Gao b College of Engneerng,
More informationOn the Virtues of Parameterized Uniform Crossover
On the Vrtues of Parameterzed Unform Crossover Wllam M. Spears Naval Research aboratory Washngton, D.C. 20375 USA spears@ac.nrl.navy.ml Kenneth A. De Jong George Mason Unversty Farfax, VA 22030 USA kdejong@ac.gmu.edu
More informationA New Token Allocation Algorithm for TCP Traffic in Diffserv Network
A New Token Allocaton Algorthm for TCP Traffc n Dffserv Network A New Token Allocaton Algorthm for TCP Traffc n Dffserv Network S. Sudha and N. Ammasagounden Natonal Insttute of Technology, Truchrappall,
More informationGA-Based Learning Algorithms to Identify Fuzzy Rules for Fuzzy Neural Networks
Seventh Internatonal Conference on Intellgent Systems Desgn and Applcatons GA-Based Learnng Algorthms to Identfy Fuzzy Rules for Fuzzy Neural Networks K Almejall, K Dahal, Member IEEE, and A Hossan, Member
More informationA New Feature of Uniformity of Image Texture Directions Coinciding with the Human Eyes Perception 1
A New Feature of Unformty of Image Texture Drectons Concdng wth the Human Eyes Percepton Xng-Jan He, De-Shuang Huang, Yue Zhang, Tat-Mng Lo 2, and Mchael R. Lyu 3 Intellgent Computng Lab, Insttute of Intellgent
More informationSignature and Lexicon Pruning Techniques
Sgnature and Lexcon Prunng Technques Srnvas Palla, Hansheng Le, Venu Govndaraju Centre for Unfed Bometrcs and Sensors Unversty at Buffalo {spalla2, hle, govnd}@cedar.buffalo.edu Abstract Handwrtten word
More informationModule Management Tool in Software Development Organizations
Journal of Computer Scence (5): 8-, 7 ISSN 59-66 7 Scence Publcatons Management Tool n Software Development Organzatons Ahmad A. Al-Rababah and Mohammad A. Al-Rababah Faculty of IT, Al-Ahlyyah Amman Unversty,
More informationImprovement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration
Improvement of Spatal Resoluton Usng BlockMatchng Based Moton Estmaton and Frame Integraton Danya Suga and Takayuk Hamamoto Graduate School of Engneerng, Tokyo Unversty of Scence, 6-3-1, Nuku, Katsuska-ku,
More informationy and the total sum of
Lnear regresson Testng for non-lnearty In analytcal chemstry, lnear regresson s commonly used n the constructon of calbraton functons requred for analytcal technques such as gas chromatography, atomc absorpton
More informationA Notable Swarm Approach to Evolve Neural Network for Classification in Data Mining
A Notable Swarm Approach to Evolve Neural Network for Classfcaton n Data Mnng Satchdananda Dehur 1, Bjan Bhar Mshra 2 and Sung-Bae Cho 1 1 Soft Computng Laboratory, Department of Computer Scence, Yonse
More informationCollaboratively Regularized Nearest Points for Set Based Recognition
Academc Center for Computng and Meda Studes, Kyoto Unversty Collaboratvely Regularzed Nearest Ponts for Set Based Recognton Yang Wu, Mchhko Mnoh, Masayuk Mukunok Kyoto Unversty 9/1/013 BMVC 013 @ Brstol,
More informationESTIMATION OF PROPER PARAMETER VALUES FOR DOCUMENT BINARIZATION
ESTIMATIO OF PROPER PARAMETER VALUES FOR OCUMET BIARIZATIO E. Badekas and. Papamarkos Image Processng and Multmeda Laboratory epartment of Electrcal & Computer Engneerng emocrtus Unversty of Thrace, 67
More informationAn Efficient Genetic Algorithm with Fuzzy c-means Clustering for Traveling Salesman Problem
An Effcent Genetc Algorthm wth Fuzzy c-means Clusterng for Travelng Salesman Problem Jong-Won Yoon and Sung-Bae Cho Dept. of Computer Scence Yonse Unversty Seoul, Korea jwyoon@sclab.yonse.ac.r, sbcho@cs.yonse.ac.r
More informationChinese Word Segmentation based on the Improved Particle Swarm Optimization Neural Networks
Chnese Word Segmentaton based on the Improved Partcle Swarm Optmzaton Neural Networks Ja He Computatonal Intellgence Laboratory School of Computer Scence and Engneerng, UESTC Chengdu, Chna Department of
More informationResearch of Neural Network Classifier Based on FCM and PSO for Breast Cancer Classification
Research of Neural Network Classfer Based on FCM and PSO for Breast Cancer Classfcaton Le Zhang 1, Ln Wang 1, Xujewen Wang 2, Keke Lu 2, and Ajth Abraham 3 1 Shandong Provncal Key Laboratory of Network
More informationConditional Speculative Decimal Addition*
Condtonal Speculatve Decmal Addton Alvaro Vazquez and Elsardo Antelo Dep. of Electronc and Computer Engneerng Unv. of Santago de Compostela, Span Ths work was supported n part by Xunta de Galca under grant
More informationFuzzy Filtering Algorithms for Image Processing: Performance Evaluation of Various Approaches
Proceedngs of the Internatonal Conference on Cognton and Recognton Fuzzy Flterng Algorthms for Image Processng: Performance Evaluaton of Varous Approaches Rajoo Pandey and Umesh Ghanekar Department of
More informationProblem Set 3 Solutions
Introducton to Algorthms October 4, 2002 Massachusetts Insttute of Technology 6046J/18410J Professors Erk Demane and Shaf Goldwasser Handout 14 Problem Set 3 Solutons (Exercses were not to be turned n,
More information6.854 Advanced Algorithms Petar Maymounkov Problem Set 11 (November 23, 2005) With: Benjamin Rossman, Oren Weimann, and Pouya Kheradpour
6.854 Advanced Algorthms Petar Maymounkov Problem Set 11 (November 23, 2005) Wth: Benjamn Rossman, Oren Wemann, and Pouya Kheradpour Problem 1. We reduce vertex cover to MAX-SAT wth weghts, such that the
More informationUser Authentication Based On Behavioral Mouse Dynamics Biometrics
User Authentcaton Based On Behavoral Mouse Dynamcs Bometrcs Chee-Hyung Yoon Danel Donghyun Km Department of Computer Scence Department of Computer Scence Stanford Unversty Stanford Unversty Stanford, CA
More informationA Bilinear Model for Sparse Coding
A Blnear Model for Sparse Codng Davd B. Grmes and Rajesh P. N. Rao Department of Computer Scence and Engneerng Unversty of Washngton Seattle, WA 98195-2350, U.S.A. grmes,rao @cs.washngton.edu Abstract
More informationLearning Non-Linearly Separable Boolean Functions With Linear Threshold Unit Trees and Madaline-Style Networks
In AAAI-93: Proceedngs of the 11th Natonal Conference on Artfcal Intellgence, 33-1. Menlo Park, CA: AAAI Press. Learnng Non-Lnearly Separable Boolean Functons Wth Lnear Threshold Unt Trees and Madalne-Style
More informationType-2 Fuzzy Non-uniform Rational B-spline Model with Type-2 Fuzzy Data
Malaysan Journal of Mathematcal Scences 11(S) Aprl : 35 46 (2017) Specal Issue: The 2nd Internatonal Conference and Workshop on Mathematcal Analyss (ICWOMA 2016) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES
More informationNetwork Intrusion Detection Based on PSO-SVM
TELKOMNIKA Indonesan Journal of Electrcal Engneerng Vol.1, No., February 014, pp. 150 ~ 1508 DOI: http://dx.do.org/10.11591/telkomnka.v1.386 150 Network Intruson Detecton Based on PSO-SVM Changsheng Xang*
More informationNAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics
Introducton G10 NAG Fortran Lbrary Chapter Introducton G10 Smoothng n Statstcs Contents 1 Scope of the Chapter... 2 2 Background to the Problems... 2 2.1 Smoothng Methods... 2 2.2 Smoothng Splnes and Regresson
More informationINTELLECT SENSING OF NEURAL NETWORK THAT TRAINED TO CLASSIFY COMPLEX SIGNALS. Reznik A. Galinskaya A.
Internatonal Journal "Informaton heores & Applcatons" Vol.10 173 INELLEC SENSING OF NEURAL NEWORK HA RAINED O CLASSIFY COMPLEX SIGNALS Reznk A. Galnskaya A. Abstract: An expermental comparson of nformaton
More informationAn algorithm for correcting mislabeled data
Intellgent Data Analyss 5 (2001) 491 2 491 IOS Press An algorthm for correctng mslabeled data Xnchuan Zeng and Tony R. Martnez Computer Scence Department, Brgham Young Unversty, Provo, UT 842, USA E-mal:
More informationClassifier Selection Based on Data Complexity Measures *
Classfer Selecton Based on Data Complexty Measures * Edth Hernández-Reyes, J.A. Carrasco-Ochoa, and J.Fco. Martínez-Trndad Natonal Insttute for Astrophyscs, Optcs and Electroncs, Lus Enrque Erro No.1 Sta.
More informationCorner-Based Image Alignment using Pyramid Structure with Gradient Vector Similarity
Journal of Sgnal and Informaton Processng, 013, 4, 114-119 do:10.436/jsp.013.43b00 Publshed Onlne August 013 (http://www.scrp.org/journal/jsp) Corner-Based Image Algnment usng Pyramd Structure wth Gradent
More informationOptimal connection strategies in one- and two-dimensional associative memory models
Optmal connecton strateges n one- and two-dmensonal assocatve memory models Lee Calcraft, Rod Adams, and Nel Davey School of Computer Scence, Unversty of Hertfordshre College lane, Hatfeld, Hertfordshre
More informationAPPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT
3. - 5. 5., Brno, Czech Republc, EU APPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT Abstract Josef TOŠENOVSKÝ ) Lenka MONSPORTOVÁ ) Flp TOŠENOVSKÝ
More informationPerformance Evaluation of Information Retrieval Systems
Why System Evaluaton? Performance Evaluaton of Informaton Retreval Systems Many sldes n ths secton are adapted from Prof. Joydeep Ghosh (UT ECE) who n turn adapted them from Prof. Dk Lee (Unv. of Scence
More informationRelated-Mode Attacks on CTR Encryption Mode
Internatonal Journal of Network Securty, Vol.4, No.3, PP.282 287, May 2007 282 Related-Mode Attacks on CTR Encrypton Mode Dayn Wang, Dongda Ln, and Wenlng Wu (Correspondng author: Dayn Wang) Key Laboratory
More informationThe Codesign Challenge
ECE 4530 Codesgn Challenge Fall 2007 Hardware/Software Codesgn The Codesgn Challenge Objectves In the codesgn challenge, your task s to accelerate a gven software reference mplementaton as fast as possble.
More informationLearning-Based Top-N Selection Query Evaluation over Relational Databases
Learnng-Based Top-N Selecton Query Evaluaton over Relatonal Databases Lang Zhu *, Wey Meng ** * School of Mathematcs and Computer Scence, Hebe Unversty, Baodng, Hebe 071002, Chna, zhu@mal.hbu.edu.cn **
More informationA mathematical programming approach to the analysis, design and scheduling of offshore oilfields
17 th European Symposum on Computer Aded Process Engneerng ESCAPE17 V. Plesu and P.S. Agach (Edtors) 2007 Elsever B.V. All rghts reserved. 1 A mathematcal programmng approach to the analyss, desgn and
More informationFitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros.
Fttng & Matchng Lecture 4 Prof. Bregler Sldes from: S. Lazebnk, S. Setz, M. Pollefeys, A. Effros. How do we buld panorama? We need to match (algn) mages Matchng wth Features Detect feature ponts n both
More informationReducing Frame Rate for Object Tracking
Reducng Frame Rate for Object Trackng Pavel Korshunov 1 and We Tsang Oo 2 1 Natonal Unversty of Sngapore, Sngapore 11977, pavelkor@comp.nus.edu.sg 2 Natonal Unversty of Sngapore, Sngapore 11977, oowt@comp.nus.edu.sg
More informationTsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance
Tsnghua Unversty at TAC 2009: Summarzng Mult-documents by Informaton Dstance Chong Long, Mnle Huang, Xaoyan Zhu State Key Laboratory of Intellgent Technology and Systems, Tsnghua Natonal Laboratory for
More informationA new segmentation algorithm for medical volume image based on K-means clustering
Avalable onlne www.jocpr.com Journal of Chemcal and harmaceutcal Research, 2013, 5(12):113-117 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCRC5 A new segmentaton algorthm for medcal volume mage based
More informationAn Improved Image Segmentation Algorithm Based on the Otsu Method
3th ACIS Internatonal Conference on Software Engneerng, Artfcal Intellgence, Networkng arallel/dstrbuted Computng An Improved Image Segmentaton Algorthm Based on the Otsu Method Mengxng Huang, enjao Yu,
More informationThe Discriminate Analysis and Dimension Reduction Methods of High Dimension
Open Journal of Socal Scences, 015, 3, 7-13 Publshed Onlne March 015 n ScRes. http://www.scrp.org/journal/jss http://dx.do.org/10.436/jss.015.3300 The Dscrmnate Analyss and Dmenson Reducton Methods of
More informationParallelization of a Series of Extreme Learning Machine Algorithms Based on Spark
Parallelzaton of a Seres of Extreme Machne Algorthms Based on Spark Tantan Lu, Zhy Fang, Chen Zhao, Yngmn Zhou College of Computer Scence and Technology Jln Unversty, JLU Changchun, Chna e-mal: lutt1992x@sna.com
More informationQuery Clustering Using a Hybrid Query Similarity Measure
Query clusterng usng a hybrd query smlarty measure Fu. L., Goh, D.H., & Foo, S. (2004). WSEAS Transacton on Computers, 3(3), 700-705. Query Clusterng Usng a Hybrd Query Smlarty Measure Ln Fu, Don Hoe-Lan
More informationLearning a Class-Specific Dictionary for Facial Expression Recognition
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 16, No 4 Sofa 016 Prnt ISSN: 1311-970; Onlne ISSN: 1314-4081 DOI: 10.1515/cat-016-0067 Learnng a Class-Specfc Dctonary for
More informationData Mining: Model Evaluation
Data Mnng: Model Evaluaton Aprl 16, 2013 1 Issues: Evaluatng Classfcaton Methods Accurac classfer accurac: predctng class label predctor accurac: guessng value of predcted attrbutes Speed tme to construct
More information