EXTENDED BIC CRITERION FOR MODEL SELECTION
|
|
- Deborah Tate
- 5 years ago
- Views:
Transcription
1 IDIAP RESEARCH REPORT EXTEDED BIC CRITERIO FOR ODEL SELECTIO Itshak Lapdot Andrew orrs IDIAP-RR-0-4 Dalle olle Insttute for Perceptual Artfcal Intellgence P.O.Box 59 artgny Valas Swtzerland phone fax e-mal secretarat@dap.ch nternet
2
3 IDIAP Research Report 0-4 EXTEDED BIC CRITERIO FOR ODEL SELECTIO Itshak Lapdot Andrew orrs OCTOBER 00 Abstract. odel selecton s commonly based on some varaton of the BIC or mnmum message length crtera, such as L and DL. In ether case the crteron s splt nto two terms: one for the model (data code length/model complexty) and one for the data gven the model (message length/data lkelhood). For problems such as change detecton, unsupervsed segmentaton or data clusterng t s common practce for the model term to comprse only a sum of sub-model terms. In ths paper t s shown that the full model complexty must also take nto account the number of sub models and the labels whch assgn data to each sub model. From ths analyss we derve an extended BIC approach (EBIC) for ths class of problem. Results wth artfcal data are gven to llustrate the propertes of ths procedure.
4 IDIAP-RR-0-4. Introducton In model selecton by mnmum two-part message length encodng, a penalty term s added to the data message length term to account for model complexty. Change detecton, segmentaton and clusterng are unsupervsed applcatons whch can apply the BIC, DL or L crtera for model selecton [,, 3]. In change detecton or segmentaton t s requred to dentfy change ponts n a data sequence at whch data should be separated and assgned to dfferent models. In clusterng unsequenced data must smlarly be assgned to some unspecfed number of one or more dfferent models. Wth the BIC model t s assumed that only the number of model parameters needs to be mnmzed, not the model code length. Ths model has been found to be successful both n segmentaton [], n clusterng [, 3]. Ths method, however, usually requres some emprcal adjustments, and does not usually take nto account the number of clusters, but only the number of parameters n the model for each data cluster. In clusterng under a mnmum duraton constrant, by Ajmera et al [4], the number of model parameters was constant although the number of clusters vared between one and 30,.e., no penalty was used accordng to the standard BIC. All these crtera were develop to estmate a model out of a known parametrc model class. In applcaton lke clusterng and change detecton t s requred to estmate more than one model from model class. In ths paper t s shown that extra terms for both the number of clusters and the labels whch assgn the data to each cluster must be added to the usual model code length for optmal model selecton. The prncple of two-part mnmum message length model selecton s brefly presented n Secton. The proposed extenson to the model code length s explaned n Secton 3. Secton 4 presents detals of the proposed extenson to the BIC message length approxmaton. Secton 5 presents some experments wth artfcal data, followed by a dscusson and concluson n Secton 6.. Two-Part essage Length nmum message length model selecton s based on the prncpal that the model whch best fts the unseen dstrbuton underlyng a gven set of model tranng data s the smplest model whch s able to ft the tranng data to some gven level of accuracy. It s very closely related to Bayesan model selecton, whch selects the model wth the maxmum posteror probablty for the gven tranng data. odel selecton uses ether one-part or two-part message length. One-part message length s used when the model (defned by a parameter vector, whch belongs to a known model class ) s fxed and known to both coder and decoder. In ths case the coder only has to send code for the data gven the esslen code _ length X. Two-part message length s used when the model model, = ( ) parameters,, are not known to the decoder, so that the coder must estmate and send code for both the model parameters and the data, esslen = code _ length + code _ length X, [5, 6]. Lke Bayesan model ( ) ( ) selecton [7], mnmum message length model selecton arses from nformaton theory as an optmal procedure for model selecton. In both cases the model code length (model complexty), as well as data code length (data lkelhood), must normally be taken nto account. Actually DL [5] and BIC [7] converged to the same formula f we replace the term of data message length by log-lkelhood and the model parameters are contnuous values that were quantzed wth unform dstrbuton over ther range.
5 3 IDIAP-RR Extended odel Code Length In DL and BIC model selecton, model complexty s approxmated as a smple functon of the number of model parameters,. It s easly shown that n clusterng, cluster model ( parameters vector of, ) wth a mxture model pdf for each of clusters, and wth a fxed combned number of mxture components, greater wll always result n a greater lkelhood, p X, = { } k = k, and hence smaller data code length. At the same tme, the number of model parameters does not change wth. Hence, f model complexty s measured by, alone, the mnmum code length clusterng wll always use as many clusters as possble, whch s absurd. It follows that BIC model selecton s not suffcent for data clusterng unless some extenson to the model structure code length (pror probablty) s taken nto account, as some functon of, n such way that, when, s constant, a larger number of clusters results n a hgher model complexty. One can argue that the full defnton of cluster model requres that the parameter vector, must be augmented by addng a parameter to specfy the number of clusters, and a set of data labels, = { L n} n= L ( s the sze of the data and assumed to be known). To analyze ths extended model we should consder two cases. In the frst case the data can be rearranged nto blocks n the same arbtrary order as the data clusters, but the order of the data wthn each block s not sgnfcant. Ths would apply, for example, f we wand to code a number of mages dvded nto themed groups. In ths case we only need to send the number of data ponts n each block, = { } k, nstead of k = L. As the total number of data ponts s known, then the sze of the last block does not need to be ncluded. If we can assume that the probablty dstrbutons for and (possbly unform) are known to both coder and decoder, then we must add the followng extra terms to the model code length: code _ length ( ) code _ length ( ) k = k _ ( ) +. Both terms code _ ( ) length and code length k must be non-redundant prefx codes that satsfed the raft nequalty message _ length( s ) ( s s S s an element n a set S that represent ether or k ). Therefore, f we allow the message length to be a fractonal, than ths quantty s gven n terms of log-probabltes as: ( P( )) log ( P( k )) log () k = In the second case the order of the data s mportant. Ths case s out the scope of ths report and wll not be dscussed. We only manson that the smplest soluton mght be to send all ( k ) the labels nstead of block length and than nstead of the term P( ). n= should present the term log ( P ( Ln )) k = log, there
6 IDIAP-RR Extended BIC (EBIC) for mult-cluster applcatons Gven two clusterng models based on the same model class, wth parameters and,, the BIC crteron for choosng whch model has the rght dmenson s and number of parameters. The, crteron to determne whch clusterng model s better; usng BIC s gven as follows: gven n terms of log-lkelhood l ( X, ) l X < l X >,, ( ) ( ) log ( ),, () The chosen model s the one wth the hgher value. The second term on the rght sde s the complexty penalty terms accordng to the dfference between numbers of parameters n each model, and the length of the nput data,. In applyng ths crteron n clusterng applcatons [, 3] have been found that t s necessary to retrospectvely ntroduce an arbtrary, emprcally found, postve scalng factor, λ, for BIC model complexty term. We now show how equaton () should be extended to take nto account the changng n cluster model complexty term gven n equaton (). Let us assume that we have two estmated models from parametrc model class of all mxture models of a gven dstrbuton famly, such as all possble Gaussan mxture model. The model wth parameters vector has, clusters and = k = k, mxture components ( k, number of mxture component n cluster k ). Frst consder the case where =. To understand how equatons () must change t wll be suffcent to fnd the values of, and,. For smplcty may assume that the number of parameters of each mxture component s a fxed at R. For a descrpton of the model accordng to the standard BIC or DL s requred to provde the followng number of parameters: R parameters for all the mxture components n all the clusters. Prors of the mxture components n each cluster, { Pm k } k, {,, m= } k =, parameters. Ths gves = R +, whch s ndependent of, and the decson s taken only accordng to the maxmum of the lkelhoods of the models. The nature of the parameters of the number of the clusters,, and the block length, k, that are nteger values, and they to be dfferent than the parameters, that assumed to be contnuous values, and can be analyzed separately n terms of the probabltes assocated wth each of these ntegers. If we wrte the BIC crteron ncludng terms for and,, than we wll get the followng: P ( ) ( k, ) P l ( X >, ) l ( X, ) ( ) ( ) P( k, ) k =,, k = < log + log + log (3) P In many cases t s reasonable that both and k, are unformly dstrbuted over fnte P + range, ( ) EBIC crteron: =. and P E B ( k, ) P max mn + BL = =. In ths case equaton (3) becomes the
7 5 IDIAP-RR-0-4 l X < l X >,, (, ) (, ) log ( ) + ( ) log ( PBL ) (4) If each segment can be any length n the nterval [, ] (case of maxmum uncertanty), than PBL = and the most smplfed verson of EBIC wll be: If, =, l X < l X >,, ( ) ( ) + log ( ),, (5), than nstead of equaton (5), the EBIC n wll be: ( ) log ( ) ε ( ) ( ) for some ( 0,] > (, ) (, ) ( ) log ( ) l X < l X (6) s the maxmal penalty. The actual penalty should therefore be scaled as log ε. As n BIC, the model penalty s a logarthmc functon of, whle the data loglkelhood s proportonal to. The model penalty term therefore more sgnfcant for small and wll have neglgble effect when s large. 5. Experments Two experments were conducted to llustrate the effect of EBIC model selecton. In the frst experment a data was generated from two Gaussans wth the same standard devatons,, t = and µ t=, t = µ, t. Two sets were generated: wth = 3 and = 8 ponts. Each Gaussan generated half of the data. Tests were made under dfferent constrans on the segment length,.e., t was assumed that several data pontes successvely generated from the same source and should be kept together. Segment length were,, 4, and 8. It should be mentoned that the hgher the segment length the less optmal a clusterng soluton, n terms of log-lkelhood, can be acheved. On the other hand more meanngful clusters may be produced. We compare one cluster wth two Gaussan mxture components aganst two clusters wth one mxture component each. Accordng to standard BIC no penalty should be used. Fgure shows the result of BIC (f BIC values are less than zero then one cluster s better otherwse two clusters are better). As can be seen a two-cluster model was always σ = and wth expectatons { µ 0.08t} 0 better. The black lne s the EBIC penalty value for ε =. It can be seen that there are no bg dfferences between BIC and EBIC except when the ambguty s hgh,.e. when there s a small amount of data, = 3, the Gaussans are close one to each other, µ < σ and there s a large duraton constrant, s = 8. Ths ndcates that when two clusters are smlar EBIC tends to prefer one more accurate cluster wth more mxture components.
8 IDIAP-RR Fg.: dfference between BIC and EBIC for dfferent expectaton values, segment length, and amount of data (a - = 3, b - = 8 ). s In the second experment µ = 0 for both Gaussans, σ = 0.7, 0.8, 0.9,.0 and all the other parameters are as n the frst experment. The results are presented n fgure. We can see that small µ n the frst experment, and σ close to one n the second experment, leads the two data sets to have smlar statstcal propertes. So, for small and large segment length the resultng clusters become smlar. Whle n BIC the decson wll be that the two-cluster model s better, EBIC wll prefer a one-cluster model. As was mentoned, a scale factor ε n all the experments was equal to one. If the scale factor was smaller, the system would be more based towards the two-cluster model. Ths parameter can be found emprcally (n the same way as the scale factor λ that s used wth the BIC crteron), or calculated accordng to some pror knowledge of another block length dstrbuton. σ =, { }
9 7 IDIAP-RR-0-4 Fg.: dfference between BIC and EBIC for dfferent standard devaton values, segment length, and amount of data (a - = 3, b - = 8 ). s 6. Dscusson It was shown that the clusterng model complexty s not only a functon of the number of parameters and ther values n the parameter vector,, but also the number of clusters, and nformaton about the labelng of each data vector { n} n= L. The labels must not be coded n a drect way, but n a compact way whch s just suffcent to permute the data nto the blocks as requred (n order to mnmze the number of parameters to be sent). The code length of such extra nformaton wll ncrease wth. It was shown that when there s small amount of data or some ambguty due to the compact nature of the data or clusterng constrans, the mportance of the addtonal penalty terms ncreases. Acknowledgment The authors want to thank the Swss Federal Offce for Educaton and Scence (OFES) n the framework of both the EC/OFES ultodal eetng anager (4) project and the Swss atonal Scence Foundaton through the atonal Center of Competence n Research (CCR) on Interactve ultmodal Informaton anagement (I) for supportng ths work.
10 IDIAP-RR References [] J. J. Olver, R. A. Baxter, and C. S. Wallace, Unsupervsed learnng usng L, Proc. 3 th Int. Conf. on achne Learnng, 996, pp. -0. []. Cettolo, Segmentaton, classfcaton and clusterng of an Italan broadcast news corpus, Proc. 6 th RIAO Conf., Aprl 000, pp [3] S. S. Chen and P. S. Gapalakrshnan, Clusterng va the Bayesan crteron wth applcatons to speech recognton, ICASSP 98, vol., 998, pp [4] J. Ajmera, H. Bourlard, I. Lapdot, and I. ccowan, Unknown-multple speaker clusterng usng H, ICSLP 0, USA, 00, pp [5] J. Rssanen, A unversal pror for ntegers and estmaton by mnmum descrpton length, The Annals of Statstcs, vol., no., pp , 983. [6] J. Olver and R. Baxter, L and Bayesansm: smlartes and dfferences: ntroducton to mnmum encodng nference Part II, Dep. Of Computer Scence, onash Unversty, Clayton, Vctora 368, Australa, Tech. Rep. TR-06, December 994. [7] G. Schwarz, Estmatng the dmenson of a model, The Annals of Statstcs, vol. 6, no., pp , 978.
CS 534: Computer Vision Model Fitting
CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust
More informationSupport Vector Machines
/9/207 MIST.6060 Busness Intellgence and Data Mnng What are Support Vector Machnes? Support Vector Machnes Support Vector Machnes (SVMs) are supervsed learnng technques that analyze data and recognze patterns.
More informationDetermining the Optimal Bandwidth Based on Multi-criterion Fusion
Proceedngs of 01 4th Internatonal Conference on Machne Learnng and Computng IPCSIT vol. 5 (01) (01) IACSIT Press, Sngapore Determnng the Optmal Bandwdth Based on Mult-crteron Fuson Ha-L Lang 1+, Xan-Mn
More informationA Robust Method for Estimating the Fundamental Matrix
Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney A Robust Method for Estmatng the Fundamental Matrx C.L. Feng and Y.S.
More informationHelsinki University Of Technology, Systems Analysis Laboratory Mat Independent research projects in applied mathematics (3 cr)
Helsnk Unversty Of Technology, Systems Analyss Laboratory Mat-2.08 Independent research projects n appled mathematcs (3 cr) "! #$&% Antt Laukkanen 506 R ajlaukka@cc.hut.f 2 Introducton...3 2 Multattrbute
More informationLearning the Kernel Parameters in Kernel Minimum Distance Classifier
Learnng the Kernel Parameters n Kernel Mnmum Dstance Classfer Daoqang Zhang 1,, Songcan Chen and Zh-Hua Zhou 1* 1 Natonal Laboratory for Novel Software Technology Nanjng Unversty, Nanjng 193, Chna Department
More informationA Binarization Algorithm specialized on Document Images and Photos
A Bnarzaton Algorthm specalzed on Document mages and Photos Ergna Kavalleratou Dept. of nformaton and Communcaton Systems Engneerng Unversty of the Aegean kavalleratou@aegean.gr Abstract n ths paper, a
More informationClassifier Selection Based on Data Complexity Measures *
Classfer Selecton Based on Data Complexty Measures * Edth Hernández-Reyes, J.A. Carrasco-Ochoa, and J.Fco. Martínez-Trndad Natonal Insttute for Astrophyscs, Optcs and Electroncs, Lus Enrque Erro No.1 Sta.
More informationTsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance
Tsnghua Unversty at TAC 2009: Summarzng Mult-documents by Informaton Dstance Chong Long, Mnle Huang, Xaoyan Zhu State Key Laboratory of Intellgent Technology and Systems, Tsnghua Natonal Laboratory for
More informationK-means and Hierarchical Clustering
Note to other teachers and users of these sldes. Andrew would be delghted f you found ths source materal useful n gvng your own lectures. Feel free to use these sldes verbatm, or to modfy them to ft your
More informationFor instance, ; the five basic number-sets are increasingly more n A B & B A A = B (1)
Secton 1.2 Subsets and the Boolean operatons on sets If every element of the set A s an element of the set B, we say that A s a subset of B, or that A s contaned n B, or that B contans A, and we wrte A
More informationSubspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;
Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features
More informationNUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS
ARPN Journal of Engneerng and Appled Scences 006-017 Asan Research Publshng Network (ARPN). All rghts reserved. NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS Igor Grgoryev, Svetlana
More informationS1 Note. Basis functions.
S1 Note. Bass functons. Contents Types of bass functons...1 The Fourer bass...2 B-splne bass...3 Power and type I error rates wth dfferent numbers of bass functons...4 Table S1. Smulaton results of type
More informationFeature Reduction and Selection
Feature Reducton and Selecton Dr. Shuang LIANG School of Software Engneerng TongJ Unversty Fall, 2012 Today s Topcs Introducton Problems of Dmensonalty Feature Reducton Statstc methods Prncpal Components
More informationThe Research of Support Vector Machine in Agricultural Data Classification
The Research of Support Vector Machne n Agrcultural Data Classfcaton Le Sh, Qguo Duan, Xnmng Ma, Me Weng College of Informaton and Management Scence, HeNan Agrcultural Unversty, Zhengzhou 45000 Chna Zhengzhou
More informationNAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics
Introducton G10 NAG Fortran Lbrary Chapter Introducton G10 Smoothng n Statstcs Contents 1 Scope of the Chapter... 2 2 Background to the Problems... 2 2.1 Smoothng Methods... 2 2.2 Smoothng Splnes and Regresson
More informationParallelism for Nested Loops with Non-uniform and Flow Dependences
Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr
More informationUnsupervised Learning and Clustering
Unsupervsed Learnng and Clusterng Why consder unlabeled samples?. Collectng and labelng large set of samples s costly Gettng recorded speech s free, labelng s tme consumng 2. Classfer could be desgned
More informationOutline. Type of Machine Learning. Examples of Application. Unsupervised Learning
Outlne Artfcal Intellgence and ts applcatons Lecture 8 Unsupervsed Learnng Professor Danel Yeung danyeung@eee.org Dr. Patrck Chan patrckchan@eee.org South Chna Unversty of Technology, Chna Introducton
More informationPositive Semi-definite Programming Localization in Wireless Sensor Networks
Postve Sem-defnte Programmng Localzaton n Wreless Sensor etworks Shengdong Xe 1,, Jn Wang, Aqun Hu 1, Yunl Gu, Jang Xu, 1 School of Informaton Scence and Engneerng, Southeast Unversty, 10096, anjng Computer
More informationA mathematical programming approach to the analysis, design and scheduling of offshore oilfields
17 th European Symposum on Computer Aded Process Engneerng ESCAPE17 V. Plesu and P.S. Agach (Edtors) 2007 Elsever B.V. All rghts reserved. 1 A mathematcal programmng approach to the analyss, desgn and
More informationy and the total sum of
Lnear regresson Testng for non-lnearty In analytcal chemstry, lnear regresson s commonly used n the constructon of calbraton functons requred for analytcal technques such as gas chromatography, atomc absorpton
More informationBiostatistics 615/815
The E-M Algorthm Bostatstcs 615/815 Lecture 17 Last Lecture: The Smplex Method General method for optmzaton Makes few assumptons about functon Crawls towards mnmum Some recommendatons Multple startng ponts
More informationUnsupervised Learning
Pattern Recognton Lecture 8 Outlne Introducton Unsupervsed Learnng Parametrc VS Non-Parametrc Approach Mxture of Denstes Maxmum-Lkelhood Estmates Clusterng Prof. Danel Yeung School of Computer Scence and
More informationClassifying Acoustic Transient Signals Using Artificial Intelligence
Classfyng Acoustc Transent Sgnals Usng Artfcal Intellgence Steve Sutton, Unversty of North Carolna At Wlmngton (suttons@charter.net) Greg Huff, Unversty of North Carolna At Wlmngton (jgh7476@uncwl.edu)
More informationTECHNIQUE OF FORMATION HOMOGENEOUS SAMPLE SAME OBJECTS. Muradaliyev A.Z.
TECHNIQUE OF FORMATION HOMOGENEOUS SAMPLE SAME OBJECTS Muradalyev AZ Azerbajan Scentfc-Research and Desgn-Prospectng Insttute of Energetc AZ1012, Ave HZardab-94 E-mal:aydn_murad@yahoocom Importance of
More informationTerm Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task
Proceedngs of NTCIR-6 Workshop Meetng, May 15-18, 2007, Tokyo, Japan Term Weghtng Classfcaton System Usng the Ch-square Statstc for the Classfcaton Subtask at NTCIR-6 Patent Retreval Task Kotaro Hashmoto
More informationWishing you all a Total Quality New Year!
Total Qualty Management and Sx Sgma Post Graduate Program 214-15 Sesson 4 Vnay Kumar Kalakband Assstant Professor Operatons & Systems Area 1 Wshng you all a Total Qualty New Year! Hope you acheve Sx sgma
More informationOutline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:
Self-Organzng Maps (SOM) Turgay İBRİKÇİ, PhD. Outlne Introducton Structures of SOM SOM Archtecture Neghborhoods SOM Algorthm Examples Summary 1 2 Unsupervsed Hebban Learnng US Hebban Learnng, Cntd 3 A
More informationEdge Detection in Noisy Images Using the Support Vector Machines
Edge Detecton n Nosy Images Usng the Support Vector Machnes Hlaro Gómez-Moreno, Saturnno Maldonado-Bascón, Francsco López-Ferreras Sgnal Theory and Communcatons Department. Unversty of Alcalá Crta. Madrd-Barcelona
More informationA Statistical Model Selection Strategy Applied to Neural Networks
A Statstcal Model Selecton Strategy Appled to Neural Networks Joaquín Pzarro Elsa Guerrero Pedro L. Galndo joaqun.pzarro@uca.es elsa.guerrero@uca.es pedro.galndo@uca.es Dpto Lenguajes y Sstemas Informátcos
More informationUser Authentication Based On Behavioral Mouse Dynamics Biometrics
User Authentcaton Based On Behavoral Mouse Dynamcs Bometrcs Chee-Hyung Yoon Danel Donghyun Km Department of Computer Scence Department of Computer Scence Stanford Unversty Stanford Unversty Stanford, CA
More informationBAYESIAN MULTI-SOURCE DOMAIN ADAPTATION
BAYESIAN MULTI-SOURCE DOMAIN ADAPTATION SHI-LIANG SUN, HONG-LEI SHI Department of Computer Scence and Technology, East Chna Normal Unversty 500 Dongchuan Road, Shangha 200241, P. R. Chna E-MAIL: slsun@cs.ecnu.edu.cn,
More informationFitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros.
Fttng & Matchng Lecture 4 Prof. Bregler Sldes from: S. Lazebnk, S. Setz, M. Pollefeys, A. Effros. How do we buld panorama? We need to match (algn) mages Matchng wth Features Detect feature ponts n both
More informationCompiler Design. Spring Register Allocation. Sample Exercises and Solutions. Prof. Pedro C. Diniz
Compler Desgn Sprng 2014 Regster Allocaton Sample Exercses and Solutons Prof. Pedro C. Dnz USC / Informaton Scences Insttute 4676 Admralty Way, Sute 1001 Marna del Rey, Calforna 90292 pedro@s.edu Regster
More informationEmpirical Distributions of Parameter Estimates. in Binary Logistic Regression Using Bootstrap
Int. Journal of Math. Analyss, Vol. 8, 4, no. 5, 7-7 HIKARI Ltd, www.m-hkar.com http://dx.do.org/.988/jma.4.494 Emprcal Dstrbutons of Parameter Estmates n Bnary Logstc Regresson Usng Bootstrap Anwar Ftranto*
More informationSVM-based Learning for Multiple Model Estimation
SVM-based Learnng for Multple Model Estmaton Vladmr Cherkassky and Yunqan Ma Department of Electrcal and Computer Engneerng Unversty of Mnnesota Mnneapols, MN 55455 {cherkass,myq}@ece.umn.edu Abstract:
More informationComplex Numbers. Now we also saw that if a and b were both positive then ab = a b. For a second let s forget that restriction and do the following.
Complex Numbers The last topc n ths secton s not really related to most of what we ve done n ths chapter, although t s somewhat related to the radcals secton as we wll see. We also won t need the materal
More informationHybridization of Expectation-Maximization and K-Means Algorithms for Better Clustering Performance
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 16, No 2 Sofa 2016 Prnt ISSN: 1311-9702; Onlne ISSN: 1314-4081 DOI: 10.1515/cat-2016-0017 Hybrdzaton of Expectaton-Maxmzaton
More informationAn Optimal Algorithm for Prufer Codes *
J. Software Engneerng & Applcatons, 2009, 2: 111-115 do:10.4236/jsea.2009.22016 Publshed Onlne July 2009 (www.scrp.org/journal/jsea) An Optmal Algorthm for Prufer Codes * Xaodong Wang 1, 2, Le Wang 3,
More informationProper Choice of Data Used for the Estimation of Datum Transformation Parameters
Proper Choce of Data Used for the Estmaton of Datum Transformaton Parameters Hakan S. KUTOGLU, Turkey Key words: Coordnate systems; transformaton; estmaton, relablty. SUMMARY Advances n technologes and
More informationContent Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers
IOSR Journal of Electroncs and Communcaton Engneerng (IOSR-JECE) e-issn: 78-834,p- ISSN: 78-8735.Volume 9, Issue, Ver. IV (Mar - Apr. 04), PP 0-07 Content Based Image Retreval Usng -D Dscrete Wavelet wth
More informationSum of Linear and Fractional Multiobjective Programming Problem under Fuzzy Rules Constraints
Australan Journal of Basc and Appled Scences, 2(4): 1204-1208, 2008 ISSN 1991-8178 Sum of Lnear and Fractonal Multobjectve Programmng Problem under Fuzzy Rules Constrants 1 2 Sanjay Jan and Kalash Lachhwan
More informationMixed Linear System Estimation and Identification
48th IEEE Conference on Decson and Control, Shangha, Chna, December 2009 Mxed Lnear System Estmaton and Identfcaton A. Zymns S. Boyd D. Gornevsky Abstract We consder a mxed lnear system model, wth both
More informationAn Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices
Internatonal Mathematcal Forum, Vol 7, 2012, no 52, 2549-2554 An Applcaton of the Dulmage-Mendelsohn Decomposton to Sparse Null Space Bases of Full Row Rank Matrces Mostafa Khorramzadeh Department of Mathematcal
More informationA MOVING MESH APPROACH FOR SIMULATION BUDGET ALLOCATION ON CONTINUOUS DOMAINS
Proceedngs of the Wnter Smulaton Conference M E Kuhl, N M Steger, F B Armstrong, and J A Jones, eds A MOVING MESH APPROACH FOR SIMULATION BUDGET ALLOCATION ON CONTINUOUS DOMAINS Mark W Brantley Chun-Hung
More informationMachine Learning: Algorithms and Applications
14/05/1 Machne Learnng: Algorthms and Applcatons Florano Zn Free Unversty of Bozen-Bolzano Faculty of Computer Scence Academc Year 011-01 Lecture 10: 14 May 01 Unsupervsed Learnng cont Sldes courtesy of
More informationProblem Definitions and Evaluation Criteria for Computational Expensive Optimization
Problem efntons and Evaluaton Crtera for Computatonal Expensve Optmzaton B. Lu 1, Q. Chen and Q. Zhang 3, J. J. Lang 4, P. N. Suganthan, B. Y. Qu 6 1 epartment of Computng, Glyndwr Unversty, UK Faclty
More informationMathematics 256 a course in differential equations for engineering students
Mathematcs 56 a course n dfferental equatons for engneerng students Chapter 5. More effcent methods of numercal soluton Euler s method s qute neffcent. Because the error s essentally proportonal to the
More informationLecture 5: Probability Distributions. Random Variables
Lecture 5: Probablty Dstrbutons Random Varables Probablty Dstrbutons Dscrete Random Varables Contnuous Random Varables and ther Dstrbutons Dscrete Jont Dstrbutons Contnuous Jont Dstrbutons Independent
More informationA New Approach For the Ranking of Fuzzy Sets With Different Heights
New pproach For the ankng of Fuzzy Sets Wth Dfferent Heghts Pushpnder Sngh School of Mathematcs Computer pplcatons Thapar Unversty, Patala-7 00 Inda pushpndersnl@gmalcom STCT ankng of fuzzy sets plays
More informationOutline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1
4/14/011 Outlne Dscrmnatve classfers for mage recognton Wednesday, Aprl 13 Krsten Grauman UT-Austn Last tme: wndow-based generc obect detecton basc ppelne face detecton wth boostng as case study Today:
More informationSome Advanced SPC Tools 1. Cumulative Sum Control (Cusum) Chart For the data shown in Table 9-1, the x chart can be generated.
Some Advanced SP Tools 1. umulatve Sum ontrol (usum) hart For the data shown n Table 9-1, the x chart can be generated. However, the shft taken place at sample #21 s not apparent. 92 For ths set samples,
More informationRelated-Mode Attacks on CTR Encryption Mode
Internatonal Journal of Network Securty, Vol.4, No.3, PP.282 287, May 2007 282 Related-Mode Attacks on CTR Encrypton Mode Dayn Wang, Dongda Ln, and Wenlng Wu (Correspondng author: Dayn Wang) Key Laboratory
More informationAdaptive Transfer Learning
Adaptve Transfer Learnng Bn Cao, Snno Jaln Pan, Yu Zhang, Dt-Yan Yeung, Qang Yang Hong Kong Unversty of Scence and Technology Clear Water Bay, Kowloon, Hong Kong {caobn,snnopan,zhangyu,dyyeung,qyang}@cse.ust.hk
More informationVanishing Hull. Jinhui Hu, Suya You, Ulrich Neumann University of Southern California {jinhuihu,suyay,
Vanshng Hull Jnhu Hu Suya You Ulrch Neumann Unversty of Southern Calforna {jnhuhusuyay uneumann}@graphcs.usc.edu Abstract Vanshng ponts are valuable n many vson tasks such as orentaton estmaton pose recovery
More informationA Background Subtraction for a Vision-based User Interface *
A Background Subtracton for a Vson-based User Interface * Dongpyo Hong and Woontack Woo KJIST U-VR Lab. {dhon wwoo}@kjst.ac.kr Abstract In ths paper, we propose a robust and effcent background subtracton
More informationFast Sparse Gaussian Processes Learning for Man-Made Structure Classification
Fast Sparse Gaussan Processes Learnng for Man-Made Structure Classfcaton Hang Zhou Insttute for Vson Systems Engneerng, Dept Elec. & Comp. Syst. Eng. PO Box 35, Monash Unversty, Clayton, VIC 3800, Australa
More informationA Fast Visual Tracking Algorithm Based on Circle Pixels Matching
A Fast Vsual Trackng Algorthm Based on Crcle Pxels Matchng Zhqang Hou hou_zhq@sohu.com Chongzhao Han czhan@mal.xjtu.edu.cn Ln Zheng Abstract: A fast vsual trackng algorthm based on crcle pxels matchng
More informationSLAM Summer School 2006 Practical 2: SLAM using Monocular Vision
SLAM Summer School 2006 Practcal 2: SLAM usng Monocular Vson Javer Cvera, Unversty of Zaragoza Andrew J. Davson, Imperal College London J.M.M Montel, Unversty of Zaragoza. josemar@unzar.es, jcvera@unzar.es,
More informationIntra-Parametric Analysis of a Fuzzy MOLP
Intra-Parametrc Analyss of a Fuzzy MOLP a MIAO-LING WANG a Department of Industral Engneerng and Management a Mnghsn Insttute of Technology and Hsnchu Tawan, ROC b HSIAO-FAN WANG b Insttute of Industral
More informationData Representation in Digital Design, a Single Conversion Equation and a Formal Languages Approach
Data Representaton n Dgtal Desgn, a Sngle Converson Equaton and a Formal Languages Approach Hassan Farhat Unversty of Nebraska at Omaha Abstract- In the study of data representaton n dgtal desgn and computer
More information12/2/2009. Announcements. Parametric / Non-parametric. Case-Based Reasoning. Nearest-Neighbor on Images. Nearest-Neighbor Classification
Introducton to Artfcal Intellgence V22.0472-001 Fall 2009 Lecture 24: Nearest-Neghbors & Support Vector Machnes Rob Fergus Dept of Computer Scence, Courant Insttute, NYU Sldes from Danel Yeung, John DeNero
More informationAPPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT
3. - 5. 5., Brno, Czech Republc, EU APPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT Abstract Josef TOŠENOVSKÝ ) Lenka MONSPORTOVÁ ) Flp TOŠENOVSKÝ
More informationA Post Randomization Framework for Privacy-Preserving Bayesian. Network Parameter Learning
A Post Randomzaton Framework for Prvacy-Preservng Bayesan Network Parameter Learnng JIANJIE MA K.SIVAKUMAR School Electrcal Engneerng and Computer Scence, Washngton State Unversty Pullman, WA. 9964-75
More informationOptimal Workload-based Weighted Wavelet Synopses
Optmal Workload-based Weghted Wavelet Synopses Yoss Matas School of Computer Scence Tel Avv Unversty Tel Avv 69978, Israel matas@tau.ac.l Danel Urel School of Computer Scence Tel Avv Unversty Tel Avv 69978,
More informationFeature Selection for Target Detection in SAR Images
Feature Selecton for Detecton n SAR Images Br Bhanu, Yngqang Ln and Shqn Wang Center for Research n Intellgent Systems Unversty of Calforna, Rversde, CA 95, USA Abstract A genetc algorthm (GA) approach
More informationR s s f. m y s. SPH3UW Unit 7.3 Spherical Concave Mirrors Page 1 of 12. Notes
SPH3UW Unt 7.3 Sphercal Concave Mrrors Page 1 of 1 Notes Physcs Tool box Concave Mrror If the reflectng surface takes place on the nner surface of the sphercal shape so that the centre of the mrror bulges
More informationReport on On-line Graph Coloring
2003 Fall Semester Comp 670K Onlne Algorthm Report on LO Yuet Me (00086365) cndylo@ust.hk Abstract Onlne algorthm deals wth data that has no future nformaton. Lots of examples demonstrate that onlne algorthm
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 15
CS434a/541a: Pattern Recognton Prof. Olga Veksler Lecture 15 Today New Topc: Unsupervsed Learnng Supervsed vs. unsupervsed learnng Unsupervsed learnng Net Tme: parametrc unsupervsed learnng Today: nonparametrc
More informationLOOP ANALYSIS. The second systematic technique to determine all currents and voltages in a circuit
LOOP ANALYSS The second systematic technique to determine all currents and voltages in a circuit T S DUAL TO NODE ANALYSS - T FRST DETERMNES ALL CURRENTS N A CRCUT AND THEN T USES OHM S LAW TO COMPUTE
More informationMachine Learning. K-means Algorithm
Macne Learnng CS 6375 --- Sprng 2015 Gaussan Mture Model GMM pectaton Mamzaton M Acknowledgement: some sldes adopted from Crstoper Bsop Vncent Ng. 1 K-means Algortm Specal case of M Goal: represent a data
More informationLECTURE : MANIFOLD LEARNING
LECTURE : MANIFOLD LEARNING Rta Osadchy Some sldes are due to L.Saul, V. C. Raykar, N. Verma Topcs PCA MDS IsoMap LLE EgenMaps Done! Dmensonalty Reducton Data representaton Inputs are real-valued vectors
More informationInvestigating the Performance of Naïve- Bayes Classifiers and K- Nearest Neighbor Classifiers
Journal of Convergence Informaton Technology Volume 5, Number 2, Aprl 2010 Investgatng the Performance of Naïve- Bayes Classfers and K- Nearest Neghbor Classfers Mohammed J. Islam *, Q. M. Jonathan Wu,
More informationClassification / Regression Support Vector Machines
Classfcaton / Regresson Support Vector Machnes Jeff Howbert Introducton to Machne Learnng Wnter 04 Topcs SVM classfers for lnearly separable classes SVM classfers for non-lnearly separable classes SVM
More information6.854 Advanced Algorithms Petar Maymounkov Problem Set 11 (November 23, 2005) With: Benjamin Rossman, Oren Weimann, and Pouya Kheradpour
6.854 Advanced Algorthms Petar Maymounkov Problem Set 11 (November 23, 2005) Wth: Benjamn Rossman, Oren Wemann, and Pouya Kheradpour Problem 1. We reduce vertex cover to MAX-SAT wth weghts, such that the
More informationFEATURE EXTRACTION. Dr. K.Vijayarekha. Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur
FEATURE EXTRACTION Dr. K.Vjayarekha Assocate Dean School of Electrcal and Electroncs Engneerng SASTRA Unversty, Thanjavur613 41 Jont Intatve of IITs and IISc Funded by MHRD Page 1 of 8 Table of Contents
More informationJournal of Chemical and Pharmaceutical Research, 2014, 6(6): Research Article
Avalable onlne www.jocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(6):2512-2520 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 Communty detecton model based on ncremental EM clusterng
More informationReducing Frame Rate for Object Tracking
Reducng Frame Rate for Object Trackng Pavel Korshunov 1 and We Tsang Oo 2 1 Natonal Unversty of Sngapore, Sngapore 11977, pavelkor@comp.nus.edu.sg 2 Natonal Unversty of Sngapore, Sngapore 11977, oowt@comp.nus.edu.sg
More informationProblem Set 3 Solutions
Introducton to Algorthms October 4, 2002 Massachusetts Insttute of Technology 6046J/18410J Professors Erk Demane and Shaf Goldwasser Handout 14 Problem Set 3 Solutons (Exercses were not to be turned n,
More informationCell Count Method on a Network with SANET
CSIS Dscusson Paper No.59 Cell Count Method on a Network wth SANET Atsuyuk Okabe* and Shno Shode** Center for Spatal Informaton Scence, Unversty of Tokyo 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
More informationAssignment # 2. Farrukh Jabeen Algorithms 510 Assignment #2 Due Date: June 15, 2009.
Farrukh Jabeen Algorthms 51 Assgnment #2 Due Date: June 15, 29. Assgnment # 2 Chapter 3 Dscrete Fourer Transforms Implement the FFT for the DFT. Descrbed n sectons 3.1 and 3.2. Delverables: 1. Concse descrpton
More informationCluster Analysis of Electrical Behavior
Journal of Computer and Communcatons, 205, 3, 88-93 Publshed Onlne May 205 n ScRes. http://www.scrp.org/ournal/cc http://dx.do.org/0.4236/cc.205.350 Cluster Analyss of Electrcal Behavor Ln Lu Ln Lu, School
More informationNGPM -- A NSGA-II Program in Matlab
Verson 1.4 LIN Song Aerospace Structural Dynamcs Research Laboratory College of Astronautcs, Northwestern Polytechncal Unversty, Chna Emal: lsssswc@163.com 2011-07-26 Contents Contents... 1. Introducton...
More information2x x l. Module 3: Element Properties Lecture 4: Lagrange and Serendipity Elements
Module 3: Element Propertes Lecture : Lagrange and Serendpty Elements 5 In last lecture note, the nterpolaton functons are derved on the bass of assumed polynomal from Pascal s trangle for the fled varable.
More informationLearning-Based Top-N Selection Query Evaluation over Relational Databases
Learnng-Based Top-N Selecton Query Evaluaton over Relatonal Databases Lang Zhu *, Wey Meng ** * School of Mathematcs and Computer Scence, Hebe Unversty, Baodng, Hebe 071002, Chna, zhu@mal.hbu.edu.cn **
More informationUnderstanding K-Means Non-hierarchical Clustering
SUNY Albany - Techncal Report 0- Understandng K-Means Non-herarchcal Clusterng Ian Davdson State Unversty of New York, 1400 Washngton Ave., Albany, 105. DAVIDSON@CS.ALBANY.EDU Abstract The K-means algorthm
More informationMachine Learning 9. week
Machne Learnng 9. week Mappng Concept Radal Bass Functons (RBF) RBF Networks 1 Mappng It s probably the best scenaro for the classfcaton of two dataset s to separate them lnearly. As you see n the below
More informationType-2 Fuzzy Non-uniform Rational B-spline Model with Type-2 Fuzzy Data
Malaysan Journal of Mathematcal Scences 11(S) Aprl : 35 46 (2017) Specal Issue: The 2nd Internatonal Conference and Workshop on Mathematcal Analyss (ICWOMA 2016) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES
More informationAn Improved Image Segmentation Algorithm Based on the Otsu Method
3th ACIS Internatonal Conference on Software Engneerng, Artfcal Intellgence, Networkng arallel/dstrbuted Computng An Improved Image Segmentaton Algorthm Based on the Otsu Method Mengxng Huang, enjao Yu,
More informationMachine Learning. Topic 6: Clustering
Machne Learnng Topc 6: lusterng lusterng Groupng data nto (hopefully useful) sets. Thngs on the left Thngs on the rght Applcatons of lusterng Hypothess Generaton lusters mght suggest natural groups. Hypothess
More informationThree supervised learning methods on pen digits character recognition dataset
Three supervsed learnng methods on pen dgts character recognton dataset Chrs Flezach Department of Computer Scence and Engneerng Unversty of Calforna, San Dego San Dego, CA 92093 cflezac@cs.ucsd.edu Satoru
More informationImproved Methods for Lithography Model Calibration
Improved Methods for Lthography Model Calbraton Chrs Mack www.lthoguru.com, Austn, Texas Abstract Lthography models, ncludng rgorous frst prncple models and fast approxmate models used for OPC, requre
More informationSolutions to Programming Assignment Five Interpolation and Numerical Differentiation
College of Engneerng and Coputer Scence Mechancal Engneerng Departent Mechancal Engneerng 309 Nuercal Analyss of Engneerng Systes Sprng 04 Nuber: 537 Instructor: Larry Caretto Solutons to Prograng Assgnent
More informationA new paradigm of fuzzy control point in space curve
MATEMATIKA, 2016, Volume 32, Number 2, 153 159 c Penerbt UTM Press All rghts reserved A new paradgm of fuzzy control pont n space curve 1 Abd Fatah Wahab, 2 Mohd Sallehuddn Husan and 3 Mohammad Izat Emr
More informationThe Greedy Method. Outline and Reading. Change Money Problem. Greedy Algorithms. Applications of the Greedy Strategy. The Greedy Method Technique
//00 :0 AM Outlne and Readng The Greedy Method The Greedy Method Technque (secton.) Fractonal Knapsack Problem (secton..) Task Schedulng (secton..) Mnmum Spannng Trees (secton.) Change Money Problem Greedy
More informationMeta-heuristics for Multidimensional Knapsack Problems
2012 4th Internatonal Conference on Computer Research and Development IPCSIT vol.39 (2012) (2012) IACSIT Press, Sngapore Meta-heurstcs for Multdmensonal Knapsack Problems Zhbao Man + Computer Scence Department,
More informationAn Iterative Solution Approach to Process Plant Layout using Mixed Integer Optimisation
17 th European Symposum on Computer Aded Process Engneerng ESCAPE17 V. Plesu and P.S. Agach (Edtors) 2007 Elsever B.V. All rghts reserved. 1 An Iteratve Soluton Approach to Process Plant Layout usng Mxed
More informationImplementation Naïve Bayes Algorithm for Student Classification Based on Graduation Status
Internatonal Journal of Appled Busness and Informaton Systems ISSN: 2597-8993 Vol 1, No 2, September 2017, pp. 6-12 6 Implementaton Naïve Bayes Algorthm for Student Classfcaton Based on Graduaton Status
More information