Multiple optimum values

Size: px
Start display at page:

Download "Multiple optimum values"

Transcription

1 1.204 Lecture 22 Unconstraned nonlnear optmzaton: Amoeba BFGS Lnear programmng: Glpk Multple optmum values A B C G E Z X F Y D X 1 X 2 Fgure by MIT OpenCourseWare. Heurstcs to deal wth multple optma: Start at many ntal ponts. Choose best of optma found. Fnd local optmum. Take a step away from t and search agan. Smulated annealng takes random steps repeatedly From Press 1

2 Nonlnear optmzaton Unconstraned nonlnear optmzaton algorthms generall lly use th e same strategy t as unconstraned Select a descent drecton Use a one dmensonal lne search to set step sze Step, and terate untl convergence Constraned optmzaton used the constrants to lmt the maxmum step sze Unconstraned optmzaton t must select maxmum step sze Step sze s problem-specfc and must be tuned Memory requrements are rarely a problem Convergence, accuracy and speed are the ssues Famly of nonlnear algorthms Amoeba (Nelder-Mead) method Solves nonlnear optmzaton problem drectly Requres no dervatves or lne search Adapts ts step sze based on change n functon value Conjugate gradent and quas-newton methods Requre functon, frst dervatves* and lne search Lne search step sze adapts as algorthm proceeds Newton-Raphson method (last lecture) Used to solve nonlnear optmzaton problems by solvng set of frst order condtons Uses step sze dx that makes f(x+dx)= 0. Lttle control. Globally convergent Newton varant has smaller step sze Needs frst and second dervatves (and functon) 2

3 Choosng among the algorthms Amoeba s smplest, most robust, slowest Crawls downhll wth no assumptons about functon No dervatves requred Conjugate gradent (Polak-Rbere) (not covered) Need frst dervatves Less storage than quas-newton but less accuracy Quas-Newton (Davdon-Fletcher-Powell or Broyden-Fletcher-Goldfarb-Shanno) Standard verson uses frst dervatves Varaton computes frst dervatves numercally Better than conjugate gradent for most problems Newton-Raphson Needs functon, frst and second dervatves Smplest code but not robust or flexble Use amoeba f you want a smple approach Amoeba algorthm The easest algorthm for unconstraned nonlnear optmzaton s known as Nelder-Mead or the amoeba It s very dfferent It requres only functon evaluatons No dervatves are requred It s less effcent than the lne search algorthms But t tends to be robust (lne methods are temperamental) It s short (~150 lnes) and relatvely easy to mplement Works n problems where dervatves are dffcult: Fngerprnt matchng Models of bran functon We ll use logt demand model estmaton as test case for all the algorthms today 3

4 Amoeba steps Smplex s volume defned by n+1 ponts n n dmensons In 3-D, t s a tetrahedron or pyramd Select a startng guess at pont P 0 Set other ponts of smplex as P = P 0 + e Take a seres of steps Most steps move pont of smplex where functon s hghest ( hghest pont ): reflecton Conserve volume of smplex -> avod degeneracy Where functon s flat, method expands smplex to take larger steps: expanson and reflecton When t reaches a valley floor, smplex contracts tself n transverse drecton and tres to ooze down the valley If tryng to pass through eye of needle t contracts tself n all drectons around ts lowest (best) pont From Press et al Amoeba steps Smplex at begnnng of step Hgh Low Reflecton Reflecton and expanson (a) Contracton (b) Multple contracton (c) (d) Possble outcomes for a step n the downhll smplex method Fgure by MIT OpenCourseWare. From Press et al 4

5 Amoeba pseudocode: mnmzaton Start at ntal guess Determne whch pont s hghest by loopng over smplex ponts and evaluatng functon at each If dfference between hghest and lowest s small, return Otherwse ooze (terate): Reflect by factor= -1 through face of smplex from hgh pont If ths s better than low pont, reflect/expand by factor of 2 If ths s worse than second hghest, contract by 2 n ths drecton If ths s worst pont, contract n all drectons around lowest pont Select new face based on new hgh pont and reflect agan Termnate f dfference between hghest and lowest ponts s small Or termnate at maxmum teratons allowed Amoeba termnaton crtera All nonlnear optmzaton algorthm termnaton crtera are dffcult Termnate when step sze s small (~10-8 wth doubl bles) Change n functon value s small (~10-14 wth doubles) Termnaton can occur at a local mnmum Restart amoeba at a mnmum t found Reset ntal smplex usng P = P 0 + e, Don t use smplex that exsted at termnaton Code s n download: publc Amoeba(double ftol) publc double[] mnmze(double[] pont, double del, MathFuncton3 mf3) // And other optonal methods Ths s very easy; code ran frst tme on demand model Experment wth startng pont and del, but t s usually easy 5

6 AmoebaTest publc class AmoebaDemandTest { publc statc vod man(strng[] args) { double[][] x= { {1, }, {1, }, // Etc. }; double[] y= {0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1}; // Almost dentcal to DemandModel from last lecture // Implements MathFuncton3, whch has method func2() // func2() returns -loglkelhood() DemandModelAmoeba d= new DemandModelAmoeba(x, y); double[] beta= {0, 0}; // Intal guess double log0= d.func2(beta); double[] ntalpont= {0.0, 0.0}; double ntaldelta= l 0.1; double ftol= 1E-14; Amoeba a= new Amoeba(ftol); beta= a.mnmze(ntalpont, ntaldelta, d); double logb= d.func2(beta); double[][] jacoban= d.jacoban(beta); d.prnt(log0, logb, beta, jacoban); } } // Output dentcal to last lecture example. 116 evals Quas-Newton methods (DFP, BFGS) We used a smlar method to BFGS n constraned optmzaton: Fnd dervatves Fnd drecton that yelds maxmum estmated objectve functon change Use lne search to fnd optmal step sze Move, and repeat DFP and BFGS are essentally the same One addtonal correcton term n BFGS Treatment of roundoff errors, tolerances s dfferent Emprcally, t seems BFGS s better than DFP 6

7 Quas-Newton methods Splt problem nto two mnmzaton problems, each wth a quadratc envelope You must understand your problem well enough to do ths Newton method Full Newton step If the mnmum regon s narrow or twsty, t s hard to get down nto t. Full Newton steps gve lttle control. (Amoeba just munches along) 7

8 Quas-Newton methods Quas Newton steps BFGS takes a fracton of the Newton step, so that f(x) decreases at some mnmum rate proportonal to the average decrease Hgh order dervatves We may be tempted to use hgher dervatves to ft our nonlnear functons. More terms n the Taylor seres expanson mean a hgher degree polynomal ft, but usng hgher order dervatves s dffcult because they are stff Ths s the challenge of nonlnear methods: use only low order dervatves to explore complex surfaces. There aren t many general ways to do ths. Solutons are problem-specfc but use BFGS or other general algorthm: Understand problem regons; have good startng pont; understand surface 8

9 Hgh order dervatves We may be tempted to use hgher dervatves to ft our nonlnear functons. More terms n the Taylor seres expanson mean a hgher degree polynomal ft, but usng hgher order dervatves s dffcult because they are stff Ths s the challenge of nonlnear methods: use only low degree dervatves to explore complex surfaces. There aren t many general ways to do ths. Solutons are problem-specfc but use BFGS or other general algorthm: Understand problem regons; have good startng pont; understand surface Hgh order dervatves We may be tempted to use hgher dervatves to ft our nonlnear functons. More terms n the Taylor seres expanson mean a hgher degree polynomal ft, but usng hgher order dervatves s dffcult because they are stff Ths s the challenge of nonlnear methods: use only low degree dervatves to explore complex surfaces. There aren t many general ways to do ths. Solutons are problem-specfc but use BFGS or other general algorthm: Understand problem regons; have good startng pont; understand surface 9

10 Hgh order dervatves We may be tempted to use hgher dervatves to ft our nonlnear functons. More terms n the Taylor seres expanson mean a hgher degree polynomal ft, but usng hgher order dervatves s dffcult because they are stff Ths s the challenge of nonlnear methods: use only low degree dervatves to explore complex surfaces. There aren t many general ways to do ths. Solutons are problem-specfc but use BFGS or other general algorthm: Understand problem regons; have good startng pont; understand surface BFGS Take a pont P as orgn. Approxmate f 2 f 1 f f(x) = f(p) + x + x 2 x x c b x + where c = f ( P) b = f P 2 f [ A] j = x x 1 2 j x A x P 2, j j x x A s the Hessan matrx. Gradent of f s : f = A x b j usng Taylor seres : 10

11 lm BFGS, p.2 BFGS constructs a sequence of matrces H > H = A 1 At pont x, usng a Taylor seres agan f ( x) = f ( x ) + A ( x x ) such that : 1 f(x) = f(x) + ( x x ) f ( x ) + ( x x ) A ( x x ) 2 To fnd a mnmum, fnd a zero of the gradent of f(x) : ( x x ) = A f ( ) Newton's method sets f(x) = 0 to fnd the next pont : 1 x BFGS, p.3 Newton s method, far from the mnmum, can Newton s method, far from the mnmum, can project us to ponts x where f(x) s greater than the current value Large steps based on a quadratc approxmaton wll not necessarly lead to mprovements n ll-behaved functon BFGS does a lne search along the Newton drecton It fnds a pont at whch the objectve has decreased Ths pont s used to update the Hessan (whch s a matrx of second dervatves) The Hessan s based on the prevous Hessan and a set of correcton terms based on f, x and x See Numercal Recpes for BFGS updatng formula Proofs are dffcult 11

12 BFGS, p.4 The seres of H converge to A -1 whle guaranteeng movement downhll Convergence takes N steps f f s quadratc The algorthm s senstve to varable scalng: Step length must be tuned Scale varables to stay n low range of double values The algorthm s not too senstve to the accuracy of the lne search BFGS dfpmn() pseudocode Compute functon, gradent at start pont P Intalze nverse Hessan to dentty matrx Fnd ntal descent drecton along gradent Loop untl convergence Lne search along drecton to fnd decrease (not mn) Update gradent and lne drecton Check f change n x s small enough to converge Check f gradent s close enough to 0 to converge Compute dfference n gradents, x for BFGS update Calculate next drecton If teraton lmt reached, termnate 12

13 BFGS lnsrch() pseudocode Takes pont, functon and gradent value at pont, and drecton (lne) to search along as nput Fnds new pont wth lower functon value Fndng mnmum along lne requres too much computaton Fndng any pont whose f s less than current sn t good enough ether; we may take too many steps We fnd a pont where the average decrease from the current pont s a fracton of the gradent projecton We requre a mnmum step sze so we don t stall We use a quadratc nterpolaton of f along the lne to choose the frst new pont We then use a cubc nterpolaton, now that we have 3 ponts ( f(0), f(1) and f(m)) for remanng ponts BFGS code Frst code set BFGS: constructor, dfpmn(), lnsrch() MathFuncton4: nterface wth func(), df() [gradent] DemandModelBFGS: Almost same as n Newton DemandModelBFGSTest: Almost same as n Newton Second code set Same BFGS, MathFuncton4, DemandModelBFGSTest DemandModelBFGS df() method computes gradent numercally, wthout analytc expressons for dervatves Both BFGS mplementatons converge quckly on same logt model coeffcents as Newton Code s subtle, especally nteracton between dfpmn() and lnsrch(). 13

14 BFGSTest publc class DemandModelBFGSTest { publc statc vod man(strng[] args) { double[][] x= { {1, }, {1, }, // Etc. }; double[] y= {0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1}; // Almost dentcal to DemandModel from last lecture // Implements MathFuncton4, wth func() and df() DemandModelBFGS d= new DemandModelBFGS(x, y); double[] beta= {0, 0}; // Intal guess double log0= d.func2(beta); double gtol= 1E-14; 14; BFGS b= new BFGS(d, gtol); double logb= b.dfpmn(beta); double[][] jacoban= d.jacoban(beta); d.prnt(log0, logb, beta, jacoban); } } // Output dentcal to last lecture example. 10 teratons Nonlnear method performance Method Iteratons Functon evaluatons Newton 6 6n 2 =24 BFGS 10 or 11 10n= 20 Amoeba Ths example s llustratve only, based on our logt test case. n s the number of varables (coeffcents n our test case). As n ncreases, amoeba performance s relatvely more compettve Amoeba and BFGS have better convergence and precson than Newton 14

15 Summary: nonlnear contnuous methods Note the smlartes n solvng dscrete and contnuous problems Startng soluton s null or {0, 0, 0, } Or a good guess or greedy f we need a good frst guess Choose ntal decson or ntal drecton/gradent Fnd ntal soluton at stage/branch or Newton/other step Look for better soluton locally through greedy/branchng/stages or new gradent Or globally through branch-and-bound/dynamc and-bound/dynamc programmng or wth advanced contnuous methods Smulated annealng, multple restarts Termnate when: Local or global optmum found, or Upper and lower bound close enough, or Maxmum teratons reached Lnear programmng Lnear programmng has a lnear objectve functon and lnear constrants Used as a subproblem several tmes already: Convex combnatons (lecture 18) Homework 8 transt network optmzaton Network equlbrum used shortest path algorthm as a specal case of lnear programmng Often used as subproblem n branch and bound algorthms We used shortest paths n our warehouse branch and bound as a specal case of lnear programmng See any nteger programmng text for a lst of problems that use lnear programmng as a subproblem 15

16 Lnear programmng From GLPK manual Andrew Makhorn. All rghts reserved. Ths content s excluded from our Creatve Commons lcense. For more nformaton, see Lnear programmng standard form Formulate constrants as a set of equatons and a set of bounded varables. Ths s easy to translate to the smplex tableau used for the computatons. Andrew Makhorn. All rghts reserved. Ths content s excluded from our Creatve Commons lcense. For more nformaton, see 16

17 LP program, p.1 mport org.gnu.glpk.*; // Glpk starts numberng at 1, not 0 publc class LP { publc statc vod man(strng[] args) { nt[] a= new nt[1+1000]; // Row ndex of coeffcent nt[] ja= new nt[1+1000]; // Col ndex of coeffcent double[] ar= new double[1+1000]; // Coeffcent double z, x1, x2, x3; // Obj value, unknowns GlpkSolver solver = new GlpkSolver(); solver.setprobname("sample"); solver.setobjdr(glpksolver.lpx_max); // Maxmzaton solver.addrows(3); solver.setrowname(1, "p"); solver.setrowbnds(1, GlpkSolver.LPX_UP, 0.0, 100.0); solver.setrowname(2, "q"); solver.setrowbnds(2, GlpkSolver.LPX_UP, 0.0, 600.0); solver.setrowname(3, "r"); solver.setrowbnds(3, GlpkSolver.LPX_UP, 0.0, 300.0); LP program, p.2 solver.addcols(3); solver.setcolname(1, "x1"); solver.setcolbnds(1, GlpkSolver.LPX_LO, 0.0, 0.0); solver.setobjcoef(1, setobjcoef(1 10.0); 0); solver.setcolname(2, "x2"); solver.setcolbnds(2, GlpkSolver.LPX_LO, 0.0, 0.0); solver.setobjcoef(2, 6.0); solver.setcolname(3, "x3"); solver.setcolbnds(3, GlpkSolver.LPX_LO, 0.0, 0.0); solver.setobjcoef(3, 4.0); a[1] = 1; ja[1] = 1; ar[1] = 1.0; /* a[1,1] = 1 */ a[2] = 1; ja[2] = 2; ar[2] = 1.0; /* a[1,2] = 1 */ a[3] = 1; ja[3] = 3; ar[3] = 1.0; /* a[1,3] = 1 */ a[4] = 2; ja[4] = 1; ar[4] = 10.0; /* a[2,1] = 10 */ a[5] = 2; ja[5] = 2; ar[5] = 4.0; /* a[2,2] = 4 */ a[6] = 2; ja[6] = 3; ar[6] = 5.0; /* a[2,3] = 5 */ a[7] = 3; ja[7] = 1; ar[7] = 2.0; /* a[3,1] = 2 */ a[8] = 3; ja[8] = 2; ar[8] = 2.0; /* a[3,2] = 2 */ a[9] = 3; ja[9] = 3; ar[9] = 6.0; /* a[3,3] = 6 */ 17

18 LP program, p.3 } solver.loadmatrx(9, a, ja, ar); solver.smplex(); z = solver.getobjval(); x1 = solver.getcolprm(1); x2 = solver.getcolprm(2); x3 = solver.getcolprm(3); System.out.prntf("\nz = %g; x1 = %g; x2 = %g; x3 = %g\n", z, x1, x2, x3); // Output: // z = ; 333; x1 = ; 3333; x2 = ; 6667; x3 = Usng GLPK under WndowsXP Download fles, documentaton from Copy glpk.jar to C:\Program Fles\Java\jdk1.6.0_05\jre\lb Copy glpk_jn.dll to the drectory of project n Eclpse Eclpse: Project->Propertes->Java Buld Path. Add external JAR: glpk.jar Create Eclpse project, wrte class as usual See GLKP documentaton (pdf). Java method names are based on C names: lpx_set_obj_dr() becomes setobjdr() glpk.jar uses the Java Natve Interface (JNI) to create Java stubs for the C natve functons n the glpk lbrary You now have a callable lbrary of lnear programmng routnes that you can use from Java GLPK also has: Mxed nteger programmng method, usng branch and bound Use ths nstead of your own B&B unless you have problem-specfc prunng Interor pont lnear programmng method 18

19 MIT OpenCourseWare Computer Algorthms n Systems Engneerng Sprng 2010 For nformaton about ctng these materals or our Terms of Use, vst:

Biostatistics 615/815

Biostatistics 615/815 The E-M Algorthm Bostatstcs 615/815 Lecture 17 Last Lecture: The Smplex Method General method for optmzaton Makes few assumptons about functon Crawls towards mnmum Some recommendatons Multple startng ponts

More information

Programming in Fortran 90 : 2017/2018

Programming in Fortran 90 : 2017/2018 Programmng n Fortran 90 : 2017/2018 Programmng n Fortran 90 : 2017/2018 Exercse 1 : Evaluaton of functon dependng on nput Wrte a program who evaluate the functon f (x,y) for any two user specfed values

More information

Improving Low Density Parity Check Codes Over the Erasure Channel. The Nelder Mead Downhill Simplex Method. Scott Stransky

Improving Low Density Parity Check Codes Over the Erasure Channel. The Nelder Mead Downhill Simplex Method. Scott Stransky Improvng Low Densty Party Check Codes Over the Erasure Channel The Nelder Mead Downhll Smplex Method Scott Stransky Programmng n conjuncton wth: Bors Cukalovc 18.413 Fnal Project Sprng 2004 Page 1 Abstract

More information

Outline. Midterm Review. Declaring Variables. Main Variable Data Types. Symbolic Constants. Arithmetic Operators. Midterm Review March 24, 2014

Outline. Midterm Review. Declaring Variables. Main Variable Data Types. Symbolic Constants. Arithmetic Operators. Midterm Review March 24, 2014 Mdterm Revew March 4, 4 Mdterm Revew Larry Caretto Mechancal Engneerng 9 Numercal Analyss of Engneerng Systems March 4, 4 Outlne VBA and MATLAB codng Varable types Control structures (Loopng and Choce)

More information

NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS

NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS ARPN Journal of Engneerng and Appled Scences 006-017 Asan Research Publshng Network (ARPN). All rghts reserved. NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS Igor Grgoryev, Svetlana

More information

S1 Note. Basis functions.

S1 Note. Basis functions. S1 Note. Bass functons. Contents Types of bass functons...1 The Fourer bass...2 B-splne bass...3 Power and type I error rates wth dfferent numbers of bass functons...4 Table S1. Smulaton results of type

More information

GSLM Operations Research II Fall 13/14

GSLM Operations Research II Fall 13/14 GSLM 58 Operatons Research II Fall /4 6. Separable Programmng Consder a general NLP mn f(x) s.t. g j (x) b j j =. m. Defnton 6.. The NLP s a separable program f ts objectve functon and all constrants are

More information

Inverse Kinematics (part 2) CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Spring 2016

Inverse Kinematics (part 2) CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Spring 2016 Inverse Knematcs (part 2) CSE169: Computer Anmaton Instructor: Steve Rotenberg UCSD, Sprng 2016 Forward Knematcs We wll use the vector: Φ... 1 2 M to represent the array of M jont DOF values We wll also

More information

Smoothing Spline ANOVA for variable screening

Smoothing Spline ANOVA for variable screening Smoothng Splne ANOVA for varable screenng a useful tool for metamodels tranng and mult-objectve optmzaton L. Rcco, E. Rgon, A. Turco Outlne RSM Introducton Possble couplng Test case MOO MOO wth Game Theory

More information

Sorting Review. Sorting. Comparison Sorting. CSE 680 Prof. Roger Crawfis. Assumptions

Sorting Review. Sorting. Comparison Sorting. CSE 680 Prof. Roger Crawfis. Assumptions Sortng Revew Introducton to Algorthms Qucksort CSE 680 Prof. Roger Crawfs Inserton Sort T(n) = Θ(n 2 ) In-place Merge Sort T(n) = Θ(n lg(n)) Not n-place Selecton Sort (from homework) T(n) = Θ(n 2 ) In-place

More information

LECTURE NOTES Duality Theory, Sensitivity Analysis, and Parametric Programming

LECTURE NOTES Duality Theory, Sensitivity Analysis, and Parametric Programming CEE 60 Davd Rosenberg p. LECTURE NOTES Dualty Theory, Senstvty Analyss, and Parametrc Programmng Learnng Objectves. Revew the prmal LP model formulaton 2. Formulate the Dual Problem of an LP problem (TUES)

More information

News. Recap: While Loop Example. Reading. Recap: Do Loop Example. Recap: For Loop Example

News. Recap: While Loop Example. Reading. Recap: Do Loop Example. Recap: For Loop Example Unversty of Brtsh Columba CPSC, Intro to Computaton Jan-Apr Tamara Munzner News Assgnment correctons to ASCIIArtste.java posted defntely read WebCT bboards Arrays Lecture, Tue Feb based on sldes by Kurt

More information

Quality Improvement Algorithm for Tetrahedral Mesh Based on Optimal Delaunay Triangulation

Quality Improvement Algorithm for Tetrahedral Mesh Based on Optimal Delaunay Triangulation Intellgent Informaton Management, 013, 5, 191-195 Publshed Onlne November 013 (http://www.scrp.org/journal/m) http://dx.do.org/10.36/m.013.5601 Qualty Improvement Algorthm for Tetrahedral Mesh Based on

More information

Modeling and Solving Nontraditional Optimization Problems Session 2a: Conic Constraints

Modeling and Solving Nontraditional Optimization Problems Session 2a: Conic Constraints Modelng and Solvng Nontradtonal Optmzaton Problems Sesson 2a: Conc Constrants Robert Fourer Industral Engneerng & Management Scences Northwestern Unversty AMPL Optmzaton LLC 4er@northwestern.edu 4er@ampl.com

More information

Course Introduction. Algorithm 8/31/2017. COSC 320 Advanced Data Structures and Algorithms. COSC 320 Advanced Data Structures and Algorithms

Course Introduction. Algorithm 8/31/2017. COSC 320 Advanced Data Structures and Algorithms. COSC 320 Advanced Data Structures and Algorithms Course Introducton Course Topcs Exams, abs, Proects A quc loo at a few algorthms 1 Advanced Data Structures and Algorthms Descrpton: We are gong to dscuss algorthm complexty analyss, algorthm desgn technques

More information

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision SLAM Summer School 2006 Practcal 2: SLAM usng Monocular Vson Javer Cvera, Unversty of Zaragoza Andrew J. Davson, Imperal College London J.M.M Montel, Unversty of Zaragoza. josemar@unzar.es, jcvera@unzar.es,

More information

Support Vector Machines

Support Vector Machines /9/207 MIST.6060 Busness Intellgence and Data Mnng What are Support Vector Machnes? Support Vector Machnes Support Vector Machnes (SVMs) are supervsed learnng technques that analyze data and recognze patterns.

More information

Solving two-person zero-sum game by Matlab

Solving two-person zero-sum game by Matlab Appled Mechancs and Materals Onlne: 2011-02-02 ISSN: 1662-7482, Vols. 50-51, pp 262-265 do:10.4028/www.scentfc.net/amm.50-51.262 2011 Trans Tech Publcatons, Swtzerland Solvng two-person zero-sum game by

More information

LECTURE : MANIFOLD LEARNING

LECTURE : MANIFOLD LEARNING LECTURE : MANIFOLD LEARNING Rta Osadchy Some sldes are due to L.Saul, V. C. Raykar, N. Verma Topcs PCA MDS IsoMap LLE EgenMaps Done! Dmensonalty Reducton Data representaton Inputs are real-valued vectors

More information

Wavefront Reconstructor

Wavefront Reconstructor A Dstrbuted Smplex B-Splne Based Wavefront Reconstructor Coen de Vsser and Mchel Verhaegen 14-12-201212 2012 Delft Unversty of Technology Contents Introducton Wavefront reconstructon usng Smplex B-Splnes

More information

Lecture 4: Principal components

Lecture 4: Principal components /3/6 Lecture 4: Prncpal components 3..6 Multvarate lnear regresson MLR s optmal for the estmaton data...but poor for handlng collnear data Covarance matrx s not nvertble (large condton number) Robustness

More information

Structural Optimization Using OPTIMIZER Program

Structural Optimization Using OPTIMIZER Program SprngerLnk - Book Chapter http://www.sprngerlnk.com/content/m28478j4372qh274/?prnt=true ق.ظ 1 of 2 2009/03/12 11:30 Book Chapter large verson Structural Optmzaton Usng OPTIMIZER Program Book III European

More information

Assignment # 2. Farrukh Jabeen Algorithms 510 Assignment #2 Due Date: June 15, 2009.

Assignment # 2. Farrukh Jabeen Algorithms 510 Assignment #2 Due Date: June 15, 2009. Farrukh Jabeen Algorthms 51 Assgnment #2 Due Date: June 15, 29. Assgnment # 2 Chapter 3 Dscrete Fourer Transforms Implement the FFT for the DFT. Descrbed n sectons 3.1 and 3.2. Delverables: 1. Concse descrpton

More information

Classification / Regression Support Vector Machines

Classification / Regression Support Vector Machines Classfcaton / Regresson Support Vector Machnes Jeff Howbert Introducton to Machne Learnng Wnter 04 Topcs SVM classfers for lnearly separable classes SVM classfers for non-lnearly separable classes SVM

More information

Radial Basis Functions

Radial Basis Functions Radal Bass Functons Mesh Reconstructon Input: pont cloud Output: water-tght manfold mesh Explct Connectvty estmaton Implct Sgned dstance functon estmaton Image from: Reconstructon and Representaton of

More information

The Codesign Challenge

The Codesign Challenge ECE 4530 Codesgn Challenge Fall 2007 Hardware/Software Codesgn The Codesgn Challenge Objectves In the codesgn challenge, your task s to accelerate a gven software reference mplementaton as fast as possble.

More information

AMath 483/583 Lecture 21 May 13, Notes: Notes: Jacobi iteration. Notes: Jacobi with OpenMP coarse grain

AMath 483/583 Lecture 21 May 13, Notes: Notes: Jacobi iteration. Notes: Jacobi with OpenMP coarse grain AMath 483/583 Lecture 21 May 13, 2011 Today: OpenMP and MPI versons of Jacob teraton Gauss-Sedel and SOR teratve methods Next week: More MPI Debuggng and totalvew GPU computng Read: Class notes and references

More information

Lecture #15 Lecture Notes

Lecture #15 Lecture Notes Lecture #15 Lecture Notes The ocean water column s very much a 3-D spatal entt and we need to represent that structure n an economcal way to deal wth t n calculatons. We wll dscuss one way to do so, emprcal

More information

Review of approximation techniques

Review of approximation techniques CHAPTER 2 Revew of appromaton technques 2. Introducton Optmzaton problems n engneerng desgn are characterzed by the followng assocated features: the objectve functon and constrants are mplct functons evaluated

More information

CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION

CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION 48 CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION 3.1 INTRODUCTION The raw mcroarray data s bascally an mage wth dfferent colors ndcatng hybrdzaton (Xue

More information

Parallel Numerics. 1 Preconditioning & Iterative Solvers (From 2016)

Parallel Numerics. 1 Preconditioning & Iterative Solvers (From 2016) Technsche Unverstät München WSe 6/7 Insttut für Informatk Prof. Dr. Thomas Huckle Dpl.-Math. Benjamn Uekermann Parallel Numercs Exercse : Prevous Exam Questons Precondtonng & Iteratve Solvers (From 6)

More information

5 The Primal-Dual Method

5 The Primal-Dual Method 5 The Prmal-Dual Method Orgnally desgned as a method for solvng lnear programs, where t reduces weghted optmzaton problems to smpler combnatoral ones, the prmal-dual method (PDM) has receved much attenton

More information

Very simple computational domains can be discretized using boundary-fitted structured meshes (also called grids)

Very simple computational domains can be discretized using boundary-fitted structured meshes (also called grids) Structured meshes Very smple computatonal domans can be dscretzed usng boundary-ftted structured meshes (also called grds) The grd lnes of a Cartesan mesh are parallel to one another Structured meshes

More information

CS246: Mining Massive Datasets Jure Leskovec, Stanford University

CS246: Mining Massive Datasets Jure Leskovec, Stanford University CS46: Mnng Massve Datasets Jure Leskovec, Stanford Unversty http://cs46.stanford.edu /19/013 Jure Leskovec, Stanford CS46: Mnng Massve Datasets, http://cs46.stanford.edu Perceptron: y = sgn( x Ho to fnd

More information

Multi-stable Perception. Necker Cube

Multi-stable Perception. Necker Cube Mult-stable Percepton Necker Cube Spnnng dancer lluson, Nobuuk Kaahara Fttng and Algnment Computer Vson Szelsk 6.1 James Has Acknowledgment: Man sldes from Derek Hoem, Lana Lazebnk, and Grauman&Lebe 2008

More information

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points; Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features

More information

Simplification of 3D Meshes

Simplification of 3D Meshes Smplfcaton of 3D Meshes Addy Ngan /4/00 Outlne Motvaton Taxonomy of smplfcaton methods Hoppe et al, Mesh optmzaton Hoppe, Progressve meshes Smplfcaton of 3D Meshes 1 Motvaton Hgh detaled meshes becomng

More information

Problem Definitions and Evaluation Criteria for Computational Expensive Optimization

Problem Definitions and Evaluation Criteria for Computational Expensive Optimization Problem efntons and Evaluaton Crtera for Computatonal Expensve Optmzaton B. Lu 1, Q. Chen and Q. Zhang 3, J. J. Lang 4, P. N. Suganthan, B. Y. Qu 6 1 epartment of Computng, Glyndwr Unversty, UK Faclty

More information

An Iterative Solution Approach to Process Plant Layout using Mixed Integer Optimisation

An Iterative Solution Approach to Process Plant Layout using Mixed Integer Optimisation 17 th European Symposum on Computer Aded Process Engneerng ESCAPE17 V. Plesu and P.S. Agach (Edtors) 2007 Elsever B.V. All rghts reserved. 1 An Iteratve Soluton Approach to Process Plant Layout usng Mxed

More information

The Greedy Method. Outline and Reading. Change Money Problem. Greedy Algorithms. Applications of the Greedy Strategy. The Greedy Method Technique

The Greedy Method. Outline and Reading. Change Money Problem. Greedy Algorithms. Applications of the Greedy Strategy. The Greedy Method Technique //00 :0 AM Outlne and Readng The Greedy Method The Greedy Method Technque (secton.) Fractonal Knapsack Problem (secton..) Task Schedulng (secton..) Mnmum Spannng Trees (secton.) Change Money Problem Greedy

More information

Mathematics 256 a course in differential equations for engineering students

Mathematics 256 a course in differential equations for engineering students Mathematcs 56 a course n dfferental equatons for engneerng students Chapter 5. More effcent methods of numercal soluton Euler s method s qute neffcent. Because the error s essentally proportonal to the

More information

Sequential search. Building Java Programs Chapter 13. Sequential search. Sequential search

Sequential search. Building Java Programs Chapter 13. Sequential search. Sequential search Sequental search Buldng Java Programs Chapter 13 Searchng and Sortng sequental search: Locates a target value n an array/lst by examnng each element from start to fnsh. How many elements wll t need to

More information

An Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices

An Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices Internatonal Mathematcal Forum, Vol 7, 2012, no 52, 2549-2554 An Applcaton of the Dulmage-Mendelsohn Decomposton to Sparse Null Space Bases of Full Row Rank Matrces Mostafa Khorramzadeh Department of Mathematcal

More information

Brave New World Pseudocode Reference

Brave New World Pseudocode Reference Brave New World Pseudocode Reference Pseudocode s a way to descrbe how to accomplsh tasks usng basc steps lke those a computer mght perform. In ths week s lab, you'll see how a form of pseudocode can be

More information

CS 534: Computer Vision Model Fitting

CS 534: Computer Vision Model Fitting CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust

More information

Introduction to Programming. Lecture 13: Container data structures. Container data structures. Topics for this lecture. A basic issue with containers

Introduction to Programming. Lecture 13: Container data structures. Container data structures. Topics for this lecture. A basic issue with containers 1 2 Introducton to Programmng Bertrand Meyer Lecture 13: Contaner data structures Last revsed 1 December 2003 Topcs for ths lecture 3 Contaner data structures 4 Contaners and genercty Contan other objects

More information

Compiler Design. Spring Register Allocation. Sample Exercises and Solutions. Prof. Pedro C. Diniz

Compiler Design. Spring Register Allocation. Sample Exercises and Solutions. Prof. Pedro C. Diniz Compler Desgn Sprng 2014 Regster Allocaton Sample Exercses and Solutons Prof. Pedro C. Dnz USC / Informaton Scences Insttute 4676 Admralty Way, Sute 1001 Marna del Rey, Calforna 90292 pedro@s.edu Regster

More information

MIXED INTEGER-DISCRETE-CONTINUOUS OPTIMIZATION BY DIFFERENTIAL EVOLUTION Part 1: the optimization method

MIXED INTEGER-DISCRETE-CONTINUOUS OPTIMIZATION BY DIFFERENTIAL EVOLUTION Part 1: the optimization method MIED INTEGER-DISCRETE-CONTINUOUS OPTIMIZATION BY DIFFERENTIAL EVOLUTION Part : the optmzaton method Joun Lampnen Unversty of Vaasa Department of Informaton Technology and Producton Economcs P. O. Box 700

More information

Learning the Kernel Parameters in Kernel Minimum Distance Classifier

Learning the Kernel Parameters in Kernel Minimum Distance Classifier Learnng the Kernel Parameters n Kernel Mnmum Dstance Classfer Daoqang Zhang 1,, Songcan Chen and Zh-Hua Zhou 1* 1 Natonal Laboratory for Novel Software Technology Nanjng Unversty, Nanjng 193, Chna Department

More information

Multicriteria Decision Making

Multicriteria Decision Making Multcrtera Decson Makng Andrés Ramos (Andres.Ramos@comllas.edu) Pedro Sánchez (Pedro.Sanchez@comllas.edu) Sonja Wogrn (Sonja.Wogrn@comllas.edu) Contents 1. Basc concepts 2. Contnuous methods 3. Dscrete

More information

Structure from Motion

Structure from Motion Structure from Moton Structure from Moton For now, statc scene and movng camera Equvalentl, rgdl movng scene and statc camera Lmtng case of stereo wth man cameras Lmtng case of multvew camera calbraton

More information

High resolution 3D Tau-p transform by matching pursuit Weiping Cao* and Warren S. Ross, Shearwater GeoServices

High resolution 3D Tau-p transform by matching pursuit Weiping Cao* and Warren S. Ross, Shearwater GeoServices Hgh resoluton 3D Tau-p transform by matchng pursut Wepng Cao* and Warren S. Ross, Shearwater GeoServces Summary The 3D Tau-p transform s of vtal sgnfcance for processng sesmc data acqured wth modern wde

More information

Active Contours/Snakes

Active Contours/Snakes Actve Contours/Snakes Erkut Erdem Acknowledgement: The sldes are adapted from the sldes prepared by K. Grauman of Unversty of Texas at Austn Fttng: Edges vs. boundares Edges useful sgnal to ndcate occludng

More information

EVALUATION OF THE PERFORMANCES OF ARTIFICIAL BEE COLONY AND INVASIVE WEED OPTIMIZATION ALGORITHMS ON THE MODIFIED BENCHMARK FUNCTIONS

EVALUATION OF THE PERFORMANCES OF ARTIFICIAL BEE COLONY AND INVASIVE WEED OPTIMIZATION ALGORITHMS ON THE MODIFIED BENCHMARK FUNCTIONS Academc Research Internatonal ISS-L: 3-9553, ISS: 3-9944 Vol., o. 3, May 0 EVALUATIO OF THE PERFORMACES OF ARTIFICIAL BEE COLOY AD IVASIVE WEED OPTIMIZATIO ALGORITHMS O THE MODIFIED BECHMARK FUCTIOS Dlay

More information

Lecture 5: Multilayer Perceptrons

Lecture 5: Multilayer Perceptrons Lecture 5: Multlayer Perceptrons Roger Grosse 1 Introducton So far, we ve only talked about lnear models: lnear regresson and lnear bnary classfers. We noted that there are functons that can t be represented

More information

An Optimal Algorithm for Prufer Codes *

An Optimal Algorithm for Prufer Codes * J. Software Engneerng & Applcatons, 2009, 2: 111-115 do:10.4236/jsea.2009.22016 Publshed Onlne July 2009 (www.scrp.org/journal/jsea) An Optmal Algorthm for Prufer Codes * Xaodong Wang 1, 2, Le Wang 3,

More information

Dijkstra s Single Source Algorithm. All-Pairs Shortest Paths. Dynamic Programming Solution. Performance. Decision Sequence.

Dijkstra s Single Source Algorithm. All-Pairs Shortest Paths. Dynamic Programming Solution. Performance. Decision Sequence. All-Pars Shortest Paths Gven an n-vertex drected weghted graph, fnd a shortest path from vertex to vertex for each of the n vertex pars (,). Dstra s Sngle Source Algorthm Use Dstra s algorthm n tmes, once

More information

CSE 326: Data Structures Quicksort Comparison Sorting Bound

CSE 326: Data Structures Quicksort Comparison Sorting Bound CSE 326: Data Structures Qucksort Comparson Sortng Bound Bran Curless Sprng 2008 Announcements (5/14/08) Homework due at begnnng of class on Frday. Secton tomorrow: Graded homeworks returned More dscusson

More information

Parallel Branch and Bound Algorithm - A comparison between serial, OpenMP and MPI implementations

Parallel Branch and Bound Algorithm - A comparison between serial, OpenMP and MPI implementations Journal of Physcs: Conference Seres Parallel Branch and Bound Algorthm - A comparson between seral, OpenMP and MPI mplementatons To cte ths artcle: Luco Barreto and Mchael Bauer 2010 J. Phys.: Conf. Ser.

More information

U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 5 Luca Trevisan September 7, 2017

U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 5 Luca Trevisan September 7, 2017 U.C. Bereley CS294: Beyond Worst-Case Analyss Handout 5 Luca Trevsan September 7, 207 Scrbed by Haars Khan Last modfed 0/3/207 Lecture 5 In whch we study the SDP relaxaton of Max Cut n random graphs. Quc

More information

2x x l. Module 3: Element Properties Lecture 4: Lagrange and Serendipity Elements

2x x l. Module 3: Element Properties Lecture 4: Lagrange and Serendipity Elements Module 3: Element Propertes Lecture : Lagrange and Serendpty Elements 5 In last lecture note, the nterpolaton functons are derved on the bass of assumed polynomal from Pascal s trangle for the fled varable.

More information

Support Vector Machines

Support Vector Machines Support Vector Machnes Decson surface s a hyperplane (lne n 2D) n feature space (smlar to the Perceptron) Arguably, the most mportant recent dscovery n machne learnng In a nutshell: map the data to a predetermned

More information

CMPS 10 Introduction to Computer Science Lecture Notes

CMPS 10 Introduction to Computer Science Lecture Notes CPS 0 Introducton to Computer Scence Lecture Notes Chapter : Algorthm Desgn How should we present algorthms? Natural languages lke Englsh, Spansh, or French whch are rch n nterpretaton and meanng are not

More information

Problem Set 3 Solutions

Problem Set 3 Solutions Introducton to Algorthms October 4, 2002 Massachusetts Insttute of Technology 6046J/18410J Professors Erk Demane and Shaf Goldwasser Handout 14 Problem Set 3 Solutons (Exercses were not to be turned n,

More information

Feature Reduction and Selection

Feature Reduction and Selection Feature Reducton and Selecton Dr. Shuang LIANG School of Software Engneerng TongJ Unversty Fall, 2012 Today s Topcs Introducton Problems of Dmensonalty Feature Reducton Statstc methods Prncpal Components

More information

CS1100 Introduction to Programming

CS1100 Introduction to Programming Factoral (n) Recursve Program fact(n) = n*fact(n-) CS00 Introducton to Programmng Recurson and Sortng Madhu Mutyam Department of Computer Scence and Engneerng Indan Insttute of Technology Madras nt fact

More information

Exercises (Part 4) Introduction to R UCLA/CCPR. John Fox, February 2005

Exercises (Part 4) Introduction to R UCLA/CCPR. John Fox, February 2005 Exercses (Part 4) Introducton to R UCLA/CCPR John Fox, February 2005 1. A challengng problem: Iterated weghted least squares (IWLS) s a standard method of fttng generalzed lnear models to data. As descrbed

More information

Solutions to Programming Assignment Five Interpolation and Numerical Differentiation

Solutions to Programming Assignment Five Interpolation and Numerical Differentiation College of Engneerng and Coputer Scence Mechancal Engneerng Departent Mechancal Engneerng 309 Nuercal Analyss of Engneerng Systes Sprng 04 Nuber: 537 Instructor: Larry Caretto Solutons to Prograng Assgnent

More information

Dijkstra s Single Source Algorithm. All-Pairs Shortest Paths. Dynamic Programming Solution. Performance

Dijkstra s Single Source Algorithm. All-Pairs Shortest Paths. Dynamic Programming Solution. Performance All-Pars Shortest Paths Gven an n-vertex drected weghted graph, fnd a shortest path from vertex to vertex for each of the n vertex pars (,). Dkstra s Sngle Source Algorthm Use Dkstra s algorthm n tmes,

More information

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 15

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 15 CS434a/541a: Pattern Recognton Prof. Olga Veksler Lecture 15 Today New Topc: Unsupervsed Learnng Supervsed vs. unsupervsed learnng Unsupervsed learnng Net Tme: parametrc unsupervsed learnng Today: nonparametrc

More information

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces Range mages For many structured lght scanners, the range data forms a hghly regular pattern known as a range mage. he samplng pattern s determned by the specfc scanner. Range mage regstraton 1 Examples

More information

Support Vector Machines. CS534 - Machine Learning

Support Vector Machines. CS534 - Machine Learning Support Vector Machnes CS534 - Machne Learnng Perceptron Revsted: Lnear Separators Bnar classfcaton can be veed as the task of separatng classes n feature space: b > 0 b 0 b < 0 f() sgn( b) Lnear Separators

More information

Outline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:

Outline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like: Self-Organzng Maps (SOM) Turgay İBRİKÇİ, PhD. Outlne Introducton Structures of SOM SOM Archtecture Neghborhoods SOM Algorthm Examples Summary 1 2 Unsupervsed Hebban Learnng US Hebban Learnng, Cntd 3 A

More information

The official electronic file of this thesis or dissertation is maintained by the University Libraries on behalf of The Graduate School at Stony Brook

The official electronic file of this thesis or dissertation is maintained by the University Libraries on behalf of The Graduate School at Stony Brook Stony Brook Unversty The offcal electronc fle of ths thess or dssertaton s mantaned by the Unversty Lbrares on behalf of The Graduate School at Stony Brook Unversty. Alll Rgghht tss Reesseerrvveedd bbyy

More information

Parallelism for Nested Loops with Non-uniform and Flow Dependences

Parallelism for Nested Loops with Non-uniform and Flow Dependences Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr

More information

EECS 730 Introduction to Bioinformatics Sequence Alignment. Luke Huan Electrical Engineering and Computer Science

EECS 730 Introduction to Bioinformatics Sequence Alignment. Luke Huan Electrical Engineering and Computer Science EECS 730 Introducton to Bonformatcs Sequence Algnment Luke Huan Electrcal Engneerng and Computer Scence http://people.eecs.ku.edu/~huan/ HMM Π s a set of states Transton Probabltes a kl Pr( l 1 k Probablty

More information

Complex Numbers. Now we also saw that if a and b were both positive then ab = a b. For a second let s forget that restriction and do the following.

Complex Numbers. Now we also saw that if a and b were both positive then ab = a b. For a second let s forget that restriction and do the following. Complex Numbers The last topc n ths secton s not really related to most of what we ve done n ths chapter, although t s somewhat related to the radcals secton as we wll see. We also won t need the materal

More information

Image Representation & Visualization Basic Imaging Algorithms Shape Representation and Analysis. outline

Image Representation & Visualization Basic Imaging Algorithms Shape Representation and Analysis. outline mage Vsualzaton mage Vsualzaton mage Representaton & Vsualzaton Basc magng Algorthms Shape Representaton and Analyss outlne mage Representaton & Vsualzaton Basc magng Algorthms Shape Representaton and

More information

Chapter 6 Programmng the fnte element method Inow turn to the man subject of ths book: The mplementaton of the fnte element algorthm n computer programs. In order to make my dscusson as straghtforward

More information

CSE 326: Data Structures Quicksort Comparison Sorting Bound

CSE 326: Data Structures Quicksort Comparison Sorting Bound CSE 326: Data Structures Qucksort Comparson Sortng Bound Steve Setz Wnter 2009 Qucksort Qucksort uses a dvde and conquer strategy, but does not requre the O(N) extra space that MergeSort does. Here s the

More information

Intro. Iterators. 1. Access

Intro. Iterators. 1. Access Intro Ths mornng I d lke to talk a lttle bt about s and s. We wll start out wth smlartes and dfferences, then we wll see how to draw them n envronment dagrams, and we wll fnsh wth some examples. Happy

More information

NGPM -- A NSGA-II Program in Matlab

NGPM -- A NSGA-II Program in Matlab Verson 1.4 LIN Song Aerospace Structural Dynamcs Research Laboratory College of Astronautcs, Northwestern Polytechncal Unversty, Chna Emal: lsssswc@163.com 2011-07-26 Contents Contents... 1. Introducton...

More information

NAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics

NAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics Introducton G10 NAG Fortran Lbrary Chapter Introducton G10 Smoothng n Statstcs Contents 1 Scope of the Chapter... 2 2 Background to the Problems... 2 2.1 Smoothng Methods... 2 2.2 Smoothng Splnes and Regresson

More information

EXTENDED BIC CRITERION FOR MODEL SELECTION

EXTENDED BIC CRITERION FOR MODEL SELECTION IDIAP RESEARCH REPORT EXTEDED BIC CRITERIO FOR ODEL SELECTIO Itshak Lapdot Andrew orrs IDIAP-RR-0-4 Dalle olle Insttute for Perceptual Artfcal Intellgence P.O.Box 59 artgny Valas Swtzerland phone +4 7

More information

Machine Learning. Support Vector Machines. (contains material adapted from talks by Constantin F. Aliferis & Ioannis Tsamardinos, and Martin Law)

Machine Learning. Support Vector Machines. (contains material adapted from talks by Constantin F. Aliferis & Ioannis Tsamardinos, and Martin Law) Machne Learnng Support Vector Machnes (contans materal adapted from talks by Constantn F. Alfers & Ioanns Tsamardnos, and Martn Law) Bryan Pardo, Machne Learnng: EECS 349 Fall 2014 Support Vector Machnes

More information

Optimizing Document Scoring for Query Retrieval

Optimizing Document Scoring for Query Retrieval Optmzng Document Scorng for Query Retreval Brent Ellwen baellwe@cs.stanford.edu Abstract The goal of ths project was to automate the process of tunng a document query engne. Specfcally, I used machne learnng

More information

A MOVING MESH APPROACH FOR SIMULATION BUDGET ALLOCATION ON CONTINUOUS DOMAINS

A MOVING MESH APPROACH FOR SIMULATION BUDGET ALLOCATION ON CONTINUOUS DOMAINS Proceedngs of the Wnter Smulaton Conference M E Kuhl, N M Steger, F B Armstrong, and J A Jones, eds A MOVING MESH APPROACH FOR SIMULATION BUDGET ALLOCATION ON CONTINUOUS DOMAINS Mark W Brantley Chun-Hung

More information

Kent State University CS 4/ Design and Analysis of Algorithms. Dept. of Math & Computer Science LECT-16. Dynamic Programming

Kent State University CS 4/ Design and Analysis of Algorithms. Dept. of Math & Computer Science LECT-16. Dynamic Programming CS 4/560 Desgn and Analyss of Algorthms Kent State Unversty Dept. of Math & Computer Scence LECT-6 Dynamc Programmng 2 Dynamc Programmng Dynamc Programmng, lke the dvde-and-conquer method, solves problems

More information

Parallel matrix-vector multiplication

Parallel matrix-vector multiplication Appendx A Parallel matrx-vector multplcaton The reduced transton matrx of the three-dmensonal cage model for gel electrophoress, descrbed n secton 3.2, becomes excessvely large for polymer lengths more

More information

Agenda & Reading. Simple If. Decision-Making Statements. COMPSCI 280 S1C Applications Programming. Programming Fundamentals

Agenda & Reading. Simple If. Decision-Making Statements. COMPSCI 280 S1C Applications Programming. Programming Fundamentals Agenda & Readng COMPSCI 8 SC Applcatons Programmng Programmng Fundamentals Control Flow Agenda: Decsonmakng statements: Smple If, Ifelse, nested felse, Select Case s Whle, DoWhle/Untl, For, For Each, Nested

More information

Meta-heuristics for Multidimensional Knapsack Problems

Meta-heuristics for Multidimensional Knapsack Problems 2012 4th Internatonal Conference on Computer Research and Development IPCSIT vol.39 (2012) (2012) IACSIT Press, Sngapore Meta-heurstcs for Multdmensonal Knapsack Problems Zhbao Man + Computer Scence Department,

More information

Topology Design using LS-TaSC Version 2 and LS-DYNA

Topology Design using LS-TaSC Version 2 and LS-DYNA Topology Desgn usng LS-TaSC Verson 2 and LS-DYNA Wllem Roux Lvermore Software Technology Corporaton, Lvermore, CA, USA Abstract Ths paper gves an overvew of LS-TaSC verson 2, a topology optmzaton tool

More information

11. APPROXIMATION ALGORITHMS

11. APPROXIMATION ALGORITHMS Copng wth NP-completeness 11. APPROXIMATION ALGORITHMS load balancng center selecton prcng method: vertex cover LP roundng: vertex cover generalzed load balancng knapsack problem Q. Suppose I need to solve

More information

Efficient Distributed Linear Classification Algorithms via the Alternating Direction Method of Multipliers

Efficient Distributed Linear Classification Algorithms via the Alternating Direction Method of Multipliers Effcent Dstrbuted Lnear Classfcaton Algorthms va the Alternatng Drecton Method of Multplers Caoxe Zhang Honglak Lee Kang G. Shn Department of EECS Unversty of Mchgan Ann Arbor, MI 48109, USA caoxezh@umch.edu

More information

Announcements. Supervised Learning

Announcements. Supervised Learning Announcements See Chapter 5 of Duda, Hart, and Stork. Tutoral by Burge lnked to on web page. Supervsed Learnng Classfcaton wth labeled eamples. Images vectors n hgh-d space. Supervsed Learnng Labeled eamples

More information

Design for Reliability: Case Studies in Manufacturing Process Synthesis

Design for Reliability: Case Studies in Manufacturing Process Synthesis Desgn for Relablty: Case Studes n Manufacturng Process Synthess Y. Lawrence Yao*, and Chao Lu Department of Mechancal Engneerng, Columba Unversty, Mudd Bldg., MC 473, New York, NY 7, USA * Correspondng

More information

Simulation: Solving Dynamic Models ABE 5646 Week 11 Chapter 2, Spring 2010

Simulation: Solving Dynamic Models ABE 5646 Week 11 Chapter 2, Spring 2010 Smulaton: Solvng Dynamc Models ABE 5646 Week Chapter 2, Sprng 200 Week Descrpton Readng Materal Mar 5- Mar 9 Evaluatng [Crop] Models Comparng a model wth data - Graphcal, errors - Measures of agreement

More information

Outline. Type of Machine Learning. Examples of Application. Unsupervised Learning

Outline. Type of Machine Learning. Examples of Application. Unsupervised Learning Outlne Artfcal Intellgence and ts applcatons Lecture 8 Unsupervsed Learnng Professor Danel Yeung danyeung@eee.org Dr. Patrck Chan patrckchan@eee.org South Chna Unversty of Technology, Chna Introducton

More information

K-means and Hierarchical Clustering

K-means and Hierarchical Clustering Note to other teachers and users of these sldes. Andrew would be delghted f you found ths source materal useful n gvng your own lectures. Feel free to use these sldes verbatm, or to modfy them to ft your

More information

12/2/2009. Announcements. Parametric / Non-parametric. Case-Based Reasoning. Nearest-Neighbor on Images. Nearest-Neighbor Classification

12/2/2009. Announcements. Parametric / Non-parametric. Case-Based Reasoning. Nearest-Neighbor on Images. Nearest-Neighbor Classification Introducton to Artfcal Intellgence V22.0472-001 Fall 2009 Lecture 24: Nearest-Neghbors & Support Vector Machnes Rob Fergus Dept of Computer Scence, Courant Insttute, NYU Sldes from Danel Yeung, John DeNero

More information