Lecture 17: Feature Subset Selection II
|
|
- Crystal Maxwell
- 5 years ago
- Views:
Transcription
1 Lecture 17: Feature Subset Selectio II Expoetial search methods Brach ad Boud Approximate Mootoicity with Brach ad Boud Beam Search Radomized alorithms Radom Geeratio plus Sequetial Selectio Simulated Aeali Geetic Alorithms Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 1
2 Brach ad Boud (B&B) (1) The Brach ad Boud alorithm, developed by Naredra ad Fukuaa i 1977, is uarateed to fid the optimal feature subset uder the mootoicity assumptio The mootoicity assumptio states that the additio of features ca oly icrease the value of the objective fuctio, this is J ( x ) < J( x, x ) < J( x, x, x ) <L< J( x, x, L, ) 1 i1 i2 i1 i2 i3 i1 i2 in i x Brach ad Boud starts from the full set ad removes features usi a depth-first stratey Nodes whose objective fuctio are lower tha the curret best are ot explored sice the mootoicity assumptio esures that their childre will ot cotai a better solutio Empty feature set Full feature set Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 2
3 Brach ad Boud (2) Alorithm The alorithm is better explaied by cosideri the subsets of M =N-M features already discarded, where N is the dimesioality of the state space ad M is the desired umber of features Sice the order of the features is irrelevat, we will oly cosider a icreasi orderi i 1 <i 2 <...i M of the feature idices, this will avoid explori states that differ oly i the orderi of their features The Brach ad Boud tree for N=6 ad M=2 is show below (umbers idicate features that are bei removed) Notice that at the level directly below the root we oly cosider removi features 1, 2 or 3, sice a hiher umber would ot allow sequeces (i 1 < i 2 < i 3 < i 4 ) with four idices Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 3
4 Brach ad Boud (3) 1. Iitialize: α=-, k=0 2. Geerate successors of the curret ode ad store them i LIST(k) 3. Select ew ode if if LIST(k) is empty o to Step 5 else i = armax delete i k [ J( x,x,...x, j) ] k i1 i2 ik 1 j LIST(k) from LIST(k) 4. Check boud if if J( xi,x,...x ) 1 i2 i < k o to 5 else if if k=m (we have the desired umber of features) o to 6 else k=k+1 o to 2 5. Backtrack to lower level set k=k-1 if if k=0 termiate alorithm else o to 3 6. Last level * Set = J( xi 1,xi 2,...xi k 1, j) ad YM = { xi 1,xi 2,... xi k } o to 5 Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 4
5 Approximate Mootoicity with B & B (AMB&B) AMB&B is a variatio of the classical Brach ad Boud alorithm AMB&B allows o-mootoic fuctios to be used, typically classifiers, by relaxi the cutoff coditio that termiates the search o a specific ode Assume that we ru B&B by setti a threshold error rate τ rather tha a umber of features M Uder AMB&B, a ive feature subset Y will be cosidered Feasible if J(Y) τ Coditioally feasible if J(Y) τ(1+ ) Ufeasible if J(Y) τ(1+ ) is a tolerace placed o the threshold to accommodate o-mootoic fuctios Rather tha limiti the search to feasible odes (like B&B does), AMB&B allows the search to explore coditioally feasible odes with the hope that these odes will lead to a feasible solutio However, AMB&B will ot retur coditioally feasible odes as solutios, it oly allows the search to explore them! Otherwise it would ot be ay differet tha B&B with a hiher threshold of τ(1+ ) Empty feature set Full feature set Coditioally feasible Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 5
6 Beam Search (1) Beam Search is a variatio of best-first search with a bouded queue to limit the scope of the search The queue oraizes states from best to worst, with the best states placed at the head of the queue At every iteratio, BS evaluates all possible states that result from addi a feature to the feature subset, ad the results are iserted ito the queue i their proper locatios It is trivial to otice that BS deeerates to Exhaustive search if there is o limit o the size of the queue. Similarly, if the queue size is set to oe, BS is equivalet to Sequetial Forward Selectio Empty feature set Full feature set Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 6
7 Beam Search (2) The example below illustrates BS for a 4-dimesioal search space ad a queue of size 3 BS caot uaratee that the optimal subset is foud: i the example, the optimal is 2-3-4(9), which is ever explored however, with the proper queue size, Beam Search ca avoid etti trapped i local miimal by preservi solutios from varyi reios i the search space root 2(6) 1(5) 3(5) 4(2) 2(3) 3(8) 4(6) 3(4) 4(5) 4(1) LIST={ } 3(7) 4(3) 4(5) 4(9) 4(5) root 1(5) 2(3) 3(4) 2(6) 3(5) 4(2) 3(8) 4(6) 4(5) 3(7) 4(3) 4(5) 4(9) 4(5) root 1(5) 2(3) 3(4) 2(6) 3(5) 4(2) 3(8) 4(6) 4(5) 3(7) 4(3) 4(5) 4(9) 4(5) root 1(5) 2(3) 3(4) 2(6) 3(5) 4(2) 3(8) 4(6) 4(5) 3(7) 4(3) 4(5) 4(9) 4(5) 4(1) 4(1) 4(1) LIST={1(5), 3(4), 2(3)} LIST={1-2(6), 1-3(5), 3(4)} LIST={1-2-3(7), 1-3(5), 3(4)} Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 7
8 Radom Geeratio plus Sequetial Selectio RGSS is a attempt to itroduce radomess ito SFS ad SBS i order to escape local miima The alorithm is self-explaatory Empty feature set 1. Repeat for a umber of iteratios 1a.Geerate a radom feature subset 1b.Perform SFS o this subset 1c. Perform SBS o this subset Full feature set Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 8
9 Simulated Aeali (1) Simulated Aeali is a stochastic optimizatio method that derives its ame from the aeali process used to re-crystallize metals Duri the aeali process i metals, the alloy is cooled dow slowly to allow its atoms to reach a cofiuratio of miimum eery (a perfectly reular crystal) If the alloy is aealed too fast, such a oraizatio caot propaate throuhout the material. The result will be a material with reios of reular structure separated by boudaries. These boudaries are potetial fault-lies where fractures are most likely to occur whe the material is stressed The laws of thermodyamics state that, at temperature T, the probability of a icrease i eery E i the system is ive by the expressio ( kt ( () P = e where k is kow as the Boltzma s costat The alorithm is a straihtforward implemetatio of these ideas Empty feature set 1. Determie a aeali schedule T(i) 2. Create a iitial solutio Y(0) 3. While T(i)>T MIN 3a.Geerate a ew solutio Y(i+1) which is a eihbor of Y(i) 3b.Compute E= - [ J(Y(i+1)) - J(Y(i)) ] 3b.If E<0 the always accept the move from Y(i) to Y(i+1) else accept the move with probability P=exp(- E/T(i)) Full feature set Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 9
10 Simulated Aeali (2) Simulated aeali is summarized with the followi idea [Hayki, 1999] Whe optimizi a very lare ad complex system (i.e., a system with may derees of freedom), istead of always oi dowhill, try to o dowhill most of the times The previous formulatio of the Simulated Aeali alorithm ca be used for ay type of miimizatio problem ad it oly requires specificatio of A trasform to eerate a local eihbor from the curret solutio (i.e. add a radom vector) For Feature Subset Selectio, the trasform will cosist of addi or removi features, typically implemeted as a radom mutatio with low probability A aeali schedule, typically T(i+1)=rT(i), with 0.0 r 1.0 A iitial temperature T(0) Selectio of the aeali schedule is critical If r is chose close too lare, the temperature decreases very slowly, allowi moves to hiher eery states to occur more frequetly. This results i slow coverece If r is chose too small, the temperature decreases very fast, ad the alorithm is likely to covere to a local miima Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 10
11 Simulated Aeali (3) A uique feature of simulated aeali is its adaptive ature At hih temperature the alorithm is oly looki at the ross features of the optimizatio surface, while at low temperatures, the fier details of the surface start to appear J(Y) Hih T Low T Y Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 11
12 Geetic Alorithms Geetic alorithms are optimizatio techiques that mimic the evolutioary process of survival of the fittest Starti with a iitial radom populatio of solutios, evolve ew populatios by mati (crossover) pairs of solutios ad mutati solutios accordi to their fitess (objective fuctio) The better solutios are more likely to be selected for the mati ad mutatio operatios ad therefore carry their eetic code from eeratio to eeratio For the problem of Feature Subset Selectio, idividual solutios are simply represeted with a biary umber (1 if the ive feature is selected, 0 otherwise), which is the oriial represetatio proposed by Hollad i 1974 Empty feature set Alorithm 1. Create a iitial radom populatio 2. Evaluate iitial populatio 2. Repeat util coverece (or a umber of eeratios) 2a.Select the fittest idividuals i the populatio 2b.Perform crossover o the selected idividuals to create offspri 2c. Perform mutatio o the selected idividuals 2d.Create the ew populatio from the old populatio ad the offspri 2e.Evaluate the ew populatio Full feature set Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 12
13 Geetic operators Sile-poit crossover Select two idividuals (parets) accordi to their fitess Select a crossover poit With probability P c (0.95 is reasoable) create two offspri by combii the parets Crossover poit selected radomly Paret i Paret j Crossover Offspri i Offspri j Biary mutatio Select a idividual accordi to its fitess With probability P M (0.01 is reasoable) mutate each oe of its bits Mutated bits Idividual Mutatio Offspri i Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 13
14 Selectio methods The selectio of idividuals is based o their fitess (the value of the objective fuctio) We will describe a selectio method called Geometric selectio Several other methods are available: Roulette Wheel, Touramet Selectio, etc. Geometric selectio The probability of selecti the r th best idividual is ive by the eometric probability mass fuctio q is the probability of selecti the best idividual (0.05 is a reasoable value) Therefore, the eometric distributio assis hiher probability to idividuals raked better, but also allows ufit idividuals to be selected I additio, it is typical to carry the best idividual of each populatio to the ext oe () r q( 1- q) r-1 P = This is called the Elitist Model Selectio probability q=0.08 q=0.04 q=0.02 q= Idividual rak Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 14
15 GAs, parameter choices for Feature selectio The choice of crossover rate P C is ot critical You will wat a value close to 1.0 to have a lare umber of offspri The choice of mutatio rate P M is very critical A optimal choice of P M will allow the GA to explore the more promisi reios while avoidi etti trapped i local miima A lare value (i.e., P M >0.25) will ot allow the search to focus o the better reios, ad the GA will perform like radom search A small value (i.e., close to 0.0) will ot allow the search to escape local miima The choice of q, the probability of selecti the best idividual is also critical A optimal value of q will allow the GA to explore the most promisi solutio, ad at the same time provide sufficiet diversity to avoid early coverece of the alorithm I eeral, poorly selected cotrol parameters will result i sub-optimal solutios due to early coverece Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 15
16 Search Strateies, summary Accuracy Complexity Advataes Disadvataes Exhaustive Sequetial Radomized Always fids the optimal solutio Good if o backtracki eeded Good with proper cotrol parameters Expoetial Hih accuracy Hih complexity Quadratic O(N EX 2) Simple ad fast Geerally low Desied to escape local miima Caot backtrack Difficult to choose ood parameters A hihly recommeded review of the material preseted i these two lectures is Justi Doak A evaluatio of feature selectio methods ad their applicatio to Computer Security Uiversity of Califoria at Davis, Tech Report CSE Itroductio to Patter Recoitio Ricardo Gutierrez-Osua Wriht State Uiversity 16
Lecture 13: Validation
Lecture 3: Validatio Resampli methods Holdout Cross Validatio Radom Subsampli -Fold Cross-Validatio Leave-oe-out The Bootstrap Bias ad variace estimatio Three-way data partitioi Itroductio to Patter Recoitio
More informationCIS 121 Data Structures and Algorithms with Java Spring Stacks and Queues Monday, February 12 / Tuesday, February 13
CIS Data Structures ad Algorithms with Java Sprig 08 Stacks ad Queues Moday, February / Tuesday, February Learig Goals Durig this lab, you will: Review stacks ad queues. Lear amortized ruig time aalysis
More informationCIS 121 Data Structures and Algorithms with Java Spring Stacks, Queues, and Heaps Monday, February 18 / Tuesday, February 19
CIS Data Structures ad Algorithms with Java Sprig 09 Stacks, Queues, ad Heaps Moday, February 8 / Tuesday, February 9 Stacks ad Queues Recall the stack ad queue ADTs (abstract data types from lecture.
More informationLecture 11: PI/T parallel I/O, part I
Lecture 11: PI/T parallel I/O, part I Geeral descriptio of the parallel I/O fuctio Bufferi Hadshaki Iput ad Output trasfers Timi Diarams Reister model of the 68230 Port Geeral Cotrol Reister (PGCR) Port
More informationCIS 121. Introduction to Trees
CIS 121 Itroductio to Trees 1 Tree ADT Tree defiitio q A tree is a set of odes which may be empty q If ot empty, the there is a distiguished ode r, called root ad zero or more o-empty subtrees T 1, T 2,
More information. Written in factored form it is easy to see that the roots are 2, 2, i,
CMPS A Itroductio to Programmig Programmig Assigmet 4 I this assigmet you will write a java program that determies the real roots of a polyomial that lie withi a specified rage. Recall that the roots (or
More informationIntroduction. Nature-Inspired Computing. Terminology. Problem Types. Constraint Satisfaction Problems - CSP. Free Optimization Problem - FOP
Nature-Ispired Computig Hadlig Costraits Dr. Şima Uyar September 2006 Itroductio may practical problems are costraied ot all combiatios of variable values represet valid solutios feasible solutios ifeasible
More informationLecture 5. Counting Sort / Radix Sort
Lecture 5. Coutig Sort / Radix Sort T. H. Corme, C. E. Leiserso ad R. L. Rivest Itroductio to Algorithms, 3rd Editio, MIT Press, 2009 Sugkyukwa Uiversity Hyuseug Choo choo@skku.edu Copyright 2000-2018
More informationHash Tables. Presentation for use with the textbook Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 2015.
Presetatio for use with the textbook Algorithm Desig ad Applicatios, by M. T. Goodrich ad R. Tamassia, Wiley, 2015 Hash Tables xkcd. http://xkcd.com/221/. Radom Number. Used with permissio uder Creative
More informationMałgorzata Sterna. Mateusz Cicheński, Mateusz Jarus, Michał Miszkiewicz, Jarosław Szymczak
Małgorzata Stera Mateusz Cicheński, Mateusz Jarus, Michał Miszkiewicz, Jarosław Szymczak Istitute of Computig Sciece Pozań Uiversity of Techology Pozań - Polad Scope of the Talk Problem defiitio MP Formulatio
More informationComputational Geometry
Computatioal Geometry Chapter 4 Liear programmig Duality Smallest eclosig disk O the Ageda Liear Programmig Slides courtesy of Craig Gotsma 4. 4. Liear Programmig - Example Defie: (amout amout cosumed
More information6.854J / J Advanced Algorithms Fall 2008
MIT OpeCourseWare http://ocw.mit.edu 6.854J / 18.415J Advaced Algorithms Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 18.415/6.854 Advaced Algorithms
More informationHeaps. Presentation for use with the textbook Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 2015
Presetatio for use with the textbook Algorithm Desig ad Applicatios, by M. T. Goodrich ad R. Tamassia, Wiley, 201 Heaps 201 Goodrich ad Tamassia xkcd. http://xkcd.com/83/. Tree. Used with permissio uder
More informationCS200: Hash Tables. Prichard Ch CS200 - Hash Tables 1
CS200: Hash Tables Prichard Ch. 13.2 CS200 - Hash Tables 1 Table Implemetatios: average cases Search Add Remove Sorted array-based Usorted array-based Balaced Search Trees O(log ) O() O() O() O(1) O()
More informationLecture 18. Optimization in n dimensions
Lecture 8 Optimizatio i dimesios Itroductio We ow cosider the problem of miimizig a sigle scalar fuctio of variables, f x, where x=[ x, x,, x ]T. The D case ca be visualized as fidig the lowest poit of
More informationLecture 6. Lecturer: Ronitt Rubinfeld Scribes: Chen Ziv, Eliav Buchnik, Ophir Arie, Jonathan Gradstein
068.670 Subliear Time Algorithms November, 0 Lecture 6 Lecturer: Roitt Rubifeld Scribes: Che Ziv, Eliav Buchik, Ophir Arie, Joatha Gradstei Lesso overview. Usig the oracle reductio framework for approximatig
More informationLecture Notes 6 Introduction to algorithm analysis CSS 501 Data Structures and Object-Oriented Programming
Lecture Notes 6 Itroductio to algorithm aalysis CSS 501 Data Structures ad Object-Orieted Programmig Readig for this lecture: Carrao, Chapter 10 To be covered i this lecture: Itroductio to algorithm aalysis
More informationBasic allocator mechanisms The course that gives CMU its Zip! Memory Management II: Dynamic Storage Allocation Mar 6, 2000.
5-23 The course that gives CM its Zip Memory Maagemet II: Dyamic Storage Allocatio Mar 6, 2000 Topics Segregated lists Buddy system Garbage collectio Mark ad Sweep Copyig eferece coutig Basic allocator
More informationHomework 1 Solutions MA 522 Fall 2017
Homework 1 Solutios MA 5 Fall 017 1. Cosider the searchig problem: Iput A sequece of umbers A = [a 1,..., a ] ad a value v. Output A idex i such that v = A[i] or the special value NIL if v does ot appear
More informationDesigning a learning system
CS 75 Machie Learig Lecture Desigig a learig system Milos Hauskrecht milos@cs.pitt.edu 539 Seott Square, x-5 people.cs.pitt.edu/~milos/courses/cs75/ Admiistrivia No homework assigmet this week Please try
More informationOnes Assignment Method for Solving Traveling Salesman Problem
Joural of mathematics ad computer sciece 0 (0), 58-65 Oes Assigmet Method for Solvig Travelig Salesma Problem Hadi Basirzadeh Departmet of Mathematics, Shahid Chamra Uiversity, Ahvaz, Ira Article history:
More informationFuzzy Rule Selection by Data Mining Criteria and Genetic Algorithms
Fuzzy Rule Selectio by Data Miig Criteria ad Geetic Algorithms Hisao Ishibuchi Dept. of Idustrial Egieerig Osaka Prefecture Uiversity 1-1 Gakue-cho, Sakai, Osaka 599-8531, JAPAN E-mail: hisaoi@ie.osakafu-u.ac.jp
More informationCIS 121 Data Structures and Algorithms with Java Fall Big-Oh Notation Tuesday, September 5 (Make-up Friday, September 8)
CIS 11 Data Structures ad Algorithms with Java Fall 017 Big-Oh Notatio Tuesday, September 5 (Make-up Friday, September 8) Learig Goals Review Big-Oh ad lear big/small omega/theta otatios Practice solvig
More informationUniversity of Waterloo Department of Electrical and Computer Engineering ECE 250 Algorithms and Data Structures
Uiversity of Waterloo Departmet of Electrical ad Computer Egieerig ECE 250 Algorithms ad Data Structures Midterm Examiatio ( pages) Istructor: Douglas Harder February 7, 2004 7:30-9:00 Name (last, first)
More informationAn Improved Shuffled Frog-Leaping Algorithm for Knapsack Problem
A Improved Shuffled Frog-Leapig Algorithm for Kapsack Problem Zhoufag Li, Ya Zhou, ad Peg Cheg School of Iformatio Sciece ad Egieerig Hea Uiversity of Techology ZhegZhou, Chia lzhf1978@126.com Abstract.
More informationAdministrative UNSUPERVISED LEARNING. Unsupervised learning. Supervised learning 11/25/13. Final project. No office hours today
Admiistrative Fial project No office hours today UNSUPERVISED LEARNING David Kauchak CS 451 Fall 2013 Supervised learig Usupervised learig label label 1 label 3 model/ predictor label 4 label 5 Supervised
More informationPattern Recognition Systems Lab 1 Least Mean Squares
Patter Recogitio Systems Lab 1 Least Mea Squares 1. Objectives This laboratory work itroduces the OpeCV-based framework used throughout the course. I this assigmet a lie is fitted to a set of poits usig
More informationData Structures Week #5. Trees (Ağaçlar)
Data Structures Week #5 Trees Ağaçlar) Trees Ağaçlar) Toros Gökarı Avrupa Gökarı October 28, 2014 Boraha Tümer, Ph.D. 2 Trees Ağaçlar) October 28, 2014 Boraha Tümer, Ph.D. 3 Outlie Trees Deiitios Implemetatio
More informationElementary Educational Computer
Chapter 5 Elemetary Educatioal Computer. Geeral structure of the Elemetary Educatioal Computer (EEC) The EEC coforms to the 5 uits structure defied by vo Neuma's model (.) All uits are preseted i a simplified
More informationcondition w i B i S maximum u i
ecture 10 Dyamic Programmig 10.1 Kapsack Problem November 1, 2004 ecturer: Kamal Jai Notes: Tobias Holgers We are give a set of items U = {a 1, a 2,..., a }. Each item has a weight w i Z + ad a utility
More informationAlgorithm. Counting Sort Analysis of Algorithms
Algorithm Coutig Sort Aalysis of Algorithms Assumptios: records Coutig sort Each record cotais keys ad data All keys are i the rage of 1 to k Space The usorted list is stored i A, the sorted list will
More informationData Structures and Algorithms Part 1.4
1 Data Structures ad Algorithms Part 1.4 Werer Nutt 2 DSA, Part 1: Itroductio, syllabus, orgaisatio Algorithms Recursio (priciple, trace, factorial, Fiboacci) Sortig (bubble, isertio, selectio) 3 Sortig
More informationPseudocode ( 1.1) Analysis of Algorithms. Primitive Operations. Pseudocode Details. Running Time ( 1.1) Estimating performance
Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Pseudocode ( 1.1) High-level descriptio of a algorithm More structured
More informationLecture 1: Introduction and Strassen s Algorithm
5-750: Graduate Algorithms Jauary 7, 08 Lecture : Itroductio ad Strasse s Algorithm Lecturer: Gary Miller Scribe: Robert Parker Itroductio Machie models I this class, we will primarily use the Radom Access
More informationA SOFTWARE MODEL FOR THE MULTILAYER PERCEPTRON
A SOFTWARE MODEL FOR THE MULTILAYER PERCEPTRON Roberto Lopez ad Eugeio Oñate Iteratioal Ceter for Numerical Methods i Egieerig (CIMNE) Edificio C1, Gra Capitá s/, 08034 Barceloa, Spai ABSTRACT I this work
More informationEE260: Digital Design, Spring /16/18. n Example: m 0 (=x 1 x 2 ) is adjacent to m 1 (=x 1 x 2 ) and m 2 (=x 1 x 2 ) but NOT m 3 (=x 1 x 2 )
EE26: Digital Desig, Sprig 28 3/6/8 EE 26: Itroductio to Digital Desig Combiatioal Datapath Yao Zheg Departmet of Electrical Egieerig Uiversity of Hawaiʻi at Māoa Combiatioal Logic Blocks Multiplexer Ecoders/Decoders
More informationCopyright 2016 Ramez Elmasri and Shamkant B. Navathe
Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe CHAPTER 19 Query Optimizatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Itroductio Query optimizatio Coducted by a query optimizer i a DBMS Goal:
More informationEE 459/500 HDL Based Digital Design with Programmable Logic. Lecture 13 Control and Sequencing: Hardwired and Microprogrammed Control
EE 459/500 HDL Based Digital Desig with Programmable Logic Lecture 13 Cotrol ad Sequecig: Hardwired ad Microprogrammed Cotrol Refereces: Chapter s 4,5 from textbook Chapter 7 of M.M. Mao ad C.R. Kime,
More informationStructuring Redundancy for Fault Tolerance. CSE 598D: Fault Tolerant Software
Structurig Redudacy for Fault Tolerace CSE 598D: Fault Tolerat Software What do we wat to achieve? Versios Damage Assessmet Versio 1 Error Detectio Iputs Versio 2 Voter Outputs State Restoratio Cotiued
More informationHeuristic Set-Covering-Based Postprocessing for Improving the Quine-McCluskey Method
Heuristic Set-Coverig-Based Postprocessig for Improvig the Quie-McCluskey Method Miloš Šeda Abstract Fidig the miimal logical fuctios has importat applicatios i the desig of logical circuits. This task
More informationA New Morphological 3D Shape Decomposition: Grayscale Interframe Interpolation Method
A ew Morphological 3D Shape Decompositio: Grayscale Iterframe Iterpolatio Method D.. Vizireau Politehica Uiversity Bucharest, Romaia ae@comm.pub.ro R. M. Udrea Politehica Uiversity Bucharest, Romaia mihea@comm.pub.ro
More informationHeuristic Approaches for Solving the Multidimensional Knapsack Problem (MKP)
Heuristic Approaches for Solvig the Multidimesioal Kapsack Problem (MKP) R. PARRA-HERNANDEZ N. DIMOPOULOS Departmet of Electrical ad Computer Eg. Uiversity of Victoria Victoria, B.C. CANADA Abstract: -
More informationquality/quantity peak time/ratio
Semi-Heap ad Its Applicatios i Touramet Rakig Jie Wu Departmet of omputer Sciece ad Egieerig Florida Atlatic Uiversity oca Rato, FL 3343 jie@cse.fau.edu September, 00 . Itroductio ad Motivatio. relimiaries
More informationData diverse software fault tolerance techniques
Data diverse software fault tolerace techiques Complemets desig diversity by compesatig for desig diversity s s limitatios Ivolves obtaiig a related set of poits i the program data space, executig the
More informationThe isoperimetric problem on the hypercube
The isoperimetric problem o the hypercube Prepared by: Steve Butler November 2, 2005 1 The isoperimetric problem We will cosider the -dimesioal hypercube Q Recall that the hypercube Q is a graph whose
More informationParabolic Path to a Best Best-Fit Line:
Studet Activity : Fidig the Least Squares Regressio Lie By Explorig the Relatioship betwee Slope ad Residuals Objective: How does oe determie a best best-fit lie for a set of data? Eyeballig it may be
More informationSolution printed. Do not start the test until instructed to do so! CS 2604 Data Structures Midterm Spring, Instructions:
CS 604 Data Structures Midterm Sprig, 00 VIRG INIA POLYTECHNIC INSTITUTE AND STATE U T PROSI M UNI VERSI TY Istructios: Prit your ame i the space provided below. This examiatio is closed book ad closed
More informationA Novel Approach to Solve Multiple Traveling Salesmen Problem by Genetic Algorithm
A Novel Approach to Solve Multiple Travelig Salesme Problem by Geetic Algorithm Adrás Király, Jáos Aboyi Uiversity of Paoia, Departmet of Process Egieerig, P.O. Box 58. Veszprém H-8200, HUNGARY, e-mail:
More informationHow do we evaluate algorithms?
F2 Readig referece: chapter 2 + slides Algorithm complexity Big O ad big Ω To calculate ruig time Aalysis of recursive Algorithms Next time: Litterature: slides mostly The first Algorithm desig methods:
More informationWhat are we going to learn? CSC Data Structures Analysis of Algorithms. Overview. Algorithm, and Inputs
What are we goig to lear? CSC316-003 Data Structures Aalysis of Algorithms Computer Sciece North Carolia State Uiversity Need to say that some algorithms are better tha others Criteria for evaluatio Structure
More informationEvolutionary Hybrid Genetic-Firefly Algorithm for Global Optimization
Iteratioal Joural of Computatioal Egieerig & Maagemet, Vol. 6 Issue, May ISSN (Olie): -789 www..org Evolutioary Hybrid Geetic-Firefly Algorithm for Global Optimizatio 7 Shaik Farook, P. Sagameswara Raju
More information9.1. Sequences and Series. Sequences. What you should learn. Why you should learn it. Definition of Sequence
_9.qxd // : AM Page Chapter 9 Sequeces, Series, ad Probability 9. Sequeces ad Series What you should lear Use sequece otatio to write the terms of sequeces. Use factorial otatio. Use summatio otatio to
More informationRunning Time. Analysis of Algorithms. Experimental Studies. Limitations of Experiments
Ruig Time Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects. The
More informationCOSC 1P03. Ch 7 Recursion. Introduction to Data Structures 8.1
COSC 1P03 Ch 7 Recursio Itroductio to Data Structures 8.1 COSC 1P03 Recursio Recursio I Mathematics factorial Fiboacci umbers defie ifiite set with fiite defiitio I Computer Sciece sytax rules fiite defiitio,
More informationSorting 9/15/2009. Sorting Problem. Insertion Sort: Soundness. Insertion Sort. Insertion Sort: Running Time. Insertion Sort: Soundness
9/5/009 Algorithms Sortig 3- Sortig Sortig Problem The Sortig Problem Istace: A sequece of umbers Objective: A permutatio (reorderig) such that a ' K a' a, K,a a ', K, a' of the iput sequece The umbers
More informationECE4050 Data Structures and Algorithms. Lecture 6: Searching
ECE4050 Data Structures ad Algorithms Lecture 6: Searchig 1 Search Give: Distict keys k 1, k 2,, k ad collectio L of records of the form (k 1, I 1 ), (k 2, I 2 ),, (k, I ) where I j is the iformatio associated
More informationISSN (Print) Research Article. *Corresponding author Nengfa Hu
Scholars Joural of Egieerig ad Techology (SJET) Sch. J. Eg. Tech., 2016; 4(5):249-253 Scholars Academic ad Scietific Publisher (A Iteratioal Publisher for Academic ad Scietific Resources) www.saspublisher.com
More informationCSC165H1 Worksheet: Tutorial 8 Algorithm analysis (SOLUTIONS)
CSC165H1, Witer 018 Learig Objectives By the ed of this worksheet, you will: Aalyse the ruig time of fuctios cotaiig ested loops. 1. Nested loop variatios. Each of the followig fuctios takes as iput a
More informationRunning Time ( 3.1) Analysis of Algorithms. Experimental Studies. Limitations of Experiments
Ruig Time ( 3.1) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step- by- step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects.
More informationAnalysis of Algorithms
Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Ruig Time Most algorithms trasform iput objects ito output objects. The
More informationAn Estimation of Distribution Algorithm for solving the Knapsack problem
Vol.4,No.5, 214 Published olie: May 25, 214 DOI: 1.7321/jscse.v4.5.1 A Estimatio of Distributio Algorithm for solvig the Kapsack problem 1 Ricardo Pérez, 2 S. Jös, 3 Arturo Herádez, 4 Carlos A. Ochoa *1,
More informationAnalysis of Algorithms
Presetatio for use with the textbook, Algorithm Desig ad Applicatios, by M. T. Goodrich ad R. Tamassia, Wiley, 2015 Aalysis of Algorithms Iput 2015 Goodrich ad Tamassia Algorithm Aalysis of Algorithms
More informationGraphs. Minimum Spanning Trees. Slides by Rose Hoberman (CMU)
Graphs Miimum Spaig Trees Slides by Rose Hoberma (CMU) Problem: Layig Telephoe Wire Cetral office 2 Wirig: Naïve Approach Cetral office Expesive! 3 Wirig: Better Approach Cetral office Miimize the total
More informationEE University of Minnesota. Midterm Exam #1. Prof. Matthew O'Keefe TA: Eric Seppanen. Department of Electrical and Computer Engineering
EE 4363 1 Uiversity of Miesota Midterm Exam #1 Prof. Matthew O'Keefe TA: Eric Seppae Departmet of Electrical ad Computer Egieerig Uiversity of Miesota Twi Cities Campus EE 4363 Itroductio to Microprocessors
More informationAnalysis of Algorithms
Aalysis of Algorithms Ruig Time of a algorithm Ruig Time Upper Bouds Lower Bouds Examples Mathematical facts Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite
More information1 Graph Sparsfication
CME 305: Discrete Mathematics ad Algorithms 1 Graph Sparsficatio I this sectio we discuss the approximatio of a graph G(V, E) by a sparse graph H(V, F ) o the same vertex set. I particular, we cosider
More informationDATA SHEET AND USER GUIDE
V L S I I P O W N E D B Y A V I R A L M I T T A L Mp3HufDec DATA SHEET AND USER GUIDE Mp3HufDec ISO 11172-3 LAYER III HUFFMAN DECODER. ISO 11172-3 Layer III or commoly called as Mp3, employs Huffma ecodi
More informationData Structures and Algorithms. Analysis of Algorithms
Data Structures ad Algorithms Aalysis of Algorithms Outlie Ruig time Pseudo-code Big-oh otatio Big-theta otatio Big-omega otatio Asymptotic algorithm aalysis Aalysis of Algorithms Iput Algorithm Output
More informationDesigning a learning system
CS 75 Itro to Machie Learig Lecture Desigig a learig system Milos Hauskrecht milos@pitt.edu 539 Seott Square, -5 people.cs.pitt.edu/~milos/courses/cs75/ Admiistrivia No homework assigmet this week Please
More informationNumerical Methods Lecture 6 - Curve Fitting Techniques
Numerical Methods Lecture 6 - Curve Fittig Techiques Topics motivatio iterpolatio liear regressio higher order polyomial form expoetial form Curve fittig - motivatio For root fidig, we used a give fuctio
More informationA Note on Least-norm Solution of Global WireWarping
A Note o Least-orm Solutio of Global WireWarpig Charlie C. L. Wag Departmet of Mechaical ad Automatio Egieerig The Chiese Uiversity of Hog Kog Shati, N.T., Hog Kog E-mail: cwag@mae.cuhk.edu.hk Abstract
More informationChapter 1. Introduction to Computers and C++ Programming. Copyright 2015 Pearson Education, Ltd.. All rights reserved.
Chapter 1 Itroductio to Computers ad C++ Programmig Copyright 2015 Pearso Educatio, Ltd.. All rights reserved. Overview 1.1 Computer Systems 1.2 Programmig ad Problem Solvig 1.3 Itroductio to C++ 1.4 Testig
More informationAbstract. Chapter 4 Computation. Overview 8/13/18. Bjarne Stroustrup Note:
Chapter 4 Computatio Bjare Stroustrup www.stroustrup.com/programmig Abstract Today, I ll preset the basics of computatio. I particular, we ll discuss expressios, how to iterate over a series of values
More informationOverview. Chapter 18 Vectors and Arrays. Reminder. vector. Bjarne Stroustrup
Chapter 18 Vectors ad Arrays Bjare Stroustrup Vector revisited How are they implemeted? Poiters ad free store Destructors Iitializatio Copy ad move Arrays Array ad poiter problems Chagig size Templates
More informationImproving Template Based Spike Detection
Improvig Template Based Spike Detectio Kirk Smith, Member - IEEE Portlad State Uiversity petra@ee.pdx.edu Abstract Template matchig algorithms like SSE, Covolutio ad Maximum Likelihood are well kow for
More informationBST Sequence of Operations
Splay Trees Problems with BSTs Because the shape of a BST is determied by the order that data is iserted, we ru the risk of trees that are essetially lists 12 21 20 32 24 37 15 40 55 56 77 2 BST Sequece
More informationEnd Semester Examination CSE, III Yr. (I Sem), 30002: Computer Organization
Ed Semester Examiatio 2013-14 CSE, III Yr. (I Sem), 30002: Computer Orgaizatio Istructios: GROUP -A 1. Write the questio paper group (A, B, C, D), o frot page top of aswer book, as per what is metioed
More informationWavelet Transform. CSE 490 G Introduction to Data Compression Winter Wavelet Transformed Barbara (Enhanced) Wavelet Transformed Barbara (Actual)
Wavelet Trasform CSE 49 G Itroductio to Data Compressio Witer 6 Wavelet Trasform Codig PACW Wavelet Trasform A family of atios that filters the data ito low resolutio data plus detail data high pass filter
More informationAnalysis Metrics. Intro to Algorithm Analysis. Slides. 12. Alg Analysis. 12. Alg Analysis
Itro to Algorithm Aalysis Aalysis Metrics Slides. Table of Cotets. Aalysis Metrics 3. Exact Aalysis Rules 4. Simple Summatio 5. Summatio Formulas 6. Order of Magitude 7. Big-O otatio 8. Big-O Theorems
More informationLecture 2: MC68000 interrupt in C language
Lecture 2: MC68000 iterrupt i C lauae Problem defiitio Vectored iterrupts The vector table Itroductio to the PI/T Basic PI/T operatio C lauae solutio Desi of Computi Systems Ricardo Gutierrez-Osua Wriht
More informationDATA STRUCTURES. amortized analysis binomial heaps Fibonacci heaps union-find. Data structures. Appetizer. Appetizer
Data structures DATA STRUCTURES Static problems. Give a iput, produce a output. Ex. Sortig, FFT, edit distace, shortest paths, MST, max-flow,... amortized aalysis biomial heaps Fiboacci heaps uio-fid Dyamic
More informationCS 683: Advanced Design and Analysis of Algorithms
CS 683: Advaced Desig ad Aalysis of Algorithms Lecture 6, February 1, 2008 Lecturer: Joh Hopcroft Scribes: Shaomei Wu, Etha Feldma February 7, 2008 1 Threshold for k CNF Satisfiability I the previous lecture,
More informationLoad balanced Parallel Prime Number Generator with Sieve of Eratosthenes on Cluster Computers *
Load balaced Parallel Prime umber Geerator with Sieve of Eratosthees o luster omputers * Soowook Hwag*, Kyusik hug**, ad Dogseug Kim* *Departmet of Electrical Egieerig Korea Uiversity Seoul, -, Rep. of
More informationThe golden search method: Question 1
1. Golde Sectio Search for the Mode of a Fuctio The golde search method: Questio 1 Suppose the last pair of poits at which we have a fuctio evaluatio is x(), y(). The accordig to the method, If f(x())
More informationExact Minimum Lower Bound Algorithm for Traveling Salesman Problem
Exact Miimum Lower Boud Algorithm for Travelig Salesma Problem Mohamed Eleiche GeoTiba Systems mohamed.eleiche@gmail.com Abstract The miimum-travel-cost algorithm is a dyamic programmig algorithm to compute
More informationOptimal Mapped Mesh on the Circle
Koferece ANSYS 009 Optimal Mapped Mesh o the Circle doc. Ig. Jaroslav Štigler, Ph.D. Bro Uiversity of Techology, aculty of Mechaical gieerig, ergy Istitut, Abstract: This paper brigs out some ideas ad
More informationVariance as a Stopping Criterion for Genetic Algorithms with Elitist Model
Fudameta Iformaticae 120 (2012) 145 164 145 DOI 10.3233/FI-2012-754 IOS Press Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model Diabadhu Bhadari, C. A. Murthy, Sakar K. Pal Ceter for
More informationA Parallel DFA Minimization Algorithm
A Parallel DFA Miimizatio Algorithm Ambuj Tewari, Utkarsh Srivastava, ad P. Gupta Departmet of Computer Sciece & Egieerig Idia Istitute of Techology Kapur Kapur 208 016,INDIA pg@iitk.ac.i Abstract. I this
More informationMassachusetts Institute of Technology Lecture : Theory of Parallel Systems Feb. 25, Lecture 6: List contraction, tree contraction, and
Massachusetts Istitute of Techology Lecture.89: Theory of Parallel Systems Feb. 5, 997 Professor Charles E. Leiserso Scribe: Guag-Ie Cheg Lecture : List cotractio, tree cotractio, ad symmetry breakig Work-eciet
More informationProject 2.5 Improved Euler Implementation
Project 2.5 Improved Euler Implemetatio Figure 2.5.10 i the text lists TI-85 ad BASIC programs implemetig the improved Euler method to approximate the solutio of the iitial value problem dy dx = x+ y,
More informationChapter 5. Functions for All Subtasks. Copyright 2015 Pearson Education, Ltd.. All rights reserved.
Chapter 5 Fuctios for All Subtasks Copyright 2015 Pearso Educatio, Ltd.. All rights reserved. Overview 5.1 void Fuctios 5.2 Call-By-Referece Parameters 5.3 Usig Procedural Abstractio 5.4 Testig ad Debuggig
More informationBig-O Analysis. Asymptotics
Big-O Aalysis 1 Defiitio: Suppose that f() ad g() are oegative fuctios of. The we say that f() is O(g()) provided that there are costats C > 0 ad N > 0 such that for all > N, f() Cg(). Big-O expresses
More informationThe Magma Database file formats
The Magma Database file formats Adrew Gaylard, Bret Pikey, ad Mart-Mari Breedt Johaesburg, South Africa 15th May 2006 1 Summary Magma is a ope-source object database created by Chris Muller, of Kasas City,
More informationFundamentals of Media Processing. Shin'ichi Satoh Kazuya Kodama Hiroshi Mo Duy-Dinh Le
Fudametals of Media Processig Shi'ichi Satoh Kazuya Kodama Hiroshi Mo Duy-Dih Le Today's topics Noparametric Methods Parze Widow k-nearest Neighbor Estimatio Clusterig Techiques k-meas Agglomerative Hierarchical
More informationImage Segmentation EEE 508
Image Segmetatio Objective: to determie (etract) object boudaries. It is a process of partitioig a image ito distict regios by groupig together eighborig piels based o some predefied similarity criterio.
More information5.3 Recursive definitions and structural induction
/8/05 5.3 Recursive defiitios ad structural iductio CSE03 Discrete Computatioal Structures Lecture 6 A recursively defied picture Recursive defiitios e sequece of powers of is give by a = for =0,,, Ca
More informationCopyright 2016 Ramez Elmasri and Shamkant B. Navathe
Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe CHAPTER 20 Itroductio to Trasactio Processig Cocepts ad Theory Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Itroductio Trasactio Describes local
More information1.2 Binomial Coefficients and Subsets
1.2. BINOMIAL COEFFICIENTS AND SUBSETS 13 1.2 Biomial Coefficiets ad Subsets 1.2-1 The loop below is part of a program to determie the umber of triagles formed by poits i the plae. for i =1 to for j =
More informationChapter 11. Friends, Overloaded Operators, and Arrays in Classes. Copyright 2014 Pearson Addison-Wesley. All rights reserved.
Chapter 11 Frieds, Overloaded Operators, ad Arrays i Classes Copyright 2014 Pearso Addiso-Wesley. All rights reserved. Overview 11.1 Fried Fuctios 11.2 Overloadig Operators 11.3 Arrays ad Classes 11.4
More informationNTH, GEOMETRIC, AND TELESCOPING TEST
NTH, GEOMETRIC, AND TELESCOPING TEST Sectio 9. Calculus BC AP/Dual, Revised 08 viet.dag@humbleisd.et /4/08 0:0 PM 9.: th, Geometric, ad Telescopig Test SUMMARY OF TESTS FOR SERIES Lookig at the first few
More information