Combinatorial Search; Monte Carlo Methods
|
|
|
- Abel Quinn
- 5 years ago
- Views:
Transcription
1 Combinatorial Search; Monte Carlo Methods Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico May 02, 2016 CPD (DEI / IST) Parallel and Distributed Computing / 31
2 Outline Parallel Search Backtrack Search Branch and Bound Monte Carlo methods CPD (DEI / IST) Parallel and Distributed Computing / 31
3 Combinatorial Search Combinatorial search algorithm: find one or more optimal or suboptimal solutions in a defined problem space Decision problems: determine if there is a solution to a given set of constraints Satisfiability (SAT) Hamiltonian path Eulerian path Optimization problems: find a solution that maximizes or minimizes a cost function under a set of constraints Shortest path Traveling salesman (TSP) Maximum satisfiability (MAX-SAT) CPD (DEI / IST) Parallel and Distributed Computing / 31
4 Combinatorial Problems Easy combinatorial problems have known algorithms that run in polynomial-time. Shortest path Eulerian path CPD (DEI / IST) Parallel and Distributed Computing / 31
5 Combinatorial Problems Easy combinatorial problems have known algorithms that run in polynomial-time. Shortest path Eulerian path The only known solutions to hard problems run in exponential-time. Satisfiability (SAT) Hamiltonian path Traveling salesman (TSP) Maximum satisfiability (MAX-SAT) CPD (DEI / IST) Parallel and Distributed Computing / 31
6 Backtrack Search Backtrack search uses a depth-first search (typically recursive) to consider alternative solutions to a combinatorial search problem. Backtrack occurs when: a node has no children all of a node s children have been explored current solution does not verify constraints CPD (DEI / IST) Parallel and Distributed Computing / 31
7 Backtrack Search Choice of 1st variable Choice of 2nd variable Choice of 3rd variable CPD (DEI / IST) Parallel and Distributed Computing / 31
8 Backtrack Search Choice of 1st variable Choice of 2nd variable Choice of 3rd variable If b is the branching factor, at level k of the tree we have b k nodes. up to level k of the tree we have bk+1 b b nodes. CPD (DEI / IST) Parallel and Distributed Computing / 31
9 Parallel Backtrack Search First strategy: suppose p = b k and give each processor a subtree. each process searches all nodes to depth k but then explores only one of subtrees rooted at level k Typically, k = log b p is much smaller than the depth of the tree. Time required by each process to traverse first k levels of state space tree inconsequential. CPD (DEI / IST) Parallel and Distributed Computing / 31
10 Parallel Backtrack Search CPD (DEI / IST) Parallel and Distributed Computing / 31
11 Parallel Backtrack Search What if p b k? define a level m up to which all processes execute redundant search in parallel. at level m, assign a subset of the b m subtrees to the p processes CPD (DEI / IST) Parallel and Distributed Computing / 31
12 Parallel Backtrack Search What if p b k? define a level m up to which all processes execute redundant search in parallel. at level m, assign a subset of the b m subtrees to the p processes What value of m to use? CPD (DEI / IST) Parallel and Distributed Computing / 31
13 Parallel Backtrack Search What if p b k? define a level m up to which all processes execute redundant search in parallel. at level m, assign a subset of the b m subtrees to the p processes What value of m to use? The larger the value of m the larger the redundant computation but better load balancing CPD (DEI / IST) Parallel and Distributed Computing / 31
14 Parallel Backtrack Search Example: p = 5 processors exploring a state space tree with branching factor b = 3 and maximum depth 10. CPD (DEI / IST) Parallel and Distributed Computing / 31
15 Branch and Bound Branch and Bound, BB, is a general algorithm for optimization problems that also performs a depth-first search, but is able to prune its search by using upper and lower estimated bounds of the cost function. Example: find the minimum number of moves to solve the 8-puzzle. CPD (DEI / IST) Parallel and Distributed Computing / 31
16 Branch and Bound CPD (DEI / IST) Parallel and Distributed Computing / 31
17 Branch and Bound could solve puzzle by pursuing breadth-first search of state space tree we want to examine as few nodes as possible can speed search if we associate with each node an estimate of minimum number of tile moves needed to solve the puzzle, given moves made so far CPD (DEI / IST) Parallel and Distributed Computing / 31
18 Branch and Bound could solve puzzle by pursuing breadth-first search of state space tree we want to examine as few nodes as possible can speed search if we associate with each node an estimate of minimum number of tile moves needed to solve the puzzle, given moves made so far Compute Manhattan distance of each tile to each final position overall sum is a lower bound on number of moves still required CPD (DEI / IST) Parallel and Distributed Computing / 31
19 Branch and Bound could solve puzzle by pursuing breadth-first search of state space tree we want to examine as few nodes as possible can speed search if we associate with each node an estimate of minimum number of tile moves needed to solve the puzzle, given moves made so far Compute Manhattan distance of each tile to each final position overall sum is a lower bound on number of moves still required If current depth + lower bound higher than an existing solution, discard subtree. CPD (DEI / IST) Parallel and Distributed Computing / 31
20 Branch and Bound CPD (DEI / IST) Parallel and Distributed Computing / 31
21 Branch and Bound CPD (DEI / IST) Parallel and Distributed Computing / 31
22 Branch and Bound CPD (DEI / IST) Parallel and Distributed Computing / 31
23 Branch and Bound CPD (DEI / IST) Parallel and Distributed Computing / 31
24 Branch and Bound CPD (DEI / IST) Parallel and Distributed Computing / 31
25 Branch and Bound CPD (DEI / IST) Parallel and Distributed Computing / 31
26 Branch and Bound CPD (DEI / IST) Parallel and Distributed Computing / 31
27 Parallel Branch and Bound Sequential algorithm uses a priority queue. Single priority queue too expensive to maintain in a distributed parallel system. Multiple Priority Queues each process maintains separate priority queue of unexamined subproblems each process retrieves subproblem with smallest lower bound to continue search occasionally processes send unexamined subproblems to other processes CPD (DEI / IST) Parallel and Distributed Computing / 31
28 Parallel Branch and Bound sequential algorithm searches minimum number of nodes (never explores nodes with lower bounds greater than cost of optimal solution) parallel algorithm may examine unnecessary nodes because each process searching locally best nodes exchanging subproblems promotes distribute of subproblems with good lower bounds, reducing amount of wasted work increases communication overhead Conditions for solution found be guaranteed optimal: all nodes in state space tree with smaller lower bounds must be explored CPD (DEI / IST) Parallel and Distributed Computing / 31
29 Monte Carlo Method Solve a problem using statistical sampling. CPD (DEI / IST) Parallel and Distributed Computing / 31
30 Monte Carlo Method Solve a problem using statistical sampling. Applications of the Monte Carlo method: integrals of arbitrary functions of 6+ dimensions predicting future values of stocks solving partial differential equations sharpening satellite images modeling cell populations finding approximate solutions to NP-hard problems CPD (DEI / IST) Parallel and Distributed Computing / 31
31 Monte Carlo Method Solve a problem using statistical sampling. Applications of the Monte Carlo method: integrals of arbitrary functions of 6+ dimensions predicting future values of stocks solving partial differential equations sharpening satellite images modeling cell populations finding approximate solutions to NP-hard problems It is estimated that over half the total cycles in parallel systems is spent with Monte Carlo methods! CPD (DEI / IST) Parallel and Distributed Computing / 31
32 Monte Carlo Method 1 Define a domain of possible inputs. 2 Generate inputs randomly from a probability distribution over the domain. 3 Perform a deterministic computation on the inputs. 4 Aggregate the results. CPD (DEI / IST) Parallel and Distributed Computing / 31
33 Example of Monte Carlo Method Computation of π: CPD (DEI / IST) Parallel and Distributed Computing / 31
34 Example of Monte Carlo Method Computation of π: CPD (DEI / IST) Parallel and Distributed Computing / 31
35 Example of Monte Carlo Method Computation of π: CPD (DEI / IST) Parallel and Distributed Computing / 31
36 Example of Monte Carlo Method Computation of π: CPD (DEI / IST) Parallel and Distributed Computing / 31
37 Example of Monte Carlo Method Computation of π: CPD (DEI / IST) Parallel and Distributed Computing / 31
38 Example of Monte Carlo Method Computation of π: CPD (DEI / IST) Parallel and Distributed Computing / 31
39 Example of Monte Carlo Method Computation of π: Area Area = πd2 /4 D 2 = π = π 4 π =3, 2 CPD (DEI / IST) Parallel and Distributed Computing / 31
40 Sample Size Defines the Error n Estimate Error 1/(2 n) 10 3, , , , , , , , , , , , , , , , , , , , , , , , , , ,00002 CPD (DEI / IST) Parallel and Distributed Computing / 31
41 Why Monte Carlo Works Mean Value Theorem b a f (x)dx = (b a)f Monte Carlo methods obtain an estimate for f : f 1 n n f (x i ) i=1 Accuracy increases at a rate of 1/ n. CPD (DEI / IST) Parallel and Distributed Computing / 31
42 Why Monte Carlo is Effective error in Monte Carlo estimate decreases by the factor 1/ n rate of convergence independent of integrand s dimension deterministic numerical integration methods do not share this property hence Monte Carlo superior when integrand has 6 or more dimensions CPD (DEI / IST) Parallel and Distributed Computing / 31
43 Why Monte Carlo is Effective error in Monte Carlo estimate decreases by the factor 1/ n rate of convergence independent of integrand s dimension deterministic numerical integration methods do not share this property hence Monte Carlo superior when integrand has 6 or more dimensions Monte Carlo methods are typically trivially amenable for parallelization. Two possible views: obtain an estimate p times faster decrease error by p CPD (DEI / IST) Parallel and Distributed Computing / 31
44 Another Example Statistical Timing Analysis (STA) Obtain the arrival times in the nodes of a circuit given that the delays are subject to variation. Types of variations: Manufacturing process variations no two chips are equal the fabrication of a chip is a sequence of hundreds of operations, each of which will be performed a little differently on each chip one of the main sources of variation in chip performances Environmental or operating conditions e.g. changes in power supply voltage or operating temperature Device fatigue phenomena e.g. electromigration, hot electron effects and NBTI (negative bias temperature instability) CPD (DEI / IST) Parallel and Distributed Computing / 31
45 Statistical Timing Analysis (STA) CPD (DEI / IST) Parallel and Distributed Computing / 31
46 STA: Arrival Times Accuracy Since delay is not given by a deterministic value, deterministic propagation of arrival times can not be done If we assume the average and worst-case values for the delays of each individual node we can do a worst-case analysis but not accurate enough Or we can assume that delay variation follows a probability distribution which is independent for each node better modeling of real conditions much harder to propagate distributions CPD (DEI / IST) Parallel and Distributed Computing / 31
47 STA: Arrival Times Accuracy Since delay is not given by a deterministic value, deterministic propagation of arrival times can not be done If we assume the average and worst-case values for the delays of each individual node we can do a worst-case analysis but not accurate enough Or we can assume that delay variation follows a probability distribution which is independent for each node better modeling of real conditions much harder to propagate distributions Monte-Carlo comes to the rescue! CPD (DEI / IST) Parallel and Distributed Computing / 31
48 STA using Monte-Carlo Repeat N times: 1 Randomly generate a delay value for each nodes according to the probabilistic distribution at each node 2 Do a deterministic propagation of the arrival times 3 Store computed arrival times CPD (DEI / IST) Parallel and Distributed Computing / 31
49 STA using Monte-Carlo Repeat N times: 1 Randomly generate a delay value for each nodes according to the probabilistic distribution at each node 2 Do a deterministic propagation of the arrival times 3 Store computed arrival times Statistical distribution is obtained: Sample mean gives us the average arrival times Standard deviation gives us a confidence interval 0.25 hist t race N x23.txt gauss(x, , ) CPD (DEI / IST) Parallel and Distributed Computing / 31
50 Random Number Generators Computers are deterministic machines: pseudo-random numbers Desirable properties of a Random Number Generator: uniformly distributed uncorrelated never cycles satisfies any statistical test for randomness reproducible machine-independent changing seed value changes sequence easily split into independent subsequences fast limited memory requirements CPD (DEI / IST) Parallel and Distributed Computing / 31
51 Random Number Generators Computers are deterministic machines: pseudo-random numbers Desirable properties of a Random Number Generator: uniformly distributed uncorrelated never cycles satisfies any statistical test for randomness reproducible machine-independent changing seed value changes sequence easily split into independent subsequences fast limited memory requirements Ideal Random Number Generators are not possible! finite number of states cycles reproducible correlation CPD (DEI / IST) Parallel and Distributed Computing / 31
52 Random Number Generators Linear Congruential Random Number Generators X i = (ax i 1 + c) Sequence depends on the seed X 0. mod M CPD (DEI / IST) Parallel and Distributed Computing / 31
53 Random Number Generators Linear Congruential Random Number Generators X i = (ax i 1 + c) Sequence depends on the seed X 0. mod M length of random sequence at most 2 M least significant bits exhibit high degree of correlation in k-dimensional space, all points fall mainly in the hyperplanes CPD (DEI / IST) Parallel and Distributed Computing / 31
54 Random Number Generators Linear Congruential Random Number Generators X i = (ax i 1 + c) Sequence depends on the seed X 0. mod M length of random sequence at most 2 M least significant bits exhibit high degree of correlation in k-dimensional space, all points fall mainly in the hyperplanes Lagged Fibonacci Random Number Generators X i = X i p X i q where is a binary operation, ie, exclusive-or, addition modulo M, etc. requires storing the last max(p, q) seed values careful selection of seed values, p, and q can result in very long periods and good randomness CPD (DEI / IST) Parallel and Distributed Computing / 31
55 Parallel Random Number Generators Centralized: master-slave approach CPD (DEI / IST) Parallel and Distributed Computing / 31
56 Parallel Random Number Generators Centralized: master-slave approach Decentralized: Leapfrog interleave random samples by p sequence splitting blocks of random samples of size p independent sequences CPD (DEI / IST) Parallel and Distributed Computing / 31
57 Parallel Random Number Generators Centralized: master-slave approach Decentralized: Leapfrog interleave random samples by p sequence splitting blocks of random samples of size p independent sequences Independent sequences tend to work well as long as each task uses different seeds. CPD (DEI / IST) Parallel and Distributed Computing / 31
58 Review Parallel Search Backtrack Search Branch and Bound Monte Carlo methods CPD (DEI / IST) Parallel and Distributed Computing / 31
59 Next Class Graph algorithms CPD (DEI / IST) Parallel and Distributed Computing / 31
Monte Carlo Methods; Combinatorial Search
Monte Carlo Methods; Combinatorial Search Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico November 22, 2012 CPD (DEI / IST) Parallel and
Parallel Programming. Parallel algorithms Combinatorial Search
Parallel Programming Parallel algorithms Combinatorial Search Some Combinatorial Search Methods Divide and conquer Backtrack search Branch and bound Game tree search (minimax, alpha-beta) 2010@FEUP Parallel
Search and Optimization
Search and Optimization Search, Optimization and Game-Playing The goal is to find one or more optimal or sub-optimal solutions in a given search space. We can either be interested in finding any one solution
Computational Methods. Randomness and Monte Carlo Methods
Computational Methods Randomness and Monte Carlo Methods Manfred Huber 2010 1 Randomness and Monte Carlo Methods Introducing randomness in an algorithm can lead to improved efficiencies Random sampling
Foster s Methodology: Application Examples
Foster s Methodology: Application Examples Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico October 19, 2011 CPD (DEI / IST) Parallel and
6. Algorithm Design Techniques
6. Algorithm Design Techniques 6. Algorithm Design Techniques 6.1 Greedy algorithms 6.2 Divide and conquer 6.3 Dynamic Programming 6.4 Randomized Algorithms 6.5 Backtracking Algorithms Malek Mouhoub, CS340
LECTURE 3 ALGORITHM DESIGN PARADIGMS
LECTURE 3 ALGORITHM DESIGN PARADIGMS Introduction Algorithm Design Paradigms: General approaches to the construction of efficient solutions to problems. Such methods are of interest because: They provide
Algorithm Design Paradigms
CmSc250 Intro to Algorithms Algorithm Design Paradigms Algorithm Design Paradigms: General approaches to the construction of efficient solutions to problems. Such methods are of interest because: They
Algorithms for Hard Problems
Algorithms for Hard Problems Monte Carlo and Markov Chain Monte Carlo methods, backtracking, branch & bound and alpha-beta pruning. Why parallel algorithms? Monte Carlo and Markov chain Monte Carlo methods
CLASS: II YEAR / IV SEMESTER CSE CS 6402-DESIGN AND ANALYSIS OF ALGORITHM UNIT I INTRODUCTION
CLASS: II YEAR / IV SEMESTER CSE CS 6402-DESIGN AND ANALYSIS OF ALGORITHM UNIT I INTRODUCTION 1. What is performance measurement? 2. What is an algorithm? 3. How the algorithm is good? 4. What are the
CAD Algorithms. Categorizing Algorithms
CAD Algorithms Categorizing Algorithms Mohammad Tehranipoor ECE Department 2 September 2008 1 Categorizing Algorithms Greedy Algorithms Prim s Algorithm (Minimum Spanning Tree) A subgraph that is a tree
Data Structures (CS 1520) Lecture 28 Name:
Traeling Salesperson Problem (TSP) -- Find an optimal (ie, minimum length) when at least one exists A (or Hamiltonian circuit) is a path from a ertex back to itself that passes through each of the other
Lecture 5: Search Algorithms for Discrete Optimization Problems
Lecture 5: Search Algorithms for Discrete Optimization Problems Definitions Discrete optimization problem (DOP): tuple (S, f), S finite set of feasible solutions, f : S R, cost function. Objective: find
Thus, it is reasonable to compare binary search trees and binary heaps as is shown in Table 1.
7.2 Binary Min-Heaps A heap is a tree-based structure, but it doesn t use the binary-search differentiation between the left and right sub-trees to create a linear ordering. Instead, a binary heap only
Chapter 9 Graph Algorithms
Introduction graph theory useful in practice represent many real-life problems can be if not careful with data structures Chapter 9 Graph s 2 Definitions Definitions an undirected graph is a finite set
Data Structures (CS 1520) Lecture 28 Name:
Traeling Salesperson Problem (TSP) -- Find an optimal (ie, minimum length) when at least one exists A (or Hamiltonian circuit) is a path from a ertex back to itself that passes through each of the other
Exact Algorithms for NP-hard problems
24 mai 2012 1 Why do we need exponential algorithms? 2 3 Why the P-border? 1 Practical reasons (Jack Edmonds, 1965) For practical purposes the difference between algebraic and exponential order is more
B553 Lecture 12: Global Optimization
B553 Lecture 12: Global Optimization Kris Hauser February 20, 2012 Most of the techniques we have examined in prior lectures only deal with local optimization, so that we can only guarantee convergence
Chapter 9 Graph Algorithms
Chapter 9 Graph Algorithms 2 Introduction graph theory useful in practice represent many real-life problems can be if not careful with data structures 3 Definitions an undirected graph G = (V, E) is a
11/22/2016. Chapter 9 Graph Algorithms. Introduction. Definitions. Definitions. Definitions. Definitions
Introduction Chapter 9 Graph Algorithms graph theory useful in practice represent many real-life problems can be slow if not careful with data structures 2 Definitions an undirected graph G = (V, E) is
Chapter 9 Graph Algorithms
Chapter 9 Graph Algorithms 2 Introduction graph theory useful in practice represent many real-life problems can be slow if not careful with data structures 3 Definitions an undirected graph G = (V, E)
Chapter 3: Solving Problems by Searching
Chapter 3: Solving Problems by Searching Prepared by: Dr. Ziad Kobti 1 Problem-Solving Agent Reflex agent -> base its actions on a direct mapping from states to actions. Cannot operate well in large environments
D.K.M.COLLEGE FOR WOMEN (AUTONOMOUS), VELLORE-1.
D.K.M.COLLEGE FOR WOMEN (AUTONOMOUS), VELLORE-1. DESIGN AND ANALYSIS OF ALGORITHM UNIT- I SECTION-A 2 MARKS 1. Define an algorithm? 2. Specify the criteria of algorithm? 3. What is Computational Procedure?
CSCE 321/3201 Analysis and Design of Algorithms. Prof. Amr Goneid. Fall 2016
CSCE 321/3201 Analysis and Design of Algorithms Prof. Amr Goneid Fall 2016 CSCE 321/3201 Analysis and Design of Algorithms Prof. Amr Goneid Course Resources Instructor: Prof. Amr Goneid E-mail: goneid@aucegypt.edu
CS510 \ Lecture Ariel Stolerman
CS510 \ Lecture02 2012-10-03 1 Ariel Stolerman Midterm Evan will email about that after the lecture, at least 2 lectures from now. The exam will be given in a regular PDF (not an online form). We will
6.856 Randomized Algorithms
6.856 Randomized Algorithms David Karger Handout #4, September 21, 2002 Homework 1 Solutions Problem 1 MR 1.8. (a) The min-cut algorithm given in class works because at each step it is very unlikely (probability
You ve already read basics of simulation now I will be taking up method of simulation, that is Random Number Generation
Unit 5 SIMULATION THEORY Lesson 39 Learning objective: To learn random number generation. Methods of simulation. Monte Carlo method of simulation You ve already read basics of simulation now I will be
Seach algorithms The travelling salesman problem The Towers of Hanoi Playing games. Comp24412: Symbolic AI. Lecture 4: Search. Ian Pratt-Hartmann
Comp24412: Symbolic AI Lecture 4: Search Ian Pratt-Hartmann Room KB2.38: email: ipratt@cs.man.ac.uk 2016 17 Outline Seach algorithms The travelling salesman problem The Towers of Hanoi Playing games Typical
Parallel Combinatorial Search on Computer Cluster: Sam Loyd s Puzzle
Parallel Combinatorial Search on Computer Cluster: Sam Loyd s Puzzle Plamenka Borovska Abstract: The paper investigates the efficiency of parallel branch-and-bound search on multicomputer cluster for the
4.1.2 Merge Sort Sorting Lower Bound Counting Sort Sorting in Practice Solving Problems by Sorting...
Contents 1 Introduction... 1 1.1 What is Competitive Programming?... 1 1.1.1 Programming Contests.... 2 1.1.2 Tips for Practicing.... 3 1.2 About This Book... 3 1.3 CSES Problem Set... 5 1.4 Other Resources...
DESIGN AND ANALYSIS OF ALGORITHMS
DESIGN AND ANALYSIS OF ALGORITHMS QUESTION BANK Module 1 OBJECTIVE: Algorithms play the central role in both the science and the practice of computing. There are compelling reasons to study algorithms.
CS 771 Artificial Intelligence. Informed Search
CS 771 Artificial Intelligence Informed Search Outline Review limitations of uninformed search methods Informed (or heuristic) search Uses problem-specific heuristics to improve efficiency Best-first,
Stochastic Algorithms
Stochastic Algorithms Some of the fastest known algorithms for certain tasks rely on chance Stochastic/Randomized Algorithms Two common variations Monte Carlo Las Vegas We have already encountered some
CSE 417 Branch & Bound (pt 4) Branch & Bound
CSE 417 Branch & Bound (pt 4) Branch & Bound Reminders > HW8 due today > HW9 will be posted tomorrow start early program will be slow, so debugging will be slow... Review of previous lectures > Complexity
Search Algorithms. IE 496 Lecture 17
Search Algorithms IE 496 Lecture 17 Reading for This Lecture Primary Horowitz and Sahni, Chapter 8 Basic Search Algorithms Search Algorithms Search algorithms are fundamental techniques applied to solve
Reference Sheet for CO142.2 Discrete Mathematics II
Reference Sheet for CO14. Discrete Mathematics II Spring 017 1 Graphs Defintions 1. Graph: set of N nodes and A arcs such that each a A is associated with an unordered pair of nodes.. Simple graph: no
Department of Computer Applications. MCA 312: Design and Analysis of Algorithms. [Part I : Medium Answer Type Questions] UNIT I
![Department of Computer Applications. MCA 312: Design and Analysis of Algorithms. [Part I : Medium Answer Type Questions] UNIT I](/thumbs/94/122075835.jpg "Department of Computer Applications. MCA 312: Design and Analysis of Algorithms. [Part I : Medium Answer Type Questions] UNIT I")
MCA 312: Design and Analysis of Algorithms [Part I : Medium Answer Type Questions] UNIT I 1) What is an Algorithm? What is the need to study Algorithms? 2) Define: a) Time Efficiency b) Space Efficiency
Contents. 1 Introduction. 2 Searching and Traversal Techniques. Preface... (vii) Acknowledgements... (ix)
Contents Preface... (vii) Acknowledgements... (ix) 1 Introduction 1.1 Algorithm 1 1.2 Life Cycle of Design and Analysis of Algorithm 2 1.3 Pseudo-Code for Expressing Algorithms 5 1.4 Recursive Algorithms
Artificial Intelligence
Artificial Intelligence Search Marc Toussaint University of Stuttgart Winter 2015/16 (slides based on Stuart Russell s AI course) Outline Problem formulation & examples Basic search algorithms 2/100 Example:
Algorithm classification
Types of Algorithms Algorithm classification Algorithms that use a similar problem-solving approach can be grouped together We ll talk about a classification scheme for algorithms This classification scheme
Algorithm Design Techniques. Hwansoo Han
Algorithm Design Techniques Hwansoo Han Algorithm Design General techniques to yield effective algorithms Divide-and-Conquer Dynamic programming Greedy techniques Backtracking Local search 2 Divide-and-Conquer
Chapter 11: Indexing and Hashing
Chapter 11: Indexing and Hashing Basic Concepts Ordered Indices B + -Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL
Assignment No 2 (Group B)
Assignment No 2 (Group B) 1 Problem Statement : Concurrent Implementation of Travelling Salesman Problem. 2 Objective : To develop problem solving abilities using Mathematical Modeling. To apply algorithmic
ACO and other (meta)heuristics for CO
ACO and other (meta)heuristics for CO 32 33 Outline Notes on combinatorial optimization and algorithmic complexity Construction and modification metaheuristics: two complementary ways of searching a solution
CSC 373: Algorithm Design and Analysis Lecture 8
CSC 373: Algorithm Design and Analysis Lecture 8 Allan Borodin January 23, 2013 1 / 19 Lecture 8: Announcements and Outline Announcements No lecture (or tutorial) this Friday. Lecture and tutorials as
Theorem 2.9: nearest addition algorithm
There are severe limits on our ability to compute near-optimal tours It is NP-complete to decide whether a given undirected =(,)has a Hamiltonian cycle An approximation algorithm for the TSP can be used
N N Sudoku Solver. Sequential and Parallel Computing
N N Sudoku Solver Sequential and Parallel Computing Abdulaziz Aljohani Computer Science. Rochester Institute of Technology, RIT Rochester, United States aaa4020@rit.edu Abstract 'Sudoku' is a logic-based
Lecture 8: The Traveling Salesman Problem
Lecture 8: The Traveling Salesman Problem Let G = (V, E) be an undirected graph. A Hamiltonian cycle of G is a cycle that visits every vertex v V exactly once. Instead of Hamiltonian cycle, we sometimes
31.6 Powers of an element
31.6 Powers of an element Just as we often consider the multiples of a given element, modulo, we consider the sequence of powers of, modulo, where :,,,,. modulo Indexing from 0, the 0th value in this sequence
The Algorithm Design Manual
Steven S. Skiena The Algorithm Design Manual With 72 Figures Includes CD-ROM THE ELECTRONIC LIBRARY OF SCIENCE Contents Preface vii I TECHNIQUES 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 2 2.1 2.2 2.3
Methods and Models for Combinatorial Optimization Exact methods for the Traveling Salesman Problem
Methods and Models for Combinatorial Optimization Exact methods for the Traveling Salesman Problem L. De Giovanni M. Di Summa The Traveling Salesman Problem (TSP) is an optimization problem on a directed
Coping with the Limitations of Algorithm Power Exact Solution Strategies Backtracking Backtracking : A Scenario
Coping with the Limitations of Algorithm Power Tackling Difficult Combinatorial Problems There are two principal approaches to tackling difficult combinatorial problems (NP-hard problems): Use a strategy
Lecture 1. Introduction
Lecture 1 Introduction 1 Lecture Contents 1. What is an algorithm? 2. Fundamentals of Algorithmic Problem Solving 3. Important Problem Types 4. Fundamental Data Structures 2 1. What is an Algorithm? Algorithm
A PACKAGE FOR DEVELOPMENT OF ALGORITHMS FOR GLOBAL OPTIMIZATION 1
Mathematical Modelling and Analysis 2005. Pages 185 190 Proceedings of the 10 th International Conference MMA2005&CMAM2, Trakai c 2005 Technika ISBN 9986-05-924-0 A PACKAGE FOR DEVELOPMENT OF ALGORITHMS
Dynamic Programming Homework Problems
CS 1510 Dynamic Programming Homework Problems 1. (2 points) Consider the recurrence relation T (0) = T (1) = 2 and for n > 1 n 1 T (n) = T (i)t (i 1) i=1 We consider the problem of computing T (n) from
CS-6402 DESIGN AND ANALYSIS OF ALGORITHMS
CS-6402 DESIGN AND ANALYSIS OF ALGORITHMS 2 marks UNIT-I 1. Define Algorithm. An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time. 2.Write a short note
Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest. Introduction to Algorithms
Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Introduction to Algorithms Preface xiii 1 Introduction 1 1.1 Algorithms 1 1.2 Analyzing algorithms 6 1.3 Designing algorithms 1 1 1.4 Summary 1 6
Probabilistic (Randomized) algorithms
Probabilistic (Randomized) algorithms Idea: Build algorithms using a random element so as gain improved performance. For some cases, improved performance is very dramatic, moving from intractable to tractable.
CSE 100: B+ TREE, 2-3 TREE, NP- COMPLETNESS
CSE 100: B+ TREE, 2-3 TREE, NP- COMPLETNESS Analyzing find in B-trees Since the B-tree is perfectly height-balanced, the worst case time cost for find is O(logN) Best case: If every internal node is completely
Chapter 6 Random Number Generation
Chapter 6 Random Number Generation Requirements / application Pseudo-random bit generator Hardware and software solutions [NetSec/SysSec], WS 2007/2008 6.1 Requirements and Application Scenarios Security
Randomized Algorithms: Selection
Randomized Algorithms: Selection CSE21 Winter 2017, Day 25 (B00), Day 16 (A00) March 15, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Selection Problem: WHAT Given list of distinct integers a 1, a 2,,
Lecture 4: Search 3. Victor R. Lesser. CMPSCI 683 Fall 2010
Lecture 4: Search 3 Victor R. Lesser CMPSCI 683 Fall 2010 First Homework 1 st Programming Assignment 2 separate parts (homeworks) First part due on (9/27) at 5pm Second part due on 10/13 at 5pm Send homework
Lecture 14: March 9, 2015
324: tate-space, F, F, Uninformed earch. pring 2015 Lecture 14: March 9, 2015 Lecturer: K.R. howdhary : Professor of (VF) isclaimer: These notes have not been subjected to the usual scrutiny reserved for
Introduction to Combinatorial Algorithms
Fall 2009 Intro Introduction to the course What are : Combinatorial Structures? Combinatorial Algorithms? Combinatorial Problems? Combinatorial Structures Combinatorial Structures Combinatorial structures
Outline. Graphs. Divide and Conquer.
GRAPHS COMP 321 McGill University These slides are mainly compiled from the following resources. - Professor Jaehyun Park slides CS 97SI - Top-coder tutorials. - Programming Challenges books. Outline Graphs.
Backtracking. Chapter 5
1 Backtracking Chapter 5 2 Objectives Describe the backtrack programming technique Determine when the backtracking technique is an appropriate approach to solving a problem Define a state space tree for
UNIT 4 Branch and Bound
UNIT 4 Branch and Bound General method: Branch and Bound is another method to systematically search a solution space. Just like backtracking, we will use bounding functions to avoid generating subtrees
mywbut.com Informed Search Strategies-I
Informed Search Strategies-I 1 3.1 Introduction We have outlined the different types of search strategies. In the earlier chapter we have looked at different blind search strategies. Uninformed search
Dynamic programming. Trivial problems are solved first More complex solutions are composed from the simpler solutions already computed
Dynamic programming Solves a complex problem by breaking it down into subproblems Each subproblem is broken down recursively until a trivial problem is reached Computation itself is not recursive: problems
CS6702 GRAPH THEORY AND APPLICATIONS 2 MARKS QUESTIONS AND ANSWERS
CS6702 GRAPH THEORY AND APPLICATIONS 2 MARKS QUESTIONS AND ANSWERS 1 UNIT I INTRODUCTION CS6702 GRAPH THEORY AND APPLICATIONS 2 MARKS QUESTIONS AND ANSWERS 1. Define Graph. A graph G = (V, E) consists
Treaps. 1 Binary Search Trees (BSTs) CSE341T/CSE549T 11/05/2014. Lecture 19
CSE34T/CSE549T /05/04 Lecture 9 Treaps Binary Search Trees (BSTs) Search trees are tree-based data structures that can be used to store and search for items that satisfy a total order. There are many types
Scientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 13 Random Numbers and Stochastic Simulation Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright
Introduction to Randomized Algorithms
Introduction to Randomized Algorithms Gopinath Mishra Advanced Computing and Microelectronics Unit Indian Statistical Institute Kolkata 700108, India. Organization 1 Introduction 2 Some basic ideas from
FINAL EXAM SOLUTIONS
COMP/MATH 3804 Design and Analysis of Algorithms I Fall 2015 FINAL EXAM SOLUTIONS Question 1 (12%). Modify Euclid s algorithm as follows. function Newclid(a,b) if a
Algorithm Design Techniques (III)
Algorithm Design Techniques (III) Minimax. Alpha-Beta Pruning. Search Tree Strategies (backtracking revisited, branch and bound). Local Search. DSA - lecture 10 - T.U.Cluj-Napoca - M. Joldos 1 Tic-Tac-Toe
Notation Index. Probability notation. (there exists) (such that) Fn-4 B n (Bell numbers) CL-27 s t (equivalence relation) GT-5.
Notation Index (there exists) (for all) Fn-4 Fn-4 (such that) Fn-4 B n (Bell numbers) CL-27 s t (equivalence relation) GT-5 ( n ) k (binomial coefficient) CL-15 ( n m 1,m 2,...) (multinomial coefficient)
Chapter 12: Indexing and Hashing. Basic Concepts
Chapter 12: Indexing and Hashing! Basic Concepts! Ordered Indices! B+-Tree Index Files! B-Tree Index Files! Static Hashing! Dynamic Hashing! Comparison of Ordered Indexing and Hashing! Index Definition
Today s s lecture. Lecture 3: Search - 2. Problem Solving by Search. Agent vs. Conventional AI View. Victor R. Lesser. CMPSCI 683 Fall 2004
Today s s lecture Search and Agents Material at the end of last lecture Lecture 3: Search - 2 Victor R. Lesser CMPSCI 683 Fall 2004 Continuation of Simple Search The use of background knowledge to accelerate
Parallel Computing in Combinatorial Optimization
Parallel Computing in Combinatorial Optimization Bernard Gendron Université de Montréal gendron@iro.umontreal.ca Course Outline Objective: provide an overview of the current research on the design of parallel
Statistical Timing Analysis Using Bounds and Selective Enumeration
IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 22, NO. 9, SEPTEMBER 2003 1243 Statistical Timing Analysis Using Bounds and Selective Enumeration Aseem Agarwal, Student
Chapter 12: Indexing and Hashing (Cnt(
Chapter 12: Indexing and Hashing (Cnt( Cnt.) Basic Concepts Ordered Indices B+-Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition
CSCE GPU PROJECT PRESENTATION SCALABLE PARALLEL RANDOM NUMBER GENERATION
CSCE 5013-002 GPU PROJECT PRESENTATION SCALABLE PARALLEL RANDOM NUMBER GENERATION Farhad Parsan 10/25/2010 OUTLINE Introduction Design Performance Live Demonstration Conclusion 2 INTRODUCTION Scalable
Chapter 12: Indexing and Hashing
Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B+-Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL
L.J. Institute of Engineering & Technology Semester: VIII (2016)
Subject Name: Design & Analysis of Algorithm Subject Code:1810 Faculties: Mitesh Thakkar Sr. UNIT-1 Basics of Algorithms and Mathematics No 1 What is an algorithm? What do you mean by correct algorithm?
Informed search algorithms
Informed search algorithms This lecture topic Chapter 3.5-3.7 Next lecture topic Chapter 4.1-4.2 (Please read lecture topic material before and after each lecture on that topic) Outline Review limitations
Informed search strategies (Section ) Source: Fotolia
Informed search strategies (Section 3.5-3.6) Source: Fotolia Review: Tree search Initialize the frontier using the starting state While the frontier is not empty Choose a frontier node to expand according
The Size Robust Multiple Knapsack Problem
MASTER THESIS ICA-3251535 The Size Robust Multiple Knapsack Problem Branch and Price for the Separate and Combined Recovery Decomposition Model Author: D.D. Tönissen, Supervisors: dr. ir. J.M. van den
Physics 736. Experimental Methods in Nuclear-, Particle-, and Astrophysics. - Statistical Methods -
Physics 736 Experimental Methods in Nuclear-, Particle-, and Astrophysics - Statistical Methods - Karsten Heeger heeger@wisc.edu Course Schedule and Reading course website http://neutrino.physics.wisc.edu/teaching/phys736/
4 INFORMED SEARCH AND EXPLORATION. 4.1 Heuristic Search Strategies
55 4 INFORMED SEARCH AND EXPLORATION We now consider informed search that uses problem-specific knowledge beyond the definition of the problem itself This information helps to find solutions more efficiently
Algorithm Design and Analysis
Algorithm Design and Analysis LECTURE 29 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/7/2016 Approximation
ITCS 6150 Intelligent Systems. Lecture 5 Informed Searches
ITCS 6150 Intelligent Systems Lecture 5 Informed Searches Informed Searches We are informed (in some way) about future states and future paths We use this information to make better decisions about which
Anany Levitin 3RD EDITION. Arup Kumar Bhattacharjee. mmmmm Analysis of Algorithms. Soumen Mukherjee. Introduction to TllG DCSISFI &
Introduction to TllG DCSISFI & mmmmm Analysis of Algorithms 3RD EDITION Anany Levitin Villa nova University International Edition contributions by Soumen Mukherjee RCC Institute of Information Technology
Travelling Salesman Problem. Algorithms and Networks 2015/2016 Hans L. Bodlaender Johan M. M. van Rooij
Travelling Salesman Problem Algorithms and Networks 2015/2016 Hans L. Bodlaender Johan M. M. van Rooij 1 Contents TSP and its applications Heuristics and approximation algorithms Construction heuristics,
Informed Search A* Algorithm
Informed Search A* Algorithm CE417: Introduction to Artificial Intelligence Sharif University of Technology Spring 2018 Soleymani Artificial Intelligence: A Modern Approach, Chapter 3 Most slides have
Informed (Heuristic) Search. Idea: be smart about what paths to try.
Informed (Heuristic) Search Idea: be smart about what paths to try. 1 Blind Search vs. Informed Search What s the difference? How do we formally specify this? A node is selected for expansion based on
CS 229 Midterm Review
CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask
Brute Force: Selection Sort
Brute Force: Intro Brute force means straightforward approach Usually based directly on problem s specs Force refers to computational power Usually not as efficient as elegant solutions Advantages: Applicable
PSD1A. DESIGN AND ANALYSIS OF ALGORITHMS Unit : I-V
PSD1A DESIGN AND ANALYSIS OF ALGORITHMS Unit : I-V UNIT I -- Introduction -- Definition of Algorithm -- Pseudocode conventions -- Recursive algorithms -- Time and space complexity -- Big- o notation --
Load Balancing and Termination Detection
Chapter 7 Load Balancing and Termination Detection 1 Load balancing used to distribute computations fairly across processors in order to obtain the highest possible execution speed. Termination detection
DIT411/TIN175, Artificial Intelligence. Peter Ljunglöf. 23 January, 2018
DIT411/TIN175, Artificial Intelligence Chapters 3 4: More search algorithms CHAPTERS 3 4: MORE SEARCH ALGORITHMS DIT411/TIN175, Artificial Intelligence Peter Ljunglöf 23 January, 2018 1 TABLE OF CONTENTS