Computational complexity
|
|
- Sophie Leonard
- 6 years ago
- Views:
Transcription
1 Computational complexity Heuristic Algorithms Giovanni Righini University of Milan Department of Computer Science (Crema)
2 Definitions: problems and instances A problem is a general question expressed in mathematical terms. Usually the same question can be expressed on many examples: they are instances of the problem. For instance: Problem: Is n prime? Instance: Is 7 prime? A solution S is the answer corresponding to a specific instance. Formally, a problem P is a function that maps instances from a set I into solutions (set S): P : I S A priori, we do not know how to compute it: we need an algorithm.
3 Definitions: algorithms An algorithm is a procedure with the following properties: it is formally defined it is deterministic it made of elementary operations it is finite. An algorithm for a problem P is an algorithm whose steps are determined by an instance I I of P and produce a solution S S A : I S An algorithm defines a function and it also computes it. If the function is the same, the algorithm is exact; otherwise, it is heuristic.
4 A heuristic algorithm should be Algorithms characteristics 1. effective: it should compute solutions with a value close to the optimum; 2. efficient: its computational complexity should be low, at least compared with an exact algorithm; 3. robust: it should remain effective and efficient for any possible input. To compute a solution, an algorithm needs some resources. The two most important ones are space (amount of memory required to store data); time (number of elementary steps to be performed to compute the final result).
5 Complexity Time is usually considered as the most critical resource because: time is subtracted from other computations more often than space; it is often possible to use very large amounts of space at a very low cost, but not the same for time; the need of space is upper bounded by the need for time, because space is re-usable. It is intuitive that in general the larger is an instance, the larger is the amount of resources that are needed to compute its solution. However how the computational cost grows when the instance size grows is not always the same: it depends on the problem and on the algorithm. By computational complexity of an algorithm we mean the speed with which the consumption of computational resources grows when the size of the instance grows.
6 Measuring the time complexity The time needed to solve a problem depends on: the specific instance to be solved the algorithm used the machine that executes the algorithm... We want a measure of the time complexity with the following characteristics: independent of the technology, i.e. it must be the same when the computation is done on different hardware; synthetic and formally defined, i.e. it must be represented by a simple and well-defined mathematical expression; ordinal, i.e. it must allow to rank the algorithms according to their complexity. The observed computing time, does not satisfy these requirements.
7 Time complexity The asymptotic worst-case time complexity of an algorithm provides the required measure in this way: 1. we measure the number T of elementary operations executed (which is computer-independent); 2. we compute a number n which determines the number of bits needed to define the size of any instance (e.g., the number of elements in the ground set in a combinatorial optimization problem); 3. we find the maximum number of elementary operations needed to solve instances of size n T (n) = max I I n T (I) n N (this reduces the complexity to a function T : N N) 4. we approximate T (n) with a simpler funcion f (n), for which we are only interested in the asymptotic trend for n + (complexity is more important when instances are larger) 5. finally we can collect these functions in complexity classes.
8 Notation: Θ means that T (n) Θ(f (n)) c 1, c 2 R +, n 0 N : c 1 f (n) T (n) c 2 f (n) for all n n 0 where c 1, c 2 and n 0 are constant values, independent on n. T (n) is between c 1 f (n) and c 2 f (n) for a suitable small value c 1 for a suitable large value c 2 for any size larger than n 0 T(n) f(n) c f(n) 2 T (n) A c f(n) 1 Asymptotically, f (n) is an estimate of T (n) within a constant factor: for large instances, the computing time is proportional to f (n). n 0 n
9 Notation: O means that T (n) O(f (n)) c R +, n 0 N : T (n) c f (n) for all n n 0 where c and n 0 do not depend on n. T (n) is upper bounded by cf (n) for a suitable large value c for any n larger than a suitable n 0 n 0 Asymptotically, f (n) is an upper bound for T (n) within a constant factor: T(n) c f(n) T (n) A for large instances the computing time is at most proportional to f (n). f(n) n
10 Notation: Ω means that T (n) Ω(f (n)) c > 0, n 0 N : T (n) c f (n) for all n n 0 where c and n 0 do not depend on n. T (n) is lower bounded by cf (n) for some suitable small value di c for any n larger than n 0 T(n) f(n) T (n) A Asymptotically, f (n) is a lower bound of T (n) within a constant factor: for large instances the computing time is at least proportional to f (n) n 0 c f(n) n
11 Combinatorial optimization In combinatorial optimization problems it is natural to define the size of an instance as the cardinality of its ground set. An explicit enumeration algorithm considers each subset S E, evaluates whether it is feasible (x X) in α(n) time, evaluates the objective function f (x) in β(n) time, records the best value found. Since the number of solutions is exponential in n, its complexity is at least exponential, even if α(n) and β(n) are polynomials (as often occurs).
12 Polynomial and exponential complexity In combinatorial optimization, the main distinction is between polynomial complexity: T (n) O ( n d) for a constant d > 0 exponential complexity: T (n) Ω(d n ) for a constant d > 1 The algorithms of the former type are efficient; those of the latter type are inefficient. In general, heuristic algorithms are polynomial and they are used when the corresponding exact algorithms are exponential. Assuming 1 operation/µsec n n 2 op. 2 n op. 1 1µ sec 2µ sec msec 1 msec msec 1 sec msec 17.9 min msec 12.7 days msec 35.7 years msec 366 centuries
13 Problem transformations and reductions Some times it is possible and convenient to reformulate an instance of a problem P into an instance of a problem Q and then to transform back the solution of the latter into a solution of the former. Polynomial transformation P Q: given any instance of P a corresponding instance of Q is defined in polynomial time the instance of Q is solved by a suitable algorithm, providing a solution S Q from S Q a corresponding solution S P is obtained in polynomial time Example: VCP SCP, MCP MISP and MISP MCP.
14 Problem transformations and reductions Polynomial reduction P Q: given any instance of P an algorithm A is executed a polynomial number of times; to solve instances of a problem Q obtained in polynomial time from the instance of P and from the results of the previous runs; from the solutions computed, a solution of the instance of P is obtained. Examples: BPP PMSP and PMSP BPP. In both cases if A is polynomial/exponential, the overall algorithm turns out to be polynomial/exponential if A is exact/heuristic, the overall algorithm turns out to be exact/heuristic
15 Optimization vs. decision A polynomial reduction links optimization and decision problems. Optimization problem: given a function f and a feasible region X, what is the minimum of f in X? f = min x X f =? Decision problem: given a function f, a value k and a feasible region X, do solutions with a value not larger than k exist? x X : f (x) k? The two problems are polynomially equivalent: the decision problem can be solved by solving the optimization problem and then comparing the optimal value with k; the optimization problem can be solved by repeatedly solving the decision problem for different values of k, tuned by dichotomous search.
16 Drawbacks of worst-case analysis The worst-case time complexity has some relevant drawbacks: it does not consider the performance of the algorithm on the easy/small instances; in practice the most difficult instances could be rare or unrealistic; it provides a rough estimate of the computing time growth, not of the computing time itself; the estimate can be very rough, up to the point it becomes useless; it may be misleading: algorithms with worse worst-case computational complexity can be very efficient in practice, even more than algorithms with better worst-case computational complexity.
17 Other complexity measures To overcome these drawbacks one could employ different definitions of computational complexity: parameterized complexity expresses T as a function of some other relevant parameter k besides the size of the instance n: T (n, k) average-case complexity assumes a probability distribution on I and it evaluates the expected value of T (I) on I n T (n) = E [T (I) I I n ] If the distribution has some parameter k, the average-case complexity is also parameterized, i.e. it provides T (n, k).
18 Average-case complexity Average-case complexity analysis and classification is more reliable when algorithms are efficient on almost all instances (e.g. the simplex algorithm for linear programming). We would like to evaluate the expected value of T (I) on I n for each n N T (n) = E [T (I) I I n ] This requires to define the probability distribution of the instances. The most frequent hypothesis is equiprobability; (when we do not have any other information.) other assumptions must be based on some specific probabilistic model of the problem (often depending on some parameters.)
19 Random instances: binary matrices Associating a probability with every instance of a problem is useful for two reasons: for a priori studying the average-case complexity of an algorithm; for a posteriori evaluating the efficiency of the algorithm. In case of heuristic algorithm we also want to evaluate their effectiveness (the value of the solutions obtained and the distance from the optimum). Random binary matrices of given size (m, n): 1. model with uniform probability p: Pr [ a ij = 1 ] = p (i = 1,...,m; j = 1,...,n) If p = 0.5 it provides equiprobability of all instances. 2. model with fixed density δ: given the mn entries of the matrix, δmn are randomly selected with uniform probability distribution and are set to 1. The two models tend to be similar for p = δ.
20 Random instances: graphs Random graphs of size n can be generated as follows: 1. Gilbert model: G(n, p), i.e. uniform probability p: Pr [(i, j) E] = p (i V, j V \{i}) Graphs with the same given number of edges m have the same probability p m : (1 p) n(n 1)/2 m (different for each m) If p = 0.5 it coincides with the model where all graphs have the same probability. 2. Erdős-Rényi model: G(n, m): given the number o edges m, m unordered vertex pairs are randomly selected with uniform probability distribution and an edge is generated for each of them. The two models tend to be similar for p = 2 m n(n 1).
21 Phase transitions Different values of the parameters of the probability distributions correspond to different regions of the instance space. For several problems we observe that the computing time of the algorithms is significantly different in different regions. In case of heuristic algorithms the same holds for the quality of the solutions. This has to do with the robustness of the algorithms. In some cases the changes occur suddenly, for some critical values of the parameters, reminding the phase transitions in physical systems.
22 Two things we can do The design and analysis of heuristic algorithms proceeds in two directions: proving theoretical properties on the algorithms, such as: worst-case time complexity (usually polynomial); average-case time complexity or parameterized time complexity; approximation guarantees; evaluating the practical usefulness of the algorithms: computing time; approximation; robustness to instances and to parameters (phase transitions). The termination is often (arbitrarily) decided on the basis of the number of iterations or the computing time elapsed or the lack of improvements for a certain time. It is used to calibrate the trade-off between approximation and computing time.
Algorithm. Algorithm Analysis. Algorithm. Algorithm. Analyzing Sorting Algorithms (Insertion Sort) Analyzing Algorithms 8/31/2017
8/3/07 Analysis Introduction to Analysis Model of Analysis Mathematical Preliminaries for Analysis Set Notation Asymptotic Analysis What is an algorithm? An algorithm is any well-defined computational
More informationComputer Science Approach to problem solving
Computer Science Approach to problem solving If my boss / supervisor / teacher formulates a problem to be solved urgently, can I write a program to efficiently solve this problem??? Polynomial-Time Brute
More informationLocal search. Heuristic algorithms. Giovanni Righini. University of Milan Department of Computer Science (Crema)
Local search Heuristic algorithms Giovanni Righini University of Milan Department of Computer Science (Crema) Exchange algorithms In Combinatorial Optimization every solution x is a subset of E An exchange
More informationConstructive and destructive algorithms
Constructive and destructive algorithms Heuristic algorithms Giovanni Righini University of Milan Department of Computer Science (Crema) Constructive algorithms In combinatorial optimization problems every
More informationPROGRAM EFFICIENCY & COMPLEXITY ANALYSIS
Lecture 03-04 PROGRAM EFFICIENCY & COMPLEXITY ANALYSIS By: Dr. Zahoor Jan 1 ALGORITHM DEFINITION A finite set of statements that guarantees an optimal solution in finite interval of time 2 GOOD ALGORITHMS?
More informationAdaptive Large Neighborhood Search
Adaptive Large Neighborhood Search Heuristic algorithms Giovanni Righini University of Milan Department of Computer Science (Crema) VLSN and LNS By Very Large Scale Neighborhood (VLSN) local search, we
More informationAlgorithm Analysis. (Algorithm Analysis ) Data Structures and Programming Spring / 48
Algorithm Analysis (Algorithm Analysis ) Data Structures and Programming Spring 2018 1 / 48 What is an Algorithm? An algorithm is a clearly specified set of instructions to be followed to solve a problem
More informationCS:3330 (22c:31) Algorithms
What s an Algorithm? CS:3330 (22c:31) Algorithms Introduction Computer Science is about problem solving using computers. Software is a solution to some problems. Algorithm is a design inside a software.
More informationComputer Science 210 Data Structures Siena College Fall Topic Notes: Complexity and Asymptotic Analysis
Computer Science 210 Data Structures Siena College Fall 2017 Topic Notes: Complexity and Asymptotic Analysis Consider the abstract data type, the Vector or ArrayList. This structure affords us the opportunity
More informationLecture 5: Running Time Evaluation
Lecture 5: Running Time Evaluation Worst-case and average-case performance Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 13 1 Time complexity 2 Time growth 3 Worst-case 4 Average-case
More informationCS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims. Lecture 10: Asymptotic Complexity and
CS/ENGRD 2110 Object-Oriented Programming and Data Structures Spring 2012 Thorsten Joachims Lecture 10: Asymptotic Complexity and What Makes a Good Algorithm? Suppose you have two possible algorithms or
More informationCS 6402 DESIGN AND ANALYSIS OF ALGORITHMS QUESTION BANK
CS 6402 DESIGN AND ANALYSIS OF ALGORITHMS QUESTION BANK Page 1 UNIT I INTRODUCTION 2 marks 1. Why is the need of studying algorithms? From a practical standpoint, a standard set of algorithms from different
More informationOutline Purpose How to analyze algorithms Examples. Algorithm Analysis. Seth Long. January 15, 2010
January 15, 2010 Intuitive Definitions Common Runtimes Final Notes Compare space and time requirements for algorithms Understand how an algorithm scales with larger datasets Intuitive Definitions Outline
More informationConstructive meta-heuristics
Constructive meta-heuristics Heuristic algorithms Giovanni Righini University of Milan Department of Computer Science (Crema) Improving constructive algorithms For many problems constructive algorithms
More informationPolynomial time approximation algorithms
Polynomial time approximation algorithms Doctoral course Optimization on graphs - Lecture 5.2 Giovanni Righini January 18 th, 2013 Approximation algorithms There are several reasons for using approximation
More informationOutline and Reading. Analysis of Algorithms 1
Outline and Reading Algorithms Running time ( 3.1) Pseudo-code ( 3.2) Counting primitive operations ( 3.4) Asymptotic notation ( 3.4.1) Asymptotic analysis ( 3.4.2) Case study ( 3.4.3) Analysis of Algorithms
More informationACO and other (meta)heuristics for CO
ACO and other (meta)heuristics for CO 32 33 Outline Notes on combinatorial optimization and algorithmic complexity Construction and modification metaheuristics: two complementary ways of searching a solution
More informationAlgorithm Analysis. Applied Algorithmics COMP526. Algorithm Analysis. Algorithm Analysis via experiments
Applied Algorithmics COMP526 Lecturer: Leszek Gąsieniec, 321 (Ashton Bldg), L.A.Gasieniec@liverpool.ac.uk Lectures: Mondays 4pm (BROD-107), and Tuesdays 3+4pm (BROD-305a) Office hours: TBA, 321 (Ashton)
More informationAlgorithms and Theory of Computation. Lecture 2: Big-O Notation Graph Algorithms
Algorithms and Theory of Computation Lecture 2: Big-O Notation Graph Algorithms Xiaohui Bei MAS 714 August 15, 2017 Nanyang Technological University MAS 714 August 15, 2017 1 / 22 O, Ω, and Θ Let T, f
More informationThe Resolution Algorithm
The Resolution Algorithm Introduction In this lecture we introduce the Resolution algorithm for solving instances of the NP-complete CNF- SAT decision problem. Although the algorithm does not run in polynomial
More informationAssignment 1 (concept): Solutions
CS10b Data Structures and Algorithms Due: Thursday, January 0th Assignment 1 (concept): Solutions Note, throughout Exercises 1 to 4, n denotes the input size of a problem. 1. (10%) Rank the following functions
More informationElementary maths for GMT. Algorithm analysis Part I
Elementary maths for GMT Algorithm analysis Part I Algorithms An algorithm is a step-by-step procedure for solving a problem in a finite amount of time Most algorithms transform input objects into output
More informationRUNNING TIME ANALYSIS. Problem Solving with Computers-II
RUNNING TIME ANALYSIS Problem Solving with Computers-II Performance questions 4 How efficient is a particular algorithm? CPU time usage (Running time complexity) Memory usage Disk usage Network usage Why
More informationAlgorithms A Look At Efficiency
Algorithms A Look At Efficiency 1B Big O Notation 15-121 Introduction to Data Structures, Carnegie Mellon University - CORTINA 1 Big O Instead of using the exact number of operations to express the complexity
More informationAnalysis of Algorithm. Chapter 2
Analysis of Algorithm Chapter 2 Outline Efficiency of algorithm Apriori of analysis Asymptotic notation The complexity of algorithm using Big-O notation Polynomial vs Exponential algorithm Average, best
More informationNP-Hardness. We start by defining types of problem, and then move on to defining the polynomial-time reductions.
CS 787: Advanced Algorithms NP-Hardness Instructor: Dieter van Melkebeek We review the concept of polynomial-time reductions, define various classes of problems including NP-complete, and show that 3-SAT
More information3 INTEGER LINEAR PROGRAMMING
3 INTEGER LINEAR PROGRAMMING PROBLEM DEFINITION Integer linear programming problem (ILP) of the decision variables x 1,..,x n : (ILP) subject to minimize c x j j n j= 1 a ij x j x j 0 x j integer n j=
More informationChapter 2: Complexity Analysis
Chapter 2: Complexity Analysis Objectives Looking ahead in this chapter, we ll consider: Computational and Asymptotic Complexity Big-O Notation Properties of the Big-O Notation Ω and Θ Notations Possible
More informationCSE 146. Asymptotic Analysis Interview Question of the Day Homework 1 & Project 1 Work Session
CSE 146 Asymptotic Analysis Interview Question of the Day Homework 1 & Project 1 Work Session Comparing Algorithms Rough Estimate Ignores Details Or really: independent of details What are some details
More informationUNIT 1 ANALYSIS OF ALGORITHMS
UNIT 1 ANALYSIS OF ALGORITHMS Analysis of Algorithms Structure Page Nos. 1.0 Introduction 7 1.1 Objectives 7 1.2 Mathematical Background 8 1.3 Process of Analysis 12 1.4 Calculation of Storage Complexity
More informationAlgorithms and Data Structures
Algorithms and Data Structures Spring 2019 Alexis Maciel Department of Computer Science Clarkson University Copyright c 2019 Alexis Maciel ii Contents 1 Analysis of Algorithms 1 1.1 Introduction.................................
More informationEND-TERM EXAMINATION
(Please Write your Exam Roll No. immediately) Exam. Roll No... END-TERM EXAMINATION Paper Code : MCA-205 DECEMBER 2006 Subject: Design and analysis of algorithm Time: 3 Hours Maximum Marks: 60 Note: Attempt
More informationCSE 421: Introduction to Algorithms Complexity
CSE 421: Introduction to Algorithms Complexity Yin-Tat Lee 1 Defining Efficiency Runs fast on typical real problem instances Pros: Sensible, Bottom-line oriented Cons: Moving target (diff computers, programming
More informationTheorem 2.9: nearest addition algorithm
There are severe limits on our ability to compute near-optimal tours It is NP-complete to decide whether a given undirected =(,)has a Hamiltonian cycle An approximation algorithm for the TSP can be used
More informationAgenda. The worst algorithm in the history of humanity. Asymptotic notations: Big-O, Big-Omega, Theta. An iterative solution
Agenda The worst algorithm in the history of humanity 1 Asymptotic notations: Big-O, Big-Omega, Theta An iterative solution A better iterative solution The repeated squaring trick Fibonacci sequence 2
More informationAlgorithm Efficiency & Sorting. Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms
Algorithm Efficiency & Sorting Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms Overview Writing programs to solve problem consists of a large number of decisions how to represent
More informationSize of a problem instance: Bigger instances take
2.1 Integer Programming and Combinatorial Optimization Slide set 2: Computational Complexity Katta G. Murty Lecture slides Aim: To study efficiency of various algo. for solving problems, and to classify
More information[ 11.2, 11.3, 11.4] Analysis of Algorithms. Complexity of Algorithms. 400 lecture note # Overview
400 lecture note #0 [.2,.3,.4] Analysis of Algorithms Complexity of Algorithms 0. Overview The complexity of an algorithm refers to the amount of time and/or space it requires to execute. The analysis
More informationLecture notes on the simplex method September We will present an algorithm to solve linear programs of the form. maximize.
Cornell University, Fall 2017 CS 6820: Algorithms Lecture notes on the simplex method September 2017 1 The Simplex Method We will present an algorithm to solve linear programs of the form maximize subject
More informationMathematical and Algorithmic Foundations Linear Programming and Matchings
Adavnced Algorithms Lectures Mathematical and Algorithmic Foundations Linear Programming and Matchings Paul G. Spirakis Department of Computer Science University of Patras and Liverpool Paul G. Spirakis
More informationCSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis. Aaron Bauer Winter 2014
CSE373: Data Structures and Algorithms Lecture 4: Asymptotic Analysis Aaron Bauer Winter 2014 Previously, on CSE 373 We want to analyze algorithms for efficiency (in time and space) And do so generally
More informationAlgorithm Efficiency & Sorting. Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms
Algorithm Efficiency & Sorting Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms Overview Writing programs to solve problem consists of a large number of decisions how to represent
More informationMath 170- Graph Theory Notes
1 Math 170- Graph Theory Notes Michael Levet December 3, 2018 Notation: Let n be a positive integer. Denote [n] to be the set {1, 2,..., n}. So for example, [3] = {1, 2, 3}. To quote Bud Brown, Graph theory
More informationDESIGN AND ANALYSIS OF ALGORITHMS. Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES
DESIGN AND ANALYSIS OF ALGORITHMS Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES http://milanvachhani.blogspot.in USE OF LOOPS As we break down algorithm into sub-algorithms, sooner or later we shall
More informationL.J. Institute of Engineering & Technology Semester: VIII (2016)
Subject Name: Design & Analysis of Algorithm Subject Code:1810 Faculties: Mitesh Thakkar Sr. UNIT-1 Basics of Algorithms and Mathematics No 1 What is an algorithm? What do you mean by correct algorithm?
More informationINSTITUTE OF AERONAUTICAL ENGINEERING
INSTITUTE OF AERONAUTICAL ENGINEERING (Autonomous) Dundigal, Hyderabad -500 043 INFORMATION TECHNOLOGY TUTORIAL QUESTION BANK Course Name : DESIGN AND ANALYSIS OF ALGORITHMS Course Code : AIT001 Class
More informationGRASP. Greedy Randomized Adaptive. Search Procedure
GRASP Greedy Randomized Adaptive Search Procedure Type of problems Combinatorial optimization problem: Finite ensemble E = {1,2,... n } Subset of feasible solutions F 2 Objective function f : 2 Minimisation
More informationSEARCHING, SORTING, AND ASYMPTOTIC COMPLEXITY
1 A3 and Prelim 2 SEARCHING, SORTING, AND ASYMPTOTIC COMPLEXITY Lecture 11 CS2110 Fall 2016 Deadline for A3: tonight. Only two late days allowed (Wed-Thur) Prelim: Thursday evening. 74 conflicts! If you
More informationBranch-and-bound: an example
Branch-and-bound: an example Giovanni Righini Università degli Studi di Milano Operations Research Complements The Linear Ordering Problem The Linear Ordering Problem (LOP) is an N P-hard combinatorial
More informationEARLY INTERIOR-POINT METHODS
C H A P T E R 3 EARLY INTERIOR-POINT METHODS An interior-point algorithm is one that improves a feasible interior solution point of the linear program by steps through the interior, rather than one that
More informationUnit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES
DESIGN AND ANALYSIS OF ALGORITHMS Unit 1 Chapter 4 ITERATIVE ALGORITHM DESIGN ISSUES http://milanvachhani.blogspot.in USE OF LOOPS As we break down algorithm into sub-algorithms, sooner or later we shall
More informationAlgorithm Analysis. College of Computing & Information Technology King Abdulaziz University. CPCS-204 Data Structures I
Algorithm Analysis College of Computing & Information Technology King Abdulaziz University CPCS-204 Data Structures I Order Analysis Judging the Efficiency/Speed of an Algorithm Thus far, we ve looked
More informationFramework for Design of Dynamic Programming Algorithms
CSE 441T/541T Advanced Algorithms September 22, 2010 Framework for Design of Dynamic Programming Algorithms Dynamic programming algorithms for combinatorial optimization generalize the strategy we studied
More informationModule 1: Asymptotic Time Complexity and Intro to Abstract Data Types
Module 1: Asymptotic Time Complexity and Intro to Abstract Data Types Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu
More informationData Structures and Algorithms CSE 465
Data Structures and Algorithms CSE 465 LECTURE 2 Analysis of Algorithms Insertion Sort Loop invariants Asymptotic analysis Sofya Raskhodnikova and Adam Smith The problem of sorting Input: sequence a 1,
More informationIntroduction to Data Structure
Introduction to Data Structure CONTENTS 1.1 Basic Terminology 1. Elementary data structure organization 2. Classification of data structure 1.2 Operations on data structures 1.3 Different Approaches to
More informationThe 2-core of a Non-homogeneous Hypergraph
July 16, 2012 k-cores A Hypergraph G on vertex set V is a collection E of subsets of V. E is the set of hyperedges. For ordinary graphs, e = 2 for all e E. The k-core of a (hyper)graph is the maximal subgraph
More informationMathematics of networks. Artem S. Novozhilov
Mathematics of networks Artem S. Novozhilov August 29, 2013 A disclaimer: While preparing these lecture notes, I am using a lot of different sources for inspiration, which I usually do not cite in the
More informationIntroduction to the Analysis of Algorithms. Algorithm
Introduction to the Analysis of Algorithms Based on the notes from David Fernandez-Baca Bryn Mawr College CS206 Intro to Data Structures Algorithm An algorithm is a strategy (well-defined computational
More informationThe Dynamic Hungarian Algorithm for the Assignment Problem with Changing Costs
The Dynamic Hungarian Algorithm for the Assignment Problem with Changing Costs G. Ayorkor Mills-Tettey Anthony Stentz M. Bernardine Dias CMU-RI-TR-07-7 July 007 Robotics Institute Carnegie Mellon University
More informationComplexity Analysis of an Algorithm
Complexity Analysis of an Algorithm Algorithm Complexity-Why? Resources required for running that algorithm To estimate how long a program will run. To estimate the largest input that can reasonably be
More informationCS240 Fall Mike Lam, Professor. Algorithm Analysis
CS240 Fall 2014 Mike Lam, Professor Algorithm Analysis Algorithm Analysis Motivation: what and why Mathematical functions Comparative & asymptotic analysis Big-O notation ("Big-Oh" in textbook) Analyzing
More informationAlgorithm Analysis. Gunnar Gotshalks. AlgAnalysis 1
Algorithm Analysis AlgAnalysis 1 How Fast is an Algorithm? 1 Measure the running time» Run the program for many data types > Use System.currentTimeMillis to record the time Worst Time Average Best» Usually
More informationAlgorithm efficiency can be measured in terms of: Time Space Other resources such as processors, network packets, etc.
Algorithms Analysis Algorithm efficiency can be measured in terms of: Time Space Other resources such as processors, network packets, etc. Algorithms analysis tends to focus on time: Techniques for measuring
More information31.6 Powers of an element
31.6 Powers of an element Just as we often consider the multiples of a given element, modulo, we consider the sequence of powers of, modulo, where :,,,,. modulo Indexing from 0, the 0th value in this sequence
More informationCopyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch.
Iterative Improvement Algorithm design technique for solving optimization problems Start with a feasible solution Repeat the following step until no improvement can be found: change the current feasible
More informationClass Note #02. [Overall Information] [During the Lecture]
Class Note #02 Date: 01/11/2006 [Overall Information] In this class, after a few additional announcements, we study the worst-case running time of Insertion Sort. The asymptotic notation (also called,
More informationAlgorithm Analysis. Part I. Tyler Moore. Lecture 3. CSE 3353, SMU, Dallas, TX
Algorithm Analysis Part I Tyler Moore CSE 5, SMU, Dallas, TX Lecture how many times do you have to turn the crank? Some slides created by or adapted from Dr. Kevin Wayne. For more information see http://www.cs.princeton.edu/~wayne/kleinberg-tardos.
More informationWhat is an algorithm?
Reminders CS 142 Lecture 3 Analysis, ADTs & Objects Program 1 was assigned - Due on 1/27 by 11:55pm 2 Abstraction Measuring Algorithm Efficiency When you utilize the mylist.index(item) function you are
More informationASYMPTOTIC COMPLEXITY
Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better. - Edsger Dijkstra ASYMPTOTIC COMPLEXITY Lecture
More informationUML CS Algorithms Qualifying Exam Fall, 2003 ALGORITHMS QUALIFYING EXAM
NAME: This exam is open: - books - notes and closed: - neighbors - calculators ALGORITHMS QUALIFYING EXAM The upper bound on exam time is 3 hours. Please put all your work on the exam paper. (Partial credit
More informationWhat is a Graphon? Daniel Glasscock, June 2013
What is a Graphon? Daniel Glasscock, June 2013 These notes complement a talk given for the What is...? seminar at the Ohio State University. The block images in this PDF should be sharp; if they appear
More information3 SOLVING PROBLEMS BY SEARCHING
48 3 SOLVING PROBLEMS BY SEARCHING A goal-based agent aims at solving problems by performing actions that lead to desirable states Let us first consider the uninformed situation in which the agent is not
More informationCS240 Fall Mike Lam, Professor. Algorithm Analysis
CS240 Fall 2014 Mike Lam, Professor Algorithm Analysis HW1 Grades are Posted Grades were generally good Check my comments! Come talk to me if you have any questions PA1 is Due 9/17 @ noon Web-CAT submission
More informationGiovanni De Micheli. Integrated Systems Centre EPF Lausanne
Two-level Logic Synthesis and Optimization Giovanni De Micheli Integrated Systems Centre EPF Lausanne This presentation can be used for non-commercial purposes as long as this note and the copyright footers
More informationASYMPTOTIC COMPLEXITY
Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better. - Edsger Dijkstra ASYMPTOTIC COMPLEXITY Lecture
More informationComputational problems. Lecture 2: Combinatorial search and optimisation problems. Computational problems. Examples. Example
Lecture 2: Combinatorial search and optimisation problems Different types of computational problems Examples of computational problems Relationships between problems Computational properties of different
More informationA CSP Search Algorithm with Reduced Branching Factor
A CSP Search Algorithm with Reduced Branching Factor Igor Razgon and Amnon Meisels Department of Computer Science, Ben-Gurion University of the Negev, Beer-Sheva, 84-105, Israel {irazgon,am}@cs.bgu.ac.il
More informationSome graph theory applications. communications networks
Some graph theory applications to communications networks Keith Briggs Keith.Briggs@bt.com http://keithbriggs.info Computational Systems Biology Group, Sheffield - 2006 Nov 02 1100 graph problems Sheffield
More informationCombinatorial Optimization
Combinatorial Optimization Frank de Zeeuw EPFL 2012 Today Introduction Graph problems - What combinatorial things will we be optimizing? Algorithms - What kind of solution are we looking for? Linear Programming
More informationLECTURE 9 Data Structures: A systematic way of organizing and accessing data. --No single data structure works well for ALL purposes.
LECTURE 9 Data Structures: A systematic way of organizing and accessing data. --No single data structure works well for ALL purposes. Input Algorithm Output An algorithm is a step-by-step procedure for
More informationCS 580: Algorithm Design and Analysis. Jeremiah Blocki Purdue University Spring 2018
CS 580: Algorithm Design and Analysis Jeremiah Blocki Purdue University Spring 2018 Recap: Stable Matching Problem Definition of a Stable Matching Stable Roomate Matching Problem Stable matching does not
More informationInformed Search Methods
Informed Search Methods How can we improve searching strategy by using intelligence? Map example: Heuristic: Expand those nodes closest in as the crow flies distance to goal 8-puzzle: Heuristic: Expand
More informationBipartite Graph based Construction of Compressed Sensing Matrices
Bipartite Graph based Construction of Compressed Sensing Matrices Weizhi Lu, Kidiyo Kpalma and Joseph Ronsin arxiv:44.4939v [cs.it] 9 Apr 24 Abstract This paper proposes an efficient method to construct
More informationEECS 477: Introduction to algorithms. Lecture 6
EECS 477: Introduction to algorithms. Lecture 6 Prof. Igor Guskov guskov@eecs.umich.edu September 24, 2002 1 Lecture outline Finish up with asymptotic notation Asymptotic analysis of programs Analyzing
More informationOptimization Techniques for Design Space Exploration
0-0-7 Optimization Techniques for Design Space Exploration Zebo Peng Embedded Systems Laboratory (ESLAB) Linköping University Outline Optimization problems in ERT system design Heuristic techniques Simulated
More information4 non-obvious examples/results. 2. The core idea in our probabilistic reformulation
1. Local weak convergence of graphs/networks Stuff that s obvious when you think about it 4 non-obvious examples/results 2. The core idea in our probabilistic reformulation of special cases of the cavity
More informationAnalysis of Algorithms
Analysis of Algorithms Data Structures and Algorithms Acknowledgement: These slides are adapted from slides provided with Data Structures and Algorithms in C++ Goodrich, Tamassia and Mount (Wiley, 2004)
More informationSEARCHING, SORTING, AND ASYMPTOTIC COMPLEXITY. Lecture 11 CS2110 Spring 2016
1 SEARCHING, SORTING, AND ASYMPTOTIC COMPLEXITY Lecture 11 CS2110 Spring 2016 Time spent on A2 2 Histogram: [inclusive:exclusive) [0:1): 0 [1:2): 24 ***** [2:3): 84 ***************** [3:4): 123 *************************
More informationAnnouncements. CSEP 521 Applied Algorithms. Announcements. Polynomial time efficiency. Definitions of efficiency 1/14/2013
Announcements CSEP 51 Applied Algorithms Richard Anderson Winter 013 Lecture Reading Chapter.1,. Chapter 3 Chapter Homework Guidelines Prove that your algorithm works A proof is a convincing argument Give
More informationSankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology. Assignment
Class: V - CE Sankalchand Patel College of Engineering - Visnagar Department of Computer Engineering and Information Technology Sub: Design and Analysis of Algorithms Analysis of Algorithm: Assignment
More informationCS S-02 Algorithm Analysis 1
CS245-2008S-02 Algorithm Analysis 1 02-0: Algorithm Analysis When is algorithm A better than algorithm B? 02-1: Algorithm Analysis When is algorithm A better than algorithm B? Algorithm A runs faster 02-2:
More informationLecture 5 Sorting Arrays
Lecture 5 Sorting Arrays 15-122: Principles of Imperative Computation (Spring 2018) Frank Pfenning, Rob Simmons We begin this lecture by discussing how to compare running times of functions in an abstract,
More informationAnalysis of Algorithms. CS 1037a Topic 13
Analysis of Algorithms CS 1037a Topic 13 Overview Time complexity - exact count of operations T(n) as a function of input size n - complexity analysis using O(...) bounds - constant time, linear, logarithmic,
More informationCSE 373 APRIL 3 RD ALGORITHM ANALYSIS
CSE 373 APRIL 3 RD ALGORITHM ANALYSIS ASSORTED MINUTIAE HW1P1 due tonight at midnight HW1P2 due Friday at midnight HW2 out tonight Second Java review session: Friday 10:30 ARC 147 TODAY S SCHEDULE Algorithm
More informationQuiz 1 Solutions. (a) If f(n) = Θ(g(n)) and g(n) = Θ(h(n)), then h(n) = Θ(f(n)) Solution: True. Θ is transitive.
Introduction to Algorithms October 17, 2007 Massachusetts Institute of Technology 6.006 Fall 2007 Professors Ron Rivest and Srini Devadas Quiz 1 Solutions Problem 1. Quiz 1 Solutions Asymptotic Notation
More informationFoundations of Computing
Foundations of Computing Darmstadt University of Technology Dept. Computer Science Winter Term 2005 / 2006 Copyright c 2004 by Matthias Müller-Hannemann and Karsten Weihe All rights reserved http://www.algo.informatik.tu-darmstadt.de/
More informationCS Introduction to Data Mining Instructor: Abdullah Mueen
CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 8: ADVANCED CLUSTERING (FUZZY AND CO -CLUSTERING) Review: Basic Cluster Analysis Methods (Chap. 10) Cluster Analysis: Basic Concepts
More informationToday s Outline. CSE 326: Data Structures Asymptotic Analysis. Analyzing Algorithms. Analyzing Algorithms: Why Bother? Hannah Takes a Break
Today s Outline CSE 326: Data Structures How s the project going? Finish up stacks, queues, lists, and bears, oh my! Math review and runtime analysis Pretty pictures Asymptotic analysis Hannah Tang and
More informationDiscrete mathematics , Fall Instructor: prof. János Pach
Discrete mathematics 2016-2017, Fall Instructor: prof. János Pach - covered material - Lecture 1. Counting problems To read: [Lov]: 1.2. Sets, 1.3. Number of subsets, 1.5. Sequences, 1.6. Permutations,
More information