: Parallel Algorithms Exercises, Batch 1. Exercise Day, Tuesday 18.11, 10:00. Hand-in before or at Exercise Day

Size: px
Start display at page:

Download ": Parallel Algorithms Exercises, Batch 1. Exercise Day, Tuesday 18.11, 10:00. Hand-in before or at Exercise Day"

Transcription

1 : Parallel Algorithms Exercises, Batch 1. Exercise Day, Tuesday 18.11, 10:00. Hand-in before or at Exercise Day Jesper Larsson Träff, Francesco Versaci Parallel Computing Group TU Wien October 16, 2014 Solutions to as many of the exercises as possible to be handed in at the latest on Tuesday, 18th November, before the exercise session (10:00-12:00, Seminarroom Argentinierstrasse): try them all, see how far you can get; discussion at the exercise session. If you use papers to find a solution (not recommended, but allowed when stuck), give reference, and explain what that paper accomplishes. Exercise 1 (warm-up): 1. Find a closed form expression for the k-term sum k Find a closed form expression for the k-term sum 1 + 1/2 + 1/4 + 1/ /2 k Find a closed form expression for the k-term sum n + n/2 + n/4 + n/ n/2 k Find a closed form expression for the k-term sum k 1 i=0 aq i Prove correctness by induction, verify against fact sheet. Exercise 2 (warm-up): Find exact, closed form solutions to the Quicksort (and, of course, Mergesort as well - why?) recurrences T (n) = T ( n/2 ) + log n T (1) = a W (n) = 2W ( n/2 ) + n W (1) = a for some suitable a > 0. Prove correctness by induction, if applicable, verify against fact sheet. 1

2 Exercise 3: Let T (n) be defined by the recurrence T (n) = nt ( n) + an T (1) = b for a constants a > 0, b > 0. Then T (n) is O(n log log n). Exercise 4: Summation with perfect speed-up: Modify the (or devise a new) summation algorithm for computing the sum s = x 0 + x x n 1 of n numbers stored in an array A such that it performs n 1 summation operations, thus has work n 1 + o(n) with potential for perfect speed-up compared to the natural, sequential algorithm (in terms of summation operations). The algorithm shall take O(log n) time steps, and more precisely run in log n steps on an EREW PRAM. Write both an explicit program for p processors, and also give the algorithm as a WT-level algorithm. Exercise 5: (Brent s Theorem): Give an explicit (not WT-level) parallel, cost-optimal algorithm for summation, that is, modify the EREW algorithm from the lecture to run in O(n/p + log n) time steps with p EREW PRAM processors. That is, do the scheduling suggested by Brent s Theorem of the work within each of the log n parallel time steps explicitly by hand. Exercise 6 (PRAM synchronization:) Assume that synchronization is necessary for some p-processor PRAM algorithm (e.g., pointer jumping in trees) to conclude a phase and enter into the next phase of the algorithm; some processors may arrive earlier than others at the synchronization point, but it is not known which processors and also not how much earlier (so padding by the compiler may not be possible). Devise a p-processor EREW PRAM synchronization algorithm that ensures that all processors can start a new phase at the same PRAM time step. What is the complexity of the algorithm? Exercise 7 (easy): Instead of using full-fledged prefix-sums for broadcasting the element stored at index k of an n-element array A to all other positions of the the array, devise an EREW algorithm that accomplishes this in log n rounds and n + o(n) operations. Could there be similar possibilities for improving the compaction operation? Exercise 8: Give a correctness proof for the recursive parallel prefix-sums algorithm (Parallel prefix-sums algorithm 2), including proof that it runs correctly on an EREW PRAM. Estimate (and improve, if possible) the operation count (number of summations, number of copy operations) and the space consumption; as exactly as possible, not only asymptotically (use Exercise 1). 2

3 Exercise 9: Show how to use the parallel prefix-sums (scan) operation to 1. evaluate a polynomial a 0 + a 1 x + a 2 x a n x n for given x. Hint: look for Horner s rule. 2. compute the sum of two n-bit binary numbers given as vectors of bits x i and y i ; the sum is also returned as a binary number with bits z i (does this also work for decimal numbers, with digits in x i, y i and output digits in z i?) 3. compute all terms x i for i = 1,... n from a linear recurrence x i = a i x i 1 + b i with x 0 = a 0 for given sequences a 0, a 1,..., a n and b 1, b 2,..., b n Exercise 10 (Segmented prefix-sums): Give an efficient (work-optimal, and O(log n) time), EREW PRAM solution to the important segmented prefix-sums problem. Input is an n-element array A, elements from a set with an associative operation +, and a Boolean marker array M, which marks the beginning of each new segment. That is, a segment is the elements A[i,..., j 1] where M[i] = true and M[j] = true and no indices k, i < k < j have M[k] = true. The task is to compute the (inclusive) prefix sums for all segments simultaneously. Devise a method for solving the segmented scan problem in O(n) work and O(log n) time steps on an EREW PRAM, either by 1. directly modifying either of the algorithms from the lectures 2. using the following operation on pairs (u, i) (v, j) = { (u + v, j) if i = j (v, j) if i j Hint: show that the operation is associative, and apply a scan operation on input (A[i], M[i]) Exercise 11: Use the segmented prefix-sums algorithm to give (in full detail) a parallel Quicksort implementation; the segmented prefix-sums problem is used to handle all active recursions at the same time, and thus solves the processor allocation problem. State the complexity in work and time, assuming a constant time (sequential) function for choosing a good pivot element. Use partition into three segments (<, =, >) at each recursive invocation at each level. Exercise 12: Consider the number of + operations on elements from the input carried out by an EREW PRAM algorithm that solves the prefix sums problem on any n-element input array (that is, in the following, bookkeeping operations in loops and so on shall be ignored, only the real operations performed on input elements and sums of input elements shall count). Devise (explicitly) an EREW PRAM algorithm that runs in O(n/p + p) time 3

4 steps and O(n) work on a p-processor EREW PRAM, and that meets the Snir trade-off of s + t = 2n 2 where s is the number of + operations as described above, and t the longest chain of such operations that have to be done in sequence (depth). Hint: use blocking into p + 1 blocks. Exercise 13 (easy): Show (by example) that Wyllie s list-ranking algorithm as presented in the WT-presentation framework is not quite a correct EREW PRAM algorithm. Give an explicit EREW PRAM formulation of Wyllie s algorithm. Modify/repair the WT-framework presentation such that is a correct CREW PRAM algorithm. Give also a correct WT presentation of the EREW algorithm. Exercise 14 (difficult): Give a detailed implementation of the EREW PRAM parallel binary search algorithm for locating m keys (given as an ordered array) in an n-element array, and show that it is correct. Formulate precise invariants (how does m 0, m 1 exactly represent a segment of the keys, how does n 0, n 1 exactly represent a segment of the array to be searched; how is fusing done without concurrent reading (hint: schedule potentially conflicting reads in different steps and/or use extra copies of the variables that are to be read by another processor); consider both left and right branches, and what happens when a segment of keys reaches a leaf (that is, when n = n 1, where A segments in general consist of the elements A[n 0,..., n 1 1]). If convenient, introduce a level and a numbering for the nodes in the search tree, i.e., if a node has number i, then its left child has number 2i and its right child number 2i + 1, and use this to determine which segments to fuse. Argue that the time complexity is O(log n + log m) using m processors. Exercise 15: Let an algorithm A to solve a problem P of size n in O(log n) time steps and O(n log n) work be given. Let also an algorithm B be given that can reduce the size of any instance of P by a constant fraction c in O(log n/ log log n) time steps and O(n) work without changing the solution. Use algorithms A and B to derive an new O(log n) time and O(n) work algorithm C to solve P. Exercise 16: Solve the processor allocation of the simple, binary search based merge algorithm, that is determine which processors do what in Steps 3 to 6. Segmented prefixsums are probably needed. In terms of constant factors, is this algorithm efficient? That is, determine when processors may stay idle, and try to minimize this as much as possible. Exercise 17 (difficult): In the lecture the following colored prefix-sums operation was used in the Cole-Vishkin work-optimal 3-coloring algorithm (bucket sorting): given an n- element input array A and an associated color array c that assigns a color to each element of A; compute, simultaneously, the prefix-sums over all subsequences of A elements with the same color. The task is to devise an efficient algorithm to solve the colored prefix-sums problem. Hint: first devise a sequential solution (simple); start by considering a constant number of colors k. For the general solution, use a binary tree and pipelining, combine this 4

5 with blocking. For how many colors is the algorithm work-optimal? The algorithm should run on an EREW PRAM. Exercise 18 (difficult): Extend the O(log n) time deterministic coin-tossing algorithm for coloring a cycle to an algorithm for 3-coloring a forest. Which PRAM model is required? Can the algorithm be made work-optimal? 5

Parallel Algorithms for (PRAM) Computers & Some Parallel Algorithms. Reference : Horowitz, Sahni and Rajasekaran, Computer Algorithms

Parallel Algorithms for (PRAM) Computers & Some Parallel Algorithms. Reference : Horowitz, Sahni and Rajasekaran, Computer Algorithms Parallel Algorithms for (PRAM) Computers & Some Parallel Algorithms Reference : Horowitz, Sahni and Rajasekaran, Computer Algorithms Part 2 1 3 Maximum Selection Problem : Given n numbers, x 1, x 2,, x

More information

15-750: Parallel Algorithms

15-750: Parallel Algorithms 5-750: Parallel Algorithms Scribe: Ilari Shafer March {8,2} 20 Introduction A Few Machine Models of Parallel Computation SIMD Single instruction, multiple data: one instruction operates on multiple data

More information

CS 598: Communication Cost Analysis of Algorithms Lecture 15: Communication-optimal sorting and tree-based algorithms

CS 598: Communication Cost Analysis of Algorithms Lecture 15: Communication-optimal sorting and tree-based algorithms CS 598: Communication Cost Analysis of Algorithms Lecture 15: Communication-optimal sorting and tree-based algorithms Edgar Solomonik University of Illinois at Urbana-Champaign October 12, 2016 Defining

More information

EE/CSCI 451 Spring 2018 Homework 8 Total Points: [10 points] Explain the following terms: EREW PRAM CRCW PRAM. Brent s Theorem.

EE/CSCI 451 Spring 2018 Homework 8 Total Points: [10 points] Explain the following terms: EREW PRAM CRCW PRAM. Brent s Theorem. EE/CSCI 451 Spring 2018 Homework 8 Total Points: 100 1 [10 points] Explain the following terms: EREW PRAM CRCW PRAM Brent s Theorem BSP model 1 2 [15 points] Assume two sorted sequences of size n can be

More information

Parallel Random Access Machine (PRAM)

Parallel Random Access Machine (PRAM) PRAM Algorithms Parallel Random Access Machine (PRAM) Collection of numbered processors Access shared memory Each processor could have local memory (registers) Each processor can access any shared memory

More information

CS2223: Algorithms Sorting Algorithms, Heap Sort, Linear-time sort, Median and Order Statistics

CS2223: Algorithms Sorting Algorithms, Heap Sort, Linear-time sort, Median and Order Statistics CS2223: Algorithms Sorting Algorithms, Heap Sort, Linear-time sort, Median and Order Statistics 1 Sorting 1.1 Problem Statement You are given a sequence of n numbers < a 1, a 2,..., a n >. You need to

More information

Real parallel computers

Real parallel computers CHAPTER 30 (in old edition) Parallel Algorithms The PRAM MODEL OF COMPUTATION Abbreviation for Parallel Random Access Machine Consists of p processors (PEs), P 0, P 1, P 2,, P p-1 connected to a shared

More information

Comparison Based Sorting Algorithms. Algorithms and Data Structures: Lower Bounds for Sorting. Comparison Based Sorting Algorithms

Comparison Based Sorting Algorithms. Algorithms and Data Structures: Lower Bounds for Sorting. Comparison Based Sorting Algorithms Comparison Based Sorting Algorithms Algorithms and Data Structures: Lower Bounds for Sorting Definition 1 A sorting algorithm is comparison based if comparisons A[i] < A[j], A[i] A[j], A[i] = A[j], A[i]

More information

Complexity and Advanced Algorithms Monsoon Parallel Algorithms Lecture 2

Complexity and Advanced Algorithms Monsoon Parallel Algorithms Lecture 2 Complexity and Advanced Algorithms Monsoon 2011 Parallel Algorithms Lecture 2 Trivia ISRO has a new supercomputer rated at 220 Tflops Can be extended to Pflops. Consumes only 150 KW of power. LINPACK is

More information

each processor can in one step do a RAM op or read/write to one global memory location

each processor can in one step do a RAM op or read/write to one global memory location Parallel Algorithms Two closely related models of parallel computation. Circuits Logic gates (AND/OR/not) connected by wires important measures PRAM number of gates depth (clock cycles in synchronous circuit)

More information

Algorithms and Data Structures: Lower Bounds for Sorting. ADS: lect 7 slide 1

Algorithms and Data Structures: Lower Bounds for Sorting. ADS: lect 7 slide 1 Algorithms and Data Structures: Lower Bounds for Sorting ADS: lect 7 slide 1 ADS: lect 7 slide 2 Comparison Based Sorting Algorithms Definition 1 A sorting algorithm is comparison based if comparisons

More information

/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Sorting lower bound and Linear-time sorting Date: 9/19/17

/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Sorting lower bound and Linear-time sorting Date: 9/19/17 601.433/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Sorting lower bound and Linear-time sorting Date: 9/19/17 5.1 Introduction You should all know a few ways of sorting in O(n log n)

More information

Parallel Models RAM. Parallel RAM aka PRAM. Variants of CRCW PRAM. Advanced Algorithms

Parallel Models RAM. Parallel RAM aka PRAM. Variants of CRCW PRAM. Advanced Algorithms Parallel Models Advanced Algorithms Piyush Kumar (Lecture 10: Parallel Algorithms) An abstract description of a real world parallel machine. Attempts to capture essential features (and suppress details?)

More information

CS256 Applied Theory of Computation

CS256 Applied Theory of Computation CS256 Applied Theory of Computation Parallel Computation IV John E Savage Overview PRAM Work-time framework for parallel algorithms Prefix computations Finding roots of trees in a forest Parallel merging

More information

Paradigms for Parallel Algorithms

Paradigms for Parallel Algorithms S Parallel Algorithms Paradigms for Parallel Algorithms Reference : C. Xavier and S. S. Iyengar, Introduction to Parallel Algorithms Binary Tree Paradigm A binary tree with n nodes is of height log n Can

More information

Treaps. 1 Binary Search Trees (BSTs) CSE341T/CSE549T 11/05/2014. Lecture 19

Treaps. 1 Binary Search Trees (BSTs) CSE341T/CSE549T 11/05/2014. Lecture 19 CSE34T/CSE549T /05/04 Lecture 9 Treaps Binary Search Trees (BSTs) Search trees are tree-based data structures that can be used to store and search for items that satisfy a total order. There are many types

More information

PRAM ALGORITHMS: BRENT S LAW

PRAM ALGORITHMS: BRENT S LAW PARALLEL AND DISTRIBUTED ALGORITHMS BY DEBDEEP MUKHOPADHYAY AND ABHISHEK SOMANI http://cse.iitkgp.ac.in/~debdeep/courses_iitkgp/palgo/index.htm PRAM ALGORITHMS: BRENT S LAW 2 1 MERGING TWO SORTED ARRAYS

More information

Chapter 1 Divide and Conquer Algorithm Theory WS 2014/15 Fabian Kuhn

Chapter 1 Divide and Conquer Algorithm Theory WS 2014/15 Fabian Kuhn Chapter 1 Divide and Conquer Algorithm Theory WS 2014/15 Fabian Kuhn Divide And Conquer Principle Important algorithm design method Examples from Informatik 2: Sorting: Mergesort, Quicksort Binary search

More information

Parallel Random-Access Machines

Parallel Random-Access Machines Parallel Random-Access Machines Marc Moreno Maza University of Western Ontario, London, Ontario (Canada) CS3101 (Moreno Maza) Parallel Random-Access Machines CS3101 1 / 69 Plan 1 The PRAM Model 2 Performance

More information

A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 3 Parallel Prefix, Pack, and Sorting

A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 3 Parallel Prefix, Pack, and Sorting A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 3 Parallel Prefix, Pack, and Sorting Dan Grossman Last Updated: November 2012 For more information, see http://www.cs.washington.edu/homes/djg/teachingmaterials/

More information

Sorting. There exist sorting algorithms which have shown to be more efficient in practice.

Sorting. There exist sorting algorithms which have shown to be more efficient in practice. Sorting Next to storing and retrieving data, sorting of data is one of the more common algorithmic tasks, with many different ways to perform it. Whenever we perform a web search and/or view statistics

More information

Problem Set 4 Solutions

Problem Set 4 Solutions Design and Analysis of Algorithms March 5, 205 Massachusetts Institute of Technology 6.046J/8.40J Profs. Erik Demaine, Srini Devadas, and Nancy Lynch Problem Set 4 Solutions Problem Set 4 Solutions This

More information

Chapter 6. Parallel Algorithms. Chapter by M. Ghaari. Last update 1 : January 2, 2019.

Chapter 6. Parallel Algorithms. Chapter by M. Ghaari. Last update 1 : January 2, 2019. Chapter 6 Parallel Algorithms Chapter by M. Ghaari. Last update 1 : January 2, 2019. This chapter provides an introduction to parallel algorithms. Our highlevel goal is to present \how to think in parallel"

More information

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR Stamp / Signature of the Invigilator

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR Stamp / Signature of the Invigilator INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR Stamp / Signature of the Invigilator EXAMINATION ( End Semester ) SEMESTER ( Autumn ) Roll Number Section Name Subject Number C S 6 0 0 2 6 Subject Name Parallel

More information

08 A: Sorting III. CS1102S: Data Structures and Algorithms. Martin Henz. March 10, Generated on Tuesday 9 th March, 2010, 09:58

08 A: Sorting III. CS1102S: Data Structures and Algorithms. Martin Henz. March 10, Generated on Tuesday 9 th March, 2010, 09:58 08 A: Sorting III CS1102S: Data Structures and Algorithms Martin Henz March 10, 2010 Generated on Tuesday 9 th March, 2010, 09:58 CS1102S: Data Structures and Algorithms 08 A: Sorting III 1 1 Recap: Sorting

More information

Algorithms and Applications

Algorithms and Applications Algorithms and Applications 1 Areas done in textbook: Sorting Algorithms Numerical Algorithms Image Processing Searching and Optimization 2 Chapter 10 Sorting Algorithms - rearranging a list of numbers

More information

1. (a) O(log n) algorithm for finding the logical AND of n bits with n processors

1. (a) O(log n) algorithm for finding the logical AND of n bits with n processors 1. (a) O(log n) algorithm for finding the logical AND of n bits with n processors on an EREW PRAM: See solution for the next problem. Omit the step where each processor sequentially computes the AND of

More information

Binary Search to find item in sorted array

Binary Search to find item in sorted array Binary Search to find item in sorted array January 15, 2008 QUESTION: Suppose we are given a sorted list A[1..n] (as an array), of n real numbers: A[1] A[2] A[n]. Given a real number x, decide whether

More information

The divide and conquer strategy has three basic parts. For a given problem of size n,

The divide and conquer strategy has three basic parts. For a given problem of size n, 1 Divide & Conquer One strategy for designing efficient algorithms is the divide and conquer approach, which is also called, more simply, a recursive approach. The analysis of recursive algorithms often

More information

Chapter 1 Divide and Conquer Algorithm Theory WS 2015/16 Fabian Kuhn

Chapter 1 Divide and Conquer Algorithm Theory WS 2015/16 Fabian Kuhn Chapter 1 Divide and Conquer Algorithm Theory WS 2015/16 Fabian Kuhn Divide And Conquer Principle Important algorithm design method Examples from Informatik 2: Sorting: Mergesort, Quicksort Binary search

More information

COMP Parallel Computing. PRAM (4) PRAM models and complexity

COMP Parallel Computing. PRAM (4) PRAM models and complexity COMP 633 - Parallel Computing Lecture 5 September 4, 2018 PRAM models and complexity Reading for Thursday Memory hierarchy and cache-based systems Topics Comparison of PRAM models relative performance

More information

The PRAM Model. Alexandre David

The PRAM Model. Alexandre David The PRAM Model Alexandre David 1.2.05 1 Outline Introduction to Parallel Algorithms (Sven Skyum) PRAM model Optimality Examples 11-02-2008 Alexandre David, MVP'08 2 2 Standard RAM Model Standard Random

More information

Sorting (Chapter 9) Alexandre David B2-206

Sorting (Chapter 9) Alexandre David B2-206 Sorting (Chapter 9) Alexandre David B2-206 1 Sorting Problem Arrange an unordered collection of elements into monotonically increasing (or decreasing) order. Let S = . Sort S into S =

More information

Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest. Introduction to Algorithms

Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest. Introduction to Algorithms Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Introduction to Algorithms Preface xiii 1 Introduction 1 1.1 Algorithms 1 1.2 Analyzing algorithms 6 1.3 Designing algorithms 1 1 1.4 Summary 1 6

More information

CSL 730: Parallel Programming

CSL 730: Parallel Programming CSL 73: Parallel Programming General Algorithmic Techniques Balance binary tree Partitioning Divid and conquer Fractional cascading Recursive doubling Symmetry breaking Pipelining 2 PARALLEL ALGORITHM

More information

Optimal Parallel Randomized Renaming

Optimal Parallel Randomized Renaming Optimal Parallel Randomized Renaming Martin Farach S. Muthukrishnan September 11, 1995 Abstract We consider the Renaming Problem, a basic processing step in string algorithms, for which we give a simultaneously

More information

Sorting (Chapter 9) Alexandre David B2-206

Sorting (Chapter 9) Alexandre David B2-206 Sorting (Chapter 9) Alexandre David B2-206 Sorting Problem Arrange an unordered collection of elements into monotonically increasing (or decreasing) order. Let S = . Sort S into S =

More information

7. Sorting I. 7.1 Simple Sorting. Problem. Algorithm: IsSorted(A) 1 i j n. Simple Sorting

7. Sorting I. 7.1 Simple Sorting. Problem. Algorithm: IsSorted(A) 1 i j n. Simple Sorting Simple Sorting 7. Sorting I 7.1 Simple Sorting Selection Sort, Insertion Sort, Bubblesort [Ottman/Widmayer, Kap. 2.1, Cormen et al, Kap. 2.1, 2.2, Exercise 2.2-2, Problem 2-2 19 197 Problem Algorithm:

More information

CPSC 311 Lecture Notes. Sorting and Order Statistics (Chapters 6-9)

CPSC 311 Lecture Notes. Sorting and Order Statistics (Chapters 6-9) CPSC 311 Lecture Notes Sorting and Order Statistics (Chapters 6-9) Acknowledgement: These notes are compiled by Nancy Amato at Texas A&M University. Parts of these course notes are based on notes from

More information

The PRAM model. A. V. Gerbessiotis CIS 485/Spring 1999 Handout 2 Week 2

The PRAM model. A. V. Gerbessiotis CIS 485/Spring 1999 Handout 2 Week 2 The PRAM model A. V. Gerbessiotis CIS 485/Spring 1999 Handout 2 Week 2 Introduction The Parallel Random Access Machine (PRAM) is one of the simplest ways to model a parallel computer. A PRAM consists of

More information

Question 7.11 Show how heapsort processes the input:

Question 7.11 Show how heapsort processes the input: Question 7.11 Show how heapsort processes the input: 142, 543, 123, 65, 453, 879, 572, 434, 111, 242, 811, 102. Solution. Step 1 Build the heap. 1.1 Place all the data into a complete binary tree in the

More information

Sorting Goodrich, Tamassia Sorting 1

Sorting Goodrich, Tamassia Sorting 1 Sorting Put array A of n numbers in increasing order. A core algorithm with many applications. Simple algorithms are O(n 2 ). Optimal algorithms are O(n log n). We will see O(n) for restricted input in

More information

IE 495 Lecture 3. Septermber 5, 2000

IE 495 Lecture 3. Septermber 5, 2000 IE 495 Lecture 3 Septermber 5, 2000 Reading for this lecture Primary Miller and Boxer, Chapter 1 Aho, Hopcroft, and Ullman, Chapter 1 Secondary Parberry, Chapters 3 and 4 Cosnard and Trystram, Chapter

More information

CS 561, Lecture 1. Jared Saia University of New Mexico

CS 561, Lecture 1. Jared Saia University of New Mexico CS 561, Lecture 1 Jared Saia University of New Mexico Quicksort Based on divide and conquer strategy Worst case is Θ(n 2 ) Expected running time is Θ(n log n) An In-place sorting algorithm Almost always

More information

Exercise 1 : B-Trees [ =17pts]

Exercise 1 : B-Trees [ =17pts] CS - Fall 003 Assignment Due : Thu November 7 (written part), Tue Dec 0 (programming part) Exercise : B-Trees [+++3+=7pts] 3 0 3 3 3 0 Figure : B-Tree. Consider the B-Tree of figure.. What are the values

More information

Parallel algorithms at ENS Lyon

Parallel algorithms at ENS Lyon Parallel algorithms at ENS Lyon Yves Robert Ecole Normale Supérieure de Lyon & Institut Universitaire de France TCPP Workshop February 2010 Yves.Robert@ens-lyon.fr February 2010 Parallel algorithms 1/

More information

The PRAM (Parallel Random Access Memory) model. All processors operate synchronously under the control of a common CPU.

The PRAM (Parallel Random Access Memory) model. All processors operate synchronously under the control of a common CPU. The PRAM (Parallel Random Access Memory) model All processors operate synchronously under the control of a common CPU. The PRAM (Parallel Random Access Memory) model All processors operate synchronously

More information

COMP Parallel Computing. PRAM (2) PRAM algorithm design techniques

COMP Parallel Computing. PRAM (2) PRAM algorithm design techniques COMP 633 - Parallel Computing Lecture 3 Aug 29, 2017 PRAM algorithm design techniques Reading for next class (Thu Aug 31): PRAM handout secns 3.6, 4.1, skim section 5. Written assignment 1 is posted, due

More information

Data Structures. Sorting. Haim Kaplan & Uri Zwick December 2013

Data Structures. Sorting. Haim Kaplan & Uri Zwick December 2013 Data Structures Sorting Haim Kaplan & Uri Zwick December 2013 1 Comparison based sorting key a 1 a 2 a n info Input: An array containing n items Keys belong to a totally ordered domain Two keys can be

More information

1 (15 points) LexicoSort

1 (15 points) LexicoSort CS161 Homework 2 Due: 22 April 2016, 12 noon Submit on Gradescope Handed out: 15 April 2016 Instructions: Please answer the following questions to the best of your ability. If you are asked to show your

More information

Parallel Systems Course: Chapter VIII. Sorting Algorithms. Kumar Chapter 9. Jan Lemeire ETRO Dept. November Parallel Sorting

Parallel Systems Course: Chapter VIII. Sorting Algorithms. Kumar Chapter 9. Jan Lemeire ETRO Dept. November Parallel Sorting Parallel Systems Course: Chapter VIII Sorting Algorithms Kumar Chapter 9 Jan Lemeire ETRO Dept. November 2014 Overview 1. Parallel sort distributed memory 2. Parallel sort shared memory 3. Sorting Networks

More information

Mergesort again. 1. Split the list into two equal parts

Mergesort again. 1. Split the list into two equal parts Quicksort Mergesort again 1. Split the list into two equal parts 5 3 9 2 8 7 3 2 1 4 5 3 9 2 8 7 3 2 1 4 Mergesort again 2. Recursively mergesort the two parts 5 3 9 2 8 7 3 2 1 4 2 3 5 8 9 1 2 3 4 7 Mergesort

More information

Parallel Models. Hypercube Butterfly Fully Connected Other Networks Shared Memory v.s. Distributed Memory SIMD v.s. MIMD

Parallel Models. Hypercube Butterfly Fully Connected Other Networks Shared Memory v.s. Distributed Memory SIMD v.s. MIMD Parallel Algorithms Parallel Models Hypercube Butterfly Fully Connected Other Networks Shared Memory v.s. Distributed Memory SIMD v.s. MIMD The PRAM Model Parallel Random Access Machine All processors

More information

What next? CSE332: Data Abstractions Lecture 20: Parallel Prefix and Parallel Sorting. The prefix-sum problem. Parallel prefix-sum

What next? CSE332: Data Abstractions Lecture 20: Parallel Prefix and Parallel Sorting. The prefix-sum problem. Parallel prefix-sum What next? CSE332: Data Abstractions Lecture 20: Parallel Prefix and Parallel Sorting Dan Grossman Spring 2010 Done: Simple ways to use parallelism for counting, summing, finding Even though in practice

More information

CSci 231 Final Review

CSci 231 Final Review CSci 231 Final Review Here is a list of topics for the final. Generally you are responsible for anything discussed in class (except topics that appear italicized), and anything appearing on the homeworks.

More information

Total Points: 60. Duration: 1hr

Total Points: 60. Duration: 1hr CS800 : Algorithms Fall 201 Nov 22, 201 Quiz 2 Practice Total Points: 0. Duration: 1hr 1. (,10) points Binary Heap. (a) The following is a sequence of elements presented to you (in order from left to right):

More information

II (Sorting and) Order Statistics

II (Sorting and) Order Statistics II (Sorting and) Order Statistics Heapsort Quicksort Sorting in Linear Time Medians and Order Statistics 8 Sorting in Linear Time The sorting algorithms introduced thus far are comparison sorts Any comparison

More information

We will show that the height of a RB tree on n vertices is approximately 2*log n. In class I presented a simple structural proof of this claim:

We will show that the height of a RB tree on n vertices is approximately 2*log n. In class I presented a simple structural proof of this claim: We have seen that the insert operation on a RB takes an amount of time proportional to the number of the levels of the tree (since the additional operations required to do any rebalancing require constant

More information

Parallel Systems Course: Chapter VIII. Sorting Algorithms. Kumar Chapter 9. Jan Lemeire ETRO Dept. Fall Parallel Sorting

Parallel Systems Course: Chapter VIII. Sorting Algorithms. Kumar Chapter 9. Jan Lemeire ETRO Dept. Fall Parallel Sorting Parallel Systems Course: Chapter VIII Sorting Algorithms Kumar Chapter 9 Jan Lemeire ETRO Dept. Fall 2017 Overview 1. Parallel sort distributed memory 2. Parallel sort shared memory 3. Sorting Networks

More information

Lecture 8 Parallel Algorithms II

Lecture 8 Parallel Algorithms II Lecture 8 Parallel Algorithms II Dr. Wilson Rivera ICOM 6025: High Performance Computing Electrical and Computer Engineering Department University of Puerto Rico Original slides from Introduction to Parallel

More information

COMP3121/3821/9101/ s1 Assignment 1

COMP3121/3821/9101/ s1 Assignment 1 Sample solutions to assignment 1 1. (a) Describe an O(n log n) algorithm (in the sense of the worst case performance) that, given an array S of n integers and another integer x, determines whether or not

More information

Lecture 5: Sorting Part A

Lecture 5: Sorting Part A Lecture 5: Sorting Part A Heapsort Running time O(n lg n), like merge sort Sorts in place (as insertion sort), only constant number of array elements are stored outside the input array at any time Combines

More information

Sorting and Selection

Sorting and Selection Sorting and Selection Introduction Divide and Conquer Merge-Sort Quick-Sort Radix-Sort Bucket-Sort 10-1 Introduction Assuming we have a sequence S storing a list of keyelement entries. The key of the element

More information

CSCE 750, Spring 2001 Notes 3 Page Symmetric Multi Processors (SMPs) (e.g., Cray vector machines, Sun Enterprise with caveats) Many processors

CSCE 750, Spring 2001 Notes 3 Page Symmetric Multi Processors (SMPs) (e.g., Cray vector machines, Sun Enterprise with caveats) Many processors CSCE 750, Spring 2001 Notes 3 Page 1 5 Parallel Algorithms 5.1 Basic Concepts With ordinary computers and serial (=sequential) algorithms, we have one processor and one memory. We count the number of operations

More information

Lecture 3. Recurrences / Heapsort

Lecture 3. Recurrences / Heapsort Lecture 3. Recurrences / Heapsort T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright

More information

Seminar on. Edge Coloring Series Parallel Graphs. Mohammmad Tawhidul Islam. Masters of Computer Science Summer Semester 2002 Matrikel Nr.

Seminar on. Edge Coloring Series Parallel Graphs. Mohammmad Tawhidul Islam. Masters of Computer Science Summer Semester 2002 Matrikel Nr. Seminar on Edge Coloring Series Parallel Graphs Mohammmad Tawhidul Islam Masters of Computer Science Summer Semester 2002 Matrikel Nr. 9003378 Fachhochschule Bonn-Rhein-Sieg Contents 1. Introduction. 2.

More information

Solutions. Suppose we insert all elements of U into the table, and let n(b) be the number of elements of U that hash to bucket b. Then.

Solutions. Suppose we insert all elements of U into the table, and let n(b) be the number of elements of U that hash to bucket b. Then. Assignment 3 1. Exercise [11.2-3 on p. 229] Modify hashing by chaining (i.e., bucketvector with BucketType = List) so that BucketType = OrderedList. How is the runtime of search, insert, and remove affected?

More information

THE EULER TOUR TECHNIQUE: EVALUATION OF TREE FUNCTIONS

THE EULER TOUR TECHNIQUE: EVALUATION OF TREE FUNCTIONS PARALLEL AND DISTRIBUTED ALGORITHMS BY DEBDEEP MUKHOPADHYAY AND ABHISHEK SOMANI http://cse.iitkgp.ac.in/~debdeep/courses_iitkgp/palgo/index.htm THE EULER TOUR TECHNIQUE: EVALUATION OF TREE FUNCTIONS 2

More information

Lecture 6 Sequences II. Parallel and Sequential Data Structures and Algorithms, (Fall 2013) Lectured by Danny Sleator 12 September 2013

Lecture 6 Sequences II. Parallel and Sequential Data Structures and Algorithms, (Fall 2013) Lectured by Danny Sleator 12 September 2013 Lecture 6 Sequences II Parallel and Sequential Data Structures and Algorithms, 15-210 (Fall 2013) Lectured by Danny Sleator 12 September 2013 Material in this lecture: Today s lecture is about reduction.

More information

CSC Design and Analysis of Algorithms

CSC Design and Analysis of Algorithms CSC 8301- Design and Analysis of Algorithms Lecture 6 Divide and Conquer Algorithm Design Technique Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide a problem instance into two

More information

Lecture 19 Sorting Goodrich, Tamassia

Lecture 19 Sorting Goodrich, Tamassia Lecture 19 Sorting 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 2004 Goodrich, Tamassia Outline Review 3 simple sorting algorithms: 1. selection Sort (in previous course) 2. insertion Sort (in previous

More information

Lecture 8: Mergesort / Quicksort Steven Skiena

Lecture 8: Mergesort / Quicksort Steven Skiena Lecture 8: Mergesort / Quicksort Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.stonybrook.edu/ skiena Problem of the Day Give an efficient

More information

DIVIDE AND CONQUER ALGORITHMS ANALYSIS WITH RECURRENCE EQUATIONS

DIVIDE AND CONQUER ALGORITHMS ANALYSIS WITH RECURRENCE EQUATIONS CHAPTER 11 SORTING ACKNOWLEDGEMENT: THESE SLIDES ARE ADAPTED FROM SLIDES PROVIDED WITH DATA STRUCTURES AND ALGORITHMS IN C++, GOODRICH, TAMASSIA AND MOUNT (WILEY 2004) AND SLIDES FROM NANCY M. AMATO AND

More information

An NC Algorithm for Sorting Real Numbers

An NC Algorithm for Sorting Real Numbers EPiC Series in Computing Volume 58, 2019, Pages 93 98 Proceedings of 34th International Conference on Computers and Their Applications An NC Algorithm for Sorting Real Numbers in O( nlogn loglogn ) Operations

More information

CSC Design and Analysis of Algorithms. Lecture 6. Divide and Conquer Algorithm Design Technique. Divide-and-Conquer

CSC Design and Analysis of Algorithms. Lecture 6. Divide and Conquer Algorithm Design Technique. Divide-and-Conquer CSC 8301- Design and Analysis of Algorithms Lecture 6 Divide and Conquer Algorithm Design Technique Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide a problem instance into two

More information

Problem Set 5 Solutions

Problem Set 5 Solutions Introduction to Algorithms November 4, 2005 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik D. Demaine and Charles E. Leiserson Handout 21 Problem Set 5 Solutions Problem 5-1. Skip

More information

Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, Merge Sort & Quick Sort

Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, Merge Sort & Quick Sort Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 2015 Merge Sort & Quick Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm

More information

would be included in is small: to be exact. Thus with probability1, the same partition n+1 n+1 would be produced regardless of whether p is in the inp

would be included in is small: to be exact. Thus with probability1, the same partition n+1 n+1 would be produced regardless of whether p is in the inp 1 Introduction 1.1 Parallel Randomized Algorihtms Using Sampling A fundamental strategy used in designing ecient algorithms is divide-and-conquer, where that input data is partitioned into several subproblems

More information

SORTING AND SELECTION

SORTING AND SELECTION 2 < > 1 4 8 6 = 9 CHAPTER 12 SORTING AND SELECTION ACKNOWLEDGEMENT: THESE SLIDES ARE ADAPTED FROM SLIDES PROVIDED WITH DATA STRUCTURES AND ALGORITHMS IN JAVA, GOODRICH, TAMASSIA AND GOLDWASSER (WILEY 2016)

More information

QuickSort

QuickSort QuickSort 7 4 9 6 2 2 4 6 7 9 4 2 2 4 7 9 7 9 2 2 9 9 1 QuickSort QuickSort on an input sequence S with n elements consists of three steps: n n n Divide: partition S into two sequences S 1 and S 2 of about

More information

Algorithms in Systems Engineering ISE 172. Lecture 12. Dr. Ted Ralphs

Algorithms in Systems Engineering ISE 172. Lecture 12. Dr. Ted Ralphs Algorithms in Systems Engineering ISE 172 Lecture 12 Dr. Ted Ralphs ISE 172 Lecture 12 1 References for Today s Lecture Required reading Chapter 6 References CLRS Chapter 7 D.E. Knuth, The Art of Computer

More information

Sorting Algorithms. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University

Sorting Algorithms. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University Sorting Algorithms CptS 223 Advanced Data Structures Larry Holder School of Electrical Engineering and Computer Science Washington State University 1 QuickSort Divide-and-conquer approach to sorting Like

More information

Chapter 1 Divide and Conquer Algorithm Theory WS 2013/14 Fabian Kuhn

Chapter 1 Divide and Conquer Algorithm Theory WS 2013/14 Fabian Kuhn Chapter 1 Divide and Conquer Algorithm Theory WS 2013/14 Fabian Kuhn Divide And Conquer Principle Important algorithm design method Examples from Informatik 2: Sorting: Mergesort, Quicksort Binary search

More information

Broadcast: Befo re 1

Broadcast: Befo re 1 Broadcast: Before 1 After 2 Spanning Tree ffl assume fixed spanning tree ffl asynchronous model 3 Processor State parent terminated children 4 Broadcast: Step One parent terminated children 5 Broadcast:Step

More information

A tion of a computing device and a set of primitive operations

A tion of a computing device and a set of primitive operations 1526 IEEE TRANSACTIONS ON COMPUTERS, VOL. 38, NO. 1 I, NOVEMBER 1989 Scans as Primitive Parallel Operations GUY E. BLELLOCH Abstmct- In most parallel random access machine (PRAM) models, memory references

More information

Parallel and Sequential Data Structures and Algorithms Lecture (Spring 2012) Lecture 16 Treaps; Augmented BSTs

Parallel and Sequential Data Structures and Algorithms Lecture (Spring 2012) Lecture 16 Treaps; Augmented BSTs Lecture 16 Treaps; Augmented BSTs Parallel and Sequential Data Structures and Algorithms, 15-210 (Spring 2012) Lectured by Margaret Reid-Miller 8 March 2012 Today: - More on Treaps - Ordered Sets and Tables

More information

A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 3 Parallel Prefix, Pack, and Sorting

A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 3 Parallel Prefix, Pack, and Sorting A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 3 Parallel Prefix, Pack, and Sorting Steve Wolfman, based on work by Dan Grossman LICENSE: This file is licensed under a Creative

More information

List Ranking. Chapter 4

List Ranking. Chapter 4 List Ranking Chapter 4 Problem on linked lists 2-level memory model List Ranking problem Given a (mono directional) linked list L of n items, compute the distance of each item from the tail of L. Id Succ

More information

Multiway Blockwise In-place Merging

Multiway Blockwise In-place Merging Multiway Blockwise In-place Merging Viliam Geffert and Jozef Gajdoš Institute of Computer Science, P.J.Šafárik University, Faculty of Science Jesenná 5, 041 54 Košice, Slovak Republic viliam.geffert@upjs.sk,

More information

Sorting Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar

Sorting Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar Sorting Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text ``Introduction to Parallel Computing'', Addison Wesley, 2003. Topic Overview Issues in Sorting on Parallel

More information

List Ranking. Chapter 4

List Ranking. Chapter 4 List Ranking Chapter 4 Problem on linked lists 2-level memory model List Ranking problem Given a (mono directional) linked list L of n items, compute the distance of each item from the tail of L. Id Succ

More information

Theory of Computing Systems 1999 Springer-Verlag New York Inc.

Theory of Computing Systems 1999 Springer-Verlag New York Inc. Theory Comput. Systems 32, 213 239 (1999) Theory of Computing Systems 1999 Springer-Verlag New York Inc. Pipelining with Futures G. E. Blelloch 1 and M. Reid-Miller 2 1 School of Computer Science, Carnegie

More information

1. Gusfield text for chapter 5&6 about suffix trees are scanned and uploaded on the web 2. List of Project ideas is uploaded

1. Gusfield text for chapter 5&6 about suffix trees are scanned and uploaded on the web 2. List of Project ideas is uploaded Date: Thursday, February 8 th Lecture: Dr. Mihai Pop Scribe: Hyoungtae Cho dministrivia. Gusfield text for chapter &6 about suffix trees are scanned and uploaded on the web. List of Project ideas is uploaded

More information

CPSC 311: Analysis of Algorithms (Honors) Exam 1 October 11, 2002

CPSC 311: Analysis of Algorithms (Honors) Exam 1 October 11, 2002 CPSC 311: Analysis of Algorithms (Honors) Exam 1 October 11, 2002 Name: Instructions: 1. This is a closed book exam. Do not use any notes or books, other than your 8.5-by-11 inch review sheet. Do not confer

More information

CSC Design and Analysis of Algorithms. Lecture 6. Divide and Conquer Algorithm Design Technique. Divide-and-Conquer

CSC Design and Analysis of Algorithms. Lecture 6. Divide and Conquer Algorithm Design Technique. Divide-and-Conquer CSC 8301- Design and Analysis of Algorithms Lecture 6 Divide and Conuer Algorithm Design Techniue Divide-and-Conuer The most-well known algorithm design strategy: 1. Divide a problem instance into two

More information

CS 506, Sect 002 Homework 5 Dr. David Nassimi Foundations of CS Due: Week 11, Mon. Apr. 7 Spring 2014

CS 506, Sect 002 Homework 5 Dr. David Nassimi Foundations of CS Due: Week 11, Mon. Apr. 7 Spring 2014 CS 506, Sect 002 Homework 5 Dr. David Nassimi Foundations of CS Due: Week 11, Mon. Apr. 7 Spring 2014 Study: Chapter 4 Analysis of Algorithms, Recursive Algorithms, and Recurrence Equations 1. Prove the

More information

Lecture Notes for Advanced Algorithms

Lecture Notes for Advanced Algorithms Lecture Notes for Advanced Algorithms Prof. Bernard Moret September 29, 2011 Notes prepared by Blanc, Eberle, and Jonnalagedda. 1 Average Case Analysis 1.1 Reminders on quicksort and tree sort We start

More information

Analysis of Algorithms - Greedy algorithms -

Analysis of Algorithms - Greedy algorithms - Analysis of Algorithms - Greedy algorithms - Andreas Ermedahl MRTC (Mälardalens Real-Time Reseach Center) andreas.ermedahl@mdh.se Autumn 2003 Greedy Algorithms Another paradigm for designing algorithms

More information

THE EULER TOUR TECHNIQUE: EVALUATION OF TREE FUNCTIONS

THE EULER TOUR TECHNIQUE: EVALUATION OF TREE FUNCTIONS PARALLEL AND DISTRIBUTED ALGORITHMS BY DEBDEEP MUKHOPADHYAY AND ABHISHEK SOMANI http://cse.iitkgp.ac.in/~debdeep/courses_iitkgp/palgo/index.htm THE EULER TOUR TECHNIQUE: EVALUATION OF TREE FUNCTIONS 2

More information

Problem. Input: An array A = (A[1],..., A[n]) with length n. Output: a permutation A of A, that is sorted: A [i] A [j] for all. 1 i j n.

Problem. Input: An array A = (A[1],..., A[n]) with length n. Output: a permutation A of A, that is sorted: A [i] A [j] for all. 1 i j n. Problem 5. Sorting Simple Sorting, Quicksort, Mergesort Input: An array A = (A[1],..., A[n]) with length n. Output: a permutation A of A, that is sorted: A [i] A [j] for all 1 i j n. 98 99 Selection Sort

More information