CS671 Parallel Programming in the Many-Core Era

Size: px
Start display at page:

Download "CS671 Parallel Programming in the Many-Core Era"

Transcription

1 1 CS671 Parallel Programming in the Many-Core Era Polyhedral Framework for Compilation: Polyhedral Model Representation, Data Dependence Analysis, Scheduling and Data Locality Optimizations December 3, 2013

2 Table of Contents 2 Polyhedral Program Optimizations Code to Model: Polyhedral Model Polyhedral Model Overview Modeling Dependence Transformations in the Model Classical Affine Transformations Affine Scheduling - One Dimensional Time Affine Scheduling - Multi-Dimensional Time Model To Code The Problem Definition Polyhedral Code Generation Approaches References

3 Polyhedral Program Optimization: A three-stage process 1. Analysis: from code to model Existing prototype tools PoCC (Clan-Candl-LetSee-Pluto-Cloog-Polylib-PIPLib-ISL-FM) URUK, Omega, Loopo GCC GRAPHITE Reservoir Labs R-Stream, IBM XL/Poly 2. Transformation in the model Build and select a program transformation 3. Code generation: from model to code Apply the transformation in the model Regenerate syntactic (AST-based) code 3

4 Lecture 7 Table of Contents 4 Polyhedral Program Optimizations Code to Model: Polyhedral Model Polyhedral Model Overview Modeling Dependence Transformations in the Model Classical Affine Transformations Affine Scheduling - One Dimensional Time Affine Scheduling - Multi-Dimensional Time Model To Code The Problem Definition Polyhedral Code Generation Approaches References

5 Motivation 5 An Example of Loop for ( i = 0; i < 3; ++i ) for ( j = 0; j < 3; ++j ) A[i][j] = i j; Loop execution: A[0][0]=0*0 A[0][1]=0*1 A[0][2]=0*2 A[1][0]=1*0 A[1][1]=1*1 A[1][2]=1*2 A[2][0]=2*0 A[2][1]=2*1 A[2][2]=2*2

6 Capture Loop Characteristics 6 What will be needed? Need to indicate total number of iterations Need to distinguish every different iteration Need to imply an order on these different iterations Challenges Compact (memory space consumption) Parametric loop bound / unbounded loop Non-unit loop stride Conditionals

7 Overview of Polyhedral Model 7 Iteration domain: A set of n-dimensional vectors Iteration Vector: x = (i, j) Iteration domain: the set of values of x Polytopes model sets of totally ordered n-dimensional vectors The set must be convex Example: Code for ( i=0; i < 5; i++ ) for ( j=0; j < 5; j++ )... endfor endfor Iter. Domain Graph J I

8 Convexity 8 Definition: Convex Set Let R be a vector space over the real numbers. A set S in R is convex iff, µ, λ S and given any t [0,1], the point (1 t) µ + t λ S In words: Drawing a line segment between any two points of S, each point of this segment is also in S.

9 Convexity in Loop Iteration Domains 9 A statement surrounded by loops with unit-stride, no conditional and constant loop bounds has a convex iteration domain, if the iteration domain exists. In general, in loops for ( i=0; i < N; i++ )... The polytope set must be convex Convexity is the central concept of polyhedral optimization J J I I

10 Affine Function I 10 Definition A function f : K m K n is affine if there exists a vector b K n and matrix A K n m such that: x K m, f ( x) = A x + b Affine half-space definition An affine half-space of K m is defined as the set of points: { x K m a x b}

11 Affine Function II x 2 Example of an affine half-space: 11 3x 1 + 2x x 1 + 2x 2 11 (0, 0) x 1 3x 1 + 2x 2 = 11

12 Polyhedron I 12 Polyhedron definition A set S K n is a polyhedron if there exists a system of a finite number of inequalities A x b such that: P = { x K n A x b} Equivalently, it is the intersection of finitely many half-spaces.

13 Polyhedron II Example of a polyhedron generated by intersection of three half-spaces: 13 x 2 3x 1 + 4x 2 = 4 2x 1 x 2 = 5 (0, 0) x 1 3x 1 + 2x 2 = 11

14 Polyhedron III A polytope is a bounded polyhedron Integer Polyhedron (Z-polyhedron) It is a polyhedron where all its extreme points are integer valued 14 Integer Hull The integer hull of a rational polyhedron P is the largest set of integer points such that each of these points is in P.

15 Polyhedron Example 15 Example of a Polytope and an Integer Hull x 2 3x 1 + 4x 2 = 4 3x 1 + 2x 2 = 11 2x 1 x 2 = 5 (0, 0) x 1

16 Polyhedron Example 15 Example of a Polytope and an Integer Hull x 2 3x 1 + 4x 2 = 4 3x 1 + 2x 2 = 11 2x 1 x 2 = 5 (0, 0) x 1

17 Polyhedron Example 15 Example of a Polytope and an Integer Hull x 2 3x 1 + 4x 2 = 4 3x 1 + 2x 2 = 11 2x 1 x 2 = 5 (0, 0) x 1

18 Define Affine Loops A special class of loop: The lowerbound and upper bound of the loop are expressed as affine expressions of the surrounding loop variables and symbolic constants. 16 Given do i 1 = L 1, U 1... do i n S 1 :... S 2 :... = L n, U n where L m = f m (i 1, i 2,..., i m 1 ) and f m (...) are all affine functions of variables (i 1,..., i m 1 ).

19 Modeling Affine Loop Iteration Domains Corresponding loop constructs: unit stride, no conditionals Polytope dimension: number of surrounding loops Constraints: set by loop bounds Let s all formulate it in this way: A * x - b 0 Add symbolic constants if necessary 17 Previous Loop Example for ( i = 0; i 5; ++i ) for ( j = 0; j 5; ++j ) A[i][j] = i j; D R : ( i j ) = 0 i 5 0 j i j 1 0

20 Lexicographical Order of Affine Index Given I = (i 1, i 2,..., i n ) and I = (i 1, i 2,..., i n), I < I iff (i 1, i 2,..., i k ) = (i 1, i 2,..., i k ) & i k+1 < i k+1 An Example of Loop for ( i = 0; i < 3; ++i ) for ( j = 0; j < 3; ++j ) A[i][j] = i j; Loop execution: 18 (0,0) (0,1) (0,2) (1,0) (1,1) (1,2)...

21 Another Iteration Space Example for ( i = 0; i 4; ++i ) for ( j = i + 1; j 10 - i; ++j ) A[i][j] = i j; Inequalities set by loop bounds: i 0 i 4 i 4 j i + 1 j 10 i j 10 + i 19 D R : ( i j ) = i j 1 0

22 The loop iteration space in graph: 20 Inequalities set by loop bounds: i 0 i 4 j i + 1 j 10 i i j i = 1 i = 4 (0, 0) j j + i = 10

23 Scan the Loop Iteration Space 21 Loop code generation: Using the lexicographical order: Assume no dependence: what if we want to sweep the space using another order? i j i = 1 i = 4 (0, 0) j j + i = 10 Scan the iteration space from loop i first. What will be the loop bounds?

24 Fourier-Motzkin elimination: Input: A polyhedron S with variables x 1, x 2, x 3,..., x n. Output: A polyhedron S with variables x 1, x 2,..., x m 1, x m+1,..., x n Generally speaking, find a projection onto dimensions other than x m Take all pairs of inequalities with opposite sign coefficients of x m, and for each generate a new valid inequality that eliminates x m Also take all inequalities from the original set which do not depend on x m Fourier-Motzkin Theorem: The above collection of inequalities defines exactly the projection of S onto x m = 0. 22

25 Assignment (2) I More examples: Example I 23 for ( i = 10; i 1000; ++i ) for ( j = i; j i + 10; ++j ) A[i,j] = 0; Example II for ( i = 0; i 100; ++i ) for ( j = 0; j i+100; ++j ) for ( k = i+j; k i - j; k++ ) X[i, j, k] = 0;

26 Assignment (2) II Example III 24 for ( i = 0; i < N; ++i ) for ( j = 0; j < i; ++j ) if ( i > M ) A[j] = 0;

27 More complicated loops I Unbounded domains: use polyhedra! Parametric loop bounds: use parametric polyhedra! Non-unit loop bounds: normalize the loop! Conditionals: Those which preserve convexity Problem remain for those which do not preserve convexity 25

28 Examples 26 Example I for ( i = 0; i < N; i += 2 ) for ( j = 0; j < N; ++j ) A[i] = 0; Example II for ( i = 0; i < N; i += 2 ) for ( j = 0; j < N; ++j ) if ( i % 3 == 1 && j % 5 == 0 ) A[i] = 0;

29 Generalized Conditionals 27 Different categories: Conjunctions ( a && b ) Disjunctions ( a b ) Non-affine ( i * j < 2 ) Data-dependent ( a[i] == 0 )

30 Relation with Operations on Polyhedra 28 Conjunction: Definition: intersection The intersection of two convex sets P 1 and P 2 is a convex set P. P = { x K m x P 1 x P 2 } Disjunction: Definition: union The union of two convex sets P 1 and P 2 is a set P: P = { x K m x P 1 x P 2 } The union of two convex sets may not be a convex set.

Tiling: A Data Locality Optimizing Algorithm

Tiling: A Data Locality Optimizing Algorithm Tiling: A Data Locality Optimizing Algorithm Previously Performance analysis of existing codes Data dependence analysis for detecting parallelism Specifying transformations using frameworks Today Usefulness

More information

Polyhedral Compilation Foundations

Polyhedral Compilation Foundations Polyhedral Compilation Foundations Louis-Noël Pouchet pouchet@cse.ohio-state.edu Dept. of Computer Science and Engineering, the Ohio State University Feb 22, 2010 888.11, Class #5 Introduction: Polyhedral

More information

The Polyhedral Model (Transformations)

The Polyhedral Model (Transformations) The Polyhedral Model (Transformations) Announcements HW4 is due Wednesday February 22 th Project proposal is due NEXT Friday, extension so that example with tool is possible (see resources website for

More information

Math 414 Lecture 2 Everyone have a laptop?

Math 414 Lecture 2 Everyone have a laptop? Math 44 Lecture 2 Everyone have a laptop? THEOREM. Let v,...,v k be k vectors in an n-dimensional space and A = [v ;...; v k ] v,..., v k independent v,..., v k span the space v,..., v k a basis v,...,

More information

Polyhedral Operations. Algorithms needed for automation. Logistics

Polyhedral Operations. Algorithms needed for automation. Logistics Polyhedral Operations Logistics Intermediate reports late deadline is Friday March 30 at midnight HW6 (posted) and HW7 (posted) due April 5 th Tuesday April 4 th, help session during class with Manaf,

More information

Iterative Optimization in the Polyhedral Model: Part I, One-Dimensional Time

Iterative Optimization in the Polyhedral Model: Part I, One-Dimensional Time Iterative Optimization in the Polyhedral Model: Part I, One-Dimensional Time Louis-Noël Pouchet, Cédric Bastoul, Albert Cohen and Nicolas Vasilache ALCHEMY, INRIA Futurs / University of Paris-Sud XI March

More information

Program transformations and optimizations in the polyhedral framework

Program transformations and optimizations in the polyhedral framework Program transformations and optimizations in the polyhedral framework Louis-Noël Pouchet Dept. of Computer Science UCLA May 14, 2013 1st Polyhedral Spring School St-Germain au Mont d or, France : Overview

More information

Loop Transformations, Dependences, and Parallelization

Loop Transformations, Dependences, and Parallelization Loop Transformations, Dependences, and Parallelization Announcements HW3 is due Wednesday February 15th Today HW3 intro Unimodular framework rehash with edits Skewing Smith-Waterman (the fix is in!), composing

More information

The Challenges of Non-linear Parameters and Variables in Automatic Loop Parallelisation

The Challenges of Non-linear Parameters and Variables in Automatic Loop Parallelisation The Challenges of Non-linear Parameters and Variables in Automatic Loop Parallelisation Armin Größlinger December 2, 2009 Rigorosum Fakultät für Informatik und Mathematik Universität Passau Automatic Loop

More information

maximize c, x subject to Ax b,

maximize c, x subject to Ax b, Lecture 8 Linear programming is about problems of the form maximize c, x subject to Ax b, where A R m n, x R n, c R n, and b R m, and the inequality sign means inequality in each row. The feasible set

More information

C&O 355 Lecture 16. N. Harvey

C&O 355 Lecture 16. N. Harvey C&O 355 Lecture 16 N. Harvey Topics Review of Fourier-Motzkin Elimination Linear Transformations of Polyhedra Convex Combinations Convex Hulls Polytopes & Convex Hulls Fourier-Motzkin Elimination Joseph

More information

Iterative Optimization in the Polyhedral Model

Iterative Optimization in the Polyhedral Model Iterative Optimization in the Polyhedral Model Louis-Noël Pouchet, INRIA Saclay / University of Paris-Sud 11, France January 18th, 2010 Ph.D Defense Introduction: A Brief History... A Quick look backward:

More information

The Polyhedral Compilation Framework

The Polyhedral Compilation Framework The Polyhedral Compilation Framework Louis-Noël Pouchet Dept. of Computer Science and Engineering Ohio State University pouchet@cse.ohio-state.edu October 20, 2011 Introduction: Overview of Today s Lecture

More information

Modeling and Analysis of Hybrid Systems

Modeling and Analysis of Hybrid Systems Modeling and Analysis of Hybrid Systems Convex polyhedra Prof. Dr. Erika Ábrahám Informatik 2 - LuFG Theory of Hybrid Systems RWTH Aachen University Szeged, Hungary, 27 September - 06 October 2017 Ábrahám

More information

Modeling and Analysis of Hybrid Systems

Modeling and Analysis of Hybrid Systems Modeling and Analysis of Hybrid Systems 6. Convex polyhedra Prof. Dr. Erika Ábrahám Informatik 2 - LuFG Theory of Hybrid Systems RWTH Aachen University Szeged, Hungary, 27 September - 06 October 2017 Ábrahám

More information

COMP331/557. Chapter 2: The Geometry of Linear Programming. (Bertsimas & Tsitsiklis, Chapter 2)

COMP331/557. Chapter 2: The Geometry of Linear Programming. (Bertsimas & Tsitsiklis, Chapter 2) COMP331/557 Chapter 2: The Geometry of Linear Programming (Bertsimas & Tsitsiklis, Chapter 2) 49 Polyhedra and Polytopes Definition 2.1. Let A 2 R m n and b 2 R m. a set {x 2 R n A x b} is called polyhedron

More information

Linear programming and duality theory

Linear programming and duality theory Linear programming and duality theory Complements of Operations Research Giovanni Righini Linear Programming (LP) A linear program is defined by linear constraints, a linear objective function. Its variables

More information

The Polyhedral Model (Dependences and Transformations)

The Polyhedral Model (Dependences and Transformations) The Polyhedral Model (Dependences and Transformations) Announcements HW3 is due Wednesday February 15 th, tomorrow Project proposal is due NEXT Friday, extension so that example with tool is possible Today

More information

Applied Integer Programming

Applied Integer Programming Applied Integer Programming D.S. Chen; R.G. Batson; Y. Dang Fahimeh 8.2 8.7 April 21, 2015 Context 8.2. Convex sets 8.3. Describing a bounded polyhedron 8.4. Describing unbounded polyhedron 8.5. Faces,

More information

Convex Geometry arising in Optimization

Convex Geometry arising in Optimization Convex Geometry arising in Optimization Jesús A. De Loera University of California, Davis Berlin Mathematical School Summer 2015 WHAT IS THIS COURSE ABOUT? Combinatorial Convexity and Optimization PLAN

More information

Combinatorial Geometry & Topology arising in Game Theory and Optimization

Combinatorial Geometry & Topology arising in Game Theory and Optimization Combinatorial Geometry & Topology arising in Game Theory and Optimization Jesús A. De Loera University of California, Davis LAST EPISODE... We discuss the content of the course... Convex Sets A set is

More information

Lecture 3. Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets. Tepper School of Business Carnegie Mellon University, Pittsburgh

Lecture 3. Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets. Tepper School of Business Carnegie Mellon University, Pittsburgh Lecture 3 Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets Gérard Cornuéjols Tepper School of Business Carnegie Mellon University, Pittsburgh January 2016 Mixed Integer Linear Programming

More information

Chapter 4 Concepts from Geometry

Chapter 4 Concepts from Geometry Chapter 4 Concepts from Geometry An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Line Segments The line segment between two points and in R n is the set of points on the straight line joining

More information

Polyhedral Compilation Foundations

Polyhedral Compilation Foundations Polyhedral Compilation Foundations Louis-Noël Pouchet pouchet@cse.ohio-state.edu Dept. of Computer Science and Engineering, the Ohio State University Feb 15, 2010 888.11, Class #4 Introduction: Polyhedral

More information

Integer Programming Theory

Integer Programming Theory Integer Programming Theory Laura Galli October 24, 2016 In the following we assume all functions are linear, hence we often drop the term linear. In discrete optimization, we seek to find a solution x

More information

POLYHEDRAL GEOMETRY. Convex functions and sets. Mathematical Programming Niels Lauritzen Recall that a subset C R n is convex if

POLYHEDRAL GEOMETRY. Convex functions and sets. Mathematical Programming Niels Lauritzen Recall that a subset C R n is convex if POLYHEDRAL GEOMETRY Mathematical Programming Niels Lauritzen 7.9.2007 Convex functions and sets Recall that a subset C R n is convex if {λx + (1 λ)y 0 λ 1} C for every x, y C and 0 λ 1. A function f :

More information

Computing the Integer Points of a Polyhedron

Computing the Integer Points of a Polyhedron Computing the Integer Points of a Polyhedron Complexity Estimates Rui-Juan Jing 1,2 and Marc Moreno Maza 2,3 1 Key Laboratoty of Mathematics Mechnization, Academy of Mathematics and Systems Science, Chinese

More information

Lecture 4: Rational IPs, Polyhedron, Decomposition Theorem

Lecture 4: Rational IPs, Polyhedron, Decomposition Theorem IE 5: Integer Programming, Spring 29 24 Jan, 29 Lecture 4: Rational IPs, Polyhedron, Decomposition Theorem Lecturer: Karthik Chandrasekaran Scribe: Setareh Taki Disclaimer: These notes have not been subjected

More information

Lecture 9 Basic Parallelization

Lecture 9 Basic Parallelization Lecture 9 Basic Parallelization I. Introduction II. Data Dependence Analysis III. Loop Nests + Locality IV. Interprocedural Parallelization Chapter 11.1-11.1.4 CS243: Parallelization 1 Machine Learning

More information

Lecture 9 Basic Parallelization

Lecture 9 Basic Parallelization Lecture 9 Basic Parallelization I. Introduction II. Data Dependence Analysis III. Loop Nests + Locality IV. Interprocedural Parallelization Chapter 11.1-11.1.4 CS243: Parallelization 1 Machine Learning

More information

Convex Hull Representation Conversion (cddlib, lrslib)

Convex Hull Representation Conversion (cddlib, lrslib) Convex Hull Representation Conversion (cddlib, lrslib) Student Seminar in Combinatorics: Mathematical Software Niklas Pfister October 31, 2014 1 Introduction In this report we try to give a short overview

More information

6.189 IAP Lecture 11. Parallelizing Compilers. Prof. Saman Amarasinghe, MIT IAP 2007 MIT

6.189 IAP Lecture 11. Parallelizing Compilers. Prof. Saman Amarasinghe, MIT IAP 2007 MIT 6.189 IAP 2007 Lecture 11 Parallelizing Compilers 1 6.189 IAP 2007 MIT Outline Parallel Execution Parallelizing Compilers Dependence Analysis Increasing Parallelization Opportunities Generation of Parallel

More information

Linear Programming in Small Dimensions

Linear Programming in Small Dimensions Linear Programming in Small Dimensions Lekcija 7 sergio.cabello@fmf.uni-lj.si FMF Univerza v Ljubljani Edited from slides by Antoine Vigneron Outline linear programming, motivation and definition one dimensional

More information

In this chapter we introduce some of the basic concepts that will be useful for the study of integer programming problems.

In this chapter we introduce some of the basic concepts that will be useful for the study of integer programming problems. 2 Basics In this chapter we introduce some of the basic concepts that will be useful for the study of integer programming problems. 2.1 Notation Let A R m n be a matrix with row index set M = {1,...,m}

More information

Integer Programming Chapter 9

Integer Programming Chapter 9 Integer Programming Chapter 9 University of Chicago Booth School of Business Kipp Martin October 25, 2017 1 / 40 Outline Key Concepts MILP Set Monoids LP set Relaxation of MILP Set Formulation Quality

More information

Legal and impossible dependences

Legal and impossible dependences Transformations and Dependences 1 operations, column Fourier-Motzkin elimination us use these tools to determine (i) legality of permutation and Let generation of transformed code. (ii) Recall: Polyhedral

More information

Data Dependences and Parallelization

Data Dependences and Parallelization Data Dependences and Parallelization 1 Agenda Introduction Single Loop Nested Loops Data Dependence Analysis 2 Motivation DOALL loops: loops whose iterations can execute in parallel for i = 11, 20 a[i]

More information

Louis-Noël Pouchet

Louis-Noël Pouchet Internship Report, jointly EPITA CSI 2006 and UPS Master 2 (March - September 2006) Louis-Noël Pouchet WHEN ITERATIVE OPTIMIZATION MEETS THE POLYHEDRAL MODEL: ONE-DIMENSIONAL

More information

CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets. Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets. Instructor: Shaddin Dughmi CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets Instructor: Shaddin Dughmi Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations

More information

FACES OF CONVEX SETS

FACES OF CONVEX SETS FACES OF CONVEX SETS VERA ROSHCHINA Abstract. We remind the basic definitions of faces of convex sets and their basic properties. For more details see the classic references [1, 2] and [4] for polytopes.

More information

Advanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs

Advanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs Advanced Operations Research Techniques IE316 Quiz 1 Review Dr. Ted Ralphs IE316 Quiz 1 Review 1 Reading for The Quiz Material covered in detail in lecture. 1.1, 1.4, 2.1-2.6, 3.1-3.3, 3.5 Background material

More information

Applications of Linear Programming

Applications of Linear Programming Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 1 Why LP? Linear programming (LP, also called linear

More information

Theory of Integers. CS389L: Automated Logical Reasoning. Lecture 13: The Omega Test. Overview of Techniques. Geometric Description

Theory of Integers. CS389L: Automated Logical Reasoning. Lecture 13: The Omega Test. Overview of Techniques. Geometric Description Theory of ntegers This lecture: Decision procedure for qff theory of integers CS389L: Automated Logical Reasoning Lecture 13: The Omega Test şıl Dillig As in previous two lectures, we ll consider T Z formulas

More information

Minimizing Memory Strides using Integer Transformations of Parametric Polytopes

Minimizing Memory Strides using Integer Transformations of Parametric Polytopes Minimizing Memory Strides using Integer Transformations of Parametric Polytopes Rachid Seghir, Vincent Loechner and Benoît Meister ICPS/LSIIT UMR CNRS-ULP 7005 Pôle API, Bd Sébastien Brant F-67400 Illkirch

More information

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets. Instructor: Shaddin Dughmi

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets. Instructor: Shaddin Dughmi CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets Instructor: Shaddin Dughmi Announcements New room: KAP 158 Today: Convex Sets Mostly from Boyd and Vandenberghe. Read all of

More information

MATH 890 HOMEWORK 2 DAVID MEREDITH

MATH 890 HOMEWORK 2 DAVID MEREDITH MATH 890 HOMEWORK 2 DAVID MEREDITH (1) Suppose P and Q are polyhedra. Then P Q is a polyhedron. Moreover if P and Q are polytopes then P Q is a polytope. The facets of P Q are either F Q where F is a facet

More information

R n a T i x = b i} is a Hyperplane.

R n a T i x = b i} is a Hyperplane. Geometry of LPs Consider the following LP : min {c T x a T i x b i The feasible region is i =1,...,m}. X := {x R n a T i x b i i =1,...,m} = m i=1 {x Rn a T i x b i} }{{} X i The set X i is a Half-space.

More information

Computing and Informatics, Vol. 36, 2017, , doi: /cai

Computing and Informatics, Vol. 36, 2017, , doi: /cai Computing and Informatics, Vol. 36, 2017, 566 596, doi: 10.4149/cai 2017 3 566 NESTED-LOOPS TILING FOR PARALLELIZATION AND LOCALITY OPTIMIZATION Saeed Parsa, Mohammad Hamzei Department of Computer Engineering

More information

ORIE 6300 Mathematical Programming I September 2, Lecture 3

ORIE 6300 Mathematical Programming I September 2, Lecture 3 ORIE 6300 Mathematical Programming I September 2, 2014 Lecturer: David P. Williamson Lecture 3 Scribe: Divya Singhvi Last time we discussed how to take dual of an LP in two different ways. Today we will

More information

Array Dependence Analysis as Integer Constraints. Array Dependence Analysis Example. Array Dependence Analysis as Integer Constraints, cont

Array Dependence Analysis as Integer Constraints. Array Dependence Analysis Example. Array Dependence Analysis as Integer Constraints, cont Theory of Integers CS389L: Automated Logical Reasoning Omega Test Işıl Dillig Earlier, we talked aout the theory of integers T Z Signature of T Z : Σ Z : {..., 2, 1, 0, 1, 2,..., 3, 2, 2, 3,..., +,, =,

More information

15.082J and 6.855J. Lagrangian Relaxation 2 Algorithms Application to LPs

15.082J and 6.855J. Lagrangian Relaxation 2 Algorithms Application to LPs 15.082J and 6.855J Lagrangian Relaxation 2 Algorithms Application to LPs 1 The Constrained Shortest Path Problem (1,10) 2 (1,1) 4 (2,3) (1,7) 1 (10,3) (1,2) (10,1) (5,7) 3 (12,3) 5 (2,2) 6 Find the shortest

More information

Advanced Operations Research Techniques IE316. Quiz 2 Review. Dr. Ted Ralphs

Advanced Operations Research Techniques IE316. Quiz 2 Review. Dr. Ted Ralphs Advanced Operations Research Techniques IE316 Quiz 2 Review Dr. Ted Ralphs IE316 Quiz 2 Review 1 Reading for The Quiz Material covered in detail in lecture Bertsimas 4.1-4.5, 4.8, 5.1-5.5, 6.1-6.3 Material

More information

Convex Optimization Lecture 2

Convex Optimization Lecture 2 Convex Optimization Lecture 2 Today: Convex Analysis Center-of-mass Algorithm 1 Convex Analysis Convex Sets Definition: A set C R n is convex if for all x, y C and all 0 λ 1, λx + (1 λ)y C Operations that

More information

Lecture 12: Feasible direction methods

Lecture 12: Feasible direction methods Lecture 12 Lecture 12: Feasible direction methods Kin Cheong Sou December 2, 2013 TMA947 Lecture 12 Lecture 12: Feasible direction methods 1 / 1 Feasible-direction methods, I Intro Consider the problem

More information

Basic Algorithms for Periodic-Linear Inequalities and Integer Polyhedra

Basic Algorithms for Periodic-Linear Inequalities and Integer Polyhedra Basic Algorithms for Periodic-Linear Inequalities and Integer Polyhedra Alain Ketterlin / Camus IMPACT 2018: January, 23, 2018 Motivation Periodic-Linear Inequalities The Omicron Test Decomposition Motivation

More information

The Chvátal-Gomory Closure of a Strictly Convex Body is a Rational Polyhedron

The Chvátal-Gomory Closure of a Strictly Convex Body is a Rational Polyhedron The Chvátal-Gomory Closure of a Strictly Convex Body is a Rational Polyhedron Juan Pablo Vielma Joint work with Daniel Dadush and Santanu S. Dey July, Atlanta, GA Outline Introduction Proof: Step Step

More information

CS 293S Parallelism and Dependence Theory

CS 293S Parallelism and Dependence Theory CS 293S Parallelism and Dependence Theory Yufei Ding Reference Book: Optimizing Compilers for Modern Architecture by Allen & Kennedy Slides adapted from Louis-Noël Pouche, Mary Hall End of Moore's Law

More information

CS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension

CS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension CS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension Antoine Vigneron King Abdullah University of Science and Technology November 7, 2012 Antoine Vigneron (KAUST) CS 372 Lecture

More information

Lesson 17. Geometry and Algebra of Corner Points

Lesson 17. Geometry and Algebra of Corner Points SA305 Linear Programming Spring 2016 Asst. Prof. Nelson Uhan 0 Warm up Lesson 17. Geometry and Algebra of Corner Points Example 1. Consider the system of equations 3 + 7x 3 = 17 + 5 = 1 2 + 11x 3 = 24

More information

Discrete Optimization 2010 Lecture 5 Min-Cost Flows & Total Unimodularity

Discrete Optimization 2010 Lecture 5 Min-Cost Flows & Total Unimodularity Discrete Optimization 2010 Lecture 5 Min-Cost Flows & Total Unimodularity Marc Uetz University of Twente m.uetz@utwente.nl Lecture 5: sheet 1 / 26 Marc Uetz Discrete Optimization Outline 1 Min-Cost Flows

More information

DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini

DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 3. 4. Standard Form Basic Feasible Solutions

More information

arxiv: v1 [math.co] 15 Dec 2009

arxiv: v1 [math.co] 15 Dec 2009 ANOTHER PROOF OF THE FACT THAT POLYHEDRAL CONES ARE FINITELY GENERATED arxiv:092.2927v [math.co] 5 Dec 2009 VOLKER KAIBEL Abstract. In this note, we work out a simple inductive proof showing that every

More information

Integer Programming ISE 418. Lecture 1. Dr. Ted Ralphs

Integer Programming ISE 418. Lecture 1. Dr. Ted Ralphs Integer Programming ISE 418 Lecture 1 Dr. Ted Ralphs ISE 418 Lecture 1 1 Reading for This Lecture N&W Sections I.1.1-I.1.4 Wolsey Chapter 1 CCZ Chapter 2 ISE 418 Lecture 1 2 Mathematical Optimization Problems

More information

UNIVERSITÉ DE PARIS-SUD 11 U.F.R. SCIENTIFIQUE D ORSAY

UNIVERSITÉ DE PARIS-SUD 11 U.F.R. SCIENTIFIQUE D ORSAY UNIVERSITÉ DE PARIS-SUD 11 U.F.R. SCIENTIFIQUE D ORSAY In Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Specialty: Computer Science Louis-Noël POUCHET Subject: ITERATIVE

More information

A Crash Course in Compilers for Parallel Computing. Mary Hall Fall, L2: Transforms, Reuse, Locality

A Crash Course in Compilers for Parallel Computing. Mary Hall Fall, L2: Transforms, Reuse, Locality A Crash Course in Compilers for Parallel Computing Mary Hall Fall, 2008 1 Overview of Crash Course L1: Data Dependence Analysis and Parallelization (Oct. 30) L2 & L3: Loop Reordering Transformations, Reuse

More information

Alan LaMielle, Michelle Strout Colorado State University March 16, Technical Report CS

Alan LaMielle, Michelle Strout Colorado State University March 16, Technical Report CS Computer Science Technical Report Enabling Code Generation within the Sparse Polyhedral Framework Alan LaMielle, Michelle Strout Colorado State University {lamielle,mstrout@cs.colostate.edu March 16, 2010

More information

Loop Nest Optimizer of GCC. Sebastian Pop. Avgust, 2006

Loop Nest Optimizer of GCC. Sebastian Pop. Avgust, 2006 Loop Nest Optimizer of GCC CRI / Ecole des mines de Paris Avgust, 26 Architecture of GCC and Loop Nest Optimizer C C++ Java F95 Ada GENERIC GIMPLE Analyses aliasing data dependences number of iterations

More information

Systems of Inequalities

Systems of Inequalities Systems of Inequalities 1 Goals: Given system of inequalities of the form Ax b determine if system has an integer solution enumerate all integer solutions 2 Running example: Upper bounds for x: (2)and

More information

TOBIAS GROSSER Parkas Group, Computer Science Department, Ècole Normale Supèrieure / INRIA 45 Rue d Ulm, Paris, 75005, France

TOBIAS GROSSER Parkas Group, Computer Science Department, Ècole Normale Supèrieure / INRIA 45 Rue d Ulm, Paris, 75005, France Parallel Processing Letters c World Scientific Publishing Company POLLY PERFORMING POLYHEDRAL OPTIMIZATIONS ON A LOW-LEVEL INTERMEDIATE REPRESENTATION TOBIAS GROSSER Parkas Group, Computer Science Department,

More information

Conic Duality. yyye

Conic Duality.  yyye Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 1 Conic Duality Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/

More information

An Overview to. Polyhedral Model. Fangzhou Jiao

An Overview to. Polyhedral Model. Fangzhou Jiao An Overview to Polyhedral Model Fangzhou Jiao Polyhedral Model A framework for performing loop transformation Loop representation: using polytopes to achieve fine-grain representation of program Loop transformation:

More information

Linear Optimization. Andongwisye John. November 17, Linkoping University. Andongwisye John (Linkoping University) November 17, / 25

Linear Optimization. Andongwisye John. November 17, Linkoping University. Andongwisye John (Linkoping University) November 17, / 25 Linear Optimization Andongwisye John Linkoping University November 17, 2016 Andongwisye John (Linkoping University) November 17, 2016 1 / 25 Overview 1 Egdes, One-Dimensional Faces, Adjacency of Extreme

More information

4 LINEAR PROGRAMMING (LP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1

4 LINEAR PROGRAMMING (LP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 4 LINEAR PROGRAMMING (LP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 Mathematical programming (optimization) problem: min f (x) s.t. x X R n set of feasible solutions with linear objective function

More information

Mathematical and Algorithmic Foundations Linear Programming and Matchings

Mathematical and Algorithmic Foundations Linear Programming and Matchings Adavnced Algorithms Lectures Mathematical and Algorithmic Foundations Linear Programming and Matchings Paul G. Spirakis Department of Computer Science University of Patras and Liverpool Paul G. Spirakis

More information

Lecture 2 - Introduction to Polytopes

Lecture 2 - Introduction to Polytopes Lecture 2 - Introduction to Polytopes Optimization and Approximation - ENS M1 Nicolas Bousquet 1 Reminder of Linear Algebra definitions Let x 1,..., x m be points in R n and λ 1,..., λ m be real numbers.

More information

Cuts from Proofs: A Complete and Practical Technique for Solving Linear Inequalities over Integers

Cuts from Proofs: A Complete and Practical Technique for Solving Linear Inequalities over Integers Cuts from Proofs: A Complete and Practical Technique for Solving Linear Inequalities over Integers Isil Dillig, Thomas Dillig, and Alex Aiken Computer Science Department Stanford University Linear Arithmetic

More information

Lecture 2. Topology of Sets in R n. August 27, 2008

Lecture 2. Topology of Sets in R n. August 27, 2008 Lecture 2 Topology of Sets in R n August 27, 2008 Outline Vectors, Matrices, Norms, Convergence Open and Closed Sets Special Sets: Subspace, Affine Set, Cone, Convex Set Special Convex Sets: Hyperplane,

More information

ACTUALLY DOING IT : an Introduction to Polyhedral Computation

ACTUALLY DOING IT : an Introduction to Polyhedral Computation ACTUALLY DOING IT : an Introduction to Polyhedral Computation Jesús A. De Loera Department of Mathematics Univ. of California, Davis http://www.math.ucdavis.edu/ deloera/ 1 What is a Convex Polytope? 2

More information

Submodularity Reading Group. Matroid Polytopes, Polymatroid. M. Pawan Kumar

Submodularity Reading Group. Matroid Polytopes, Polymatroid. M. Pawan Kumar Submodularity Reading Group Matroid Polytopes, Polymatroid M. Pawan Kumar http://www.robots.ox.ac.uk/~oval/ Outline Linear Programming Matroid Polytopes Polymatroid Polyhedron Ax b A : m x n matrix b:

More information

Polar Duality and Farkas Lemma

Polar Duality and Farkas Lemma Lecture 3 Polar Duality and Farkas Lemma October 8th, 2004 Lecturer: Kamal Jain Notes: Daniel Lowd 3.1 Polytope = bounded polyhedron Last lecture, we were attempting to prove the Minkowsky-Weyl Theorem:

More information

The Simplex Algorithm

The Simplex Algorithm The Simplex Algorithm Uri Feige November 2011 1 The simplex algorithm The simplex algorithm was designed by Danzig in 1947. This write-up presents the main ideas involved. It is a slight update (mostly

More information

CS522: Advanced Algorithms

CS522: Advanced Algorithms Lecture 1 CS5: Advanced Algorithms October 4, 004 Lecturer: Kamal Jain Notes: Chris Re 1.1 Plan for the week Figure 1.1: Plan for the week The underlined tools, weak duality theorem and complimentary slackness,

More information

Investigating Mixed-Integer Hulls using a MIP-Solver

Investigating Mixed-Integer Hulls using a MIP-Solver Investigating Mixed-Integer Hulls using a MIP-Solver Matthias Walter Otto-von-Guericke Universität Magdeburg Joint work with Volker Kaibel (OvGU) Aussois Combinatorial Optimization Workshop 2015 Outline

More information

CS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm. Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm. Instructor: Shaddin Dughmi CS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm Instructor: Shaddin Dughmi Algorithms for Convex Optimization We will look at 2 algorithms in detail: Simplex and Ellipsoid.

More information

Solving the Live-out Iterator Problem, Part I

Solving the Live-out Iterator Problem, Part I Louis-Noël Pouchet pouchet@cse.ohio-state.edu Dept. of Computer Science and Engineering, the Ohio State University September 2010 888.11 Outline: Reminder: step-by-step methodology 1 Problem definition:

More information

Polyhedral-Based Data Reuse Optimization for Configurable Computing

Polyhedral-Based Data Reuse Optimization for Configurable Computing Polyhedral-Based Data Reuse Optimization for Configurable Computing Louis-Noël Pouchet 1 Peng Zhang 1 P. Sadayappan 2 Jason Cong 1 1 University of California, Los Angeles 2 The Ohio State University February

More information

Polyhedral Computation Today s Topic: The Double Description Algorithm. Komei Fukuda Swiss Federal Institute of Technology Zurich October 29, 2010

Polyhedral Computation Today s Topic: The Double Description Algorithm. Komei Fukuda Swiss Federal Institute of Technology Zurich October 29, 2010 Polyhedral Computation Today s Topic: The Double Description Algorithm Komei Fukuda Swiss Federal Institute of Technology Zurich October 29, 2010 1 Convexity Review: Farkas-Type Alternative Theorems Gale

More information

Fourier-Motzkin and Farkas Questions (HW10)

Fourier-Motzkin and Farkas Questions (HW10) Automating Scheduling Logistics Final report for project due this Friday, 5/4/12 Quiz 4 due this Monday, 5/7/12 Poster session Thursday May 10 from 2-4pm Distance students need to contact me to set up

More information

UNIVERSITÉ DE PARIS-SUD 11 U.F.R. SCIENTIFIQUE D ORSAY

UNIVERSITÉ DE PARIS-SUD 11 U.F.R. SCIENTIFIQUE D ORSAY UNIVERSITÉ DE PARIS-SUD 11 U.F.R. SCIENTIFIQUE D ORSAY In Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Specialty: Computer Science Louis-Noël POUCHET Subject: ITERATIVE

More information

Using Facets of a MILP Model for solving a Job Shop Scheduling Problem

Using Facets of a MILP Model for solving a Job Shop Scheduling Problem Using Facets of a MILP Model for solving a Job Shop Scheduling Problem Master s Thesis in Complex Adaptive Systems VIKTOR FORSMAN Department of Mathematical Sciences Gothenburg University Gothenburg, Sweden

More information

The Polyhedral Model Is More Widely Applicable Than You Think

The Polyhedral Model Is More Widely Applicable Than You Think The Polyhedral Model Is More Widely Applicable Than You Think Mohamed-Walid Benabderrahmane 1 Louis-Noël Pouchet 1,2 Albert Cohen 1 Cédric Bastoul 1 1 ALCHEMY group, INRIA Saclay / University of Paris-Sud

More information

Polyhedral Search Space Exploration in the ExaStencils Code Generator

Polyhedral Search Space Exploration in the ExaStencils Code Generator Preprint version before issue assignment Polyhedral Search Space Exploration in the ExaStencils Code Generator STEFAN KRONAWITTER and CHRISTIAN LENGAUER, University of Passau, Germany Performance optimization

More information

Introducing the GCC to the Polyhedron Model

Introducing the GCC to the Polyhedron Model 1/15 Michael Claßen University of Passau St. Goar, June 30th 2009 2/15 Agenda Agenda 1 GRAPHITE Introduction Status of GRAPHITE 2 The Polytope Model in GRAPHITE What code can be represented? GPOLY - The

More information

1 Linear Programming. 1.1 Optimizion problems and convex polytopes 1 LINEAR PROGRAMMING

1 Linear Programming. 1.1 Optimizion problems and convex polytopes 1 LINEAR PROGRAMMING 1 LINEAR PROGRAMMING 1 Linear Programming Now, we will talk a little bit about Linear Programming. We say that a problem is an instance of linear programming when it can be effectively expressed in the

More information

AMS : Combinatorial Optimization Homework Problems - Week V

AMS : Combinatorial Optimization Homework Problems - Week V AMS 553.766: Combinatorial Optimization Homework Problems - Week V For the following problems, A R m n will be m n matrices, and b R m. An affine subspace is the set of solutions to a a system of linear

More information

GRAPHITE: Polyhedral Analyses and Optimizations

GRAPHITE: Polyhedral Analyses and Optimizations GRAPHITE: Polyhedral Analyses and Optimizations for GCC Sebastian Pop 1 Albert Cohen 2 Cédric Bastoul 2 Sylvain Girbal 2 Georges-André Silber 1 Nicolas Vasilache 2 1 CRI, École des mines de Paris, Fontainebleau,

More information

Lecture 5: Properties of convex sets

Lecture 5: Properties of convex sets Lecture 5: Properties of convex sets Rajat Mittal IIT Kanpur This week we will see properties of convex sets. These properties make convex sets special and are the reason why convex optimization problems

More information

11 Linear Programming

11 Linear Programming 11 Linear Programming 11.1 Definition and Importance The final topic in this course is Linear Programming. We say that a problem is an instance of linear programming when it can be effectively expressed

More information

Introduction to Modern Control Systems

Introduction to Modern Control Systems Introduction to Modern Control Systems Convex Optimization, Duality and Linear Matrix Inequalities Kostas Margellos University of Oxford AIMS CDT 2016-17 Introduction to Modern Control Systems November

More information

Artificial Intelligence

Artificial Intelligence Artificial Intelligence Combinatorial Optimization G. Guérard Department of Nouvelles Energies Ecole Supérieur d Ingénieurs Léonard de Vinci Lecture 1 GG A.I. 1/34 Outline 1 Motivation 2 Geometric resolution

More information