Implicit threading, Speculation, and Mutation in Manticore
|
|
|
- Alban Short
- 6 years ago
- Views:
Transcription
1 Implicit threading, Speculation, and Mutation in Manticore John Reppy University of Chicago April 30, 2010
2 Introduction Overview This talk is about some very preliminary ideas for the next step in the design of the Parallel ML (PML) language, which is part of the Manticore system. Specifically, we are exploring three related ways to grow the language: 1. better support for speculative parallelism 2. support for shared state between implicitly-parallel computations 3. supporting nondeterminism to increase parallelism This talk focuses on the first two. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 2
3 Introduction Manticore The Manticore Project The Manticore project is our effort to address the programming needs of commodity applications running on multicore SMP systems Prototype language design supports different levels of parallelism High-level declarative mechanisms for fine-grain parallelism No shared memory Preserve determinism where possible TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 3
4 Introduction Manticore The Manticore Project The Manticore project is our effort to address the programming needs of commodity applications running on multicore SMP systems Prototype language design supports different levels of parallelism High-level declarative mechanisms for fine-grain parallelism No shared memory Preserve determinism where possible TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 3
5 Introduction Manticore The Manticore Project The Manticore project is our effort to address the programming needs of commodity applications running on multicore SMP systems Prototype language design supports different levels of parallelism High-level declarative mechanisms for fine-grain parallelism No shared memory Preserve determinism where possible TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 3
6 Introduction Manticore The Manticore Project The Manticore project is our effort to address the programming needs of commodity applications running on multicore SMP systems Prototype language design supports different levels of parallelism High-level declarative mechanisms for fine-grain parallelism No shared memory Preserve determinism where possible TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 3
7 Introduction Manticore The Manticore Project The Manticore project is our effort to address the programming needs of commodity applications running on multicore SMP systems Prototype language design supports different levels of parallelism High-level declarative mechanisms for fine-grain parallelism No shared memory Preserve determinism where possible TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 3
8 Introduction Manticore The Manticore Project The Manticore project is our effort to address the programming needs of commodity applications running on multicore SMP systems Prototype language design supports different levels of parallelism High-level declarative mechanisms for fine-grain parallelism No shared memory Preserve determinism where possible TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 3
9 Introduction Manticore The Manticore Project (continued...) Our initial language design is called Parallel ML (PML). Sequential core language based on subset of SML: strict with no mutable storage. A variety of lightweight implicitly-threaded constructs for fine-grain parallelism. Explicitly-threaded parallelism based on CML: message passing with first-class synchronization. Prototype implementation with good scaling on 16-way parallel hardware for a range of applications. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 4
10 Introduction Manticore The Manticore Project (continued...) Our initial language design is called Parallel ML (PML). Sequential core language based on subset of SML: strict with no mutable storage. A variety of lightweight implicitly-threaded constructs for fine-grain parallelism. Explicitly-threaded parallelism based on CML: message passing with first-class synchronization. Prototype implementation with good scaling on 16-way parallel hardware for a range of applications. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 4
11 Introduction Manticore The Manticore Project (continued...) Our initial language design is called Parallel ML (PML). Sequential core language based on subset of SML: strict with no mutable storage. A variety of lightweight implicitly-threaded constructs for fine-grain parallelism. Explicitly-threaded parallelism based on CML: message passing with first-class synchronization. Prototype implementation with good scaling on 16-way parallel hardware for a range of applications. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 4
12 Introduction Manticore The Manticore Project (continued...) Our initial language design is called Parallel ML (PML). Sequential core language based on subset of SML: strict with no mutable storage. A variety of lightweight implicitly-threaded constructs for fine-grain parallelism. Explicitly-threaded parallelism based on CML: message passing with first-class synchronization. Prototype implementation with good scaling on 16-way parallel hardware for a range of applications. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 4
13 Implicit threading Implicit threading PML provides several light-weight syntactic forms for introducing parallel computation. Nested Data-parallel arrays provide fine-grain data-parallel computations over sequences. Parallel tuples provide a basic fork-join parallel computation. Parallel bindings provide data-flow parallelism with cancellation of unused subcomputations. Parallel cases provide non-deterministic speculative parallelism. These forms are declarations that a computation is a good candidate for parallel execution, but the details are left to the implementation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 5
14 Implicit threading Implicit threading PML provides several light-weight syntactic forms for introducing parallel computation. Nested Data-parallel arrays provide fine-grain data-parallel computations over sequences. Parallel tuples provide a basic fork-join parallel computation. Parallel bindings provide data-flow parallelism with cancellation of unused subcomputations. Parallel cases provide non-deterministic speculative parallelism. These forms are declarations that a computation is a good candidate for parallel execution, but the details are left to the implementation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 5
15 Implicit threading Implicit threading PML provides several light-weight syntactic forms for introducing parallel computation. Nested Data-parallel arrays provide fine-grain data-parallel computations over sequences. Parallel tuples provide a basic fork-join parallel computation. Parallel bindings provide data-flow parallelism with cancellation of unused subcomputations. Parallel cases provide non-deterministic speculative parallelism. These forms are declarations that a computation is a good candidate for parallel execution, but the details are left to the implementation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 5
16 Implicit threading Implicit threading PML provides several light-weight syntactic forms for introducing parallel computation. Nested Data-parallel arrays provide fine-grain data-parallel computations over sequences. Parallel tuples provide a basic fork-join parallel computation. Parallel bindings provide data-flow parallelism with cancellation of unused subcomputations. Parallel cases provide non-deterministic speculative parallelism. These forms are declarations that a computation is a good candidate for parallel execution, but the details are left to the implementation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 5
17 Implicit threading Implicit threading PML provides several light-weight syntactic forms for introducing parallel computation. Nested Data-parallel arrays provide fine-grain data-parallel computations over sequences. Parallel tuples provide a basic fork-join parallel computation. Parallel bindings provide data-flow parallelism with cancellation of unused subcomputations. Parallel cases provide non-deterministic speculative parallelism. These forms are declarations that a computation is a good candidate for parallel execution, but the details are left to the implementation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 5
18 Implicit threading Implicit threading PML provides several light-weight syntactic forms for introducing parallel computation. Nested Data-parallel arrays provide fine-grain data-parallel computations over sequences. Parallel tuples provide a basic fork-join parallel computation. Parallel bindings provide data-flow parallelism with cancellation of unused subcomputations. Parallel cases provide non-deterministic speculative parallelism. These forms are declarations that a computation is a good candidate for parallel execution, but the details are left to the implementation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 5
19 Implicit threading Implicit threading design space We can organize these features by how well behaved they are. non-speculative deterministic speculation nondeterministic speculation speculation + mutation ( ) [ ] pval pcase Manticore 1.0 TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 6
20 Implicit threading Implicit threading design space We can organize these features by how well behaved they are. non-speculative deterministic speculation nondeterministic speculation speculation + mutation ( ) [ ] pval pcase Manticore 1.0 TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 6
21 Speculation The need for speculation Amdahl s Law tells us that as the number of cores increases, execution time will be dominated by sequential code. Speculation is an important tool for introducing parallelism in otherwise sequential code. PML supports both deterministic and nondeterministic speculation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 7
22 Speculation The need for speculation Amdahl s Law tells us that as the number of cores increases, execution time will be dominated by sequential code. Speculation is an important tool for introducing parallelism in otherwise sequential code. PML supports both deterministic and nondeterministic speculation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 7
23 Speculation The need for speculation Amdahl s Law tells us that as the number of cores increases, execution time will be dominated by sequential code. Speculation is an important tool for introducing parallelism in otherwise sequential code. PML supports both deterministic and nondeterministic speculation. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 7
24 Speculation Parallel bindings Parallel bindings allow deterministic speculation. For example, the computation of b may not be needed in the following program: datatype tree = LF of long ND of tree * tree fun treemul (LF n) = n treemul (ND(t1, t2)) = let pval b = treemul t2 val a = treemul t1 in if (a = 0) then 0 else a*b end TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 8
25 Speculation Parallel case Parallel case supports non-deterministic speculation when we want the quickest answer (e.g., search problems). For example, consider picking a leaf of the tree: fun treepick (LF n) = n treepick (ND(t1, t2)) = ( pcase treepick t1 & treepick t2 of? & n => n n &? => n) There is some similarity with join patterns. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 9
26 Speculation Extending speculation to collections Using pcase, one can define speculative functions over lists: fun pexists f [] = false pexists f (x::xs) = (pcase f x & pexists f xs of true &? => true? & true => true false & false => false) We plan to provide similar operations on our data-parallel arrays, which will avoid the sequential bottleneck of the list-based operations. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 10
27 Mutation The need for shared mutable state Mutable storage is a very powerful communication mechanism: essentially a broadcast mechanism supported by the memory hardware. Sequential algorithms and data-structures gain significant (asymptotic) performance benefits from shared memory (e.g., union-find with path compression). Some algorithms seem hard/impossible to parallelize without shared state (e.g., mesh refinement). But shared memory makes parallel programming hard, so we want to be cautious in adding it to PML. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 11
28 Mutation The need for shared mutable state Mutable storage is a very powerful communication mechanism: essentially a broadcast mechanism supported by the memory hardware. Sequential algorithms and data-structures gain significant (asymptotic) performance benefits from shared memory (e.g., union-find with path compression). Some algorithms seem hard/impossible to parallelize without shared state (e.g., mesh refinement). But shared memory makes parallel programming hard, so we want to be cautious in adding it to PML. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 11
29 Mutation The need for shared mutable state Mutable storage is a very powerful communication mechanism: essentially a broadcast mechanism supported by the memory hardware. Sequential algorithms and data-structures gain significant (asymptotic) performance benefits from shared memory (e.g., union-find with path compression). Some algorithms seem hard/impossible to parallelize without shared state (e.g., mesh refinement). But shared memory makes parallel programming hard, so we want to be cautious in adding it to PML. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 11
30 Mutation The need for shared mutable state Mutable storage is a very powerful communication mechanism: essentially a broadcast mechanism supported by the memory hardware. Sequential algorithms and data-structures gain significant (asymptotic) performance benefits from shared memory (e.g., union-find with path compression). Some algorithms seem hard/impossible to parallelize without shared state (e.g., mesh refinement). But shared memory makes parallel programming hard, so we want to be cautious in adding it to PML. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 11
31 Mutation The design challenge How do we add shared memory while preserving PML s declarative programming model for fine-grain parallelism? Some races are okay in an implicitly-threaded setting. Deadlock is not okay in an implicitly-threaded setting. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 12
32 Mutation The design challenge How do we add shared memory while preserving PML s declarative programming model for fine-grain parallelism? Some races are okay in an implicitly-threaded setting. Deadlock is not okay in an implicitly-threaded setting. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 12
33 Mutation The design challenge How do we add shared memory while preserving PML s declarative programming model for fine-grain parallelism? Some races are okay in an implicitly-threaded setting. Deadlock is not okay in an implicitly-threaded setting. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 12
34 Mutation Example: Minimax search Algorithms, such as minimax search and SAT solving, can benefit from sharing information between branches of the search. For game search, transposition tables are used to turn the search tree into a DAG. Transposition tables provide a kind of function memoization. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 13
35 Mutation Example: Minimax search Algorithms, such as minimax search and SAT solving, can benefit from sharing information between branches of the search. For game search, transposition tables are used to turn the search tree into a DAG. Transposition tables provide a kind of function memoization. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 13
36 Mutation Example: Minimax search Algorithms, such as minimax search and SAT solving, can benefit from sharing information between branches of the search. For game search, transposition tables are used to turn the search tree into a DAG. Transposition tables provide a kind of function memoization. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 13
37 Mutation Transposition tables X X X X X O O X X O O X X O X X O X X O X TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 14
38 Mutation Transposition tables (continued...) For Tic-Tac-Toe, using a transposition table results in a 34-fold reduction in the number of board positions searched. Thus, a sequential search that uses a transposition table is likely to beat a parallel search that doesn t. Want to support sharing information between parallel branches. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 15
39 Mutation Transposition tables (continued...) For Tic-Tac-Toe, using a transposition table results in a 34-fold reduction in the number of board positions searched. Thus, a sequential search that uses a transposition table is likely to beat a parallel search that doesn t. Want to support sharing information between parallel branches. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 15
40 Mutation Transposition tables (continued...) For Tic-Tac-Toe, using a transposition table results in a 34-fold reduction in the number of board positions searched. Thus, a sequential search that uses a transposition table is likely to beat a parallel search that doesn t. Want to support sharing information between parallel branches. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 15
41 Mutation Possible feature: Blackboards Implementing memoization in a general-purpose language poses a lot of challenges, such as efficient equality testing and hashing for user-defined types. An alternative are blackboards, which are essentially shared hash tables. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 16
42 Mutation Possible feature: Blackboards Implementing memoization in a general-purpose language poses a lot of challenges, such as efficient equality testing and hashing for user-defined types. An alternative are blackboards, which are essentially shared hash tables. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 16
43 Mutation Adding blackboards to PML Write-once blackboards; returns value if already set. val setifnotset : ( a bb * key * a) -> a option Memoize a function so that it is evaluated no more than once per call. fun memo (bb, keyfn, f) x = let val iv = ivarnew() in case setifnotset (bb, keyfn x, iv) of SOME iv => ivarget iv NONE => let val y = f x in ivarset (iv, y); y end end This approach fits well with our existing mechanisms. TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 17
44 Mutation What is the right level of abstraction? One approach is to provide a collection of parallel data structures, such as blackboards. Alternatively, we can provide lower-level mechanisms. For example, blackboards can be built from synchronous variables (MVars and IVars), but synchronous variables introduce the possibility of deadlock. Perhaps STM is the solution? TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 18
45 Mutation What is the right level of abstraction? One approach is to provide a collection of parallel data structures, such as blackboards. Alternatively, we can provide lower-level mechanisms. For example, blackboards can be built from synchronous variables (MVars and IVars), but synchronous variables introduce the possibility of deadlock. Perhaps STM is the solution? TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 18
46 Mutation What is the right level of abstraction? One approach is to provide a collection of parallel data structures, such as blackboards. Alternatively, we can provide lower-level mechanisms. For example, blackboards can be built from synchronous variables (MVars and IVars), but synchronous variables introduce the possibility of deadlock. Perhaps STM is the solution? TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 18
47 Mutation What is the right level of abstraction? One approach is to provide a collection of parallel data structures, such as blackboards. Alternatively, we can provide lower-level mechanisms. For example, blackboards can be built from synchronous variables (MVars and IVars), but synchronous variables introduce the possibility of deadlock. Perhaps STM is the solution? TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 18
48 Mutation People Lars Bergstrom Matthew Fluet Mike Rainey Adam Shaw Yingqi Xiao with help from Nic Ford Korie Klein Joshua Knox Jon Riehl Ridg Scott University of Chicago Rochester Institute of Technology University of Chicago University of Chicago University of Chicago University of Chicago University of Chicago University of Chicago University of Chicago University of Chicago TMW 2010 Implicit threading, Speculation, and Mutation in Manticore 19
The Manticore Project
The Manticore Project John Reppy University of Chicago January 20, 2009 Introduction People The Manticore project is a joint project between the University of Chicago and the Toyota Technological Institute.
Concurrent ML. John Reppy January 21, University of Chicago
Concurrent ML John Reppy jhr@cs.uchicago.edu University of Chicago January 21, 2016 Introduction Outline I Concurrent programming models I Concurrent ML I Multithreading via continuations (if there is
THE UNIVERSITY OF CHICAGO PARALLEL FUNCTIONAL PROGRAMMING WITH MUTABLE STATE A DISSERTATION SUBMITTED TO
THE UNIVERSITY OF CHICAGO PARALLEL FUNCTIONAL PROGRAMMING WITH MUTABLE STATE A DISSERTATION SUBMITTED TO THE FACULTY OF THE DIVISION OF THE PHYSICAL SCIENCES IN CANDIDACY FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
Multicore programming in Haskell. Simon Marlow Microsoft Research
Multicore programming in Haskell Simon Marlow Microsoft Research A concurrent web server server :: Socket -> IO () server sock = forever (do acc
Status Report: The Manticore Project
Status Report: The Manticore Project http://manticore.cs.uchicago.edu Matthew Fluet Toyota Technological Institute at Chicago fluet@tti-c.org Nic Ford Mike Rainey John Reppy Adam Shaw Yingqi Xiao University
CSE341 Section 3. Standard-Library Docs, First-Class Functions, & More
CSE341 Section 3 Standard-Library Docs, First-Class Functions, & More Adapted from slides by Daniel Snitkovskiy, Nick Mooney, Nicholas Shahan, Patrick Larson, and Dan Grossman Agenda 1. SML Docs Standard
Processor speed. Concurrency Structure and Interpretation of Computer Programs. Multiple processors. Processor speed. Mike Phillips <mpp>
Processor speed 6.037 - Structure and Interpretation of Computer Programs Mike Phillips Massachusetts Institute of Technology http://en.wikipedia.org/wiki/file:transistor_count_and_moore%27s_law_-
Parallel Functional Programming Lecture 1. John Hughes
Parallel Functional Programming Lecture 1 John Hughes Moore s Law (1965) The number of transistors per chip increases by a factor of two every year two years (1975) Number of transistors What shall we
CSE341: Programming Languages Lecture 9 Function-Closure Idioms. Dan Grossman Winter 2013
CSE341: Programming Languages Lecture 9 Function-Closure Idioms Dan Grossman Winter 2013 More idioms We know the rule for lexical scope and function closures Now what is it good for A partial but wide-ranging
Lectures 24 and 25: Scheduling; Introduction to Effects
15-150 Lectures 24 and 25: Scheduling; Introduction to Effects Lectures by Dan Licata April 12 and 17, 2011 1 Brent s Principle In lectures 17 and 18, we discussed cost graphs, which can be used to reason
Programming Idioms for Transactional Events
Matthew Kehrt Laura Effinger-Dean Michael Schmitz Dan Grossman University of Washington {mkehrt,effinger,schmmd,djg}@cs.washington.edu Abstract Transactional events (TE) are an extension of Concurrent
CSE 341: Programming Languages
CSE 341: Programming Languages Dan Grossman Spring 2008 Lecture 6 Nested pattern-matching; course motivation Dan Grossman CSE341 Spring 2008, Lecture 6 1 Patterns What we know: case-expresssions do pattern-matching
Products and Records
Products and Records Michael P. Fourman February 2, 2010 1 Simple structured types Tuples Given a value v 1 of type t 1 and a value v 2 of type t 2, we can form a pair, (v 1, v 2 ), containing these values.
CSE341 Spring 2017, Midterm Examination April 28, 2017
CSE341 Spring 2017, Midterm Examination April 28, 2017 Please do not turn the page until 12:30. Rules: The exam is closed-book, closed-note, etc. except for one side of one 8.5x11in piece of paper. Please
CSE341: Programming Languages Lecture 9 Function-Closure Idioms. Dan Grossman Fall 2011
CSE341: Programming Languages Lecture 9 Function-Closure Idioms Dan Grossman Fall 2011 More idioms We know the rule for lexical scope and function closures Now what is it good for A partial but wide-ranging
Handout 2 August 25, 2008
CS 502: Compiling and Programming Systems Handout 2 August 25, 2008 Project The project you will implement will be a subset of Standard ML called Mini-ML. While Mini- ML shares strong syntactic and semantic
Denotational Semantics. Domain Theory
Denotational Semantics and Domain Theory 1 / 51 Outline Denotational Semantics Basic Domain Theory Introduction and history Primitive and lifted domains Sum and product domains Function domains Meaning
Parallel Programming Patterns Overview CS 472 Concurrent & Parallel Programming University of Evansville
Parallel Programming Patterns Overview CS 472 Concurrent & Parallel Programming of Evansville Selection of slides from CIS 410/510 Introduction to Parallel Computing Department of Computer and Information
Problems with Concurrency
with Concurrency February 14, 2012 1 / 27 s with concurrency race conditions deadlocks GUI source of s non-determinism deterministic execution model interleavings 2 / 27 General ideas Shared variable Shared
Garbage Collection for Multicore NUMA Machines
Garbage Collection for Multicore NUMA Machines Sven Auhagen University of Chicago sauhagen@cs.uchicago.edu Lars Bergstrom University of Chicago larsberg@cs.uchicago.edu Matthew Fluet Rochester Institute
Programming at Scale: Concurrency
Programming at Scale: Concurrency 1 Goal: Building Fast, Scalable Software How do we speed up software? 2 What is scalability? A system is scalable if it can easily adapt to increased (or reduced) demand
Standard ML. Data types. ML Datatypes.1
Standard ML Data types ML Datatypes.1 Concrete Datatypes The datatype declaration creates new types These are concrete data types, not abstract Concrete datatypes can be inspected - constructed and taken
CSE 332: Data Structures & Parallelism Lecture 15: Analysis of Fork-Join Parallel Programs. Ruth Anderson Autumn 2018
CSE 332: Data Structures & Parallelism Lecture 15: Analysis of Fork-Join Parallel Programs Ruth Anderson Autumn 2018 Outline Done: How to use fork and join to write a parallel algorithm Why using divide-and-conquer
CSCI-GA Scripting Languages
CSCI-GA.3033.003 Scripting Languages 12/02/2013 OCaml 1 Acknowledgement The material on these slides is based on notes provided by Dexter Kozen. 2 About OCaml A functional programming language All computation
Verification and Parallelism in Intro CS. Dan Licata Wesleyan University
Verification and Parallelism in Intro CS Dan Licata Wesleyan University Starting in 2011, Carnegie Mellon revised its intro CS curriculum Computational thinking [Wing] Specification and verification Parallelism
Threads and Continuations COS 320, David Walker
Threads and Continuations COS 320, David Walker Concurrency Concurrency primitives are an important part of modern programming languages. Concurrency primitives allow programmers to avoid having to specify
CSE341: Programming Languages Lecture 11 Type Inference. Dan Grossman Spring 2016
CSE341: Programming Languages Lecture 11 Type Inference Dan Grossman Spring 2016 Type-checking (Static) type-checking can reject a program before it runs to prevent the possibility of some errors A feature
Concurrency and Parallelism. CS152: Programming Languages. Lecture 19 Shared-Memory Parallelism and Concurrency. Concurrency vs. Parallelism.
CS152: Programming Languages Lecture 19 Shared-Memory Parallelism and Concurrency Dan Grossman Spring 2011 Concurrency and Parallelism PL support for concurrency/parallelism a huge topic Increasingly important
CS152: Programming Languages. Lecture 19 Shared-Memory Parallelism and Concurrency. Dan Grossman Spring 2011
CS152: Programming Languages Lecture 19 Shared-Memory Parallelism and Concurrency Dan Grossman Spring 2011 Concurrency and Parallelism PL support for concurrency/parallelism a huge topic Increasingly important
A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 2 Analysis of Fork-Join Parallel Programs
A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 2 Analysis of Fork-Join Parallel Programs Dan Grossman Last Updated: January 2016 For more information, see http://www.cs.washington.edu/homes/djg/teachingmaterials/
THE UNIVERSITY OF CHICAGO IMPLEMENTATION TECHNIQUES FOR NESTED-DATA-PARALLEL LANGUAGES A DISSERTATION SUBMITTED TO
THE UNIVERSITY OF CHICAGO IMPLEMENTATION TECHNIQUES FOR NESTED-DATA-PARALLEL LANGUAGES A DISSERTATION SUBMITTED TO THE FACULTY OF THE DIVISION OF THE PHYSICAL SCIENCES IN CANDIDACY FOR THE DEGREE OF DOCTOR
Mutation. COS 326 David Walker Princeton University
Mutation COS 326 David Walker Princeton University Mutation? 2 Thus far We have considered the (almost) purely functional subset of Ocaml. We ve had a few side effects: printing & raising exceptions. Two
Spring 2012 Homework 10
15-150 Spring 2012 Homework 10 Out: 24 April, 2012 Due: 2 May, 2012, 0900 EST 1 Introduction This homework will give you additional practice with imperative programming in SML. It is slightly short to
Problems with Concurrency. February 19, 2014
with Concurrency February 19, 2014 s with concurrency interleavings race conditions dead GUI source of s non-determinism deterministic execution model 2 / 30 General ideas Shared variable Access interleavings
Thinking parallel. Decomposition. Thinking parallel. COMP528 Ways of exploiting parallelism, or thinking parallel
COMP528 Ways of exploiting parallelism, or thinking parallel www.csc.liv.ac.uk/~alexei/comp528 Alexei Lisitsa Dept of computer science University of Liverpool a.lisitsa@.liverpool.ac.uk Thinking parallel
David Walker. Thanks to Kathleen Fisher and recursively to Simon Peyton Jones for much of the content of these slides.
David Walker Thanks to Kathleen Fisher and recursively to Simon Peyton Jones for much of the content of these slides. Optional Reading: Beautiful Concurrency, The Transactional Memory / Garbage Collection
Part I: Written Problems
CSci 4223 Homework 3 DUE: Friday, March 22, 11:59 pm Instructions. Your task is to answer 2 written problems, and to write 16 SML functions (excluding local helper functions) as well as test cases for
Parallel Programming: Background Information
1 Parallel Programming: Background Information Mike Bailey mjb@cs.oregonstate.edu parallel.background.pptx Three Reasons to Study Parallel Programming 2 1. Increase performance: do more work in the same
Parallel Programming: Background Information
1 Parallel Programming: Background Information Mike Bailey mjb@cs.oregonstate.edu parallel.background.pptx Three Reasons to Study Parallel Programming 2 1. Increase performance: do more work in the same
Inductive Data Types
Inductive Data Types Lars-Henrik Eriksson Functional Programming 1 Original slides by Tjark Weber Lars-Henrik Eriksson (UU) Inductive Data Types 1 / 42 Inductive Data Types Today Today New names for old
Written Presentation: JoCaml, a Language for Concurrent Distributed and Mobile Programming
Written Presentation: JoCaml, a Language for Concurrent Distributed and Mobile Programming Nicolas Bettenburg 1 Universitaet des Saarlandes, D-66041 Saarbruecken, nicbet@studcs.uni-sb.de Abstract. As traditional
Lecture 16: Recapitulations. Lecture 16: Recapitulations p. 1
Lecture 16: Recapitulations Lecture 16: Recapitulations p. 1 Parallel computing and programming in general Parallel computing a form of parallel processing by utilizing multiple computing units concurrently
Testing for Linearizability
Testing for Linearizability 01 Testing for Linearizability Gavin Lowe Testing for Linearizability 02 Linearizability Linearizability is a well-established and accepted correctness condition for concurrent
2012 June UGC NET Solved Question Paper in Computer Science and Applications, Paper III
> 2012 June UGC NET Solved Question Paper in Computer Science and Applications, Paper III 1. Consider the following pseudocode segment: K: =0 for i 1 := l to n for i 2 := 1 to i 1 : : : for i m := 1 to
Barbara Chapman, Gabriele Jost, Ruud van der Pas
Using OpenMP Portable Shared Memory Parallel Programming Barbara Chapman, Gabriele Jost, Ruud van der Pas The MIT Press Cambridge, Massachusetts London, England c 2008 Massachusetts Institute of Technology
F28PL1 Programming Languages. Lecture 12: Standard ML 2
F28PL1 Programming Languages Lecture 12: Standard ML 2 Declaration introduce a variable associate identifier with value - val identifier = expression; > val identifier = value : type identifier - any sequence
Artificial Intelligence. Game trees. Two-player zero-sum game. Goals for the lecture. Blai Bonet
Artificial Intelligence Blai Bonet Game trees Universidad Simón Boĺıvar, Caracas, Venezuela Goals for the lecture Two-player zero-sum game Two-player game with deterministic actions, complete information
Lecture 3: Recursion; Structural Induction
15-150 Lecture 3: Recursion; Structural Induction Lecture by Dan Licata January 24, 2012 Today, we are going to talk about one of the most important ideas in functional programming, structural recursion
1 of 6 Lecture 7: March 4. CISC 879 Software Support for Multicore Architectures Spring Lecture 7: March 4, 2008
1 of 6 Lecture 7: March 4 CISC 879 Software Support for Multicore Architectures Spring 2008 Lecture 7: March 4, 2008 Lecturer: Lori Pollock Scribe: Navreet Virk Open MP Programming Topics covered 1. Introduction
Overview. Cayuga. Autometa. Stream Processing. Stream Definition Operators. Discussion. Applications. Stock Markets. Internet of Things
Overview Stream Processing Applications Stock Markets Internet of Things Intrusion Detection Central Idea Classical Queries: Queries Change, Data Fixed View Maintenance: Data Changes, Queries Fixed, Slow
Chapter 3 Linear Structures: Lists
Plan Chapter 3 Linear Structures: Lists 1. Lists... 3.2 2. Basic operations... 3.4 3. Constructors and pattern matching... 3.5 4. Polymorphism... 3.8 5. Simple operations on lists... 3.11 6. Application:
CSE341: Programming Languages Lecture 6 Nested Patterns Exceptions Tail Recursion. Zach Tatlock Winter 2018
CSE341: Programming Languages Lecture 6 Nested Patterns Exceptions Tail Recursion Zach Tatlock Winter 2018 Nested patterns We can nest patterns as deep as we want Just like we can nest expressions as deep
Lecture 23: Parallelism. To Use Library. Getting Good Results. CS 62 Fall 2016 Kim Bruce & Peter Mawhorter
Lecture 23: Parallelism CS 62 Fall 2016 Kim Bruce & Peter Mawhorter Some slides based on those from Dan Grossman, U. of Washington To Use Library Getting Good Results Create a ForkJoinPool Instead of subclass
COS 326 David Walker. Thanks to Kathleen Fisher and recursively to Simon Peyton Jones for much of the content of these slides.
COS 326 David Walker Thanks to Kathleen Fisher and recursively to Simon Peyton Jones for much of the content of these slides. Optional Reading: Beautiful Concurrency, The Transactional Memory / Garbage
On the Logical Foundations of Staged Computation
On the Logical Foundations of Staged Computation Frank Pfenning PEPM 00, Boston, MA January 22, 2000 1. Introduction 2. Judgments and Propositions 3. Intensional Types 4. Run-Time Code Generation 5. The
Parallelism and runtimes
Parallelism and runtimes Advanced Course on Compilers Spring 2015 (III-V): Lecture 7 Vesa Hirvisalo ESG/CSE/Aalto Today Parallel platforms Concurrency Consistency Examples of parallelism Regularity of
Chapter 3 Linear Structures: Lists
Plan Chapter 3 Linear Structures: Lists 1. Two constructors for lists... 3.2 2. Lists... 3.3 3. Basic operations... 3.5 4. Constructors and pattern matching... 3.6 5. Simple operations on lists... 3.9
A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 2 Analysis of Fork-Join Parallel Programs
A Sophomoric Introduction to Shared-Memory Parallelism and Concurrency Lecture 2 Analysis of Fork-Join Parallel Programs Steve Wolfman, based on work by Dan Grossman (with small tweaks by Alan Hu) Learning
Supplementary Notes on Concurrent ML
Supplementary Notes on Concurrent ML 15-312: Foundations of Programmg Languages Frank Pfenng Lecture 25 November 21, 2002 In the last lecture we discussed the π-calculus, a mimal language with synchronous
CSE 341 Section 5. Winter 2018
CSE 341 Section 5 Winter 2018 Midterm Review! Variable Bindings, Shadowing, Let Expressions Boolean, Comparison and Arithmetic Operations Equality Types Types, Datatypes, Type synonyms Tuples, Records
Amdahl s Law. AMath 483/583 Lecture 13 April 25, Amdahl s Law. Amdahl s Law. Today: Amdahl s law Speed up, strong and weak scaling OpenMP
AMath 483/583 Lecture 13 April 25, 2011 Amdahl s Law Today: Amdahl s law Speed up, strong and weak scaling OpenMP Typically only part of a computation can be parallelized. Suppose 50% of the computation
Potential violations of Serializability: Example 1
CSCE 6610:Advanced Computer Architecture Review New Amdahl s law A possible idea for a term project Explore my idea about changing frequency based on serial fraction to maintain fixed energy or keep same
TensorFlow: A System for Learning-Scale Machine Learning. Google Brain
TensorFlow: A System for Learning-Scale Machine Learning Google Brain The Problem Machine learning is everywhere This is in large part due to: 1. Invention of more sophisticated machine learning models
Advanced C Programming Winter Term 2008/09. Guest Lecture by Markus Thiele
Advanced C Programming Winter Term 2008/09 Guest Lecture by Markus Thiele Lecture 14: Parallel Programming with OpenMP Motivation: Why parallelize? The free lunch is over. Herb
CS4230 Parallel Programming. Lecture 3: Introduction to Parallel Architectures 8/28/12. Homework 1: Parallel Programming Basics
CS4230 Parallel Programming Lecture 3: Introduction to Parallel Architectures Mary Hall August 28, 2012 Homework 1: Parallel Programming Basics Due before class, Thursday, August 30 Turn in electronically
Imprecise Exceptions - Exceptions in Haskell
Imprecise Exceptions - Exceptions in Haskell Christopher Krauß Universität des Saarlandes Informatik 22nd March 2006 Outline Introduction Exceptions Where do we use exceptions? Different kinds of exceptions
Mini-ML. CS 502 Lecture 2 8/28/08
Mini-ML CS 502 Lecture 2 8/28/08 ML This course focuses on compilation techniques for functional languages Programs expressed in Standard ML Mini-ML (the source language) is an expressive core subset of
Chapter 5. Multiprocessors and Thread-Level Parallelism
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 5 Multiprocessors and Thread-Level Parallelism 1 Introduction Thread-Level parallelism Have multiple program counters Uses MIMD model
Ornaments in ML. Thomas Williams, Didier Rémy. April 18, Inria - Gallium
Ornaments in ML Thomas Williams, Didier Rémy Inria - Gallium April 18, 2017 1 Motivation Two very similar functions let rec add m n = match m with Z n S m S (add m n) let rec append ml nl = match ml with
CS 11 Haskell track: lecture 1
CS 11 Haskell track: lecture 1 This week: Introduction/motivation/pep talk Basics of Haskell Prerequisite Knowledge of basic functional programming e.g. Scheme, Ocaml, Erlang CS 1, CS 4 "permission of
Tic Tac Toe Game! Day 8
Tic Tac Toe Game! Day 8 Game Description We will be working on an implementation of a Tic-Tac-Toe Game. This is designed as a two-player game. As you get more involved in programming, you might learn how
Scope. Chapter Ten Modern Programming Languages 1
Scope Chapter Ten Modern Programming Languages 1 Reusing Names Scope is trivial if you have a unique name for everything: fun square a = a * a; fun double b = b + b; But in modern languages, we often use
A retargetable control-flow construct for reactive, parallel and concurrent programming
Joinads A retargetable control-flow construct for reactive, parallel and concurrent programming Tomáš Petříček (tomas.petricek@cl.cam.ac.uk) University of Cambridge, UK Don Syme (don.syme@microsoft.com)
Practice Problems for the Final
ECE-250 Algorithms and Data Structures (Winter 2012) Practice Problems for the Final Disclaimer: Please do keep in mind that this problem set does not reflect the exact topics or the fractions of each
Programming Languages and Compilers (CS 421)
Programming Languages and Compilers (CS 421) Elsa L Gunter 2112 SC, UIUC http://courses.engr.illinois.edu/cs421 Based in part on slides by Mattox Beckman, as updated by Vikram Adve and Gul Agha 10/20/16
n (0 1)*1 n a*b(a*) n ((01) (10))* n You tell me n Regular expressions (equivalently, regular 10/20/ /20/16 4
Regular Expressions Programming Languages and Compilers (CS 421) Elsa L Gunter 2112 SC, UIUC http://courses.engr.illinois.edu/cs421 Based in part on slides by Mattox Beckman, as updated by Vikram Adve
Introduction to ML. Mooly Sagiv. Cornell CS 3110 Data Structures and Functional Programming
Introduction to ML Mooly Sagiv Cornell CS 3110 Data Structures and Functional Programming Typed Lambda Calculus Chapter 9 Benjamin Pierce Types and Programming Languages Call-by-value Operational Semantics
15-853:Algorithms in the Real World. Outline. Parallelism: Lecture 1 Nested parallelism Cost model Parallel techniques and algorithms
:Algorithms in the Real World Parallelism: Lecture 1 Nested parallelism Cost model Parallel techniques and algorithms Page1 Andrew Chien, 2008 2 Outline Concurrency vs. Parallelism Quicksort example Nested
Computer Architecture
Jens Teubner Computer Architecture Summer 2016 1 Computer Architecture Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de Summer 2016 Jens Teubner Computer Architecture Summer 2016 2 Part I Programming
Enforcing Textual Alignment of
Parallel Hardware Parallel Applications IT industry (Silicon Valley) Parallel Software Users Enforcing Textual Alignment of Collectives using Dynamic Checks and Katherine Yelick UC Berkeley Parallel Computing
Tasks with Effects. A Model for Disciplined Concurrent Programming. Abstract. 1. Motivation
Tasks with Effects A Model for Disciplined Concurrent Programming Stephen Heumann Vikram Adve University of Illinois at Urbana-Champaign {heumann1,vadve}@illinois.edu Abstract Today s widely-used concurrent
Lectures 27 and 28: Imperative Programming
15-150 Lectures 27 and 28: Imperative Programming Lectures by Dan Licata April 24 and 26, 2012 In these lectures, we will show that imperative programming is a special case of functional programming. We
Program Graph. Lecture 25: Parallelism & Concurrency. Performance. What does it mean?
Program Graph Lecture 25: Parallelism & Concurrency CS 62 Fall 2015 Kim Bruce & Michael Bannister Some slides based on those from Dan Grossman, U. of Washington Program using fork and join can be seen
A Transactional Model and Platform for Designing and Implementing Reactive Systems
A Transactional Model and Platform for Designing and Implementing Reactive Systems Justin R. Wilson A dissertation presented to the Graduate School of Arts and Sciences of Washington University in partial
Haskell. COS 326 Andrew W. Appel Princeton University. a lazy, purely functional language. slides copyright David Walker and Andrew W.
Haskell a lazy, purely functional language COS 326 Andrew W. Appel Princeton University slides copyright 2013-2015 David Walker and Andrew W. Appel Haskell Another cool, typed, functional programming language
Mathematica for the symbolic. Combining ACL2 and. ACL2 Workshop 2003 Boulder,CO. TIMA Laboratory -VDS Group, Grenoble, France
Combining ACL2 and Mathematica for the symbolic simulation of digital systems AL SAMMANE Ghiath, BORRIONE Dominique, OSTIER Pierre, SCHMALTZ Julien, TOMA Diana TIMA Laboratory -VDS Group, Grenoble, France
Open Nesting in Software Transactional Memory. Paper by: Ni et al. Presented by: Matthew McGill
Open Nesting in Software Transactional Memory Paper by: Ni et al. Presented by: Matthew McGill Transactional Memory Transactional Memory is a concurrent programming abstraction intended to be as scalable
Relational model continued. Understanding how to use the relational model. Summary of board example: with Copies as weak entity
COS 597A: Principles of Database and Information Systems Relational model continued Understanding how to use the relational model 1 with as weak entity folded into folded into branches: (br_, librarian,
Fall Lecture 3 September 4. Stephen Brookes
15-150 Fall 2018 Lecture 3 September 4 Stephen Brookes Today A brief remark about equality types Using patterns Specifying what a function does equality in ML e1 = e2 Only for expressions whose type is
CS153: Compilers Lecture 15: Local Optimization
CS153: Compilers Lecture 15: Local Optimization Stephen Chong https://www.seas.harvard.edu/courses/cs153 Announcements Project 4 out Due Thursday Oct 25 (2 days) Project 5 out Due Tuesday Nov 13 (21 days)
Reinhard v. Hanxleden 1, Michael Mendler 2, J. Aguado 2, Björn Duderstadt 1, Insa Fuhrmann 1, Christian Motika 1, Stephen Mercer 3 and Owen Brian 3
Sequentially Constructive Concurrency * A conservative extension of the Synchronous Model of Computation Reinhard v. Hanxleden, Michael Mendler 2, J. Aguado 2, Björn Duderstadt, Insa Fuhrmann, Christian
CSE 341 : Programming Languages
CSE 341 : Programming Languages Lecture 9 Lexical Scope, Closures Zach Tatlock Spring 2014 Very important concept We know function bodies can use any bindings in scope But now that functions can be passed
OpenMP tasking model for Ada: safety and correctness
www.bsc.es www.cister.isep.ipp.pt OpenMP tasking model for Ada: safety and correctness Sara Royuela, Xavier Martorell, Eduardo Quiñones and Luis Miguel Pinho Vienna (Austria) June 12-16, 2017 Parallel
Introduction to ML. Based on materials by Vitaly Shmatikov. General-purpose, non-c-like, non-oo language. Related languages: Haskell, Ocaml, F#,
Introduction to ML Based on materials by Vitaly Shmatikov slide 1 ML General-purpose, non-c-like, non-oo language Related languages: Haskell, Ocaml, F#, Combination of Lisp and Algol-like features (1958)
4.5 Pure Linear Functional Programming
4.5 Pure Linear Functional Programming 99 4.5 Pure Linear Functional Programming The linear λ-calculus developed in the preceding sections can serve as the basis for a programming language. The step from
CSE332: Data Abstractions Lecture 23: Programming with Locks and Critical Sections. Tyler Robison Summer 2010
CSE332: Data Abstractions Lecture 23: Programming with Locks and Critical Sections Tyler Robison Summer 2010 1 Concurrency: where are we Done: The semantics of locks Locks in Java Using locks for mutual
Shared Memory Parallelism - OpenMP
Shared Memory Parallelism - OpenMP Sathish Vadhiyar Credits/Sources: OpenMP C/C++ standard (openmp.org) OpenMP tutorial (http://www.llnl.gov/computing/tutorials/openmp/#introduction) OpenMP sc99 tutorial
Programming Languages and Techniques (CIS120)
Programming Languages and Techniques () Lecture 13 February 12, 2018 Mutable State & Abstract Stack Machine Chapters 14 &15 Homework 4 Announcements due on February 20. Out this morning Midterm results
Speculative Parallelization Technology s only constant is CHANGE. Devarshi Ghoshal Sreesudhan
Speculative Parallelization Technology s only constant is CHANGE Devarshi Ghoshal Sreesudhan Agenda Moore s law What is speculation? What is parallelization? Amdahl s law Communication between parallely
An Introduction to Parallel Programming
An Introduction to Parallel Programming Ing. Andrea Marongiu (a.marongiu@unibo.it) Includes slides from Multicore Programming Primer course at Massachusetts Institute of Technology (MIT) by Prof. SamanAmarasinghe