Introduction to High Performance Computing and X10

Transcription

1 Introduction to High Performance Computing and X10 Special Topic For Comp 621 Vineet Kumar

2 High Performance Computing Supercomputing Grid computers, Multi-cores, clusters massively parallel computing Used for highly calculation intensive tasks like quantum physics, weather forecasting, climate research, oil and gas exploration, molecular modeling, etc mostly scientific research

3 What is X10? A Programming language for HPC systems Strongly statically typed Object oriented Java/C# like sequential core

4 X10 for performance & productivity at scale An evolution of Java for concurrency, scale out, and heterogeneity The X10 language provides: Ability to specify fine-grained concurrency Ability to distribute computation across large-scale clusters Ability to represent heterogeneity at the language level Simple programming model for computation offload Modern OO language features (build libraries/frameworks) Support for writing deadlock-free and determinate code Interoperability with Java Interoperability with native libraries (eg BLAS)

5 X10:History Started in 2004 as part of the DARPA's HPCS project Aim is to develop multi petaflop systems Winner of 2009 HPC challenge for Best performance IBM is looking ahead of HPCS project

6 X10 Target Environments High-end large clusters BlueGene/P, Power7IH, x86 + IB Goal: deliver scalable performance competitive with C+MPI Medium-scale commodity systems ~100 nodes (~1000 core and ~1 terabyte main memory) Scale out environments, but MTBF is days not minutes, Programs that run in minutes/hours at this scale Goal: deliver main-memory performance with simple programming model (accessible to Java programmers) Developer laptops Linux, Max, Windows. Eclipse-based IDE, debugger, etc Goal: support developer productivity

7 X10 programming model Legend Thread/Process Address Space Memory Reference Parallel activities Asynchronous PGAS (Partitioned Global Address Space) Global address space, but with explicit locality Activities can be dynamically created and joined Abstracts hardware details, while still allowing programmer to express locality and concurrency.

8 X10 Compilation flow X10 Compiler Front-End X10 Source Parsing / Type Check X10 AST AST Optimizations AST Lowering X10 AST C++ Back-End C++ Code Generation Java Code Generation Java Back-End C++ Source Java Source XRC C++ Compiler XRX Java Compiler XRJ Native Code Bytecode Native X10 JNI Managed X10 Native Env X10RT Java VMs

9 X10 type system 4 kinds of values Objects, Struct values, functions and null Objects and struct values are together called containers supplies structs and objects Boolean, Byte, Short, Char, Int, Long, Float, Double, Complex and String.

10 Classes Single inheritance, multiple interfaces Static and instance methods Static val fields Instances fields may be val or var Values of class types may be null

11 Structs No inheritance May implement interfaces All fields are val (immutable) All methods are final Cheaper than class objects struct Complex { val real:double; val img :Double; def this(r:double, i:double) { real = r; img = i; def operator + (that:complex) { return Complex(real + that.real, img + that.img);... val x = new Array[Complex](1..10)

12 Arrays Points indices of individual elements in an array, represented as n-dimensional tuples, implicit conversion from Array[Int](1) to Point Rank is the dimensionality or shape of the point/region/array Regions collections of points of same rank (which is also the rank of the region) An array's underlying region is denoted by array.region

13 Life, the universe and everything

14 Points Points can be constructed from integers or Array[Int](1)s by the Point.make factory methods: val origin_1 : Point{rank==1 = Point.make(0); val origin_2 : Point{rank==2 = Point.make(0,0); val origin_5 : Point{rank==5 = Point.make([0,0,0,0,0]); Implicit conversion to Point from Array[Int](1) val p : Point = [1,2,3]; val q : Point{rank==5 = [1,2,3,4,5]; val r : Point(3) = [11,22,33]; //Point(i) specifies the number of dimensions

15 Regions Val MAX_HEIGHT=20; Val Null = Region.makeUnit(); //Empty 0-dimensional region Val R1 = ; // IntRange Val R2 = R1 as Region(1); Val R3 = (0..99) * (-1..MAX_HEIGHT); Val R4 = Region.makeUpperTriangular(10); Val R4 = R5 = R4 && R3; // intersection of two regions Val R5 = [p1,p2,p3] //p1,p2 and p3 are points

16 Arrays A: Array[T](R, f) constructs an array of elements of type T on region R, with A(p) initialized to f(p) for each point p in R val A1 = new Array[Int]((0..9)*(0..9), (p:point(2))=> p(0)*p(1)); //creates a multiplication table Array operations without knowing dimensions for (p in a) a(p) = Math.log(a(p));

17 Type inference Field and local variable types inferred from initializer type val x = 1; x has type int{self==1 val y = 1..2*1..10; y has type Region{rank==2,rect Loop index types inferred from region R: Region{rank==2 for (p in R) {... p has type Point{rank==2 Method return types inferred from method body def m() {... return true... return false... m has return type Boolean Programming Tip: Often important to not declare types of local vals. Enables type inference to compute the most precise possible type.

18 Concurrency controls async finish atomic when

19 async Stmt ::= async Stmt Creates a new child activity that executes statement S Returns immediately S may reference variables in enclosing blocks Activities cannot be named Activity cannot be aborted or cancelled // Compute the Fibonacci // sequence in parallel. def fib(n:int):int { if (n < 2) return 1; val f1:int; val f2:int; finish { async f1 = fib(n-1); f2 = fib(n-2); return f1+f2;

20 finish finish S Execute S, but wait until all (transitively) spawned asyncs have terminated. Rooted exception model Trap all exceptions thrown by spawned activities. Throw an (aggregate) exception if any spawned async terminates abruptly. implicit finish at main activity finish is useful for expressing synchronous operations on (local or) remote data. Stmt ::= finish Stmt // Compute the Fibonacci // sequence in parallel. def fib(n:int):int { if (n < 2) return 1; val f1:int; val f2:int; finish { async f1 = fib(n-1); f2 = fib(n-2); return f1+f2;

21 atomic atomic S Execute statement S atomically Atomic blocks are conceptually executed in a serialized order with respect to all other atomic blocks. An atomic block body (S)... must be nonblocking must not create concurrent activities (sequential) must not access remote data (local) restrictions checked dynamically Stmt ::= atomic Statement MethodModifier ::= atomic def add(x:t) { atomic { this.lis.add(x); this.size = this.size + 1; // push data onto concurrent // list-stack val node = new Node(data); atomic { node.next = head; head = node;

22 When (Conditional atomic) when (E) S Activity suspends until a state in which the guard E is true. In that state, S is executed atomically and in isolation. Guard E is a boolean expression must be nonblocking must not create concurrent activities (sequential) must not access remote data (local) must not have side-effects (const) Stmt ::= WhenStmt WhenStmt ::= when ( Expr ) Stmt WhenStmt or (Expr) Stmt class OneBuffer { var datum:object = null; var filled:boolean = false; def send(v:object) { when (!filled ) { datum = v; filled = true; def receive():object { when ( filled ) { val v = datum; datum = null; filled = false; return v;

23 Questions? I will try my best. I promise!!!

24 Short question 1 Method good will always terminate Method bad does not guarantee termination WHY? static def good() { val c = new Cell[Boolean](false); async { atomic {c() = true; when( c() ); static def bad() { val c = new Cell[Boolean] (false); async { c() = true; when( c() );

25 Short Question 2 Val R1 = (1..2)*(3..4)*(5..6); for (p in R1) { console.out.println(arr(p)); What is the order in which the points are read?

26 Long question Write a X10 program to implement a parallel merge sort!!!

27 References SC'11 presentation Developing Scalable Parallel Applications in X10 by David Cunningham, David Grove, Vijay Saraswat, and Olivier Tardieu X10 Language specification version 2.2 Wikipedia article on number X10 homepage x10-lang.org