Side-effect checking for Scala. Lukas Rytz, Martin Odersky, Adriaan Moors Scala Days 2011

Transcription

1 Side-effect checking for Scala Lukas Rytz, Martin Odersky, Adriaan Moors Scala Days 2011

2 Why effect checking? We have at least one post a week where the correct answer is wholly or in part, effect system. [@matthewpocock on scala-internals, May 19] Libraries expect pure code: atomic blocks in STMs, distributed code, DSLs (e.g. Parsers), pattern guards Checkable documentation, guaranteed behavior 2

3 What are side-effects? Everything a function does besides computing the result: read / write a file, modify state, throw an Exception, consume resources, call another side-effecting function Programmers care about side-effects but not always about all of them! 3

4 Control your effects Goal: a system to annotate and verify sideeffects 4

5 Control your effects Goal: a system to annotate and verify sideeffects Popular techniques: As a type and effect system, an extension to the type system Using monads, equivalent to an effect type system [Phil Wadler and Peter Thiemann, 2003] 4

6 Control your effects Goal: a system to annotate and verify sideeffects Popular techniques: As a type and effect system, an extension to the type system Using monads, equivalent to an effect type system [Phil Wadler and Peter Thiemann, 2003] 4

7 Java s checked exceptions An effect system! Methods declare their side-effects Overriding methods need to conform 5

8 Java s checked exceptions An effect system! Methods declare their side-effects Overriding methods need to conform...with one major problem: Writing a method which is polymorphic in the effect is heavily verbose Also, they cannot be turned off 5

9 Java s checked exceptions An effect system! interface StringTrans { public string trans(string s); Methods declare their side-effects Overriding methods need to conform...with one major problem: class Text { public void translines(stringtrans f) {... Writing a method which is polymorphic in the effect is heavily verbose Also, they cannot be turned off 5

10 Java s checked exceptions An effect system! interface StringTrans<E extends Exception> { public Methods string declare trans(string their side-effects s) throws E; Overriding methods need to conform...with one major problem: class Text { public void <E extends Exception> translines(stringtrans<e> f) throws E {... Writing a method which is polymorphic in the effect is heavily verbose Also, they cannot be turned off 5

11 Java s checked exceptions An effect system! Methods declare their side-effects Overriding methods need to conform...with one major problem: Writing a method which is polymorphic in the effect is heavily verbose Also, they cannot be turned off 5

12 Our main goals A syntactically lightweight system for polymorphic effects Open for different kinds of side-effects A framework to facilitate implementing new effect systems 6

13 Properties of effects Idempotent Not ordered ( first A then B ) May semantics: side-effects of a method may occur, but they don t need to 7

14 Lightweight polymorphism The Gist: a higher-oder function has the side-effect of its argument class List[A].. { def map[b](f: A => B): = means: union of the side-effects of all methods of all parameters 8

15 @poly is too coarse In the object-oriented is too coarse: trait Counter { def get: Int; def inc(): Unit def read(c: Counter): = { c.get 9

16 Calls on Parameters Annotate which methods of a parameter are called method m is called on parameter a 10

17 Calls on Parameters Annotate which methods of a parameter are called method m is called on parameter a trait Counter { def get: Int; def inc(): Unit def read(c: Counter): = { c.get 10

18 Parameter Calls annotations form an effect system! Calls to methods on arguments are the effects: def f(a: A, b: B): b.bar()) = { a.foo() a.foo() b.boom() 11

19 Parameter Calls annotations form an effect system! Calls to methods on arguments are the effects: def f(a: A, b: B): b.bar()) = { a.foo() a.foo() b.boom() Effects are idempotent 11

20 Parameter Calls annotations form an effect system! Calls to methods on arguments are the effects: def f(a: A, b: B): b.bar()) = { a.foo() a.foo() b.boom() No ordering 11

21 Parameter Calls annotations form an effect system! Calls to methods on arguments are the effects: def f(a: A, b: B): b.bar()) = { a.foo() a.foo() b.boom() May -semantics 11

22 Parameter Calls annotations form an effect system! Calls to methods on arguments are the effects: def f(a: A, b: B): b.bar()) = { a.foo() a.foo() Effect type error b.boom() 11

23 Parameter Calls annotations form an effect system! Calls to methods on arguments are the effects: def f(a: A, b: B): b.bar()) = { a.foo() a.foo() Effect type error b.boom() We first infer and/or check effects Other effect systems use annotations to implement polymorphism 11

24 map trait List[A] { def map[b](f: A => B): = this match { case x :: xs => f(x) :: xs.map(f) case Nil => Nil 12

25 map trait List[A] { def map[b](f: A => B): = this match { case x :: xs => f(x) :: xs.map(f) case Nil => Nil OK, but map is not actually implemented that way... 12

26 Real-world map The actual implementation trait TraversableLike[+A, +Repr] extends... { this: Repr => def map[b, That](f: A => B) (implicit bf: CanBuildFrom[Repr, B, That]): That = { val b = bf(this) for (a <- this) b += f(a) //foreach(a => b += f(a)) b.result 13

27 Real-world map The actual implementation trait TraversableLike[+A, +Repr] extends... { this: Repr => def map[b, That](f: A => B) (implicit bf: CanBuildFrom[Repr, B, That]): That = { val b = bf(this) for (a <- this) b += f(a) //foreach(a => b += f(a)) b.result We need an effect system for tracking state modifications 13

28 Effect system for state Light in annotation Modular (no whole-program pointer analysis) Allow modifications to local variables and local objects...got inspired by a recently published system called JPure [David J. Pearce, Conference on Compiler Construction, Saarbrücken, 2011] 14

29 Locality Fields marked define the locality of a class Invariant: an object can never be part of more than one locality class HashSet[A] val table: Array[List[A]] 15

30 Locality Fields marked define the locality of a class Invariant: an object can never be part of more than one locality HashSet table class HashSet[A] val table: Array[List[A]] Array 15 List a next List... A... List...

31 Modification effects describe the localities a method modifies class HashSet[A] val table: Array[List[A]] def add(a: A): = { val l = table(a.##) if (!l.contains(a)) table(a.##) = a :: l 16

32 Modification effects describe the localities a method modifies class HashSet[A] val table: Array[List[A]] def add(a: A): = { val l = table(a.##) if (!l.contains(a)) table(a.##) = a :: l A method is pure if it modifies only freshlyallocated objects 16

33 State effect example class HashSet[A] val table: Array[List[A]] def add(a: A): = { val l = table(a.##) if (!l.contains(a)) table(a.##) = a :: l def makeset(): = { val s = new HashSet[Int]() s.add(1) s 17

34 State effect example class HashSet[A] val table: Array[List[A]] def add(a: A): = { val l = table(a.##) if (!l.contains(a)) table(a.##) = a :: l s is freshly allocated def makeset(): = { val s = new HashSet[Int]() s.add(1) s 17

35 State effect example class HashSet[A] val table: Array[List[A]] def add(a: A): = { val l = table(a.##) if (!l.contains(a)) table(a.##) = a :: l s is freshly allocated def makeset(): = { val s = new HashSet[Int]() s.add(1) s mask modification of fresh object 17

36 Freshness Modifications to fresh local objects can be masked new returns a fresh annotation to mark other methods that return a fresh object 18

37 CanBuildFrom, Builder trait CanBuildFrom[-From, -Elem, +To] { def apply[a](f: From): Builder[Elem, 19

38 CanBuildFrom, Builder Returns a fresh builder trait CanBuildFrom[-From, -Elem, +To] { def apply[a](f: From): Builder[Elem, 19

39 CanBuildFrom, Builder Returns a fresh builder trait CanBuildFrom[-From, -Elem, +To] { def apply[a](f: From): Builder[Elem, trait Builder[-B, +That] { def +=(b: B): def result: 19

40 The real map, revisited trait TraversableLike[+A, +Repr] extends... { this: Repr => def map[b, That](f: A => B) (implicit bf: CanBuildFrom[Repr, B, That]): That = { val b = bf(this) for (a <- this) b += f(a) //foreach(a => b += f(a)) b.result 20

41 The real map, revisited trait TraversableLike[+A, +Repr] extends... { this: Repr => def map[b, That](f: A => B) (implicit bf: CanBuildFrom[Repr, B, That]): That = { val b = bf(this) for (a <- this) b += f(a) //foreach(a => b += f(a)) b.result trait CanBuildFrom[-Repr, -B, +That] { def apply(r: Repr): Builder[B, 20

42 The real map, revisited trait TraversableLike[+A, +Repr] extends... { this: Repr => def map[b, That](f: A => B) (implicit bf: CanBuildFrom[Repr, B, That]): That = { val b = bf(this) for (a <- this) b += f(a) //foreach(a => b += f(a)) b.result trait Builder[-B, +That] { def +=(b: B): 20

43 The real map, revisited trait TraversableLike[+A, +Repr] extends... { this: Repr => def map[b, That](f: A => B) (implicit bf: CanBuildFrom[Repr, B, That]): That = { val b = bf(this) for (a <- this) b += f(a) //foreach(a => b += f(a)) b.result def foreach[u](f: A => U): 20

44 The real map, revisited trait TraversableLike[+A, +Repr] extends... { this: Repr def map[b, That](f: A => B) (implicit bf: CanBuildFrom[Repr, B, That]): That = { val b = bf(this) for (a <- this) b += f(a) //foreach(a => b += f(a)) b.result 20

45 Limitations Caches break purity Locality is fixed at definition-site: In (mutable) Set[T], the elements in the set are not part of the set s locality Given a Set[Set[T]], modifying one of the inner sets cannot be interpreted as modifying the outer set. 21

46 (Near) future work Interaction with non-annotated libraries Define default behavior: when should effects be inferred? Benchmark on the Scala library Obtain feedback by releasing a first version to the Community 22

47 Thank you! I m happy to answer questions. 23