F-ing Applicative Functors

Size: px

Start display at page:

Download "F-ing Applicative Functors"

Doreen Townsend
5 years ago
Views:

1 F-ing Applicative Functors Andreas Rossberg, Google Claudio Russo, MSR Derek Dreyer, MPI-SWS ML Workshop, Copenhagen 2012/09/13

2 The Functor Schism

3 The Functor Schism SML: generative functors return fresh abstract types with each application

4 The Functor Schism SML: generative functors return fresh abstract types with each application OCaml: applicative functors return same abstract types with each application (to the same argument)

5 The Functor Schism SML: generative functors return fresh abstract types with each application OCaml: applicative functors return same abstract types with each application (to the same argument)

6 Example: Set

7 Example: Set signature SET = { type elem type set val empty : set val add : elem set set val member : elem set bool }

8 Example: Set signature SET = { type elem type set val empty : set val add : elem set set val member : elem set bool } signature ORD = { type t val leq : t t bool }

9 Example: Set signature SET = { type elem type set val empty : set val add : elem set set val member : elem set bool } signature ORD = { type t val leq : t t bool } module Set (Elem : ORD) :> (SET where type elem = Elem.t) = {... }

10 Example: Set, generative module A = Set Int A.member 3 (A.add 2 A.empty)

11 Example: Set, generative module A = Set Int module B = Set Int A.member 3 (B.add 2 B.empty)

12 Example: Set, generative module A = Set Int module B = Set Int A.member 3 (B.add 2 B.empty) (* ill-typed, A.set B.set *)

13 Example: Set, applicative module A = Set Int module B = Set Int A.member 3 (B.add 2 B.empty) (* well-typed, A.set = Set(Int).set = B.set *)

14 The story, so far Leroy [POPL 1995] OCaml Russo [Thesis 1998 / ENTCS 2003] Moscow ML Shao [ICFP 1999] Dreyer & Crary & Harper [POPL 2003]

15 Plan Issues Proposal Formalisation

16 Why applicative functors?

17 Why applicative functors? Literature: motivated by higher-order functors

18 Why applicative functors? Literature: motivated by higher-order functors Practice: compilation units importing functors are higher-order functors in disguise

19 Example: Map signature MAP = { type key type map α val empty : map α val add : key α map α map α val lookup : key map α option α } module Map : (Key : ORD) MAP with type key = Key.t

20 Example: Map signature MAP = { type key type map α val empty : map α val add : key α map α map α val lookup : key map α option α val domain : map α? } module Map : (Key : ORD) MAP with type key = Key.t

21 With generative Set, variant 1 signature MAP = { type key type map α module Set : SET with type elem = key val empty : map α val add : key α map α map α val lookup : key map α option α val domain : map α Set.set } module Map : (Key : ORD) MAP with type key = Key.t

22 With generative Set, variant 2 signature MAP = { type key type map α type set val empty : map α val add : key α map α map α val lookup : key map α option α val domain : map α set } module Map : (Key : ORD) (Set : SET with type elem = Key.t) MAP with type key = Key.t with type set = Set.t

23 With applicative Set signature MAP = { module Key : ORD type map α val empty : map α val add : Key.t α map α map α val lookup : Key.t map α option α val domain : map α Set(Key).set } module Map : (Key : ORD) MAP with type key = Key.t

24 Why applicative functors? Two independent units can use the same generic data type without any need for cooperation A third unit using both is still able to exchange sets between them seamlessly (a.k.a. diamond import)

25 Observation 0 Applicative functors are useful for modularity.

26 Example: First-class modules signature S = { type t; val v : t; val f : t t } val p1 = pack { type t = int; val v = 6; val f = negate } : S val p2 = pack { type t = string; val v = uh ; val f = id } : S

27 Example: First-class modules signature S = { type t; val v : t; val f : t t } val p1 = pack { type t = int; val v = 6; val f = negate } : S val p2 = pack { type t = string; val v = uh ; val f = id } : S val p = ref p1 module F {} = unpack!p : S

28 Example: First-class modules signature S = { type t; val v : t; val f : t t } val p1 = pack { type t = int; val v = 6; val f = negate } : S val p2 = pack { type t = string; val v = uh ; val f = id } : S val p = ref p1 module F {} = unpack!p : S module M1 = F {} p := p2 module M2 = F {}

29 Example: First-class modules signature S = { type t; val v : t; val f : t t } val p1 = pack { type t = int; val v = 6; val f = negate } : S val p2 = pack { type t = string; val v = uh ; val f = id } : S val p = ref p1 module F {} = unpack!p : S module M1 = F {} p := p2 module M2 = F {} M1.f M2.v (* oops, ka-boom! *)

30 Observation 1 Applicativity can break type safety (with 1st-class modules).

31 Example: Abstract names signature NAME = { type name val new : () name val eq : name name bool } module Name {} :> NAME = { type name = int val count = ref 0 val new () = ++count ;!count val eq = Int.eq }

32 Example: Abstract names module A = Name {} module B = Name {} val a = A.new () val b = B.new ()

33 Example: Abstract names module A = Name {} module B = Name {} val a = A.new () val b = B.new () a = b

34 Example: Abstract names module A = Name {} module B = Name {} val a = A.new () val b = B.new () a = b (* oops, true! *)

35 Observation 2 Applicativity can break abstraction safety (of impure functors).

36 Example: Set ordering module A = Set {type t = int; val leq = Int.leq} module B = Set {type t = int; val leq = Int.geq}

37 Example: Set ordering module A = Set {type t = int; val leq = Int.leq} module B = Set {type t = int; val leq = Int.geq} val s1 = A.add 2 A.empty val s2 = B.add 3 s1

38 Example: Set ordering module A = Set {type t = int; val leq = Int.leq} module B = Set {type t = int; val leq = Int.geq} val s1 = A.add 2 A.empty val s2 = B.add 3 s1 A.member 2 s2

39 Example: Set ordering module A = Set {type t = int; val leq = Int.leq} module B = Set {type t = int; val leq = Int.geq} val s1 = A.add 2 A.empty val s2 = B.add 3 s1 A.member 2 s2 (* oops, false! *)

40 Observation 3 Applicativity can break abstraction safety (of pure functors).

41 Example: Set ordering, in Ocaml module A = Set {type t = int; val leq = Int.leq} module B = Set {type t = int; val leq = Int.geq}

42 Example: Set ordering, in Ocaml module A = Set {type t = int; val leq = Int.leq} module B = Set {type t = int; val leq = Int.geq} val s1 = A.add 2 A.empty val s2 = B.add 3 s1

43 Example: Set ordering, in Ocaml module A = Set {type t = int; val leq = Int.leq} module B = Set {type t = int; val leq = Int.geq} val s1 = A.add 2 A.empty val s2 = B.add 3 s1 (* type error, A.set B.set, because Set({...}).set not a path *)

44 Set ordering in Ocaml, take 2

45 Set ordering in Ocaml, take 2 module F (X : {}) = { type t = int val leq = if isfullmoon () then Int.leq else Int.geq }

46 Set ordering in Ocaml, take 2 module F (X : {}) = (* returns one of the previous modules! *) { type t = int val leq = if isfullmoon () then Int.leq else Int.geq }

47 Set ordering in Ocaml, take 2 module F (X : {}) = (* returns one of the previous modules! *) { type t = int val leq = if isfullmoon () then Int.leq else Int.geq } module Unit = {} module A = Set (F Unit) module B = Set (F Unit)

48 Set ordering in Ocaml, take 2 module F (X : {}) = (* returns one of the previous modules! *) { type t = int val leq = if isfullmoon () then Int.leq else Int.geq } module Unit = {} module A = Set (F Unit) module B = Set (F Unit) val s1 = A.add 2 A.empty val s2 = B.add 3 s1 A.member 2 s2

49 Set ordering in Ocaml, take 2 module F (X : {}) = (* returns one of the previous modules! *) { type t = int val leq = if isfullmoon () then Int.leq else Int.geq } module Unit = {} module A = Set (F Unit) module B = Set (F Unit) val s1 = A.add 2 A.empty val s2 = B.add 3 s1 A.member 2 s2 (* oops, false! A.set = Set(F Unit).set = B.set *)

50 Observation 4 Impure applications in paths breaks abstraction safety (even of pure functors).

51 Type Equivalence in Ocaml module A = Set {type t = int; val leq = Int.leq} module B = Set {type t = int; val leq = Int.leq} val s1 = A.add 2 A.empty val s2 = B.add 3 s1 A.member 2 s2 (* type error, A.set B.set, because Set({...}).set not a path *)

52 Type Equivalence in Ocaml (2) module IntOrd = {type t = int; val leq = Int.leq} module IntOrd = IntOrd module A = Set IntOrd module B = Set IntOrd val s1 = A.add 2 A.empty val s2 = B.add 3 s1 A.member 2 s2 (* type error, A.set = Set(IntOrd).set Set(IntOrd ).set = B.set *)

53 Observation 5 Syntactic path equivalence is overly restrictive.

54 Hm...

55 Overcoming the Schism

56 Overcoming the Schism Support both applicative and generative functors

57 Overcoming the Schism Support both applicative and generative functors A functor is applicative if and only if it is pure

58 Overcoming the Schism Support both applicative and generative functors A functor is applicative if and only if it is pure type system tracks purity

59 Overcoming the Schism Support both applicative and generative functors A functor is applicative if and only if it is pure type system tracks purity Two modules are equivalent if and only if they define equivalent types and values

60 Overcoming the Schism Support both applicative and generative functors A functor is applicative if and only if it is pure type system tracks purity Two modules are equivalent if and only if they define equivalent types and values type system tracks value identity (while avoiding dependent types)

61 Purity

62 Purity Only one form of functor expression, deemed pure iff: it does not unpack a first-class module it does not apply an impure functor all value bindings are non-expansive (value restriction)

63 Purity Only one form of functor expression, deemed pure iff: it does not unpack a first-class module it does not apply an impure functor all value bindings are non-expansive (value restriction) Two forms of functor type impure: (X : S1) S2 pure: (X : S1) S2

64 Abstract Values

65 Abstract Values Every value binding is identified by an abstract value

66 Abstract Values Every value binding is identified by an abstract value Mere renamings retain identity (e.g., val x = A.y)

67 Abstract Values Every value binding is identified by an abstract value Mere renamings retain identity (e.g., val x = A.y) Other bindings define fresh abstract value

68 Abstract Values Every value binding is identified by an abstract value Mere renamings retain identity (e.g., val x = A.y) Other bindings define fresh abstract value Specifications (in signatures) declare abstract values

69 Abstract Values Every value binding is identified by an abstract value Mere renamings retain identity (e.g., val x = A.y) Other bindings define fresh abstract value Specifications (in signatures) declare abstract values Formally, abstract values are phantom type variables, quantified and matched in same manner as abstract types

70 Abstract Values Every value binding is identified by an abstract value Mere renamings retain identity (e.g., val x = A.y) Other bindings define fresh abstract value Specifications (in signatures) declare abstract values Formally, abstract values are phantom type variables, quantified and matched in same manner as abstract types Refinement of SML90 s structure sharing

71 Module Syntax Modules M ::= X {B} M.X fun X:S M XX X:>S Signatures S ::= X {D} M.X (X:S) S (X:S) S S where type X=T Bindings B ::= val X=E type X=T module X=M signature X=S include M B;B ɛ Declarations D ::= val X:T type X=T type X:K module X:S signature X=S include S D;D ɛ

72 F-ing Formalisation

73 F-ing Elaboration, recap Signatures Γ S α.σ Modules Γ M : α.σ e

74 F-ing Elaboration, recap Signatures Γ S α.σ Γ α.σ :Ω Modules Γ M : α.σ e Γ e : α.σ

75 Semantic Signatures, recap Σ ::= [τ] [= τ:κ] {l :Σ} α 1.Σ 1 α 2.Σ 2 (term) (type) (structure) (functor)

76 Example: Set Signature Set : (Elem : ORD) (SET where type elem = Elem.t) α.{ t : [= α : Ω], leq :[α α bool] } β. { elem : [= α : Ω], set : [= β : Ω], empty :[β], add :[α β β], member :[α β bool] }

77 Elaboration, revised Signatures Γ S α.σ Modules Γ M : ϕ α.σ e (ϕ ::= P I)

78 Semantic Signatures, revised π ::= α τ Σ ::= [= π:τ] [= τ:κ] {l :Σ} α 1.Σ 2 ϕ α 2.Σ 2 (path) (term) (type) (structure) (functor)

79 Semantic Signatures, revised π ::= α τ Σ ::= [= π:τ] [= τ:κ] {l :Σ} α 1.Σ 2 ϕ α 2.Σ 2 (path) (term) (type) (structure) (functor) Impure functor: Pure functor: α 1.Σ 1 I α 2.Σ 2 α 2. α 1.Σ 1 P Σ 2

80 Functor Signatures Γ S α.σ

81 Functor Signatures Γ S α.σ Γ S 1 α 1.Σ 1 Γ, α 1,X:Σ 1 S 2 α 2.Σ 2 Γ (X:S 1 ) S 2 α 1. Σ 1 α 2.Σ 2

82 Functor Signatures Γ S α.σ Γ S 1 α 1.Σ 1 Γ, α 1,X:Σ 1 S 2 α 2.Σ 2 Γ (X:S 1 ) S 2 α 1. Σ 1 α 2.Σ 2 Γ S 1 α 1.Σ 1 Γ, α 1,X:Σ 1 S 2 α 2.Σ 2 Γ (X:S 1 ) S 2 α 2. α 1. Σ 1 Σ 2 [α 2 α 1/α 2 ] α 1 : κ 1 α 2 : κ 2 α 2 : κ 1 κ 2

83 Example: Set Signature Set : (Elem : ORD) (SET where type elem = Elem.t) β Ω Ω. α.{ t : [= α : Ω], leq :[α α bool] } { elem : [= α : Ω], set : [= βα: Ω], empty :[βα], add :[α βα βα], member :[α βα bool] }

84 Example: Set Signature Set : (Elem : ORD) (SET where type elem = Elem.t) ββ 1 β 2 β 3. αα 1.{ t : [= α : Ω], leq : [= α 1 : α α bool] } { elem : [= α : Ω], set : [= β α α 1 : Ω], empty : [= β 1 : β α α 1 ], add : [= β 2 : α β α α 1 β α α 1 ], member : [= β 3 : α β α α 1 bool] }

85 Functor Expressions Γ M : ϕ α.σ e

86 Functor Expressions Γ M : ϕ α.σ e Γ S α 1.Σ 1 Γ, α 1,X:Σ 1 M : I α 2.Σ 2 e Γ fun X:S M : P α 1. Σ 1 α 2.Σ 2 λα 1.λX:Σ 1.e

87 Functor Expressions Γ M : ϕ α.σ e Γ S α 1.Σ 1 Γ, α 1,X:Σ 1 M : I α 2.Σ 2 e Γ fun X:S M : P α 1. Σ 1 α 2.Σ 2 λα 1.λX:Σ 1.e Γ S α 1.Σ 1 Γ, α 1,X:Σ 1 M : P α 2.Σ 2 e Γ fun X:S M : P α 2. α 1. Σ 1 Σ 2???

88 Elaboration Invariant, revised Signatures Γ S α.σ Γ α.σ :Ω Modules Γ M : I α.σ e Γ e : α.σ

89 Elaboration Invariant, revised Signatures Γ S α.σ Γ α.σ :Ω Modules Γ M : I α.σ e Γ e : α.σ Γ M : P α.σ e

90 Elaboration Invariant, revised Signatures Γ S α.σ Γ α.σ :Ω Modules Γ M : I α.σ e Γ e : α.σ Γ M : P α.σ e e : α. Γ.Σ

91 Functor Expressions Γ M : ϕ α.σ e Γ S α 1.Σ 1 Γ, α 1,X:Σ 1 M : I α 2.Σ 2 e Γ fun X:S M : P α 1. Σ 1 α 2.Σ 2 λγ.λα 1.λX:Σ 1.e Γ S α 1.Σ 1 Γ, α 1,X:Σ 1 M : P α 2.Σ 2 e Γ fun X:S M : P α 2. α 1. Σ 1 Σ 2 e

92 Sealing Γ M : ϕ α.σ e Γ(X) =Σ Γ S α.σ Γ Σ α.σ τ f Γ X :> S: P α.σ[α Γ/α] pack λγ.τ, λγ.f X α : κ α : Γ κ

93 Elaborating Specifications

94 Elaborating Specifications Γ D Ξ

95 Elaborating Specifications Γ K κ α Γ type X:K α.{x : [= α : κ α ]} Γ D Ξ

96 Elaborating Specifications Γ D Ξ Γ K κ α Γ type X:K α.{x : [= α : κ α ]} Γ T : κ τ Γ type X=T {X : [= τ : κ]}

97 Elaborating Specifications Γ D Ξ Γ K κ α Γ type X:K α.{x : [= α : κ α ]} Γ T : κ τ Γ type X=T {X : [= τ : κ]} Γ T :Ω τ Γ val X:T α.{x : [= α : Ω]}

98 Elaborating Specifications Γ D Ξ Γ K κ α Γ type X:K α.{x : [= α : κ α ]} Γ T : κ τ Γ type X=T {X : [= τ : κ]} Γ T :Ω τ Γ val X:T α.{x : [= α : Ω]} Γ P : [= π : τ] e Γ val X=P {X : [= π : τ]}

99 Elaborating Bindings

100 Elaborating Bindings Γ B :Ξ e

101 Elaborating Bindings Γ T : κ τ Γ B :Ξ e Γ type X=T : ϕ {X : [= τ : κ]} {X =[τ : κ]}

102 Elaborating Bindings Γ T : κ τ Γ B :Ξ e Γ type X=T : ϕ {X : [= τ : κ]} {X =[τ : κ]} Γ E : τ e Γ val X=E : I α.{x : [= α : τ]} pack {},{X =[e]}

103 Elaborating Bindings Γ T : κ τ Γ B :Ξ e Γ type X=T : ϕ {X : [= τ : κ]} {X =[τ : κ]} Γ E : τ e Γ val X=E : I α.{x : [= α : τ]} pack {},{X =[e]} Γ E : τ e E non-expansive Γ val X=E : P α.{x : [= α : τ]} pack λγ.{},λγ.{x =[e]}

104 Elaborating Bindings Γ T : κ τ Γ B :Ξ e Γ type X=T : ϕ {X : [= τ : κ]} {X =[τ : κ]} Γ E : τ e Γ val X=E : I α.{x : [= α : τ]} pack {},{X =[e]} Γ E : τ e E non-expansive Γ val X=E : P α.{x : [= α : τ]} pack λγ.{},λγ.{x =[e]} Γ P : ϕ [= π : τ] e Γ val X=P : ϕ {X : [= π : τ]} {X = e}

105 Bonus: Sharing specifications

106 Bonus: Sharing specifications opaque & transparent value specifications val x : t vs. val x = A.y

107 Bonus: Sharing specifications opaque & transparent value specifications val x : t vs. val x = A.y opaque & transparent module specifications module X : S vs. module X = A.Y

108 Bonus: Sharing specifications opaque & transparent value specifications val x : t vs. val x = A.y opaque & transparent module specifications module X : S vs. module X = A.Y value & module refinements S where val X.y = z or S where module X.Y = Z

109 Bonus: Sharing specifications opaque & transparent value specifications val x : t vs. val x = A.y opaque & transparent module specifications module X : S vs. module X = A.Y value & module refinements S where val X.y = z or S where module X.Y = Z singleton signatures like X.Y

110 Conclusion

111 Conclusion Applicative functors are delicate

112 Conclusion Applicative functors are delicate Applicative pure, generative impure

113 Conclusion Applicative functors are delicate Applicative pure, generative impure Module equivalence = type equivalence + value equivalence

114 Conclusion Applicative functors are delicate Applicative pure, generative impure Module equivalence = type equivalence + value equivalence F-ing modules allows fairly elegant formalisation

115 Conclusion Applicative functors are delicate Applicative pure, generative impure Module equivalence = type equivalence + value equivalence F-ing modules allows fairly elegant formalisation Gory details in draft article:

116 Thank you!

117 Outtakes

118 Applicative Semantics for Functors is difficult! Leroy Russo Shao Dreyer+ unrestricted 1stclass modules impure abstraction safe module equivalence no loss of type equivalences Generative functors can be turned applicative after the fact 2 Though you have to try harder

119 Example module Set = fun Elem : ORD { type elem = Elem.t type set = list elem val empty = [] val add x s = case s of [] [x] y :: s' if not (Elem.leq x y) then y :: add x s' else if Elem.leq y x then s else x :: s val mem x s = case s of [] false y :: s' Elem.leq y x and (Elem.leq x y or mem x s') } :> SET where type elem = Elem.t

Derek Dreyer. WG2.8 Meeting, Iceland July 16-20, 2007

Derek Dreyer. WG2.8 Meeting, Iceland July 16-20, 2007 Why Applicative Functors Matter Derek Dreyer Toyota Technological Institute at Chicago WG2.8 Meeting, Iceland July 16-20, 2007 Matthias Felleisen Gets In a Good Jab Matthias Felleisen, POPL PC Chair report,