A Modular Package Language for Haskell

Transcription

1 A Modular Package Language for Haskell Scott Kilpatrick MPI-SWS Joint work with: Derek Dreyer (MPI-SWS) Simon Marlow (Microsoft Research) Simon Peyton Jones (Microsoft Research) University of Pennsylvania 28 November 2012

2 Haskell Packages Today... 2

3 Modules Organize Code unit of programming & reasoning Data/Set.hs module Data.Set where data Set a =... add s x = GHC Haskell compiler; compiles modules access other mods code via importing Foo.hs module Foo where import Data.Set data T =... Set... foo x y =... add... 3

4 Packages Organize Modules unit of distribution & authorship alicelib.cabal name: alicelib version: 1.0 build-depends: data == 2.* exposed-modules: Alice use names & versions to track package dependencies Alice.hs module Alice where import Data.Set <code> Cabal package specification and tool suite for building/ distributing packages 4

5 Building Packages alicelib.cabal name: alicelib version: 1.0 build-depends: data == 2.* exposed-modules: Alice data == 2.* data Alice.hs module Alice where import Data.Set <code> Ingrid 5

6 Four weaknesses: 6

7 Problem #1: Uncheckable alicelib.cabal name: alicelib version: 1.0 build-depends: data == 2.* exposed-modules: Alice Alice.hs module Alice where import Data.Set <code> Alice must build data to type check Alice.hs Builds, say, data-2.1 But Ingrid picked 2.4! Informal policy re: version numbers and types Hoping, not checking 7

8 Problem #2: Linear Development alicelib boblib Pkgs must be created in dependency order alicelib, then boblib, then carollib carollib 8

9 Problem #2: Linear Development alicelib impl s Alice API Pkgs must be created in dependency order alicelib, then boblib, boblib impl s Bob API then carollib Program against APIs Develop in any order carollib Link together in the end 9

10 Problem #2: Linear Development Alice.hs module Alice where x = 5 impl s Alice API x :: Int Bob.hs module Bob where import Alice y = iszero x impl s Bob API y :: Bool incremental vs. separate compilation Carol.hs module Carol where import Bob z = not y 10

11 Problem #3: Too Inflexible data-2.* qc-2.* public private alicelib-2.0 data >= 2 && < 4 public boblib-1.0 qc-3.* private Ingrid wants carollib public public carollib-0.0 Ingrid 11

12 Problem #3: Too Inflexible data-2.* data >= 2 && < 4 alicelib-2.0 qc-2.* boblib-1.0 qc-3.* Ingrid wants carollib Cabal requires single version for all depend s Can satisfy data constraints but not qc carollib-0.0 Ingrid Cabal naively complains Code would be fine if Cabal considered types 12

13 Problem #4: Weak Reusability Ingrid alicelib-1.0 Ingrid.hs module Ingrid where import Alice import Data3.0Stuff <code> Ingrid uses alicelib and data-3.0 together Can t satisfy dependency But Alice code would also work with data-3.0 data-2.* data-3.0 data-2.* dependency was too limiting! 13

14 Diagnosis: Informal APIs boblib.cabal name: boblib version: 1.0 build-depends: data >= 2 && < 4 qc == 3.*... I 2 I 1 Package name and version range form an API! Informal, intended API No formal checking 14

15 Packages Need Interfaces! 15

16 Using Mixins Packages are mixin modules Namespaces containing defined and undefined components, each with name and interface Link together by name to fill in components Permits recursive linking * based on MixML [Dreyer and Rossberg, ICFP 2008] 16

17 Mixin Solution package data-2-sigs where Data.Set :: [ data Socket];... package alicelib-1.0 where include data-2-sigs; Alice = import Data.Set <code> dependency abstracted over an interface, not a name and version range use structural matching to link in anything that provides a compatible Data.Set 17

18 Contributions Package language and type system based on mixins Introduces proper abstraction of dependencies Offers greater flexibility, checkability, reuse Largely agnostic to underlying language 18

19 Outline 1. Introduction 2. Language features 3. Semantics 4. The rest 19

20 Package Definitions and Bindings package threemods where A = [First.hs]; B = [Second.hs]; C = B Defines package with 3 bindings (2 distinct mods) Separates file names from bindings names -- First.hs x = 5 -- Second.hs import A y = x + 5 Binding order explicit Earlier bindings bound in the context of later ones 20

21 Signatures (Holes) package onehole where A :: [Sig.hsig]; B = [Other.hs] Signatures are abstract mods; defined eventually Variable references look like defined mods -- Sig.hsig x :: Int only static info -- Other.hs import A y = x + 5 But someone must eventually impl them! Package definition encapsulates holes; others instantiate them 21

22 Package Dependencies package twomods where A = [First.hs]; B = [Second.hs] package threeagain where include twomods; C = B include injects bindings into present namespace Realizes dependencies Gets both mods & holes Included bindings merge with explicit bindings All part of this package 22

23 Linking Bindings Link/merge bindings by name Unifies compatible module components Performs linking, not shadowing Three cases when linking two components with same name: mod/mod mod/sig sig/sig 23

24 mod/mod Linking package top where Top = [T.hs] package left where include top; Left = [L.hs] package right where include top; Right = [R.hs] package bottom where include left; include right; Bottom = [B.hs] unifies identical* Top modules from left and right * identity discussed later 24

25 sig/sig Linking package mylib where View :: [JustX.hsig];... x :: Int package yourlib where View :: [JustY.hsig];... y :: Bool package ourlib where include mylib; within mylib, View has x :: Int include yourlib; yourlib, y :: Bool... ourlib, { x :: Int y :: Bool unifies View sigs into glb sig 25

26 mod/sig Linking package httplib where Socket :: [ ]; HTTP =... data Socket new :: () -> Socket package socketlib where Socket = [ ] data Socket = MkS Int new _ = MkS 4 package main where include socketlib; include httplib checks that the mod matches the sig 26

27 Outline 1. Introduction 2. Language features 3. Semantics 4. The rest 27

28 Module Identity Each module is given an identity Parameterized by identities of imports Captures dependency graph package mod import graph identities package idents where A = [1.hs]; B = [2.hs]; C = [3.hs]; D = C; A C B D ν A #(1.hs) ν B #(2.hs) ν A ν C #(3.hs) ν A ν B ν D ν C 28

29 Applicative Identities Distinguish instantiations of modules Share identical instantiations Permit multiple instantiations in one package package onehole where A :: [ x :: Int ]; B = [Other.hs] ν B #(Other.hs) α A, for α A = ν impl1, ν impl2,... 29

30 Type Representation Need to know when types equivalent! module where import A (T) gett :: () -> T module where import B (T) putt :: T -> ()... putt (gett ())... T = T?? 30

31 Type Representation Today, Haskell types represented as GHC.Types.Bool defining module s name syntactic constructor We represent types similarly [ν]bool defining module s semantic identity syntactic constructor 31

32 package httpd where Socket :: [ ]; Server = [ ] data T import Socket(T) data Conn = MkConn T package yourlib where include httpd; include socket-1.0 package mylib where include httpd; include socket-1.0 package main where A = include yourlib ; B = include mylib ; Main = [ ] import A.Server(Conn) import B.Server(Conn)... only one Conn type: [#(Server.hs) ν Sock1.0 ]Conn 32

33 package httpd where Socket :: [ ]; Server = [ ] data T import Socket(T) data Conn = MkConn T package yourlib where include httpd; include socket-1.0 package mylib where include httpd; include socket-1.5 package main where A = include yourlib ; B = include mylib ; Main = [ ] import A.Server(Conn) import B.Server(Conn)... two distinct Server identities two distinct Conn types: [#(Server.hs) ν Sock1.0 ]Conn [#(Server.hs) ν Sock1.5 ]Conn 33

34 Elaboration Package specifications/tools are too weak Our language improves those weaknesses and offers a translation down into the ordinary Haskell modules packages with interfaces elaboration plain Haskell modules GHC compilation 34

35 Elaboration Idea: Create a mod file for each identity Rewrite logical imports to hard links Similar to C++ template expansion Soundness (Conjecture!): Well-typed package elaborates to a set of import-closed, well-typed Haskell modules 35

36 Elaboration package httpd where Socket :: [ ]; Server = [ ] data T import Socket(T) data Conn = MkConn T α module data T where module #(Server.hs) α where import α (T) data Conn = MkConn T a GHC boot file imports are rewritten to access semantic hard links 36

37 Elaboration package main where A = include yourlib ; B = include mylib ; Main = [ ] import A.Server(Conn) import B.Server(Conn) from Socket1.0 module ν 1.0 where data T = module #(Server.hs) ν 1.0 import ν 1.0 (T) data Conn = MkConn T where module #(Main.hs) (#(Server.hs) ν 1.0 )(#(Server.hs) ν 1.0 ) import #(Server.hs) ν 1.0 (Conn) import #(Server.hs) ν 1.0 (Conn)... Socket impl, single Server instance, and Main importing it (x2) where 37

38 Elaboration package main where A = include yourlib ; B = include mylib ; Main = [ ] import A.Server(Conn) import B.Server(Conn)... module import import from Socket1.0 module ν 1.0 where data T = from Socket1.5 module ν 1.5 where data T = module #(Server.hs) ν 1.0 where module #(Server.hs) ν 1.5 import ν 1.0 (T) import ν 1.5 (T) data Conn = MkConn T data Conn = MkConn T #(Main.hs) (#(Server.hs) ν 1.0 )(#(Server.hs) ν 1.5 ) #(Server.hs) ν 1.0 #(Server.hs) ν 1.5 (Conn) (Conn) where source file for Server duplicated where the two distinct Conn types are clear 38

39 Outline 1. Introduction 2. Language features 3. Semantics 4. The rest 39

40 Not Covered Haskell type classes Mixins vs. functors Package thinning Recursion within/between packages Type checking and specification 40

41 Conclusion Today s untyped pkgs suffer weaknesses Pkgs benefit from typed module system Abstract over dependencies Clean elaboration into ordinary source files 41

42 Conclusion Today s untyped pkgs suffer weaknesses Pkgs benefit from typed module system Abstract over dependencies Clean elaboration into ordinary source files Thanks! 41

43 Backup Slides 42

44 Why Not (ML) Modules? Different model of program modularity Pkg lang encapsulates entire Haskell progs: Core lang terms are whole Haskell modules Core lang types are whole Haskell signatures val x : int = 3 ( ) signature where val M : data T = y :: T ( ) module where data T = MkT Int y = MkT 5 43

45 Why Not (ML) Modules? Could represent packages with functors With dependencies as functor params mylib-1.0 fun( ){ ( )} B : BASESIG4, val Mine = Q : QCSIG2 module where import B.Prelude... 44

46 Why Not (ML) Modules? Need dependencies to agree on things Functors require verbose sharing specs! How to avoid that? fun( B : BASESIG4, *) { } M : MYLIBSIG,... Y : YRLIBSIG * s.t. B = (base of M) = (base of Y) 45

47 Thinning Use thinning to filter packages, P only p At defn site, expose only public modules At use site, get only desired mods from pkg containers = mylib = filters out Tree Set =...; Tree =... include (containers only Set); Foo = [...Set...] filters out Set only Foo 46

48 Closer Look at Thinning We can t ignore holes via thinning! Forms partition on pkg s dep. graph 1. Directly desired modules 2. Indirectly desired modules (trans. closure) 3. Irrelevant modules Discard the irrelevant stuff 47

49 Closer Look at Thinning containers = include basesig; Set = [...Prelude...]; include arraysig; Graph = [...Array...] inherits an Array hole from arraysig; Graph uses that hole foolib = foolib : include containers; Foo = [...Set...] Prelude Set Array Graph Foo hole def d hole def d def d Foo doesn t need Graph, but still clients must provide Array! 48

50 Closer Look at Thinning containers = include basesig; Set = [...Prelude...]; include arraysig; Graph = [...Array...] : Prelude Set Array Graph hole def d hole def d containers only Set necessary desired irrelevant Prelude Set Array Graph : Prelude Set hole def d foolib = include (containers only Set); Foo = [...Set...] : Prelude Set Foo hole def d def d 49

51 Semantics Static semantics based on MixML s New additions: Module identities / stamps Initial static shaping pass 50

52 Pkg Shaping Type checking for pkg s structure only Performs no GHC type checking Determines dep. graph of all entities Necessary for recursive modules But handled differently in MixML 51

53 Judgments Typing Γ; Σ E :Σ Shaping Γ sh E : α.(l; Σ) Thinning }} \ (L; Ξ) only p (L p ; Ξ ); ν ; p 52

54 MixML Linking { R } { } { } R L 1 locates α 1 R 1 # Σ 2 Γ; R R 1 L 1 ; β 1 mod 1 : Σ 1 L 2 locates α 2 R 2 # Σ 1 Γ, X: Σ 1 ; R R 2 L 2 ; β 2 stat mod 2 : Σ 2 (L 1 ; Σ 1 ) (L 2 ; Σ 2) δ α 1, α 2 fresh Γ, X: δσ 1 ; R R 2 δl 2 ; β 2 mod 2 : Σ 2 δσ 1 + Σ 2 Σ Γ; R R 1 R 2 ; β 1, β 2 (X = mod 1 ) with mod 2 : Σ (LINK) 53

55 Related Work SMLSC [Swasey et al., ML 2006] Package impls and package interfaces Unify dependencies when linking Nix [Dolstra et al.] Packages as a functional language, but untyped 54