Runtime types

Transcription

1 Runtime types LexiFi March 2017 Bloomberg/LexiFi workshop

2 OCaml and LexiFi LexiFi has been (happily) using OCaml since Strong involvment in the OCaml ecosystem: Member of the OCaml Consortium. Active role in core OCaml team since Major upstreamed contributions: native dynlink, first-class modules, extension points and PPX, advanced compiler warnings, etc. Open source packages (gen_js_api, dead_code_analyzer, ocaml-vdom, landmarks, csml, cmt_annot, ppx_tools). Funding community efforts (meetups, interns, OCaml consultants).

3 The LexiFi s OCaml compiler LexiFi has always maintained a local fork of the OCaml compiler. Redistributing this fork is made possible by the OCaml Consortium special license. Control of our entire implementation stack has proved extremely useful in several situations, e.g.: Cherry picking serious bug fixes ahead of OCaml releases. Early days of the Bbg/LexiFi project: tweaks for shipping IBox as a static library supportng older libc versions. Another customer struggles with long compilation times => Linear Scan allocator (now upstreamed). Better support for relocating the toolchain.

4 The LexiFi s OCaml compiler Ongoing effort to reduce the diff with upstream in order to reduce maintenance efforts: try to upstream local extensions, revert less useful ones. We usually synchronize with OCaml on official releases (once validated on our code base) + local cherry-picking of major bug fixes. Moving towards a full-fledged OCaml distribution and a better integration with the ecosystem (OPAM).

5 Local changes: syntax Alphanumeric infix operators. Date literals. Embedded module signatures.

6 Local changes: standard library A few useful additions to Set/Map, and optimized existing functions. List.map tail-recursive.

7 Local changes: type-checking Lazy local let-bindings. Structural subtyping of declared types. Strict dependency flag for more robust builds. Warnings on missed punning. Implicit subtyping (deprecated).

8 Local changes: backend More optimizations for lazy values. Embedded x86/amd64 machine code generator and COFF file generation.

9 Local changes: runtime types Key component for many of our internal libraries. Relatively small patch to the compiler. Collaboration with academic researchers for future upstream inclusion (unclear).

10 Type-driven generic programming in OCaml Type-driven generic programming can avoid a lot of boilerplate code: serialization (variants, JSON, XML), UI (e.g. BBUI), property-based testing, SQL mapping, interacting with dynamic languages (e.g. our template language). Unfortunately, the OCaml compiler erases types. 0, \000, false, None, [] have the same runtime representation.

11 Type-driven generic programming in OCaml Syntactic approach: generate code at compilation time (Camlp4/PPX). Runtime types: let the compiler generate values that represent types, provide libraries that allows writing generic operations based on these representations. Hybrid approach: PPX generates type witnesses, then manipulated by libraries. Disclaimer: runtime types not suited for performance-critical code (although a lot of care has been put to make type-safe inspection of type efficient).

12 Types of types An untyped low-level concrete representation of types: type stype = DT_int DT_string DT_list of stype... A layer of phantom typing to connect a static type and its dynamic representation: type 'a ttype (* internally = stype, but static guarantee that the representation is coherent with 'a *) Higher-level types to allow writing type-safe generic functions, such as Mlfi_xtypes.xtype.

13 Building a type representation explicitly let t = (ttype_of: int) in assert (stype_of_ttype t = DT_int)

14 Building a type representation with unification let t : int ttype = (ttype_of: _) in assert (stype_of_ttype t = DT_int)

15 Implicit parameters Missing labeled parameters of type ttype are automatically inserted: let f ~(t:int ttype) () = t let t = f () (* equivalent to: f ~t:(ttype_of: _) () and to: f ~t:(ttype_of: int ttype) () *)

16 Implicit parameters and polymorphic functions let f ~(t: 'a ttype) (_x : 'a) = t let d = f 42

17 Example: dynamics type dyn = Dyn: 'a ttype * 'a -> dyn let dyn ~(t : 'a ttype) (x: 'a) = Dyn (t, x)

18 Supported types: simple cases type stype = DT_int DT_float DT_string DT_date DT_tuple of stype list DT_list of stype DT_array of stype DT_option of stype DT_arrow of string * stype * stype...

19 Supported types type stype =... DT_abstract of string * stype list (* for types with a global path (e.g. (int, string) Foo.Bar.t *) DT_object of (string * stype) list (* only non-polymorphic methods, recursion not allowed *) DT_prop of (string * string) list * stype (* type t =... + [foo = "bar"; bar = "foo"] *) DT_var of int (* internal use *)

20 Supported types: data types type stype =... DT_node of node (* sum/record, allows recursion *) and node = { mutable rec_descr: node_descr; rec_name: string; rec_args: stype list; (*... some internal fields *) } and node_descr = DT_variant of variant_descr DT_record of record_descr

21 Data types and variant_descr = { variant_constrs: (string * props * variant_args) list; } and variant_args = C_tuple of stype list C_inline of stype (* a DT_record node *) and record_descr = { record_fields: (string * props * stype) list; record_repr: record_repr; }

22 Abstract types Only globally accessible abstract types can use the DT_abstract representation. For other abstract types (in functor s argument or result, or locally abstract types), one must export the ttype explicitly with the same name as the type: module F(X: sig end) : sig type foobar val foobar: foobar ttype end = struct type foobar = int let foobar = (ttype_of: foobar) end Current limitation: this only works for non-parametric abstract types.

23 Abstract type and functor module type S = sig type t val t: t ttype end module F(X: S) : S = struct type t = X.t list let t = (ttype_of: t) end

24 Locally abstract type let f (type t) (t : t ttype) = (ttype_of: t list)

25 Explicit binding DEPRECATED FORM! let f (t: 'a ttype) = let type : 'a = t in (ttype_of: 'a list)

26 Using runtime types: dynamic equality Dynamic witness of the equality of two types: module TypEq : sig type (_, _) t = Eq: ('a, 'a) t... end Checking equality: val ttypes_equality: 'a ttype -> 'b ttype -> ('a, 'b) TypEq.t option

27 Dynamics type dyn = Dyn: 'a ttype * 'a -> dyn let dyn ~t x = Dyn (t, x) let undyn (type t) ~(t : t ttype) (Dyn (s, x)) : t = match ttypes_equality t s with Some TypEq.Eq -> x None -> failwith "Invalid type" val dyn: t:'a ttype -> 'a -> dyn val undyn: t:'a ttype -> dyn -> 'a

28 Variants type variant = V_unit V_bool of bool V_int of int V_float of float V_string of string V_date of date V_tuple of variant list V_list of variant list V_array of variant array V_option of variant option V_record of (string * variant) list V_constructor of string * variant option V_variant of variant V_lazy of variant Lazy.t val variant: t:'a ttype -> 'a -> variant val of_variant: t:'a ttype -> variant -> 'a

29 stype <-> ttype val stype_of_ttype: 'a ttype -> stype type sttype = Ttype: 'a ttype -> sttype val sttype_of_stype: stype -> sttype

30 stype -> ttype: example let roundtrip s x = let (Ttype t) = sttype_of_stype s in variant ~t (of_variant ~t x) val roundtrip: stype -> variant -> variant

31 Type-safe deconstruction of ttype val xtype_of_ttype: 'a ttype -> 'a xtype type 'a xtype = Unit: unit xtype Bool: bool xtype Int: int xtype Float: float xtype String: string xtype...

32 Example: a generic value printer let print (type t) ~(t : t ttype) (x : t) = match xtype_of_ttype t with Unit -> (* t == unit *) print_string "()" Bool -> (* t == bool *) Printf.printf "%b" x Int -> (* t == int *) Printf.printf "%i" x Float -> (* t == float *) Printf.printf "%f" x String -> (* t == string *) Printf.printf "%S" x...

33 Type-safe deconstruction, built-in parametric types type 'a xtype =... Option: 'b ttype * 'b xtype Lazy.t -> 'b option xtype List: 'b ttype * 'b xtype Lazy.t -> 'b list xtype Array: 'b ttype * 'b xtype Lazy.t -> 'b array xtype Lazy: ('b ttype * 'b xtype Lazy.t) -> 'b Lazy.t xtype Prop: ((string * string) list * 'a ttype * 'a xtype Lazy.t) -> 'a xtype...

34 Example: a generic value printer, continued let rec print (type t) ~(t : t ttype) (x : t) = match xtype_of_ttype t with... List (s, _) -> (* t == 'b list; s : 'b ttype *) print_string "["; List.iter (fun y -> print ~t:s y; print_string ";") x; print_string "]" Does not work because polymorphic recursion must be made explicit: let rec print: type t. t:t ttype -> t -> unit = fun ~t x ->...

35 Sum types type 'a xtype = Sum: 'a Sum.t -> 'a xtype... module Sum: sig type 's t val constructors: 's t -> 's ecstr array val get_constructor_index: 's t -> 's -> int val lookup_constructor: 's t -> string -> int... end type 's ecstr = C: ('s, 't) C.t -> 's ecstr module C : sig type ('s, 't) t val ttype: (_, 't) t -> 't ttype val name: _ t -> string val project: ('s, 't) t -> 's -> 't option val inject:('s, 't) t -> 't -> 's... end

36 Sum type let rec print: type t. t:t ttype -> t -> unit = fun ~t x -> match xtype_of_ttype t with... Sum sum -> let (C cstr) = Sum.constructor sum x in Printf.printf "%s " (C.name cstr); let y = C.project_exn cstr x in print ~t:(c.ttype cstr) y

37 Record types type 'a xtype = Record: 'a Record.t -> 'a xtype... module Record = struct type 's t val fields: 's t -> 's efield list val build: 's t -> 's field_builder -> 's val make: 's t -> ('s record_builder -> unit) -> 's... end type 's efield = F: ('s, 't) F.t -> 's efield type 's field_builder = { mk: 't. ('s, 't) F.t -> 't } type 'a record_builder module F : sig type ('s, 't) t val ttype: (_, 't) t -> 't ttype val name: _ t -> string val get: ('s, 't) t -> 's -> 't val set: ('s, 't) t -> 's record_builder -> 't -> unit...

38 Record types let rec print: type t. t:t ttype -> t -> unit = fun ~t x -> match xtype_of_ttype t with... Record record -> print_string "{"; List.iter (fun (F field) -> Printf.printf "%s = " (F.name field); print ~t:(f.ttype field) (F.get field x); print_string ";" ) (Record.fields record); print_string "}"

39 Type-indexed tables Using monomorphic data structures indexed by stype. type 'a pr = ('a -> unit) type cprinter = Pr: 'a ttype * 'a pr -> cprinter let prs: cprinter_printer TypeHMap.t ref = ref TypeHMap.empty let register_printer ~t pr = prs := TypeHMap.add (stype_of_ttype t) (Pr (t, pr))!prs

40 Type-indexed tables Keep ttype around and check for dynamic equality: let assert_eq t1 t2 = match ttypes_equality t1 t2 with None -> assert false Some eq -> eq let find_printer (type t) (t : t ttype) : t pr option = let s = stype_of_ttype t in match TypeHMap.find_opt s!prs with Some (Pr (ty, pr)) -> (* pr: 'a pr; ty : 'a ttype *) let TypEq.Eq = assert_eq t ty in (* 'a == t *) Some pr None -> None

41 Type-indexed tables let rec print: type t. t:t ttype -> t -> unit = fun ~t x -> match find_printer t with Some pr -> pr x None -> match xtype_of_ttype t with...