Tool-assisted spec development

Transcription

1 Typed Clojure Tool-assisted spec development Ambrose Bonnaire-Sergeant Sam Tobin-Hochstadt

2 Spec is awesome Expressive specification language Runtime instrumentation Generative testing Documentation Parsing

3 Show of hands: Who s avoided writing a spec because? Too inexperienced with spec Reverse engineering too costly to justify Partially wrote spec, but will fill it in later

4 Show of hands: Who s updated a spec because? Typo in the spec Function/data out of date with spec

5 We need help writing our specs

6 Idea Spec models how our program currently operates So run the program, and observe! Use observations to generate specs

7 Analogy: Megaphone as a function function

8 Goal: What is the spec for a megaphone? function

9 Step 1: Instrumentation function

10 Step 1: Instrumentation function

11 Step 2: Observe running program function

12 Step 2: Observe running program function

13 Step 3: Summarize execution : :

14 Step 3: Summarize execution : # # : #

15 The challenge: Deriving good specs from program execution

16 Values The observed value :foo 1 any? 2 keyword? 3 #{:foo} Specs Our choices to spec value

17 Values :foo Don t care 1 any? 2 keyword? 3 #{:foo} Specs Homogeneous Enumeration

18 Values {1 2, 3 4} Instance check 1 map? Homogeneous 2(map-of int? int?) 3 (map-of #{1 3} #{2 4}) Enumerate 4 (map-of pos? pos?) Specs Constrain

19 Values {:x 2, :y 3} 1 (map-of keyword? int?) Homogeneous 2 (def ::x int?) (def ::y int?) (s/keys :req-un [::x ::y]) Specs Entity map

20 {:shape :square :width 3} Homogeneous 1 2 (map-of keyword? (or int? keyword?)) (def ::shape keyword?) (def ::width int?) (def ::height int?) (s/keys :req-un [::shape ::width]) Entity map 3 (defmulti shape :shape) (defmethod shape [_] :square (s/keys :req-un [::shape ::width])) Values Tagged Entity map (multi-spec shape :shape) Specs

21 {:shape :square :width 3} {:shape :rect :width 3 :height 4} Second tag (map-of keyword? (or int? keyword?)) (def ::shape keyword?) (def ::width int?) Optional entries (def ::height int?) (s/keys :req-un [::shape ::width] :opt-un [::height]) (defmulti shape :shape) (defmethod shape [_] :square (s/keys :req-un [::shape ::width])) (defmethod shape [_] :rect (s/keys :req-un [::shape ::width ::height])) Values (multi-spec shape :shape) Specs

22 {:shape :square, :width 3 :inner {:shape :rect, :width 1, :height 3, Values :inner {:shape :square, :width 3}}} 1 map? 2 (map-of ) 3 (keys ) (defmulti shape :shape) Recursively (defmethod shape [_] :square defined, (s/keys :req-un [::shape ::width]) tagged entity :opt-un [::inner])) 4 (multi-spec ) Tagged Entity map 5 (defmethod shape [_] :rect map (s/keys :req-un [::shape ::width ::height] :opt-un [::inner])) (s/def ::inner (multi-spec shape :shape)) (multi-spec shape :shape)

23 Phew! Many implicit decisions

24 How do we make sense of all this? 1 View possible specs as a lattice 2 Observing different values moves along lattice

25 How do we make sense of all this? 1 View possible specs as a lattice 2 Observing different values moves along lattice

26 How do we make sense of all this? 1 View possible specs as a lattice 2 Observing different values moves along lattice

27 How do we make sense of all this? 1 View possible specs as a lattice 2 Observing different values moves along lattice

28 How do we make sense of all this? 1 View possible specs as a lattice 2 Observing different values moves along lattice

29 Build lattice using heuristics about Clojure idioms

30 Observed Chosen Spec Values number? Observed values Chosen spec (or double? integer?) double? integer?

31 Observed Chosen Spec Values number? Observed doubles (or double? integer?) 6.2 Heuristic double? integer?

32 Observed Chosen Spec Values number? Observed BigDecimal (or double? integer?) M double? integer?

33 Observed Chosen Spec Values any? {:shape :rect } (map-of keyword? any?) (map-of keyword? (or )) (multi-spec shape :shape) Entity map (s/keys :req-un [::shape])

34 Observed Chosen Spec Values any? {:shape :rect } (map-of keyword? any?) {:shape :circle } Tagged Entity map (map-of keyword? (or )) (multi-spec shape :shape) (s/keys :req-un [::shape])

35 Observed Chosen Spec Values any? Homogeneous map {:shape :rect } (map-of keyword? any?) (map-of keyword? (or )) {:shape :circle } {:x 1 :y 2} (multi-spec shape :shape) (s/keys :req-un [::shape])

36 Observed Chosen Spec Values any? {:shape :rect } (map-of keyword? any?) {:shape :circle } (map-of keyword? (or )) {:x 1 :y 2} Observed non-map data (multi-spec shape :shape) 42 sym :kw (s/keys :req-un [::shape])

37 Demo

38 Real-world Examples

39 The good

40 Names Reuse argument names Good spec tags

41 Simple collection manipulation Succinct

42 Multiple arities

43 Multi-specs {:op :vector..} {:op :map..} {:op :var..}

44 Useful aliases Extract common map into alias Usage Usage

45 Mutually recursive maps

46 Mutually recursive maps

47 The bad

48 Lost kw :min kw arg arguments :max kw arg

49 Over-specificity Only unit tested 3-tuple!

50 The ugly

51 No rest arg inference 6 args 13 args

52 The future

53 Polymorphic types

54 Polymorphic types Fragments of polymorphic types

55 Questions, Demos, and magic tricks?

56 Thanks

57 ambrosebs.com/auto-ann.html

58 Inference results via side effects

59 Step 1: 1) Generate naive type environment from dynamic inference results point : [Long Long -> {:x Long :y Long}]

60 Step 2: 2) Create local recursive types ( vertically ) f : g : h : b :

61 Step 2: 2) Create local recursive types ( vertically ) f : g : h : b :

62 Step 3: 3) Merge possibly-recursive types globally ( horizontally ) f : g : h : b :

63 Step 3: 3) Merge possibly-recursive types globally ( horizontally ) f : g : h, b :

64 Instrumentation

65 Instrumentation

66 Paths [emit (arg)] [emit (ret)] [emit (ret)]