Running Probabilistic. Programs Backwards

Transcription

1 Running Probabilistic Programs Backwards Neil Toronto * Jay McCarthy David Van Horn * * University of Maryland Vassar College ESOP /04/14

2 Roadmap Probabilistic inference, and why it s hard 1

3 Roadmap Probabilistic inference, and why it s hard Limitations of current probabilistic programming languages (PPLs) 1

4 Roadmap Probabilistic inference, and why it s hard Limitations of current probabilistic programming languages (PPLs) Contributions 1

5 Roadmap Probabilistic inference, and why it s hard Limitations of current probabilistic programming languages (PPLs) Contributions Uncomputable, compositional ways to not limit language 1

6 Roadmap Probabilistic inference, and why it s hard Limitations of current probabilistic programming languages (PPLs) Contributions Uncomputable, compositional ways to not limit language Computable, compositional ways to not limit language 1

7 (let ([x (flip 0.5)]) x) Programming Coin Flips 2

8 (let ([x (flip 0.5)]) x) Programming Coin Flips 0.5 2

9 (let ([x (flip 0.5)]) x) Programming Coin Flips

10 (let ([x (flip 0.5)] [y (flip 0.5)]) (cons x y)) Programming Coin Flips , 0.5, 2

11 (let ([x (flip 0.5)] [y (flip 0.5)]) (cons x y)) Programming Coin Flips ,, 0.5,, 2

12 (let ([x (flip 0.5)] [y (flip 0.5)]) (cons x y)) Programming Coin Flips ,, 0.5,, 2

13 Programming Coin Flips (let* ([x (flip 0.5)] [y (flip (if (equal? x heads) ))]) (cons x y)) ,, 0.5,, 2

14 Programming Coin Flips (let* ([x (flip 0.5)] [y (flip (if (equal? x heads) ))]) (cons x y)) ,, 0.5,,

15 Programming Coin Flips ,, 0.5,,

16 Programming Coin Flips ,, 0.5,,

17 Programming Coin Flips , 0.5, 0.3 2

18 Stochastic Ray Tracing sto cha stic /stō-ˈkas-tik/ adj. fancy word for "randomized" 2

19 Stochastic Ray Tracing sto cha stic /stō-ˈkas-tik/ adj. fancy word for "randomized" 2

20 Stochastic Ray Tracing ap er ture /ˈap-ə(r)-chər/ n. fancy word for "opening" 2

21 Stochastic Ray Tracing ap er ture /ˈap-ə(r)-chər/ n. fancy word for "opening" 2

22 Stochastic Ray Tracing Simulate projecting rays onto a sensor... 2

23 Stochastic Ray Tracing... and collect them to form an image 2

24 Programming Stochastic Ray Tracing Normally thousands of lines of code 2

25 Programming Stochastic Ray Tracing Normally thousands of lines of code Bears little resemblance to the physical process 2

26 Programming Stochastic Ray Tracing Normally thousands of lines of code Bears little resemblance to the physical process In DrBayes, it s simple physics simulation: (define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (dot v n))]) (if (> denom 0) (let ([t (/ (+ d (dot p0 n)) denom)]) (if (> t 0) (collision t (vec+ p0 (vec* v t)) n) #f)) #f))) 2

27 Programming Stochastic Ray Tracing Normally thousands of lines of code Bears little resemblance to the physical process In DrBayes, it s simple physics simulation: (define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (dot v n))]) (if (> denom 0) (let ([t (/ (+ d (dot p0 n)) denom)]) (if (> t 0) (collision t (vec+ p0 (vec* v t)) n) #f)) #f))) Other PPLs really aren t up to this yet 2

28 Programming Stochastic Ray Tracing Normally thousands of lines of code Bears little resemblance to the physical process In DrBayes, it s simple physics simulation: (define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (dot v n))]) (if (> denom 0) (let ([t (/ (+ d (dot p0 n)) denom)]) (if (> t 0) (collision t (vec+ p0 (vec* v t)) n) #f)) #f))) Other PPLs really aren t up to this yet The issue is one of theory, not engineering effort 2

29 Simpler Example Assume (random) returns a value uniformly in 3

30 Simpler Example Assume (random) returns a value uniformly in Density function for value of (random): p(x) x 3

31 Simpler Example Assume (random) returns a value uniformly in Density function for value of (random): p(x) x 3

32 Simpler Example Assume (random) returns a value uniformly in Density function for value of (random): p(x) x 3

33 Simpler Example Assume (random) returns a value uniformly in Density function for value of (max 0.5 (random)): pₘ(x) =???.75 pₘ(x) x 3

34 Simpler Example Assume (random) returns a value uniformly in Density function for value of (max 0.5 (random)): pₘ(x) =???.75 pₘ(x) x 3

35 Simpler Example Assume (random) returns a value uniformly in Density function for value of (max 0.5 (random)): pₘ(x) =???.75 pₘ(x) x 3

36 Simpler Example Assume (random) returns a value uniformly in Density function for value of (max 0.5 (random)): pₘ(x) doesn't exist.75 pₘ(x) x 3

37 What Can t Densities Model? 4

38 What Can t Densities Model? Results of discontinuous functions (bounded measuring devices) (let ([temperature (normal 99 1)]) (min 100 temperature)) 4

39 What Can t Densities Model? Results of discontinuous functions (bounded measuring devices) (let ([temperature (normal 99 1)]) (min 100 temperature)) Variable-dimensional things (union types) (if test? none (just x)) 4

40 What Can t Densities Model? Results of discontinuous functions (bounded measuring devices) (let ([temperature (normal 99 1)]) (min 100 temperature)) Variable-dimensional things (union types) (if test? none (just x)) Infinite-dimensional things (recursion) 4

41 What Can t Densities Model? Results of discontinuous functions (bounded measuring devices) (let ([temperature (normal 99 1)]) (min 100 temperature)) Variable-dimensional things (union types) (if test? none (just x)) Infinite-dimensional things (recursion) In general: the distributions of program values 4

42 Probability Measures Like already-integrated densities, but a primitive concept 5

43 Probability Measures Like already-integrated densities, but a primitive concept Measure of (random) is, defined by 5

44 Probability Measures Like already-integrated densities, but a primitive concept Measure of (random) is, defined by 5

45 Probability Measures Like already-integrated densities, but a primitive concept Measure of (random) is, defined by Measure of (max 0.5 (random)) defined by 5

46 Probability Measures Like already-integrated densities, but a primitive concept Measure of (random) is, defined by Measure of (max 0.5 (random)) defined by This term assigns probability 5

47 Probability Measures Like already-integrated densities, but a primitive concept Measure of (random) is, defined by Measure of (max 0.5 (random)) defined by This term assigns probability Need a way to derive measures from code 5

48 Probability Measures Via Preimages Interpret (max 0.5 (random)) as, defined 6

49 Probability Measures Via Preimages Interpret (max 0.5 (random)) as, defined Derive measure of (max 0.5 (random)) as 6

50 Probability Measures Via Preimages Interpret (max 0.5 (random)) as, defined Derive measure of (max 0.5 (random)) as where 6

51 Probability Measures Via Preimages Interpret (max 0.5 (random)) as, defined Derive measure of (max 0.5 (random)) as where Factored into random and deterministic parts: 6

52 Probability Measures Via Preimages Interpret (max 0.5 (random)) as, defined Derive measure of (max 0.5 (random)) as where Factored into random and deterministic parts: In other words, compute measures of expressions by running them backwards 6

53 Crazy Idea is Feasible If... Seems like we need: 7

54 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source 7

55 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets 7

56 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets 7

57 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets Efficient way to compute areas of preimage sets 7

58 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets Efficient way to compute areas of preimage sets Proof of correctness w.r.t. standard interpretation 7

59 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets Efficient way to compute areas of preimage sets Proof of correctness w.r.t. standard interpretation WAT 7

60 What About Approximating? Conservative approximation with rectangles: 8

61 What About Approximating? Conservative approximation with rectangles: 8

62 What About Approximating? Restricting preimages to rectangular subdomains: 9

63 What About Approximating? Restricting preimages to rectangular subdomains: 9

64 What About Approximating? Restricting preimages to rectangular subdomains: 9

65 What About Approximating? Restricting preimages to rectangular subdomains: 9

66 What About Approximating? Restricting preimages to rectangular subdomains: 9

67 What About Approximating? Restricting preimages to rectangular subdomains: 9

68 What About Approximating? Restricting preimages to rectangular subdomains: 9

69 What About Approximating? Restricting preimages to rectangular subdomains: 9

70 What About Approximating? Restricting preimages to rectangular subdomains: 9

71 What About Approximating? Restricting preimages to rectangular subdomains: 9

72 What About Approximating? Restricting preimages to rectangular subdomains: 9

73 What About Approximating? Restricting preimages to rectangular subdomains: 9

74 What About Approximating? Sampling: exponential to quadratic (e.g. days to minutes) 10

75 What About Approximating? Sampling: exponential to quadratic (e.g. days to minutes) 10

76 Contribution: Making This Crazy Idea Feasible Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets Efficient way to compute volumes of preimage sets Proof of correctness w.r.t. standard interpretation 11

77 Contribution: Making This Crazy Idea Feasible Standard interpretation of programs as pure functions from a random source Efficient way to compute abstract preimage subsets Efficient representation of arbitrary sets Efficient way to compute volumes of preimage sets Proof of correctness w.r.t. standard interpretation 11

78 Contribution: Making This Crazy Idea Feasible Standard interpretation of programs as pure functions from a random source Efficient way to compute abstract preimage subsets Efficient representation of abstract sets Efficient way to compute volumes of preimage sets Proof of correctness w.r.t. standard interpretation 11

79 Contribution: Making This Crazy Idea Feasible Standard interpretation of programs as pure functions from a random source Efficient way to compute abstract preimage subsets Efficient representation of abstract sets Efficient way to sample uniformly in preimage sets Proof of correctness w.r.t. standard interpretation 11

80 Contribution: Making This Crazy Idea Feasible Standard interpretation of programs as pure functions from a random source Efficient way to compute abstract preimage subsets Efficient representation of abstract sets Efficient way to sample uniformly in preimage sets Efficient domain partition sampling Proof of correctness w.r.t. standard interpretation 11

81 Contribution: Making This Crazy Idea Feasible Standard interpretation of programs as pure functions from a random source Efficient way to compute abstract preimage subsets Efficient representation of abstract sets Efficient way to sample uniformly in preimage sets Efficient domain partition sampling Efficient way to determine whether a domain sample is actually in the preimage (just use standard interpretation) Proof of correctness w.r.t. standard interpretation 11

82 How Many Meanings? Start with pure programs, then lift by threading a random store 11

83 How Many Meanings? Start with pure programs, then lift by threading a random store Nonrecursive, nonprobabilistic programs:,, 11

84 How Many Meanings? Start with pure programs, then lift by threading a random store Nonrecursive, nonprobabilistic programs:,, Add 3 semantic functions for recursion and probabilistic choice 11

85 How Many Meanings? Start with pure programs, then lift by threading a random store Nonrecursive, nonprobabilistic programs:,, Add 3 semantic functions for recursion and probabilistic choice Full development needs 2 more to transfer theorems from measure theory... 11

86 How Many Meanings? Start with pure programs, then lift by threading a random store Nonrecursive, nonprobabilistic programs:,, Add 3 semantic functions for recursion and probabilistic choice Full development needs 2 more to transfer theorems from measure theory oh, and 1 more to collect information for Monte Carlo integration 11

87 How Many Meanings? Start with pure programs, then lift by threading a random store Nonrecursive, nonprobabilistic programs:,, Add 3 semantic functions for recursion and probabilistic choice Full development needs 2 more to transfer theorems from measure theory oh, and 1 more to collect information for Monte Carlo integration Tally: = 9 semantic functions, 11 or 12 rules each 11

88 Enter Category Theory Moggi (1989): Introduces monads for interpreting effects 11

89 Enter Category Theory Moggi (1989): Introduces monads for interpreting effects Other kinds of categories: idioms, arrows 11

90 Enter Category Theory Moggi (1989): Introduces monads for interpreting effects Other kinds of categories: idioms, arrows Arrow defined by type constructor combinators: and these 11

91 Enter Category Theory Moggi (1989): Introduces monads for interpreting effects Other kinds of categories: idioms, arrows Arrow defined by type constructor combinators: and these Arrows are always function-like 11

92 Reducing Complexity Function arrow: is just 11

93 Reducing Complexity Function arrow: is just 11

94 Reducing Complexity Function arrow: is just 11

95 Reducing Complexity Function arrow: is just 11

96 Reducing Complexity Function arrow: is just 11

97 Reducing Complexity Function arrow: is just 11

98 Reducing Complexity Function arrow: is just 11

99 Reducing Complexity Function arrow: is just 11

100 Reducing Complexity Function arrow: is just 11

101 Pair Preimages 12

102 Pair Preimages 12

103 Pair Preimages : 12

104 Pair Preimages and : 12

105 Pair Preimages : 12

106 Correctness Theorems For Low, Low Prices Define 12

107 Correctness Theorems For Low, Low Prices Define Derive and others so that distributes; e.g. 12

108 Correctness Theorems For Low, Low Prices Define Derive and others so that distributes; e.g. Distributive properties makes proving this very easy: Theorem (correctness). For all,. 12

109 Correctness Theorems For Low, Low Prices Define Derive and others so that distributes; e.g. Distributive properties makes proving this very easy: Theorem (correctness). For all,. In English: computes preimages under. 12

110 Correctness Theorems For Low, Low Prices Define Derive and others so that distributes; e.g. Distributive properties makes proving this very easy: Theorem (correctness). For all,. In English: computes preimages under. Other correctness proofs are similarly easy: prove 5 distributive properties 12

111 Correctness Theorems For Low, Low Prices Define Derive and others so that distributes; e.g. Distributive properties makes proving this very easy: Theorem (correctness). For all,. In English: computes preimages under. Other correctness proofs are similarly easy: prove 5 distributive properties Can add (random) and recursion to all semantics in one shot 12

112 Abstraction Rectangle: An interval or union of intervals, a subset of for rectangles and, or 13

113 Abstraction Rectangle: An interval or union of intervals, a subset of for rectangles and, or Easy representation; easy intersection, join (which overapproximates union), empty test, etc. 13

114 Abstraction Rectangle: An interval or union of intervals, a subset of for rectangles and, or Easy representation; easy intersection, join (which overapproximates union), empty test, etc. Define (and therefore ) by replacing sets and set operations with rectangles and rectangle operations 13

115 Abstraction Rectangle: An interval or union of intervals, a subset of for rectangles and, or Easy representation; easy intersection, join (which overapproximates union), empty test, etc. Define (and therefore ) by replacing sets and set operations with rectangles and rectangle operations Recursion is somewhat tricky requires fine control over recursion depth or if choices 13

116 In Theory... Theorem (sound). computes overapproximations of the preimages computed by. Consequence: Sampling in abstract preimages doesn t leave anything out 14

117 In Theory... Theorem (sound). computes overapproximations of the preimages computed by. Consequence: Sampling in abstract preimages doesn t leave anything out Theorem (decreasing). the given subdomain. never returns preimages larger than Consequence: Refining abstract preimage sets never results in a worse approximation 14

118 In Theory... Theorem (sound). computes overapproximations of the preimages computed by. Consequence: Sampling in abstract preimages doesn t leave anything out Theorem (decreasing). the given subdomain. never returns preimages larger than Consequence: Refining abstract preimage sets never results in a worse approximation Theorem (monotone). is monotone. Consequence: Partitioning and then refining never results in a worse approximation 14

119 In Practice... Theorems prove this always works: 15

120 In Practice... Theorems prove this always works: 15

121 In Practice... Theorems prove this always works: 15

122 In Practice... Theorems prove this always works: 15

123 In Practice... Theorems prove this always works: 15

124 In Practice... Theorems prove this always works: 15

125 In Practice... Theorems prove this always works: 15

126 In Practice... Theorems prove this always works: 15

127 In Practice... Theorems prove this always works: 15

128 Program Domain Values 16

129 Program Domain Values Program inputs are infinite binary trees: 16

130 Program Domain Values Program inputs are infinite binary trees: Every expression in a program is assigned a node 16

131 Program Domain Values Program inputs are infinite binary trees: Every expression in a program is assigned a node Implemented using lazy trees of random values 16

132 Program Domain Values Program inputs are infinite binary trees: Every expression in a program is assigned a node Implemented using lazy trees of random values No probability density for domain, but there is a measure 16 16

133 Example: Stochastic Ray Tracing 17

134 Example: Probabilistic Verification (struct/drbayes float-any ()) (struct/drbayes float (value error)) 18

135 Example: Probabilistic Verification (struct/drbayes float-any ()) (struct/drbayes float (value error)) (define/drbayes (flsqrt x) (if (float-any? x) x (let ([v (float-value x)] [e (float-error x)]) (cond [(negative? v) (float-any)] [(zero? v) (float 0 0)] [else (float (sqrt v) (+ (- 1 (sqrt (- 1 e))) (* 1/2 epsilon)))])))) 18

136 Example: Probabilistic Verification (struct/drbayes float-any ()) (struct/drbayes float (value error)) (define/drbayes (flsqrt x) (if (float-any? x) x (let ([v (float-value x)] [e (float-error x)]) (cond [(negative? v) (float-any)] [(zero? v) (float 0 0)] [else (float (sqrt v) (+ (- 1 (sqrt (- 1 e))) (* 1/2 epsilon)))])))) Idea: sample e where (> (float-error e) threshold) 18

137 Example: Probabilistic Verification (struct/drbayes float-any ()) (struct/drbayes float (value error)) (define/drbayes (flsqrt x) (if (float-any? x) x (let ([v (float-value x)] [e (float-error x)]) (cond [(negative? v) (float-any)] [(zero? v) (float 0 0)] [else (float (sqrt v) (+ (- 1 (sqrt (- 1 e))) (* 1/2 epsilon)))])))) Idea: sample e where (> (float-error e) threshold) Verified flhypot, flsqrt1pm1, flsinh in Racket s math library, as well as others 18

138 Examples: Other Inference Tasks Typical Bayesian inference Hierarchical models Bayesian regression Model selection 19

139 Examples: Other Inference Tasks Typical Bayesian inference Hierarchical models Bayesian regression Model selection Atypical Programs that halt with probability < 1, or never halt Probabilistic context-free grammars with context-sensitive constraints 19

140 Summary Probabilistic inference is hard, so PPLs have been popping up 20

141 Summary Probabilistic inference is hard, so PPLs have been popping up Interpreting every program requires measure theory 20

142 Summary Probabilistic inference is hard, so PPLs have been popping up Interpreting every program requires measure theory Defined a semantics that computes preimages 20

143 Summary Probabilistic inference is hard, so PPLs have been popping up Interpreting every program requires measure theory Defined a semantics that computes preimages Measuring abstract preimages or sampling in them carries out inference 20

144 Summary Probabilistic inference is hard, so PPLs have been popping up Interpreting every program requires measure theory Defined a semantics that computes preimages Measuring abstract preimages or sampling in them carries out inference Can do a lot of cool stuff that s normally inaccessible 20

145 21