SpaceSearch: A Library for Building and Verifying Solver-Aided Tools

Size: px

Start display at page:

Download "SpaceSearch: A Library for Building and Verifying Solver-Aided Tools"

Lambert Washington
5 years ago
Views:

1 SpaceSearch: A Library for Building and Verifying Solver-Aided Tools Konstantin Steven S. Stefan Weitz Lyubomirsky Heule Emina Torlak Michael D. Ernst Zachary Tatlock

4 Reduction

5 Reduction SMT

6 Reduction SMT

7 SpaceSearch Reduction SMT

8 The Border Gateway Protocol

9 The Border Gateway Protocol

10 The Border Gateway Protocol

11 The Border Gateway Protocol

12 The Border Gateway Protocol

13 The Border Gateway Protocol neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outcustomer permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list outprovider permit

14 The Border Gateway Protocol neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outcustomer permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list outprovider permit

15 The Border Gateway Protocol neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outcustomer permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list outprovider permit

16 The Border Gateway Protocol neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outcustomer permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list outprovider permit

17 The Border Gateway Protocol neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outcustomer permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list outprovider permit

18 The Border Gateway Protocol neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outcustomer permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list outprovider permit

The Border Gateway Protocol neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4!

19 The Border Gateway Protocol neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outcustomer permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list outprovider permit

20 Specification Bagpipe Spec Holds neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration Spec Violation

21 Specification Bagpipe Spec Holds neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration Spec Violation

22 Bagpipe t:trace( ), check(, t) Specification Spec Holds neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration Spec Violation

23 Bagpipe Specification t:trace( ), check(, t) configure terminal Spec Holds neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration Spec Violation

24 Bagpipe Specification t:trace( ), check(, t) fin Reduce [OOPSLA 16] t:inittrace( ), check(, t) Spec Holds neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration Spec Violation

25 Bagpipe Specification t:trace( ), check(, t) fin Reduce [OOPSLA 16] t:inittrace( ), check(, t) Spec Holds neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration Spec Violation

Bagpipe Specification neighbor 8.212.226.20 remote-as 2 neighbor 172.16.254.1 remote-as 3 neighbor 95.112.23.51 remote-as 4! prefix lists ip prefix-list outc permit 0.0.0.0/0 le 32 ip prefix-list outpeer permit 8.

26 Bagpipe Specification neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration t:trace( ), check(, t) fin Reduce t:inittrace( ), check(, t) Reduce? SMT [OOPSLA 16] Spec Holds Spec Violation

27 t:inittrace( ), check(, t) neighbor

28 t:inittrace( ), check(, t) neighbor inittrace( )

29 t:inittrace( ), check(, t) neighbor inittrace( ) { t:inittrace( ) check(, t) }

30 t:inittrace( ), check(, t) neighbor inittrace( ) { t:inittrace( ) check(, t) } search( ) = None

31 SpaceSearch Interface empty = singleton(a) = a union(, ) =

32 SpaceSearch Interface empty = singleton(a) = a union(, ) = a a b f(a) bind(s,f) = Ux:S f(x) = bind(, ) = b f(b)

33 SpaceSearch Interface empty = search( ) = None singleton(a) = a search( a ) = Some a union(, ) = a a b f(a) bind(s,f) = Ux:S f(x) = bind(, ) = b f(b)

34 Bagpipe Specification t:trace( ), check(, t) fin Reduce t:inittrace( ), check(, t) Spec Holds neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration Spec Violation

35 Bagpipe Specification neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration t:trace( ), check(, t) fin Reduce t:inittrace( ), check(, t) Reduce search( bind(inittracespace( ), (λ t. if check(, t) then empty else singleton(t))) = None Spec Holds Spec Violation

36 Bagpipe Specification neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration t:trace( ), check(, t) fin Reduce t:inittrace( ), check(, t) Reduce search( bind(inittracespace( ), (λ t. if check(, t) then empty else singleton(t))) = None Extract? SMT Spec Holds Spec Violation

37 Meet

38 Meet x y. (x /\ y) ( x \/ y) De Morgan's Law

39 Meet x y. (x /\ y) ( x \/ y) De Morgan's Law (let ((x (symbolic-bool)) (y (symbolic-bool)))

40 Meet x y. (x /\ y) ( x \/ y) De Morgan's Law (let ((x (symbolic-bool)) (y (symbolic-bool))) (eq? (and x y) (not (or (not x) (not y))))

41 Meet x y. (x /\ y) ( x \/ y) De Morgan's Law (solve (let ((x (symbolic-bool)) (y (symbolic-bool))) (if (eq? (and x y) (not (or (not x) (not y)))) (assert false) counter-example))

42 Meet x y. (x /\ y) ( x \/ y) De Morgan's Law (solve (let ((x (symbolic-bool)) (y (symbolic-bool))) (if (eq? (and x y) (not (or (not x) (not y)))) (assert false) counter-example)) type-driven state merging (declare-const x Bool) (declare-const y Bool) (define-const a Bool (and x y)) (define-const b Bool (not (or (not x) (not y)))) (assert (not (and (=> a b) (=> b a)))) (check-sat)

43 SpaceSearch Extraction => union(s,t) => (lambda (v) (if (symbolic-bool) (s v) (t v))) empty => (lambda (v) (assert false)) search(s) => (solve s) single(a) => (lambda (v) a) bind(s,f) => (lambda (v) (f (s v) v))

44 Bagpipe Specification neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration t:trace( ), check(, t) fin Reduce t:inittrace( ), check(, t) Reduce search( bind(inittracespace( ), (λ t. if check(, t) then empty else singleton(t))) = None Spec Holds Spec Violation

45 Bagpipe Specification neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Configuration t:trace( ), check(, t) fin Reduce t:inittrace( ), check(, t) Reduce search( bind(inittracespace( ), (λ t. if check(, t) then empty else singleton(t))) = None Extract /SMT Spec Holds Spec Violation

46 Summary SpaceSearch Interface & Semantics Extraction

47 Summary SpaceSearch Interface & Semantics Extraction More in the paper: Infinite Search Spaces Other Backends Parallelization Incrementalization

48 Evaluation SpaceSearch Interface & Semantics Extraction Bagpipe 1. BGP Verification

49 Evaluation SpaceSearch Interface & Semantics Extraction Bagpipe 1. BGP Verification 2. SQL Rewrite

50 Evaluation SpaceSearch Interface & Semantics Extraction Bagpipe 1. BGP Verification 2. SQL Rewrite 3. x86 Semantics

51 1. BGP Verification Specification Correct Bagpipe neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Counter Example Configuration

52 1. BGP Verification 10 Juniper Scenarios No Martian Block To External Gao & Rexford Bagpipe 14 Specs Hold neighbor remote-as 2 neighbor remote-as 3 neighbor remote-as 4! prefix lists ip prefix-list outc permit /0 le 32 ip prefix-list outpeer permit /24 le p prefix-list ou permit Internet2 >100K BelWü >200K Selfnet >50 19 Spec Violations, No False Positives

53 2. SQL Rewrite Verification SELECT DISTINCT x.name FROM Employee AS x, Employee AS y WHERE x.name = y.name = SELECT DISTINCT name FROM Employee

54 2. SQL Rewrite Verification SELECT DISTINCT x.name FROM Employee AS x, Employee AS y WHERE x.name = y.name = SELECT DISTINCT name FROM Employee ConjunctiveQuery: SELECT DISTINCT a,b FROM A,B WHERE a = b, b = c

55 2. SQL Rewrite Verification = Query A Reduce search( counterexamples(, )) Equal Extract Counter Example Query B SMT

56 3. x86 Semantics Validation ADD x, y Instruction search( bind(int32, (λ x. bind(int32, (λ y. if stoke(add x, y) = rocksalt(add x, y) then empty else singleton(x,y))))) Extract Equal Counter Example SMT

57 3. x86 Semantics Validation ADD x, y Instruction search( bind(int32, (λ x. bind(int32, (λ y. if stoke(add x, y) = rocksalt(add x, y) then empty else singleton(x,y))))) Extract SMT Equal Counter Example Bugs found: 7 Rocksalt Bugs 1 Stoke Bug

58 Related Work Solver Aided Languages: Rosette Torlak et al. PLDI 14 Smten Uhler et al. CAV 13 Solver Aided Tool Verification: XCert Tatlock et al. PLDI 10 Verified SAT Solvers & SAT Tactics: Marić TCS 10 Oe et al. VMCAI 12

59 Thank You github.com/konne88/spacesearch Bagpipe 1. BGP Verification SpaceSearch Interface & Semantics Extraction 2. SQL Rewrite 3. x86 Semantics Konstantin Weitz Steven S. Lyubomirsky Stefan Heule Emina Torlak Michael D. Ernst Zachary Tatlock

SpaceSearch: A Library for Building and Verifying Solver-Aided Tools

SpaceSearch: A Library for Building and Verifying Solver-Aided Tools Konstantin Weitz University of Washington, USA weitzkon@cs.washington.edu Emina Torlak University of Washington, USA emina@cs.washington.edu