Verification and Parallelism in Intro CS. Dan Licata Wesleyan University

Transcription

1 Verification and Parallelism in Intro CS Dan Licata Wesleyan University

2 Starting in 2011, Carnegie Mellon revised its intro CS curriculum Computational thinking [Wing] Specification and verification Parallelism

3 CMU Intro Courses Fundamentals of Computing Imperative Computation Functional Computation Computer Systems Data Structures and Algorithms Software Construction

4 CMU Intro Courses Fundamentals of Computing Imperative Computation Functional Computation Computer Systems Data Structures and Algorithms Software Construction

5 Course Design Imperative computation (Fa 11): Frank Pfenning, Tom Cortina, William Lovas Functional computation (Sp 11): me, Bob Harper Data structures and algorithms (Fa 12): Guy Blelloch, Margaret Reid-Miller, Kanat Tangwongsan taught and refined by many people since!

6 Status First group of students just graduated Courses are generally well-liked Anecdotally, students seemed stronger than before in some upper-level classes Preliminary studies by Carol Frieze indicate they preserve the women-cs fit at CMU

7 My role Designed Functional Computation Taught Spring 11, Fall 11, Spring 12 Now at Wesleyan Taught Imperative Computation this spring; teaching Imperative next fall and Functional next spring

8 CMU Wesleyan Premier tech school Students admitted to CS ~600 students per year in intro ~150 CS majors per year Liberal arts school, strong in sciences Not ~100 students per year in intro ~30 CS majors per year

9 Imperative has a basic programming prereq CMU Imperative has a math co-req Functional has a math prereq Wesleyan Many students have never programmed before No math pre/co-reqs, though many students have it

10 Hypothesis CMU courses can work elsewhere, with some adaptation to context Wesleyan this spring: 2/3rds of CMU Imperative less programming background shorter semester fewer student hours per week

11 Course Design Imperative Computation Functional Computation Data Structures and Algorithms

12 Course Design parallelism Imperative Computation Functional Computation Data Structures and Algorithms

13 Course Design parallelism Imperative Computation Functional Computation Data Structures and Algorithms verification

14 Trade-offs Breadth vs depth Outward vs inward Motivation vs skills Short- vs long-term Systems vs theory Jump in vs training wheels

15 Trade-offs Breadth vs depth Outward vs inward Motivation vs skills Short- vs long-term Systems vs theory Jump in vs training wheels

16 Objectives computational thinking imperative and functional programming algorithm design and analysis specification and verification

17 Computational Thinking what vs how correctness and safety efficiency parallelism randomness self-reference modularity functions as data ephemerality vs persistence

18 Objectives computational thinking imperative and functional programming algorithm design and analysis specification and verification

19 Imperative C0: teaching language based on C, designed by Frank Pfenning and Rob Arnold functions, variables, loops, arrays, pointers&structs rudimentary interfaces type safe, bounds checked, garbage collected

20 Imperative Then transition to C memory management void* and casting function pointers

21 Imperative

22 Arrays

23 Search

24 Pointers and structs

25 Data structures

26 Functional Standard ML numbers, pairs, lists, trees, datatypes functions as arguments and results signatures, structures, functors exceptions, mutation, IO

27 Functional Why not Ocaml: no parallel implementation (Manticore for SML) Haskell: laziness complicates cost analysis F#: want the ML module system

28 Functional

29 Functional

30 Objectives computational thinking imperative and functional programming algorithm design and analysis specification and verification

31 Imperative ephemeral data structures arrays searching in-place sorting linked lists stacks queues unbounded arrays priority queues hash tables DFS/BFS balanced BSTs tries spanning trees union find

32 Imperative ephemeral data structures arrays searching in-place sorting unbounded arrays priority queues hash tables DFS/BFS linked lists stacks queues

33 Analysis big-o worst case amortized expected case

34 Functional persistent data structures lists trees sorting regular expression matching n-body simulation balanced BSTs game tree search

35 Analysis recurrence relations closed forms log

36 Data structures and algorithms divide and conquer sequences sets and tables randomization dynamic programming BFS/DFS shortest paths treaps leftist heaps k-grams

37 Functional DS&A lists trees sorting n-body simulation game tree search divide and conquer sequences sets and tables randomization BFS/DFS shortest paths treaps dynamic programming leftist heaps

38 Functional DS&A lists trees sorting n-body simulation game tree search divide and conquer sequences sets and tables randomization BFS/DFS shortest paths treaps dynamic programming leftist heaps always have parallelism in mind!

39 Imperative mutable data structures arrays searching in-place sorting linked lists stacks queues unbounded arrays priority queues hash tables DFS/BFS tries spanning trees union find Functional/ DS&A lists trees sorting n-body simulation game tree search divide and conquer sequences sets and tables randomization BFS/DFS shortest paths treaps dynamic programming leftist heaps

40 Imperative mutable data structures arrays searching in-place sorting linked lists stacks queues unbounded arrays priority queues hash tables DFS/BFS tries spanning trees union find Functional/ DS&A lists trees sorting n-body simulation game tree search divide and conquer sequences sets and tables randomization BFS/DFS shortest paths treaps dynamic programming leftist heaps always have verification in mind!

41 Objectives computational thinking imperative and functional programming parallel algorithm design and analysis specification and verification

42 Specification and Verification

43 Specification and Verification Imperative: pre/post-conditions and loop invariants, expressed using boolean-valued functions Functional: mathematical statements about calculational behavior of programs

44 Slogans proof-oriented programming deliberate programming proof-directed debugging

45 Imperative

46 Specifications safety behavioral data structure invariants

47 Safety

48 Behavior

49 Behavior

50 Proofs Loop invariants hold initially Loop invariants preserved by one iteration Loop invariants imply the postcondition

51

52 Preservation of LI A[s] 0 lower i upper A[lower,i) sorted A[lower,i) <= A[i,upper) A[s] <= A[i+1,upper) A[lower,i+1) sorted A[lower,i+1) <= A[i+1,upper)

53 Binary Search

54 Binary Search Assignment: binary search for first occurrence

55 Data structure invs

56 Data structure invs

57 Representation invs

58 Preservation

59 Data structure invs

60 Heaps

61 Data structure invs

62 Contracts

63 Functional

64 Specifications Beyond requires/ensures Specs relating multiple functions Propositions, not booleans

65 Computing by calculation

66 Contextual equivalence

67

68 Props, not bools

69 Props, not bools

70 Props, not bools want to reason logically about existential, not about try-all-splits implementation

71 Props, not bools inner inductive definition of L*

72 Props, not bools

73 Regexp

74 Regexp

75 Regexp termination bug

76 Proof-directed debugging

77 Modularity

78 Representation invs

79 Representation invs

80

81 Verification safety behavioral specifications representation invariants + modularity booleans and propositions contracts

82 Slogans proof-oriented programming deliberate programming proof-directed debugging

83 Parallelism

84 Parallelism recognize the dependencies deterministic parallelism language-based cost model asymptotic analysis

85 Parallelism!= Concurrency parallelism: multiple processors/cores. property of the machine. concurrency: interleaving of threads. property of the application.

86 Parallelism!= Concurrency sequential concurrent serial traditional algorithms traditional OS parallel deterministic parallelism general parallelism

87 Parallel Calculation

88 Parallel Calculation verification is the same as without parallelism

89 Work and span work is usual serial time complexity work <e1,e2> = work(e1) + work(e2) span is parallel time complexity span <e1,e2> = max(span(e1), span(e2))

90 Mergesort

91 Work [7,1,3,6,8,4,2,5]

92 Work [7,1,3,6,8,4,2,5] [7,1,3,6] [8,4,2,5]

93 Work [7,1,3,6,8,4,2,5] [7,1,3,6] [8,4,2,5] [7,1] [3,6] [8,4] [2,5]

94 Work [7,1,3,6,8,4,2,5] [7,1,3,6] [8,4,2,5] [7,1] [3,6] [8,4] [2,5] [7] [1] [3] [6] [8] [4] [2] [5]

95 Work [7,1,3,6,8,4,2,5] [7,1,3,6] [8,4,2,5] [7,1] [3,6] [8,4] [2,5] [7] [1] [3] [6] [8] [4] [2] [5] W(n) = n + 2 W(n/2) is O(n log n)

96 Span [7,1,3,6,8,4,2,5] [7,1,3,6] [8,4,2,5] [7,1] [3,6] [8,4] [2,5] [7] [1] [3] [6] [8] [4] [2] [5]

97 Span [7,1,3,6,8,4,2,5] [7,1,3,6] [8,4,2,5] [7,1] [3,6] [8,4] [2,5] [7] [1] [3] [6] [8] [4] [2] [5]

98 Span [7,1,3,6,8,4,2,5] [7,1,3,6] [8,4,2,5] [7,1] [3,6] [8,4] [2,5] [7] [1] [3] [6] [8] [4] [2] [5] S(n) = n + S(n/2) is O(n)

99 Mergesort on lists: O(n log n) work O(n) span on trees: O(n log n) work O( (log n) 3 ) span

100

101 Design principles keep work low (ideally work-efficient) then minimize span

102 Brent s principle: time to run on p processors is O(max(work/p,span))

103 Sequences

104 n-body simulation quadratic work logarithmic span

105 Barnes-Hut divide space recursively into quadrants, approximating contribution of bodies that are too far away by their center of mass O(n log(n)) work logarithmic span

106 Game tree search Minimax: at each node, work is O(children) span is O(log children) Alpha-beta pruning: work is better but span is linear Jamboree: trade work for span

107 Data structures and algorithms divide and conquer sequences sets and tables randomization dynamic programming BFS/DFS shortest paths graph contractability treaps leftist heaps

108 Implementation Nested parallelism can be realized in Nesl Manticore (SML) Parallel Haskell Cilk TPL (C#/F#) OpenMP

109 Parallelism recognize the dependencies deterministic parallelism language-based cost model asymptotic analysis

110 Objectives computational thinking imperative and functional programming parallel algorithm design and analysis specification and verification

111 Trade-offs Breadth vs depth Outward vs inward Motivation vs skills Short- vs long-term Systems vs theory Jump in vs training wheels

112 CMU Intro Courses Fundamentals of Computing Imperative Computation Functional Computation Computer Systems Data Structures and Algorithms Software Construction

113 Activities lecture lab/recitation homework lots of TA homework help time lots of TA grading time

114 Experiment CMU courses can work elsewhere, with some adaptation to context Wesleyan this spring: 2/3 of CMU Imperative Wesleyan next year: 3/4 of CMU Functional?