Lurch: A Word Processor that Can Grade Students Proofs

Transcription

1 Lurch: A Word Processor that Can Grade Students Proofs Nathan C. Carter, ncarter@bentley.edu Bentley University, Waltham, MA, USA joint work with Kenneth G. Monks, monks@scranton.edu University of Scranton, Scranton, PA, USA April 2014 N. Carter (Bentley Univ.) Lurch April / 40

2 What is Lurch? Introduction A free word processor that can check the steps of a proof Our main focus now is on introduction-to-proof courses (propositional/predicate logic, introductory set theory, number theory) Free and open-source for all major desktop platforms (Windows, Mac, Linux) N. Carter (Bentley Univ.) Lurch April / 40

3 Lurch Mission Statement Introduction Part 1 Lurch should be as indistinguishable from the ordinary activities of mathematics as possible, except for the additional services it provides. That is, it should only add to your math experience, not change your math experience. Part 2 Lurch should provide the software infrastructure the mathematical community needs for validating rigorous mathematics. That is, it should validate mathematical content created by you a spell-checker for mathematical rigor. N. Carter (Bentley Univ.) Lurch April / 40

4 Introduction Not a proof-checker in the usual sense of the phrase Differences from Mizar, Coq, et al.: Does not do any proofs for you All rules are customizable, even the rules of logic Focused on student learning Simple user interface N. Carter (Bentley Univ.) Lurch April / 40

5 Introduction Relevant results from education research Frequent, immediate feedback is good for learning Students work on assignments until they re judged correct More time on task is also good for learning Neater homework is easier to grade Professors report spending saved grading time with students N. Carter (Bentley Univ.) Lurch April / 40

6 User Interface Simple word processing Standard formatting tools L A TEX-style mathematical typesetting Inserting images Printing Saving Exporting to HTML N. Carter (Bentley Univ.) Lurch April / 40

7 User Interface Simple word processing Standard formatting tools L A TEX-style mathematical typesetting Inserting images Printing Saving Exporting to HTML N. Carter (Bentley Univ.) Lurch April / 40

8 But does it work? User Interface (only partially complete, with errors, as an example) N. Carter (Bentley Univ.) Lurch April / 40

9 User Interface Making bubbles Select text you want Lurch to understand Click the meaningful expression button Lurch will bubble the text N. Carter (Bentley Univ.) Lurch April / 40

10 Making bubbles User Interface The bubble reports the understood meaning It is shown iff the cursor is in it Three types of bubbles ME Property Context N. Carter (Bentley Univ.) Lurch April / 40

11 User Interface Bubble benefits No ambiguity No style restrictions No language restrictions Minimal overhead Pedagogical benefits N. Carter (Bentley Univ.) Lurch April / 40

12 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles Simple MEs include integer, float, string, variable, etc.: "hello" x Compound MEs include function applications and bindings: 150 x y t, t 0 s, t s = 1 Every step in any argument the user writes must be an ME. More complex structures (e.g., rule definitions) can also be expressed as MEs. Each ME is stored internally as an OpenMath tree. Example: N. Carter (Bentley Univ.) Lurch April / 40

13 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles Simple MEs include integer, float, string, variable, etc.: "hello" x Compound MEs include function applications and bindings: 150 x y t, t 0 s, t s = 1 Every step in any argument the user writes must be an ME. More complex structures (e.g., rule definitions) can also be expressed as MEs. Each ME is stored internally as an OpenMath tree. Example: N. Carter (Bentley Univ.) Lurch April / 40

14 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles Simple MEs include integer, float, string, variable, etc.: "hello" x Compound MEs include function applications and bindings: 150 x y t, t 0 s, t s = 1 Every step in any argument the user writes must be an ME. More complex structures (e.g., rule definitions) can also be expressed as MEs. Each ME is stored internally as an OpenMath tree. Example: N. Carter (Bentley Univ.) Lurch April / 40

15 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles Simple MEs include integer, float, string, variable, etc.: "hello" x Compound MEs include function applications and bindings: 150 x y t, t 0 s, t s = 1 Every step in any argument the user writes must be an ME. More complex structures (e.g., rule definitions) can also be expressed as MEs. Each ME is stored internally as an OpenMath tree. Example: N. Carter (Bentley Univ.) Lurch April / 40

16 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles Simple MEs include integer, float, string, variable, etc.: "hello" x Compound MEs include function applications and bindings: 150 x y t, t 0 s, t s = 1 Every step in any argument the user writes must be an ME. More complex structures (e.g., rule definitions) can also be expressed as MEs. Each ME is stored internally as an OpenMath tree. Example: N. Carter (Bentley Univ.) Lurch April / 40

17 Data storage What does OpenMath look like? ME bubble contents OpenMath XML encoding 77 <OMI>77</OMI> x x 9 <OMV name="x"/> <OMA> <OMS name="minus" cd="arith1"/> <OMV name="x"/> <OMI>9</OMI> </OMA> N. Carter (Bentley Univ.) Lurch April / 40

18 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles Simple MEs include integer, float, string, variable, etc.: "hello" x Compound MEs include function applications and bindings: 150 x y t, t 0 s, t s = 1 Every step in any argument the user writes must be an ME. More complex structures (e.g., rule definitions) can also be expressed as MEs. Each ME is stored internally as an OpenMath tree. Example: N. Carter (Bentley Univ.) Lurch April / 40

19 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles Simple MEs include integer, float, string, variable, etc.: "hello" x Compound MEs include function applications and bindings: 150 x y t, t 0 s, t s = 1 Every step in any argument the user writes must be an ME. More complex structures (e.g., rule definitions) can also be expressed as MEs. Each ME is stored internally as an OpenMath tree. Example: N. Carter (Bentley Univ.) Lurch April / 40

20 Data storage Where do rules come from? Answer #1 Anyone can add a rule to any document, if you know how to structure ME bubbles correctly to define it. Click a bubble s tag to change its role (e.g., if-then rule ). Nest bubbles to form a more complex structure (e.g., premise and conclusion MEs inside a rule ME). Label the rule so that it can be cited later. N. Carter (Bentley Univ.) Lurch April / 40

21 Data storage Where do rules come from? Answer #2 Some smart rules are built into the software and can be useful for skipping steps students have moved beyond. Simple arithmetic Example: = 39.5 Propositional tautologies (and set-theoretic versions) Examples: (x > 1 x < 1) (x < 1 x > 1) S ( {x f (x) = 10} C ) = (S {x f (x) = 10}) ( S C ) Future: CAS Examples: d dx sin x = cos x (t + 1) 3 = t 3 + 3t 2 + 3t + 1 N. Carter (Bentley Univ.) Lurch April / 40

22 Data storage Where do rules go? It is common to place a set of rules that go together into a single document, and use that document as a dependency. Lurch comes with many math topics built-in, each defined as a set of rules in a document, usable as a dependency. These files can be copied and changed, or new ones created from scratch. No programming is required. Thus entirely new mathematical systems can be built from the ground up in the word processor itself. Users can share them on the web, as mentioned earlier. N. Carter (Bentley Univ.) Lurch April / 40

23 Data storage An example library (dependency) N. Carter (Bentley Univ.) Lurch April / 40

24 Data storage An example library (dependency) N. Carter (Bentley Univ.) Lurch April / 40

25 Data storage An example library (dependency) N. Carter (Bentley Univ.) Lurch April / 40

26 Data storage An example library (dependency) N. Carter (Bentley Univ.) Lurch April / 40

27 Data storage An example library (dependency) N. Carter (Bentley Univ.) Lurch April / 40

28 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles Simple MEs include integer, float, string, variable, etc.: "hello" x Compound MEs include function applications and bindings: 150 x y t, t 0 s, t s = 1 Every step in any argument the user writes must be an ME. More complex structures (e.g., rule definitions) can also be expressed as MEs. Each ME is stored internally as an OpenMath tree. Example: N. Carter (Bentley Univ.) Lurch April / 40

29 Three types of bubbles Data storage 1 Meaningful Expressions (MEs), red bubbles 2 Properties, blue bubbles Used to decorate adjacent MEs with one of three types of attributes: labels, like L A TEX \label reasons, citing what justifies a step of work premises, in situations where this is necessary Do not change MEs semantics, only add to them (typically using OMATTR, but not always). Examples: N. Carter (Bentley Univ.) Lurch April / 40

30 Data storage Three types of bubbles 1 Meaningful Expressions (MEs), red bubbles 2 Properties, blue bubbles 3 Contexts, green bubbles Used to group together collections of MEs that belong together. Example: In Exercise 1, you might declare f (x) = π x. Then in Exercise 2, you declare f (x) = x In order to avoid the conclusion that π x = x 1000, we must keep these exercises in separate contexts. N. Carter (Bentley Univ.) Lurch April / 40

31 Data storage Three types of bubbles 1 Meaningful Expressions (MEs), red bubbles 2 Properties, blue bubbles 3 Contexts, green bubbles N. Carter (Bentley Univ.) Lurch April / 40

32 Data storage Three types of bubbles 1 Meaningful Expressions (MEs), red bubbles 2 Properties, blue bubbles 3 Contexts, green bubbles N. Carter (Bentley Univ.) Lurch April / 40

33 Data storage Three types of bubbles 1 Meaningful Expressions (MEs), red bubbles 2 Properties, blue bubbles 3 Contexts, green bubbles N. Carter (Bentley Univ.) Lurch April / 40

34 Demo Demo Small example proof using built-in rule libraries Theorem: If x {3} then x 9 = 27. Defining a custom rule, such as this one Bizarre rule: We say that x y is fuzzy if and only if x = y. Proving a theorem from custom rules Theorem: If x y is fuzzy then y x is fuzzy. N. Carter (Bentley Univ.) Lurch April / 40

35 Validation Validation specification Each type of ME has a different validation specification. These specifications together describe Lurch s validation algorithm. Example 1: Constant and variable declarations Valid iff the following conditions are met: 1 The identifier is not already declared, i.e., it is not in the scope of the declaration of the same identifier either as a variable or a constant. 2 The declaration does not have a reason assigned to it. N. Carter (Bentley Univ.) Lurch April / 40

36 Validation specification Validation Each type of ME has a different validation specification. These specifications together describe Lurch s validation algorithm. Example 1: Constant and variable declarations N. Carter (Bentley Univ.) Lurch April / 40

37 Validation Validation specification Each type of ME has a different validation specification. These specifications together describe Lurch s validation algorithm. Example 1: Constant and variable declarations Valid iff the following conditions are met: 1 The identifier is not already declared, i.e., it is not in the scope of the declaration of the same identifier either as a variable or a constant. 2 The declaration does not have a reason assigned to it. N. Carter (Bentley Univ.) Lurch April / 40

38 Validation Validation specification Each type of ME has a different validation specification. These specifications together describe Lurch s validation algorithm. Example 1: Constant and variable declarations Valid iff the following conditions are met: 1 The identifier is not already declared, i.e., it is not in the scope of the declaration of the same identifier either as a variable or a constant. 2 The declaration does not have a reason assigned to it. N. Carter (Bentley Univ.) Lurch April / 40

39 Validation Inspiration re: variables D.J. Velleman / Annals of Pure and Applied Logic 144 (2006) This paper introduces rules for variable declarations in natural deduction with the following properties. They obviate the need for complex checks one must perform before applying quantifier introduction and elimination rules. They match the way mathematicians often write in practice, and, in turn, what is expected of students in mathematics. N. Carter (Bentley Univ.) Lurch April / 40

40 Validation Inspiration re: variables D.J. Velleman / Annals of Pure and Applied Logic 144 (2006) We will say that a variable v is a declared variable at a line of a derivation if that line falls within the scope of a declaration of v. Variable declarations will be governed by the following natural rules: A variable may not occur free in a line of a derivation if it is not a declared variable at that line. A declaration of a variable may not occur in a line of a derivation if the line is in the scope of a previous declaration of the same variable. N. Carter (Bentley Univ.) Lurch April / 40

41 Inspiration re: variables Validation Universal Generalization (UG): If you begin a subordinate derivation with the line Declare v, and then you deduce ϕ(v) in this subordinate derivation, then you may end the subordinate derivation and infer xϕ(x). Universal Instantiation (UI): If t is a term and all variables occurring in t are declared variables, then from xϕ(x) you may infer ϕ(t). Existential Generalization (EG): If t is a term and all variables occurring in t are declared variables, then from ϕ(t) you may infer xϕ(x). Existential Instantiation (EI): You may use the line xϕ(x) to justify the line Declare v: ϕ(v). This line both declares the variable v and also asserts the statement ϕ(v). N. Carter (Bentley Univ.) Lurch April / 40

42 Inspiration re: variables Validation Same four rules in Lurch: N. Carter (Bentley Univ.) Lurch April / 40

43 Validation Validation specification Example 2: MEs with reasons attached Valid iff the following conditions are met: 1 The user must not have attempted to use more than one reason to justify the ME. 2 The cited reason must refer to a rule usable at the location cited. 3 The cited rule itself must have passed validation by Lurch. 4 The user must not have cited more premises than required, and all premises the user did cite must be MEs usable at the location cited. 5 If too few premises were cited, Lurch must be able to automatically locate the others immediately preceding the ME to be justified. 6 The forms of the premises and conclusion must match those of the rule s premises and conclusion and about ten other conditions not listed here... N. Carter (Bentley Univ.) Lurch April / 40

44 How is Lurch doing? Does Lurch help students? Classroom testing Bentley University in 2008 Preliminary version, much older than current release University of Scranton in Spring 2013 Introduction to proof course Walked students along a spectrum from formal proofs to informal ones Bentley University in Fall 2013 Introduction to formal logic course Propositional logic, predicate logic, basic proofs in analysis N. Carter (Bentley Univ.) Lurch April / 40

45 How is Lurch doing? Starting the journey: Formal proofs N. Carter (Bentley Univ.) Lurch April / 40

46 How is Lurch doing? On the way: Semiformal proofs N. Carter (Bentley Univ.) Lurch April / 40

47 How is Lurch doing? At the destination: Informal proofs (only partially complete, with errors, as an example) N. Carter (Bentley Univ.) Lurch April / 40

48 How is Lurch doing? Investigating success (informally) On frequent and immediate feedback... I liked that we could check our work as we are doing it it makes the learning more immediate by providing constant feedback even outside of class. It allowed me to guarantee when I got something right, which was helpful early on when I was unsure about what a proof was. On learning the mathematical system the instructor put into Lurch... I liked using Lurch because I was able to see what was needed for rules to work. Lurch became easier as I learned the rules and definitions, so using it in turn helped me learn those. On time spent on homework when using Lurch... I worked until I was correct. Anecdotal evidence from first assignment, Spring 2013 N. Carter (Bentley Univ.) Lurch April / 40

49 How is Lurch doing? Investigating success (informally) Do students want to use Lurch? Lurch use in both testing courses was optional. Students who chose not to use Lurch fell into two categories. Confident in the course material and didn t feel a need for help. (One such student was incorrectly confident.) Annoyed at being told they were wrong. (These usually converted when they saw the benefits.) Late in the 2013 testing course, when students were asked to use L A TEX for proofs, some voluntarily typed them in Lurch first to receive the benefits of validation. N. Carter (Bentley Univ.) Lurch April / 40

50 Plans for the future What s next? Improvements needed to existing app Efficiency: Long documents bog the software down Type system: Bubble tags that read, say, polynomial x R,... Visualizations of document and proof structure: E.g., arrows among bubbles to show premise and property relations Even less obtrusive bubbling interface Find and bubble the largest contiguous parseable text around the cursor Same for reason names and premise labels Automatic bubbling of entire lines of formal proofs N. Carter (Bentley Univ.) Lurch April / 40

51 What s next? Bigger plans for the future Move to a web app? We ve begun experiments to determine if this is a worthwhile endeavor. Pros End user does not need to install or update anything Usable on all platforms easily, including tablets Less of a barrier to potential adopters trying it out Less to maintain (no installer, no document layout code, etc.) Easier access to new technologies (MathJax, MathQuill, etc.) Cons Not yet clear that this is entirely possible Would require rebuilding many existing features It s harder to build a beautiful UI on the web N. Carter (Bentley Univ.) Lurch April / 40

52 What s next? Getting involved Testing in courses not taught by the developers Writing documents defining new mathematical systems Software development (C++, Qt, JavaScript, CoffeeScript) Design ideas, proofs of correctness Join us! N. Carter (Bentley Univ.) Lurch April / 40