User s Guide: JFLAP Finite Automata Feedback/Grading Tool

Transcription

1 User s Guide: JFLAP Finite Automata Feedback/Grading Tool Daphne A. Norton dan0337@cs.rit.edu February 14, Preface This tool is intended for use with JFLAP, the Java Formal Language and Automata Package (see It provides feedback to the user regarding their solutions to problems involving Deterministic Finite Automata (DFAs) and Nondeterministic Finite Automata (NFAs). The program can simply state that an answer is correct or incorrect; this option may be useful for instructors grading homework assignments or take-home examinations. Alternatively, when equivalence is checked to determine if the student s model accepts the same language as the instructor s answer key, but the student has an incorrect solution, the feedback may include a specific witness string which is accepted by exactly one of these two automata. This feature can even be useful outside the classroom, when looking for a string demonstrating inequivalence. There are a few ways to run the program: 1. Within JFLAP, an added menu option can used to check two finite automata for equivalence. Unlike the existing menu option, if the automata are NOT equivalent, this program provides extra feedback. It displays a short witness string which is accepted by one automaton, but not by the other. 2. This program can also be run standalone, without the JFLAP GUI. This allows the user the option to see verbose output, if they want to follow exactly what the algorithm is doing as it compares the automata. The witness can be turned on or off, as desired. Two different equivalence-checking algorithms are currently available. One is the Hopcroft O(n lg n) partitioning algorithm, which actually runs more slowly if a witness is created. The other is a faster, nearly linear merging algorithm by Hopcroft and Karp; it remains nearly linear even if a witness is produced. For small automata of the size typically created in JFLAP, users will not notice a significant performance difference. However, the merging algorithm has an added advantage, in that it always produces (one of) the shortest possible witness string(s). 3. It can be used as a grading tool. For instance, it can be run via try, the homework submission system which is used frequently at RIT. Instructors 1

2 who are familiar with try will find that it is easy to set it up to accept and grade JFLAP assignments. 2 Instructions 2.1 Getting Started You will need to ensure that the Java JRE version 1.5 or higher has been installed on your computer. Java is available for download on the Sun site at java.sun.com. To download JFLAP, browse to jflap.org and click on Get JFLAP. Since the grading code was written to work with JFLAP 6.4, it is recommended that you download version 6.4. This file is named JFLAP.jar. For the customized feedback/grading code: Obtain JFLAP_grade_dn.jar from If this site is no longer available, the code should be available from the RIT Computer Science Graduate Projects/Theses site, or contact Edith Hemaspaandra ( If you wish to use the new menu option within JFLAP, the easiest approach is to update the JFLAP jar with the class files contained in JFLAP_grade_dn.jar. To do this in Unix, put both jars in the same directory, extract the contents of JFLAP_grade_dn.jar (jar -xvf JFLAP_grade_dn.jar), and update JFLAP.jar: jar -uvf JFLAP.jar equivalence/*.class gui/action/fawitnessaction.class gui/menu/menubarcreator*.class To start the JFLAP program in a Windows environment, simply double-click on the jar file. To start it in Unix, use: java -jar JFLAP.jar For more information, see the directions and Useful References below. 2.2 The Get Distinguishing String Menu Option JFLAP has a lot of nice features, including the ability to determine if two finite automata are equivalent. However, what if you have two NFAs (and/or DFAs), and you would like a simple example string to show exactly what makes them different? The code in this package provides a convenient Get Distinguishing String menu option. To access it, make sure you have done the setup in Getting Started above. Then: 1. Start up JFLAP. 2

3 2. Click on Finite Automaton. 3. Draw two finite automata (as per usual in JFLAP), or choose the menu option File > Open... and browse to the previously saved.jff files you want to compare. You will need to have at least two automata open. 4. In one of the windows, choose the menu option Test > Get Distinguishing String. See Figure A pop-up will allow you to select the second automaton from a drop-down. Only the files you have open will be shown. Click OK when ready. 6. A sample of the final output is shown in Figure 2. This is a string (witness) which one automaton accepts, and the other does not. If you compare two equivalent automata, there will not be a witness for them, as they both accept the same language (the same set of strings), so the program will state that they are equivalent. Note that if you run the program multiple times on the same automata, the possibility exists that you will not get the same witness every time. This is because the order of iteration over HashSets in Java can vary over time, and the witness which is produced depends upon the order in which the states and transitions are processed. However, note that the witness will be as short as it possibly can be, since the program does a breadth-first search (a state-merging algorithm) to find it. See the author s project paper for a full discussion. 2.3 Verbose Mode Outside the JFLAP GUI Normally, the Unix command to run the jar for grading purposes is: java -jar JFLAP_grade_dn.jar studentfile.jff answerkey.jff true where the two.jff files are the student s answer and the instructor s answer key. The order of these two files does not matter. The last parameter, true, can be changed to false if the instructor does not want to display a witness if the answer is wrong. No additional detailed output is provided. A fourth parameter of false may be added to this command if you wish to use the Hopcroft partitioning algorithm behind the scenes instead of the default merging algorithm. However, to study the equivalence-checking algorithms in more depth, see the author s paper and the Java source code which is included in the.jar file. If you are interested in tracing exactly what the near-linear merging algorithm or the n lg n Hopcroft partitioning algorithm is doing as it compares two automata, you can save any two finite automata as.jff files and then run the tool in verbose mode. For example, if you run the partitioning algorithm, then the 3

4 Figure 1: The menu option to get a sample string that distinguishes two automata. Figure 2: The witness string demonstrating why two automata are different. 4

5 inverse transition function, processing lists, partition blocks, and so forth will be printed to standard out. It will also show the details of how a witness string is produced, if the witness option is turned on. Moreover, the witness string from the partitioning algorithm will be a relatively short string, but it is not guaranteed to be the absolute shortest possible string. If you run the program multiple times and look at the output, you may see varying results. To set this up, the main method from EquivalenceNlgNWitness.java needs to be executed. The executable jar file JFLAP_grade_dn.jar is set up to use the Grader.main() method by default. Therefore, if you want an executable jar, one way to set this up is to change the main class in the manifest: 1. Extract the contents of the jar (jar -xvf JFLAP_grade_dn.jar). 2. Create a Manifest.txt file formatted as follows: (a) For the merging equivalence-checking algorithm: Main-Class: equivalence.equivalencemergewitness Class-Path: JFLAP.jar (b) Alternatively, for the partitioning algorithm: Main-Class: equivalence.equivalencenlgnwitness Class-Path: JFLAP.jar 3. Build a new jar file, using a command such as this one to create a file named JFLAP_equiv_dn.jar. jar -cfm JFLAP_equiv_dn.jar Manifest.txt *.java equivalence/*.class gui/action/fawitnessaction.class gui/menu/menubarcreator*.class 4. Then if you have two automata in files nfa1.jff and nfa2.jff, the program can be run with a command like java -jar JFLAP_equiv_dn.jar nfa1.jff nfa2.jff true true where the first boolean turns on the witness, and the second turns on verbose mode. Either one (or both) can be set to false, if preferred. 5

6 Another alternative is to execute the class files directly, outside of the jar. Simply extract the JFLAP 6.4 jar (jar -xvf JFLAP.jar), then extract the grading tool jar on top of it (jar -xvf JFLAP_grade_dn.jar). (This will overwrite a couple of the class files from the original JFLAP jar file; see the description of the new and modified classes in the appendix of this User s Guide.) Now it is feasible to get verbose output via syntax such as: java equivalence.equivalencemergewitness nfa1.jff nfa2.jff true true... or java equivalence.equivalencenlgnwitness nfa1.jff nfa2.jff true true... or to grade files via: java equivalence.grader studentfile.jff answerkey.jff true 2.4 Information for the Instructor To make it easier to grade assignments, it is recommended that the instructor specify the exact JFLAP filename to be submitted for each problem. For example, if there are five questions, the student s files might be named using a convention like problem1.jff, problem2.jff,... problem5.jff. This allows grading to be automated. Note that a convention should also be established for the assignment so that students know whether to draw transitions to a dead (trap) state, or to completely leave these transitions out of the diagram. JFLAP itself allows missing transitions in its definition of DFAs. However, the alphabet used in the answer key.jff file must match the alphabet used in the student s submitted file. In some cases, a missing trap state may result in a missing alphabet symbol within the automaton. If the alphabets do not match, then the two automata cannot be compared automatically, and a message will indicate that equivalence cannot be tested due to differing alphabets. In this situation, the program will assume the automata are not equivalent. 2.5 Using this Program with try at RIT This section provides instructions on how to configure try to use the JFLAP automata grading program. 6

7 2.5.1 Submitting Assignments to be Graded Whenever an assignment is due, the instructor will need to set up a try project to accept submissions. Based on the setup, they should then tell the class exactly how to submit their assignment to try from the Unix command line. The account name, project name, and list of files must be provided. For example, suppose the instructor s account is called grading-acct. The assignment (project code) might be called jflap-submit. Perhaps it was a short assignment, and there were only two problems. If the names of the two corresponding JFLAP files were problem1.jff and problem2.jff, then the command to submit the assignment from the Unix command line would be: try grading-acct jflap-submit problem1.jff problem2.jff The output might be something like: Copying files...done Grading JFLAP assignment... Checking problem 1 The answer is correct! Checking problem 2 Incorrect: the string 00 is an example. You have successfully completed project jflap-submit! When an answer is wrong, the instructor may choose to display an example (here, 00), or to simply state that the answer is incorrect. The student may be able to fix the problem and then resubmit all of the files. Students should make sure that their filenames match the names that try is expecting, or the grading program will not be able to identify the files correctly. A Student s Guide to try provides additional troubleshooting tips. (See the Useful References below.) Setting Up try Projects in the Instructor s Unix Grading Account The following steps assume that the user is an instructor with a Unix account on a system where try has already been installed. (At RIT, professors typically have a special grading account so that try is isolated from their personal account. Refer to Reek s documentation for a discussion of the potential security issues.) First, an entry must be placed in a.tryrc file in the grading home directory to configure the try project. For example: 7

8 # This is a sample.tryrc file for a project # called jflap-submit, to grade JFLAP DFAs/NFAs. # The try directory is trydir, # and the project directory is projdir. Files from the # most recent submission are saved. jflap-submit trydir projdir save Note that the try directory should be created in the home directory, and then the project directory must exist in the try directory, so the path is: /home/account/trydir/projdir/. To enforce security, verify that the permissions on all files and directories follow the recommendations in the try documentation. Next, for try to work, a build or build_safe script is required to exist in the project directory. However, no compilation is necessary to grade the JFLAP.jff files. Therefore, simply create an empty build file (such as via the command touch build). Finally, the run script in the project directory will perform the actual grading. The JFLAP jar and grading jar will need to be available to the account and specified on the classpath. The instructor must also create the.jff answers to the assignment, and save them in an accessible location as well, such as in the project directory. Here is an example of a simple working run script: # This is a sample run file to grade JFLAP DFAs/NFAs # using the try system. # Based in part on samples in Kenneth A Reek s paper, # "The Try System." # The answer key consists of the instructor s files, # named solution1.jff, # solution2.jff, and solution3.jff, which are # JFLAP XML files representing # the desired automata. The student must submit corresponding # files named problem1.jff, problem2.jff, and problem3.jff. echo Grading JFLAP assignment... # Student files are placed in their working directory within # the try directory. This working directory is the current # directory while this script is running. # Instructor files should be located elsewhere, such as the # project directory. Put the grading jar and the standard # JFLAP 6.4 jar on the classpath. CLASSPATH=${CLASSPATH}:/home/stu13/s8/dan0337/tryDir/ projdir/jflap_grade_dn.jar: 8

9 /home/stu13/s8/dan0337/trydir/projdir/jflap.jar # For statistics on the answers correct=0 incorrect=0 submitted=0 missing=0 for T in do echo Checking problem $T # Comment out this call, which does not use Grader.java or # jar file. (The Manifest in the grading jar dictates main # class of Grader, but this call is a way to run the # algorithm with verbose output instead of grading.) # java../projdir/equivalence.equivalencenlgnwitness../projdir/solution$t.jff problem$t.jff true true # This gives the student a witness string as feedback # if the answer is wrong. # Change the "true" to "false" to omit the witness. java -jar../projdir/jflap_grade_dn.jar../projdir/solution$t.jff problem$t.jff true > out.$t # The out file will contain the student s result, unless # the.jff file(s) couldn t be read. # Show the student the result and track the grade. if [ -s out.$t ] then submitted= expr $submitted + 1 cat out.$t resultstatus= grep -c "The answer is correct" out.$t if [ $resultstatus -eq 1 ] then correct= expr $correct + 1 else incorrect= expr $incorrect + 1 fi else # Zero size on the out file means the program didn t # run successfully. echo Could not process or read file $T - did you use the right filename and submit it? missing= expr $missing + 1 9

10 fi done # try_log can only be executed once, so summarize the # results on one line. try_log JFLAP FAs correct $correct, incorrect $incorrect, submitted $submitted, missing $missing exit 0 This run script simply tracks the number of correct, incorrect, submitted, and missing files. The instructor could modify the script to assign a specific number of points to each problem, change file names, and so forth. In this example, the files from the latest submission will be stored in the working directory for the student (since save is indicated in.tryrc), and the log entry for this student might say: student-acct jflap-submit 08/10/12 16:59:16 Completed: JFLAP FAs correct 1, incorrect 1, submitted 2, missing 1 In this case, the student submitted one correct answer and one incorrect answer. Moreover, apparently the instructor was expecting a third file which the student was missing. With the current configuration in this example, the student can execute the try command again until everything is correct. 3 Useful References As a starting point if you are looking for more information: 1. Helpful reference information on JFLAP is available at the org site. This includes tutorials, software updates, and even a wiki. 2. The JFLAP manual may also be useful: Susan H. Rodger and Thomas W. Finley, JFLAP: An Interactive Formal Languages and Automata Package, Jones & Bartlett Publishers, Sudbury, MA, The try software, complete with manual and README doc, can be obtained on Kenneth Reek s Web page at: software.html. 4. His presentation paper and slides regarding try can be found at: http: // 10

11 5. A Student s Guide to try is located at Misc/try.html or It was originally written by Margaret M. Reek, and later updated by Trudy Howles. (Older versions may also exist on the RIT Web; the latest is version 2.8 from 2004.) 6. For additional background on the new grading tool, please refer to the author s M.S. project paper, Algorithms for Testing Equivalence of Finite Automata, with a Grading Tool for JFLAP. See the site: cs.rit.edu/~dan0337/. The appendix below may also be of interest to programmers. 4 Appendix: About the Program For those who are curious, here is a description of the code behind the grading tool. This program was written in Java, so it is cross-platform. As the original JFLAP 6.2 source code documentation indicates, Java versions 1.4 and 1.5 are required to compile and run the main JFLAP program. JFLAP 6.4 requires 1.5, and the additional features for the grading tool have been coded in Java version 1.5. This version allows for type checking on generics at compile time, prior to running the program. In an IDE such as Eclipse, which automatically builds the program as you write, this means errors can be found while you code. This kind of type-checking is an invaluable time-saver to the programmer, particularly when implementing and debugging complex algorithms (such as equivalence checkers) that involve numerous containers like vectors and HashSets, which can hold any type of Object whatsoever. The classes used for grading are all in the equivalence package. These include: EquivalenceWitness: An abstract class used for testing two JFLAP finite automata for equivalence. Contains useful methods for reading.jff files, setting options, getting output, etc. Superclass of the two equivalencechecking algorithms described next. EquivalenceNlgNWitness: This class uses the n lg n Hopcroft algorithm to compare two finite automata to determine if they recognize the same language. The deciding factor is whether or not the start states of the two automata can be distinguished from each other via partitioning. This particular implementation of the algorithm halts immediately if it finds at any point that the start states are distinguishable (meaning that the automata are not equivalent and do not recognize the same language). Moreover, the user can supply a boolean (true/false) to request extra feedback (or not). Specifically, if the automata are not equivalent and this boolean is set to true, the program provides the user with a witness string which is accepted by one automaton, but not by the other. This setting is good for 11

12 providing feedback to a student, but it requires more overhead (more time and space) than the basic algorithm. Refer to the project paper for details. To run the program from main(), the user must provide the names of the two files (automata) to compare, a boolean specifying whether or not to create a witness, and a boolean to turn verbose mode on or off (to dictate how much information is printed to standard out). Note that NFAs and automata with missing transitions are acceptable as input, as the class will convert them to complete DFAs prior to executing the equivalencechecking code. Subset construction will, of course, slow down the program, but to the average JFLAP user it will not have a noticeable impact. EquivalenceMergeWitness: Similar in purpose to the previous class; however, it uses a faster, near-linear merging algorithm by Hopcroft and Karp. This approach acts under the assumption that the start states are equivalent. Using the start states as the origin, the program systematically follows transitions, merging the states it encounters for the same input string, as they must be equivalent for the automata to be equivalent. If at any point an accepting state and a non-accepting state are merged, then the assumption that the start states are equivalent must be false, so the automata cannot be equivalent. The witness generated by this class is guaranteed to be one of the shortest possible, as a breadth-first search strategy is used. Note that even if the witness option is turned on, the run time for this particular algorithm still remains nearly linear. For more analysis, see the project paper. Grader: Essentially a wrapper around EquivalenceWitness to facilitate grading. It simply states whether a student s answer is correct or not, based on the instructor s.jff file. If the answer is incorrect, it can be set to provide a witness, if so desired, in order that the student can receive feedback and learn from their mistake. By default, EquivalenceMergeWitness is implemented, although the user can pass a parameter to the program to use EquivalenceNlgNWitness instead. StateAlphaPair: This class simply tracks a state paired with an alphabet symbol. Used in the other classes (for the transition function and its inverse). StatePair: For easy lookup from a Collection, this class contains two states. One state should come from each of the two automata being compared, in a consistent order. StateStateSymbol: A subclass of StatePair. Tracks an additional alphabet symbol which is associated with the states. StateDisjointSets: A class that stores states in sets which can be merged efficiently. The set containing a particular state can also be found quickly. Used in EquivalenceMergeWitness to allow near-linear time overall performance. (Implements the tree structure with heuristics as described by 12

13 Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, 2nd ed., p. 508.) In addition, the algorithms needed to be available to be called by the JFLAP GUI. Therefore, a couple of classes were created and/or modified for the existing JFLAP program: gui.action.fawitnessaction: This class instantiates and runs the EquivalenceMergeWitness class described above. It will display a pop-up with the witness string if two FAs (finite automata) are not equivalent. Otherwise, it states that they are equivalent. gui.menu.menubarcreator: This is existing JFLAP code which displays the menu bar. A new menu option, Get Distinguishing String, was added to the menu in the gettestmenu() method. This calls the FAWitnessAction. Via the normal JFLAP GUI, users are able to select any two finite automata to compare. Additional files used for building the project: buildjar: A script used to create the jar easily from the Unix command line. Both the source files and compiled class files are included in the jar. The script consists of the single (one-line) command: jar -cfm JFLAP_grade_dn.jar Manifest.txt *.java equivalence/*.class gui/action/fawitnessaction.class gui/menu/menubarcreator*.class Manifest.txt: A text file which is used to create a manifest file for the jar. It specifies the main class to use when the jar is executed (the Grader class, although this can be changed to run one of the algorithms directly instead, as described earlier in this guide). It also puts the JFLAP.jar into the classpath for this jar. Here is what it looks like: 4.1 Copyright Main-Class: equivalence.grader Class-Path: JFLAP.jar Programmers are welcome to modify and reuse the code written by Daphne Norton for this project, as long at it is not sold for commercial purposes. Please be sure to acknowledge the author. The original JFLAP code is copyrighted as follows: 13

14 Susan H. Rodger, Thomas Finley Computer Science Department Duke University April 24, 2003 Supported by National Science Foundation DUE Copyright (c) 2003 All rights reserved. Redistribution and use in source and binary forms are permitted provided that the above copyright notice and this paragraph are duplicated in all such forms and that any documentation, advertising materials, and other materials related to such distribution and use acknowledge that the software was developed by the author. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED AS IS AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. 14