Automatic assessment of students program codes

Transcription

1 Automatic assessment of students program codes Ahmed Hashim June 22, Introduction Automatic assessment provides an effective method for giving immediate 24/7 feedback service for students of mass courses [MKS02]. The aim of automatic assessment system is not only to reduce the grading time on the teaching staff, but also to increase the quality of students programs by assessing them and providing helpful feedback to the students. Nowadays, there are several programming languages, which are found to be difficult for a novice programmer to understand. According to Robins [RRR03], students have often difficulties in building mental model of computer programs, since a programming language differs from the structure of natural language. Even when the students have learned the programming concepts and languages, they may still lack the skills for using this knowledge to create computer programs. Moreover, novice programmers are usually not very good at evaluating their own programs, as even incorrect programs can seem to work as desired [AMJ04]. For this reason, automatic assessment of students program codes without direct helpful feedback is a useless task. In this paper, we consider techniques for automatic assessment of students program codes. Section 2 describes the three common kinds of programming languages and provides example codes for each one. Section 3 describes the assessing students programs and shows the common errors of the novice programmers. An automatic assessment system is proposed in Section 4. Finally, conclusions are drawn in Section 5. 1

2 2 Programming languages Programming languages are formal languages that have been designed to express computations. For example, the notation that mathematicians use is a formal language that is particularly good at denoting relationships among numbers and symbols, while natural languages are the languages that people speak, like English, Spanish, and French. This section describes the three common types of programming languages: 1. Procedural programming languages, 2. Logic programming languages, and 3. Object-oriented programming languages. 2.1 Procedural programming language Procedural programming is a way of writing code as an ordered list of instructions, telling the computer what to do. Procedural programming languages are considered as the third generation languages after machine-level, and assembly programming languages. Examples of procedural programming languages are BASIC, Pascal, C and FORTRAN. According to Niklaus Wirth [Wir78], a procedural program is defined as Algorithm + DataStructure. In principle, a programmer has to think in terms of data structures and algorithms for manipulating the data structures in procedural programming languages. For example, a procedural program may instruct the computer to read a number, multiply it by itself, and display the result. This is shown in Program 1, which is written in BASIC language. 2.2 Logic programming language A logic programming language is based on different way of thinking than procedural programming language. The former is based on the formal logic system with operational semantics, which is defined by deduction in that system [Der85], while the latter is based on the separation between instructions and data structures. Logic programming languages and functional programming languages are members of the same class, which is called Declarative programming language. Declarative programming languages are considered 2

3 Program 1 : Calculate the square number Input: Number is any real number Output: Result is Number Number 1 BEGIN 2 CLS clean the screen 3 INPUT Enter a number, Number 4 Result = Number Number 5 PRINT The Result is, Result 6 END as the fourth generation languages. Examples of logic programming languages are Prolog, Mercury and Oz, while examples of functional programming languages are Lisp and Scheme. According to Robert Kowalski [Kow79], a logical program is defined as Logic + Control. In logic programming, a programmer has to think about correctness at the logical level and about efficiency at the operational level. For example, the squaring function for natural number X may be written as a set of tuples as {(0, 0), (1, 1), (2, 4), (3, 9),...}. We can write the abstract form of the same function as sqr = (X, X 2 ).. Program 3 shows the squaring function, which is written in PROLOG language. Program 2 : Calculate the square number Input: X is any real number Output: Y is the result of X X 1 sqr(x, Y ) : Y is X X. 2.3 Object-oriented programming language The object-oriented programming language is a type of programming in which programmers define not only the data type of a data structure, but also the types of operations that can be applied to the data structure. In 3

4 this way, the data structure becomes an object that includes both data and functions. In addition, programmers can create relationships between one object and another. Examples of object-oriented programming language are Ada, C++, and Java. Programming in an object-oriented language requires the programmer to think in terms of a hierarchy of objects and the properties possessed by the objects. Program 3 shows the squaring function, which is written in Java language. Program 3 : Calculate the square number Input: N umber Output: Result 1 class TestSquare{ 2 public static void main(string atg[]){ 3 TestSquare obj=new TestSquare(); 4 int x = System.in.read(); 5 int y = obj.square(x); 6 System.out.println( The result is +y); 7 } 8 public int square(int x){return x*x;} 9 } As a result, several programming languages increase the complexity of automatic assessment process, since any program, which is written by any of the described programming languages, should be assessed based on that language. 3 Assessing students programs Student programmers have difficulty finding and fixing syntax and logic errors in their programs, since several programming languages do not provide enough information to diagnose the difficulties of the students program codes during the compilation process. This section describes the common errors of the novice programmers and gives a short overview of existing systems for assessing programs. 4

5 Typically common errors of the novice programmers can be classified into two categories: 1. Syntax errors, and 2. Logical errors. 3.1 Syntax errors A syntax error is a fatal error that the programmer must correct before being able to fully compile the program. Usually compilers diagnose the syntax errors by displaying an error message of the problem and also the location where it occurs. Examples of syntax errors are mistyping the name of the variable or forgetting to close a string, mismatched parentheses or braces. For example, a novice programmer may mistype the name of the variable. This is shown in line 4 of Program 4. Program 4 : Calculate the square number Input: Number is any real number Output: Result is Number Number 1 BEGIN 2 CLS 3 INPUT Enter a number, Number 4 Result = Num Num Remark for a syntax error 5 PRINT The Result is, Result 6 END 3.2 Logical errors A logical error is a mistake in the meaning or logic of the program. Unlike syntax error, logical errors are the most difficult ones to locate because there is generally no information available about the source of error. That is why the logical errors are more interesting than the syntax errors, because the former allow the teacher to identify the most common conceptual errors, which are made by novice programmers. Examples of logical errors include type mismatch, uninitialized variables, dividing by zero, and an infinite loop. For example, a novice programmer may forget to initialize the variables. This is 5

6 shown in line 1 of Program 5. Program 5 : Calculate the square number Input: X is a real number Output: Y is the result of X X 1 sqr(x, Y ) : Y is Y X. Remark for a semantic error or logical error As a result, to assess the students programs, we need to detect and diagnose the logical errors, which may be found in these programs. 4 Automatic assessment systems An automatic assessment system is software that takes the students program codes as an input, assesses the programs without need for human intervention, and generates useful feedback based on the assessment criteria. This section describes related research in the area of automatic assessments and presents the architecture of our proposed automatic assessment system. 4.1 Related research A student program can be assessed in various ways which include correctness, efficiency, complexity and style. Examples of existed systems are ASSYST [JU97] and CAP [Sch95]. Both of systems perform a static analysis. A static analysis is the process of examining source code without executing the program. The main aim of static analysis is to determine the quality of students programs. ASSYST is (ASsessment SYSTem) tool that assess the students programs. ASSYST assesses and grades a student program automatically based on five phases: correctness, efficiency, complexity, style and data testing. ASSYST tool may use to assess Ada and C programming languages. Figure 1 shows the assessment process of ASSYST. 6

7 Figure 1: ASSYST assessment process CAP is (Code Analyzer for Pascal) tool that analyze only the subset of Pascal language. The tool checks the student program and informs the student what is wrong, why it is wrong and how to fix the problem. CAP can diagnose three types of errors: syntax errors, logical errors, and style errors. A style error is an incorrect in the commenting declarations, commenting formal parameters, or indenting nested statements [MMNS83]. Figure 2 shows the assessment process of CAP. Figure 2: CAP assessment process 7

8 4.2 Automatic assessment architecture The main goal of our proposed automatic assessment system is to assess the students programs based on correctness, efficiency, complexity, style and data testing. Also, the system must generate helpful and clear feedback that shows what is wrong, why it is wrong and how to fix the problem. This feedback enhances the student s learning experience and increases the quality of students programs as well. In addition, the system must be capable of storing and retrieving the assessment process information, which includes the student s program, the student s mistakes, and the feedback of the system. Otherwise it will not help the educational process. Figure 3: Overview of the architecture Figure 3 illustrates a proposed automatic assessment system from components point of view. The system combines the functionalities of both of assessment and code analyzer into three main components: code analyzer, style analyzer and performance analyzer. Code analyzer detects the syntax and logical errors, and generates code report. Style analyzer detects the style and comment errors, and generates style report. Performance analyzer detects program complexity, program outputs, and program efficiency. performance analyzer generates performance report. The system generates the 8

9 feedback by combining the three reports, and informs the student the final results. 5 Conclusions The advantage of automatic assessment system is that it provides helpful and immediate feedback to the student without need for human intervention. However, designing and implementing automatic assessment of students program codes is still a complicated task, because it depends not only on programming the type of the programming language which is used, but also on the methods and techniques that are used to evaluate the students program codes. The main objective of this paper is to show the requirements of automatic assessment in order to achieve automatic assessment of students program codes. Finally, we hope that our paper will promote other systems to foucs on the requirements of automatic assessment, and apply these requirements in thier environments, as well. References [AMJ04] Kirsti Ala-Mutka and Hannu-Matti Jarvinen. Assessment process for programming assignments. pages , Washington, DC, USA, IEEE Computer Society. [Der85] Nachum Dershowitz. Synthetic programming. Artif. Intell., 25(3): , [JU97] [Kow79] [MKS02] David Jackson and Michelle Usher. Grading student programs using assyst. pages , New York, NY, USA, ACM Press. Robert Kowalski. Algorithm = logic + control. Commun. ACM, 22(7): , Lauri Malmi, Ari Korhonen, and Riku Saikkonen. Experiences in automatic assessment on mass courses and issues for designing virtual courses. pages 55 59, New York, NY, USA, ACM Press. [MMNS83] Richard J. Miara, Joyce A. Musselman, Juan A. Navarro, and Ben Shneiderman. Program indentation and comprehensibility. Commun. ACM, 26(11): ,

10 [RRR03] Robins, Rountree, and Rountree. Learning and teaching programming: A review and discussion. Computer Science Education, 13(2): , [Sch95] [Wir78] Tom Schorsch. Cap: an automated self-assessment tool to check pascal programs for syntax, logic and style errors. pages , New York, NY, USA, ACM Press. Niklaus Wirth. Algorithms + Data Structures = Programs. Prentice Hall PTR, Upper Saddle River, NJ, USA,