Intermediate Programming String-Interpretation and Graph-Algorithms Exercise Sheet 4

Transcription

1 Institute of Scientific Computing Technische Universität Braunschweig Prof. Hermann G. Matthies, Ph. D. Dr. Elmar Zander Winter Term 2013/14 November 14, 2014 Intermediate Programming String-Interpretation and Graph-Algorithms Exercise Sheet 4 Interpretation of Expressions In the development of computational or interactive software, it is often nescessary to interprete complex user input and to transform this input into a system response of the software system. Typing in commands to the command-line of UNIX is an example of such interaction between a user and an interactive language interpreter. Examples as reading of complex data sets from a file into a program or the development of applications for the Internet require the manipulation and interpretation of strings. In this exercise the manipulation and interpretation of strings is focused. Implementing interpreters for simple expressions is not hard, compared to writing interpreters for complex languages as programming languages. For such a task professional tools like JavaCC or ANTLR should be used, which provide code generators for such interpreters. For simple expressions as we will have to handle here, Java provides integrated support. For describing formal languages, the Backus-Naur-Form (BNF) is an established tool ( Language to Describe Formal Languages In the meta-language BNF (Meta-language means something like Language for describing Languge ) so-called productions can be used to define which form a class of expressions can take. Example: <digit without Null> ::= A digit without Null is a 1, or a 2, or a 3,... This production rule describes, what digit without Null is. Rules can also define sequences, in which certain symbols occur in predefined order: <digit> ::= 0 <digit without Null> <two-digit number> ::= <digit without Null> <digit> <ten to nineteen> ::= 1 <digit> <fourtytwo> ::= 42 A digit is therefore defines as a 0 or a digit without Null. A two-digit number is defined as a digit without Null followed by a digit. Fourtytwo id defined as a 4 followed by a 2. Repitations of symbols can be defined in BNF also, therefore recursions of production-rules have to be used. A production rule with the left-hand symbol on the right-hand side is such a recursion: <digit string> ::= <digit> <digit> <digit string>

2 Reading: A digit string is a digit or a digit followed by a digit string. With the help of such productions we can easily define arbitrary complex expressions. We want to use the BNF on this exercise sheet to describe a very simple language for defining graphs. Therefore we at first present you the basics os graph theory. Graphs for modelling Routes A Graph in the sense of mathematical graph theory consists of a set (the so-called nodes or vertices of the graph) and of direct connections between the them (the so-called edges of the graph). Mathematically speaking, a graph G is a tupel (V, E), where V is a set of nodes and E is a set of edges. An example which fulfulls this definition is given by: G 1 = ({1, 2, 3}, {(1, 2), (3, 1), (3, 2)}) This graph G 1 has three nodes, the digits 1, 2 and 3; and it has three nodes from the digit 1 to 2 and from digit 3 to both 1 and 2. It is also possible to illustrate graphs graphically, the graph G 1 is for example visualised in 3 figure 1. Graphs like these have applications in many domains of science and are of great importance. They are used in electrotechnology e.g. in circuit engineering to modell circuits, in computer-science to model processes and data-structures and in mathematics to model descrete topologies; the list of 1 2 applications can be made arbitrary long. Pracicle relevance in every-day life can be found e.g. in stree-navigation, where places are represented by nodes Figure 1: Graphic representation of Graph G 1 and connection-ways between the nodes are modelled represented by edges. The graph in figure 2 shows a modell of the connection-ways between the traffic-nodes Braunschweig, Hannover and Hamburg. The connections between the nodes are annotated with the name of the autobahn connecting the cities. With the help of this graph you can already calculate a route to get from Braunschweig to Hamburg. If all cities of Germany with all streets where present in the graph, we would be able to find a path between arbitrary nodes in the graph. A2 Braunschweig Hannover A2 A7 A7 Hamburg Braunschweig A2 dist=62.32 A2 dist=61.31 A7 dist=148.2 Hannover Hamburg A7 dist=149.3 Figure 2: A simplified graph of the connections between Hamburg, Hannover and Braunschweig. Figure 3: A simplified graph of the connections between Hamburg, Hannover and Braunschweig with annotated distances. It is obvious that any route we find in the graph does not have to be nescessarily the shortest, because in the graph are no informations present for the distances between the nodes, which are nescessary for

3 finding a route with minimal distance. This function is known from every modern navigation system. For increasing the information content of the graph more data can be added to nodes and/or edges. Such additional information may be the distances (or in other words the cost) of an edge, the traffic jam likelihood, the coordinates of the nodes,... This is exemplary done in figure 3. If the data represent the real situation you can realy use the graph for finding the shortest path between traffic nodes in the real world. By enriching the graph with further information, many other things can be computed easily; the application is only bounded by fantasy. A Language for describing Graphs In the last paragraph we gave a mathematical definition of graphs; formaly a graph consists of a set of nodes and a set of edges. G = (V, E) For developing a language for the description of these entities, we need in our new language expressions to define them. An example for such language in BNF is: <NAME> <REAL> ::= <STRING a-z 0-9 _ > ::= <FLOAT> <NODEDEFINITION> ::= NODE ( <NAME> ) <EDGEDEFINITION> ::= EDGE ( <NAME>, <NAME>, <REAL> ) <HELPCALL> ::= HELP <QUITCALL> ::= QUIT As you can see we neglected many details in the first two definitions, we did not give a precise formal definition of the productions. The language defined here is called regular. That is, the language is simple compared to other languages which can also be specified in BNF. Therefore we can evaluate strings containing this language by using so-called regular expressions in Java (class java.util.regex.pattern). Those expressions can immediately be derived from the BNF definition, and using placeholders we are able to extract particular values like node names and edge weights from strings that comply with the language definition. Exercise 1: Shortest Path Algoritms Floyd-Warshall At first you should read and understand the source code given at ADV/files/exercise4/graph.zip. If you have understood the structure you will realize that the presented program is easy to extend. Therefore do not get put off if you do not understand it immediately. Programs which evaluate formally defined languages are called parsers. Thus the program provided by us contains a parser as well, which is implemented in the file Parser.java. Please have a look at this file first. In the method parsecommmand(string command) the parser attempts to identify one of our BNF-defined language elements (NODE( ), HELP etc.) in the given string command and on success extracts the particular parameters. The identified elements are then interpreted as instructions to construct a graph. Elements that do not contain any parameters can be parsed using simple string comparisons (e.g. command.equalsignorecase("quit")), for all other elements we use the abovementioned regular expressions to identify the structure of the given command (NODE( ) or EDGE(,, ). For this purpose a Matcher object is constructed from each regular expression in conjunction with the input string. The Matcher checks if the input string matches the regular expression it was constructed from. If it does, the parameter values are extracted from the input. In a regular expression, extractable parameters

4 are marked with parentheses. Those special placeholders are called groups. Groups are indexed from left to right starting from 1 and their values can be retrieved using the method Matcher.group(int position). 1 else if(command.equalsignorecase("quit")) { 2 interpreter.quit(); 3 } 4 else if(edge_regex.matcher(command).matches()) { 5 // regex matches, extract given values 6 Matcher matcher = EDGE_REGEX.matcher(command); 7 8 // must be called again to make group access available 9 matcher.matches(); // extract values 12 String sourcename = matcher.group(1); 13 String targetname = matcher.group(2); 14 // convert to double value 15 double weight = Double.parseDouble(matcher.group(3)); // invoke corresponding interpreter method 18 interpreter.addedge(sourcename, targetname, weight); 19 } Listing 1: Excerpt from method Parser.parseCommand The development of regular expressions would require in-depth explanation of the complex syntax and therefore is not covered in this exercise. We will provide regular expressions in the following tasks as needed. See for detailed information. When the parser recognizes a command it invokes the corresponding method of the interpreter object that was specified in the parser constructor. The intepreter performs the actual construction of the graph and controls the remainder of the program. We could have constructed the graph directly within the parser, however this functional separation has some advantages and is done in complex software systems. This way we can for instance simply exchange the current parser with a different one without having to reimplement the construction of the graph. If the parser cannot identify any valid command, it throws a ParseException, which is caught in the main class of the program (Main.java). The main class just reads entire lines from the console and hands them over to the parser. The interpreter (file Interpreter.java) gradually adds nodes and edges to the graph object. The graph is implemented using the classes Graph, Node and Edge. Please read the source code comments of these classes carefully, as you will need them later in this exercise to implement some algorithms. The presented program allows to read in graphs from files and can perform arbitrary algorithms on them. An important problem concerning complex graphs is the search for the shortest path from one node to another, minimising the cost. In the example given in figure 3 it was almost trivial to find the shortest path, because there was no alternative path available. If the considered graph is larger than this one, and

5 1 void addedge(string sourcename, String targetname, double weight) { 2 graph.addedge(sourcename, targetname, weight); 3 } 4 5 void addnode(string nodename) { 6 graph.addnode(nodename); 7 } Listing 2: Excerpt from file Interpreter.java alternative routes are possible, the solution of the shortest path problem gets much harder. In the worst case even every possible route has to be systematically explored. This proceeding implements the so-called Floyd-Warshall algorithm, which you will have to implement in this task. The algorithm would not be used in modern navigation-systems, but it is easy to understand and implement, so it is good enough for us. A big problem of the Floyd-Warshall proceeding is its high (cubic) computational complexity in the number of nodes. Additionally the memory complexity of this proceeding is squared in the number of nodes, which means that a small graph with just nodes requires already memory fields and (with a size of double of 8 Bytes) 8MB are required for this small problem. If the problem-size is increased to , the required memory is already 800 MB. A modern navigation-system has to solve the problem with many more nodes and many less hardware, which makes the Floyd-Warshall algorithm not applicable in this domain. But neglecting this problem, we implement this algorithm for our purpose of learning. Note: The Floyd-Warshall algorithm is presented here as pseudocode. It determines the shortest distance between node source and node target in graph. FLOYDWARSHALL(source, target, graph) 1 n = number of nodes in graph //diagonal entries initially zero, all other entries infinity (Double.MAX VALUE) 2 costs = n n matrix 3 for each edge in graph 4 do costs[edge.source.id][edge.target.id] = edge.weight //floyd-warshall phase 5 for k 1 to n 6 do for i 1 to n 7 do for j 1 to n 8 do costs[i][j] = MIN(costs[i][j], costs[i][k] + costs[k][j]) 9 return costs[source.id][target.id] Note: When implementing the given pseudocode, please bear in mind that java array indices start at 0.

6 Task a) Implement the Floyd-Warshall algorithm in the file Graph.java, which you can find at The method header public double getshortestdistance(string sourcename, String targetname) is provided already, you just have to implement the body of the method. Also utilize the provided objects and methods of the graph data structure. Task b) Integrate and implement a new command SHORTESTDISTANCE(Source,Target) into the interpreter, which calculates and displays the shortest distance between the two nodes Source and Target. Employ the algorithm implemented in the previous task. The BNF of the new command is: <SHORTESTDISTANCECALL> ::= SHORTESTDISTANCE ( <NAME>, <NAME> ) Use the following regular expression: shortestdistance\\(([\\w]+),([\\w]+)\\) In order to parse the command and extract its parameters, use the already implemented commands NODE and EDGE as examples. The name of the source node resides in group 1, the target node name in group 2. Task c) Start the program with java Main and use file graph.in as its input (for example via input redirection). In this file we have defined a simple test graph and compute three sample routes. Verify your implementation using the following results: hamburg ---(210,52)---> braunschweig hamburg ---(328,30)---> muenster muenster ---(494,52)---> berlin Test your program with further input. Task d) Add new nodes and edges to the file graph.in and test your algorithm further. By now we considered how to save and restore complex data structures from a file. In the practice of software engineering, this is a common task. Another important technique is dynamic data structures, because without them programs would be less flexible and certain algorithms cannot be implemented. On a prior exercise sheet we considered already sorting and searching procedures, which used an array as basis data type. Many algorithms require structures as list in which new entries can also be efficiently added in the middle (this is not possible with an array) or trees allowing search queries, which require less than log 2 n steps. Since dynamic data structures play an important role, we are going to demonstrate this technique by implementing a so-called doubly linked list in the following task. The list concept depicted in figure 4 consists of two classes:

7 Class CircularList The main class CircularList in the upper left provides operations for manipulating the list (add, remove, determine its size). For that purpose the class maintains a reference to the first entry of the list (its head). 1 public class CircularList<T> implements Iterable<T> { 2 3 private static class Entry<T> { 4 Entry<T> next; 5 Entry<T> prev; 6 T value; 7 } 8 9 // first element 10 private Entry<T> head; 11 public int size(); 12 public void append(t value); Figure 4: Concept of a doubly linked list, which can be implemented in Java 13 //... etc 14 } Listing 3: Implementation of the list depicted in figure 4 Class Entry The actual list consists of objects of the class Entry. For each stored value there is an Entry object. These entries have references to their particular successor (next) and predecessor (prev) in addition to the actual value (field value). Therefore this concept is called a doubly linked list. For instance, in order to access the second entry of such a list, one could use the expression list.head.next.value. In fact single chaining using successor references only would be sufficient to store a list structure. However, the additional predecessor references simplify some list operations that would otherwise have to determine the predecessor of an entry using linear search. Furthermore doubly linked lists can easily be traversed in reverse order. Since the last entry of the list is linked to the head again, the depicted list is also circular. This for instance simplifies the special cases of inserting an element at the head or the tail of a list. In particular there are no nullpointers as each entry always has a successor and a predecessor. Also the last element of the list can be accessed quickly, since it is just the predecessor of the head. In listing 3 we show a part of the implementation of the discussed list concept in Java. The listing contains all the data fields, which are directly named after the fields in figure 4. Furthermore, with the type parameter T the list implementation is generic (as already introduced in Exercise Sheet 3), allowing values of arbitrary types to be stored.

8 Note: Class Entry is defined within class CircularList and is therefore called an inner class. Inner classes are used to express a tight coupling to the defining class. This applies to our Entry class: the list entries are used solely to organize the payload data within the CircularList class and are of no direct importance to other classes. Therefore the Entry class is also marked as private so that classes other than CircularList cannot even access it. The usage of static inner classes does not differ from the usage of regular classes, yet there is one exception: In order to use an inner class from another class, the name of the inner class must be preceded with the name of its enclosing (i.e. defining) class. For example, our entry class would be called CircularList.Entry rather than just Entry. Iterators Iterators are used to traverse a data structure one element after the other, for instance to print each value at the terminal or to evaluate a certain property of each element. Conceptionally, an iterator is a pointer to one element of the data structure, with operations to advance it to the next unvisited element and to check if there are still remaining unvisited elements. The third, right upper component of figure 4 illustrates this: If the list is to be traversed using the iterator, the dashed arrow successively points to each entry of the list. Code using this iterator therefore gradually gains access to each value. In Java, iterators are usually implemented as objects that are created by the particular data structure (list etc.). For iterators Java provides a standardized interface called java.util.iterator, which specifies the elementary iterator operations as next() and hasnext() (see com/javase/7/docs/api/java/util/iterator.html for full description). In one of the following tasks you are to develop a particular implementation for this interface. For further information on iterators, see section and of Textbook Introduction to Programming Using Java [Eck: Sixth Edition]. The java.lang.iterable interface ( java/lang/iterable.html) is closely related to the Iterator interface. It specifies a single parameterless method iterator() that returns an iterator. It is advisable to have every class which provides an iterator implement this interface. Objects of those classes may then be used in the enhanced for loop (sometimes called foreach). Therefore we have already added the interface to our implementation in listing 3. In listing 4 we show an example of the relationship between iterators, Iterable and the enhanced for loop. Revisit section of Textbook Introduction to Programming Using Java [Eck: Sixth Edition] for a detailed explanation. The Principle of separating data-structure from algorithms Writing a program using a list, two different approaches are possible: Embedded the operations to manipulate a list can be written directly into the code a new list entry then requires explicit code inside the algorithm to add the entry to the list, meaning to change the references next and prev of the respective entries locally in the algorithmic code. Encapsulated the operations to manipulate a list can be written separately in form of classes and methods. Methods like insert,append,... which manipulate a list have to be provided. Embedding of manipulation operations results in a strong interweaving of technical code for datastructures and algorithmic code of the application. This interweaving makes it harder to modify the

9 1 class List1 { 2 Iterator iterator(); 3 //... 4 } 5 6 List1 list1 = new List1(); 7 8 // Iterable not implemented, enhanced for loop cannot be used to traverse list1 9 for(iterator it = list1.iterator(); it.hasnext();) { 10 Object next = it.next(); 11 // } class List2 implements Iterable { 15 Iterator iterator(); 16 // } List2 list2 = new List2(); // enhanced for loop simplifies list traversal by hiding the iterator and is otherwise equivalent to the for loop in line 9 22 for(object next : list2) { 23 // } Listing 4: Interface Iterable and enhanced for loop resulting program. Pretend a situation where the list implementation of program is to be changed, e.g. the process of integrating a new entry into the list is to be changed. In the embedded variant the complete application code has to be browsed to find places where the list is manipulated. Therefore interweaving through embedding code obviously has serious disadvantages. In this exercise the advantageous separation of algorithms and data structures is to be trained by implementing a list. Therefore we provide a skeleton implementation for the already discussed linked list concept in file CircularList.java. In a later task, the list is to be used to store routes between cities. Since all needed methods of the class CircularList are already declared, we are able to provide you with a test program which verifies the correct function of your implementation. You can find the test program in the file CircularListTest.java. Exercise 2: List Implementation Task a) Download the template files from list.zip.

10 Task b) Complete the circular list implementation by implementing the methods declared in CircularList.java. Adhere precisely to the specification of each method (documented in the source code). Note: In order to implement the method iterator() you have to define another inner class which implements the Iterator. Afterwards test your implementation with java CircularListTest. Make sure all tests pass without any errors since the following tasks depend on the list being implemented correctly. Task c) Configure Ant to create a dedicated Jar library from your list implementation, which you will use in the following tasks. Application of the CircularList on evaluating routes In the last exercise the Floyd-Warschall Algorithm has been employed to find the shortest distance between two nodes in a graph. Here we will use the algorithm to also calculate the shortest route to take. In this application the Floyd-Warschall algorithm computes a A N n n matrix (where n is the number of nodes in the graph). The algorithm computes for each two nodes i and j an entry A i,j = k which gives the identification number of a node on the shortest route. Because nodes are identified in the program by integer numbers (int), we need an additional integer matrix for this task, which you can create with int[n][n] similarly to double matrices.

11 Note: The Floyd-Warshall algorithm to calculate the shortest path between source and target and its length is presented here as pseudocode: FLOYDWARSHALLROUTE(source, target, graph) 1 n = number of nodes in graph //diagonal entries initially zero, all other entries infinity (Double.MAX VALUE) 2 costs = n n matrix //all entries initially -1 3 route = n n matrix 4 for each edge in graph 5 do costs[edge.source.id][edge.target.id] = edge.weight 6 route[edge.source.id][edge.target.id] = edge.target.id //floyd-warshall phase 7 for k 1 to n 8 do for i 1 to n 9 do for j 1 to n 10 do if (costs[i][j] > costs[i][k] + costs[k][j]) 11 do costs[i][j] = costs[i][k] + costs[k][j] 12 route[i][j] = k 13 waypoints EVALUATEOPTIMALROUTE(route, source.id, target.id) 14 return costs[source.id][target.id] The lines 13 and 14 are both meant to be results of this algorithm. Note: This is the pseudocode for the function EVALUATEOPTIMALROUTE. EVALUATEOPTIMALROUTE(route, source id, target id) 1 result = list of integers 2 waypoint = route[source id][target id] 3 if (source id!= target id and target id waypoint) 4 do subroute1 = EVALUATEOPTIMALROUTE(route, source id, waypoint) 5 result.appendlist(subroute1) 6 subroute2 = EVALUATEOPTIMALROUTE(route, waypoint, target id) 7 result.appendlist(subroute2) 8 else 9 result.append(waypoint) 10 return result

12 Exercise 3: The Shortest Path Finally we have all tools ready to calculate the shortest path between two nodes in a graph. Task a) Implement the above algorithm in the file Graph.java. For the FLOYDWARSHALLROUTE function implement the body of the method public double getshortestpath(string sourcename, String targetname, CircularList<Node> path), and for EVALUATEOPTIMALROUTE use private CircularList<Integer> evaluateoptimalroute(int[][] route, int sourceid, int targetid) accordingly. Employ the CircularList class you have developed on this exercise sheet for the implementation of these methods. Note: The path parameter of the getshortestpath(...) method is to contain an empty list at first, which will be filled with the intermediate nodes of the calculated path (waypoints variable) by the algorithm. Thereby the method can return both the path and the distance. Please note that this list contains full Node objects, whereas the algorithm internally uses node ids only. Therefore you have to eventually convert the list of node ids into a node object list. We provide the method getnode(int id) to assist with this conversion. Task b) Similarly to task 1b), integrate and implement a new command SHORTESTPATH(Source,Target) into the interpreter. The BNF of the new command is: <SHORTESTPATHCALL> ::= SHORTESTPATH ( <NAME>, <NAME> ) Use the following regular expression, again with group numbers 1 and 2. shortestpath\\(([\\w]+),([\\w]+)\\) Have the command display the complete route between the two given nodes. The example output depicted in listing 5 was calculated using the test graph again, therefore you may use it as a reference to verify your implementation.

13 1 hamburg ---(210,52)---> braunschweig 2 via: 3 hannover 4 hamburg ---(328,30)---> muenster 5 via: 6 hannover 7 muenster ---(494,52)---> berlin 8 via: 9 hannover 10 braunschweig 11 magdeburg Listing 5: Example output of SHORTESTPATH