CS 2605 Lab 10 Spring 2008

Size: px

Start display at page:

Download "CS 2605 Lab 10 Spring 2008"

Leslie Cross
5 years ago
Views:

1 Text Parsing and Indexing: A Minimal db Goal In this lab, you will explore basic text parsing in C++. Learning Objectives understanding how to use getline() to break down input data understanding the logical view of a file as a sequence of bytes, the notion of offsets within a file, and how to use tellg() and seekg() to find and use offsets understanding the concept of an in-memory index structure for a collection of records design of an appropriate interface for a simple index class Mini-FAQ C++ and Files Logically, a file on disk is just a sequence of bytes. (The physical reality is somewhat more complex, but that's not an issue as long as you're using a high-level library to deal with the file.) So, each byte in the file is located at a specific, unique offset from the beginning of the file, just as the cells of an array are located at specific, unique offsets from the beginning of the array: T h e e f f e c t o f d e r e f e n You've already been introduced to I/O using several of the stream classes in the Standard Library. In this lab we'll make use of some features you may not have encountered yet. Input file streams maintain an internal pointer, called the get pointer, which stores the offset of the next byte of the file which would be returned if you performed a read operation. The get pointer is automatically updated, as needed, when a stream operator or member function is called. There are also three member functions which allow the user to make good use of file offsets: tellg() Returns the current value of the get pointer (i.e., the file offset); the return type is streampos, which is an integer variant seekg(offset) Resets the get pointer to that value; if the value is negative or exceeds the offset of the last byte of the file, the behavior is undefined. seekg(offset, direction) Adjusts the get pointer according to the value of the direction parameter: beg sets the get pointer to offset cur increments the get pointer by offset end sets the get pointer to a location offset bytes before the end of the file If the resulting location is invalid, the effect is undefined. For the record, output streams have an internal put pointer, which is the offset at which the next byte would be written, and similar functions tellp() and seekp() that allow the user to manipulate the put pointer. However, we won't be making use of those during this lab. 1

2 When parsing formatted input, the getline() function is often very useful: getline(inputstream, targetstring, stopcharacter) Parses inputstream from the current position forward until the stopcharacter is found, or the end of the stream is reached, putting the intervening characters into the targetstring. If the stopcharacter is found, it is removed from the stream, but not added to targetstring. This provides a simple way to parse input that consists of string "tokens" that are separated by specified characters, like a tab-separated file exported from a spreadsheet program. getline() does not skip over leading whitespace. Note that getline() does suffer one serious limitation. Since it places the data into a string object, it cannot be used to read numeric data without performing extra steps to convert the data after reading it. You might also want to know about the input manipulator ws. You can use this to advance the get pointer of an input stream to the next non-whitespace character (without removing that character from the stream). For example: Now it's time to put some of this to work. cin >> ws; Part I: Finding record offsets Choose a driver for this part. Your task is to write a function that can parse a file of relatively simple records and return a list of the file offsets at which those records begin. There's no reason at this time to make this a class member function. Download the file db01.txt from the lab page. The file contains a small number of records like this: AC P00350 DE 6-phosphogluconate dehydrogenase, decarboxylating (EC ). OS Escherichia coli KW Gluconate utilization; Oxidoreductase; Pentose shunt; NADP; KW Complete proteome. CC -!- CATALYTIC ACTIVITY: 6-phospho-D-gluconate + NADP(+) = D-ribulose CC 5-phosphate + CO(2) + NADPH. CC -!- PATHWAY: Hexose monophosphate shunt. CC -!- SIMILARITY: Belongs to the 6-phosphogluconate dehydrogenase CC family. CC CC This SWISS-PROT entry is copyright. It is produced through a collaboration CC between the Swiss Institute of Bioinformatics and the EMBL outstation - CC the European Bioinformatics Institute. There are no restrictions on its CC use by non-profit institutions as long as its content is in no way CC modified and this statement is not removed. Usage by and for commercial CC entities requires a license agreement (See CC or send an to license@isb-sib.ch). CC SQ MSKQQIGVVG MAVMGRNLAL NIESRGYTVS IFNRSREKTE EVIAENPGKK LVPYYTVKEF... Each record consists of six (or less) sections and a record terminator "". Each line of a record begins with a twocharacter line code indicating which section of the file it belongs to. For our current purpose, you don't need to worry about the significance of those line codes. But, you do need to know that the only line codes are the ones shown in the sample above, and that a line code is always followed by one or more spaces. The record terminator is followed immediately by a newline character. You must implement a function that meets the following specification: 2

3 The function parses File, and returns a vector of unsigned int values which are the file offsets of the records in that file. Pre: File contains a sequence of records of the form described above. Returns: A vector containing the offsets of the protein records found, listed in the same order the records occur in the file. vector<unsigned int> findoffsets(ifstream& File); All that's required is that your solution conforms to this interface, and that it computes the correct solution. Remember that so far we don't care about the specific data that's contained in the records, just finding the appropriate file offsets. For the given sample data file, the correct offsets would be: Note: The data files are supplied in Unix format, which is what your installation of Cygwin and g++ should be configured for. Be careful to not mess that up. In particular, download the files by right-clicking and selecting Save Target or Save Link, and do not open the data files in Notepad or modify them in any way. The file offsets do depend on the file format, so if you change it then you'll get different results. Put your function in a separate cpp file, named PartI.cpp, and create an appropriate header file, named PartI.h. If you write helper functions include them in the same cpp file. Test your solution by writing a simple driver and using the supplied data file. This is pretty straightforward; be sure you refer to the FAQ at the beginning of the lab specification. When you're sure it's correct, zip up the header file and the cpp file containing your function, and any helper functions it uses, and submit that to the Curator under the heading Lab10PI. Each member of your group must make a submission. This part will be auto-graded. The maximum score is 100. Part II: Building an index of the records Switch drivers for this part. Determining the offsets of the records is just the first step to producing a useful index of the records. An index enables the user to specify the value of a key field within a record and obtain the location of the matching record (or records if the key value isn't unique). The records we're dealing with consist of a number of fields; the AC, or accession code, field is a unique identifier for a record, and you will begin the process of building an index of records using their accession codes as the key. This requires creating two classes. First, you need a class that encapsulates the indexing data for a record. The interface for this class is determined entirely by the types of data it will store. The generic approach would be to implement a templated class, and in fact there's already an appropriate one in the STL. On the other hand, there's also something to be said for creating a custom class for the specific problem at hand, in which case, you might implement something very like the following class: 3

4 class ACEntry { public: ACEntry(const std::string& AC, unsigned int Offset); std::string AC() const; unsigned int Offset() const; private: std::string mac; unsigned int moffset; }; You are not required to implement the class shown above; the user will not ever access objects of this type directly. However, you must implement some such class, and the index must store one of those objects for each protein record that was found in the data file, and at this point it's acceptable to store them in a vector. However, the vector doesn't provide an appropriate interface from the user's perspective. The user of an index expects to carry out transactions of the form: "here is an accession code, give me the file offset of the corresponding record". Notice that the user doesn't even need to (or want to) be aware of the type of object we've put into the vector. So, we need a new class to provide an interface that's appropriate to a user of an index. The following class declares a minimal interface for an index: class ACIndex { public: ACIndex(std::ifstream& File); unsigned int offsetof(const std::string& AC); private: up to you... }; Implement this class. The constructor takes a file stream, already opened on a file like the one described in Part I, parses it, and builds the list of accession code index entries. You should be able to recycle much of your code from Part I here. Put the class declaration and implementation in files named PartII.h and PartII.cpp. You should write a test driver and test your index, using the offsets found in Part I to verify your results. When you're sure it's correct, zip up the header file and the cpp file containing your solution, and submit that to the Curator under the heading Lab10PII. Each member of your group must make a submission. This part will be auto-graded. The maximum score is 500. Part III: Finding records by offset Switch drivers for this part. For this part, you'll implement a companion function to the one you implemented in Part I: The function parses File, and returns a string object containing the protein record data beginning at Offset. Pre: File contains a sequence of records of the form described above. Offset is the file offset of the first byte of a record in the file. Returns: A string containing the bytes of the specified record, exactly as they occur in the file (including newline characters). string findprotrecord(ifstream& File, unsigned int Offset); 4

5 Again, you should refer to the discussion at the beginning of the specification for useful C++ features. Since you know the offsets of the records in the given data file, you can easily test your solution. If you have a correct solution and you insert the returned string into an output stream, you should see precisely the same display of the requested record that you would see if you view the original file in a text editor. Put your function in a separate cpp file, named PartIII.cpp, and create an appropriate header file, named PartIII.h. If you write helper functions include them in the same cpp file. You should write a test driver and test your index, using the offsets found in Part I and the data file to verify your results. When you're sure it's correct, zip up the header file and the cpp file containing your solution, and submit that to the Curator under the heading Lab10PIII. Each member of your group must make a submission. This part will be auto-graded. The maximum score is 500. Part IV: Finding a field of a record Switch drivers for this part. The function above returns the entire record as a string object, but if we were to implement an application program to serve user queries about a collection of records, we would certainly want to be able to provide finetuned access to individual fields of a record. Your next task is to implement the following function: The function parses File, and returns a string object containing the SQ field of the protein record beginning at Offset. Pre: File contains a sequence of records of the form described above. Offset is the file offset of the first byte of a record in the file. Returns: A string containing the bytes of the SQ field, without the line code or the leading spaces (or the trailing newline). string findaasequence(ifstream& File, unsigned int Offset); There are at least two way to approach this. You could copy and modify your solution to Part III so that it returns only part of the protein record from the file. Or, you could call your solution to Part III and retrieve the entire record from the file into a string object, and then parse the string (using an istringstream) to extract the specified data. Either approach is acceptable. Put your function in a separate cpp file, named PartIV.cpp, and create an appropriate header file, named PartIV.h. If you write helper functions include them in the same cpp file. When you're sure it's correct, zip up the header file and the cpp file containing your solution, and submit that to the Curator under the heading Lab10PIV. Each member of your group must make a submission. This part will be auto-graded. The maximum score is 100. Backup Lab Work Before leaving the lab both you and your partner should backup your submitted lab work. It is your responsibility to be able to produce a copy of your work. The Curator server is reasonably reliable, and you can download your submissions from it, but it is always best to maintain your own backups. A USB key or ing yourself a copy of your submitted archive are good options. 5

Protein Sequence Database

Protein Sequence Database A protein is a large molecule manufactured in the cell of a living organism to carry out essential functions within the cell. The primary structure of a protein is a sequence