Protein Sequence Database

Transcription

1 Protein Sequence Database A protein is a large molecule manufactured in the cell of a living organism to carry out essential functions within the cell. The primary structure of a protein is a sequence of amino acids. There are 20 common amino acids, each of which has a chemical name (e.g., "Glycine"), a three-letter abbreviation (e.g., "Gly"), and a one-letter code (e.g., "G"). See for a table about the chemistry of the amino acids and for information about how the amino acids fit into the genetic code. The twenty one-letter codes are: A, C, D, E, F, G, H, I, K, L, M, N, P, Q, R, S, T, V, W, Y For the purpose of representing and manipulating the primary structure of a protein, it suffices to use the one-letter codes in a string. For example, MLQSIIKNIWIPMKPYYTKVYQEIWIGMGLMGFIVYKIRAADKRSKALKASAPAPGHH is the amino acid sequence for a human protein called "6.8 kda mitochondrial proteolipid". In this project, amino acid sequences will always be upper-case, with no white space inserted. There are many online databases from which protein sequences can be obtained. One is SWISS-PROT, which can be found at As of March 27, 2004, SWISS-PROT contained database entries for 146,720 amino acid sequences. Each database entry has much more information about a protein than its sequence, as can be seen by going to This is the entry for the protein whose sequence was given above. A shorter sample is given in Figure 4 on page 8 of this specification. Besides the amino acid sequence (found in a formatted form under the heading "Sequence information" near the bottom of the page) other important information in the SWISS-PROT entry includes: The primary accession code (e.g. "P56378") that is always a unique 6-character identifier for the entry A protein name (e.g., "6.8 kda mitochondrial proteolipid") that is an official name for the protein that indicates its function in a cell and One or more source organism fields (e.g., "Homo sapiens (Human)") that give the species where the protein is found. A molecular weight given to the nearest mass unit. For our purposes in this assignment, rather than use a text file of SWISS-PROT entries, the program will manipulate a binary database file that contains condensed protein records consisting of a primary accession number, a protein name, a source organism, the amino acid sequence, and the molecular weight. More precisely, each database record will be stored in the following format: Significance Type Comments Accession code sequence of characters alphanumeric, always 6 characters Protein name length unsigned short Protein name sequence of characters no meaningful restriction on contents Number of source organisms unsigned short The following pair of entries will be repeated the specified number of times Organism name length unsigned short Organism name sequence of characters no meaningful restriction on contents Amino acid sequence length unsigned short Amino acid sequence sequence of characters upper-case, from code set given above Molecular weight of protein unsigned int Last modified: 4/13/2004 2:10 PM 1

2 Since there are three fields of variable size, the total record length will also vary. The first four bytes of the database file will be an unsigned integer specifying the number of records stored in the file. Note that there are absolutely no stated limits on the lengths of the strings that occur in the protein records. Figure 1 below illustrates the binary layout of a protein record in the database file. The highlighting indicates the correspondence between the plain text and the binary representation. The first byte of the hex dump belongs to the previous record in the database file, and the last 15 bytes belong to the next record. Figure 1 Sample Protein Record in Plain Text and Binary Format: P protein gamma (Protein kinase C inhibitor protein-1) (KCIP-1). Homo sapiens (Human) Mus musculus (Mouse) Rattus norvegicus (Rat) VDREQLVQKARLAEQAERYDDMAAAMKNVTELNEPLSNEERNLLSVAYKNVVGARRSSWRVISSIEQKTSADGNEKK IEMVRAYREKIEKELEAVCQDVLSLLDNYLIKNCSETQYESKVFYLKMKGDYYRYLAEVATGEKRATVVESSEKAYS EAHEISKEHMQPTHPIRLGLALNYSVFYYEIQNAPEQACHLAKTAFDDAIAELDTLNEDSYKDSTLIMQLLRDNLTL WTSDQQDDDGGEGNN A 0B 0C 0D 0E 0F ABCDEF :8F D 33 2D P35214E :8F F E D 6D protein gamma (P 0000:8F F E 20 6B 69 6E rotein kinase C 0000:8F E F F inhibitor protei 0000:8F40 6E 2D B D E n-1) (KCIP-1) :8F F 6D 6F E Homo sapiens ( 0000:8F D 61 6E D D Human)..Mus musc 0000:8F C D 6F ulus (Mouse)..Ra 0000:8F E 6F ttus norvegicus 0000:8F F C B (Rat)ö.VDREQLVQK 0000:8FA C D ARLAEQAERYDDMAAA 0000:8FB0 4D 4B 4E C 4E C 53 4E MKNVTELNEPLSNEER 0000:8FC0 4E 4C 4C B 4E NLLSVAYKNVVGARRS 0000:8FD B SWRVISSIEQKTSADG 0000:8FE0 4E 45 4B 4B D B NEKKIEMVRAYREKIE 0000:8FF0 4B 45 4C C 53 4C 4C 44 4E KELEAVCQDVLSLLDN 0000: C 49 4B 4E B YLIKNCSETQYESKVF 0000: C 4B 4D 4B C YLKMKGDYYRYLAEVA 0000: B B TGEKRATVVESSEKAY 0000: B D SEAHEISKEHMQPTHP 0000: C 47 4C 41 4C 4E IRLGLALNYSVFYYEI 0000: E C 41 4B QNAPEQACHLAKTAFD 0000: C C 4E B 44 DAIAELDTLNEDSYKD 0000: C 49 4D 51 4C 4C E 4C 54 4C STLIMQLLRDNLTLWT 0000: E 4E 0B 6E 00 SDQQDDDGGEGNN.n. 0000: F D 33 2D O70456! A more extensive hex dump of a sample binary database file is shown in Figure 2 on page 6 of this specification. Last modified: 4/13/2004 2:10 PM 2

3 Assignment: You will implement a system that maintains a database of amino acid sequences (proteins) stored in the format described above. There is no stated limit on the number of records that may be in the file, so all data structures must be fully dynamic. Your system will build and maintain several in-memory index data structures to support these operations: Inserting a new protein record Retrieving for protein records based on accession number, source organism, or amino acid sequence Retrieving protein records based on subsequences. You will implement a single C++ program to perform all system functions. Program Invocation: The program will take the size of the hash table and the names of three files from the command line, like this: ProteinDB <HT size> <database file name> <command script file name> <log file name> If the database file is not found, open a new file using the given name, and begin execution with an empty database. Naturally, if the script file is not found, the program should log an error message and exit. Data and File Structures: There will be a persistent database file, in the format described earlier. Adding a new protein record to the database requires updating the indexing data structures in memory and the database file on disk. Each of the search keys is simply an ASCII string, and so the keys can be compared using the standard relational operators. The amino acid sequence and accession codes are unique (primary) that is, no two different proteins will have the same accession code or amino acid sequence. The other fields are not guaranteed to be primary. The index for the amino acid sequences will use a hash table to store index entries, consisting of the amino acid sequence and the file offset at which the record begins. The hash table will contain the number of slots specified on the command line. Since open hashing is used, there is never any reason to resize the table. The amino acid sequences will be hashed using the ELFhash function discussed in the course notes. Rather than use a probing strategy, the hash table will store a container in each slot. The container in each slot should be capable of storing an arbitrary number of records (although we hope that will not really be necessary), and it should support efficient search, insertion and deletion. Within those constraints, it's your choice 2. The index structure for the accession codes is your choice, subject to the requirement that it support efficient search, insertion and deletion. The index entries in the accession code index will be similar to those in the amino acid sequence index, in that they will store two values however, the second value in an accession index entry will be the corresponding amino acid sequence. The index for the source organisms cannot be a hash table because there will be many protein records from a single organism. Since this index must also support efficient search, insertion, and deletion, we will use a binary tree structure. The ideal option would be a balanced tree, like an AVL. However, if you cannot do that reliably, you may use a plain BST instead, for a penalty of 5%. (There will be no partial credit given for semi-functional AVL trees.) The source organism index will store index entries containing the organism name and a list of corresponding amino acid sequences (NOT file offsets!). This means that retrievals based on the accession code or the source organism name will require first searching the accession code or source organism index and then performing one or more searches of the hash table that indexes the amino acid sequences 1. Aside from where specific data structures are required, you may use any STL components you like. At the start of execution, your program should parse the binary database file and build both index structures. Each index object should have the ability to write a nicely-formatted display of itself to an output stream. Last modified: 4/13/2004 2:10 PM 3

4 Other System Elements: There should be an overall controller that validates the command line arguments and manages the initialization of the indices. The controller should hand off execution to a command processor that manages retrieving commands from the script file, and making the necessary calls to carry out those commands. Command File: The execution of the program will be driven by a script file. Lines beginning with a semicolon character ('') are comments and should be ignored. Each non-comment line of the command file will specify one of the commands described below. Each line consists of a sequence of tokens, which will be separated by single tab characters. A newline character will immediately follow the final token on each line. The command file is guaranteed to conform to this specification, so you do not need to worry about error-checking when reading it. The following commands must be supported: show_sequence<tab><sequence> Log all the data fields in the protein record that stores the sequence given in <sequence>. show_accession<tab><accession> Log all the data fields in the protein record that has primary accession number <accession>. show_organism<tab><species> Log all the data fields in every protein record that includes the organism name given in <species>. At the end of the list, log the number of matching protein records that were found. add<tab><accession> Add a new protein record with primary accession code <accession>, if it does not already exist in the database. The protein name, organism(s), molecular weight, and sequence for the new record will be found on the following input lines in that order. The lines will be labeled with the flags DE, OS, MW and SQ, respectively. Each flag will be followed by three spaces. Note that it is possible there will be multiple organism lines. update<tab><accession> Update the protein record with primary accession code <accession>. If the given accession code is not already in the database, treat this as an add command. The protein name, organism(s), molecular weight, and sequence to update the record will be given on the following lines in the same format as for the add command. Updating the database file poses some challenges. There is no guarantee that the updated record will be the same length as the original record. Therefore, when updating an existing record you will do something (your choice) to mark the original record in the file as invalid, and then append the updated record to the database file. Note that this may require modifying all the index structures. find_subsequence<tab><subsequence> 3 In this command, <subsequence> must be a non-empty sequence of the twenty amino acid codes (all upper case). You are to log the primary accession code of every protein record that has a sequence that contains <subsequence> as a (contiguous) subsequence within it. You should be able to use one of the index structures to guide your search, especially if you provide iterator support, but this does require a full traversal of the database. Sequences should be logged in order of primary accession code. This will be treated as extra-credit. debug<tab>[accession organism sequence] Log the contents of the specified index in a fashion that makes the internal structure of the index clear. If the index is stored in a hash table, you should display information only for non-empty slots. clear<tab> Delete the contents of the entire database. open<tab><database file name> Read the protein records in the specified binary database file and add them to the database, if they are not duplicates. This will include adding the records to the original database file, just like an add command. Last modified: 4/13/2004 2:10 PM 4

5 exit<tab> Terminate program execution. A sample command script is included in Figure 3 on page 7 of this document. As a general rule, every command should result in some output. In particular, error messages should be logged if searches yield no protein records. Instrumentation: Each index must be instrumented so that it logs information about each search it performs. The information should identify each index record that is accessed during the index search, and should be written to the log file. Log File Description: Since this assignment will be graded by TAs, rather than the Curator, the format of the output is left up to you. Of course, your output should be clear, concise, well labeled, and correct. The first two lines should contain your name, section specification (e.g., "CS :00 TTh"), and project title (e.g., "Major Project 2: Protein Sequence Database"). The remainder of the log file output should come directly from your processing of the command file. You are required to echo each command that you process to the log file so that it's easy to determine which command each section of your output corresponds to. Each command should be numbered, starting with 1, and the output from each command should be well formatted, and delimited from the output resulting from processing other commands. A complete sample log will be posted shortly on the course website. Submitting Your Program: You will submit this assignment to the Curator System (read the Student Guide), where it will be archived for grading at a demo with a TA. For this assignment, you must submit a gzip'd tar file containing all the source code files for your implementation (i.e., header files and cpp files). Submit only the header and cpp files. Submit nothing else. In order to correct submission errors and late-breaking implementation errors, you will be allowed up to five submissions for this assignment. You may choose which one will be evaluated at your demo. The Student Guide and link to the submission client can be found at: Evaluation: Shortly before the due date for the project, we will announce which TA will be grading your project and post signup sheets inside the McB 124 lab. You will schedule a demo with your assigned TA. At the demo, you will perform a build, and run your program on the demo test data, which we will provide to the TAs. The TA will evaluate the correctness of your results. In addition, the TA will evaluate your project for good internal documentation and software engineering practice. Remember that your implementation will be tested in the McB 124 lab environment. If you use a different development platform, it is entirely your responsibility to make sure your implementation works correctly in the lab. Note that the evaluation of your project will depend substantially on the quality of your code and documentation. See the Programming Standards page on the course website for specific requirements that should be observed in this course. Pledge: Each of your program submissions must be pledged to conform to the Honor Code requirements for this course. Specifically, you must include the pledge statement provided on the Submitting Assignments page of the course website. Last modified: 4/13/2004 2:10 PM 5

6 Figure 2 Hex Dump of Sample Binary DB File: A 0B 0C 0D 0E 0F ABCDEF :0000 A C Q9V2L2B.Puta 0000: D 61 6D 69 6E 6F C tive 1-aminocycl 0000:0020 6F F E 65 2D 31 2D F opropane-1-carbo 0000: C D 69 6E xylate deaminase 0000: E 35 2E E (EC ) : F 63 6F Pyrococcus aby 0000: A 01 4D B C 4C ssij.mhpkvdallsr 0000: C C FPRITLIPWETPIQYL 0000: C B PRISRELGVDVYVKRD 0000: C C E 4B B 4C 45 DLTGLGIGGNKIRKLE 0000:00A0 46 4C 4C C FLLGDALSRGCDTVIT 0000:00B E C IGAVHSNHAFVTALAA 0000:00C0 4B 4B 4C 47 4C C 49 4C KKLGLGAVLILRGEEV 0000:00D0 4C 4B 47 4E 59 4C 4C 44 4B 4C 4D LKGNYLLDKLMGIETR 0000:00E E C 4D 4B IYEADNSWELMKVAEE 0000:00F C 4B B 4B VAEELKGEGKKPYIIP 0000: C PGGASPVGTLGYIRGV 0000: C B 4B 4C 47 4C GELYTQVKKLGLRIDT 0000: C 4C 4C VVDAVGSGGTYAGLLL 0000: E GSAIVNAEWSVVGIDV 0000: B 41 4B B 4E 4C SSATEKAKERVKNLVE 0000:0150 4B 54 4B 45 4C 4C E 56 4B KTKELLGINVKVQEPR 0000: B B IYDYGFGAYGKIVKEV 0000: B 4C 49 4B D C 4C 4C 44 AKLIKSVGTMEGLLLD 0000: B C 4D 44 4C 41 4B PVYTGKAFYGLMDLAK 0000:0190 4B C C C KGDLGESVLFIHTGGL 0000:01A D 4C 45 4C 4C 56 AA PGIFHYGEEMLELLVª 0000:01B0 8B Q8U4R3S.Putat 0000:01C D 61 6D 69 6E 6F C 6F ive 1-aminocyclo 0000:01D F E 65 2D 31 2D F 78 propane-1-carbox 0000:01E0 79 6C D 69 6E ylate deaminase 0000:01F E 35 2E E (EC ) (A 0000: D 69 6E E CC deaminase) : F 63 6F Pyrococcus fur 0000: F D B C 4C iosusi.mhpkvqsll 0000: B C SKFPRVELIPWETPIQ 0000: C 50 4E B 4C B YLPNISKLVGADIYVK 0000: C C E 4B B RDDLTGLGIGGNKIRK 0000:0260 4C C 4C B LEYLLGDAIIRKADVI 0000: E C ITVGAVHSNHAFVTGL 0000: B 4B 4C C 56 4C B AAKKLGFDVVLVLRGK 0000: C E 59 4C 4C 44 4B 49 4D EELRGNYLLDKIMGIE 0000:02A B C 4D 4B TRVYEAKDSFELMKYA 0000:02B B 45 4C B B EEVAKELEEKGRKPYI... Last modified: 4/13/2004 2:10 PM 6

7 Figure 3 Sample Command Script: Script file for the Protein Database Project Display some records: show_accession Q9V2L2 show_accession P11576 show_sequence ILNSPDRACNLAKQAFDEAISELDSLGEESYKDSTLIMQLLXDNLTLWTSDTNEDGGDEIK show_organism Drosophila melanogaster (Fruit fly) Add a new record: add P56378 DE 6.8 kda mitochondrial proteolipid OS Homo sapiens (Human) MW SQ MLQSIIKNIWIPMKPYYTKVYQEIWIGMGLMGFIVYKIRAADKRSKALKASAPAPGHH Echo the added record: show_accession P56378 Now dump the database and the indices debug accession debug organism debug sequence Quit: exit Last modified: 4/13/2004 2:10 PM 7

8 Figure 4 A sample SWISS-PROT protein database entry: ID 1431_MAIZE STANDARD PRT 261 AA. AC P49106 DT 01-FEB-1996 (Rel. 33, Created) DT 01-FEB-1996 (Rel. 33, Last sequence update) DT 01-OCT-1996 (Rel. 34, Last annotation update) DE like protein GF14-6. GN GRF1. OS Zea mays (Maize). OC Eukaryota Viridiplantae Streptophyta Embryophyta Tracheophyta OC Spermatophyta Magnoliophyta Liliopsida Poales Poaceae OC PACCAD clade Panicoideae Andropogoneae Zea. OX NCBI_TaxID=4577 RN [1] RP SEQUENCE FROM N.A. RX MEDLINE= PubMed= RA de Vetten N.C., Ferl R.J. RT "Two genes encoding GF14 (14-3-3) proteins in Zea mays. Structure, RT expression, and potential regulation by the G-box binding complex." RL Plant Physiol. 106: (1994). CC -!- FUNCTION: Is associated with a DNA binding complex to bind to CC the G box, a well-characterized cis-acting DNA regulatory element CC found in plants genes. CC -!- SIMILARITY: Belongs to the family. CC CC This SWISS-PROT entry is copyright. It is produced through a collaboration CC between the Swiss Institute of Bioinformatics and the EMBL outstation - CC the European Bioinformatics Institute. There are no restrictions on its CC use by non-profit institutions as long as its content is in no way CC modified and this statement is not removed. Usage by and for commercial CC entities requires a license agreement (See CC or send an to license@isb-sib.ch). CC DR EMBL S77133 AAB DR PIR T01752 T DR HSSP P A38. DR MaizeDB DR InterPro IPR DR Pfam PF DR PRINTS PR ZETA. DR SMART SM _3_3 1. DR PROSITE PS _1 1. DR PROSITE PS _2 1. KW Multigene family. SQ SEQUENCE 261 AA MW 25EC70725A5F6801 CRC64 MASAELSREE NVYMAKLAEQ AERYEEMVEF MEKVAKTVDS EELTVEERNL LSVAYKNVIG ARRASWRIIS SIEQKEEGRG NEDRVTLIKD YRGKIETELT KICDGILKLL ETHLVPSSTA PESKVFYLKM KGDYYRYLAE FKTGAERKDA AENTMVAYKA AQDIALAELA PTHPIRLGLA LNFSVFYYEI LNSPDRACSL AKQAFDEAIS ELDTLSEESY KDSTLIMQLL RDNLTLWTSD ISEDPAEEIR EAPKRDSSEG Q // Last modified: 4/13/2004 2:10 PM 8

9 Notes: The reason that the other two indices refer to the hash table index, instead of storing file offsets themselves, is that this simplifies maintaining the index structures if the binary database file is changed. For example, if lots of updates are performed, the database file will contain lots of "dead" records. It would be natural to read and re-write the database file periodically to eliminate those dead records but that will change the file offsets for many of the records. Since the hash table index is the only one that actually stores file offsets, it's the only index that would have to be rebuilt when this is done. The ELFHash function is good enough on long strings that you would expect to not have too many collisions if the hash table size is reasonable (i.e., about twice the number of records actually stored). Given that observation, it's acceptable to use an efficient, dynamic, linear structure in each slot. The find_subsequence command will be treated as an extra-credit component of the assignment. It will be evaluated (if you implement it) as a separate section of the testing, and will be worth up to 10/100 added to your final score. Last modified: 4/13/2004 2:10 PM 9