Generating Better Conformations for Roadmaps in Protein Folding

Transcription

1 Generating Better Conformations for Roadmaps in Protein Folding Jeff May, Lydia Tapia, and Nancy M. Amato Parasol Lab, Dept. of Computer Science, Texas A&M University, College Station, TX {ltapia, Abstract Probabilistic roadmap methods (PRM) in robotics have been successfully applied to find accurate protein folding pathways with a fast and robust framework for proteins with around 6 3 residues. We can create a roadmap that can be queried multiple times for energy feasible pathways to particular folded states. Misfolded proteins are the cause of proteopathic diseases and are of interest when studying folding pathways. For example, Alzheimer s disease, which is caused by the misfolding of Amyloid Precursor Protein, has between 365 to 77 amino acids[5]. Our goal is to generate better quality roadmaps for larger proteins. For the node generation phase, we generate new protein conformations from old conformations by perturbing them at random points along the protein chain. Certain portions of the protein contain bonds in the folded state that would restrict movement of the protein. Rigidity analysis allows us to classify these residues of a protein, and separate the protein, into two regions: rigid and flexible. We can specify different probabilities of perturbing the conformation for each region to improve the node generation process[2]. In this paper focus on improving the node generation process for PRMs. First, we look at the effect that these parameter values have on the resulting conformations. With this information we construct a Markov Decision Process (MDP) policy learning algorithm to dynamically tune parameter values for the node generation process. We show that policy learning shows promise to automate the task of specifying good parameters.

2 2 Introduction Using graphs to represent protein folding pathways is quick and robust method for finding multiple pathways in a very short amount of time using a graph traversal method. To generate the roadmap for protein folding, we borrow the probabilistic roadmap method (PRM) from robotics and model the protein as we would a robot with many armed linkages fixed to each other. Our goal is to find pathways that are likely to represent the way in which proteins will fold to reach particular conformations. Instead of having our robot maneuver around obstacles, it moves through a funnel-like potential energy landscape along energy feasible paths. PRMs have been implemented for a wide variety of applications[6]. For this research, we focus on the roadmap construction phase, specifically, the node generation process. We generate a new node by perturbing one or more of the angles of a parent conformation from the roadmap. If it is energy feasible, we add it to the roadmap. We start the process from the native state, the conformation representing the folded and functional protein in its natural environment. Rigidity analysis is a simple and fast method for classifying regions of the protein. We can use different probabilities of perturbing the protein in these regions to improve the node generation process. The issue then becomes picking the right parameter values for this process. By using a Markov Decision Process to reward the use of good parameters, we can dynamically tune the parameters during the node generation process produce nodes within an area of interest. For our process, we divide the energy landscape funnel into bins and attempt to fill each bin to a desired number of nodes. The process requires finding a sufficient number of nodes in each layer. By adapting the selection of parameters with MDP Policy Learning, we automatically find useful parameters for different areas of the funnel. This will save researchers valuable time for selecting good parameters on different varieties of proteins. 3 Related Work 3. Modeling Protein Folding with PRMs A probabilistic roadmap method (PRM) is a technique from robotics that is used for planning the motion of a robot from a start position to a goal position. It relies on the ability to abstract all the possible ways a robot can be positioned into what is called configuration space. The number of degrees of freedom (DoFs) that is required to describe the exact position of the robot determines the dimensionality of its configuration space. Mapping this space completely becomes an intractable problem very quickly. Instead we can use probabilistic methods, such as PRMs, to solve these problems with probabilistic completeness, where as the number of samples of configuration space increases, the probability of completely covering the space approaches %. We model a protein conformation as a chain of linkages, representing the backbone position of the protein[9]. Each residue has two torsional directions of rotation. A conformation, in our model, is a set of all the angles between the backbones of the amino acids/residues in the protein. Since proteins have 2 torsional degrees of freedom between each pair of amino acids, they have twice as many DoFs as they have amino acids. The set of all possible conformations (C-space) has as many dimensions as the protein has degrees of freedom. This makes the task of covering the landscape difficult for even small proteins, and it rapidly becomes more difficult for larger proteins. PRMs have two phases: a construction phase and a query phase. A roadmap is basically a simple graph containing valid configurations of a robot and collision-free connections between them. Once this graph is obtained, it can solve many queries for pathways in a short amount of time. The challenge of PRMs, and the focus of this research, is to generate good quality roadmaps. The quality of a roadmap depends on its use. For protein folding, we are interested in mapping the potential energy landscape to provide energy-feasible pathways between various protein conformations. A good quality roadmap will have well placed nodes that are well connected and cover the features of the energy landscape. 3.2 Roadmap Construction for Proteins To create a roadmap, we sample various points in C-space. Our methods bias sampling to increase density of nodes near the native state. The potential energy of these conformations / nodes are calculated. We use this information to decide whether to add the node to the roadmap or not. Once we have a set of nodes, we connect the nodes and test whether the transition between each of the connected conformations are energy feasible, discarding those that are not.

3 3.3 Node Generation for Better Protein Conformations We sample the configuration space of the protein by storing nodes that represent particular conformations. We classify regions of these conformations using rigidity analysis[2]. Each residue in the conformation is classified as part of a rigid, flexible, or dependently flexible region. A rigid region is one in which the bonds on the molecule would prevent the movement of the residues in any direction. A flexible region is defined by the characteristic that the residues are not held by bonds and are not constricted from changing their angle, and where doing so would not necessitate that the rest of the protein change. A dependently flexible region is one in which there is flexibility in a region, however changing the angle in any of the residues would require the rest of the protein to move. We generate nodes incrementally by randomly perturbing a subset of the nodes we currently have in the roadmap. To perturb a given conformation, we move along the amino acid chain until we decide to bend the angle of a particular residue using the following parameters: P flex the probability of perturbing a flexible region P rigid the probability of perturbing a rigid region θ std the angle (or quantity) of perturbation This method works better than the other methods and requires a smaller roadmap to compute the correct formation orders[2]. However, this method introduces a new problem: choosing the right parameter values. Choosing between many possible permutations of parameter values can be cumbersome for a researcher. A machine learning process for selecting good parameter values would ease the process of choosing the right parameter values. 3.4 Rigidity Analysis We use a rigidity analysis technique belonging to the class of approaches called the pebble game[, ] to better simulate motion. It is fast and efficient; we can apply it to every conformation we sample. The pebble game is a constraint counting algorithm which determines the DoF in a two-dimensional graph, along with its rigid/flexible regions. In 2D, the pebble game assigns each vertex two pebbles, representing its two DoF, see Figure a. Each edge/constraint is examined to determine if it is independent or redundant. If two free pebbles can be placed on both endpoints of the edge, then it is marked independent and covered by a pebble from one of its incident vertices. Once an edge is covered by a pebble, it remains covered, although which vertex the pebble comes from may change. Pebbles may be rearranged as shown in Figure a. If pebbles cannot be rearranged to get two free pebbles on both of an edges endpoints, then the edge is marked redundant and indicates a rigid region. In the end, the remaining free pebbles indicate the graphs DoF. The 2D pebble does not generalize to 3D for arbitrary graphs, but it can be applied to 3D bond-bending network[]. A bond-bending network is a truss structure with constraints between nearest neighbors and nextnearest neighbors. A protein, with xed bond lengths and bond angles, can be modeled as a bond-bending network, called the bar-joint model, where atoms are modeled as vertices with 3 DoF and bonds are modeled as edges, see Figure b. It has been successfully used by several applications to study protein rigidity and flexibility[2, 8, 8, 5]. An alternative model, the body-bar model, represents atoms as rigid bodies with 6 DoF and the torsional bonds between them as 5 bars/constraints[22] see Figure c. Both models are conjectured to be equivalent[]. move on move off (a) (b) (c) Figure : (a) The result of the pebble game on a 2D graph. Pebbles may be free (white) or covering (black). Constraints are marked as independent (solid) or redundant (dashed). Pebbles may be rearranged as shown. Rigidity models for a sample molecule: (b) bar-joint and (c) body-bar.

4 3.5 Connecting the Roadmap In the second step, connections (edges) are made between sampled conformations with similar structure. Weights are assigned to directed edges to reflect the energetic feasibility of transitioning between the two endpoint conformations. This combination of nodes and weighted edges forms a roadmap that approximates the energy landscape. This roadmap encodes thousands of folding pathways. The most energetically feasible pathways in the roadmap can be extracted using these weights. Connections between two nodes, q and q 2, are labeled with edge weights that reflect the energetic feasibility of transitioning between them. This is done by first identifying all the intermediate nodes, q = c, c,..., c n, c n = q 2, that connect q to q 2. For each pair of consecutive conformations c i and c i+, the probability P i of transitioning from c i to c i+ depends on the difference between their potential energies E i = E(c i+)e(c i): P i = { e E i kt if E i > if E i () This keeps the detailed balance between two adjacent states and enables the edge weight to be computed by summing the logarithms of the probabilities for all pairs of consecutive conformations in the sequence. With this edge weight definition, we can use simple graph search algorithms to extract the most energetically feasible pathways in the roadmap between two given states (e.g. from the unfolded state to the folded state). 3.6 Potential Energy Calculation Our method is flexible and allows any potential function to be used. In this paper, we use a coarse potential function similar to [6]. We use a step function approximation of the van der Waals potential component and model side chains as spheres with zero DoF. If any two spheres are too close (i.e., less than 2.4Å during sampling and.å during connection), a very high potential is returned. Otherwise, the potential is: U tot = K d {[(d i d ) 2 + d 2 c] /2 d c} + E hp (2) restraints where K d is kcal/mol and d = d c = 2. The rst term represents constraints favoring known secondary structure through main-chain hydrogen bonds and disulphide bonds, and the second term is the hydrophobic effect. The hydrophobic eect is computed as follows: if two hydrophobic residues are within 6 of each other, then the potential is decreased by kj/mol. In our previous work [2, 2], we provided methods for building an approximate map of a proteins potential energy landscape[2, 2] and an RNAs folding landscape[2]. We have published results from our approximate maps for proteins up to 48 residues easily built on a desktop PC[2]. Our roadmaps give an approximate view of the protein folding landscape. In the past, we have successfully extracted low-energy pathways, validated secondary structure formation order, and seen general and consistent trends in reaction coordinates such as native contacts present and RMSD. 3.7 Energy feasibility We use the potential energy calculation from 3.6 to decide the probability of accepting a conformation into the roadmap. A conformation q, with potential energy E(q), is considered energy feasible with the probability: P(accept q) = if E(q) < E min if E min E(q) E max (3) if E(q) > E max E max E(q) E max E min where E min is the potential energy of the open chain and E max is 2E min. 3.8 Native contacts present A native contact is a pair of C α atoms that are within a specified range in a given conformation as well as in the native state. For our system, we use at a distance of less than 7Å to define whether a pair of C α atoms are in contact. The number of native contacts present in any conformation can be used as an indication of how close it is to the native state. These contacts are used to define the formation of secondary structures in a conformation and to calculate the formation orders in a roadmap.

5 4 Methodology 4. Studying the Parameters The first stage of our research was to study the effect that different parameter values have on the resulting conformations. We collected data on the resulting conformations, such as the number of native contacts, the potential energy, and whether the node was energy feasible or in collision. When building a model of the energy landscape, we have two primary goals. First, we expect good coverage of the energy landscape. We would like a set of conformations that represent the allowable motions of the molecule. Second, we would like high-quality conformations. A high-quality conformation is one that will be feasibly undertaken, e.g., low energy. 4.2 MDP Policy Learning for Selecting Parameter Sets 4.2. Markov Decision Process in PRMs MDP policy learning has been previously used to impact PRM sampling. In robotics applications, there are many proposed sampling methods[3,, 4, 4, 7, 3] whose efficiency and effectiveness has been seen to be highly correlated with the planning space and the problem construction[7]. A method called Hybrid PRM uses MDP policy learning with a library of possible sampling methods, the actions. It has been shown to automatically learn which sampling methods will best cover an unclassified problem[9]. Markov decision processes (MDP) occur when an autonomous agent can sense outcomes from its actions in the environment. From the action and outcome relationship, policies can be learned to choose optional actions to achieve the agent s goals. Markov decision processes occur in many domains and has been solved in problems including mobile robot control, game playing, and robot motion planning. In a MDP, the learning agent perceives a set of states, S, that describe its current environment. It also has a set of actions, A, that it can select from. At each time step t, it can select an action at that is taken. The outcome of that action makes some impact on the environment β(s t, a t) that results in a new set of perceptions, s t+. The result of that outcome is measured in a reward function, r(s t, a t), that gives the agent some knowledge of the utility of its action a t. Learning progresses by the agent s pursuit of maximal rewards. Solutions to MDPs are commonly solved through dynamic programming and reinforcement learning. Algorithm 4. MDP Policy Learning in Node Generation Input. A set actions A defined by the parameters that can be selected a set of states S that defines current perception, a starting conformation c : for timestep, t, from to n do 2: Select action a t from probabilistically from rankings R a of set A 3: Generate c t where c t β(a t, c t ) 4: Generate reward r t based on new perceptions, s t+ where s t+ δ(s t, a t ) 5: Apply reward r t to P a where a is a t 6: end for Output. A set rankings of R a that defines the utility of each A Reward Function for Layers Method In order to insure that we have a diverse spread of nodes, we store conformations with similar numbers of native contacts into a common bin. Each bin holds conformations within a certain range of native contacts. We reward parameter sets for producing a diverse conformation. For each bin, we attempt to fill it to a desired amount. Rewards are generated for Algorithm 4. as follows based on the resulting conformation: R max: the conformation is within the bin we are attempting to fill R partial : the conformation is within another bin that is not currently filled R penalty : the conformation has an energy greater than E max or it is in a layer that is already full

6 4.2.3 Choosing a Parameter Set To choose a set of parameters for node generation (see Algorithm 4.), we use the roulette wheel selection method. This method is analogous to having a slot for every permutation of the possible parameter values for P flex, P rigid, θ flex, and θ rigid on a roulette wheel, where the size of the slot P i is dependent on its score r i relative to the scores of all the other parameter sets R. It probabilistically selects a set of parameters a from A by performing a random walk along the circumference of the wheel. Every permutation starts with an equal probability of being chosen. Once it has received a reward, it adds it to its score. Algorithm 4.2 Selecting a Parameter Set in Policy Learning Input. A set of all possible parameter set permutations A, a set of scores R for each value in A initially set to, a learning rate L : if random value v [, ) < L then 2: return a random parameter set, a A 3: else 4: Select a random velocity, v, such that: v < ( A + R) 5: if v < A then 6: return a random parameter set, a A 7: else 8: i 9: while v > and r i R do : v vr i : i i + 2: end while 3: return a i 4: end if 5: end if Output. a parameter set, a A As we add scores, the probability of selecting a particular parameter set changes. Therefore, at a given stage in Algorithm 4.2, the probability of selecting the parameter set a i is given by: ( n ) P i = ( + R i) / R i, where n = R (4) 5 Results 5. Parameter Study To test the effect the parameters had on the resulting nodes, we generated maps for proteins G, BDD, and 2CRS, each with 4 nodes. We ran the tests by randomly selecting parameter set from the value sets, {,,,,, } for P flex and P rigid and {.27,.55,.83,.38,.277,.555,.} for θ std in radians. We collected the potential energy, number of native contacts, and whether it was energy feasible. By plotting average rate of energy feasible nodes generated per attempt for every parameter set combination we attain Figure 2. In this figure, the spacial dimensions (x, y,z) correspond to the value of the parameter in the set (P flex, P rigid, θ std ). We use a fourth dimension of color to represent the energy feasibility rate, where a dark color represents a high energy feasibility rate, since a smaller proportion of nodes are discarded, and a lighter color represents low energy feasibility, thus a greater proportion is discarded. In Figure 3, we plot the average number of native contacts that each parameter combination produces. Both Figure 2 and 3 were generated from the same test on protein G (PDB ID: GB) using 4, nodes. If the parameter values varied independently from each other, we would see that the colors along that slice of the 3 dimensional space would remain consistent. In Figure 2, we see that the color varies dramatically, this demonstrates that they are greatly dependent on each other. Although the parameter values behave similar to our expectations, specifying the right parameters is more difficult than choosing the values independent of each other.

7 Proportion of Energy Feasible Nodes stdangle prigid pflex Figure 2: Effect of the parameters P rigid, P flex, and θ std on the rate of energy feasible generated for Protein G (PDB ID GB). Average Native Contacts.75.7 stdangle prigid pflex Figure 3: Effect of the parameters P rigid, P flex, and θ std on the average number of native contacts of the generated nodes for Protein G (PDB ID GB). 5.2 Policy Learning After implementing the policy learning, we examined the number of times a parameter set was chosen. We used a set {,,, } for the P flex and P rigid values, a subset of the values from the previous tests. These tests were run with R max =, R partial = 5, and R penalty = -.2. We used a learning rate L of.5. After filling half of the bins, the scores are reset: R. However, in these tests, we used different angle values for flexible and rigid regions, θ flex and θ rigid. For Figure 4, we converted these θ values to a single angle value θ std = P flex θ flex +P rigid θ rigid. We would expect that the parameter values would match the energy feasibility from Figure 2, with the added prospect of finding diverse conformations. MDP policy learning has been previously used to impact PRM sampling. The results in Figure 4 show that the algorithm was learning to use parameter sets for reasons other than just energy feasibility. By examining the color at P flex =, P rigid =, and the θ std is greatest, we see that this parameter set was chosen many times, despite having a lower energy feasibility, this is caused by the parameter set being rewarded for generating diverse nodes.

8 Number of Selections stdangle prigid pflex Figure 4: The number of times the parameter set of {P rigid, P flex, θ std } was chosen using policy learning with a learning rate of.5 on Protein G (PDB ID GB). 6 Conclusions and Future Work The preliminary tests showed that the outcome of selecting a particular parameter value depended strongly on the other parameter values selected (Figure 2). This makes the problem of specifying good parameter values before running each simulation difficult. In order to alleviate the researcher from having to specify only good parameter values, we can automatically learn which of the values specified are performing well, and bias the selection process in favor of those values. The policy learning method dynamically adapts to the outcomes of the node generation process. By examining the values that were selected, we can find what parameter sets the algorithm found to work best at that time. These values are not always obvious and can vary depending on which conformations have been generated. By specifying a wide range of values, policy learning can hone in on the useful ones. Since we have to specify a learning rate along with our parameter, we add another parameter to specify. For future work, we could study the effect the learning rate has for different sized and shaped proteins. We could also test other learning algorithms. However, policy learning is quick and simple method for choosing actions, and we have shown that it can be used to ease the process of selecting good parameter values. We can also test this algorithm on larger proteins to see if it is capable of adapting and generating better quality maps in shorter amounts of time. With enough improvement, we could hopefully study larger proteins to find folding pathways for proteopathic diseases, such as Alzhiemer s or Mad Cow disease. We have a publicly available protein folding and motion server at: Given a single protein conformation/pdb file, our server will simulate the folding behavior and provide several different views of extracted folding pathways. Given multiple protein conformations/pdb files, our server will simulate the transition pathways between the different input states. We invite the community to help enrich our web-based motion database by submitting PDB files for study. References [] N. M. Amato, O. B. Bayazit, L. K. Dale, C. V. Jones, and D. Vallejo. OBPRM: An obstacle-based PRM for 3D workspaces. In Robotics: The Algorithmic Perspective, pages 55 68, Natick, MA, 998. A.K. Peters. Proc. Third Workshop on Algorithmic Foundations of Robotics (WAFR), Houston, TX, 998. [2] N. M. Amato, K. A. Dill, and G. Song. Using motion planning to map protein folding landscapes and analyze folding kinetics of known native structures. J. Comput. Biol., (3-4): , 23. Special issue of Int. Conf. Comput. Molecular Biology (RECOMB) 22. [3] R. Bohlin and L. E. Kavraki. Path planning using Lazy PRM. In Proc. IEEE Int. Conf. Robot. Autom. (ICRA), pages , 2. [4] V. Boor, M. H. Overmars, and A. F. van der Stappen. The Gaussian sampling strategy for probabilistic roadmap planners. In Proc. IEEE Int. Conf. Robot. Autom. (ICRA), volume 2, pages 8 23, May 999.

9 [5] F. Chiti and C. Dobson. Protein misfolding, functional amyloid, and human disease. Annu. Rev. Biochem., 75: , 26. [6] H. Choset, K. M. Lynch, S. Hutchinson, G. A. Kantor, W. Burgard, L. E. Kavraki, and S. Thrun. Principles of Robot Motion: Theory, Algorithms, and Implementations. MIT Press, Cambridge, MA, June 25. [7] R. Geraerts and M. H. Overmars. Reachablility-based analysis for probabilistic roadmap planners. Robotics and Autonomous Systems, 55: , 27. [8] B. M. Hespenheide, A. Rader, M. Thorpe, and L. A. Kuhn. Identifying protein folding cores from the evolution of flexible regious during unfolding. J. Mol. Gra. Model., 2:95 27, 22. [9] D. Hsu, G. Sánchez-Ante, and Z. Sun. Hybrid PRM sampling with a cost-sensitive adaptive strategy. In Proc. IEEE Int. Conf. Robot. Autom. (ICRA), pages , 25. [] D. Jacobs. Generic rigidity in three-dimensional bond-bending networks. J. Phys. A: Math. Gen., 3: , 998. [] D. Jacobs and M. Thorpe. Generic rigidity percolation: The pebble game. Phys. Rev. Lett., 75(22):45 454, 995. [2] D. J. Jacobs, A. Rader, L. A. Kuhn, and M. Thorpe. Protein flexiblility predictions using graph theory. Proteins Struct. Funct. Genet., 44:5 65, 2. [3] L. E. Kavraki, P. Švestka, J. C. Latombe, and M. H. Overmars. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. Robot. Automat., 2(4):566 58, August 996. [4] S. M. LaValle and J. J. Kuffner. Randomized kinodynamic planning. In Proc. IEEE Int. Conf. Robot. Autom. (ICRA), pages , 999. [5] M. Lei, M. I. Zavodszky, L. A. Kuhn, and M. F. Thorpe. Sampling protein conformations and pathways. J. Comput. Chem., 25:33 48, 24. [6] M. Levitt. Protein folding by restrained energy minimization and molecular dynamics. J. Mol. Biol., 7: , 983. [7] C. L. Nielsen and L. E. Kavraki. A two level fuzzy PRM for manipulation planning. Technical Report TR2-365, Computer Science, Rice University, Houston, TX, 2. [8] A. Rader, B. M. Hespenheide, L. A. Kuhn, and M. Thorpe. Protein unfolding: Rigidity lost. Proc. Natl. Acad. Sci. USA, 99(6): , 22. [9] M. J. Sternberg. Protein Structure Prediction. OIRL Press at Oxford University Press, 996. [2] X. Tang, S. Thomas, L. Tapia, and N. M. Amato. Tools for simulating and analyzing RNA folding kinetics. In Proc. Int. Conf. Comput. Molecular Biology (RECOMB), pages , 27. [2] S. Thomas, X. Tang, L. Tapia, and N. M. Amato. Simulating protein motions with rigidity analysis. In Proc. Int. Conf. Comput. Molecular Biology (RECOMB), pages , 26. [22] W. Whiteley. Some matroids from discrete applied geometry. Contemp. Math., 97:7 3, 996.