Fuzzy Relational System for Identification of Gene Regulatory Network

Transcription

1 Fuzzy Relational System for Identification of Gene Regulatory Network Papia Das 1, Pratyusha Rakshit, Amit Konar, Mita Nasipuri 1, Atulya K. Nagar 3 1 CSE Dept., ETCE, Jadavpur University, Kolkata, India 3 Department of Math & Computer Science, Liverpool Hope University, Liverpool, UK Abstract- Generating inferences from a gene regulatory network is important to understand the fundamental cellular processes, involving gene functions, and their relations. The availability of time-series gene expression data makes it possible to investigate the gene activities of the whole genomes. Under this framework, gene interaction is explained through a set of fuzzy relational matrices. By transforming quantitative expression values into linguistic terms, the proposed technique defines a measure of fuzzy dependency among genes. Based on the fact that the measured time points are limited, we present an Artificial Bee Colony-based search algorithm to unveil potential genetic network constructions that fit well with the time-series data and explore possible gene interactions. Keywords- gene regulatory network; fuzzy relational system; fuzzy membership distribution; artificial bee colony optimization algorithm; differential evolution algorithm. 1 Introduction Genes in living organisms form a virtual network through interaction with each other. This interaction mechanism is called gene regulatory network (GRN). GRNs form dynamic and distributed systems which control the expressions of the various genes in the cell. They explicitly represent the causality of developmental processes and explain exactly how genomic sequence encodes the regulation of expression of the sets of genes that progressively generate developmental patterns and execute the construction of multiple states of differentiation. The complex control systems underlying development have probably been evolving for more than a billion years. These control systems consist of many thousands of modular DNA sequences. Each such module receives and integrates multiple inputs, in the form of regulatory proteins (activators and repressors) that recognize specific sequences within them. The end result is the precise transcriptional control of the associated genes. Some regulatory modules control the activities of the genes encoding regulatory proteins. Functional linkages between these particular genes, and their associated regulatory modules, define the core networks underlying development. This regulatory mechanism of genes provides an insight into the interaction between different genes. With the rapid advancement of DNA microarray technologies, inferring genetic regulatory networks from timeseries gene expression data has become critically important in revealing fundamental cellular processes, investigating functions of genes and proteins, and understanding complex relations and interactions between genes. Several methods have been proposed to model maps of gene interaction, including Bayesian networks [1], dynamic Bayesian networks with hidden Markov model [], and Boolean networks [3]. More recently, neural networks have also been applied to the problem of gene expression data analysis [4]. Boolean networks have been used to infer underlying GRN structures. In a Boolean network, the state of a gene is represented by a Boolean variable (ON or OFF) and interactions between genes are represented by Boolean functions. Boolean networks require that a number of assumptions be made to simplify analysis. Unfortunately, the validity of these assumptions has been questioned by many researchers, especially those in the biological community. To these researchers, there is a perceived lack of connection between simulation results and empirically testable hypotheses. Instead of Boolean networks, Bayesian networks can also be used for GRN inferences. Bayesian network is a probabilistic model that describes the multivariate probability distribution of a set of genes whose interdependencies are known. A Bayesian network allows the conditional dependencies and independencies to be displayed by means of a directed acyclic graph. However, this approach to the learning of network structures is a NP-hard problem, especially for high-dimensional data such as gene expression data. Another problem that needs to be tackled when using the Bayesian network approaches for gene expression data analysis is concerned with the effect of small sample sizes. A stochastic model of gene interactions capable of handling missing variables is proposed in []. It can be represented as a dynamic Bayesian network particularly well suited to tackle the stochastic nature of gene regulation and gene expression measurement. Parameters of the model are learned through a penalized likelihood maximization technique. The model referred to here is based on several strong assumptions, such as stationary or additive regulation. The model needs farther improvement in order to represent more realistic phenomena, such as non-linear and combinatorial regulations. Currently, with the advancements of the DNA micro array technology, it has become possible to simulate gene regulatory network from gene expression time-series data. In [6], a mathematical model for GRN has been proposed using fuzzy recurrent neural network to determine the numerical interaction values between genes. Due to the large number of model parameters and the small number of data sets available, the system of equations in GRN identification problem is highly underdetermined and ambiguous. GRN weights usually are multimodal functions of the gene expression time series

2 data. Hence, the solution sets of weights are non-unique, and naturally the solution does not guarantee the optimal selection of weights of the network. In this context, it is necessary to propose models that attempt to get good predictions, reducing the need for prior knowledge. For one to infer the structure of a GRN, it is important to identify, for each gene in the GRN, whether other genes can affect its expression and how they can affect it. To better infer GRN structures, we propose a technique which is able to discover interesting fuzzy dependency relationships among genes. It can represent discovered fuzzy dependency relationships explicitly as if a gene is highly expressed, its dependant gene is then lowly expressed etc. These relationships can reveal biologically meaningful gene regulatory relationships that could be used to infer underlying GRN structures. In this work, we present a fuzzy logic based algorithm for analyzing gene expression data, and employ an Artificial Bee Colony (ABC) optimization algorithm [7] to find the optimal membership function of normalized gene responses as well the fuzzy relation between genes. The membership function thus obtained are then defuzzified by centroidal defuzzification technique, and the results are found to be promising. Using fuzzy logic, we have developed a technique to identify logical relationships between genes. The fuzzy logic has proved to be an important tool due to its ability to represent non-linear systems, its friendly language to express knowledge and the ability to incorporate and edit fuzzy rules. It can handle very noisy, high-dimensional time series gene expression data and can represent discovered fuzzy dependency relationships explicitly. These discovered relationships not only make hidden regularities easily interpretable, it also determines if a gene is supposed to be activated or inhibited and can be used to predict how a gene would be affected by other genes from an unseen sample (i.e., expression data that are not in the original database). The proposed technique has been tested with real expression data. The performance of the current work is significantly better than the one reported in [6] considering root mean square error and convergence speed of the procedure. ABC seems to be promising for this optimization problem because of the following reasons: 1) providing better solution quality to find out fuzzy membership distribution of relation between genes in GRN, ) combining local search methods with global search methods attempting to balance exploration and exploitation processes giving high speed of convergence, and 3) preventing the search technique from premature convergence problem providing global search ability with the help of scout unit. The paper is organized as follows. First, the conventional concept of fuzzy sets and relations is described briefly in section. In section 3, we describe the fuzzy relational approach to solve GRN identification problem. The cost function used to determine the quality of a solution is proposed in section 4. In section 5, we describe the ABC optimization algorithm used to find the relational matrices between genes in the network and we explain the fuzzy technique to represent the membership values of gene response in section 6. In section 7, we present the simulated results and in section 8, we demonstrate the use of our model to simulate a gene regulatory network using real gene expression time series data. Section 9 concludes the paper. An Overview of Fuzzy Sets and Relations.1 Definition 1 A fuzzy set A is a set of ordered pairs, given by A {( x, A( x)) : x X} (1) where X is a universal set of objects (also called the universe of discourse) and µ A (x) is the grade of membership of the object x in A. Usually, µ A (x) lies in the closed interval of [0,1].. Definition A membership function µ A (x) is characterized by the following mapping: ( x) : x [0,1 ], x X () A where x is a real number describing an object or its attribute, X is the universe of discourse and A is a subset of X..3 Definition 3 A fuzzy relation is a fuzzy set defined in the Cartesian product of crisp sets X 1, X,, X n. A fuzzy relation R(x 1, x,.., x n ) thus is defined as R x1, x,.., xn { R ( x1, x,.., xn) /( x1, x ( x, x,.., x ) X X... X } 1 n 1 n,.., x ) where X X... X [0,1]. (3) R : 1 n In binary fuzzy relation instead of n universes we need only universes..4 Definition 4 A fuzzy implication relation for a given rule: IF x is A i THEN y is B i is formally denoted by Ri( x, y) { R ( x, y) /( x, y)} i (4) where the membership function µ Ri (x, y) is constructed intuitively by many alternative ways. Here we have used Mamdani Implication. Mamdani proposed the following implication function: R ( x, y) min[ A ( x), B ( x)] i i i (5).5 Definition 5 Let us consider two fuzzy relations R 1 and R defined on X Y and Y Z respectively. The max-min composition of R 1 and R is a fuzzy set defined by R3 R1o R (6) { R ( x, z)/( x, z)} 3 where ( x, z) ( x, y), ( y, z)) x X, y Y, z Z}. R3 max{min( R y 1 R.6 Definition 6 Let us consider a fuzzy production rule: IF x is A THEN y is B, and a fuzzy fact: x is A /.The Generalized Modus Ponens (GMP) inference rule then infers y is B /. Here A, B, A /, and B / are fuzzy sets such that A / is close to A, and B / is close to B. The inference rule also states that the closer the A / to A, the closer the B / to B. Symbolically, the GMP can be stated as follows: Given: IF x is A THEN y is B. Given: X is A /. Inferred: y is B /. For evaluation of membership distribution of y is B /, µ B (y), we need to know the membership distribution of x is n

3 A /, µ A (x), and the membership of the fuzzy relation for the given IF-THEN rule, µ R (x, y). According to GMP / ( y) / ( x) o ( x, y) (7) B A R where µ A (x) and µ R (x, y) are row vector and matrices of compatible dimension respectively. 3 Solving the GRN Identification Problem by Fuzzy Relational Approach To describe the proposed technique, let us assume that we are given a set of gene expression time series data G={G 1,, G j,, G N }, consisting of N time series collected from experiments with N genes. Each of these N time series consists, in turn, of T data points collected at T different time instances. Here we have considered that the response value of gene g j at time instance t, G j (t) has a fuzzy membership distribution µ A (G j (t)), and the corresponding fuzzy set A is given by the doublet (G k j (t) µ A (G k j (t)), where jϵ[1, N] and kϵ[1,f]. The G j (t) is evaluated by centroidal defuzzification procedure given by G j( t ) F k j G ( t ) A ( G (t )) k 1 (8) F A ( k 1 G k j k j (t )) As an example let F=5; so that a particular gene expression at time instance t can be represented as { , , , , }, and after the de-fuzzification it becomes( )/( )= At this point, we want the attention of the reader on the above fuzzy set A; the members of fuzzy set A are 0., 0.4, 0.6, 0.8, and 1.0. Now, gene expression can be described in two different states such as highly expressed and lowly expressed to a varying degree based on a set of membership functions. For our application here, we define two different states, highly expressed and lowly expressed in terms of two fuzzy sets as shown in Figure 1. In our proposed work, we are considering two fuzzy sets A 1 = [0.1, 0.4] and A = [0.5, 1.0]. Here µ A1 (G j (t)) in fuzzy set A 1 indicates the degree of membership of G j (t) to be low and µ A (G j (t)) in fuzzy set A indicates the degree of membership of G j (t) to be high. Let A=A 1 UA. Hence, gene expression is considered to be low with a high membership value of gene response within a range of 0.1 to 0.4 and otherwise gene expression is considered to be high. From the membership distribution of µ A1 (G i (t=0)), µ A (G i (t=0)), µ A1 (G j (t=0)) and µ A (G j (t=0)) we can construct 4 fuzzy relational matrices for each pair of gene responses G i (t) and G j (t), i, jϵ[1, N] following Mamdani rule of Fuzzy implication. Fuzzy Membership Values of Gene Expression low high Normalized Gene Expression Values Figure 1. Fuzzy membership distribution of gene expression The descriptions of four relational matrices are given as follows. 1) Ri _low, j _ low ( k,l ) Min( ( k G i ( t )), ( l A1 0 A1 G j (t 0 ))) ) Ri _low, j _ high ( k,l ) Min( k (G i (t )), l A1 0 A(G j ( t 0 ))) 3) Ri _ high, j _ low ( k,l ) Min( k (G i ( t )), ( l A 0 A1 G j (t 0 ))) 4) Ri _ high, j _ high ( k,l ) Min( k (G i ( t )), l A 0 A(G j (t 0 ))) k,l [1,F]. The corresponding fuzzy production rules are given as follows. PR1: IF g i s response is low THEN g j s response is low. PR: IF g i s response is low THEN g j s response is high. PR3: IF g i s response is high THEN g j s response is low. PR4: IF g i s response is high THEN g j s response is high. Now, the entire fuzzy relational matrix between response of genes g i and g j is given by R i,j which is formed using 4 relational sub-matrices. R i,j = R i_low,j_low R i_high,j_low R i_low,j_high R i_high,j_high Hence there will be such N N relational matrices each of dimension F F. Now our objective is to determine the membership distribution of gene g i at next time instance t+1. Let this is denoted as µ A (G i (t+1)). Once the relational matrix R i,j has been formed between two genes g i and g j, we can evaluate µ A (G i (t+1)) by max-min composition between R i,j and µ A (G j (t)), for i, j[1,n], as given by GMP inference rule N µ A (G i (t+1))=max[µ A (G j (t))or j,i ], i,j [1,N] (10) j=1 where µ(g j (t))or j,i =max[min{ µ (G j k (t), R j,i (k,l)}] (11) k F (9)

4 4 Proposed Cost Function The proposed cost function in this work is designed keeping in mind the main issue of accurately identifying the existing relationship between genes in the network.handling this issue is a tough job, since we do not have any knowledge except the available gene espression time series data. Therefore, a judicious choice of cost function can greatly influence the accuracy of the simulated network. To meet this issue, we evaluate the accuracy of the produced gene expression of our simulated network obtained using the fuzzy relational system by comparing it with the original gene expression with the hope that if the fuzzy relational matrices correctly identify the logical relationships between two genes then the difference (error) between the two set of gene expressions will be less. The error has been calculated by taking the squared difference between original gene expression, Gi_org(t), and experimental gene expression, Gi_cal(t), given by 1 T N cos t _ fn ( Gi _ org ( t) Gi _ cal ( t)) (1) N T t1i 1 5 Artificial Bee Colony Optimization algorithm (ABC) In ABC algorithm, the colony of artificial bees contains three groups of bees: Onlooker bee makes decision to choose a food source. Employed bee selects a food source. Scout bee carries out random search for food source. Here, the position of a food source represents a possible solution of the optimization problem and the nectar amount of a food source corresponds to the fitness of the associated solution. The number of employed bees and onlooker bees is equal to the number of solutions in the population. ABC consists of following steps: 5.1 Initialization ABC generates a randomly distributed initial population P (G=0) of Np solutions (food source positions). Each solution X i (i=0, 1,,, Np -1) is a D dimensional vector. 5. Placement of employed bees on the food sources An employed bee produces a modification on the position in her memory depending on the local information (visual information) as stated by equation (14) and tests the nectar amount of the new source. Provided that the nectar amount of the new one is higher than that of the previous one, the bee memorizes the new position and forgets the old one. Otherwise she keeps the position of the previous one in her memory. 5.3 Placement of onlooker bees on the food sources An onlooker bee evaluates the nectar information from all employed bees and chooses a food source depending on the probability value associated with that food source, p i, calculated by the following expression: fit i (13) p i 1 Np j0 fit j where fit i is the fitness value of the solution i evaluated by its employed bee. After that, as in case of employed bee, onlooker bee produces a modification on the position and checks the nectar amount of the candidate source. Onlooker bee memorizes the better position only. In order to find a solution X / i in the neighborhood of X i, a solution parameter j and another solution X k are selected on random basis. Except for the value of chosen parameter j, all other parameter values of X / i are same as in the solution X i, for example, X / i =( x i0, x i1,, x i(j-1), x ij, x i(j+1),, x i(d-1) ). The value / / of x ij parameter in X i solution is computed using the following expression: x ij = x ij +u(x ij- x kj ) (14) where u is a uniform variable in [-1, 1] and k is any number between 0 to Np-1 but not equal to i. 5.4 Send scouts for discovering the new food sources In the ABC algorithm, if a position cannot be improved further through a predefined number of cycles called limit, the food source is abandoned. This abandoned food source is replaced by the scouts by randomly producing a position. After that again steps (B), (C) and (D) will be repeated until the stopping criteria is met. 6 Extraction of Fuzzy Relationship between Genes Using ABC In our paper, we have used the well known Artificial Bee Colony (ABC) optimization algorithm to find the simulated network. To spread the initial candidate solutions as far possible in the search space with the hope that some of the solutions may be close to the original solution we have used a chaos system [5] in ABC. The process of producing the chaos is as follows: Z k+1 =µz k (1-Z k ) (15) where k = 0, 1,, 3 Θ, Θ is the number of chaotic iteration, µ is the control parameter. Z k takes any value between 0 and 1; it is the selected value in the kth iteration. We indeed found that this initialization improve the overall convergence rate of the artificial bee colony optimization algorithm. Each individual food source of ABC represents a complete solution. As an example one solution of the N=4 gene network contains N F=4F data points where F is the number of elements in each of the N=4 fuzzy sets. These sets represent the membership values of gene responses in the network. We maintain a pop_size number of individual food sources all the time in the population pool. The population pool of the ABC optimization algorithm for the four gene network with F=10 can be represented pictorially as a two dimensional matrix as shown in Figure. In Figure, F=10 and µ A (G i k ) represents the fuzzy membership values of the gene expression G i of any individual food source, k=1,,, 10 with the Fuzzy members as {0.1, 0., 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}. At each step of ABC we evaluate fuzzy membership distribution of gene response, de-fuzzify each membership, calculate the cost function, and make the appropriate decision whether to keep that particular food source for the next generation or not.

5 µ A(G 1 1 ) µ A(G 1 ) µ A(G 1 3 )µ A(G 1 4 ) µ A(G 1 5 ) µ A(G 1 6 ) µ A(G 1 7 ) µ A(G 1 8 ) µ A(G 1 9 ) µ A(G 1 10 ) µ A(G 1 ) µ A(G ) µ A(G 3 )µ A(G 4 ) µ A(G 5 ) µ A(G 6 ) µ A(G 7 ) µ A(G 8 ) µ A(G 9 ) µ A(G 10 ) µ (G 3 1 ) µ A(G 3 ) µ A(G 3 3 )µ A(G 3 4 ) µ A(G 3 5 ) µ A(G 3 6 ) µ A(G 3 7 ) µ A(G 3 8 ) µ A(G 3 9 ) µ A(G 3 10 ) µ A(G 4 1 ) µ A(G 4 ) µ A(G 4 3 )µ A(G 4 4 ) µ A(G 4 5 ) µ A(G 4 6 ) µ A(G 4 7 ) µ A(G 4 8 ) µ AG 4 9 ) µ A(G 4 10 ) Figure. Individual solution used in optimization algorithm 7 Simulation Results The gene regulatory network identification problem is implemented in a Pentium processor. The results are generated with 4 time series data, one for each of 4 genes. The experiments are conducted for F=5, 10, and Experiment with Artificial Bee Colony Figure 3 shows 16 fuzzy relational matrices R i,j between responses of genes g i and g j, i, j [1,4] with 1000 iterations for ABC algorithm with limit=100 and 300 iterations for chaotic initialization algorithm. 7. Experiment with Differential Evolution Figure 4 shows 16 fuzzy relational matrices R i,j between responses of genes g i and g j, i, j[1,4] with 1000 iterations for DE algorithm with Cr=0.9 and 300 iterations for chaotic initialization algorithm. 7.3 Results on the time series data Using the relational matrices obtained from ABC- and DEbased simulations, and de-fuzzifying values of gene responses at t=1,,, 150, we obtain the calculated gene expression time-series data. The relative performance of ABC-, DEbased simulations using our approach as well as the fuzzy recurrent neural approach proposed in [6], can be studied through the plot (Figure 5(i)-(iv)). Each plot consists of the gene expression levels at different time instances obtained by our approach (using ABC and DE), work proposed in[6] and the original time series data for a particular gene. Now we compare the derived time series plot with the original gene expression time series data. It is evident from the figures that ABC- based simulation using fuzzy relational system has outperformed the other two approaches. 7.4 Cost function evaluation In order to compare the ability of ABC- and DE- based simulations to provide better solution with less cost function value, we plot the cost function value of the best solution obtained in each iteration of ABC- and DE-based simulations in Figure 6. It is apparent that for a fixed number of iteration ABC provides better solution than DE. g1 response 7.5 Performance analysis To analyze the performance of the proposed approach for identification of gene regulatory network, we measure the following two parameters Root mean square error (RMSE) The performance metric used here to determine how close the estimated gene responses are close to the original values of gene expressions is Root Mean Square Error (RMSE) given as 1 T N RMSE ( Gi _ org ( t) Gi _ cal ( t)) (16) N T t1i 1 Here, T=150 and N=4.We obtain the following results from the plot of time series data in Fig.5. RMSE for ABC-based simulation with fuzzy relational system= 3.039% RMSE for DE-based simulation with fuzzy relational system= % RMSE for DE-based simulation with recurrent fuzzy neural model as in [6] = % 7.5. Run time After carrying out the experiment in a Pentium dual port computer using ABC optimization and DE algorithms, we find out Run_time ABC =59 minutes Run_time DE =3 minutes ABC- based simulation takes more time than DE- based simulation due to complexity involved in ABC. In Table-I, we represent the mean fuzzy relational matrix indicating relationship between expression of genes g and g 1 obtained using ABC-based simulation after 5 runs with F=10. A close inspection of Table-I indicates that membership value of expression of gene g is high (low) when that of gene g 1 is low (high). It indicates that gene g 1 regulates expression of gene g by inhibiting its response. 8 Inferring GRN Using Real Data Set We have used our model to infer the gene regulatory network of e.coli. Bacteria S.O.S DNA repair network consisting of nearly 30 genes regulated at the transcription level. Four experiments have been conducted with different UV light intensities and eight major genes have been documented. These genes are uvrd, lexa, umud, reca, uvra, uvry, ruva, polb. This data set is available in the website [ We have conducted same experiment as with the above artificial data. The identified gene responses are represented in Figure 8. TABLE-I: Fuzzy Relational Matrix between expression of genes g 1 and g g response low high low high

6 Figure 3. Fuzzy relational matrices R i,j, i,jϵ [1,4], obtained from ABC-based simulation Figure 4. Fuzzy relational matrices R i,j, i,jϵ [1,4], obtained from DE-based simulation

7 Figure 5(i). Plot of time series data for gene 1 Figure 5(ii). Plot of time series data for gene Figure 5(iii). Plot of time series data for gene 3 Figure 5(iv). Plot of time series data for gene 4 Figure 6. Minimum cost function value in each iteration of ABC- and DEbased simulation Figure 8. The measured gene expression profile of e. coli. 9 Conclusion In this paper, we have presented an effective fuzzy technique for the discovery of GRNs from time series gene expression data. We design the fuzzy rules according to expressing level of gene, and fuzzy set theory. The proposed technique can discover fuzzy dependency relationships in high-dimensional and very noisy data. Based on the discovered fuzzy dependency relationships, the user can not only determine those genes affecting a target gene but also can identify whether or not the target gene is supposed to be activated or inhibited. The simulation results on both the artificial and the real data demonstrate that the proposed method is very promising in capturing the nonlinear dynamics of genetic regulatory systems and unveiling the potential gene interaction relation. 10 References [1] P. Spirtes, C. Glymour, R. Scheines, S. Kauffmann, V. Aimale and F. Wimberly, Constructing Bayesian Network Models of Gene Expression Network from Microarray Data, Proc. Atlantic Symp. Computational Biology, Genome Information Systems and Technology,000. [] E. Perrin, L. Rolaivola, A. Mazurie, S. Bottani, J. Mallet, and F. D Alche-Buc, Gene Network Inference using Dynamic Bayesian Networks, Bioinformatics, vol.19(): ,003. [3] S. Laing, S. Fuhrman and R. Somogyi, REVEAL, A general reverse engineering algorithm for inference of genetic network architechtures, Proc. Pacific Symp. Biocomputing 3, [4] A Nnarayanan,E.C. Keedwell,J. Gamalielsson and S. Tataneni, Single Layer Artificial Neural Network for gene expression analysis, Proc. Neurocomputing Conf.,vol.61:17-40,004. [5] C. Lng, S.Q. Li, Chaotic spreading sequences with multiple access performance better than random sequences. IEEE transaction on Circuit and System-I, Fundamental Theory and Application, 47(3): , 000. [6] D. Datta, A. Konar, R. Janarthanan, Extraction of interaction information among genes from gene expression time series data, NaBIC 009. [7] B. Basturk, and Dervis Karaboga, An Artificial Bee Colony (ABC) Algorithm for Numeric function Optimization EEE Swarm Intelligence Symposium 006, May 1-14, 006, Indianapolis, Indiana, USA. Figure 7. E.coli S.O.S. DNA repair network, activation is represented by + sign and inhabitation by -