Reconstruction of Time Series using Optimal Ordering of ICA Components

Transcription

1 Reconstruction of Tie Series using Optial Ordering of ICA Coponents Ar Goneid and Abear Kael Departent of Coputer Science & Engineering, The Aerican University in Cairo, Cairo, Egypt e-ail: Abstract We investigate the application of Independent Coponent Analysis (ICA) and the process of optial ordering of independent coponents in reconstructing tie series generated by ixed independent sources. We use a odified fast neural learning ICA algorith with a non-linearity dependent on the statistical properties of the observed tie series to obtain independent coponents (IC s). Experiental results are presented on the reconstruction of both artificial tie series and actual tie series of currency exchange rates using different error easures. The area of the error profile is introduced as a iniizing paraeter to obtain optial ordered lists of IC s for the different series. We copare different error easures and different algoriths for deterining optial ordering lists. Our results support the use of an Euclidean error easure for evaluating reconstruction errors and are in favor of a ethod for obtaining optial ordering lists based on iniizing the error profile between contributions of independent coponents in the lists and the observed tie series. For the ajority of the series considered, we find that quite acceptable reconstructions can be obtained with only the first few doinant IC s in the lists. Keywords: Independent coponent analysis; Independent coponent ordering; Data Reconstruction ***** I. INTRODUCTION Independent Coponent Analysis (ICA) has been considered as a powerful tool in generating independent features in probles where sets of observed tie series ay be considered as results of linearly ixed instantaneous source signals [1,2]. It has been successfully applied in a wide variety of probles (see survey in [3]). In the application of ICA in tie series analysis, there is great value in the reconstruction of observed data for trend discovery and forecasting [4,5]. The process requires the optial ordering of IC s for the reconstruction process. Although this proble had soe attention in the literature, the ethods suggested vary considerably. For exaple, the coponents are sorted according to their non-gaussianity [6], or by selecting a subset of the coponents based on the utual inforation between the observations and the individual coponents [7]. Also, in the work [8], the L nor of each individual coponent is used to decide on the coponent ordering where the order is deterined based on each individual coponent only. More recently, coponent ordering is suggested to be based on coponent power [9]. On the other hand, the works in [4] and [10] consider the joint contributions of IC s in data reconstruction, which naturally leads the coponent ordering to a typical cobinatorial optiization proble. In the present paper, we investigate the process of reconstruction of observed tie series using ICA. We use a odified fast neural learning ICA algorith with a nonlinearity dependent on the statistical properties of the observed tie series. Experiental results are presented on the reconstruction of both artificial tie series and actual tie series of currency exchange rates using different error easures. The area of the error profile is introduced as a iniizing paraeter to obtain optial ordered lists of IC s for the different series. We copare different error easures and different algoriths for deterining optial ordering lists. The paper is organized as follows: section II introduces the ICA of tie series and the odified fast ICA algorith used in the present work; section III describes the process of reconstruction of observed tie series; section IV presents the different ethods for the deterination of optial ordering lists; sections V and VI give results of experientation; and finally section VII presents the suary and conclusion of our work. A. The ICA Model II. ICA OF TIME SERIES Consider the observed k tie series X = x(t) =[x 1 (t), x k (t)] T, 1 t N to be the instantaneous linear ixture of unknown statistically independent coponents Y = y(t) =[y 1 (t),..y k (t)] T, i.e., X = A Y, where A is an unknown k x k nonsingular ixing atrix. Given X, the basic ICA proble is to find an estiate y of Y and the ixing atrix A such that y = W X = W A Y = G Y Y, where W = A -1 is the unixing atrix, and G = W A is usually called the Global Transfer Function or Global Separating-Mixing (GSM) Matrix. The linear apping W is such that the unixed signals y are statistically independent. However, the sources are recovered only up to scaling and perutation. In practice, the estiate of the unixing atrix W is not exactly the inverse of the ixing atrix A. Hence, the departure of G fro the 297

2 identity atrix I can be a easure of the error in achieving coplete separation of sources. B. The Neural Learning ICA Algorith For coputing the independent coponents (IC s) fro the observed tie series, we have adopted the odified algorith used before in [10], which is based on the Fast ICA algorith originally given by [11]. Basically, the algorith uses a fixedpoint iteration ethod to axiize the negentropy using a Newton iteration ethod. We assue that the observation atrix X of k tie series and N saples has been preprocessed by centering followed by whitening or sphering to reove correlations. Centering reoves eans via the transforation X X-E{X} and whitening is done using a linear transfor (PCA like) Z = VX, where V is a whitening atrix. A popular whitening atrix is V = D -1/2 E T, where E and D are the eigenvector and eigenvalue atrices of the covariance atrix of X, respectively. The resulting new atrix Z is therefore characterized by E{ZZ T } = I and E{Z} = 0. After obtaining the unixing atrix W fro whitened data, the total unixing atrix is then W W V. The algorith estiates several or all coponents in parallel using syetric orthogonalization by setting W (W W T ) -1/2 W in each iteration. In the odified version [10] of the algorith, the perforance during the iterative learning process is easured using the atrix G = W A, which is supposed to converge to a perutation of the scaled identity atrix at coplete separation of the IC s. This is done by decoposing G = Q P, where P is a positive definite stretching atrix and Q is an orthogonal rotational atrix. The cosine of the rotation angle is to be found on the diagonal of Q so that a convergence criterion is taken as Δ diag(q) in < ε, where ε is a threshold value. This leads to the noralized perforance (error) easure, E3 introduced in [10] as: e3 E3 {0,1} (1) 2 kk ( 1) with, k k e3 { g M M 1} i1 j1 k k { g M M 1} j1 i1 ij i i ij j j where g ij is the ij th eleent of the atrix G of diensions k x k, M i = ax k g ik is the absolute value of the axiu eleent in row ( i ) and M j = ax k g kj is the corresponding quantity for colun ( j ). The algorith is suarized in the following steps: Preprocess observation atrix X to get Z Choose rando initial orthonoral vectors w i to for intial W and rando A Set W old W Iterate: (2) 1. Do Syetric orthogonalization of W by setting W (W W T ) -1/2 W 2. Copute dewhitened atrix A and new G = W A and do polar decoposition of G = Q P 3. Copute error E3 4. If not the first iteration, test for convergence: Δ diag(q) in < ε 5. If converged, break. 6. Set W old W 7. For each coponent w i of W, update using learning rule w i E{z f (w i z)} E{ f (w i z)} w i xˆ ˆ L x (, ) (,, ) i i Li t u i s t s1 After convergence, dewhiten using W W V Copute independent coponents y = W X In the update step, z is a colun vector of one saple fro Z, and the expectation is the average over the N saples in Z. C. Choice of Non-linearity In the above neural learning algorith, a non-linearity f(y) (and its derivative f (y)) is essential in the optiization process and for the learning rule that updates the estiates of the unixing atrix W and, overall, it is iportant for the stability and robustness of the convergence process. It is coon to use non-linearities f (x) that are derived fro assued source odels such that f x = ( p(x)/ x) / p(x), where p(x) is the PDF of the source. For super-gaussian sources, the PDF p(x) = 1/cosh(x) leads to the general purpose nonlinearity (,,, ) exp{ b b x a b C x } C, a f (x) = tanh(x). For sub-gaussian sources, it was shown in [12] that the PDF can be represented by a biodal Exponential Power Distibution (EPD) syetric ixture density p( x, a, b, ) (1/ 2){ ( x, a, b, ) ( x, a, b, )}, 0 where, (4) The function (4) is the single EPD with scale paraeter (a), a shape paraeter (b) and location paraeter μ. The above syetric density leads to a non-linearity p( x) / x f( x) (5) px ( ) where, b1 η ρ b x μ sign (x μ), b (3) 1 (6) Considering the stability profiles of such ixture odel, the work in [12] has given for sub-gaussian sources the following set of paraeters for the non-linearity given by equation (3): 298

3 a = 3, b = 2, = 3 The error profile area is therefore a sensitive easure of IC s order given by the list L i. Using such area, an optial ordering list ay be obtained as: III. RECONSTRUCTION OF OBSERVED TIME SERIES USING opt L arg in ( ( )) INDEPENDENT COMPONENTS i L A L (12) i i A. Contribution of Independent Coponents to Observed Tie Series Assue we have a list L i of independent coponents indices expressing a specific coponent ordering. The process of reconstructing tie series x i fro the estiated independent coponents Y j, j = 1 k can be done by suing their contributions in the order given by the list L i. Following [4], [10], the contribution ay be expressed by the 3-D space: u i j t W i j Y t j k 1 (,, ) (, ) j ( ), 1 where W -1 (i,j) is the (i,j)th eleent in the inverse of the W atrix. The reconstructed series of x i by the first independent coponents in the list L i is given by the su of the contributions of the individual coponent, i.e., (8) where (s) denotes the s th eleent of L i.. B. Reconstruction Error Profile The reconstruction error for x i can be coputed under a certain error easure (e.g. Euclidean Distance or MSE, Average Mutual Inforation (AMI), etc). At a given tie (t) in the tie series x i, this error ay be denoted by the quantity q( x ( t), xˆ ( L, t )). For the whole series, the average is given by: i i i Q( ) Q( x, xˆ ( L )) aver q( x ( t), xˆ ( L, t)) The dependence of the above quantity on represents the error profile for the reconstruction of series x i using list L i. Naturally, it will also depend on the error easure used. C. Error Profile Area i i i i i i 1tN The error profile Q() depends on the list L i and the used error easure. For coparison purposes, we introduce the error profile area as a collective error easure for a given profile. Fro observations, we find that typical error profiles exhibit an approxiate exponential decrease with increasing. If we express the error as Q = C exp, > 0, then the area under the error profile will be A( L ) Q( ) d ( C / )exp( ) i i i 1 In practice, we copute the area as k1 1 A( Li ) { Q( ) Q( 1) in( Q( ), Q( 1)} 2 1 (7) (9) (10) (11) IV. DETERMINATION OF OPTIMAL ORDERING LISTS We investigate five different ethods for deterining the optial list for the reconstruction of an observed series x i. These ethods are suarized as follows: A. Exhaustive Search Method (ES) A list L i of k indices of IC s has k! perutations. Exhaustive search will exaine the error profile for each of these perutations for a given observed series x i. Based on a given error easure, the resulting k! error profile areas A(L i ) opt are then used to find L i as given by (12). This ethod involves k! reconstruction steps. B. The L Nor Method (LN) The contribution of independent coponent (j) to the reconstruction of observed series x i is given by the quantity u(i,j,t). In this ethod, the L nor of each of these individual coponents is used to decide on the coponent ordering. The optial list represents the coponent indices sorted in descending order of their L nor. C. Exclude the Least Contributing IC First (EL) This is basically the Testing-and Acceptance (TnA) ethod given by [4]. The ethod first selects fro the set of k IC s the coponent that when excluded fro the list will iniize the reconstruction error. This coponent is then reoved fro the set of IC s and its index becoes the last in the order list L i. The process is repeated on what reains in the coponent set to select the second-last in the order list, and so on. The algorith for this ethod is given in the following: Algorith Exclude-Least-First L i = { } is an epty list; Z = set of indices 1: k; r = 1 is the reverse order of the least contributing IC; While (r k) For all indices j reaining in Z Find the su of contributions excluding coponent (j), C ( t) u( i,, t) ij Z, j Find the error ij of reconstructing x i using C ij ; End For α = index (j) for which ij is iniu = index of IC of least contribution; L i (r) α; Reove α fro set Z; r r +1; End While 299

4 L i Reverse (L i ); V. EXPERIMENTS WITH ARTIFICIALLY GENERATED This algorith will involve k(k + 1)/2 1 reconstruction steps. D. Maxiizing the Average Mutual Inforation (AMI) Average utual inforation (AMI) easures the dependence between pairs of rando variables. It can be used in selecting a subset of the coponents based on the utual inforation between the observations and the individual coponents [7]. In the present work, an optial list ay be obtained by sorting in ascending order the AMI between individual contributions u(i,j,t) and the observed series x i. This ethod involves only k reconstruction steps. DATA A. ICA of Artificially Generated Tie Series In order to validate the present ethodology and to copare between different ethods for deterining reconstruction lists, we have used k = 6 artificially generated tie series, each of length N = The generated series S were ixed by a rando ixing atrix A to obtain the siulated observed series X = A S. Fig. 2 shows portions of the source signals S, the ixed signals X and the IC s obtained fro the fast ICA algorith given in Section II.B. For choosing the appropriate non-linearity, we have coputed the noralized kurtosis K for the ixed series X as given by K= E{x 4 } / (E{x 2 }) 2 3. with the results shown in Table (1). Figure 1. Dependence of AMI on for bivariate Gaussian series (x,y) Several algoriths exist for coputing the AMI between two series. [13]. A fast ipleentation given in [14] will be used here. In order to test the AMI algorith, it is well known [e.g. 15] that if (x,y) are bivariate noral, then the AMI between x and y depends only on the correlation coefficient between the. Specifically in this case, AMI x y 2 (, ) (1/ 2)log 2(1 ) We have generated two rando Gaussian series x and y with a correlation coefficient between the in the range in steps of The AMI obtained fro the algorith is copared with the theoretical values as shown in Fig. 1 for series length of N = The validity of the algorith is evident fro the coparison of coputed and theoretical AMI values. E. Miniizing the Reconstruction Error for Individual IC s Contribution (ME) Given a certain error easure, the list is obtained by sorting in ascending order the error between individual contributions u(i,j,t) and the observed series x i. This ethod involves only k reconstruction steps. Figure 2. Artificial source signals, ixed series and obtained IC s TABLE 1. Kurtosis of ixed series X Series Kurtosis Table (1) shows that the ixed signals are all supergaussian (K > 0) and a suitable non-linearity would be the general purpose one f (x) = tanh(x). B. Error Measures For evaluating reconstruction errors, we have selected 3 error easures as follows: 1. Relative Haing Distance (RHD) The Q-easure using RHD is given as: Q x i, x Li = RHD x i, x Li = where, = 1 N 1 R i t = sign[x i t + 1 x i (t)], N 1 t=1 [R i t R Li (t)] 2 (13) 300

5 R Li t = sign[x Li t + 1 x Li (t)] 2. Euclidean Distance or Mean Square Error (MSE) The Q-easure using MSE is given as: Q x i, x Li = MSE x i, x Li = 1 N N t=1 [x Li t x i t ] 2 (14) 3. AMI Distance (AMID) The Average Mutual Inforation (AMI) ay be used as an error easure equivalent to the distance between AMI (ax) and AMI (x,y). In the present work, we use AMI Q- easure in the for: Q(x i, x Li ) = AMID x i, x Li = AMI x i, x i AMI( x i, x Li (15) C. Results for Series Reconstruction In order to reconstruct observed series X fro the obtained IC s, we have to first deterine the error easure ost appropriate for such process. We have derived trial ordered lists of IC indices using the ethod of iniizing the MSE and coputed the error profiles for the 3 different error easures (RHD, MSE, and AMID). Fig. 3a shows an exaple for x 1 of such profiles representing the dependence of the reconstruction error Q() on the nuber of contributing IC s () taken in order fro the ordered list. It can be seen fro the figure that the MSE easure gives the desired lowest profile. Such result is also found in the reconstruction of all other series X. This result is also evident fro coputations of the error profile area as shown in Fig. 3b where A(L) is plotted against the indices of the observed series X. Fro these results, it is possible to conclude that the MSE error easure is superior to RHD and AMID for the evaluation of reconstruction error. Figure 3b. Reconstruction error area Turning now to the proble of deterining optial lists for reconstruction, we have ade calculations of the error profiles and error profile areas for the 5 ethods given in Section IV using MSE as the error easure. Fig. 4 shows a coparison of the different ethods for the observed artificial series X. Figure 4. Reconstruction error area for observed series It can be seen fro this figure that the L nor ethod (LN) cannot copete with the other ethods. It is also clear that the three ethods of Exhaustive Search (ES), Exclude Least (EL), and Miniizing MSE (ME) give exactly the sae results and are superior to all other ethods, including axiizing AMI. Table (2) gives an exaple of the optial lists obtained for the reconstruction of observed series x 1 again highlighting the conclusions drawn fro Fig. 4. In particular, the ES, EL, and ME (with MSE easure) ethods give exactly siilar lists. TABLE 2. Optial IC lists for observed series x 1 Figure 3a. Exaple error profiles Method List ES LN EL AMI ME (MSE)

6 On the other hand, the AMI ethod gives slightly different lists resulting in slightly higher reconstruction error, while the lists obtained by the LN ethod cannot be judged to be optial and therefore produce uch higher reconstruction errors. Although tables like Table (2) for the other 5 series are not given here, it can be shown that siilar conclusions apply to all the 6 observed series X. D. Obtained Reconstruction Ordered Lists There is an exact atch between the ordered lists obtained by the 3 ethods of ES, EL and ME using the MSE error easure and all give the sae iniu reconstruction error. Recall that the nuber of reconstruction steps required by these three ethods are: k!, k(k+1)/2 1, and k respectively, where k is the nuber of observed tie series. For our case of k = 6, the nuber of reconstruction steps are 720, 20 and 6, respectively. Hence, we ay consider the ME (MSE) ethod to be of least coplexity. Using this last ethod, we obtain the following set of ordered lists for the 6 artificially generated tie series: L 1 = [ ] L 2 = [ ] L 3 = [ ] L 4 = [ ] L 5 = [ ] L 6 = [ ] Note that exactly the sae sets are also obtained by the Exhaustive Search (ES) and the Exclude Least (EL) ethods. VI. EXPERIMENTS WITH FINANCIAL DATA A. Exchange Rate Tie Series Dataset As a second set of experients, actual tie series of 6 foreign exchange rate series were selected representing USD versus Brazilian Real (BRL), Canadian Dollar (CAD), Danish Krone (DKK), Japanese Yen (JPY), Swedish Krona (SEK), and Swiss Franc (CHF) in the period fro January 4, 2010 till Deceber 31, The dataset size was 6 tie series over 1504 days collected fro different historical exchange rates data sources such as [16, 17, 18]. Fig. 5 shows these tie series. In order to choose the appropriate non-linearity for the fast ICA algorith given in Section II.B., we have coputed the noralized kurtosis K for the above series X with the results shown in Table (3). The observed series represent a ixture between super-gaussian (K > 0) and sub-gaussian signals (K < 0). We found that our fast ICA algorith would converge in fewer nuber of iterations if we use the ixture non-linearity given by (5) rather than the general purpose non-linearity f (x) = tanh(x). Figure 5. Exchange rate tie series TABLE 3. Kurtosis of observed series X Series Kurtosis B. Results for Series Reconstruction Using the IC s obtained fro the fast ICA algorith, we have derived trial ordered lists of IC indices using the ethod of iniizing the MSE and coputed the error profiles for the 3 different error easures (RHD, MSE, and AMID). Fig.6a shows an exaple for x 5 (SEK) of such profiles representing the dependence of the reconstruction error Q() on the nuber of contributing IC s () taken in order fro the ordered list. Figure 6a. Exaple error profiles This result is also evident fro coputations of the error profile area as shown in Fig.6b where A(L) is plotted against the indices of the observed series X. It can be seen fro that figure that the MSE easure also gives the desired lowest profile for the financial series. Such result is also found in the reconstruction of all other series X. 302

7 TABLE 4. Optial IC lists for exchange rate series x 2 (CAD) Method List ES LN EL AMI ME Figure 6b. Reconstruction error area For deterining optial lists for reconstruction, we have ade calculations of the error profiles and error profile areas for the 5 ethods given in Section IV using MSE as the error easure. Fig. 7 shows a coparison of the different ethods for the observed artificial series X C. Results for Ordered Lists and Contributions of IC s Although the ES, EL, and ME ethods give exactly siilar lists, the ME ethod is superior since it involves only k = 6 reconstruction steps. Using this ethod, we obtain the set of ordered lists for the 6 exchange rate tie series X. Table (5) shows the obtained ordered lists. TABLE 5. Obtained ordered lists Series Ordered List BRL CAD DKK JPY SEK CHF Table (6) gives the percentage cuulative contribution of the first IC s fro the lists to the reconstruction of the exchange rate tie series. In this table, the contributions are calculated as 1 -MSE x i, x Li / variance x i = 1- Q(), since the series have zero ean and unit variance. Figure 7. Reconstruction error area for observed Exchange rate series X: Coparison of Methods for obtaining optial lists It can be seen fro Fig. 7 that the three ethods of Exhaustive Search (ES), Exclude Least (EL), and Miniizing MSE (ME) give exactly the sae results and are superior to all other ethods, a result that is in accordance with our findings for artificially generated series. Table (4) gives an exaple of the optial lists obtained for the reconstruction of observed series x 2 (CAD) again highlighting the conclusions drawn fro Fig. 7. In particular, the ES, EL, and ME (with MSE easure) ethods give exactly siilar lists. It can also be seen fro Table (4) that the LN ethod gives slightly different lists resulting in slightly higher reconstruction error, while the lists obtained by the AMI ethod produce uch higher reconstruction errors. Although tables like Table (4) for the other 5 series are not given here, it can be shown that siilar conclusions apply to all the 6 exchange rate series. TABLE 6. Cuulative Contributions of IC s (%) Series =1 =2 =3 =4 =5 =6 BRL CAD DKK JPY SEK CHF The results given in Table (6) indicate that ost of the observed exchange rate series considered here can be reconstructed to an excellent degree using the first 4 IC s in their respective ordered lists (contributions > 97 %). For the ajority of series, quite acceptable reconstructions (> 93 %) can also be obtained with only the first 3 IC s in the lists D. Coparison between Reconstructed and Observed Series As an exaple, Fig. 8 copares the observed CAD/USA series with the reconstructed ones using the first one, the first two and first 3 IC s in their ordered lists. 303

8 Figure 8. Reconstruction of exchange rate tie series X 2 (CAD), Red (Observed), Blue (constructed) Figure 10. Reconstruction of exchange rate tie series X 6 (CHF), Red (Observed), Blue (constructed) It can be seen fro these figures that the reconstruction of observed series can preserve the general trends with one or two doinant IC s and that quite acceptable atching can be realized with only the doinant 3 IC s in the lists. Figure 9. Reconstruction of exchange rate tie series X 5 (SEK), Red (Observed), Blue (constructed) A second exaple is shown in Fig. 9 for the SEK/USD series. Notice that the series copared in these figures have zero ean and unit variance as obtained fro the preprocessing of the data for the ICA algorith. A third exaple is also shown in Fig. 10 for the CHF/USD series. VII. SUMMARY AND CONCLUSIONS We have used Independent Coponent Analysis (ICA) in the process of reconstructing observed tie series that ight have been generated by ixed independent sources. A odified fast ICA algorith is adopted with enhanceents achieved by using non-linearities dependent on the noralized kurtosis of the observed series. We have experiented with both artificially generated series and actual observed series of currency exchange rates. The artificial data were all super-gaussian for which we have used a general purpose non-linearity. The exchange rate data were a ixture between super-gaussian and sub-gaussian signals. For this latter dataset, we found that our fast ICA algorith would converge in fewer iterations if we use a ixture non-linearity derived fro a biodal Exponential Power Distribution (EPD) syetric ixture density. For the reconstruction of the observed series fro the obtained IC s, one has to obtain ordered lists of such coponents using an error easure that would iniize the error between observed and reconstructed series. We have experiented with three different error easures naely, Relative Haing Distance (RHD), Mean Square Error (MSE) and Average Mutual Inforation Distance (AMID). The area of the error profile is introduced as a sensitive easure of IC s order given by a list and to copare between the perforances of the different error easures for a given ordered list. We find that the MSE easure is superior to the other two ethods for all of the series concerned. 304

9 We have also copared five different ethods for International Joint Conference on Neural Networks, vol. 2, deterining the optial lists. These utilize Exhaustive Search pp , 1999 (ES), the L nor (LN), Exclude Least contribution (EL), [8] A.D. Back, and A.S. Weigend, "A first application of independent coponent analysis to extracting structure fro Maxiizing Average Mutual Inforation (AMI) and stock returns, Int. J. Neural Systes, vol. 8(4), pp , Miniizing Mean square Error (MSE). Naturally ES gives 1997 optial ordered lists but it requires k! reconstruction steps [9] A. J. Hendrikse, R.N.J. Veldhuis, and L. J. Spreeuwers, where k is the nuber of IC s used. "Coponent ordering in independent coponent analysis Fro experients with artificial and exchange rate datasets, based on data power", 28th Syposiu on Inforation Theory we find that both EL and MSE algoriths offer excellent in the Benelux, pp , 2007 results since they give exactly the sae ordered lists as the ES [10] A. Kael, A. Goneid, and D. Mokhtar, Ordering of ethod. However, we consider the MSE to be the ost doinant independent coponents in tie series Analysis efficient since it requires only k reconstruction steps copared using fast ICA algorith, Egyptian Coputer Science Journal (ISSN ), vol. 41 2, pp. 1-10, 2017 to k (k+1)/2-1 steps for the EL algorith and k! steps for the ES algorith. [11] A. Hyvärinen, Fast and robust fixed-point algoriths for On the other hand, the LN and AMI algoriths soeties independent coponent analysis, IEEE Trans. on Neural produce lists that are not optial and therefore cannot copete Networks, vol. 10(3), pp , 1999 with the other three algoriths. Here we ight argue that the [12] A. Goneid, A. Kael, and I. Farag, "Generalized Mixture LN ethod does not necessarily consider the whole Models for Blind Source Separation", Egyptian Coputer contribution of a given IC to the reconstruction process. As for Science Journal, (ISSN ), vol. 34-1, pp. 1-14, 2010 the AMI algorith, it is observed that AMI coputations [13] R. Thoas, N. Moses, E. Seple, and A. Strang, An converge to correct results only for long series length [13] and efficient algorith for the coputation of average utual inforation: Validation and ipleentation in Matlab, that AMI values are not very sensitive to correlations in the Journal of Matheatical Psychology, vol. 61, pp , 2014 pairs of series when they are characterized by low correlation [14] Aslak Grinsted, values. For coparing observed exchange rate series with the 0-average-utual-inforation, 2006 reconstructed ones, we have produced results for the percentage [15] C. J. Cellucci, A. M. Albano, and P.E. Rapp, Statistical contribution of cuulative IC s fro the ordered lists obtained validation of utual inforation calculations: Coparison of using the MSE algorith. For the ajority of the series alternative nuerical algoriths, Phys. Rev. E, Vol. 71, pp. considered, we find that quite acceptable reconstructions (> , 2005 %) can be obtained with only the first 3 doinant IC s in the [16] [17] lists. Even the reconstruction with only the first, or with the [18] first and the second IC s, the trends in the observed series are acceptably preserved. REFERENCES [1] A. Hyvärinen, J. Karhunen, and E. Oja, "Independent Coponent Analysis", John Wiley and sons, New York, NT, 2001 [2] A. Hyvärinen, Independent coponent analysis: recent advances, Phil. Trans. R. Soc. A 371: , [3] A. Mansour, and M. Kawaoto, ICA papers classified according to their applications and perforances, IEICE Trans. Fundaentals, vol. E86-A, No. 3, pp , 2003 [4] Cheung, Yiu-ing, and Lei Xu. "Independent coponent ordering in ICA tie series analysis", Neurocoputing, vol. 41.1, pp , 2001 [5] Chi-Jie Lu, Tian-Shyug Lee and Chih-Chou Chiu, Tie series forecasting using independent coponent analysis and support vector regression, Decision Support Systes, vol. 47-2, pp , 2009 [6] A. Hyvärinen, Survey on independent coponent analysis, Neural Coputing Surveys 2, pp , 1999 [7] A. D. Back, and T.P. Trappenberg, Input variable selection using independent coponent analysis, Proceedings of 305