Turbo Compressed Sensing with Partial DFT Sensing Matrix

Transcription

1 Turbo Compressed Sensing with Partial DFT Sensing Matrix Junjie Ma, Xiaojun Yuan, Member, IEEE and Li Ping, Fellow, IEEE arxiv: v2 [cs.it] 9 Sep 204 bstract In this letter, we propose a turbo compressed sensing algorithm with partial discrete Fourier transform DFT sensing matrices. Interestingly, the state evolution of the proposed algorithm is shown to be consistent with that derived using the replica method. umerical results demonstrate that the proposed algorithm outperforms the well-known approximate message passing MP algorithm when a partial DFT sensing matrix is involved. Index Terms Compressed sensing, approximate message passing MP, partial DFT matrix, state evolution, replica method. I. ITRODUCTIO Partial discrete Fourier transform DFT sensing matrices have found many applications [] and an efficient signal recovery algorithm is highly desirable for related compressed sensing problems. pproximate message passing MP [2] [4] is an iterative algorithm for this purpose. The state evolution of MP with independent and identically distributed i.i.d. Gaussian sensing matrices is shown to be consistent with that derived using the replica method [4]. This implies that MP can potentially provide near-optimal performance when i.i.d. Gaussian sensing matrices are involved. However, the situation is different for partial DFT sensing matrices whose entries are not independently drawn. Recent results in [5], [6] pointed out that, using the replica method, the optimal reconstruction performance of a system based on a partial DFT matrix is different from that based on an i.i.d. Gaussian matrix. In this letter, we propose a turbo-type iterative algorithm [7] for the problem. The proposed algorithm involves two local processors. One processor handles the information related to a partial DFT sensing matrix using the linear minimum meansquare error LMMSE principle. The other processor handles the sparsity information. Our main contribution is a novel way to compute extrinsic messages related to the sparsity information. The state evolution of the proposed algorithm coincides with that predicted by the replica method [5], [6]. This indicates the potentially excellent performance of the proposed algorithm, as confirmed by Monte Carlo simulations. II. PROLEM DESCRIPTIO Consider the following linear system y = F partial x + n, J. Ma and Li Ping are with the Department of Electronic Engineering, City University of Hong Kong, Hong Kong, SR, China. junjiema2- c@my.cityu.edu.hk; eeliping@cityu.edu.hk. X. Yuan is with the School of Information Science and Technology, ShanghaiTech University yuanxj@shanghaitech.edu.cn. where x C is a sparse signal to be estimated, y C M the received signal, and n C 0, σ 2 I the Gaussian noise. F partial consists of M randomly selected and reordered rows of the unitary DFT matrix F C, where the m, nth entry of F is given by e 2πjm n / with j =. The entries of the sparse signal x is assumed to be i.i.d., with the jth entry of x following the ernoulli-gaussian distribution [4]: { 0 probability = λ, x j C 0, λ 2 probability = λ. In 2, the variance of each x j is normalized, i.e., E[ x j 2 ] =. The partial DFT matrix in can be rewritten as F partial = SF, 3 where S is a selection matrix consisting of M randomly selected and reordered rows of the identity matrix. Define z = F x. 4 Together with 3, we rewrite the system model in as. Standard Turbo lgorithm y = Sz + n. 5 The proposed algorithm is based on the turbo principle in iterative decoding [7]. efore introducing the proposed solution, we will first discuss a standard algorithm and explain its potential problem. For the problem in, the block diagram of a standard turbo detector is illustrated in Fig. a. It consists of two modules: module is an LMMSE estimator and module a sparsity combiner. The LMMSE estimator produces a coarse extrinsic estimate of x based on the observation y. The sparsity combiner refines the estimate using the sparse distribution in 2. Here, the extrinsic output [7] of a module is fed to the other module as the a priori input. The two modules are executed iteratively until convergence. t the end of the iteration, the final estimate of x is based on the a posteriori output of the sparsity combiner. We next discuss the detailed operations of the algorithm in Fig. a. Module : ssumption : The entries of x are i.i.d. with a priori mean and variance vpri. The a priori information about x is obtained from the feedback of the sparsity combiner, which will be discussed

2 2 a standard turbo detector. Fig.. b The proposed turbo detector. lock diagrams of a standard turbo algorithm and the proposed turbo algorithm. ext represents extrinsic information computation. later. With ssumption, the a priori mean of z = F x is given by = F xpri 6 and the variance is. From 5, the LMMSE estimator and the mean-square error MSE matrix of z are respectively given by [8] = + vpri SH y S, 7a + σ2 V post = I 2 + σ2 SH S. 7b From x = F H z, the LMMSE estimator of x is The associated MSE matrix is = F H. 8 F H V post F. 9 It can be verified that the diagonals of F H V post F, which are the a posteriori MSEs, are identical and given by = M 2 + σ2. 0 Using the concise formulas in [9], [0], the extrinsic LMMSE estimate and the MSE of x can be computed by x ext = v ext v ext = xpri, a. b 2 Module : The LMMSE estimator effectively makes a Gaussian assumption on x and ignores the sparsity information of x. The function of the sparsity combiner is to refine the LMMSE estimate of x by combining the sparsity information in 2. ssumption 2: is modeled as an additive white Gaussian noise WG observation of x, i.e., = x + w, 2 where w C 0, I and is independent of x. Here, xpri and are updated by the extrinsic output of module, i.e. = xext and = vext. 3 ased on ssumption 2, the minimum mean-square error MMSE estimator of x conditioned on is a componentwise operation and given by [ j, = E ] [ ] x j = E x j j,, j, 4 where j, and xpri j, denote the jth entry of xpost and respectively. E[ ] is with respect to the joint distribution of x and characterized by 2. The detailed operations of the above MMSE estimation can be found in, e.g., [6]. The conditional variance corresponding to 4 is given by [ j, = var ] [ ] x j = var x j j,, j, 5 where var[a b] E [ a E[a b] 2 ] b. We next compute the extrinsic estimate of each x j by excluding the contribution of j,. Under ssumption 2, the MMSE estimation in 4 is a component-wise operation. Excluding the contribution of x j, the extrinsic estimate of x j becomes where j, j, x ext j, = E [ x j j,] = E [xj ] = 0, j, 6 is obtained from xpri by excluding the jth entry. The extrinsic estimate of module will be treated as a priori mean for module in the next iteration. The following observations are useful: The LMMSE operation ensures that module in Fig. a is optimal in the LMMSE sense if the sparsity information is ignored and no iteration is involved. ote that MP cannot make such a claim due to the distributive nature of message passing.

3 3 However, from 6, the extrinsic estimate of module is zero and so iterative processing does not provide any further improvement. In what follows, we will develop an alterative processor in Fig. b that maintains the advantage but avoid the disadvantage.. Proposed Turbo Compressed Sensing lgorithm The proposed algorithm is illustrated in Fig. b. Module computes extrinsic information of x and Module computes the extrinsic information of z. This is different from the standard approach in Fig. a where both modules compute extrinsic information of the same variable x. Module : Module includes the LMMSE estimator of z and two IDFTs. The operations of Module are roughly the same as that in Fig. a, except that the input is in Fig. b. 2 Module : s discussed in Section II--2, the sparsity combiner produces no extrinsic estimate of x. In the proposed algorithm, module now computes the extrinsic estimate of z instead of x. ssumption 3: The a posteriori distributions of z conditioned on are Gaussian, i.e. where j, vector and Pr z j = C j,, vpost, j, 7 is the jth entry of the following a posteriori mean = F, 8 is the a posteriori variance given by = j= j,, 9 where j, is the variance of xpost j, in 5. Intuitively, when is large, ssumption 3 can be justified by the mixing effect of the DFT and the central limit theorem. Eqn. 8 is due to z = F x. s the entries of x are a priori independent from ssumption 2 and the sparsity combiner is a component-wise operation, the entries of x are also a posteriori independent, and so 9 follows. From ssumption 2, the a priori estimate = F xpri is an WG observation of z, i.e. = F xpri = z + F w. 20 s w is i.i.d. Gaussian with mean zero and variance, F w has the same distribution and we have Pr j, zj = C j,, vpri, j. 2 From 20, j, and zpri j, given z j. It can then be verified that Pr z j Pr z j j, are conditionally independent Pr j, z j, j 22 With slight abuse of notation, here and also in 2 C m, v denotes a Gaussian function of z j with mean m and variance v. where denotes equality up to a constant scaling factor independent of z j. ased on 7, 2 and 22, the extrinsic distribution Prz j j, is Gaussian [9], [] and given by Pr z j j, = C zj,, ext v ext, j 23 where z ext j, = v ext v ext = z post j, zpri j,, 24a. 24b The extrinsic mean/variance in 24 will be treated as a priori mean/variance for module in the next iteration. ote that in the standard turbo detector in Section II-, module produces no extrinsic output, as shown in 6. This is the main difference between the proposed algorithm and the standard detector. 4 Overall lgorithm In the first iteration, = 0 and vpri =. The operations of module and module are executed iteratively until convergence. The DFT/IDFT operations in Fig. b can be efficiently implemented using the fast Fourier transform FFT. lso, the order of the ext operations in Fig. b see and 24 and DFT/IDFT can be changed, and then one pair of DFT/IDFT can be saved. This is straightforward and we omit the details. III. STTE EVOLUTIO Following [2] [4], we analyze the large-system performance of the proposed scheme by using state evolution.. State Evolution We characterize the performance of the iterative algorithm by a recursion of two states, and vpri. In the following, for notational brevity, we define η and v. 25 We define the following MMSE of the sparse signal estimation given an WG observation with SR η mmseη E [ x E[x x + ξ] 2], 26 where x is a sparse signal modeled as 2 and ξ C 0, η. From 5 and based on ssumption 2, = [ ] var x j j, mmseη. 27 j= Under ssumptions -3, we have the following proposition. Proposition : The state evolution of the proposed turbo compressed sensing algorithm is characterized by η t+ = M v, t + σ 2 v t = v t+ mmseη t+ η t+, 28a 28b

4 4 where the subscript t and t + indicate the iteration indices. The state evolution in 28 is derived by combining 0, b, 5, 9, 24b, 25-27, together with some straightforward manipulations.. Fixed Point of State Evolution Denote by η the convergence value of η. Combining 28a and 28b and eliminating v, η can be characterized by the following fixed point equation: M σ2 mmseη η 2 M mmseη + σ 2 η + = One solution of 29 is given by η = mmse + σ 2 mmse + σ 2 2 4σ 2 mmse M 2 σ 2 mmse, 30 where mmse represents mmseη. ote that 29 has two solutions, but it can be shown that the other solution is not a valid convergence point. It can be verified that 30 is consistent with that in [5, 7 and 37] derived using the replica method. It can also be shown that 28 is equivalent to [6, 7-8]. We omit the details here due to space limitation. MSE IV. UMERICL EXMPLES i.i.d. Gaussian-MP-evolution partial DFT-MP-simulation partial DFT-proposed: dashed line: simulation, marker: evolution Iteration Fig. 2. Comparisons of the proposed algorithm and MP. = 892, M = , λ = 0.4, and SR = 50 d. In Fig. 2, we compare the MSE performance of the proposed algorithm with partial DFT matrices and MP with i.i.d. Gaussian matrices. For a fair comparison, the variance of each entry in the i.i.d. Gaussian matrix is normalized to /. Here, the implementation of MP is based on [4]. In simulation, MSE is obtained by averaging over 2000 realizations. We see that the proposed algorithm converges faster than MP and also achieves lower convergence MSE. Moreover, the state evolution analysis agrees well with simulation. We can also directly apply MP to the case with a partial DFT sensing matrix. From Fig. 2, we see that MP with partial DFT matrices outperforms the case with i.i.d. Gaussian matrices. This performance difference also indicates that the state evolution of MP is not accurate when applied to partial DFT matrices. MSE i.i.d. Gaussian partial DFT MP-evolution proposed line: simulation marker: evolution partial DFT MP-simulation Iteration Fig. 3. Comparisons of the proposed algorithm and MP. = 32768, M = , λ = 0.4, and SR = 50 d. This is reasonable because the state evolution of MP is developed for i.i.d. Gaussian matrices. In Fig. 3, we reduce the measurement ratio M/. is set to sufficiently large so that the simulation performance agrees well with state evolution. We see that MP for partial DFT performs much worse than the proposed algorithm in this setup. V. COCLUSIO D DISCUSSIOS The state evolution in Section III- is developed based on three assumptions. umerical results in Section IV demonstrate that the state evolution developed based on these assumptions is accurate. It is an interesting future research topic to establish more rigorous justifications for the state evolution. The analysis in [3] for MP with i.i.d. Gaussian sensing matrices may shed light on this problem. REFERECES [] E. J. Candès, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inf. Theory, vol. 52, no. 2, pp , [2] D. L. Donoho,. Maleki, and. Montanari, Message-passing algorithms for compressed sensing, Proceedings of the ational cademy of Sciences, vol. 06, no. 45, pp , [3] M. ayati and. Montanari, The dynamics of message passing on dense graphs, with applications to compressed sensing, IEEE Trans. Inf. Theory, vol. 57, no. 2, pp , Feb 20. [4] S. Rangan. Generalized approximate message passing for estimation with random linear mixing. Preprint, 200. [Online]. vailable: [5]. Tulino, G. Caire, S. Verdu, and S. Shamai, Support recovery with sparsely sampled free random matrices, IEEE Trans. Inf. Theory, vol. 59, no. 7, pp , July 203. [6] C. K. Wen and K. K. Wong. nalysis of compressed sensing with spatially-coupled orthogonal matrices. Preprint, 204. [Online]. vailable: [7] C. errou and. Glavieux, ear optimum error correcting coding and decoding: turbo-codes, IEEE Trans. Commun., vol. 44, no. 0, pp , Oct 996. [8] S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. J: Prentice-Hall PTR, 993. [9] Q. Guo and D. Huang, concise representation for the soft-in soft-out lmmse detector, IEEE Commun. Lett., vol. 5, no. 5, pp , May 20. [0] X. Yuan, L. Ping, C. Xu, and. Kavcic. chievable rates of mimo systems with linear precoding and iterative lmmse detection. Preprint, 20. [Online]. vailable: pdf

5 [] H.-. Loeliger, J. Dauwels, J. Hu, S. Korl, L. Ping, and F. Kschischang, The factor graph approach to model-based signal processing, Proceedings of the IEEE, vol. 95, no. 6, pp , June