SDP Memo 048: Two Dimensional Sparse Fourier Transform Algorithms

Transcription

1 SDP Memo 048: Two Dimensional Sparse Fourier Transform Algorithms Document Number SDP Memo 048 Document Type MEMO Revision Author Haihang You, Sheng Shi, Runkai Yang Release Date Document Classification Status Draft Lead Author Designation Affiliation Haihang You Professor Institute of Computing Technology, Chinese Academy of Sciences Signature & Date:

2 SDP Memo Disclaimer The SDP memos are designed to allow the quick recording of investigations and research done by members of the SDP. They are also designed to raise questions about parts of the SDP design or SDP process. The contents of a memo may be the opinion of the author, not the whole of the SDP. Table of Contents 1 Introduction 3 2 Two Dimensional Sparse Fast Fourier Transform 3 3 Adaptive Tuning Sparse Fast Fourier Transform 5 4 Conclusion 9 References 9 Page 2 of 10

3 1 Introduction Advances on communication and transmission techniques over Internet have been witnessed under the background of the big data. The amount of signals increases dramatically, and the corresponding high efficiency signal processing techniques are demanded urgently. The discrete Fourier transform (DFT) is one of the most fundamental and important numerical algorithms which plays central roles in signal processing, communications, and audio/image/video compression. The Fast Fourier Transform (FFT) greatly simplifies the complexity of DFT, and shows importance in a broad range of applications. It is well known that most signals are sparsity in frequency domain, and this sparsity property has been widely used in various applications including compressed sensing and radio astronomy. Therefore, for sparse signals, the W(n) lower bound for the complexity of DFT is no longer valid. It is necessary to study the new strategy of the Fourier transform based on signal sparsity. In 2012, Hassanieh et al proposed one-dimensional Sparse Fast Fourier Transform (SFFT) [1][2] which is faster than traditional DFT. However, the two dimensional spare Fourier transform cannot simply implement by utilizing two separate one-dimensional Sparse Fourier transform. Since two-dimensional transform for image signals are more widely used in practival applications, in [3], we proposed a method for two-dimensional Sparse Fast Fourier transform. However, the SFFT implementation has the drawback that it only works reliably for very specific input parameters, especially signal sparsity k. This drawback hinders the extensive applications of SFFT. Therefore, in this paper, we propose a new Adaptive Tuning Sparse Fast Fourier Transform (ATSFFT) [4] to improve the accuracy and robustness of SFFT. By adaptive tuning technology, ATSFFT can probe the sparsity k automatically and achieve the Fourier transform of signals. Experimental results show that ATSFFT not only can control the error better than existing SFFT, but also performs faster than SFFT. 2 Two Dimensional Sparse Fast Fourier Transform Several conventions and notations are used in this paper. A space-domain image is represented as a 2D matrix x 2 C N N, the Fourier spectrum of the image is represented as ˆx. We assume that N is a power of 2, the notation [N] is defined as the set {0,1,...,N 1}, and [N] [N]=[N] 2 denotes the N N grid {(i, j) : i 2 [N], j 2 [N]}. The image support is denoted by supp(x) [N] [N]. All matrix indices are calculated modulo the matrix size, A set of matrix elements can be written as a matrix subscripted with a set of indices, for example x I,J = {x i, j i 2 I, j 2 J}. In addition, we assume that B is a power of 2, and N can be divisible by B. We define w = e 2pi/N to be a primitive N-th root of unity. In the following sections, we will use the following definition of the 2D-DFT without the constant scaling factor: ˆx i, j = Â u2[n] Â v2[n] x u,v w iu+ jv, (i, j) 2 W N W N = {(i, j) 0 apple i apple N 1,0 apple j apple N 1} (1) This makes some proofs easier, but is not relevant in practical implementations. We briefly describe the structure of SFFT. The presentation of all details is beyond the scope of this paper and can be referred to [3] for more information. A simplified flow chart of SFFT is shown in Fig. 1 which originally appeared in[3]. The hash function is the core of SFFT. Hash function includes random spectrum permutation, filtering and subsampling in frequency domain. Page 3 of 10

4 Location Seeking Image and sparsity k Hash Function (Permute, Filter, Subsampling+FFT ) Cutoff (keep dk coordinates of maximum magnitude) Reverse Hash Function L iteration Location Seeking Image and sparsity k Hash Function (Permute, Filter, Subsampling+FFT ) Cutoff (keep dk coordinates of maximum magnitude) Reverse Hash Function coordinates Only keep coordinates occurred in at least half of the location loops coordinates Coefficient Estimation Image and sparsity k Hash Function (Permute, Filter, Subsampling+FFT ) Estimate Coefficient coordinates Coefficient Estimation Image and sparsity k L iteration For each coordinate output the median of coefficients Hash Function (Permute, Filter, Subsampling+FFT ) Estimate Coefficient Figure 1: A simplified flow diagram of SFFT SFFT does not have access to the input image s Fourier spectrum, as that would involve performing DFT. The permutation allows to permute the image s Fourier spectrum by modifying the space domain signal x. This can be done by spectrum factor P s1,s 2,t 1,t 2 where s 1,s 2 are the random step sizes and t 1,t 2 are the offsets. (P s1,s 2,t 1,t 2 x) i, j = x s1 i+t 1,s 2 j+t 2 (2) In order to achieve a sub-linear computational complexity, SFFT uses the Guassian flat window function to extract a part of the input image for computations. According to the convolution theorem, filtering process can expand the area of non-zero coefficients, in preparation for the subsequent sub-sampling and reverse steps, and further increase the probability of detecting non-zero coefficients. SFFT effectively reduces dimension by subsampling input space-domain image and summing up the result. As the signal is sparse in frequency domain, dimension reduction can reduce the complexity of searching position and amplitude of non-zero elements. The basic idea behind hash function is to hash the N N coefficients of the input image into a small number of B B bins. B is determined by k, which is set to p Nk. From these bins, SFFT only keeps coordinates of top dk points with largest magnitudes. The actual locations of the Fourier coefficients are then approximated. h s1,s 2 (i, j)=round(s 1 s 2 ij B2 ),i 2 [N], j 2 [N] (3) N2 Page 4 of 10

5 10 Run Time Vs Matrix Size (Sparsity = 50) 10 Run Time Vs Matrix Size (Sparsity = 100) SFFT FFTW SFFT FFTW 1 1 Run Time (sec) 0.1 Run Time (sec) Matrix Size Matrix Size Figure 2: The process of permutation, filtering and subsampling on image (k = 2) As shown in Fig. 1, the SFFT consists of multiple executions of two kinds of operations: location seeking and coefficient estimation. Location seeking is to generate a list of candidate coordinates which have high probability of being indices of the k nonzero coefficients in frequency domain. While coefficient estimation is used to precisely determine the frequency coefficients. We compare 2D-SFFT with 2D-FFT in FFTW algorithms library[5]. Experiment 1: The sparsity parameter k is fixed to constant (k = 50,100), and the runtime of the compared algorithms for 6 different length of image matrix from 2 8 to 2 13 are shown in Fig. 2. As expected, Fig. 2 shows that the runtimes of 2D-SFFT and 2D-FFTW are approximately linear in the log scale. However, the slope of the line for 2D-SFFT is less than the slop for 2D-FFT, 2D-SFFT has the faster runtime than 2D-FFTW for a large range of image size. 3 Adaptive Tuning Sparse Fast Fourier Transform The drawback of SFFT implementation is that it works reliably with prior knowledge of signal sparsity k. This drawback hinders the wide range of applications of SFFT. Although it is well known to all that most signals are sparse, it is almost impossible to foreknow the sparsity k of the signals in advance. The hash function hashes the N N coefficients of the input image into a small number of B B bins. An example of a image in frequency domain (k = 2) is shown in Fig.3(1). We find that after the hash function, the number of maximum points is the same with the number of original image non-zero coefficients in frequency domain as shown in Fig.3(2). Therefore, by finding the local maximum points, we can avoid prior knowledge sparsity k. ATSFFT initially sets a small value to B 0 and k 0. After hashing, we can get k 1 by finding and counting the number of local maximum values in B B matrix. Then, B 1 can be updated by analyzing the relationship between k 0 and k 1. For the convenience of description, the change ratio factor r of k is defined as follows, r i = 1 k i k i 1. (4) The iterative process is shown as follows Page 5 of 10

6 (a) Original image ( ,k = 2) (b) The image after hash function in frequency domain Figure 3: The relationship of the number of local maximum points and sparisty k 8 < B i+1 = : e 1 B i, 0appler i < d 1 B i, d 1 appler i < d 2 (5) (1 + e 2 )B i, d 2 appler i where 0 < d 1 < d 2 < 1 and 0 < e 1 < e 2 < 1. When k i approaches k i 1 within a very small range of d 1, we consider B i adapts to the full scatter point set where nonzero elements can adequately distributed, so B i+1 need to be decreased to find the better size B. When k i approaches k i 1 within a appropriate range between d 1 and d 2, we consider B i adapts to the appropriate scatter point set. If we further reduce the size of B, it will lead to large area overlap. Therefore, B i+1 is kept the same with B i. When k i fluctuates within a large range of k i 1, we consider B i is too small to lead large area overlap, so we need to continue to increase B. The visualized representation is shown in Fig. 3. Since B is a power of 2, and N can be divisible by B. A simplified flow diagram of ATSFFT is shown in Fig. 5. Compared with SFFT, ATSFFT does not need sparsity k in advance. By running multiple adaptive tuning iterations, it is able to identify candidate coordinates with a high probability of being one of the k nonzero coordinates. Given the set of coordinates I, ATSFFT can use coefficient estimation to precisely determine the frequency coefficients, which basically remove the phase change due to the permutation and the effect of the filtering. ATSFFT is compared with two representative algorithms, SFFT and FFT in FFTW algorithms library[5]. The size of image matrix N is fixed to constant (N = 2 10,2 11,2 12 ) and the runtime of the three algorithms with the sparsity k (k = 50,100,200,500,1000) is shown in Table 1. Observing the Table 1, we can get two conclusions: (i) ATSFFT is not only faster than FFTW, but also faster than SFFT which uses the fixed B; (ii) ATSFFT can control the error better than SFFT. That is, the error of ATSFFT is lower than SFFT at the same number of cycles. More intuitively, the runtime comparison on different sparsity degrees (N = 2 1 1,2 1 2) is shown in Fig. 6. We can see that when the image size is fixed, as the increase of sparsity k, the ATSFFT is faster than SFFT, while the runtime of FFTW is essentially constant, which depends on image size not sparsity k. The runtime comparison on different sparsity degrees (N = 2 1 0) is shown in Fig. 7. We can get the same conclusion that ATSFFT is faster than SFFT. In addition, SFFT is faster than Page 6 of 10

7 Figure 4: The relationship between B and r FFTW when sparsity k < 200, while 2D-SFFT presents disadvantage when sparsity k > 200. This is because the increase of sparsity k will leads to large area overlap. SFFT must increase the number of iterations to achieve the error standard, which will greatly increase running time. Therefore, SFFT performs better than FFTW when image has sufficient sparsity. By comparing ATSFFT against SFFT, the applicable range of ATSFFT is much wider than SFFT. Table 1: The runtime and error of SFFT, ASFFT and FFTW N = 1024 time(s) MSE k FFTW SFFT ATSFFT FFTW SFFT ATSFFT E E E E E E E E E E E E E E E-03 N = 2048 time(s) MSE E E E E E E E E E E E E E E E-06 N = 4096 time(s) MSE E E E E E E E E E E E E E E E-06 Page 7 of 10

8 Figure 5: A simplified flow diagram of ASFFT Figure 6: The runtime comparison on different sparsity degrees (k = 2048, 4096) Page 8 of 10

9 Figure 7: The runtime comparison of different sparsity degrees (k = 1024) 4 Conclusion FFT is an important image processing tool. Consider most images posses sparsity in frequency domain, we propose a novel 2D-SFFT which take advantage of the sparsity characteristics of images based on existing research. Experiment results show that 2D-SFFT presents a big advantage than traditional 2D-FFT. The reference implementation of the SFFT algorithm exists and has been proven to be faster than modern FFT library in cases of sufficient sparsity. However, the drawback of SFFT implementation is that it works reliably with foreknown signal sparsity k. It hinders the wide range of applications of SFFT. We present Adaptive Tuning Sparse Fast Fourier Transform ASFFT. In the case of unknown sparsity k in advance, ASFFT can identify the sparsity by adaptive dynamic iterative tuning and then complete the Fourier transform of signal. Experimental results show that ATSFFT not only can control the error better than SFFT, but also performs faster than SFFT. SFFT has a broad range of applications such as Spectrum Sensing, Magnetic Resonance Imaging (MRI), Light Field Photography, and Radio Astronomy. Especially, The Square Kilometer Array (SKA) is a radio telescope in development in Australia and South Africa. It spreads over an area of one square kilometer providing the highest resolution images ever captured in astronomy. Using existing SFFT[1] algorithms for processing images of the sky 100 faster than FFT. As this paper is exploratory, There are many algorithm application questions that future work should be considered. References [1] Haitham Hassanieh, Piotr Indyk, Dina Katabi and Eric Price, Simple and practical algorithm for sparse Fourier transform, Proceedings of the Twenty-Third Annual ACM-SIAM Symposium on Discrete Algorithms, 2012, Japan, pp [2] Haitham Hassanieh, Piotr Indyk, Dina Katabi and Eric Price, Nearly optimal sparse fourier transform, Proceedings of the 44th Symposium on Theory of Computing Conference, 2012, New York, pp [3] S. Shi, R. Yang and H. You, A new two-dimensional Fourier transform algorithm based on image sparsity, 2017 IEEE International Conference on Acoustics, Speech and Signal Process- Page 9 of 10

10 ing (ICASSP), New Orleans, LA, 2017, pp doi: /ICASSP [4] S. Shi, R. Yang, X. Zh, H. You and D. F, An Adaptive Tuning Sparse Fourier Transform, 2017 Pacific-Rim Conference on Multimedia (PCM), Harbin, China, 2017, pp [5] M. Frigo and S. G. Johnson, FFTW: an adaptive software architecture for the FFT, Acoustics, Speech and Signal Processing, Proceedings of the 1998 IEEE International Conference on, Seattle, WA, 1998, pp Page 10 of 10