Uniqueness in Bilinear Inverse Problems with Applications to Subspace and Sparsity Models

Size: px

Start display at page:

Download "Uniqueness in Bilinear Inverse Problems with Applications to Subspace and Sparsity Models"

Kevin Banks
5 years ago
Views:

Kiryung Lee and Yoram Bresler Coordinated Science

Engineering University of Illinois, Urbana-Champaign Email:

1 Uniqueness in Bilinear Inverse Problems with Applications to Subspace and Sparsity Models Yanjun Li Joint work with Kiryung Lee and Yoram Bresler Coordinated Science Laboratory Department of Electrical and Computer Engineering University of Illinois, Urbana-Champaign SampTA 2015 May 29, 2015, American University, Washington, D.C. 1

2 1 Bilinear Inverse Problems 2 Identifiability in Bilinear Inverse Problems 3 Identifiability in Blind Gain and Phase Calibration (BGPC) 4 Discussion 2

3 Inverse Problems Image deblurring Tomography MRI Solving equation Ax = b, subject to x Ω 3

4 Inverse Problems Image deblurring Tomography MRI Solving equation Ax = b, subject to x Ω 3

5 Inverse Problems Image deblurring Tomography MRI Solving equation Ax = b, subject to x Ω 3

6 Bilinear Inverse Problems What if both the input and the system are unknown? Blind image deblurring Calibrationless pmri Dictionary learning 4

Bilinear Inverse Problems As shown in both examples, GRAPPA

PILS and SMASH contain visually aliasing artifacts (arrows).

experiments are not What if both the input and sufficiently the

method, and the Figure 5 Simulated sensitivity maps of the

Due to the space limit, only the magnitude parts of the complex

PILS method may show banding artifacts at high undersampling

It also shows that SMASH works better with linear spine array

SPACE-RIP and SENSE these examples because all methods performed

7 Bilinear Inverse Problems As shown in both examples, GRAPPA resulted in the high SNR and low AP. PILS and SMASH contain visually aliasing artifacts (arrows). The results show that the coil sensitivities in these experiments are not What if both the input and sufficiently the system localized for the are PILSunknown? method, and the Figure 5 Simulated sensitivity maps of the eight-channel head array. Due to the space limit, only the magnitude parts of the complex sensitivity maps are shown. PILS method may show banding artifacts at high undersampling factors. It also shows that SMASH works better with linear spine array than with the head array for which the spatial harmonics are difficult to synthesize. GRAPPA yields slightly better reco struction quality than SPACE-RIP and SENSE these examples because all methods performed sens tivity estimation using self-calibrated data. Computational Complexity. The overall comput tional complexity of the five reconstruction metho is shown in Fig. 11. As an example, the number required complex multiplications is shown using t (a) Coil 1 Coil 2 Coil 3 Coil 4 Blind image deblurring Calibrationless pmri Dictionary learning (b) Coil 1 Coil 2 Coil 3 Coil 4 Coil 5 Coil 6 Coil 7 Coil 8 Figure 6 In-vivo MR images acquired using (a) four-channel spine array and (b) eight-channel head array. Concepts in Magnetic Resonance Part B(Magnetic Resonance Engineering) DOI /cmr 4

8 Bilinear Inverse Problems What if both the input and the system are unknown? Blind image deblurring Calibrationless pmri Dictionary learning 4

9 Bilinear Inverse Problem (BIP) Definition (Bilinear Mapping) F : X Y Z such that: F(a 1 x 1 + a 2 x 2, y) = a 1 F(x 1, y) + a 2 F(x 2, y), F(x, b 1 y 1 + b 2 y 2 ) = b 1 F(x, y 1 ) + b 2 F(x, y 2 ). Definition (Bilinear Inverse Problem) Given bilinear measurement z = F(x 0, y 0 ), and constraint sets Ω X, Ω Y : (BIP) find (x, y), s.t. F(x, y) = z, x Ω X, y Ω Y. In what sense can the solution be unique? Under what condition is the solution unique? 5

10 Bilinear Inverse Problem (BIP) Definition (Bilinear Mapping) F : X Y Z such that: F(a 1 x 1 + a 2 x 2, y) = a 1 F(x 1, y) + a 2 F(x 2, y), F(x, b 1 y 1 + b 2 y 2 ) = b 1 F(x, y 1 ) + b 2 F(x, y 2 ). Definition (Bilinear Inverse Problem) Given bilinear measurement z = F(x 0, y 0 ), and constraint sets Ω X, Ω Y : (BIP) find (x, y), s.t. F(x, y) = z, x Ω X, y Ω Y. In what sense can the solution be unique? Under what condition is the solution unique? 5

11 Bilinear Inverse Problem (BIP) Definition (Bilinear Mapping) F : X Y Z such that: F(a 1 x 1 + a 2 x 2, y) = a 1 F(x 1, y) + a 2 F(x 2, y), F(x, b 1 y 1 + b 2 y 2 ) = b 1 F(x, y 1 ) + b 2 F(x, y 2 ). Definition (Bilinear Inverse Problem) Given bilinear measurement z = F(x 0, y 0 ), and constraint sets Ω X, Ω Y : (BIP) find (x, y), s.t. F(x, y) = z, x Ω X, y Ω Y. In what sense can the solution be unique? Under what condition is the solution unique? 5

12 1 Bilinear Inverse Problems 2 Identifiability in Bilinear Inverse Problems 3 Identifiability in Blind Gain and Phase Calibration (BGPC) 4 Discussion 6

13 Previous Work Lifting for blind deconvolution [Ahmed et al., 2014], phase retrieval [Candès et al., 2013]: for bilinear mapping F : C m C n Z, define linear mapping G : C m n Z such that G(xy T ) = F(x, y). (Lifted BIP) find M, s.t. G(M) = z, M Ω M = {xy T : x Ω X, y Ω Y }. Identifiability up to scaling [Choudhary and Mitra, 2014]: the set of rank-2 matrices in the null space of G Limitations: Euclidean spaces Scaling ambiguity Interpretability 7

14 Previous Work Lifting for blind deconvolution [Ahmed et al., 2014], phase retrieval [Candès et al., 2013]: for bilinear mapping F : C m C n Z, define linear mapping G : C m n Z such that G(xy T ) = F(x, y). (Lifted BIP) find M, s.t. G(M) = z, M Ω M = {xy T : x Ω X, y Ω Y }. Identifiability up to scaling [Choudhary and Mitra, 2014]: the set of rank-2 matrices in the null space of G Limitations: Euclidean spaces General vector spaces Scaling ambiguity More ambiguities Interpretability Interpretable conditions 7

15 A Blind Deconvolution Problem Given the measurement z = x 0 y 0 : find (x, y), s.t. x y = z, x C n, y C n. Let S l denote the circular shift by l. Every pair (x, y) = ( σs l (x 0 ), 1 σ S l(y 0 ) ) is a solution. Scaling ambiguity and shift ambiguity 8

16 A Blind Deconvolution Problem Given the measurement z = x 0 y 0 : find (x, y), s.t. x y = z, x C n, y C n. Transformation group on C n C n associated with circular convolution: { ( T = W : W(x, y) = σs l (x), 1 ) } σ S l(y), for σ 0, l Z. Equivalence class: [(x 0, y 0)] T = { (x, y) C n C n : (x, y) = ( σs l (x 1 ) } 0), σ S l(y 0), for σ 0, l Z 9

17 Identifiability up to a Transformation Group Given the bilinear measurement z = F(x 0, y 0 ): (BIP) find (x, y), s.t. F(x, y) = z, x Ω X, y Ω Y. Transformation group T on Ω X Ω Y associated with F satisfies: Closed under composition. Contains identity transformation 1. Contains inverse transformations: W and W 1. F is preserved: F = F W, W T. Definition (Identifiability up to Transformation Group T) Every solution satisfies (x, y) = W(x 0, y 0 ), for some W T. Example: Every solution is a scaled and/or circularly shifted version of (x 0, y 0). 10

18 Identifiability up to a Transformation Group Given the bilinear measurement z = F(x 0, y 0 ): (BIP) find (x, y), s.t. F(x, y) = z, x Ω X, y Ω Y. Transformation group T on Ω X Ω Y associated with F satisfies: Closed under composition. Contains identity transformation 1. Contains inverse transformations: W and W 1. F is preserved: F = F W, W T. Definition (Identifiability up to Transformation Group T) Every solution satisfies (x, y) = W(x 0, y 0 ), for some W T. Example: Every solution is a scaled and/or circularly shifted version of (x 0, y 0). 10

19 Main Result Given the bilinear measurement z = F(x 0, y 0 ): (BIP) find (x, y), s.t. F(x, y) = z, x Ω X, y Ω Y. Theorem A sufficient condition for identifiability is: 1 Vector x 0 can be identified up to the transformation group. 2 Once x 0 is identified, the recovery of y 0 is unique. Under mild conditions on F and T, the above conditions are also necessary. Example: 1 Every solution x is a scaled and/or circularly shifted version of x 0. 2 Given x 0, the solution y 0 to non-blind deconvolution is unique. 11

20 1 Bilinear Inverse Problems 2 Identifiability in Bilinear Inverse Problems 3 Identifiability in Blind Gain and Phase Calibration (BGPC) 4 Discussion 12

21 Blind Gain and Phase Calibration (BGPC) find (λ, Φ), s.t. diag(λ)φ = Y, λ C n, Φ C n N. Y = diag(λ) Φ Φ 0 : unknown signals; λ 0 : unknown gain and phase, λ (j) 0 = g (j) e ip(j) Ill-posed! Ω Φ = {Φ = AX : X Ω X }: Subspace: A is a basis for a lower-dimensional subspace. Joint sparsity: A is a basis or a frame, columns of X are jointly s-sparse. (BGPC) find (λ, X), s.t. diag(λ)ax = Y, λ C n, X Ω X. 13

22 Blind Gain and Phase Calibration (BGPC) find (λ, Φ), s.t. diag(λ)φ = Y, λ C n, Φ C n N. Y = diag(λ) Φ Φ 0 : unknown signals; λ 0 : unknown gain and phase, λ (j) 0 = g (j) e ip(j) Ill-posed! Ω Φ = {Φ = AX : X Ω X }: Subspace: A is a basis for a lower-dimensional subspace. Joint sparsity: A is a basis or a frame, columns of X are jointly s-sparse. (BGPC) find (λ, X), s.t. diag(λ)ax = Y, λ C n, X Ω X. 13

23 Blind Gain and Phase Calibration (BGPC) find (λ, Φ), s.t. diag(λ)φ = Y, λ C n, Φ C n N. Y = diag(λ) Φ Φ 0 : unknown signals; λ 0 : unknown gain and phase, λ (j) 0 = g (j) e ip(j) Ill-posed! Ω Φ = {Φ = AX : X Ω X }: Subspace: A is a basis for a lower-dimensional subspace. Joint sparsity: A is a basis or a frame, columns of X are jointly s-sparse. (BGPC) find (λ, X), s.t. diag(λ)ax = Y, λ C n, X Ω X. 13

24 Applications in Which BGPC Arises Inverse rendering in computational relighting [Nguyen et al., 2013] Blind gain and phase calibration in sensor array processing [Paulraj and Kailath, 1985] Synthetic aperture radar (SAR) autofocus [Morrison et al., 2009] Multichannel blind deconvolution [Abed-Meraim et al., 1997] (BGPC) find (λ, X), s.t. diag(λ)ax = Y, λ C n, X Ω X. 14

25 Applications in Which BGPC Arises Inverse rendering in computational relighting [Nguyen et al., 2013] Blind gain and phase calibration in sensor array processing [Paulraj and Kailath, 1985] Synthetic aperture radar (SAR) autofocus [Morrison et al., 2009] Multichannel blind deconvolution [Abed-Meraim et al., 1997] (BGPC) find (λ, X), s.t. diag(λ)ax = Y, λ C n, X Ω X. 14

26 Applications in Which BGPC Arises Inverse rendering in computational relighting [Nguyen et al., 2013] Blind gain and phase calibration in sensor array processing [Paulraj and Kailath, 1985] Synthetic aperture radar (SAR) autofocus [Morrison et al., 2009] Multichannel blind deconvolution [Abed-Meraim et al., 1997] (BGPC) find (λ, X), s.t. diag(λ)ax = Y, λ C n, X Ω X. 14

1985] Synthetic aperture radar (SAR) autofocus [Morrison et al.

27 Applications in Which BGPC Arises Inverse rendering in computational relighting [Nguyen et al., 2013] Blind gain and phase calibration in sensor array processing [Paulraj and Kailath, 1985] Synthetic aperture radar (SAR) autofocus [Morrison et al., 2009] Multichannel blind deconvolution [Abed-Meraim et al., 1997] (BGPC) find (λ, X), s.t. diag(λ)ax = Y, λ C n, X Ω X. 14

28 Sample Complexity find (λ, X), s.t. diag(λ)ax = Y, λ C n, X Ω X. Y = diag(λ) n A m X N Sample complexity: condition on N (number of channels, snapshots, lighting conditions, etc.) in terms of n (length of the signals), m (dimension of subspace) or s (sparsity level) Sufficient Condition Subspace Constraint [Nguyen et al., 2013] Inverse rendering N m Joint Sparsity Constraint? Necessary Condition [Morrison et al., 2009] SAR autofocus N n 1 n m?? Gap?? 15

29 Subspace Constraint The matrix A C n m is tall (n > m). find (λ, X), s.t. diag(λ)ax = Y, λ C n, X C m N. Y diag(λ) A = n m Transformation group: T = {W σ : W σ (λ, X) = (σλ, 1 σ X), σ 0}. X N Definition (Decomposability of A) span(a ) = span(a (J,:) ) span(a (J c,:) ). Example: J = {1, 3}, J c = {2}. A =,

30 Subspace Constraint The matrix A C n m is tall (n > m). find (λ, X), s.t. diag(λ)ax = Y, λ C n, X C m N. Y diag(λ) A = n m Transformation group: T = {W σ : W σ (λ, X) = (σλ, 1 σ X), σ 0}. X N Definition (Decomposability of A) span(a ) = span(a (J,:) ) span(a (J c,:) ). Example: J = {1, 3}, J c = {2}. 1 2 A = 1 1, 2 4 [ ] 1 [ ] 1 R 2 = span 2 span

31 Subspace Constraint The matrix A C n m is tall (n > m). find (λ, X), s.t. diag(λ)ax = Y, λ C n, X C m N. Y diag(λ) A = n m Transformation group: T = {W σ : W σ (λ, X) = (σλ, 1 σ X), σ 0}. X N Definition (Decomposability of A) span(a ) = span(a (J,:) ) span(a (J c,:) ). Example: J = {1, 3}, J c = {2}. 1 2 A (J,:) =, 2 4 span [ ]

32 Subspace Constraint The matrix A C n m is tall (n > m). find (λ, X), s.t. diag(λ)ax = Y, λ C n, X C m N. Y diag(λ) A = n m Transformation group: T = {W σ : W σ (λ, X) = (σλ, 1 σ X), σ 0}. X N Definition (Decomposability of A) span(a ) = span(a (J,:) ) span(a (J c,:) ). Example: J = {1, 3}, J c = {2}. A (Jc,:) = 1 1, span [ ]

33 Subspace Constraint The matrix A C n m is tall (n > m). find (λ, X), s.t. diag(λ)ax = Y, λ C n, X C m N. Y diag(λ) A = n m Transformation group: T = {W σ : W σ (λ, X) = (σλ, 1 σ X), σ 0}. X N Definition (Decomposability of A) span(a ) = span(a (J,:) ) span(a (J c,:) ). Example: J = {1, 3}, J c = {2}. 1 2 A = 1 1, 2 4 [ ] 1 [ ] 1 R 2 = span 2 span

34 Subspace Constraint Y diag(λ) A X = N n m Theorem (Sufficient Condition for Identifiability up to Scaling) [Nguyen et al., 2013] 1 All the entries of λ 0 are nonzero. 2 X 0 has full row rank. 3 A has full column rank, and its row space is not decomposable. If A is tall and X 0 is fat, then almost all λ 0, X 0, and A satisfy the above conditions. Sample complexity: N m Dimension of subspace Number of signals 17

35 Joint Sparsity Constraint With DFT Matrix Y diag(λ) = n F n X N The matrix A = F C n n is the normalized DFT matrix. find (λ, X), s.t. diag(λ)f X = Y, λ C n, X {X C n N : X has at most s nonzero rows}. Definition (Periodicity of the Joint Support) {j 1, j 2,, j s } = {j 1 + l, j 2 + l,, j s + l} (modulo n). Example: {1, 3, 5} = {1 + 2, 3 + 2, 5 + 2} (modulo 6) {2, 3, 5, 6} = {2 + 3, 3 + 3, 5 + 3, 6 + 3} (modulo 6) 18

36 Joint Sparsity Constraint With DFT Matrix Y diag(λ) = n F n X N The matrix A = F C n n is the normalized DFT matrix. find (λ, X), s.t. diag(λ)f X = Y, entrywise product circular convolution λ C n, X {X C n N : X has at most s nonzero rows}. Transformation group: scaling and circular shift Definition (Periodicity of the Joint Support) {j 1, j 2,, j s } = {j 1 + l, j 2 + l,, j s + l} (modulo n). Example: {1, 3, 5} = {1 + 2, 3 + 2, 5 + 2} (modulo 6) {2, 3, 5, 6} = {2 + 3, 3 + 3, 5 + 3, 6 + 3} (modulo 6) 18

37 Joint Sparsity Constraint With DFT Matrix Y diag(λ) = n F n X N The matrix A = F C n n is the normalized DFT matrix. find (λ, X), s.t. diag(λ)f X = Y, entrywise product circular convolution λ C n, X {X C n N : X has at most s nonzero rows}. Transformation group: scaling and circular shift Definition (Periodicity of the Joint Support) {j 1, j 2,, j s } = {j 1 + l, j 2 + l,, j s + l} (modulo n). Example: {1, 3, 5} = {1 + 2, 3 + 2, 5 + 2} (modulo 6) {2, 3, 5, 6} = {2 + 3, 3 + 3, 5 + 3, 6 + 3} (modulo 6) 18

38 Joint Sparsity Constraint With DFT Matrix Y diag(λ) F X = n n N Theorem (Sufficient Condition for Identifiability up to Scaling and Circular Shift) 1 All the entries of λ 0 are nonzero. 2 X 0 has s nonzero rows and rank s. 3 The joint support of the columns of X 0 is not periodic. Assume that N s, X 0 has exactly s nonzero rows. If the joint support is contiguous, then almost all λ 0 and X 0 satisfy the above conditions. If n and s are coprime, then almost all λ 0 and X 0 satisfy the above conditions. Sample complexity: N s Number of signals 19

39 Joint Sparsity Constraint With DFT Matrix Y diag(λ) F X = n n N Theorem (Sufficient Condition for Identifiability up to Scaling and Circular Shift) 1 All the entries of λ 0 are nonzero. 2 X 0 has s nonzero rows and rank s. 3 The joint support of the columns of X 0 is not periodic. Assume that N s, X 0 has exactly s nonzero rows. If the joint support is contiguous, then almost all λ 0 and X 0 satisfy the above conditions. If n and s are coprime, then almost all λ 0 and X 0 satisfy the above conditions. Sample complexity: N s NumberSparsity of signals level 19

40 Tightness of Sample Complexities Sufficient Condition Subspace Constraint N m Joint Sparsity Constraint (DFT) N s Necessary Condition N n 1 n m N n 1 n s Numerical Experiment N N m s 0 20

41 1 Bilinear Inverse Problems 2 Identifiability in Bilinear Inverse Problems 3 Identifiability in Blind Gain and Phase Calibration (BGPC) 4 Discussion 21

42 Discussion Summary: Identifiability in BIP up to transformation group General framework with two-step approach. Identifiability in BGPC under subspace/joint sparsity constraints Interpretable results in terms of sample complexities. Future work: Bridging the gap Imposing constraints on λ Analyzing other BIPs within the framework Our journal paper: 22

43 Thank you! 23

44 References K. Abed-Meraim, Wanzhi Qiu, and Y. Hua. Blind system identification. 85(8): , Aug ISSN A. Ahmed, B. Recht, and J. Romberg. Blind deconvolution using convex programming. 60(3): , Mar ISSN Emmanuel J. Candès, Thomas Strohmer, and Vladislav Voroninski. Phaselift: Exact and stable signal recovery from magnitude measurements via convex programming. Commun. Pure Appl. Math., 66(8): , ISSN Sunav Choudhary and Urbashi Mitra. Identifiability scaling laws in bilinear inverse problems. arxiv preprint arxiv: , R.L. Morrison, M.N. Do, and Jr. Munson, D.C. MCA: A multichannel approach to SAR autofocus. 18(4): , Apr ISSN Ha Q. Nguyen, Siying Liu, and Minh N. Do. Subspace methods for computational relighting. Proc. SPIE, 8657: , A. Paulraj and T. Kailath. Direction of arrival estimation by eigenstructure methods with unknown sensor gain and phase. In Proc. Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), volume 10, pages IEEE, Apr

Detecting Burnscar from Hyperspectral Imagery via Sparse Representation with Low-Rank Interference

Detecting Burnscar from Hyperspectral Imagery via Sparse Representation with Low-Rank Interference Minh Dao 1, Xiang Xiang 1, Bulent Ayhan 2, Chiman Kwan 2, Trac D. Tran 1 Johns Hopkins Univeristy, 3400