PHASE retrieval has been an active research topic for decades [1], [2]. The underlying goal is to estimate an unknown

Transcription

1 DOLPHIn Dictionary Learning for Phase Retrieva Andreas M. Timann, Yonina C. Edar, Feow, IEEE, and Juien Maira, Member, IEEE arxiv:60.063v [math.oc] 3 Aug 06 Abstract We propose a new agorithm to earn a dictionary for reconstructing and sparsey encoding signas from measurements without phase. Specificay, we consider the task of estimating a two-dimensiona image from squared-magnitude measurements of a compex-vaued inear transformation of the origina image. Severa recent phase retrieva agorithms expoit underying sparsity of the unknown signa in order to improve recovery performance. In this work, we consider such a sparse signa prior in the context of phase retrieva, when the sparsifying dictionary is not known in advance. Our agorithm jointy reconstructs the unknown signa possiby corrupted by noise and earns a dictionary such that each patch of the estimated image can be sparsey represented. Numerica experiments demonstrate that our approach can obtain significanty better reconstructions for phase retrieva probems with noise than methods that cannot expoit such hidden sparsity. Moreover, on the theoretica side, we provide a convergence resut for our method. Index Terms MLR-DICT, MLR-LEAR, OPT-NCVX, OPT-SOPT) Machine Learning, Signa Reconstruction, Image Reconstruction I. INTRODUCTION PHASE retrieva has been an active research topic for decades [], []. The underying goa is to estimate an unknown signa from the moduus of a compex-vaued inear transformation of the signa. With such noninear measurements, the phase information is ost hence the name phase retrieva ), rendering the recovery task i-posed and, perhaps not surprisingy, NP-hard [3]. Traditiona approaches consider cases where the soution is unique up to a goba phase shift, which can never be uniquey resoved, and devise signa reconstruction agorithms for such settings. Uniqueness properties and the empirica success of recovery agorithms usuay hinge on oversamping the signa, i.e., taking more measurements than the number of signa components. The most popuar techniques for phase retrieva are based on aternating projections, see [4], [5], [6] for overviews. These methods usuay require precise prior information about the signa such as knowedge of the support set) and often converge to erroneous resuts. More recent approaches incude semidefinite programming reaxations [7], [8], [9], [0], [] and gradientbased methods such as Wirtinger Fow [], [3]. In recent years, new phase retrieva techniques were deveoped for recovering sparse signas, which are inear combinations of ony a few atoms from a known dictionary [8], [4], [3], [5]. With a sparsity assumption, these agorithms obtained better recovery performance than traditiona non-sparse approaches. The main idea is akin to compressed sensing, where one works with fewer inear) measurements than signa components [6], [7], [8]. An important motivation for deveoping sparse recovery techniques was that many casses of signas admit a sparse approximation in some basis or overcompete dictionary [9], [0], []. Whie sometimes such dictionaries are known expicity, better resuts have been achieved by adapting the dictionary to the data, e.g., for image denoising [0]. Numerous agorithms have been deveoped for this task, see, e.g., [9], [], [3]. In this traditiona setting, the signa measurements are inear and a arge database of training signas is used to train the dictionary. In this work, we propose a dictionary earning formuation for simutaneousy soving the signa reconstruction and sparse representation probems given noninear, phaseess and noisy measurements. To optimize the resuting nonconvex) objective function, our agorithm referred to as DOLPHIn DictiOnary Learning for PHase retrieva) aternates between severa minimization steps, thus monotonicay reducing the vaue of the objective unti a stationary point is found if step sizes are chosen appropriatey). Specifiay, we iterate between best fitting the data and sparsey representing the recovered signa. DOLPHIn combines projected gradient descent steps to update the signa, iterative shrinkage to obtain a sparse approximation [4], and bock-coordinate descent for the dictionary update [3]. A. M. Timann is with the TU Darmstadt, Research Group Optimization, Doivostr. 5, 6493 Darmstadt, Germany e-mai: timann@mathematik.tudarmstadt.de). Y. C. Edar is with the Department of Eectrica Engineering, Technion Israe Institute of Technoogy, Haifa 3000, Israe e-mai: yonina@ee.technion.ac.i). J. Maira is with Inria, Lear Team, Laboratoire Jean Kuntzmann, CNRS, Université Grenobe Apes, 655, Avenue de Europe, Montbonnot, France e-mai: juien.maira@inria.fr). The work of J. Maira was funded by the French Nationa Research Agency [Macaron project, ANR-4-CE ]. The work of Y. Edar was funded by the European Union s Horizon 00 research and innovation programme under grant agreement ERC-BNYQ, and by the Israe Science Foundation under Grant no. 335/4. This work has been submitted to the IEEE for possibe pubication. Copyright may be transferred without notice, after which this version may no onger be accessibe.

2 In various experiments on image reconstruction probems, we demonstrate the abiity of DOLPHIn to achieve significanty improved resuts when the oversamping ratio is ow and the noise eve high, compared to the recent state-of-the-art Wirtinger Fow WF) method [], which cannot expoit sparsity if the dictionary is unknown. In this two-dimensiona setting, we break an image down into sma patches and train a dictionary such that each patch can be sparsey represented using this dictionary. The patch size as we as the amount of overap between patches can be freey chosen, which aows us to contro the trade-off between the amount of computation required to reconstruct the signa and the quaity of the resut. The paper is organized as foows: In Sections II and III, we introduce the DOLPHIn framework and agorithm. Then, in Section IV, we present numerica experiments and impementation detais, aong with discussions about hyper-)parameter seection and variants of DOLPHIn. We concude the paper in Section V. The appendices provide further detais on the mathematica derivation of the DOLPHIn agorithm and its properties. A short preiminary version of this work appeared in the conference paper [5]. II. PHASE RETRIEVAL MEETS DICTIONARY LEARNING In mathematica terms, the phase retrieva probem can be formuated as soving a noninear system of equations: Find x X C N s.t. f i x) = y i i =,..., M, ) where the functions f i : C N C are inear operators and the scaars y i are noninear measurements of the unknown origina signa ˆx in X, obtained by removing the phase information. The set X represents constraints corresponding to additiona prior information about ˆx. For instance, when deaing with rea-vaued bounded signas, this may typicay be a box constraint X = [0, ] N. Other common constraints incude information about the support set that is, the set of nonzero coefficients of ˆx. Cassica phase retrieva concerns the recovery of ˆx given the squared) moduus of the signa s Fourier transform. Other commony considered cases pertain to randomized measurements f i are random inear functions) or coded diffraction patterns, i.e., concatenations of random signa masks and Fourier transforms see, e.g., [], []). A. Prior and Reated Work The most popuar methods for cassica phase retrieva Fienup s agorithm [5] and many reated approaches [], [4], [6], [6], [7] are based on aternating projections onto the sets Y := {x C N s.t. f i x) = y i i} or {x C N s.t. f i x) = y i i}) and onto the set X. However, the nonconvexity of Y makes the projection not uniquey defined and possiby hard to compute. The success of such projection-based methods hinges criticay on precise prior knowedge which, in genera, wi not be avaiabe in practice) and on the choice of a projection operator onto Y. Utimatey, convergence to ˆx up to goba phase) is in genera not guaranteed and these methods often fai in practice. Further agorithmic techniques to tacke ) incude two different semidefinite reaxation approaches, PhaseLift [7] and PhaseCut []. PhaseLift ifts ) into the space of compex) positive semidefinite rank- matrices via the variabe transformation X := xx. Then, the noninear constraints f i x) = y i are equivaent to inear constraints with respect to the matrix variabe X. By suitaby reaxing the immediate but intractabe rank-minimization objective, one obtains a convex semidefinite program SDP). Simiary, PhaseCut introduces a separate variabe u for the phase, aowing to eiminate x, and then ifts u to obtain an equivaent probem with a rank--constraint, which can be dropped to obtain a different SDP reaxation of ). Despite some guarantees on when these reaxations are tight, i.e., aow for correcty recovering the soution to ) again up to a goba phase factor), their practica appicabiity is imited due to the dimension of the SDP that grows quadraticay with the probem dimension. A recent method that works in the origina variabe space is the so-caed Wirtinger Fow agorithm []. Here, ) is recast as the optimization probem min x C N 4M M fi x) ), y i ) i= which is approximatey soved by a gradient descent agorithm. Note that in the case of compex variabes, the concept of a gradient is not we-defined, but as shown in [], a strongy reated expression termed the Wirtinger derivative can be used instead and indeed reduces to the actua gradient in the rea case. For the case of i.i.d. Gaussian random measurements, oca convergence with high probabiity can be proven for the method, and a certain spectra initiaization provides sufficienty accurate signa estimates for these resuts to be appicabe. Further variants of the Wirtinger Fow WF) method that have been investigated are the Truncated WF [8], which invoves improving search directions by a statisticay motivated technique to fiter out components that bear too much infuence, and Threshoded WF [3], which aows for improved reconstruction of sparse signas i.e., ones with ony a few significant components or nonzero eements), in particuar when the measurements are corrupted by noise. The concept of sparsity has been successfuy empoyed in the context of signa reconstruction from inear measurements, perhaps most prominenty in the fied of compressed sensing [6], [7], [8], [9] during the past decade. There, the task is to recover an unkown signa ˆx C N from M < N inear measurements that is, finding the desired soution among the

3 3 infinitey many soutions of an underdetermined system of inear equations. For signas that are exacty or approximatey) sparse with respect to some basis or dictionary, i.e., when ˆx Dâ for a matrix D and a vector â that has few nonzero entries, such recovery probems have been shown to be sovabe in a very broad variety of settings and appications, and with a host of different agorithms. Dictionaries enabing sparse signa representations are sometimes, but not generay, known in advance. The goa of dictionary earning is to improve upon the sparsity achievabe with a given anaytica) dictionary, or to find a suitabe dictionary in the first pace. Given a set of training signas, the task consists of finding a dictionary such that every training signa can be we-approximated by inear combinations of just a few atoms. Again, many methods have been deveoped for this purpose see, e.g., [9], [0], [], [], [3]) and demonstrated to work we in different practica appications. Signa sparsity or compressabiity) can aso be beneficiay expoited in phase retrieva methods, cf. [8], [9], [3], [4], [5]. However, to the best of our knowedge, existing methods assume that the signa is sparse itsef or sparse with respect to a fixed pre-defined dictionary. This motivates the deveopment of new agorithms and formuations to jointy earn suitabe dictionaries and reconstruct input signas from noninear measurements. B. Dictionary Learning for Phase Retrieva In this paper, we consider the probem of phase retrieva by focusing on image reconstruction appications. Therefore, we wi work in a two-dimensiona setting directy. However, it shoud be noted that a expressions and agorithms can aso easiy be formuated for one-dimensiona signas ike ), as detaied in Appendix A. We wi aso consider the case of noisy measurements, and wi show that our approach based on dictionary earning is particuary robust to noise, which is an important feature in practice. Concretey, we wish to recover an image ˆX in [0, ] N N from noise-corrupted phaseess noninear measurements Y := F ˆX) + N, 3) where F : C N N C M M is a inear operator, N is a rea matrix whose entries represent noise, and the compex moduus and squares are taken component-wise. As mentioned earier, signa sparsity is known to improve the performance of phase retrieva agorithms, but a sparsifying transform is not aways known in advance, or a better choice than a predefined seection can sometimes be obtained by adapting the dictionary to the data. In the context of image reconstruction, this motivates earning a dictionary D in R s n such that each s s patch ˆx i of ˆX, represented as a vector of size s = s s, can be approximated by ˆx i Da i with a sparse vector a i in R n. Here, n is chosen a priori and the number of patches depends on whether the patches are overapping or not. In genera, D is chosen such that n s. With inear measurements, the paradigm woud be simiar to the successfu image denoising technique of [0], but the probem 3) is significanty more difficut to sove due to the moduus operator. Before detaiing our agorithm for soving 3), we introduce the foowing notation. Because our approach is patch-based as most dictionary earning formuations), we consider the inear operator E : C N N C s p that extracts the p patches x i which may overap or not) from an image X and forms the matrix EX) = x,..., x p ). Simiary, we define the inear operator R : C s p C N N that reverses this process, i.e., buids an image from a matrix containing vectorized patches as its coumns. Thus, in particuar, we have REX)) = X. When the patches do not overap, the operator R simpy paces every sma patch at its appropriate ocation in a arger N N image. When they overap, the operator R averages the pixe vaues from the patches that fa into the same ocation. Further, et A := a,..., a p ) in R n p be the matrix containing the patch representation coefficient vectors as coumns. Then, our desired sparse-approximation reation x i Da i for a i can be expressed as EX) DA. With this notation in hand, we may now introduce our method, caed DOLPHIn DictiOnary Learning for PHase retrieva). We consider an optimization probem which can be interpreted as a combination of an optimization-based approach to phase retrieva minimizing the residua norm with respect to the set of noninear equations induced by the phaseess measurements, cf. ) and a patch-based) dictionary earning mode simiar to that used for image denoising in [0]. The mode contains three variabes: The image, or phase retrieva soution X, the dictionary D and the matrix A containing as coumns the coefficient vectors of the representation X RDA). The phase retrieva task consists of estimating X and the dictionary earning or sparse coding task consists of estimating D and A; a common objective function provides feedback between the two objectives, with the goa of improving the phase retrieva reconstruction procedure by encouraging the patches of X to admit a sparse approximation. Formay, the DOLPHIn formuation consists of minimizing min X,D,A 4 Y FX) + µ F EX) DA p + λ F a i s.t. X [0, ] N N, D D. 4) Here, X F denotes the Frobenius matrix-norm, which generaizes the Eucidean norm to matrices. The parameters µ, λ > 0 in the objective 4) provide a way to contro the trade-off between the data fideity term from the phase retrieva probem i=

4 4 Agorithm Dictionary earning for phase retrieva DOLPHIn) Input: Initia image estimate X 0) [0, ] N N, initia dictionary D 0) D R s n, parameters µ, λ > 0, maximum number of iterations K, K Output: Learned dictionary D = D K), patch representations A = A K), image reconstructions X = X K) and RDA) : for = 0,,,..., K + K =: K do : choose step size γ A as expained in Section III-A and update A +) S γ A λ/µ A γ A D ) D) A ) EX ) ) )) 3: choose step size γ X as expained in Section III-D or IV-B and update X +) P X X ) γ X R F FX) FX) Y) )) + µr R EX) DA ))), where R is defined in Section III-B 4: if < K then 5: do not update the dictionary: D +) D ) 6: ese 7: set B EX ) )A ) and C A )A ) 8: for j =,..., n do 9: if C jj > 0 then ) 0: update j-th coumn: D +) ) j C jj B j D ) C j + D) ) j : ese : reset j-th coumn: e.g., D +) ) j random N 0, ) vector in R s ) 3: project D +) ) j max{, D +) ) j D } +)) j and the approximation sparsity of the image patches. To that effect, we use the -norm, which is we-known to have a sparsity-inducing effect [30]. In order to avoid scaing ambiguities, we aso restrict D to be in the subset D := {D R s n : d j j =,..., n} of matrices with coumn -norms at most, and assume n < p otherwise, each patch is triviay representabe by a -sparse vector a i by incuding x i / x i as a coumn of D). The mode 4) coud aso easiy be modified to incude further side constraints, a different type of noninear measurements, or mutipe images or measurements, respectivey; we omit these extensions for simpicity. III. ALGORITHMIC FRAMEWORK Simiar to cassica dictionary earning [9], [], [], [3] and phase retrieva, probem 4) is nonconvex and difficut to sove. Therefore, we adopt an agorithm that provides monotonic decrease of the objective whie converging to a stationary point see Section III-D beow). The agorithmic framework we empoy is that of aternating minimization: For each variabe A, X and D in turn, we take one step towards soving 4) with respect to this variabe aone, keeping the other ones fixed. Each of these subprobems is convex in the remaining unfixed optimization variabe, and we-known efficient agorithms can be empoyed accordingy. We summarize our method in Agorithm, where the superscript denotes the adjoint operator for a matrix Z, Z is thus the conjugate transpose), R ) extracts the rea part of a compex-vaued argument, and denotes the Hadamard eement-wise) product of two matrices. The agorithm aso invoves the cassica soft-threshoding operator S τ Z) := max{0, Z τ} signz) and the Eucidean projection P X Z) := max{0, min{, Z}} onto X := [0, ] N N ; here, a operations are meant component-wise. To avoid training the dictionary on potentiay useess eary estimates, the agorithm performs two phases whie the iteration counter is smaer than K, the dictionary is not updated. Beow, we expain the agorithmic steps in more detai. Note that DOLPHIn actuay produces two distinct reconstructions of the desired image, namey X the per se image variabe ) and RDA) the image assembed from the sparsey coded patches). Our numerica experiments in Section IV show that in many cases, RDA) is in fact sighty or even significanty better than X with respect to at east one quantitative quaity measure and is therefore aso considered a possibe reconstruction output of Agorithm at east in the noisy setups we consider in this paper). Nevertheess, X is sometimes more visuay appeaing and can be used, for instance, to refine parameter settings if it differs strongy from RDA)) or to assess the infuence of the patch-based reguarization on the pure non-sparse Wirtinger Fow method corresponding to the formuation where λ and µ are set to zero. We discuss suitabe choices and sensitivity of the mode to these parameters in detai in Section IV-D. Technicay, RDA) might contain entries not in X, so one shoud project once more. Throughout, we often omit this step for simpicity; differences if any) between RDA) and P X RDA)) were insignificant in a our tests.

5 5 A. Updating the Patch Representation Vectors Updating A i.e., considering 4) with D and X fixed at their current vaues) consists of decreasing the objective p D) a i x i ) + λ ) µ a i, 5) i= which is separabe in the patches i =..., p. Therefore, we can update a vectors a i independenty and/or in parae. To do so, we choose to perform one step of the we-known agorithm ISTA see, e.g., [4]), which is a gradient-based method that is abe to take into account a non-smooth reguarizer such as the -norm. Concretey, the foowing update is performed for each i =,..., p: a i +) = S γ A λ/µ a i ) γa D ) D) a i ) xi )) ). 6) This update invoves a gradient descent step the gradient with respect to a i of the smooth term in each summand of 5) is D) D) a i ) ) xi, respectivey) foowed by soft-threshoding. Constructing A+) from the a i +) as specified above is equivaent to Step of Agorithm. The step size parameter γ A can be chosen in 0, /L A ), where L A is an upper bound on the Lipschitz constant of the gradient; here, L A = D ) D ) = D ) woud be appropriate, but a ess computationay demanding strategy is to use a backtracking scheme to automaticay update L A [4]. A technica subtety is noteworthy in this regard: We can either find one γ A that works for the whoe matrix-variabe update probem this is what is stated impicity in Step or we coud find different vaues, say γ a,i, for each coumn a i, i =,..., p, of A separatey. Our impementation does the atter, since it empoys a backtracking strategy for each coumn update independenty. B. Updating the Image Estimate With D = D ) and A = A +) fixed, updating X consists of decreasing the objective 4 Y FX) + µ F EX) DA 7) F with X X = [0, ] N N. This probem can be seen as a reguarized version of the phase retrieva probem with reguarization parameter µ) that encourages the patches of X to be cose to the sparse approximation DA obtained during the previous inner) iterations. Our approach to decrease the vaue of the objective 7) is by a projected gradient descent step. In fact, for µ = 0, this step reduces to the Wirtinger fow method [], but with necessary modifications to take into account the constraints on X rea-vauedness and variabe bounds [0, ]). The gradient of ϕx) := 4 Y FX) F with respect to X can be computed as ϕx) = R F FX) FX) Y) )), by using the chain rue. For ψx) := µ EX) DA F, the gradient is given by ψx) = µe EX) DA ) = µr R EX) DA ), where R is an N N matrix whose entries r ij equa the number of patches the respective pixe x ij is contained in. Note that if the whoe image is divided into a compete set of nonoverapping patches, R wi just be the a-ones matrix; otherwise, the eement-wise mutipication with R undoes the averaging of pixe vaues performed by R when assembing an image from overapping patches. Finay, the gradient w.r.t. X of the objective in 7) is ϕx)+ ψx) R N N, and the update in Step 3 of Agorithm is indeed shown to be a projected gradient descent step. Typicay, a backtracking ine search) strategy is used for choosing the step size γ X ; see Theorem in Section III-D for a seection rue that gives theoretica convergence, and aso Section IV-B for a heuristic aternative. C. Updating the Dictionary To update the dictionary, i.e., to approximatey sove 4) w.r.t. D aone, keeping X and A fixed at their current vaues, we empoy one pass of a bock-coordinate descent BCD) agorithm on the coumns of the dictionary [3]. The objective to decrease may be written as p Da i +) xi +) s.t. D D, 8) i= and the update rue given by Steps 4 3 corresponds 3 to one iteration of [, Agorithm ] appied to 8). 3 In [, Ago. ], and in our impementation, we simpy normaize the coumns of D; it is easiy seen that any soution with d j < for some j is suboptima w.r.t. 4)) since raising it to aows to reduce coefficients in A and thus to improve the -term of the DOLPHIn objective 4). However, using the projection is more convenient for proving the convergence resuts without adding more technica subteties w.r.t. this aspect.

6 6 To see this, note that each coumn update probem has a cosed-form soution: d j p n ) +) = P p a i j x i a i kd k) = i= ai j ) i= k= k j max{, w j q j w } j q j) with w j := i ai j ) and q j := i ai j x i k j ai k dk) ; here, we abbreviated a i := a i +), xi := x i +). If w j = 0, and thus a i j = 0 for a i =,..., p, then coumn dj is not used in any of the current patch representations; in that case, the coumn s update probem has a constant objective and is therefore soved by any d with d, e.g., a normaized random vector as in Step of Agorithm. The computations performed in Steps 8 3 of Agorithm are equivaent to these soutions, expressed differenty using the matrices B and C defined there. Note that the operations coud be paraeized to speed up computation. D. Convergence of the Agorithm As mentioned at the beginning of this section, with appropriate step size choices, DOLPHIn Agorithm ) exhibits the property of monotonicay decreasing the objective function vaue 4) at each iteration. In particuar, many ine-search type step size seection mechanisms aim precisey at reducing the objective; for simpicity, we wi simpy refer to such subroutines as suitabe backtracking schemes beow. Concrete exampes are the ISTA backtracking from [4, Section 3] we can empoy in the update of A, or the rue given in Theorem for the projected gradient descent update of X a different choice is described in Section IV-B); further variants are discussed, e.g., in [3]. Proposition : Let A ), X ), D ) ) be the current iterates after the -th iteration) of Agorithm with step sizes γ X and γ A determined by suitabe backtracking schemes or arbitrary 0 < γ A < / D ) D ), resp.) and et f i,j,k denote the objective function vaue of the DOLPHIn mode 4) at A i), X j), D k) ). Then, DOLPHIn either terminates in the + )-th iteration, or it hods that f +,+,+ f,,. Proof: Since we use ISTA to update A, it foows from [4] that f +,, f,,. Simiary, a suitabe backtracking strategy is known to enforce descent in the projected gradient method when the gradient is ocay Lipschitz-continuous, whence f +,+, f +,,. Finay, f +,+,+ f +,+, foows from standard resuts for BCD methods appied to convex probems, see, e.g., [33]. Combining these inequaities proves the caim. The case of termination in Proposition can occur when the backtracking scheme is combined with a maxima number of tria steps, which are often used as a safeguard against numerica stabiity probems or as a heuristic stopping condition to terminate the agorithm if no sufficient) improvement can be reached even with tiny step sizes. Note aso that the assertion of Proposition triviay hods true if a step sizes are 0; naturay, a true descent of the objective requires a stricty positive step size in at east one update step. In our agorithm, step size positivity can aways be guaranteed since a these updates invove objective functions whose gradient is Lipschitz continuous, and backtracking essentiay finds step sizes inversey proportiona to oca) Lipschitz constants. Due to non-expansiveness, the projection in the X-update poses no probem either). Adopting a specific Armijo-type step size seection rue for the X-update aows us to infer a convergence resut, stated in Theorem beow. To simpify the presentation, et f XX) := 4 Y FX) F + µ EX) D )A +) F and ΓX := f XX )) cf. ϕx) + ψx) in Section III-B). Theorem : Let η 0, ) and η > 0. Consider the DOLPHIn variant consisting of Agorithm with K = and the foowing Armijo rue to be used in Step 3: Determine γ X as the argest number in { ηη k } k=0,,,... such that f XP X X ) γ XΓX )) f XX )) P X X ) γ XΓX ) X ) F and set X +) := P X X ) γ XΓX ). γ X If µ and, for some 0 < ν ν, it hods that ν γ X, γa or γ a,i ), p i= ai ) ) j ν for a and j and i), then every accumuation point of the sequence {A ), X ), D ) )} =0,,,... of DOLPHIn iterates is a stationary point of probem 4). Proof: The proof works by expressing Agorithm as a specific instantiation of the coordinate gradient descent CGD) method from [34] and anayzing the objective descent achievabe in each update of A, X and D, respectivey. The technica detais make the rigorous forma proof somewhat engthy; therefore, we ony sketch it here and defer the fu proof to Appendix B. The CGD method works by soving subprobems to obtain directions of improvement for bocks of variabes at a time in our case, the coumns of) A, the matrix X, and the coumns of D correspond to such bocks and then taking steps aong these directions. More specificay, the directions are generated using a suitaby parameterized) stricty convex quadratic approximation of the objective buit using the gradient). Essentiay due to the strict convexity, it is then aways possibe to make a positive step aong such a direction that decreases the origina) objective, uness stationarity aready hods. Using a certain Armijo ine-search rue designed to find such positive step sizes which achieve a sufficient objective reduction, [34, Theorem ] ensures under mid further assumptions, which in our case essentiay transate to the stated boundedness requirement of the step sizes) that every accumuation point of the iterate sequence is indeed a stationary point of the addressed bock-separabe) probem.

7 7 To embed DOLPHIn into the CGD framework, we can interpret the difference between one iterate and the next w.r.t. the variabe bock under consideration) as the improvement direction, and proceed to show that we can aways choose a step size equa to in the Armijo-criterion from [34] cf. 9) and 46) therein). For this to work out, we need to impose sighty stricter conditions on other parameters used to define that rue than what is needed in [34]; these conditions are derived directy from known descent properties of the D- and A-updates of our method essentiay, ISTA descent properties as in [4]). That way, the D- and A-updates automaticay satisfy the specific CGD Armijo rue, and the actua backtracking scheme for the X-update given in the present theorem can be shown to assert that our X-update does so as we. The step sizes used in DOLPHIn coud aso be reinterpreted in the CGD framework as scaing factors of diagona Hessian approximations of the combined objective to be used in the direction-finding subprobems. With such simpe Hessian approximations, the obtained directions are then indeed equivaent to the iterate-differences resuting from the DOLPHIn update schemes.) The caim then foows directy from [34, Theorem e) and its extensions discussed in Section 8)]. A more forma expanation for why the step sizes can be chosen positive in each step can be found on page 39 of [34]; the boundedness of approximate Hessians is stated in [34, Assumption ]. Arguaby, assuming step sizes are bounded away from zero by a constant may become probematic in theory imagine an Armijo-generated step size sequence converging to zero), but wi not pose a probem in practice where one aways faces the imitations of numerica machine precision. Note aso that, in practice, the number of ine-search trias can be effectivey reduced by choosing η based on the previous step size [34].) Our impementation uses a different backtracking scheme for the X-update see Section IV-B) that can be viewed as a cheaper heuristic aternative to the stated Armijo-rue which sti ensures monotonic objective descent and hence is suitabe in the context of Proposition ), aso enabes stricty positive steps, and empiricay performs equay we. Finay, we remark that the condition µ in Theorem can be dropped if the reevant objective parts of probem 4) are not rescaed for the A- and D-updates, respectivey. To concude the discussion of convergence, we point out that one can obtain a inear rate of convergence for DOLPHIn with the Armijo rue from Theorem, by extending the resuts of [34, Theorem cf. Section 8)]. IV. NUMERICAL RESULTS In this section, we discuss various numerica experiments to study the effectiveness of the DOLPHIn agorithm. To that end, we consider severa types of inear operators F within our mode 4) namey, different types of Gaussian random operators and coded diffraction modes). Detais on the experimenta setup and our impementation are given in the first two subsections, before presenting the main numerica resuts in Subsection IV-C. Our experiments demonstrate that with noisy measurements, DOLPHIn gives significanty better image reconstructions than the Wirtinger Fow method [], one recent state-of-the-art phase retrieva agorithm, thereby showing that introducing sparsity via a simutaneousy) earned dictionary is indeed a promising new approach for signa reconstruction from noisy, phaseess, noninear measurements. Furthermore, we discuss sensitivity of DOLPHIn with regard to various hyper-)parameter choices Subsections. IV-D, IV-E and IV-F) and a variant in which the -reguarization term in the objective is repaced by expicit constraints on the sparsity of the patch-representation coefficient vectors a i Subsection. IV-G). A. Experimenta Setup We consider severa inear operators F corresponding to different types of measurements that are cassica in the phase retrieva iterature. We denote by F the normaized) D-Fourier operator impemented using fast Fourier transforms), and introduce two compex Gaussian matrices G C M N, H C M N, whose entries are i.i.d. sampes from the distribution N 0, I/) + in 0, I/). Then, we experiment with the operators FX) = GX, FX) = GXG, FX) = GXH, and the coded diffraction pattern mode F M X ) FX) =. F M m X ), F Z) = m Mj F Z j ) ), 9) where Z j := Z {j )N+,...,jN }, i.e., Z = Z,..., Z m)) and the M j s are admissibe coded diffraction patterns CDPs), see for instance [, Section 4.]; in our experiments we used ternary CDPs, such that each M j is in {0, ±} N N. Later, we wi aso consider octanary CDPs with M j {± /, ±i /, ± 3, ±i 3} C N N.) To reconstruct ˆX, we choose an oversamping setting where M = 4N, M = 4N and/or m =, respectivey. Moreover, we corrupt our measurements with additive white Gaussian noise N such that SNRY, F ˆX) +N) = 0 db for the Gaussiantype, and 0 db for CDP measurements, respectivey. Note that these settings yied, in particuar, a reativey heavy noise eve for the Gaussian cases and a reativey ow oversamping ratio for the CDPs. j=

8 8 Fig.. a) b) c) d) e) f) g) h) Test images. a) c): cameraman, house and peppers size 56 56). d) h): ena, barbara, boat, fingerprint and mandri size 5 5). B. Impementation Detais We choose to initiaize our agorithm with a simpe random image X0) in X to demonstrate the robustness of our approach with respect to its initiaization. Nevertheess, other choices are possibe. For instance, one may aso initiaize X0) with a power-method scheme simiar to that proposed in [], modified to account for the rea-vauedness and box-constraints. The dictionary is initiaized as D0) = I, FD ) in Rs s, where FD corresponds to the two-dimensiona discrete cosine transform see, e.g., [0]). To update A, we use the ISTA impementation from the SPAMS package4 [3] with its integrated backtracking ine search for LA ). Regarding the step sizes γ`x for the update of X Step 3 of Agorithm ), we adopt the foowing simpe strategy, which is simiar to that from [4] and may be viewed as a heuristic to the Armijo rue from Theorem : Whenever the gradient step eads to a reduction in the objective function vaue, we accept it. Otherwise, we recompute the step with γ`x haved unti a reduction is achieved; here, as a safeguard against numerica issues, we impemented a imit of 00 trias forcing termination in case a were unsuccessfu), but this was never reached in any of our computationa experiments. Regardess of whether γ`x was reduced or not, we reset its vaue to.68γ`x for the next round; the initia step size is γ0x = 04 /f0), where f0) is the objective function of the DOLPHIn mode 4), evauated at X0), D0) and east-squares patch representations arg mina kex0) ) D0) AkF. Note that, whie this rue deviates from the theoretica convergence Theorem, Propositon and the remarks foowing it remain appicabe.) Finay, we consider nonoverapping 8 8 patches and run DOLPHIn Agorithm ) with K = 5 and K = 50; the reguarization/penaty parameter vaues can be read from Tabe I there, my is the number of eements of Y). We remark that these parameter vaues were empiricay benchmarked to work we for the measurement setups and instances considered here; a discussion about the stabiity of our approach with respect to these parameter choices is presented beow in Section IV-D. Further experiments with a sparsity-constrained DOLPHIn variant and using overapping patches are discussed in Section IV-G. Our DOLPHIn code is avaiabe onine on the first author s webpage5. C. Computationa Experiments We test our method on a coection of typica grayscae) test images used in the iterature, see Figure. A experiments were carried out on a Linux 64-bit quad-core machine.8 GHz, 8 GB RAM) running Matab R06a singe-thread). We evauate our approach with the foowing question in mind: Can we improve upon the quaity of reconstruction compared to standard phase retrieva agorithms? Standard methods cannot expoit sparsity if the underying basis or dictionary is timann/

9 9 TABLE I TEST RESULTS FOR m Y GAUSSIAN-TYPE AND CODED DIFFRACTION PATTERN CDP) MEASUREMENTS. WE REPORT MEAN VALUES GEOMETRIC MEAN FOR CPU TIMES) PER MEASUREMENT TYPE, OBTAINED FROM THREE INSTANCES WITH RANDOM X 0) AND RANDOM NOISE FOR EACH OF THE THREE AND FIVE 5 5 IMAGES, W.R.T. THE RECONSTRUCTIONS FROM DOLPHIN X DOLPHIN AND P X RDA))) WITH PARAMETERS µ, λ) AND REAL-VALUED, [0, ]-CONSTRAINED) WIRTINGER FLOW X WF ), RESPECTIVELY. CPU TIMES IN SECONDS, PSNR IN DECIBELS) instances 5 5 instances F type reconstruction µ, λ)/m Y time PSNR SSIM a i 0 µ, λ)/m Y time PSNR SSIM a i 0 G ˆX X DOLPHIn 0.5,0.05) ,0.05) P X RDA)) X WF G ˆXG X DOLPHIn 0.5,0.0) ,0.0) P X RDA)) X WF G ˆXH X DOLPHIn 0.5,0.0) ,0.0) P X RDA)) X WF CDP cf. 9)) X DOLPHIn 0.05,0.003) ,0.003) P X RDA)) X WF a) b) c) d) Fig.. DOLPHIn exampe: Image origina is the 5 5 fingerprint picture, measurements are noisy Gaussian G ˆX M = 4N, noise-snr 0 db), µ, λ) = 0.5, 0.05)m Y. a) fina dictionary excerpt, magnified), b) image reconstruction X DOLPHIn, c) image reconstruction RDA) from sparsey coded patches, d) reconstruction X WF after 75 WF iterations. Fina PSNR vaues:.37 db for RDA), 3.74 db for X DOLPHIn, 8.9 db for X WF ; fina SSIM vaues: for RDA), 0.85 for X DOLPHIn, for X WF ; average a i 0 is unknown; as we wi see, the introduced patch-) sparsity indeed aows for better recovery resuts at east in the oversamping and noise regimes considered here). To evauate the achievabe sparsity, we ook at the average number of nonzeros in the coumns of A after running our agorithm. Generay, smaer vaues indicate an improved suitabiity of the earned dictionary for sparse patch coding high vaues often occur if the reguarization parameter λ is too sma and the dictionary is earning the noise, which is something we woud ike to avoid). To assess the quaity of the image reconstructions, we consider two standard measures, namey the peak signa-to-noise ratio PSNR) of a reconstruction as we as its structura simiarity index SSIM) [35]. For PSNR, arger vaues are better, and SSIM-vaues coser to aways ranging between 0 and ) indicate better visua quaity. Tabe I dispays the CPU times, PSNR- and SSIM-vaues and mean patch representation vector sparsity eves obtained for the various measurement types, averaged over the instance groups of the same size. The concrete exampes in Figures and 3 show the resuts from DOLPHIn and pain Wirtinger Fow WF; the rea-vaued, [0, ]-box constrained variant, which corresponds to running Agorithm with µ = 0 and omitting the updates of A and D). In a tests, we et the Wirtinger Fow method run for the same number of iterations 75) and use the same starting points as for the DOLPHIn runs. Note that instead of random X 0), we coud aso use a spectra initiaization simiar to the one proposed for the unconstrained) Wirtinger Fow agorithm, see []. Such initiaization can improve WF reconstruction at east in the noiseess case), and may aso provide better initia estimates for DOLPHIn. We have experimented with such a spectra approach and found the resuts comparabe to what is achievabe with random X 0), both for WF and DOLPHIn. Therefore, we do not report these experiments in the paper. The DOLPHIn method consistenty provides better image reconstructions than WF, which ceary shows that our approach successfuy introduces sparsity into the phase retrieva probem and expoits it for estimating the soution. As can be seen from Tabe I, the obtained dictionaries aow for rather sparse representation vectors, with the effect of making better use of the information provided by the measurements, and aso denoising the image aong the way. The atter fact can be seen

10 0 a) b) c) Fig. 3. DOLPHIn exampe: Image origina is a 86 photo of the Wadspirae buiding in Darmstadt, Germany; measurements are noisy CDPs obtained using two ternary masks), noise-snr 0 db, µ, λ) = 0.05, 0.007)mY. a) image reconstruction XDOLPHIn, b) image reconstruction RDA) from sparsey coded patches, c) reconstruction XWF after 75 WF iterations. Fina PSNR vaues: 3.40 db for RDA), 4.7 db for XDOLPHIn,.63 db for XWF ; fina SSIM vaues: for RDA), for XDOLPHIn, for XWF ; average kai k0 is.8. Tota reconstruction time roughy 30 min DOLPHIn) and 0 min WF), resp.) Origina image taken from Wikimedia Commons, under Creative Commons Attribution-Share Aike 3.0 Unported icense. a) b) c) d) Fig. 4. DOLPHIn exampe Wadspirae image, zoomed-in pixe parts magnified). a) origina image, b) image reconstruction XDOLPHIn, b) reconstruction RDA) from sparsey coded patches, c) reconstruction XWF after 75 WF iterations. The sight bock artefacts visibe in b) and c) are due to the nonoverapping patch approach in experiments and coud easiy be mitigated by introducing some patch overap cf., e.g., Fig. 6). in the exampes Fig. and 3, see aso Fig. 4) and aso inferred from the significanty higher PSNR and SSIM vaues for the estimates XDOLPHIn and RDA) or PX RDA)), resp.) obtained from DOLPHIn compared to the reconstruction XWF of the WF agorithm which cannot make use of hidden sparsity). The gain in reconstruction quaity is more visibe in the exampe of Fig. 3 cf. Fig. 4) than for that in Fig., though both cases assert higher quantitative measures. Furthermore, note that DOLPHIn naturay has higher running times than WF, since it performs more work per iteration aso, different types of measurement operators require different amounts of time to evauate). Note aso that storing A and D instead of an actua image X such as the WF reconstruction) requires saving ony about haf as many numbers incuding integer index pairs for the nonzero entries in A). As indicated earier, the reconstruction RDA) is quite often better than XDOLPHIn w.r.t. at east one of either PSNR or SSIM vaue. Nonetheess, XDOLPHIn may be visuay more appeaing than RDA) even if the atter exhibits a higher quantitative quaity measure as is the case, for instance, in the exampe of Figures 3 and 4); Furthermore, occasionay XDOLPHIn achieves notaby better quantitative) measures than RDA); an intuitive expanation may be that if, whie the sparse coding of patches served we to eiminate the noise and by means of the patch-fit objective term to successfuy push the X-update steps toward a soution of good quaity, that soution eventuay becomes so good, then the fact that RDA) is designed to be ony an approximation of X) predominates. On the other hand, XDOLPHIn is sometimes very cose to XWF, which indicates a suboptima setting of the parameters µ and λ that contro how much feedback the patch-fitting objective term introduces into the Wirtinger-Fow-ike X-update in the DOLPHIn agorithm. We discuss parameter choices in more detai in the foowing subsection. D. Hyperparameter Choices and Sensitivity The DOLPHIn agorithm requires severa parameters to be specified a priori. Most can be referred to as design parameters; the most prominent ones are the size of image patches s s ), whether patches shoud overap or not not given a name here), and the number n of dictionary atoms to earn. Furthermore, there are certain agorithmic parameters in a broad sense) that need to be fixed, e.g., the iteration imits K and K or the initia dictionary D0) and image estimate X0). The arguaby most important parameters, however, are the mode or reguarization parameters µ and λ. For any fixed combination of design and agorithmic parameters in a certain measurement setup fixed measurement type/mode and assumed) noise eve), it is conceivabe that one can find some vaues for µ and λ that work we for most instances, whie the converse choosing, say, iteration imits for fixed µ, λ and other parameters is ceary not a very practica approach.

11 max SSIM average a i 0 λ yieding max SSIM scaed by /my) λ yieding max SSIM scaed by /my) λ/my a) inf noise SNR b) number of CDPs c) max SSIM average a i 0 λ yieding max SSIM scaed by /my) λ yieding max SSIM scaed by /my) λ/my d) 0 0 inf noise SNR e) oversamping factor f) Fig. 5. Infuence of parameter λ on best achievabe SSIM vaues for reconstructed images and sensitivity w.r.t. samping ratios and noise eves, for different measurement types. Fixed other parameters: µ = 0.m Y, K = 5, K = 50, s = s = 8 nonoverapping patches), D 0) = I, F D ), X 0) X random. a) c): Averages over reconstructions from ternary CDP measurements of the three images. The pots show a) the best achievabe SSIM for λ/m Y {0.000, 0.000,..., 0.0} eft vertica axis, soid ines) and average patch-sparsity of corresponding soutions right vertica axis, dashed ines) for noise eve SNRY, F ˆX) ) = 0 db and b) choice of λ yieding best SSIM for different noise eves, for number of masks back), 5 bue), 0 red) or 0 green), respectivey; c) choice of λ to achieve best SSIM for different number of masks, for noise-snrs 0 back), 5 bue), 0 red), 30 yeow), 50 ight bue) and green), respectivey. d) f): Averages over reconstructions from Gaussian measurements Y = G ˆX ) of the five 5 5 images. The pots dispay the same kind of information as a) c), but in d) with λ/m Y {0.000, 0.005, 0.00,..., 0.095} for noise-snr 5 db and in e) for different noise eves, for samping ratios M /N = back), 4 bue) and 8 red), respectivey; and in f) with M /N = for noise-snrs 0 back), 5 bue), 0 red), 30 green) and yeow). As is common for reguarization parameters, a good genera-purpose way to choose µ and λ a priori is unfortunatey not known. To obtain the specific choices used in our experiments, we fixed a the other parameters incuding noise SNR and oversamping ratios), then for each measurement mode) ran preiminary tests to identify vaues µ for which good resuts coud be produced with some λ, and finay fixed µ at such vaues and ran extensive benchmark tests to find λ vaues that give the best resuts. For DOLPHIn, µ offers some contro over how much feedback from the current sparse approximation of the current image estimate is taken into account in the update step to produce the next image iterate overy arge vaues hinder the progress made by the Wirtinger-Fow-ike part of the X-update, whie too sma vaues marginaize the infuence of the approximation RDA), with one consequence being that the automatic denoising feature is essentiay ost. Nevertheess, in our experiments we found that DOLPHIn is not strongy sensitive to the specific choice of µ once a certain regime has been identified in which one is abe to produce meaningfu resuts for some choice of λ). Hence, λ may be considered the most important parameter; note that this intuitivey makes sense, as λ contros how strongy sparsity of the patch representations is actuay promoted, the expoitation of which to obtain improved reconstructions being the driving purpose behind our deveopment of DOLPHIn. Figure 5 iustrates the sensitivity of DOLPHIn with respect to λ, in terms of reconstruction quaity and achievabe patchsparsity, for different noise eves, and exampary measurement types and probem sizes. In this figure, image quaity is measured by SSIM vaues aone; the pots using PSNR instead ook very simiar and were thus omitted. Note, however, that the parameters λ yieding the best SSIM and PSNR vaues, respectivey, need not be the same.) As shown by a) and d), there is a cear correation between the best reconstruction quaity that can be achieved in noisy settings) and the average sparsity of the patch representation vectors a i. For arger noise, ceary a arger λ is needed to achieve good resuts see b) and e) which shows that a stronger promotion of patch-sparsity is an adequate response to increased noise, as is known for inear sparsity-based denoising as we. Simiary, increasing the number of measurements aows to pick a smaer λ whatever the noise eve actuay is, as can be seen in c) and f), respectivey. The dependence of the best λ on the noise eve appears

12 TABLE II TEST RESULTS FOR DOLPHIN VARIANTS WITH DIFFERENT COMBINATIONS OF INNER ITERATION NUMBERS a AND d FOR THE A- AND D-UPDATES, RESP. REPORTED ARE THE BEST MEAN VALUES ACHIEVABLE VIA EITHER X DOLPHIN OR P X RDA))) WITH ANY OF THE CONSIDERED COMBINATIONS a, d) {, 3, 5} {, 3, 5} FIRST ROWS FOR EACH MEASUREMENT TYPE) AND THE WORST AMONG THE BEST VALUES FOR EACH COMBINATION SECOND ROWS), ALONG WITH THE RESPECTIVE COMBINATIONS YIELDING THE STATED VALUES. ALL OTHER PARAMETERS ARE IDENTICAL TO THOSE USED IN THE EXPERIMENTS FOR DEFAULT DOLPHIN a = d = ), CF. TABLE I instances 5 5 instances F type time PSNR SSIM a i 0 time PSNR SSIM a i 0 G ˆX.6, 3) 4.7, 5) , 5) 3.69, 5) 68.8, 3) 4.43, 3) , 3) 6.5, 5) , 5) , 3) , 3) , ) , 5) , 3) , 5) , 5) G ˆXG 5.0, 3) 3.7, 5) , 5) 6.7 5, 5) , ) 3.44, 5) , ) , 5) , ) , 5) 0.788, 3) 7.55, 3) , 3) , ) , 3).6, 3) G ˆXH 50.95, 3) 3.68, ) 0.735, ) , {, 5}) , ) 3.44, {3, 5}) , 3) , 5) , ) , 3) , 3) 7.59, 3) , 5) , {3, 5}) , 3).57, 5) CDP cf. 9)) 8.56, ) 7.9 3, ) , ) 7.7, 3) 35.66, 3) , ) , 3).40, 3).40 5, 5) , 5) , ) , ) , 3) 7.33, {, 3}) 0.788, 5) , ) to foow an exponentia curve w.r.t. the reciproca SNR) which is dampened by the samping ratio, i.e., becoming ess steep and pronounced the more measurements are avaiabe, cf. b) and e). Indeed, again referring to the subpots c) and f), at a fixed noise eve the best λ vaues seem to decrease exponentiay with growing number of measurements. It remains subject of future research to investigate these dependencies in more detai, e.g., to come up with more or ess genera functiona) rues for choosing λ. E. Impact of Increased Inner Iteration Counts It is worth considering whether more inner iterations i.e., consecutive update steps for the different variabe bocks ead to further improvements of the resuts and / or faster convergence. In genera, this is an open question for bock-coordinate descent agorithms, so the choices are typicay made empiricay. Our defaut choices of a = ISTA iterations for the A- update Step in Agorithm ), x = projected gradient descent steps for the X-update Step 3) and d = iterations of the BCD scheme for the D-update Steps 4 3) primariy refect the desire to keep the overa iteration cost ow. To assess whether another choice might yied significant improvements, we evauated the DOLPHIn performance for a combinations of a {, 3, 5} and d {, 3, 5}, keeping a other parameters equa to the settings from the experiments reported on above. We aso tried these combinations together with an increased iteration count for the X-update, but aready for x = or x = 3 the resuts were far worse than with just projected gradient descent step; the reason can ikey be found in the fact that without adapting A and D to a modified X-iterate, the patch-fitting term of the objective tries to keep X cose to a then-unsuitabe estimate RDA) based on oder X-iterates, which apparenty has a quite notabe negative infuence on the achievabe progress in the X-update oop.) The resuts are summarized in condensed format in Tabe II, from which we can read off the spread of the best and worst resuts among the best ones achievabe with either X or P X RDA))) for each measurement-instance combination among a combinations a, d) {, 3, 5}. Fu tabes for each test run can be found aongside our DOLPHIn code on the first author s webpage.) As the tabe shows, the resuts are a quite cose; whie some settings ead to sparser patch representations, the overa quaity of the best reconstructions for the various combinations usuay differ ony sighty, and no particuar combination stands out ceary as being better than a others. In particuar, comparing the resuts with those in Tabe I, we find that our defaut settings provide consistenty good resuts; they may be improved upon with some other combination of a and d, but at east with the same tota iteration horizon K + K = 75), the improvements are often ony margina. Since the overa runtime reductions if any) obtained with other choices for a, d) are aso very sma, there does not seem to be a cear advantage to using more than a singe iteration for either update probem. F. Infuence of the First DOLPHIn Phase Our agorithm keeps the dictionary fixed at its initiaization for the first K iterations in order to prevent the dictionary from training on reativey useess first reconstruction iterates. Indeed, if a variabes incuding D are updated right from the beginning i.e., K = 0, K = 75), then we end up with inferior resuts keeping a other parameters unchanged): The obtained patch representations are much ess sparse, the quaity of the image estimate RDA) decreases drasticay, and aso the quaity of the reconstruction X becomes notaby worse. This demonstrates that the dictionary apparenty earns too much noise when updated from the beginning, and the positive effect of fitering out quite a ot of noise in the first phase by reguarizing with sparsey coded patches using a fixed initia dictionary is amost competey ost. To give an exampe, for the CDP testset on images, the average vaues obtained by DOLPHIn when updating aso the dictionary from the first iteration onward are: 9.4 seconds runtime versus 8.56 for defaut DOLPHIn, cf. Tabe I), mean patch sparsity a i vs.

13 3 TABLE III TEST RESULTS FOR SPARSITY-CONSTRAINED DOLPHIN, USING OVERLAPPING PATCHES, FOR m Y GAUSSIAN-TYPE AND CODED DIFFRACTION PATTERN CDP) MEASUREMENTS. REPORTED ARE MEAN VALUES GEOMETRIC MEAN FOR CPU TIMES) PER MEASUREMENT TYPE, OBTAINED FROM THREE INSTANCES WITH RANDOM X 0) AND RANDOM NOISE FOR EACH OF THE THREE AND FIVE 5 5 IMAGES, W.R.T. THE RECONSTRUCTIONS FROM DOLPHIN X DOLPHIN AND P X RDA))) WITH PARAMETERS µ, µ ) AND REAL-VALUED, [0, ]-CONSTRAINED) WIRTINGER FLOW X WF ), RESPECTIVELY. CPU TIMES IN SECONDS, PSNR IN DECIBELS) instances 5 5 instances F type reconstruction µ, µ )/m Y time PSNR SSIM a i 0 µ, µ )/m Y time PSNR SSIM a i 0 G ˆX X DOLPHIn 0.005,0.0084) ,0.0084) P X RDA)) X WF G ˆXG X DOLPHIn 0.005,0.0084) ,0.0084) P X RDA)) X WF G ˆXH X DOLPHIn 0.005,0.0084) ,0.0084) P X RDA)) X WF CDP cf. 9)) X DOLPHIn 0.005,0.0084) ,0.0084) P X RDA)) X WF ), PSNR 6.80 db and 8.88 db vs. 7.5 and 6.58) and SSIM-vaues and vs and ) for the reconstructions X and P X RDA)), respectivey. On the other hand, one might argue that if the first iterates are reativey worthess, the effort of updating A in the first DOLPHIn phase i.e., the first K iterations) coud be saved as we. However, the infuence of the objective terms invoving D and A shoud then be competey removed from the agorithm for the first K iterations; otherwise, the patch-fitting term wi certainy hinder progress made by the X-update because it then amounts to trying to keep X cose to the initia estimate RDA), which obviousy needs not bear any resembance to the sought soution at a. Thus, if both D and A are to be unused in the first phase, then one shoud temporariy set µ = 0, with the consequence that the first phase reduces to pure projected gradient descent for X with respect to the phase retrieva objective i.e., essentiay, Wirtinger Fow. Therefore, proceeding ike this simpy amounts to a different initiaization of X. Experiments with this DOLPHIn variant K = 5 initia WF iterations foowed by K = 50 fu DOLPHIn iterations incuding A and D; a other parameters again eft unchanged) showed that the achievabe patch-sparsities remain about the same for the Gaussian measurement types but become much worse for the CDP setup, and that the average PSNR and SSIM vaues become often ceary) worse in virtuay a cases. In the exampe of measurements G ˆX of the 5 5 test images, the above-described DOLPHIn variant runs 6.34 seconds on average as ess effort is spent in the first phase, this is naturay ower than the 68.6 seconds defaut DOLPHIn takes), produces sighty ower average patch-sparsity 5.88 vs for defaut DOLPHIn), but for both X and P X RDA)), the PSNR and SSIM vaues are notaby worse.3 db and 0.55 db vs. 4.4 db and.66 db, and and 0.59 vs and , resp.). The reason for the observed behavior can be found in the inferior performance of WF without expoiting patch-sparsity cf. aso Tabe I); note that the resuts aso further demonstrate DOLPHIn s robustness w.r.t. the initia point apparenty, the initia point obtained from some WF iterations is not more hepfu for DOLPHIn than a random first guess. Finay, it is aso worth considering what happens if the dictionary updates are turned off competey, i.e., K = 0. Then, DOLPHIn reduces to patch-sparsity reguarized Wirtinger Fow, a WF variant that apparenty was not considered previousy. Additiona experiments with K = 75, K = 0 and D = D 0) = I, F D ) other parameters eft unchanged) showed that this variant consistenty produces higher sparsity i.e., smaer average number of nonzero entries) of the patch representation coefficient vectors, but that the best reconstruction X or P X RDA))) is aways significanty inferior to the best one produced by our defaut version of DOLPHIn. The first observation is expained by the fact that with D fixed throughout, the patch coding A-update) ony needs to adapt w.r.t. new X-iterates but not a new D; at east if the new X is not too different from the previous one, the former representation coefficient vectors sti yied an acceptabe approximation, which no onger hods true if the dictionary was aso modified. Whie it shoud aso me mentioned that the patch-sparsity reguarized version performs much better than pain WF aready, the second observation ceary indicates the additiona benefit of working with trained dictionaries, i.e., superiority of DOLPHIn aso over the reguarized WF variant. Fu tabes containing resuts for a testruns reported on in this subsection are again avaiabe onine aong with our DOLPHIn code on the first author s website. G. Sparsity-Constrained DOLPHIn Variant From Tabe I and Figure 5, a) and d), it becomes apparent that a sparsity eve of 8±4 accompanies the good reconstructions by DOLPHIn. This suggests use in a DOLPHIn variant we aready briefy hinted at: Instead of using -norm reguarization,

14 4 we coud incorporate expicit sparsity constraints on the a i. The corresponding DOLPHIn mode then reads Y FX) F + µ EX) DA F min X,D,A 4 s.t. X [0, ] N N, D D, a i 0 k i =,..., p, 0) where k is the target sparsity eve. We no onger need to tune the λ parameter, and can et our previous experimenta resuts guide the choice of k. Note that the ony modification to Agorithm concerns the update of A Step ), which now requires soving or approximating min EX) DA s.t. a i F 0 k i =,..., p. In our impementation, we do so by running Orthogona Matching Pursuit OMP) [36] for each coumn a i of A separatey unti either the sparsity bound is reached or x i Da i ε, where we set a defaut of ε := 0.. The vaue of ε is again a parameter that might need thorough benchmarking; obviousy, it is reated to the amount of noise one wants to fiter out by sparsey representing the patches higher noise eves wi require arger ε vaues. The 0. defaut worked quite we in our experiments, but coud probaby be improved by benchmarking as we.) The OMP code we used is aso part of the SPAMS package. The effect of the parameter µ is more pronounced in the sparsity-constrained DOLPHIn than in Agorithm ; however, it appears its choice is ess dependent on the measurement mode used. By just a few experimenta runs, we found that good resuts in a our test cases can be achieved using K = K = 5 iterations, where in the first K with fixed dictionary), we use a vaue of µ = µ = 0.005m Y aong with a sparsity bound k = k = 4, and in the fina K iterations in which the dictionary is updated), µ = µ =.68µ = m Y and k = k = 8. Resuts on the same instances considered before using the same D 0) ) are presented in Tabe III, with the exception that for the CDP case, we used compex-vaued octanary masks cf. []) here; the initia image estimates and measurement noise were again chosen randomy. Note aso that for these test, we used compete sets of overapping patches. This greaty increases p and hence the number of subprobems to be soved in the A-update step, which is the main reason for the increased running times compared to Tabe I for the standard DOLPHIn method. It shoud however aso be mentioned that OMP requires up to k iterations per subprobem, whie in Agorithm we expicity restricted the number of ISTA iterations in the A-update to just a singe one.) A concrete exampe is given in Figure 6; here, we consider the coor mandri image, for which the reconstruction agorithms given just two quite heaviy noise-corrupted octanary CDP measurements) were run on each of the three RGB channes separatey. The sparsity-constrained DOLPHIn reconstructions appear superior to the pain WF soution both visuay and in terms of the quaity measures PSNR and SSIM. V. DISCUSSION AND CONCLUSION In this paper, we introduced a new method, caed DOLPHIn, for dictionary earning from noisy noninear measurements without phase information. In the context of image reconstruction, the agorithm fuses a variant of the recent Wirtinger Fow method for phase retrieva with a patch-based dictionary earning mode to obtain sparse representations of image patches, and yieds monotonic objective decrease or with appropriate step size seection) convergence to a stationary point for the nonconvex combined DOLPHIn mode. Our experiments demonstrate that dictionary earning for phase retrieva with a patch-based sparsity is a promising direction, especiay for cases in which the origina Wirtinger Fow approach fais due to high noise eves and/or reativey ow samping rates). Severa aspects remain open for future research. For instance, regarding the generay difficut task of parameter tuning, additiona benchmarking for to-be-identified instance settings of specia interest coud give further insights on how to choose, e.g., the reguarization parameters in reation to varying noise eves. It may aso be worth deveoping further variants of our agorithm; the successfu use of 0 -constraints instead of the - penaty, combining OMP with our framework, is just one exampe. Perhaps most importanty, future research wi be directed towards the cassic phase retrieva probem in which one is given the squared) magnitudes of Fourier measurements, see, e.g., [], [], [5]. Here, the WF method fais, and existing other projection-based) methods are not aways reiabe either. The hope is that introducing sparsity via a earned dictionary wi aso enabe improved reconstructions in the Fourier setting. To evauate the quaity of the earned dictionary, one might aso ask how DOLPHIn compares to the straightforward approach to first run standard) phase retrieva and then earn dictionary and sparse patch representations from the resut. Some preiminary experiments see aso those in Section IV-F pertaining to keeping both A and D fixed in the first DOLPHIn phase) indicate that both approaches produce comparabe resuts in the noisefree setting, whie our numerica resuts demonstrate a denoising feature of our agorithm that the simpe approach obviousy acks. Simiary, it wi be of interest to see if the dictionaries earned by DOLPHIn can be used successfuy within sparsity-aware methods e.g., the Threshoded WF proposed in [3], if that were modified to hande oca patch-based) sparsity instead of goba priors). In particuar, earning dictionaries for patch representations of images from a whoe cass of images woud then be an interesting point to consider. To that end, note that the DOLPHIn mode and agorithm can easiy be extended to mutipe

15 5 a) b) e) c) f) d) g) h) Fig. 6. Sparsity-Constrained DOLPHIn exampe: Image origina is the 5 5 RGB mandri picture, measurements are noisy CDPs obtained using two compex-vaued octanary masks, noise-snr 0 db, per coor channe), µ = my, µ = my other parameters as described in Section IV-G). D0) = I, FD ) for R-channe, fina dictionary then served as initia dictionary for G-channe, whose fina dictionary in turn was initia dictionary for B-channe; X0) X random for each channe. a) fina dictionary excerpt) for B-channe b) image reconstruction XDOLPHIn, c) image reconstruction RDA) from sparsey coded patches, d) reconstruction XWF after 50 WF iterations. Fina PSNR vaues: 0.53 db for RDA), 0.79 db for XDOLPHIn, 4.47 db for XWF ; fina SSIM vaues: for RDA), 0.54 for XDOLPHIn, 0.96 for XWF ; average kai k0 is Means over a coor channes.) e) h): zoomed-in pixe parts magnified) of e) origina image, f) XDOLPHIn, g) RDA) and h) XWF. input images whose patches are a to be represented using a singe dictionary by summing up the objectives for each separate image, but with the same D throughout. Another interesting aspect to evauate is by how much reconstruction quaity and achievabe sparsity degrade due to the oss of phase or, more generay, measurement noninearity), compared to the inear measurement case. A PPENDIX A O NE -D IMENSIONAL DOLPHI N In the foowing, we derive the DOLPHIn agorithm for the one-dimensiona setting ). In particuar, the gradient) formuas for the D-case can be obtained by appying the ones given beow to the vectorized image x = vecx) stacking coumns of X on top of each other to form the vector x), the vectorized matrix a = veca), and interpreting the matrix F CM N as describing the inear transformation corresponding to F in terms of the vectorized variabes. We now have a patch-extraction matrix Pe Rps N which gives us Pe x = x )>,..., xp )> )> in the vectorized Dcase, xi then is the vectorized i-th patch of X, i.e., Pe corresponds to E). Simiary, we have a patch-reassemby matrix Pa RN ps ; then, the reassembed signa vector wi be Pa a )> D>,..., ap )> D> )> so Pa corresponds to R). Note > that Pa = P e = P> Pe ; in particuar, x = Pa Pe x, and P> e Pe e Pe is a diagona matrix for which each diagona entry is associated to a specific vector component and gives the number of patches this component is contained in. Thus, if x = vecx) is a vectorized D-image, P> X) with R as defined in Section III-B.) Note that for nonoverapping patches, e Pe x = vecr > Pe is simpy a permutation matrix, and Pa = P> e so Pa Pe = I). Aso, appying just Pe actuay reassembes a signa from patches by simpy adding the component s vaues without averaging. We wish to represent each patch as xi Dai with sparse coefficient vectors ai ; with a := a )>,..., ap )> )> and D := Ip D, this sparse-approximation reation can be written as Pe x D a. Our mode to tacke the D-probem ) reads min x,d,α s.t. 4 y Fx + x X := [0, ]N, µ Pe x D a + λkak D D; ) here, y := Fx + n, with x the origina signa we wish to recover and n a noise vector. The update formuas for a separatey for a,..., ap ) and D remain the same as described before, see Sections III-A and III-C, respectivey. However, the update probem for the phase retrieva soution now derived from ), with D and a fixed at their current vaues becomes decreasing the objective 4 y Fx + µ Pe x D a with x X. )