Nature Methos Bayesian localization microscopy reveals nanoscale poosome ynamics Susan Cox, Ewar Rosten, James Monypenny, Tijana Jovanovic-Talisman, Dylan T Burnette, Jennifer Lippincott-Schwartz, Gareth E Jones & Rainer Heintzmann Supplementary File Supplementary Figure Supplementary Figure 2 Supplementary Figure 3 Supplementary Figure 4 Supplementary Note Title Blinking of fluorophores in fixe poosome sample Vinculin filament bining angles in poosomes Linescans showing resolution of 3B imaging in live cell samples. State transition iagram for our moel of a fluorophore Detaile escription of 3B algorithm Note: Supplementary Vieos -7 an Supplementary Software are available on the Nature Methos website.
2 Supplementary Figure : The two graphs show a time-scan, each in a region with an isolate fluorophore, through a single pixel of the ataset use in Figure 2c g. Single frame images of the corresponing fluorophore are shown next to each graph. Only the last 200 frames of the 500 frame ataset (which were usually iscare) were use in orer to isolate iniviual fluorophores. The upper graph shows a im, slowly blinking fluorophore with three appearances. The fluorophore is localise to a stanar eviation of 3 nm using 9 frames (threshole about 6200). The lower graph shows a bright fluorophore with two brief appearances. The fluorophore is localise to a stanar eviation of 9.5 nm using 6 frames (threshole above 6400). 6600 Spot 6400 6200 Intensity 6000 5800 5600 5400 5200 300 350 400 450 500 Frame number 7400 7200 Spot 2 7000 6800 6600 Intensity 6400 6200 6000 5800 5600 5400 5200 300 350 400 450 500 Frame number
3 Supplementary Figure 2 : We have compare the istribution of bining angles at junctions between vinculin strans in images obtaine using our 3B analysis technique to the angles foun using STED. The STED ata has been taken from (M. Wale, J.M., G.E.J., R.H., S.C., Unpublishe ata), which also escribes the metho use to measure the angles. From the STED ata,,780 angles were measure an from the localization ata, 338 were measure. As can be seen, the results from the two microscopy techniques are in agreement, but the higher resolution of our technique prouces a sharper peak in the angle istribution. 0.02 STED 3B Analysis Probability ensity 0.05 0.0 0.005 0 0 50 00 50 Angle (egrees)
4 Supplementary Figure 3 : Linescans showing resolution of 3B imaging in live cell samples. (a), (b) two images of a poosome 20 secons apart (see Figure 3a) with (c), () linescans. (e), (f) two poosomes (see Figure 4) with linescans (g), (h). Structures aroun 50nm apart can be seen in linescans (c), (g) an (h). Scalebars are 200 nm. a b c 0.8 a 0.8 b Density (AU) 0.6 0.4 Density (AU) 0.6 0.4 0.2 0.2 0 50 00 50 200 250 Distance (nm) 0 50 00 50 200 250 Distance (nm) e f g e h 0.8 Density (AU) 0.6 0.4 0.2 0 50 00 50 200 250 Distance (nm) Density (AU) 0.8 0.6 0.4 0.2 f 0 50 00 50 200 250 Distance (nm)
5 Supplementary Figure 4 : State transition iagram for our moel of a fluorophore. P P 4 6 P State 0 emitting P 3 State not emitting State 2 bleache P 2 P 5
6 Supplementary Note : Detaile escription of metho. Moelling the image Emitting fluorophores are moele as proucing a Gaussian shape spot in the image. This spot has four continuous variables: the position (x,y), the raius, an the brightness (integrate intensity). We assume that fluorophores can occur anywhere in the region being analyze with equal probability. Size an brightness are taken to be inepenent an prior istribution over the variables is set to log-normal, as this is a convenient istribution which prevents negative values. We normalize images by high-pass filtering with large Gaussian kernel to remove the backgroun offset followe by scaling the image to set the stanar eviation of intensity to one. This is possible because the images that we have analyse have very large areas containing only large amounts of out of focus light, which means that there are only small errors in the estimate of the noise variance. If the image is mostly foregroun, then the noise variance can be estimate using manually selecte backgroun regions (which inclue the still present out-of-focus fluorescence). The prior on brightness is applie relative to the normalize image. Image noise is moele as Gaussian with zero mean an the stanar eviation (σ) of, since the backgroun level is high. Inference Below, we use the convention common in the Bayesian machine learning literature that P( ) enotes both continuous an iscrete probability istributions an the symbol is use in both the iscrete an continuous case. A complete list of symbols is given at the en of this note. Consier the case of eciing to place a fluorophore, F, versus keeping the null hypothesis that no fluorophore is present, N, given the ata, D. We woul like to etermine the relative probabilities P(F D) P(N D) an using Bayes rule we obtain the Bayesian moel comparison equation: P(F D) P(N D) = P(D F)P(F) P(D N)P(N). Since P(F) an P(N) are constants, we nee to calculate P(D F) an P(D N). The latter is trivial as it is just the probability of observing all pixels given the noise moel. The former is compute by evaluating the moel evience which is compute using the marginalization: P(D F) = P(D,a,b F)b a, () a R 4 b Z N 3 where a are the four continuous parameters, b is the fluorophore state in each frame, an N is the number of frames. For a single fluorophore, the integration over b can be performe with the forwar algorithm 24. We perform the integral over the continuous parameters using using Laplace s Approximation 28 which involves fining a MP, which is the MAP (maximum a posteriori) estimate of a. We also require the MAP probability, P(D,a MP F), an the Hessian, log P(D,a F) amp, We fin a MP using nonlinear conjugate graient 29. We nee to compare moels with ifferent numbers of fluorophores. For a moel with M fluorophores, F M, a now has 4M continuous parameters: four per fluorophore. Also, b = {b,,b M }, where b i Z N 3 is the state sequence for the ith fluorophore. Equation becomes P(D F M ) = a R 4M b Z MN 3 P(D,a,b F M )b a, Exact integration over state sequences rapily becomes intractable with increasing M so we perform integration using MCMC (Markov Chain Monte Carlo) sampling. The iscrete states are sample using Gibbs sampling 30 since we can sample from Markov chains efficiently using forwar filtering backwars sampling 3. For convenience, we now make the efinitions P(D,a,b F M ) = P (a,b) = e E(a,b), where E is the log probability. Also, we efine Z = P(D F M ) = P (a,b)a b, Z a = P (a,b)b = P(a,b). We use Laplace s approximation to integrate over a. Recall that one requirement is computation of Z amp which requires optimization over a to fin a MP. The latter is compute using the MCMC technique of slow growth thermoynamic integration 32,33 because while it is har to compute Z amp irectly it is easier to compute the integral of the erivative of the log with respect to some parameter. The noise stanar eviation, σ is the most convenient parameter so instea of keeping it fixe to, we treat it as a parameter an compute: an P (a,b) Z σmax log Z a () = log Z a (σ max ) σ log Z a(σ)σ. (2) For sufficiently large values of σ max (e.g. 0 0 ), the HMM states become effectively inepenent of the ata an so log Z a (σ max ) = NKlog 2πσ max2, where K is the number of pixels. As iscusse before, fining a requires optimization over a to fin the MAP fluorophore positions. For one fluorophore, we use conjugate graient to optimize over a. However, since the sampling introuces noise in estimating Z a, it it not possible to know if a true maximum or an artifact of noise has been foun. Therefore we have evelope a hybri of the forwar algorithm an MCMC to compute erivatives of Z a very precisely. By yieling accurate estimates of the erivatives, we can avoi resampling uring each loop of the optimization, which makes the process vastly more efficient an prevents the optimization from getting stuck on an artifact of noise.
7 Since our metho only yiels accurate graients, we aapt the conjugate graient algorithm. While searching along the conjugate irection, instea of searching for a local maximum of Z a, we search until the ot prouct of the first erivative an the search irection changes sign, an then ientify the zero crossing. We take small steps to avoi jumping over the local maximum. Our hybri metho works as follows. Derivatives of logarithms can be reaily foun using S samples rawn using MCMC 32 (where b is an instance of a sample of b): When fining the expectation of an arbitrary function φ over an arbitrary istribution P(x) (with x = {x,,x n } in the arbitrary space x X,x i X i ) using MCMC, the integration over ifferent variables can be separate: a log Z a = E(a,b) P(a,b)b P(a) a S E(a, b) a b P(b a) (3). X φ(x)p(x)x = X n X 2 [ X φ(x)p(x x 2,x n )x ] P(x 2,,x n )(x 2,,x n ). (4) Since rawing samples from the joint raws samples from the marginal, this can be approximate easily using MCMC: S ( x 2,, x n) P(x 2,,x n) [ ] φ(x, x 2,, x n )P(x x 2,, x n )x. (5) X Substituting the erivative of log Z a from Equation 3 into equation 5 gives: a log Z E(a, b) a = P(a,b)b a Z MN 3 P(a) S ( b 2,, b M) P(b 2,,b M a) [ Z N 3 ] E(b, b 2,, b M,a) P(b b 2,, b M,a)b. a (6) In a manner similar to Equation 3, the inner integral in Equation 6 can be written as: a log b Z N 3 P (b b 2,, b M,a)b. (7) This equation is the marginalization equation for a single fluorophore, conitione on a sample of all other fluorophore states an it can therefore be efficiently compute with the forwar algorithm. We can therefore compute graients with respect to the parameters of a spot by marginalizing over the spot s states with the forwar algorithm an the remaining spot states with MCMC. Using this hybri of MCMC an the forwar algorithm, we get smooth, accurate ifferentials because we o not recompute the samples uring the hill climbing stage. This allows for very efficient optimization of the real-value fluorophore parameters an is crucial to the success of our metho. Algorithm Even with these improvements, it is too computationally expensive to compare large moels with Bayesian moel comparison. In practice, we make many local ecisions which incrementally moify the moel, one fluorophore at a time. We take one fluorophore uner consieration to be either ae or remove. We either select a fluorophore for removal, or a a new fluorophore at a ranom position. We now have two moels: one with that fluorophore an one without. For the purposes of moel selection an marginalization, we allow only the parameters of that one spot to vary, an then ecie which moel to keep. After a number of such ecisions have been mae, we re-optimize the entire moel, then repeat the moel selection process. This yiels the algorithm:
8. Select initial spot positions for a moel. 2. Optimize moel using hybri MCMC an aapte conjugate graient. The step size is limite so that L of the step is 0.5. One pass of optimization optimizes each fluorophore in turn with the orering of fluorophores taken in the four compass irections. We repeat this for 4 passes (one for each compass irection), using 0 samples. 3. Repeat 20 times: (a) Compute free energy (Z a ) of the moel assuming fluorophores are fixe using 000 iterations of thermoynamic iterations where the noise stanar eviation for iteration i is.25 i/0. (b) Either a a new fluorophore or select one at ranom. (c) Optimize the new/selecte spot using the hybri metho with 20 samples. () Compute Z a of the new moel using thermoynamic integration. (e) Compute free energy by computing the Hessian using 00 outer samples, an 000 inner samples. 4. Goto 2 Note that when computing the Hessian with respect to the parameters of a single spot, we o not have an equivalent to the hybri algorithm, so we use a neste iteration scheme. During the neste scheme, for each sample of all states we raw (outer sample), we raw a number of samples (inner samples) from the Markov chain of interest using forwar filtering, backwars sampling. Since that algorithm is so efficient, this allows us to get consierably more accuracy with little extra computational cost. Note that in general, estimating a suitable point for terminating an MCMC proceure is ifficult 32. In practice, the algorithm is run until significant changes in the reconstructe image can no longer be observe. This typically requires several hunre complete iterations. The resulting high resolution image is then blurre with a Gaussian kernel with a stanar eviation of not less than σ = 2 pixels, then subsample by a factor of two to give the ensity image. The ensity image is then given a false colormap, to create the final colormappe image. The ensity image intensity is then scale by an arbitrary amount an a glow color scheme is then applie using the following formula: r = min,3 g = min,(max 0,3 ) b = min,(max 0,3 2), where is a pixel intensity in the scale ensity image, an r, g an b are the corresponing reg, green an blue pixel intensities in the final colormappe image. All the 3B images shown in the paper are generate using this metho, except for Figure 2a, which shows a single MCMC sample. Aitional references 29. Press, W. H., Teukolsky, S. A., Vetterling, W. H., an Flannery, B. P. Numerical Recipes in C. Cambrige University Press, (999). 30. Geman, S. an Geman, D. Stochastic relaxation, Gibbs istributions, an the Bayesian restoration of images. IEEE T. Pattern Anal. 6(6), 72 74 (984). 3. Gosill, S. J., Doucet, A., an West, M. Monte carlo smoothing for nonlinear time series. J. Am. Stat. Assoc. 99(465), 56 68 March (2004). 32. Neal, R. M. Probabilistic inference using markov chain monte carlo methos. Technical Report CRG-TR-93-, Dept. of Computer Science, University of Toronto, (993). 33. Bash, P., Singh, U., Langrige, R., an Kollman, P. A. Free energy calculations by computer simulation. Science 22, 245 268 (987). Reconstruction The metho for generating a final image from the MAP estimates is very similar to a iscrete approximation to kernel ensity estimation. The first stage is to buil up an image containing the fluorophore ensity. An empty image is initially create which is generally a much higher resolution than the input ata. All MAP positions are then quantize to the nearest pixel in the high resolution ensity image. The resulting quantize locations are then accumulate into the image. In orer to retain an approximation of the brightness, the accumulate fluorophores are weighte with the MAP brightness.
9 Table of symbols Symbol Meaning N Number of frames in an image sequence. M Number of fluorophores. S Number of samples. Z 3 Integers moulo 3, i.e. Z 3 {0,,2}. This is generally use to inicate that a variable is in one of the three states, emitting (light), not emitting an bleache. Z N 3 An N imensional quantity, each element of which is in Z 3. A variable in Z N 3 is generally the states of a single fluorophore in all frames. Z MN 3 An MN imensional quantity in Z 3. A variable in this space contains the state of every fluorophore in every frame. b A variable containing the state of every fluorophore in every frame. R 4 Real numbers in 4 imensions. a A variable containing the four continuous parameters of every fluorophore: x an y position, brightness an size. N Moel class for the null hypothesis (ata is generate by noise). F Moel class inicating that the ata is generate by one fluorophore an noise. F M Moel class inicating that the ata is generate by M fluorophores an noise. σ Noise stanar eviation, usually set to. P(D,a,b F M ) Probability that the ata is generate by a M fluorophores with specific parameters an a specific state sequence, multiplie by priors over b an a. P (a,b) Shorthan for above. P(D F M ) Probability that ata is generate by M fluorophores. Equal to P (a,b)ab Z Shorthan for above. Note that P (a,b)/z ab =. P(a,b) Normalize istribution P (a,b)/z. b A sample of b rawn from the istribution P(b a). P(a) Prior probability of a. Equivalent to P(a F M ). E(a,b) Log probability such that P (a,b) = e E(a,b). P(b 0 ) Prior probability of the fluorophore state in the first frame. Equivalent to P(b F M ). This is set to give a 50/50 chance of being in states 0 or. a 0 Initial spot parameters. P(D,a F M ) Probability that the ata is generate by a M fluorophores with specific parameters, but with the state sequence marginalize out. Z a Shorthan for above. Z a ( ) Z a parameterize with the noise stanar eviation. Since σ is usually, the parameterization is usually omitte. a MP The value of a which maximizes Z a. X An arbitrary space. x An arbitrary variable in X. φ An arbitrary function of x. P P 6 Transition matrix probabilities. Parameter values an ajustments: P P 6 Discusse in the main text. P(a) All elements of P(a) are inepenent. The position prior is set to be uniform over the area uner analysis an zero outsie. The size prior is set to be a log-normal istribution with σ = 0. an µ set so that the moe is at the correct FWHM of the PSF for the microscope. The brightness prior is log-normal with values aroun σ = 3 µ =. These parameters nee to be tune to the ata base on the approximate brightness of the fluorophores. a 0 The initial values of the brightness an size are set to the moal values of the prior. The initial number of spots nees to be selecte by the user. The initial positions are approximately uniform over the area of interest. The parts of P(a) which relate to the pixel resolution an relative brightness may nee to be ajuste for ifferent samples. The part of P(a) an a 0 relating to the area of analysis will nee to be altere between ifferent runs of the algorithm.