Rowland NMR Toolkit (RNMRTK) Overview

Transcription

1 Rowland NMR Toolkit (RNMRTK) Overview Originally developed as a platform for developing new NMR data processing methods Now widely used as a general processing platform Provides Rich set of apodization (window) functions DFT processing General processing tools (phasing, reversing, truncating, etc.) Robust, efficient Linear Prediction (LP) extrapolation MaxEnt reconstruction (deconvolution, uniform or nonuniform sampling) universal data reader exports to nmrpipe, XEASY, Felix Generates synthetic data, noise (testing/error analysis) Based on a program VNMR developed by Jeff Hoch in the late 1970 s Redesigned in 1985 and then in the early 1990s Free for Academic use and included in NMRbox l 1 -norm real (LONER) has recently been added

2 RNMRTK Suite of Programs section Manages the shared memory sections rnmrtk The main processing program in the Toolkit Loading, apodization, DFT, phasing, saving, etc. flip Forward linear prediction msa / msa2d / msa3d Maximum entropy reconstruction in one, two, and three dimensions inject Add synthetic peaks to time domain data select / zsample Expands or compresses NUS data seepln / contour graphical display for 1D and 2D data Many others

3 RNMRTK Shared memory With a suite of programs for performing tasks How do you get data output from one command to the next command without needing to write out intermediate files? Pipes (nmrpipe strategy) Shared memory (RNMRTK strategy) A shared memory section is a chunk of persistent memory where NMR data can be stored and used by different programs. section RNMRTK program to manage (create / delete) shared memory sections section -c The size of the shared memory section is the product of the arguments multiplied by 4 (32 bit data) plus 512 extra bytes for a header. Shared memory sections must be large enough to store all the data including processing section -d Deletes the shared memory section

4 RNMRTK Shared memory The maximum size of shared memory sections is defined by kernel values shmmax and shmall. NMRbox takes care of these advanced settings. Example: cd UCHC-2017/large-HNCO procpar.prl 25 The data set size is 512 x 128 x 128 complex points (1024 x 256 x 256 actual size) section -c Shared memory section is just large enough to hold time domain data However, it is not large enough to handle any zero-fills rnmrtk loadvnmr./fid rnmrtk seepar section -c rnmrtk rnmrtk%> seepar Deletes the shared memory section and replaces it. Existing data is deleted! rnmrtk%> loadvnmr./fid rnmrtk%> FFT rnmrtk%> DIM t3 rnmrtk%> FFT rnmrtk%> seepar rnmrtk%> exit section -d Shared memory sections will stay persistent until deleted or a reboot

5 rnmrtk Command for performing many of the traditional processing techniques Can be run: one command at a time in interactive mode scripted Row (column) oriented. Many commands need to have the dimension set RNMRTK rnmrtk

6 RNMRTK Issuing commands Arguments are entered as different classes: floats (defined as a number with a decimal) setpar SF integers (defined as a number without a decimal) zerofill 2048 string (defined as a character string without a period) sinebell square 70.0 filename (defined as a character string with a decimal point) loadvnmr./fid load hsqc.sec The order of arguments entered is important, but only within a given class of argument sstdc COS sstdc COS 20 rnmrtk is case insensitive (except for filenames) DIM command sets the dimension which is necessary for many commands rnmrtk << EOF LOAD test.sec LOAD command does not need dimension set DIM t1 FFT FFT command must have dimension set EOF

7 RNMRTK Issuing commands From the command line: bash% rnmrtk loadvnmr./fid bash% rnmrtk setpar SF bash% * DIM command not persistent In interactive mode bash% rnmrtk rnmrtk% loadvnmr./fid rnmrtk% DIM t2 rnmrtk% FFT rnmrtk% exit bash% As a script bash% processing_script.com bash% Typical script #! bin/bash section c rnmrtk << EOF loadvnmr./fid setpar PPM dim t2 zerofill 1024 fft 0.5 phase realpart EOF rnmrtk << EOF dim t1 zerofill 128 fft 0.5 realpart save test.sec EOF Here doc Cannot have spaces at the beginning of lines

8 Varian rnmrtk loadvnmr./fid RNMRTK Load data Bruker Need to create a parameter file called ser.par Example: Dom T1 T2 T3 N 64 C 140 C 512 C sw sf ppm Quad Order of dimensions in memory (after loading) STATES STATES STATES Format Little-endian Int Layout t2:280 t1:128 t3:1024 Layout defines order of dimensions in input file Note spacing between value and C # of padding bytes at beginning of fid # of padding bytes at end of fid # of header bytes Endian is defined by the computer architecture NOTE: spectrum-translator being developed to handled data conversions so little need to manually create load script anymore

9 cd UCHC-2017/large-HNCO more rnmrtk.com RNMRTK DFT Notes: SETPAR can be used to define parameters not set properly during conversion SEFIX1 is used to swap data that has been collected with sensitivity enhancement and not pre-shuffled during conversion./rnmrtk.com

10 Maximum Entropy Reconstruction (MaxEnt) MaxEnt can be performed in one dimension (msa), two dimension (msa2d) and three dimensions (msa4d) The acquisition dimension is often processed with a DFT and not with MaxEnt There are two modes for processing with MaxEnt Constant aim Must define def and aim Constant lambda Must define def and lambda When all the dimensions are NOT processed with MaxEnt, such as when the acquisition dimension is processed with a DFT, you MUST use Constant Lambda mode 2D Data sets Option 1: Process F2 with DFT, and F1 with msa (lambda mode) Option 2: Process F2-F1 planes with msa2d (aim mode) 3D Data sets Option 1: Process F3 with DFT and F1-F2 planes with msa2d (lambda mode) Option 2: Process F3-F2-F1 cube with msa3d (aim mode) 4D Data sets Option 1: Process F4 with DFT and F1-F2-F3 cubes with msa3d (lambda mode)

11 How to choose aim? AIM should be set to a value close to the RMS of the noise. Values too high will cause weak features of the spectrum to be lost Values too low will cause even minute details of the noise to be included (will try to force an exact match to the noise) Clear non-linearities are observed in peak intensities and are dependent on the value of aim unapodized DFT

12 Large values of def lead to reconstructions resembling the DFT Small values of def suppress noise but distort relative peak intensities Like aim, setting def to values near the RMS is generally the most appropriate DEF is less sensitive that AIM and can be altered by orders of magnitude How to choose def?

13 DEF & AIM High DEF, Low AIM Very low DEF, Low AIM Low DEF and Low AIM

14 LAMBDA During a MaxEnt reconstruction in constant aim mode there is a value called lambda that will be written to standard out. When processing data where the acquisition dimension has been processed with the FT you first need to determine values of def and aim based on the spectromter and spectra and then run MaxEnt in constant aim mode, record the converged lambda, and re-run in constant lambda mode. This is critical as in constant aim mode each plane will be scaled by a different amount and thus cause distortions in the peaks. Constant lambda holds the scaling constant.

15 RNMRTK msa2d 2 MODES Maximum Entropy Reconstruction in 2 dimensions Reads parameters from a file and dimensions are specified on invocation line Ex: msa2d t1 t2 msa2d.param Constant Aim parameter file DEBUG 1 NLOOPS 400 DEF 0.1 AIM 2.5 SCALEFIRST 0.5 SCHED./sample_schedule NUSE NOUT PHASE LW JVALUE Max loops, convergence may be quicker NUS Uniform Data will be extrapolated Constant LAMBDA parameter file DEBUG 1 NLOOPS 400 DEF 0.1 LAMBDA 1.0 SCALEFIRST 0.5 SCHED./sample_schedule NUSE NOUT PHASE LW JVALUE For 3D NUS datasets the acquisition dimension is typically processed with conventional FT and then the indirect dimensions are processed with msa2d

16 Let s try a MaxEnt Reconstruction cd UCHC-2017/large-HNCO more msa2d_param wc -l schedule3.scd There are 400 / 16,384 FIDs collected (2.4%)./msa2d.com schedule3.scd test nmrdraw -in test_nudft_proj.ft2 nmrdraw -in test_nudft.ft3 nmrdraw -in test_msa2d_proj.ft2 nmrdraw -in test_msa2d.ft3 You can compare the results to the DFT using 100% of the FIDs nmrdraw -in ft-f1_proj.ft2 nmrdraw -in ft-f2_proj.ft2 nmrdraw -in ft_f3f1.ft3

17 RNMRTK msa2d Monitoring convergence Very important! Ensure that the value of test is less than 0.1 unless fewer than NLOOPS required Without deconvolution convergence typically occurs is around 40 iterations if proper def, aim, lambda values were use. High numbers of loops suggest msa2d is modeling noise With deconvolution the number of loops may be significantly higher. How to check values for Test and the maximum number of loops more msa2d.txt tail msa2d.txt grep Test msa2d.txt Only look at lines that start with Test grep Loop msa2d.txt sort -n -k2