----------------------------------------------------------------- Compiling Optimal Acquisition and Sampling Tables : COAST DSR, October 2004, Program Version 1.2 DSR, July 2003, Program Version 1.2 Manual Version 0.2 (October, 2004) Manual Version 0.1a (July, 2003) ----------------------------------------------------------------- 1. Introduction 1.1 Acknowledgments 1.2 Versions 1.3 Purpose 1.4 Philosophy 1A. What's New in v1.2? 1A.1 True States-TPPI for Bruker 1A.2 Simpler Varian Schedules 1A.3 Modified "500" Schedules 1A.4 Removed Dead Schedules 1A.5 Continuous Dwell Times 1A.6 Minor Typos 2. Installing Coast 2.1 Step by Step 2.2 Microsoft Windows 3. Pulse Programming 3.1 Acquiring Quadruples 3.2 Bruker 3.3 Varian 4. Running Coast 4.1 The Look and Feel 4.1.1 The log and preview window. 4.1.2 The configuration window. 4.1.2 The schedule window. 4.2 Choosing Parameters 4.2.1 Which Spectral Width? 4.2.2 Which Sampling Schedule? 4.2.3 How much S/N is enough? 5. Process nonuniform Data with MaxEnt Reconstruction 5.1 Processing Files 5.2 Getting Started 5.2.1 Starting with Bruker 5.2.2 Starting with Varian 5.2.3 Remaining Steps 5.3 General Impressions 6. Frequently Asked Questions 6.1 Does Coast run on my Mac? 6.2 Is Coast the same as the Rowland Toolkit? 6.3 But really, which schedule should I use? 6.4 Can I use one schedule for many experiments? 6.5 Can I generate schedules on the fly? 6.6 How good are the line width calculations, really? GLOSSARY tau-c : overall rotational correlation time t1 : first indirect evolution period t2 : second indirect evolution period nonlinear sampling : (phased out and replaced by 'nonuniform sampling') nonuniform sampling : acquiring different numbers of transients for different evolution periods 1. Introduction This is the manual for Coast, a graphical program for configuring sampling schedules for protein 3D NMR pulse sequences. (**IMPORTANT**) Information considered to be of unusual significance in the manual will be accompanied by this marker. 1.1 Acknowledgements Coast is written and maintained by D. Rovnyak. The work was done at the Harvard Medical School (HMS) and the Harvard/MIT Center for Magnetic Resonance. The support of an NIH-NRSA training fellowship is gratefully acknowledged (CA89940). Development of pulse programs was a collaborative effort among several individuals : Dmitri Ivanov (Ph.D., HMS), Jim Sun (Ph.D., HMS), Dominique Frueh (Ph.D., HMS), Wolfgang Bermel (Ph.D., Bruker Biospin) and Clemens Anklin (Ph.D., Bruker Biospin). The advice of Greg Heffron (Ph.D., HMS), Tony Bielecki (Ph.D., Center for Magnetic Resonance), and George Gray (Ph.D., Varian Inc.) was also instrumental. This project has been conducted under the supervision of Gerhard Wagner (Ph.D., HMS) and has depended critically upon close collaboration with Jeff Hoch (Ph.D., Rowland Inst.) and Alan Stern (Ph.D., Rowland Inst.). Initial testing by Monika Oberer (Ph.D, HMS), Natalia Beglova (Ph.D. Brigham and Women's), and Vladimir Guelev (Ph.D., HMS) was critical to the develop- ment of Coast. Coast is named for the author's time spent on the beautiful north shore of New England. The significant enhancements and ongoing developments of nonuniform sampling pulse programs for Bruker and Varian instruments have been performed by Jim Sun, Dominique Frueh, Hari Arthanari, and probably many others I have missed with apologies. 1.2 Versions Coast is updated on a semi-regular basis. All versions prior to 2.0 are to be considered functional pre-releases which may have some 'personality', or lack all features planned for v2.0. 1.3 Purpose Coast helps to smooth the process of performing nonuniform sampling and processing the resulting data. Coast is not intended to replace or even overlap with rnmrtk programming and documentation. (**IMPORTANT) In time (near or distant) it is hoped that native vendor support from the major NMR suppliers will eliminate the need for Coast. Indeed as you gain more experience with Coast you will likely develop enough proficiency to no longer use it. To some degree, Coast acts as a training tool. Since it is not possible to imagine every scenario in which you may want to apply nonuniform sampling, Coast can show you the ropes on a useful subset of experiments to help you see how to do nonuniform sampling on experiments that we did not consider. More generally, my hope is that Coast will make nonuniform sampling more approachable and fun for people to try. 1.4 Philosophy Coast is a text script that is not compiled. This is a nice way to satisfy the GPL since the executable and the source code are the same. Coast aims to be accessible for proof-reading and modifying the code. It is true that compiling the script would theoretically improve the performance. But the script is not very demanding, with the possible exception of viewing the different schedules. You are welcome to compile this if you want but remember that you can redistribute the compiled program only if it is accompanied by the source. 1.5 Disclaimer (**IMPORTANT**) Coast is a work in progress and is likely to be rough around the edges, particularly in the GUI behavior. Implementations of non-uniform sampling may evolve more rapidly than Coast is able to accomodate. Please email any bugs, comments, suggestions (rovnyak@alum.mit.edu). The GPL is in full effect (no warranty, express or implied, etc...). 1A. What's new in v1.2? 1A.1 True States-TPPI for Bruker During the development of pulse programs to perform nonuniform sampling, we encountered an initial inability to perform States-TPPI phase cycling in both indirect dimensions for the Bruker implementation. The fix was developed recently by Jim Sun. This version of coast supports this new pulse programming method for Bruker. This greatly simplifies the appearance of sampling schedules in Bruker and also makes configuring these pulse programs on Bruker instruments much more intuitive. You will now generate just one sampling file for use in nonuniform 3D experiments on Bruker. You will need to enter the spectrometer information by hand into Coast. This is described more fully in the instructions. 1A.2 Simpler Varian Schedules Similarly the Varian sequences were improved in the Wagner lab to use much simpler sampling schedules than utilized previously. This version of coast also implements these simpler schedules used in the current release of nonuniform sampling pulse programs for varian. 1A.3 Modified "500" Schedules This is actually a consequence of the fix for Bruker. The new method for Bruker requires that the first point correspond to 0 evolution in both indirect dimensions. This is the point (1,1). The scheduled 500ee and 500cc did not include this point. The first two points are (1,3) and (1,5) in 500ee in v1.1 The first two points are (1,1) and (1,2) in 500ee in v1.2 The first two points are (1,2) and (1,6) in 500cc in v1.1 The first two points are (1,1) and (1,2) in 500cc in v1.2 1A.4 Removed Dead Schedules There was a set of 500xx_new schecules in v1.1 that were experimental and which were not offered in the user interface. These have not been used by us or others and were deemed 'dead code' and removed. If you decide to print out the code for v1.2 for any reason, this will save you about 3 pages. 1A.5 Continuous Dwell Times In the original pulse sequences supported by v1.1 of coast, it was necessary to supply sampling schedules with explcit evolution times, so that quite a lot of tedious handling of dwell times and spectral widths was performed in coast. This was handled by forcing evolution increments to be rounded to the nearest half-microsecond. Thankfully the new sequences don't need this anymore and all configuration of spectral widths occurs at the console in the usual fashion. Now coast does not set your spectral widths or dwell times at all. You will supply this information later -which can be any dwell time at all - when you setup processing scripts in coast. 1A.6 Minor Typos Several minor typos in the manual and the comments have been fixed. They were all spelling related. 2. Installing Coast Coast is a standalone program using Tcl/Tk for scripting and graphics functionality. Coast is intended to be compliant to Tcl v7.4 or higher and Tk v4.0 or higher. 2.1 Step by Step First, you will need to have Tcl/Tk installed. Check for wish, which is often found at '/usr/bin/wish' for most flavors of unix (inluding linux). Wish is the combined Tcl/Tk interpreter that any tcl/tk application uses. Type 'which wish' to find out if and where wish is in your path. Note that even if wish is not in your path, it may be present on your system, so look around hard before doing a fresh install. If you are running Apple OSX, use fink to install tcl/tk. (http://fink.sourceforge.net). Second, once you know the path to your wish interpreter, modify the first line of coast to point to this location. Third, if you primarily use a single machine, you can enter the vendor and field strength into Coast. In the coast script file, find the startup procedure, the first procedure in the Coast file (about 1-2 pages from the top), and change the appropriate values. Finally, place Coast in an appropriate location, probably somewhere in your path like /usr/local/bin, for example. You do not need root to run Coast. 2.2 Microsoft Windows Coast has been found to work on several flavors of Linux, Apple OSX and Irix, but has not been tested yet on any version of the Microsoft Windows operating system. Probably it will be fine, but feel free to drop me an email describing any system on which you have run Coast (rovnyak@alum.mit.edu). 3. Pulse Programming 3.1 Acquiring Quadruples We have performed nonuniform sampling on Bruker Avance and Varian Inova consoles, and also custom-designed consoles (D. Ruben, R.G. Griffin, MIT/Harvard Center for Magnetic Resonance). The main concept we use is the acquisition of 'quadruples'. By a quadruple, we mean that four spectra are recorded for a given set of t1 and t2 evolution times: (R,R) (R,I) (I,R (I,I) where R and I refer to setting phases to obtain real and imaginary parts. This gives States mode detection in both indirect evolution periods. If States-TPPI is desired, it is necessary to determine if an evolution time is even or odd with respect to the dwell time and to add 180 degree phase shifts to the relevent pulses and receiver when it is indicated. (**IMPORTANT**) The receiver must be 180-degree shifted in tandem with pulses when carrying out States-TPPI with nonuniform sampling. In consecutive (aka linear) sampling, this is not needed since processing programs offer features to change the signs of alternating FIDs. There is no feature in Coast or the Rowland toolkit to adjust the sign of nonuniformly accumulated FIDs post-acquisition. This is mainly something for Bruker users to consider, since Bruker tends to advocate not shifting the receiver for States-TPPI. 3.2 Bruker (** Updated in Version 1.2 **) On Bruker, Coast currently uses xwinnmr version 3.1 or higher. For nonuniform sampling two dimensions of a 3D experiment, you will use one vc list. When you write the vclist from coast the copy it to, /u/exp/stan/nmr/lists/vc/ and use names consisting of 6-7 characters. Keep in mind that the 'expt' command works with nonuniform sampling (it usually requires extra time to compute). It should be kept in mind that operations on lists in xwinnmr (e.g. incrementing to the next value) can only be stably performed during the recycle time. For the Bruker implementation, the quadruples are acquired by using appropriate statements in the acquisition portion of the pulse program; the vc list has one value per line. The following are notes on the construction of a vclist adapated from J. Sun's notes. The implementation uses solely integers to represent which samples to use (selected from the grid of uniformly distributed samples that would be acquired with conventional nD-NMR data acquisition). The pulse programs automatically acquire a quadruple for zero evolution in both indirect periods. Thus any sampling schedule must contain this point. IN rnmrtk it is common to call this the (1,1) point, however vc lists count from zero so in order to convert an rnmrtk schedule -and schedules in coast use this format - the first point is omitted, and all other points are t1-1 and t2-1. A final dummy point of (0,0) must be appended. 3.3 Varian On Varian, we currently use Biopack (c) (a.k.a. proteinpack) pulse sequences, with minor modifications. This has several obvious benefits over writing new sequences from scratch, but foremost it allows the user to optimize their experiments using the Biopack, and then switch to the nonuniform pulse program only when they are ready to run the actual experiment. The 'time' command still works with nonuniform sampling. 4. Running Coast If you have placed coast in your path, you can start it by typing the name 'coast'. Of course feel free to give it any name you like. 4.1 The Look and Feel The coast window is divided into three general areas. They are : 4.1.1 The log and preview window. This is the left window, which will provide feedback and other information as you work in coast. For example if you double-click on any schedule, the coordinates for that schedule will be displayed here. The buttons at the top are: Tools - Config - Write - Load - Quit. Tools : This gives a pulldown menu with choices to (a) estimate measuring time for any configured experiments and (b) toggle balloon help on or off. Config : Coast can be customized here. Any changes you make here will *not* be saved with a summary file. Default values for these items are hard-coded into Coast. You can change them permanently simply by editing Coast. Write : Use this button to write formatted schedules or summary files. It is deactivated unless the log window displays a formatted sampling schedule or a summary of configured experiments. There are no restrictions on the file names that you can choose as far as Coast is concerned, but (**IMPORTANT**) if you are working on Bruker instrumentation, restrict the names of sampling schedules to just 6 or 7 characters. Load : Load a previously saved summary file. 4.1.2 The configuration window. This is the center of the screen, and is divided into three areas : Sample Parameters - Experiment Configuration - Processing. Sample Parameters - (**IMPORTANT**) Only the static field is required. The static field is used to convert between Hertz and ppm in the configuration window below. Enter a molecular weight and Coast will estimate the overall rotational correlation time, tau-c. Or you may enter tau-c directly if you have a good idea of it. Once a tau-c value is present, you can click the line widths button. Experiment Configuration - Here you can set up sampling schedules for any experiment that is supported by Coast. Currently Coast is set up for the 6 canonical 3D backbone experiments, but one can appreciate that schedules generated here can be used for other types of experiments. To configure an experiment : (a) click the checkbutton next to any experiment to select it as the 'working experiment'. Anything you do will apply only to the working experiment. You can have more than one working experiment by clicking two or more boxes. (b) click inside the box of a schedule to associate that schedule with the working experiment. You will see a short code appear next to the checkbox (e.g. 500ce) which is a shorthand notation for the schedule you have selected. (c) (** Updated in Version 1.2 **) The section for spectral widths has NO bearing on the sampling schedules, but is currently requried to proceed with coast. For Varian and Bruker, you will enter spectral widths at the console when setting up the experiment in the normal fashion. The section here in coastg is not used until processing, so put in any sw's you want and fix these later when you process the data. It is suggested that you click the button 'Standard SW's' to load spectral widths for the working experiment#. Again, you are under NO obligation to use these at the console. They are merely place-holders until you use coast to help generate processing scripts. #These are conservative spectral widths based upon an analysis of the shifts deposited in the BioMagResBank. You can keep these values or modify them. (d) The label for the working experiment will become a button when Coast notices that all items are present. When you click this button a 'formatted schedule' is displayed in the left-hand portion of coast. This will look like one column (for use with Bruker) or two coloumns (for use with Varian). (e) When a formatted schedule is displayed in Coast, you can print this schedule to a file, as you will see the 'Write Sched' button become active in the left window. This file can then be transferred to the appropriate experiment directory if you are using varian or to the /u/exp/stan/lists/vc/ directory if bruker. 4.1.3 The schedule window. This is the right third of the screen and displays prebuilt sampling schedules that you can choose from. Use the blue arrows to flip through all of the sampling schedules. For each page there is a fixed number of samples in the three displayed schedules, but each has a different distribution. The top schedule is exponentially distributed in both dimensions (ee); the middle schedule is exponential in one dimension and random in the other (ce); the bottom schedule is randomly distributed in both dimensions (cc). Random distributions are useful for constant-time dimension (usually 15N). 4.2 Choosing Parameters 4.2.1 Which Spectral Width? When in doubt, use the standard spectral widths that Coast suggests. These are safe (but not excessively wide either) and will miss shifts only in pathological cases. A common practice is to accept the Coast defaults for 13C but to use the smallest spectral width in 15N as determined from 15N-1H HSQC experiments. Coast will report the unused portion of constant time evolution, assuming that the maximum time available is 12.4 ms. Take this with a grain of salt. In both cases the actual amount you can use is a bit less due to miscellaneous delays. Aim to use about 12 ms. 4.2.2 Which Sampling Schedule? See also FAQ 5.3. Key features of the sampling schedules are: (a) the first few points are linearized. This is not strictly necessary but I have found, in a mostly anecodtal fashion, that this makes it easier to do the MaxEnt reconstructions. (b) every schedule has the maximal point, such as the point (64,40) in the 500 pt family of schedules. This is a bit arbitrary but it seems reaosonable to have this point to delimit the sampling space. (c) each family has three distributions : double exponential (ee), random-exponential (ce), and random-random (cc). The 'c' is for 'constant-time'. (d) the 500 point and 415 point schedules have found the widest use, however consider getting started with either the 600 or 700 point schedules first as these are particularly conservative and it will probably be a little easier to select def and lambda as a first- time user. (**IMPORTANT**) The greater the degree of data reduction of nonuniform sampling versus linear sampling, the more care is needed in performing the reconstruction. 4.2.3 How much S/N is enough? This is a very (**IMPORTANT**) point. Don't ask anything of MaxEnt reconstruction that you would not ask of the Fourier transform. In other words, it is necessary to obtain as much sensitivity in the first FID as you would obtain if you planned on using the fast Fourier transform. 5. Processing nonuniform Data with MaxEnt Reconstruction 5.1 Processing Files For any configured experiment, Coast will create the necessary files for processing the resulting data. These files are: 2d.sched : the schedule for nonuniform sampling, in integer format fid.par : the spectrometer/acquisition parameters needed to read the raw data into the Rowland NMR Toolkit. proc1d.sh : a shell script for carrying out the FFT of the directly acquired proton dimension proc_msa2d.sh : a shell script for carrying out FFT of the directly acquired proton dimension followed by two-dimensional MaxEnt reconstruction of the C-N planes msa2d.par : parameters for the MaxEnt reconstruction; most significant of these are Def and Lambda 5.2 Getting Started 5.2.1 Starting with Bruker (a) copy all data and parameters into the directory where you will do all processing (b) rename the ser file to 'fid.dat' (c) have you performed digital to analog conversion? You will be asked this by Coast. If the answer is no, Coast will compute the linear phase shift that must be applied to the processed direct dimension to correctt for the time shift that Bruker's oversampling introduces into the FID. 5.2.2 Starting with Varian (a) copy all data and parameter files into the directory where you will do all processing (b) rename the fid file to 'fid.dat' (c) dsp/non-dsp data do not require special treatment 5.2.3 Remaining Steps (d) allocate a section for rnmrtk with the 'section' command. On most systems you will want to allocate 2048 256 256, which is the number of real + imaginary points in each dimension. So this would be suitable if the direct dimension used 1024 samples, and the indirect dimensions were expanded to 128 samples each. (i.e. one sample equals the real and imaginary parts) (e) try the proc1d.sh script and view the results with the rnmrtk program seepln; take this opportunity to decide on apodization and other pre/post processing items that you may want for the direct dimension. Coast specifies extracting the left half of the spectrum, which assumes an amide detected experiment. Coast also tends to use one 90-degree shifted sine bell window function as a compromise between artifiact suppression and resolution degradation. (**IMPORTANT**) A common mistake is to acquire the direct dimension for longer times than are necessary, which introduces extra noise. Try shrinking the direct dimension before FFT. (f) Update proc_msa2d.sh with everything that you decided on in step (e). (g) Open msa2d.par. There are several options here, and more are are available, so it is advised that you look over the docs for the msa2d program. (**IMPORTANT**) On Mac OSX running on a G4 chip, you should have installed an executable called 'mas2dv' which is the vector-enhanced version of msa2d. DEBUG : how verbose do you want msa2d to be. NLOOPS : how many iterations in the algorithm; you should not have to modify this DEF : related to the least sensitive signal that is present in your spectrum LAMBDA : related to enforcing consistency between the frequency spectrum that msa2d produces, and the nonuniform data SCHED : name of the file containing the schedule NOUT : the number of samples to expand the processed data into in the t1 and t2 dimensions, respectively LW : allows you to convolute the time domain data with an exponential function to achieve line broadening or line narrowing. (e) It is suggested to accept defaults in the msa2d.par file. Typical values for both DEF and LAMBDA are found in the range (0.1 - 50), and in fact usually end up just with the range (1-10). (f) Try the proc_msa2d.sh script next. If you have a dual processor machine, remember to set MP_SET_NUMTHREADS =2 in your shell environ- ment first. bash : export MP_SET_NUMTHREADS=2 csh : setenv MP_SET_NUMTHREADS 2 Use 'weak' values of DEF and LAMBDA at the start. For example the default suggestions from Coast are DEF=10 and LAMBDA=1, which are actually fairly strong values. It is generally acceptable to set LAMBDA to 0.1 and DEF to 20 just to get a very fast reconstruction. (g) Phase and inspect the resulting spectrum. If all is well you can now move on to optimizing DEF and LAMBDA. It is important to note that the reconstructions are relatively insensitive to these values so that you likely only need to run proc_msa2d.sh just 1-3 more times. (h) As desktop computing has very rapidly advanced, MaxEnt has become very accessible on the desktop, to the point that it is not difficult to make DEF too small and LAMBDA too large without unreasonably extending the computational time. The telltale signs for this are usually in the noise. Noise should resemble what you are accustomed to seeing with the FFT; most spectroscopists have a good eye for recognizing 'healthy' gaussian distributed noise. Look for regular or irregular spikes in the noise; look for noise that is not constant accross the spectrum. Look for images that resemble folding. 5.3 General Impressions It is recommended to work on a system that you are familiar with at the outset to develop confidence with the method, and then move on to unknown samples and data sets. Have faith in the methodolgy - there is a huge temptation to assume something is wrong in the processing/sampling without looking critically at acquisition paramaters when something does not look right in the spectra. 6. Frequently Asked Questions 6.1 Does Coast run on my Mac? With the proper steps, Coast runs very well on Mac OSX. Most of Coast was developed on an OSX laptop. See 'Installing Coast' for detailed instructions. I was not able to get Coast to run very well with Apple's aqua version of Tcl/Tk, so be sure to install tcl/tk using fink. 6.2 Is Coast the same as the Rowland Toolkit? No. Coast is a standalone program that provides scripts that the Rowland NMR Toolkit can use to read and process nonuniformly acquired data. 6.3 But really, which schedule should I use? This is not always an easy question to answer, but consider both the 500 pt schedules and the 415 pt schedules for backbone assignment experiments. The 500 pt schedules span (64x40) in 13C and 15N, respectively, and the 415 pt schedules span (60x32). If you have a system which is sensitivity challenged (low concentration and/or high MW), the 415 pt schedules give you a lot of free time to put back into collecting additional transients. If the sensitivity is extremely poor, consider perhaps the 350 pt family to have even more time (at the expense of resolution). If your sensitivity is decent, the 500 point schedules will give better resolution. 6.4 Can I use one schedule for many experiments? It is common to be able to use one schedule many times, particularly if you used conservative spectral widths for that schedule. For example, with the backbone assignment experiments it is typical to use one schedule for both experiments in a 'pair'; that is the schedule for the HNCA can also be used for the HN(CO)CA, and the schedule for the HNCO can be used for the HN(CA)CO, etc. 6.5 Can I generate schedules on the fly? No. Currently Coast offers only a limited selection of prebuilt schedules. These are all highly suitable for the currently supported backbone assignment experiments, but it is likely that you may wish to try nonuniform sampling in many other experiments for which these schedules are not as well suited. To use a different schedule, consider using the programs sampsched and sampsched2d, which are components of the Rowland NMR toolkit. Generating sampling schedules interactively with Coast will be offered in v2.0 (the framework is already completed for doing this, but the GUI is lacking). 6.6 How good are the line width calculations, really? They are decent, but could be off by up to 30-40% depending upon each situation. The correspondence between molecular weight and tau-c (the overall rotational correlation time) is weak so if you use Coast to estimate tau-c, then this is probably the biggest source of error. Internal dynamics, local structure, deuteration efficiency, etc. are not accounted for either. But if you have independent knowledge of tau-c, the line width estimates are probably quite close to the theoretical values. Remember that these should be treated as average line widths for a given system. This is an area where further devleopment could be interesting.