The BUSTER Environment

Contents

• Overview
• Files and Directories Layout
• Take a walk on the server side
• User files
• Principle
• Examples
• The CCP4 suite of programs and the MTZ format
• How to prepare MTZ files for BUSTER
• What CCP4 programs does the BUSTER interface use ?
• The BUSTER-TNT interface
• How to prepare PDB files for BUSTER
• User input for TNT
• The TNT scripts
• Changing the command inputs to the TNT scripts
• The BUSTER Control Panel and Preferences Page
• Control Panel
• Shutdown
• Resume
• Restart
• Save
• Preferences
• User Level
• Job submission scheme
• Displaying maps
• Contour level
• Region Selection
• Viewing tool
• Peak picking
• Skeletonisation
• Atomic models display
• Archiving files
• References

Overview

Most of the material presented in this Chapter is currently best consulted by browsing through the tutorials. The only aspect of the BUSTER environment that is not dealt with elsewhere is the protocol by which BUSTER communicates with TNT, and the extent to which the user can modify it. The present state of the software in this respect is far from its intended final form, and the highest priority will be given to providing fuller and more flexible access to TNT. The limited capabilities available at present have nevertheless proved adequate for tackling a variety of problems and should be usable in most cases with little or no change.

Files and Directories Layout

Take a walk on the server side

At the heart of BUFFET's server side is the apache http daemon. It serves all the http requests from the user's browser, usually by running so-called cgi scripts. Because it serves the GUI for every user, the http daemon may run under a different UNIX account, using a strict directory layout. Users need to know only the root of this directory tree: this location is referred to as $BDG_home, and is included in the welcome message sent to every BUSTER user.

The directory $BDG_home/bin and its sub-directories are readable by BUSTER users. They can find additional scripts for configuration and data preparation in those directories.

User's files

Principle

Figure 1. BUSTER user's files layout. Square boxes are directories, and rounded ones are soft links. Italics indicate placeholders for your jobs and/or machine names.

The $BDG_busterfiles directory is the root of the BUSTER directory layout for each user. It can be located anywhere, provided that it can be accessed by all the machines running BUSTER (the MASTERs) and by the machine running BUFFET (the SERVER). It must have write permission for the user (obviously) and for the server's UNIX account. The command files for BUSTER are stored in subdirectories in the cardfiles directory. BUSTER looks for PDB and MTZ files in the datafiles and puts the results and the log in the logfiles directory. The logfiles directory contains only links to the actual directories in which the results are stored. Final results can be archived in the savefiles directory, keeping only most important files. Every BUSTER job is labelled by a job name followed by a serial number. Each log file can be easily connected to its corresponding set of command files.

The dispaching of output on locally mounted disks for each MASTER scatters the results on several quite different locations. Some users may want to keep their results centralised, even at the expense of run-time efficiency. In those cases, all links logfiles.MASTER may point to the same directory. The default settings at user creation time is the create a directory called logfiles_local in $BDG_busterfiles and force all logfiles.MASTER to point towards this directory, as seen on Figure 2. A "project" layer has been added to avoid the accumulation of unrelated datasets in the same cardfiles and datafiles directory. Each BUSTER project is entirely enclosed into a single directory, which acts as the $BDG_busterfiles in figures 1 and 2. It allows for example more meaningful names to be given to jobs (i.e. RigBodRef.1 or CompleteModel.5) besides less cluttered cardfiles, datafiles and logfiles directories. All projects have the same layout, but every one of them need to have a different root directory replacing $BDG_busterfiles. When projects are used in conjunction with the dispaching of log files on locally mounted disks (see Figure 1), the various logfiles.MASTER must be different from project to project, to avoid possible conflict between job names.

Figure 2. BUSTER user's files layout. Output files are grouped in a single directory, regardless of the machine of which the job is run

Examples

Examples of both kind of installation are given below. In both cases, the BUSTER server is on machine gibbs, running on the software account; there are 7 configured masters machines: bayes, babinet, fourier, gibbs, green, jeffreys and newton. As the BUSTER user (blanc) is in the same UNIX group as the buster server software, permissions were restricted to disallow any operation from other groups.

blanc@newton<1> ls -al ~/projects/IF3-C
total 57
drwxr-x---   4 blanc    user        8192 Oct  9 12:09 ..
drwxrwx---   7 blanc    user        8192 Oct  9 12:08 .
-rw-r-----   1 software user         272 Oct  9 12:08 .busteraccess.gibbs
drwxrwx---   2 software user        8192 Oct  9 12:06 cardfiles
drwxrwx---   2 software user        8192 Oct  9 12:06 datafiles
drwxrwx---   2 software user        8192 Oct  9 12:06 logfiles
drwxrwx---   2 software user        8192 Oct  9 12:06 savefiles
drwxrwx---   2 software user        8192 Oct  9 12:06 logfiles_local
lrwxrwxrwx   1 software user           5 Oct  9 12:07 logfiles.babinet -> logfiles_local
lrwxrwxrwx   1 software user           5 Oct  9 12:07 logfiles.bayes -> logfiles_local
lrwxrwxrwx   1 software user           5 Oct  9 12:07 logfiles.fourier -> logfiles_local
lrwxrwxrwx   1 software user           5 Oct  9 12:07 logfiles.gibbs -> logfiles_local
lrwxrwxrwx   1 software user           5 Oct  9 12:07 logfiles.green -> logfiles_local
lrwxrwxrwx   1 software user           5 Oct  9 12:07 logfiles.jeffreys -> logfiles_local
lrwxrwxrwx   1 software user           5 Oct  9 12:07 logfiles.newton -> logfiles_local
blanc@newton<2>

In the above example, the logs will all be stored in the ~/projects/IF3-C/logfiles_local directory, regardless on which machine the job was run on. The outputs are grouped at the expense of network efficiency when the jobs are run.

blanc@gibbs<3> ls -al ~/projects/Barnase
total 57
drwxrwx---   4 blanc    user        8192 Oct  9 12:09 ..
drwxrwx---   7 blanc    user        8192 Oct  9 12:12 .
-rw-r-----   1 software user         272 Oct  9 12:09 .busteraccess.gibbs
drwxrwx---   2 software user        8192 Oct  9 12:09 cardfiles
drwxrwx---   2 software user        8192 Oct  9 12:09 datafiles
drwxrwx---   2 software user        8192 Oct  9 12:09 logfiles
drwxrwx---   2 software user        8192 Oct  9 12:09 savefiles
drwxrwx---   2 software user        8192 Oct  9 12:09 logfiles_local
lrwxrwxrwx   1 software user          34 Oct  9 12:11 logfiles.bayes -> /mnt/bayes1/blanc/projects/Barnase
lrwxrwxrwx   1 software user          36 Oct  9 12:12 logfiles.babinet -> /mnt/babinet1/blanc/projects/Barnase
lrwxrwxrwx   1 software user          36 Oct  9 12:12 logfiles.fourier -> /mnt/fourier1/blanc/projects/Barnase
lrwxrwxrwx   1 software user          34 Oct  9 12:12 logfiles.gibbs -> logfiles_local
lrwxrwxrwx   1 software user          34 Oct  9 12:12 logfiles.green -> /mnt/green1/blanc/projects/Barnase
lrwxrwxrwx   1 software user          37 Oct  9 12:12 logfiles.jeffreys -> /mnt/jeffreys1/blanc/projects/Barnase
lrwxrwxrwx   1 software user          35 Oct  9 12:12 logfiles.newton -> /mnt/newton1/blanc/projects/Barnase
blanc@gibbs<4>

On the contrary, the example above show how to setup directories so that the log files are kept on locally mounted disks for each master machine. In this case, all directories /mnt/*/blanc/projects/Barnase need to be created by the user at project creation time. Unfortunately, the server cannot create those directories, because they sit in disk areas owned by the user blanc.

The CCP4 suite of programs and the MTZ format

In the CCP4 system, all crystallographic data relevant to a given problem are stored in a single multi-column binary file. The format of this binary file is called MTZ. We will only provide here a BUSTER-oriented summary of MTZ.

How to prepare MTZ files for BUSTER

If your data processing and scaling etc. have been done using CCP4 tools, you already have a file in MTZ format, that will be read directly by BUSTER. For each run, an automatic conversion to TNT's asymmetric unit and sort order is performed by the startup script, using CCP4's sftools.

If you have a file in some other format (PHASES, X-PLOR etc.), it may take a few more steps to prepare an MTZ file. CCP4 provides an all-purpose converter, called f2mtz, that will take an ASCII file and convert it into an MTZ file. In addition, please note that BUSTER reads data only in the form of structure factor amplitudes (FP). Transformation from intensity data (for unique reflexions) to structure factor amplitudes and anomalous differences is usually done using the CCP4 program truncate.

Reduction to unique Miller indices, as well as the application of scales (possibly calculated elsewhere) to the data is usually done in CCP4 program scala.

Documentation for these programs can usually be found (if CCP4 is installed on your system) by typing 'man <name-of-program>'. If the man page is not found, you have a second chance with 'more $CDOC/<name-of-program>.doc'.

What CCP4 programs does the BUSTER interface use ?

• fft : Fast Fourier Transform (Ten Eyck, 1973) to obtain the Log-likelihood gradient maps.
• npo : Traces contour maps, to be viewed with xplot84driver
• peakmax : finds the peaks in density maps; it always run in the asymmetric unit

The BUSTER-TNT interface

How to prepare PDB files for BUSTER

When present, NCS description must be made available to BUSTER in the form of a file having the *.hardncs or *.softncs file extension in the datafiles directory. This file must be created by the user; for those unfamiliar with TNT's NCS syntax, a very terse introduction is given appendix A. It must be noted that, unlike usual TNT scripts, BUSTER expects an expanded set of coordinates, even in the case of constrained NCS.

User input for TNT

The TNT scripts

Changing the command inputs to the TNT scripts

The BUSTER Control Panel and Preferences Page

Control Panel

• About this page : describes the BUSTER Control Panel
• Report Bugs : is used to send a message to the developers of BUSTER.
• Change password :  is used to change your password for using the BUSTER hypertext server. After you change your password, you will be asked to re-authenticate once after you have changed your password.
• Preferences : is used to view and changesettings which are used by the interface.
• Start : is used to prepare input for a BUSTER job. On the right of that button, you can see a menu of previous inputs, including the templates that we deliver with the release. You may use the parameters of a previous calculation as a template for the parameters of your present crystallographic problem, or start with a form filled with defaults by selecting 'None'. In these pages, you will find context-sensitive help by clicking on highlighted words or expressions.
• Request : is used to perform an action on a job which has already been started. The type of action is selected in the menu immediately on the right, and the job name is selected in the next menu on the right. The types of actions are :
• Nothing : the default. No action taken.
• Shutdown : stop the job as soon as possible.
• Resume : resume a job; if it is a refinement, it will resume from the point where it had stopped.
• Restart : restart a job, overwriting any files in the output directory.
• Examine : is used view cardfiles, datafiles, and logfiles. Cardfiles are the input instructions to BUSTER (also called CRD files); datafiles are all other input files used by BUSTER, such as partial structures, observed data, etc.; logfiles are result directories (one per run) produced by BUSTER
• Projects : is used to manage projects. New projects can be created, or some old project can be selected as default project. Note that the default project cannot be changed while a job is still running.
• Save : archive a subset of the output files and remove all the others, to clean the output directory and disk; you can choose the list of files to be archived from the preferences page.
• Delete : is used to remove both cardfiles and logfiles from disk, for a given run. Attention : these results tend to accumulate, especially when you first start using BUSTER and make a few mistakes. Do not forget to 'clean up' from time to time, using this facility.
• Browse : used to look at various documents relevant to BUSTER.
• About : for finding out more about this page, and about the BUSTER Development Group (Gérard Bricogne's group in Cambridge, developing SHARP and BUSTER).
• Mail : is used for sending mail to the BUSTER usergroup, the site administrator (set locally as the account that runs BUSTER - please ensure that this account is mail-capable), and the developers of BUSTER (to report bugs directly to us, but we strongly advise you to report bugs to your BUSTER site administrator first).

Preferences

• User Level :  The user level selected here will determine the degree in detail of the input form you are presented. Some calculation options and parameter settings are only available for the advanced and expert users. More specifically:
1. Standard: the simplest of the input forms will be displayed; defaults are taken wherever possible; it is not possible to use a map to generate a prior envelope; no fine tuning of the TNT weights is possible; one cannot change the connect/contact TNT files.
2. Advanced: the input form will allow the choice of the model and parameters to be used to model the bulk-solvent scattering; the radii and blurring factors for the generation of the envelopes can also be set; it is possible to use a map to generate a prior envelope, but only a low-pass filter can be used; occupancy refinement and rigid body refinement are possible; fine tuning of the TNT weights is possible; one can change the connect/convert TNT files
3. Expert: much like the Advanced, but now the high-pass filter can be used when generating a prior from a map; it is possible to toggle on/off the individual scaling parameters.
• Job submission scheme :  The preferred type of submission. In most cases, only interactive submission is allowed, but when several MASTERs are available, the user can select his/her preferred machine, submitting jobs either via remote shell, or through a queuing system, obviously the preferred way.

For more details please see the chapter about Submission mechanisms and queuing systems from the Installation manual.

• Displaying maps :  Various maps can be displayed: the MaxEnt Map (i.e. the density obtained modulating the prior-prejudice); the Prior (i.e. the distribution for the missing atoms, used as a starting point for the Maximum Entropy structure completion); and the usual centroid, 2Fo-Fc and Fo-Fc maps. Note that for the two difference maps are weighted differences of mFo and DFc.
• Map display : 
Here you can select the colours and contour levels that will be used by the plotting program (typically O) for the display of various maps. Contour levels are understood in units of map standard deviation from its mean value.
• Region Selection : You can either display the map in the whole unit cell, in the asymmetric unit, or contour it around a PDB model. The PDB models for the masking of the maps are to be picked up from the datafiles directory (with prefix "model_"), from the PDBs used in the BUSTER masks generation, or are refined fragments. The map-masking step is done via a call to the CCP4 program mapmask. The radius around the PDB can be selected here, as well as the viewing tool.

At the moment, the only choice is between plotmtv, that produces 2D contour plots of map sections, and "O" (Jones et. al.,1991). Hooks can be provided to connect other types of viewers to BUSTER (Xfit, Xplot84driver/npo...); these facilities are disabled by default, and you should contact your BUSTER administrator for more information.
If you select the "Save Maps and PDB files" option, you can write to disk a tar archive file with the objects you might want to display with the program of your choice.

• Peak picking :  The first parameter entered here governs the number of peaks in the map that will displayed, with cross flags; the peak list is built by the CCP4 program peakmax. The second parameter is a threshold above which a peak in the map is retained for checking the peak list.
• Skeletonisation :  If requested, the skeleton for the map is traced by the Uppsala program mapman, using the base level and increment entered here.
• Atomic models display : A number of objects can be displayed together with the maps. If you use O to examine these maps, you can (and are encouraged to) display the unit cell, and a number of pdb models:
• The unit cell: the crystallographic unit cell
• The skeleton: bones of tentative protein model drawn in the electron density map
• The peaks: peaks extracted from the difference map
• The fragment: the model for the substructure used as a fragment before refinement
• The rest: the (tentative) model for the fragment and the missing atoms that was used to define the missing atoms region, if any was given
• The refined fragment: the model for the substructure used as a fragment during the course of the refinement
• External model: any PDB in the datafiles directory whose name starts with "model_"

Note that, even if you tick any of the boxes for the PDB models, the requested model will be displayed only if the interface finds the corresponding PDB file in the appropriate directory.

• Archiving files : The user may wish to delete some files after a successful BUSTER run, or to save disk space when archiving final BUSTER jobs, i.e. when the user chooses to "save" a job from the interface. The selection of files to be kept is under partial user's control. The files that can be saved are:
• The refined fragments can be saved at various cycles during the course of the refinement,
• The graphs of the correlation coefficients can also be saved, and
• The Log-Likelihood gradient maps (which contain the primary information driving the refinement, before any convolution with atomic density) too.
• The external map generated during the phasing stage can be kept,
• The Maximum Entropy map showing the result of the completion of the missing structure, and
• The Prior map showing the region in which density is allowed to grow during ME completion can be saved too.
• The BUSTER phases and amplitudeqs file is an MTZ file containing the Fourier coefficients used to generate most maps used for inspection: the centroid, the Fo-Fc and 2Fo-Fc are obtained by FFT on the coefficients in this file. Because it contains the main results of the BUSTER run, this box is usually ticked.
• The TNT output contains mainly information about the quality of the refined fragment in terms of deviations from ideal stereochemistry and NCS overlap. It is very verbose, but quite informative.
• More : If you want control over more parameters to be given in this page, we are ready to upgrade it in the next version. Please write at buster-develop@globalphasing.com

References

1. CCP4. Collaborative Computational Project, Number 4 (1994). Acta Cryst. D50, 760-763.
2. Ten Eyck, L. F. (1973). Acta Cryst. A29, 486.
3. Bricogne, G. & Irwin, J.J. (1996) in Macromolecular Refinement - Proceedings of the CCP4 Study Weekend. SERC Daresbury Laboratory, Warrington, England; 85-92.
4. Bricogne, G. & Irwin J.J. (1997). In Crystallographic Computing 7 edited by P.E. Bourne and K.D.Watenpaugh, 1-9.
5. Tronrud, D. E., Ten Eyck, L. F., & Matthews, B. W. (1987). An Efficient General-Purpose Least-Squares Refinement Program for Macromolecular Structures. Acta Crystallogr A, 43, 489-501.
6. Tronrud, D. E. (1992). Conjugate-Direction Minimization - An Improved Method for the Refinement of Macromolecules. Acta Crystallogr A, 48 (November), 912-916.
7. Tronrud, D. E. (1996). Knowledge-Based B-Factor Restraints for the Refinement of Proteins. J App Cryst, 29 (2), 100-104.
8. Tronrud, D. E. (1997). The TNT Refinement Package. in Macromolecular Crystallography, Part B, Eds Charlie Carter and Robert Sweet, Volume 277 in Methods in Enzymology, pp 306-319.
9. Tronrud, D. E. (1999). The Efficient Calculation of the Normal matrix in least-squares refinement of macromolecular Structures. Acta Crystallogr A, 55, 700-703.