The Eiger detector is a new detector from Dectris that also produces data in a new format: HDF5. We added support for this into autoPROC based on Eiger 16M data provided by Dectris and SLS.


The solutions shown below (via imginfo wrapper scripts) are not our preferred way of handling these newer detectors. We hope that updates to the software at beamlines will result in missing data to be added or incorrect items being corrected.

Eiger 16M at SLS, X06SA (PX-I)

See also:


% process -h5 some_master.h5 -d outdir | tee out.lis

should be all that is required for processing. All other standard autoPROC flags and options can be used.

Thanks to Dirk Reinert (Boehringer-Ingelheim), Meitian Wang (SLS), Andreas Foerster (Dectris), Joachim Diez (Expose), Stefan Brandstetter (Dectris), Zac Panepucci (SLS), Kay Diederichs (Konstanz) and many more.

Eiger 4M at ESRF, MASSIF-3

See also:

Earlier datasets:

         process dist=123.45 beam="1234.5 1234.6" ...

After downloading and making it executable via

% chmod +x

you can run something like

% process -h5 some_master.h5 \
        autoPROC_XdsKeyword_UNTRUSTED_RECTANGLE="257 483 551 809" \
        imginfo=`pwd`/ osc=0.05 \
        -d outdir | tee out.lis

(obviously adapting to your local setup and dataset characteristics).

Thanks to David von Stetten (ESRF) for test data and discussions.

Eiger 9M at APS 21-ID-D

See also:


      % process -h5 h.001_master.h5 imginfo=`pwd`/ osc=0.5 -d 01 | tee 01.lis

Thanks to Joe Brunzelle (LS-CAT) for providing test data.

Eiger 9M at Soleil Proxima2

See also:


        /entry/instrument/detector/detectorSpecific/eiger_fw_version = eiger-1.5.2-101.git5b1f904.release
        /entry/instrument/detector/detectorSpecific/software_version = 1.5.2
        /entry/instrument/detector/detector_number = E-18-0102


        /entry/instrument/detector/detectorSpecific/eiger_fw_version = eiger-1.6.1-104.git3f17ecb.release
        /entry/instrument/detector/detectorSpecific/software_version = 1.6.1
        /entry/instrument/detector/detector_number = E-18-0102
            % process -h5 Lyso_U2_1_master.h5 imginfo=`pwd`/ autoPROC_XdsKeyword_SENSOR_THICKNESS=0.45 -d 01 | tee 01.lis

Note: autoPROC releases from 20160324 onwards contain already this wrapper, so running

 % process -h5 h.001_master.h5 autoPROC_XdsKeyword_SENSOR_THICKNESS=0.45 -d 01 | tee 01.lis

should work.

Thanks to Martin Savko and Bill Shepard (Soleil) as well as Nicholas Keep (Birkbeck, London) for providing test data.

Eiger 4M at Photon Factory BL1A

See also:


      % process -h5 h.001_master.h5 imginfo=`pwd`/ osc=0.5 -d 01 | tee 01.lis

Note: autoPROC releases from 20160324 onwards contain already this wrapper, so running

 % process -h5 h.001_master.h5 osc=0.5 -d 01 | tee 01.lis

should work.

Thanks to Takaaki Fukami (Chugai Pharmaceutical) for providing feedback and test data.

Eiger 9M at SPring-8 BL32XU


% process -h5 some_master.h5 ReversePhi=yes -d outdir | tee out.lis

should be all that is required for processing. All other standard autoPROC flags and options can be used.

Thanks to Keitaro Yamashita (SPring-8) for providing information and test data.

Eiger 16M at MAX IV (BioMAX)

The master file contains the definition of the rotation axis, but autoPROC currently doesn't read this item. So one needs to run

% process autoPROC_XdsKeyword_ROTATION_AXIS="0 -1 0" -h5 some_master.h5 ...

for now.

Thanks to Jie Nan and Uwe Mueller for providing test data and help.

Eiger 16M at Australian Synchrotron (MX-2)

As of 20170920, HDF5 datasets from the Eiger 16M at MX-2 of the Australian Synchrotron record inconsistent information about rotation axes in the master.h5 file: both "Omega" and "Phi" are marked as rotation axes. Both will contain start and end values according to the oscillation angle and the "imginfo" tool from autoPROC returns something like

 >>> Image format detected as HDF5/Eiger

     Image number 1/2000

 ===== Header information:
 date                                = 20 Sep 2017 15:31:54.976
 distance                       [mm] = 380.032
 wavelength                      [A] = 0.953729
 sensor thickness               [mm] = 0.450
 Phi-angle (start, end)     [degree] = 180.00000 180.10001
 Oscillation-angle in Phi   [degree] = 0.10001
 Omega-angle (start, end)   [degree] = 180.00000 180.10001
 Oscillation-angle in Omega [degree] = 0.10001
 Chi-angle                  [degree] = 0.00000
 Kappa-angle                [degree] = 0.00000
 2-Theta angle              [degree] = 0.00000
 Pixel size in X                [mm] = 0.075000
 Pixel size in Y                [mm] = 0.075000
 Number of pixels in X               = 4150
 Number of pixels in Y               = 4371
 Beam centre in X               [mm] = 155.187
 Beam centre in X            [pixel] = 2069.160
 Beam centre in Y               [mm] = 167.042
 Beam centre in Y            [pixel] = 2227.230
 Overload value                      = 38079

 WARNING: more than one angle marked with different start/stop values!
 WARNING: oscillation around more than one axis?

Obviously, not both axes (Omega and Phi) would have rotated during the experiment (since this would correspond to a total rotation of 0.2 degree in case of a Kappa-goniostat with Kappa=0.0 ... or of a total rotation of 0.0 if they rotate in opposite directions at that setting). So the above will confuse autoPROC and could result in failed or poor processing.

Until this has been fixed at the beamline - and in order to (re)process already collected datasets - a workaround is required to patch those header information up. Please download the script, make it executable via

% chmod 0755 /where/ever/

and use it in autoPROC:

% process imginfo=/where/ever/ ...

Please note that

     % process BeamCentreFrom=header:y,x ReversePhi=yes ...

Thanks to Richard Birkinshaw (University Melbourne) for providing example datasets for testing. The above script will be included into the next autoPROC release.

HDF5 converter

autoPROC provides a converter for HDF5 datasets into CBF format (including full mini-cbf headers). It supports both bitshuffle as well as LZ4 compression. Please note that is will only convert HDF5 datasets without applying any of the necessary fixes discussed above (which are necessary because of issues mostly at the metadata population stage). In order to obtain correct miniCBF headers, the metadata of the input HDF5 file needs to be correct.

The converter can be used to extract single images via e.g.

% hdf2mini-cbf -m some_master.h5 -image 47

or the complete dataset using

% hdf2mini-cbf -m some_master.h5

For additional help please see

% hdf2mini-cbf -h
USAGE: hdf2mini-cbf [-h] [-linkrange <i1> <i2>] [-image <imgnum>] -m <master.h5> [-o <prefix>] [-uncompressed|-compressed|-gzip-level <level>]

Additional options/features are

An additional tool is provided for very fast conversion by distributing the extraction/conversion to multiple compute nodes. See

% aP_convert_hdf5 -h

for details.

USAGE: aP_convert_hdf5 [-h] [-v] [-t] [-compressed] -m <master> -o <prefix>

       -h                    : show help

       -v                    : increase verbosity (default = 0)

       -t                    : don't actually run the command(s), but show
                               what would be done

       -m <master>           : HDF5 *_master.h5 file

       -o <prefix>           : prefix (default=__169826_) for output mini-cbf
                               files (default=__169826__######.cbf)

       -compressed           : (optional) write *.cbf.gz files directly

Some settings - current default are adequate for SLS computing setup - can be given on the command-line (as par=value):

Note: the resulting miniCBF files take full advantage of the "signed 32-bit integer" data type used for storing the image array (see "X-Binary-Element-Type:" item). MOSFLM only supports pixel values from (and including) -2 and higher. It will therefore interpret other pixel markers below that threshold incorrectly, leading to wrong or completely failing processing (thanks to Harry Powell for confirmation). We are working on a MOSFLM-compatibility option for our hdf2mini-cbf converter for the next release. XDS will interpret the full "signed 32-bit integer" range correctly, but we have no information for any other data-processing package.

Using to read HDF5 files in XDS directly.

A collaboration between Dectris and the XDS developers one can now specify an external library, such that XDS can read HDF5 files directly. See eg. Dectris.

The use of this in autoPROC is not yet automatic. Meanwhile you can do:

% process -h5 data/abc_1_master.h5 \
           autoPROC_XdsKeyword_LIB=/path/to/ \
           -d aP-01 >& aP-01.txt

That is:

The library is available from Dectris for CentOS6 and CentOS7. It is also available in source form from GitHub.

See also recent CCP4bb discussion.