Content:


Introduction

Eiger X detectors from Dectris can create datasets in 16- or 32-bit image depth. This is usually specified in the /entry/instrument/detector/bit_depth_image dataset of a HDF5 *_master.h5 file:

  h5dump -d /entry/instrument/detector/bit_depth_image some_master.h5

will give something like

HDF5 "some_master.h5" {
DATASET "/entry/instrument/detector/bit_depth_image" {
   DATATYPE  H5T_STD_I32LE
   DATASPACE  SCALAR
   DATA {
   (0): 16
   }
}
}

Another way to check is via e.g.

  h5dump -d /entry/data/data_000001 some_master.h5

which would show something like

HDF5 "some_master.h5" {
DATASET "/entry/data/data_000001" {
   DATATYPE  H5T_STD_U16LE
...

See also:


Problem

A data image should always have a clear indication which pixels are active and which are inactive (for whatever reason) and should therefore be ignored by the processing software. This is usually done by given a particular pixel a negative value (e.g. -1 or -2): data-processing software then knows to ignore such a pixel (since a nagative number of counts don't make much sense).

Constantly inactive pixels are usualy described in a so-called pixel mask and the data reading part of software will apply the information accordingly, i.e. mask those pixels before transfering the raw data array to the processing part of the software.

However, there are also pixels that are only sometimes inactive (while in other circumstances they are active). These can't be included into the static/constant pixel mask and must be marked as such in the actual image array. This is done via the value

2^16 - 1   # for 16-bit (unsigned int) images
2^32 - 1   # for 32-bit (unsigned int) images

It is then the responsibility of the data-reading software, to (1) know if the image comes in 16- or 32-bit, (2) check for the appropriate marker value and (3) change the pixel value to be negative.

Different software handles this to a different degree:

Software Result Remark
hdf2mini-cbf (part of autoPROC, Global Phasing) hdf2mini-cbf_16.png correct detection of inactive pixel
H5ToXDS (Dectris) H5ToXds_16.png missed detection of inactive pixel
eiger2cbf (see also here) eiger2cbf_16.png missed detection of inactive pixel
Neggia plugin for XDS (Dectris) aP.05.neggia_COLSPOT_crop.png incorrect handling of inactive pixel, leading to nearly 25% of spots getting rejected.
Durin plugin for XDS (Diamond Light Source) aP.05.durin_COLSPOT_crop.png correct handling of inactive pixel
 

Does it matter? After all, these should only ever be a few pixels, right?

The problem occurs when the spot search in COLSPOT is performed in XDS. Those temporarily active pixels can look like spots to XDS - mainly because they can span several images before becoming active and seem like real spots. The OVERLOAD value has no effect at that stage: the idea being that overloads in e.g. CCD data (much more common than with pixel array detectors) can still be useful for indexing, even if later they would be ignored in integration.

So it doesn't really matter, right?

Not quite - because the spot list might contain a larger number of very strong (fake) "spots" when those inactive pixels are given to XDS unmasked/unmarked. This means that the notion of "strong" and "weak" spots is skewed and we see a surprisingly large number of "weak" spots wrongly excluded from the output spot list within COLSPOT.

So what? I'll be only missing all those weak reflections ...

True - if you are prepared to ignore weak reflections and pretend that the crystal created only those strong diffraction peaks. However, it seems a bad idea to just ignore a substantial number of spots that could tell you about the correct spacegroup/cell, translational NCS or any other crystal characteristics.


Solution

There are several approaches/stages:

  • With software as it is now: use the correct software for image conversion and processing:
    • hdf2mini-cbf for image conversion
    • durin plugin for processing (e.g. in autoPROC adding the keyword autoPROC_Img2Xds_UseXdsPlugins_DectrisHdf5=durin-plugin)