This page is work in progress - please update from time to time.

Content:

Introduction

14 July 2025

A new XDS binary has been released that seems to deal with the problems of background determination analysed and reported back in March 2025.

Quick initial overview

Let's first go back to those artificail test images (without and with marked pixels) used in the original analysis. We can again compare the background (after the XDS/INIT step) between the different XDS versions for the initial background images (top row) and those with the additional markers (bottom row):

type close-up (click for larger version)
single montage_BKGINIT_500-250_wide.png
vertical strip montage_BKGINIT_1250-250_wide.png
horizontal strip montage_BKGINIT_500-750_wide.png

The marked/masked pixels are shown in red. The 2024 and previous 2025 versions showed a clear (negative, i.e. lower background value) imprint of the masked pixel in the surrounding background table. This is now handled better by the 20250714 binary. Looking at the differences:

type wide view (click for larger version)
single montage_1-0_BKGINIT_500-250_wide.png
vertical strip montage_1-0_BKGINIT_1250-250_wide.png
horizontal strip montage_1-0_BKGINIT_500-750_wide.png

This shows still a small "imprint" of the masked pixels, but it is much less pronounced with the latest 20250714 binary.

Another (real-life) example - 4FQN

Let's take 4FQN as an example: 

rsync -av rsync://data.sbgrid.org/10.15785/SBGRID/561 .
ln -s 561 Images
mkdir BackgroundImages

ln -s ../Images/OF-T2P3_1_00011.cbf BackgroundImages/ForBackgroundOnly_00001.cbf
ln -s ../Images/OF-T2P3_1_00012.cbf BackgroundImages/ForBackgroundOnly_00002.cbf
ln -s ../Images/OF-T2P3_1_00013.cbf BackgroundImages/ForBackgroundOnly_00003.cbf
ln -s ../Images/OF-T2P3_1_00014.cbf BackgroundImages/ForBackgroundOnly_00004.cbf
ln -s ../Images/OF-T2P3_1_00015.cbf BackgroundImages/ForBackgroundOnly_00005.cbf
ln -s ../Images/OF-T2P3_1_00016.cbf BackgroundImages/ForBackgroundOnly_00006.cbf
ln -s ../Images/OF-T2P3_1_00017.cbf BackgroundImages/ForBackgroundOnly_00007.cbf
ln -s ../Images/OF-T2P3_1_00018.cbf BackgroundImages/ForBackgroundOnly_00008.cbf
ln -s ../Images/OF-T2P3_1_00019.cbf BackgroundImages/ForBackgroundOnly_00009.cbf
ln -s ../Images/OF-T2P3_1_00020.cbf BackgroundImages/ForBackgroundOnly_00010.cbf
ln -s ../Images/OF-T2P3_1_00051.cbf BackgroundImages/ForBackgroundOnly_00011.cbf
ln -s ../Images/OF-T2P3_1_00052.cbf BackgroundImages/ForBackgroundOnly_00012.cbf
ln -s ../Images/OF-T2P3_1_00053.cbf BackgroundImages/ForBackgroundOnly_00013.cbf
ln -s ../Images/OF-T2P3_1_00054.cbf BackgroundImages/ForBackgroundOnly_00014.cbf
ln -s ../Images/OF-T2P3_1_00055.cbf BackgroundImages/ForBackgroundOnly_00015.cbf
ln -s ../Images/OF-T2P3_1_00056.cbf BackgroundImages/ForBackgroundOnly_00016.cbf
ln -s ../Images/OF-T2P3_1_00057.cbf BackgroundImages/ForBackgroundOnly_00017.cbf
ln -s ../Images/OF-T2P3_1_00058.cbf BackgroundImages/ForBackgroundOnly_00018.cbf
ln -s ../Images/OF-T2P3_1_00059.cbf BackgroundImages/ForBackgroundOnly_00019.cbf
ln -s ../Images/OF-T2P3_1_00060.cbf BackgroundImages/ForBackgroundOnly_00020.cbf
ln -s ../Images/OF-T2P3_1_00091.cbf BackgroundImages/ForBackgroundOnly_00021.cbf
ln -s ../Images/OF-T2P3_1_00092.cbf BackgroundImages/ForBackgroundOnly_00022.cbf
ln -s ../Images/OF-T2P3_1_00093.cbf BackgroundImages/ForBackgroundOnly_00023.cbf
ln -s ../Images/OF-T2P3_1_00094.cbf BackgroundImages/ForBackgroundOnly_00024.cbf
ln -s ../Images/OF-T2P3_1_00095.cbf BackgroundImages/ForBackgroundOnly_00025.cbf
ln -s ../Images/OF-T2P3_1_00096.cbf BackgroundImages/ForBackgroundOnly_00026.cbf
ln -s ../Images/OF-T2P3_1_00097.cbf BackgroundImages/ForBackgroundOnly_00027.cbf
ln -s ../Images/OF-T2P3_1_00098.cbf BackgroundImages/ForBackgroundOnly_00028.cbf
ln -s ../Images/OF-T2P3_1_00099.cbf BackgroundImages/ForBackgroundOnly_00029.cbf
ln -s ../Images/OF-T2P3_1_00100.cbf BackgroundImages/ForBackgroundOnly_00030.cbf

and then run different XDS version using 4FQN_XDS.INP as XDS.INP.

Let's then look at four particular pixel regions (images are centred on the pixel with +- 10 pixels border): we show (left) a plot using GPX2 (with actual values) and on the right one that tries to use color over the full range:

first image all images 20230630 20250430 20250714
4FQN_Images_OF-T2P3_1_00001_358_277_combo.png 4FQN_ForBackgroundOnly_358-277.png 4FQN_20230630_BKGINIT_358_277_combo.png 4FQN_20250430_BKGINIT_358_277_combo.png 4FQN_20250714_BKGINIT_358_277_combo.png
4FQN_Images_OF-T2P3_1_00001_796_260_combo.png 4FQN_ForBackgroundOnly_358-277.png 4FQN_20230630_BKGINIT_796_260_combo.png 4FQN_20250430_BKGINIT_796_260_combo.png 4FQN_20250714_BKGINIT_796_260_combo.png
4FQN_Images_OF-T2P3_1_00001_384_296_combo.png 4FQN_ForBackgroundOnly_358-277.png 4FQN_20230630_BKGINIT_384_296_combo.png 4FQN_20250430_BKGINIT_384_296_combo.png 4FQN_20250714_BKGINIT_384_296_combo.png
4FQN_Images_OF-T2P3_1_00001_384_304_combo.png 4FQN_ForBackgroundOnly_358-277.png 4FQN_20230630_BKGINIT_384_304_combo.png 4FQN_20250430_BKGINIT_384_304_combo.png 4FQN_20250714_BKGINIT_384_304_combo.png

Misfits along module gap boundaries

Reported to XDS developers on 20250718

Does that improved behaviour (close to masked pixels) of the 20250714 binary also affect the classification of integrated reflections as so-called "misfits" in CORRECT?

Let's look at the number of misfits as a function of X and Y coordinate, respectively:

4FQN_plot_misfits_xy.png

This shows (click on image for bigger version) a larger number of misfits close to the (indicated) module gaps - both in X (vertical gaps) and Y (horizontal gaps). It seems as if something else is at play here, requiring further investigation.

Note: the default and recommended autoPROC behaviour is to not use reflection data after the CORRECT step, but taking the INTEGRATE.HKL file directly into AIMLESS for scaling and subsequent STARANISO analysis. So the outlier analysis might be different and less visible around the module gaps - although we expect this to be an intrinsic characteristic of the integrated intensities and not of the outlier detection mechanism itself (so it should/could affect AIMLESS to a similar extent).