Beam tilt estimation - group micrographs?

Issue #3 resolved
Oliver Clarke created an issue

Hi Sjors,

The option of refining beam tilt seems like it could be very helpful for those of us collecting using moderate image shift!

In such cases (e.g. see attached), we have decent (?) prior values for beam tilt on a per micrograph basis, and it would be great to be able to refine these per micrograph or by dividing the dataset into beam tilt groups (e.g. 9 groups in the attached example).

Currently we are doing this ad hoc by splitting the dataset into nine separate datasets, but being able to use all the data at once while refining beam tilt values seems like it would give better results.

Cheers Oli


Comments (25)

  1. Sjors Scheres repo owner

    Hi Oli, This is a useful suggestion. It will take a bit of time to do this... thanks, Sjors

  2. Sjors Scheres repo owner

    Meanwhile, just proceed like you are doing, and use joinStar jobtype to combine all nine again later. A bit cumbersome, but doing exactly this is what got Wim's data from 2.9 to 2.2A, like he showed at the GRC. We are now writing this up. S

  3. Takanori Nakane

    Hi Oli,

    Now you can define rlnBeamTiltGroupName in your particle STAR file and each group will be refined separetely.

    Takanori & Jasenko

  4. Oliver Clarke reporter

    Hi Takanori, Sjors, Jasenko,

    Anchi Cheng has a suggestion re this feature, see attached - she suggests that supporting refinement of the center of the beam tilt distribution (when the input star file already contains per micrograph beam tilt values estimated from image shift) would be a useful addition. I guess this would mean refining one beam tilt group, but using the overall beam tilt to adjust each estimated per micrograph value present in the input star.

    Cheers Oli

    Screen Shot 2018-10-02 at 10.41.24 AM.png

  5. Ricardo Righetto


    We implemented rlnBeamTiltGroupName support in FOCUS where we give a label (an integer number) to each movie corresponding to the beam tilt group it belongs to, in the output file from FOCUS. However it seems that RELION-3 throws away this column upon particle extraction. Is it a bug or are we doing something wrong?

    Thanks, Ricardo

  6. Takanori Nakane

    Hi Ricardo,

    The commit 7c7a874 will copy rlnBeamTiltGroupName from the micrograph STAR file when it is absent in the particle STAR file.

  7. Ricardo Righetto

    Hello again,

    It doesn't seem to work as expected - when we extract particles from a micrograph STAR file with the rlnBeamTiltGroupName column, the first particle from each micrograph seems to take the correct value, but the remainder of the particles for that micrograph receives an 'empty' value in this column. Grouped BeamTilt refinement later then doesn't work as expected. If it helps you I could send you the corresponding STAR files via e-mail.

    Thanks once again!

  8. Ricardo Righetto

    Hello again Takanori,

    Now the particle STAR files get the rlnBeamTiltGroupName values correctly from the micrograph STAR file, thanks a lot for correcting this. Now the problem is with the beam tilt estimation itself: RELION still refines the whole dataset as a single beam tilt group. Do we have to pass a special flag or something to relion_ctf_refine_mpi in order to enable grouped beam tilt refinement?

    Thanks a lot!

  9. Takanori Nakane

    I am sorry but we realised that we had two redundant columns rlnBeamTiltGroupName and rlnBeamTiltClass. The first is string and the latter is integer. We decided to remove the first (commit 6a1a78a9). So please use rlnBeamTiltClass instead.

  10. Ricardo Righetto

    Hi Takanori,

    Sorry, I forgot to confirm earlier that the grouped beam tilt refinement is now working nice! The output has quite some verbose on the beam tilt classes - I guess some kind of sanity check - but that's fine. For example see:

    Thanks a lot for solving this issue! Merry Christmas to you and the RELION team.

  11. Ricardo Righetto

    Hi Takanori and RELION team,

    I'm afraid the issue that commit 7c7a874 was supposed to fix is back: if an imported contains the rlnBeamTiltClass column, it is again throwing it away upon particle extraction...

    Thanks in advance!

  12. Takanori Nakane

    When did the problem come back? Are you extracting from AutoPick, or re-extracting from

  13. Ricardo Righetto

    I'm using the latest version of the source code from this repo and noticed the problem for the first time today. I am extracting from AutoPick.

  14. Ricardo Righetto

    On 13-12-2018 I confirmed that the extraction was working. That was the last time I tried to do this before today for a dataset with multiple rlnBeamTiltClass values. At that time it was from a re-extraction from a file and we were still using the rlnBeamTiltGroupName label, which was later deprecated in favor of rlnBeamTiltClass, which I manually changed in the .star file. I just tested and at the moment it's not working either for AutoPick nor re-extraction with my .star files.

  15. Takanori Nakane

    Hi, it works just fine here. My input micrograph STAR file contains the following columns:

    _rlnMicrographName #1 
    _rlnCtfImage #2 
    _rlnDefocusU #3 
    _rlnDefocusV #4 
    _rlnCtfAstigmatism #5 
    _rlnDefocusAngle #6 
    _rlnVoltage #7 
    _rlnSphericalAberration #8 
    _rlnAmplitudeContrast #9 
    _rlnMagnification #10 
    _rlnDetectorPixelSize #11 
    _rlnCtfFigureOfMerit #12 
    _rlnCtfMaxResolution #13 
    _rlnBeamTiltClass #14
  16. Ricardo Righetto

    Strange. My input file contains the following columns:

    _rlnMicrographName #1 _rlnCtfImage #2 _rlnDefocusU #3 _rlnDefocusV #4 _rlnDefocusAngle #5 _rlnVoltage #6 _rlnSphericalAberration #7 _rlnAmplitudeContrast #8 _rlnMagnification #9 _rlnDetectorPixelSize #10 _rlnCtfFigureOfMerit #11 _rlnCtfMaxResolution #12 _rlnPhaseShift #13 _rlnBeamTiltClass #14

    But the extracted contains:

    _rlnCoordinateX #1 _rlnCoordinateY #2 _rlnAutopickFigureOfMerit #3 _rlnClassNumber #4 _rlnAnglePsi #5 _rlnImageName #6 _rlnMicrographName #7 _rlnMagnification #8 _rlnDetectorPixelSize #9 _rlnCtfMaxResolution #10 _rlnCtfFigureOfMerit #11 _rlnVoltage #12 _rlnDefocusU #13 _rlnDefocusV #14 _rlnDefocusAngle #15 _rlnSphericalAberration #16 _rlnCtfBfactor #17 _rlnCtfScalefactor #18 _rlnPhaseShift #19 _rlnAmplitudeContrast #20 _rlnOriginX #21 _rlnOriginY #22

    I even did git pull (already up-to-date) and re-compiled to make sure I am running the latest version.

  17. Takanori Nakane

    Are you sure your PATH points to the right directory? Did you also try make clean before make?

  18. Ricardo Righetto

    You are right: the problem was with my PATH in my cluster submission script for CPU jobs that was pointing to a wrong, outdated version. All other submission scripts were fine, that's why I didn't notice it before. It works indeed. Sorry for the confusion!

  19. Log in to comment