Beam tilt estimation - group micrographs?
Hi Sjors,
The option of refining beam tilt seems like it could be very helpful for those of us collecting using moderate image shift!
In such cases (e.g. see attached), we have decent (?) prior values for beam tilt on a per micrograph basis, and it would be great to be able to refine these per micrograph or by dividing the dataset into beam tilt groups (e.g. 9 groups in the attached example).
Currently we are doing this ad hoc by splitting the dataset into nine separate datasets, but being able to use all the data at once while refining beam tilt values seems like it would give better results.
Cheers Oli
Comments (25)
-
repo owner -
repo owner Meanwhile, just proceed like you are doing, and use joinStar jobtype to combine all nine again later. A bit cumbersome, but doing exactly this is what got Wim's data from 2.9 to 2.2A, like he showed at the GRC. We are now writing this up. S
-
reporter Great, thanks Sjors!
Oli
-
Hi Oli,
Now you can define
rlnBeamTiltGroupName
in your particle STAR file and each group will be refined separetely.Takanori & Jasenko
-
- changed status to resolved
-
reporter That's great, thanks so much Takanori and Jasenko! Look forward to trying it!
-
reporter Hi Takanori, Sjors, Jasenko,
Anchi Cheng has a suggestion re this feature, see attached - she suggests that supporting refinement of the center of the beam tilt distribution (when the input star file already contains per micrograph beam tilt values estimated from image shift) would be a useful addition. I guess this would mean refining one beam tilt group, but using the overall beam tilt to adjust each estimated per micrograph value present in the input star.
Cheers Oli
-
Hi,
We implemented rlnBeamTiltGroupName support in FOCUS where we give a label (an integer number) to each movie corresponding to the beam tilt group it belongs to, in the output micrographs_ctf.star file from FOCUS. However it seems that RELION-3 throws away this column upon particle extraction. Is it a bug or are we doing something wrong?
Thanks, Ricardo
-
Hi Ricardo,
The commit 7c7a874 will copy rlnBeamTiltGroupName from the micrograph STAR file when it is absent in the particle STAR file.
-
Thanks Takanori, will test it on our micrograph STAR files!
-
Hello again,
It doesn't seem to work as expected - when we extract particles from a micrograph STAR file with the rlnBeamTiltGroupName column, the first particle from each micrograph seems to take the correct value, but the remainder of the particles for that micrograph receives an 'empty' value in this column. Grouped BeamTilt refinement later then doesn't work as expected. If it helps you I could send you the corresponding STAR files via e-mail.
Thanks once again!
-
Confirmed. Sorry, I wasn't careful enough. Fixing now...
-
Hi Ricardo,
The commit 99d96a3 should fix the problem.
-
Hello again Takanori,
Now the particle STAR files get the rlnBeamTiltGroupName values correctly from the micrograph STAR file, thanks a lot for correcting this. Now the problem is with the beam tilt estimation itself: RELION still refines the whole dataset as a single beam tilt group. Do we have to pass a special flag or something to relion_ctf_refine_mpi in order to enable grouped beam tilt refinement?
Thanks a lot!
-
I am sorry but we realised that we had two redundant columns
rlnBeamTiltGroupName
andrlnBeamTiltClass
. The first is string and the latter is integer. We decided to remove the first (commit 6a1a78a9). So please userlnBeamTiltClass
instead. -
Hi Takanori,
Sorry, I forgot to confirm earlier that the grouped beam tilt refinement is now working nice! The output has quite some verbose on the beam tilt classes - I guess some kind of sanity check - but that's fine. For example see: https://pastebin.com/BkFfLZQt
Thanks a lot for solving this issue! Merry Christmas to you and the RELION team.
-
Hi Takanori and RELION team,
I'm afraid the issue that commit 7c7a874 was supposed to fix is back: if an imported micrographs.star contains the rlnBeamTiltClass column, it is again throwing it away upon particle extraction...
Thanks in advance!
-
When did the problem come back? Are you extracting from AutoPick, or re-extracting from particles.star?
-
I'm using the latest version of the source code from this repo and noticed the problem for the first time today. I am extracting from AutoPick.
-
When was it working before? Were you also extracting from AutoPick at that time?
-
On 13-12-2018 I confirmed that the extraction was working. That was the last time I tried to do this before today for a dataset with multiple rlnBeamTiltClass values. At that time it was from a re-extraction from a particles.star file and we were still using the rlnBeamTiltGroupName label, which was later deprecated in favor of rlnBeamTiltClass, which I manually changed in the .star file. I just tested and at the moment it's not working either for AutoPick nor re-extraction with my .star files.
-
Hi, it works just fine here. My input micrograph STAR file contains the following columns:
_rlnMicrographName #1 _rlnCtfImage #2 _rlnDefocusU #3 _rlnDefocusV #4 _rlnCtfAstigmatism #5 _rlnDefocusAngle #6 _rlnVoltage #7 _rlnSphericalAberration #8 _rlnAmplitudeContrast #9 _rlnMagnification #10 _rlnDetectorPixelSize #11 _rlnCtfFigureOfMerit #12 _rlnCtfMaxResolution #13 _rlnBeamTiltClass #14
-
Strange. My input micrographs.star file contains the following columns:
_rlnMicrographName
#1_rlnCtfImage#2_rlnDefocusU#3_rlnDefocusV#4_rlnDefocusAngle#5_rlnVoltage #6 _rlnSphericalAberration#7_rlnAmplitudeContrast#8_rlnMagnification#9_rlnDetectorPixelSize #10 _rlnCtfFigureOfMerit#11_rlnCtfMaxResolution#12_rlnPhaseShift#13_rlnBeamTiltClass #14But the extracted particles.star contains:
_rlnCoordinateX
#1_rlnCoordinateY#2_rlnAutopickFigureOfMerit#3_rlnClassNumber#4_rlnAnglePsi#5_rlnImageName #6 _rlnMicrographName#7_rlnMagnification#8_rlnDetectorPixelSize#9_rlnCtfMaxResolution #10 _rlnCtfFigureOfMerit#11_rlnVoltage#12_rlnDefocusU#13_rlnDefocusV #14 _rlnDefocusAngle #15 _rlnSphericalAberration#16_rlnCtfBfactor #17 _rlnCtfScalefactor#18_rlnPhaseShift#19_rlnAmplitudeContrast#20_rlnOriginX#21_rlnOriginY#22I even did git pull (already up-to-date) and re-compiled to make sure I am running the latest version.
-
Are you sure your
PATH
points to the right directory? Did you also trymake clean
beforemake
? -
You are right: the problem was with my PATH in my cluster submission script for CPU jobs that was pointing to a wrong, outdated version. All other submission scripts were fine, that's why I didn't notice it before. It works indeed. Sorry for the confusion!
- Log in to comment
Hi Oli, This is a useful suggestion. It will take a bit of time to do this... thanks, Sjors