Frame alignment

Submitted by Ruda Santos on Fri, 2015-10-16 06:09

Forums

Magdistortion

Hi,

Do you perform mag distortion correction before or after drift correction/whole frame alignment? I used unblur to align my image stack and when I did the mag correction before alignment the power spectrum looked totally off, but it seemed normal after alignment and correction but the 3D refinement is 10 angstroms worse. Thanks!

timgrant Fri, 2015-10-16 08:39

Hi,

In general it shouldn't matter which way round you do it. I tend to do it on the super resolution frames, then immediately Fourier bin the frames by 2 and continue processing with these. This limits the effect of the interpolation.

What are the distortion values you are using in your correction?

Tim

Ruda Santos Fri, 2015-10-16 09:54

This dataset was taken at Janelia Farm, so we used the distortion parameters from the paper for 29,000 magnification. The images look really different when unblur is used before or after the magnification correction, as well as the CTF and the power spectrum (on eman2). I calculated the CTF using ctffind 4.0.16 and when I use unblur before the magnification correction the Thon rings with good fit are around 10.1 angstroms, whereas when I use unblur after the fit is 6.1 angstroms. The image also looks much clearer using unblur after distortion correction, but for some reason this way is not working although it seems better at first.

timgrant Fri, 2015-10-16 10:04

Hi,

If you are using the values in the paper, then you are introducing an edge into the frames. This can interfere with the frame alignment, and may explain why you get bad results if you correct the frames. If you want to correct the distortion before the frame alignment then I recommend only stretching the images. This is discussed in the bottom left paragraph on page 3 of the paper.

However, correcting your images after alignment should be fine, and whilst you will still have an edge, it shouldn't really matter. I'm not sure if i understood your first post, did you mean that after correcting the distortion your resolution was 10A worse than the same data without correction? Can you be a bit more specific about the test you did, and the results you obtained?

Thanks,

Tim

Ruda Santos Fri, 2015-10-16 10:42

Hi Tim,

Yes, I have a dataset with 2600 images that were took in January. I processed them without distortion correction and got 8A on the 3D Refinement using Relion 1.3. Now I used IMOD to convert from tif to mrc, then aligned the frames with unblur (saving the aligned frames) and finally used mag distortion correction. I ran the processing with Relion 1.3 and on the 3D classification/Refinement the map doesn't get better than 21A. I made another test using the same software on another dataset and got the same result. Just to confirm, let's say I try just stretching the images, I'd finally need to adjust the apix for every image before processing, right? Thanks!

Ruda

timgrant Fri, 2015-10-16 10:48

Hi,

Yes that is correct. However, if you used the provided values on already aligned images then you shouldn't need to redo it.

A drop from 8A to 21A suggests something is going very very wrong, more so than can be explained by applying a magnification distortion or even really movie alignment unless the movie alignment is going astronomically wrong, which would be obvious by looking at the images.

Did you redo all the processing, including the particle picking?

Thanks,

Tim

Ruda Santos Fri, 2015-10-16 10:57

Hi,

Yes I redid everything and got on average the same number of particles, the 2D classification also looked similar. What I don't understand is why the CTF results and power spectrum look so different if I align them before and after. I will try correcting first and also using the stretch only parameters anyway to see if I get a different result.

Thanks,

Ruda

timgrant Fri, 2015-10-16 11:09

Hi,

The edge was likely interfering with your alignment. which is why you get a difference before / after. The difference will probably go away if you do the stretch only parameters.

As i understand it, this won't explain your poor resolution as that was obtained correcting after alignment?

It's also worth noting that you probably don't have to worry about magnification distortion. You typically wouldn't have to worry about it until you get to 3 or 4 Angstroms resolution, and even then only on fairly large molecules.

Please let me know if you continue to have problems.

Tim

jgalaz Tue, 2018-06-26 05:37

[Sorry, I mistakenly posted my comment as a reply to the original post by Ruda. It seems more appropriate after your comment that "it shouldn't matter which way you do it" (I'll go back and delete the other repeated comment)]:

Hi,

I'm getting widely different results for "distortion angle" and "total distortion" from mag_distortion_estimate_1.0.1 depending on whether I run it on the raw frames or the motion-corrected sum.

If it's true that it shouldn't matter, I would expect for the results to be very close.

For example, running this command on the raw frames:

~/Downloads/mag_distortion_estimate_1.0.1/bin/mag_distortion_estimate_openmp_1_12_16.exe > stack7_d4_automag.log <<-'__MARKER__'
stack7_d4.mrc
stack7_d4_spectra.mrc
stack7_d4_rot.mrc
stack7_d4_corrected.mrc
1.12514030933
NO
__MARKER__

I get the following results:

Input stack filename [stack1.mrc] : stack7_d4.mrc
Output spectra filename [stack1_spectra.mrc] : stack7_d4_spectra.mrc
Output rot. avg. filename [stack1_rot.mrc] : stack7_d4_rot.mrc
Output corrected spectra [my_amp_corrected.mrc] : stack7_d4_corrected.mrc
Image Pixel size (Å) [1.0] : 1.12514030933
Set expert options? [NO] : NO
The following distortion parameters were found :-
Distortion Angle = 137.8
Major Scale = 1.002
Minor Scale = 0.998
Stretch only parameters would be as follows :-
Distortion Angle = 137.8
Major Scale = 1.004
Minor Scale = 1.000
Corrected Pixel Size = 1.123
The Total Distortion = .40%

While running this command on the aligned sum:

~/Downloads/mag_distortion_estimate_1.0.1/bin/mag_distortion_estimate_openmp_1_12_16.exe > aligned7_d4_automag.log <<-'__MARKER__'
aligned7_d4.mrc
aligned7_d4_spectra.mrc
aligned7_d4_rot.mrc
aligned7_d4_corrected.mrc
1.12514030933
NO
__MARKER__

Yields:

Input stack filename [aligned9_d4.mrc] : aligned7_d4.mrc
Output spectra filename
[aligned9_d4_spectra.mrc] : aligned7_d4_spectra.mrc
Output rot. avg. filename [aligned9_d4_rot.mrc] : aligned7_d4_rot.mrc
Output corrected spectra
[aligned9_d4_corrected.mrc] : aligned7_d4_corrected.mrc
Image Pixel size (Å) [1.078] : 1.12514030933
Set expert options? [NO] : NO
**warning(FileDelete): attempt to delete file which does not exist: .mag_distortion_estimate_VKmPmDbUNMjcxXsG
Reading images...
Calculating Spectra...
Finding distortion parameters...
100% [==============================] done!
The following distortion parameters were found :-
Distortion Angle = 78.7
Major Scale = 1.013
Minor Scale = 0.988
Stretch only parameters would be as follows :-
Distortion Angle = 78.7
Major Scale = 1.025
Minor Scale = 1.000
Corrected Pixel Size = 1.111
The Total Distortion = 2.52%

Does this suggests that
1) mag_distorsion_estimate performs less well on frames (perhaps due to lower signal? Not sure how the script would go about things when raw frames are the input; are they summed up first?), or rather
2) motion correction is actually making the images worse (introducing distortions that were not there), or
3) motion correction is making real distortions more apparent and easy to measure,
4) the distortion numbers are so low that no correction is actually needed?
5) the difference between distortion numbers obtained with each of the two procedures is too small to be relevant (the distortion angle difference seems quite large, though...)???? (at what distortion percentage does it start mattering?).

Anyway, the difference between numbers looks similar (sometimes better, sometimes worse) for other images I evaluated manually (one by one). The data were collected on a Titan Krios with a K2 camera and an energy filter, unfortunately not in super-resoluition mode. The specimen consists of large virus particles (~70 nm in diameter) in vitro.

Has anyone figured out whether one of the two procedures (finding mag distortion parameters before vs after motion correction) consistently yields more accurate results so that I can stick to the better practice when I process large dataset in conjunction with motion correction (e.g., MotionCor2 can take in the parameters found by mag_distorsion_estimate)?

Thanks for your help!

timgrant Tue, 2018-06-26 13:25

Hi,

Are you running this on images of particles?

mag_distortion_estimate is designed to be run on a stack of 10-20 images of different areas of a gold covered cross grating. You will need to take the images at the magnification you want to calibrate. Once you have run estimate on the cross grating images, you can use mag_distortion_correct to correct your particle images.

Cheers,

Tim

jgalaz Thu, 2018-06-28 01:03

Hi Tim,

Yes, I was running the script directly on data images.

Now that you've made me aware that the parameters need to be derived from gold covered cross grating grid images, we will perform the experiment properly to calibrate the data from our microscopes (we have the grids and our microscopy managers have agreed to do the experiment). Thanks for clarifying! Hopefully, we'll see significant improvements in our data after accounting for mag distortion :-)