Warning: skipped refinement step in 4.0.10

Submitted by Marcus on Thu, 2015-02-19 10:57

Forums

ctffind & ctftilt

Dear Alexis,

first of all thanks for all the work on ctffind.
However, I was now running into a problem using the newest version of ctffind. For almost all my micrographs, ctffind is not able to refine the ctf values. In version 4.0.7 it works and therefore there is quite a significant difference in the estimated ctf values. The output of the logfile is below. Also the image of the ctf estimation is looking weird, as there seem to be things left out (see attached image).

**error (parse_command_line_arguments): unrecognised flag: --old-school-input

** Welcome to CTFFind **
Version - 4.0.10

Input Mode: Batch
Date & Time: 2015-02-18 17:16:14

Copyright 2015 Howard Hughes Medical Institute. All rights reserved.
Use is subject to Janelia Farm Research Campus Software Copyright 1.1
license terms ( http://license.janelia.org/license/jfrc_copyright_1_1.html )

CS[mm], HT[kV], AmpCnst, XMAG, DStep[um]
2.3 300.0 0.10 39683.0 5.000

Summary information for file mics/align_15feb03a_XXX_2_00046gr_00007sq_v01_00038hl_00002en.frames.mrc
Number of columns, rows, sections: 3710 3710 1
MRC data mode: 2
Bit depth: 32
Pixel size: .000 .000 .000

Working on micrograph 1 of 1

SEARCHING CTF PARAMETERS...

100% [==============================] done!

DFMID1 DFMID2 ANGAST CC
19000.00 19000.00 -20.00 0.13643

REFINING CTF PARAMETERS...
DFMID1 DFMID2 ANGAST CC
**warning(VA04): maximum change does not alter function
19000.00 19000.00 -20.00 0.13643 Final Values

**warning(Image::ComputeRotationalAverageOfPowerSpectrum): unable to rescale average experimental power
Estimated defocus values : 19000.00 , 19000.00 Angstroms
Estimated azimuth of astigmatism: -20.00 degrees
Score : .13643
Thon rings with good fit up to : 5.8 Angstroms

Alexis Thu, 2015-02-19 11:13

Hi Marcus,

Thanks for your message.

There are possibly a few things going on there:
- firstly, it seems the flag --old-school-input was not recognized, or that it was, but the program still output an error message (first line of your output)
- something went wrong during the refinement. Perhaps you had dAst set to 0.0?
- the cross-supression part of the algorithm clearly went overboard on this; it looks like you're using a smaller box than what I was developing with (256 and 512 are the two box sizes I was using when developing)

To debug this further, could you show me the full input to ctffind? Also, ideally please send me (email) this micrograph so I can reproduce & debug on my end.

Sorry for these errors, and thanks for reporting them! I hope in the next few weeks/months ctffind4 will stabilise.

Alexis

Marcus Thu, 2015-02-19 13:29

Hi Alexis,

1) yes, I also recognized the flag warning, but since the program ran I assumed that this error output is just not correct. In the .7 version the program just stopped immediately in case the flag was not there.

2) Yes, I did set dAst to 0.0 (as it worked in all previous versions)
I quickly checked a non zero value (eg 100) and the refinement is running. However the ctf image still has these empty spaces. What values would you suggest to use in this case?
And what is most worrying for me is that in this case (and for dAst 0) for example the estimation of ctffind .7 version differs by 1 micron from the .10 estimation

Working on micrograph 1 of 1

SEARCHING CTF PARAMETERS...

100% [==============================] done!

DFMID1 DFMID2 ANGAST CC
19000.00 19000.00 -20.00 0.13643

REFINING CTF PARAMETERS...
DFMID1 DFMID2 ANGAST CC
19099.58 19069.55 -46.27 0.13940 Final Values

**warning(Image::ComputeRotationalAverageOfPowerSpectrum): unable to rescale average experimental power
Estimated defocus values : 19099.58 , 19069.55 Angstroms
Estimated azimuth of astigmatism: -46.27 degrees
Score : .13940
Thon rings with good fit up to : 6.7 Angstroms

3) And also this is correct. My box size is 128, and this worked in all previous versions. Using box size 256 actually gave me again similar ctf estimations than i got with the .7 version and the ctf image looks normal:

Number of columns, rows, sections: 3710 3710 1
MRC data mode: 2
Bit depth: 32
Pixel size: .000 .000 .000

Working on micrograph 1 of 1

SEARCHING CTF PARAMETERS...

100% [==============================] done!

DFMID1 DFMID2 ANGAST CC
28000.00 30250.00 40.00 0.23395

REFINING CTF PARAMETERS...
DFMID1 DFMID2 ANGAST CC
28073.30 30187.86 41.68 0.23408 Final Values

4) My input was following
mpirun -np 48 -machinefile ~/machinefile_1sthalf.dat `which relion_run_ctffind_mpi` --i "sel_mic.star" --o "sel_mic_ctf.star" --ctfWin -1 --CS 2
.26 --HT 300 --AmpCnst 0.1 --XMAG 39683 --DStep 5 --Box 128 --ResMin 30 --ResMax 7 --dFMin 10000 --dFMax 60000 --FStep 250 --dAst 0 --ctffind3_
exe "../ctffind4.10/ctffind --old-school-input"

5) if you are anyway still interested in this micrograph I can send it to you via mail

Cheers
Marcus

Alexis Thu, 2015-02-19 14:02

Hi Marcus,

1) --old-school-input: OK, there is bug here - this message should not appear. I will investigate

2) I forgot to document this properly, but the meaning of dAst has changed slightly since previous releases. The meaning of 0.0 for this value in ctffind3 was not well documented. It caused ctffind3 (and ctffind <= 4.0.8) to not impose any restraint on the level of astigmatism. This runs counter intuition I believe, since setting this to 0.0 might be expected to mean that we don't want to allow any/much astigmatism.
So I changed this. Now, if the user gives 0.0<=dAst<=10.0 (now called "expected/tolerated astigmatism" in ctffind4), there will be a very strong restraint on astigmatism. To switch off the restraint altogether, the user must set this value to something negative.
The default suggested by ctffind4 is 100.0A, which gives a soft restraint on astigmatism. I have already discussed this with Sjors and the next release of Relion will also use 100.0A as a default. I recommend you do this yourself in the meantime.

3) I never tested 128 box size, and it's clear that the central cross suppression is mis-calibrated in this case, causing what you are calling empty space. I will fix this for the next release. In the meantime, please use 256 box size - hope that's OK.

5) If things are working with the 256 box size, I don't need to test your micrograph, I can debug the bugs you describe without it.

Thanks again for your careful testing!
Alexis

Alexis Mon, 2015-02-23 17:34

Hi Marcus,

I've just put a new version (4.0.11) on the ctffind page. I believe it fixes the bugs you reported. Please try it and let me know if you find more problems.

Thanks
Alexis