ctffind 4.1.13

Submitted by Smith_Lee on Thu, 2020-05-07 09:03

Forums

ctffind & ctftilt

 

I have installed ctffind 4.1.13 from ctffind-4.1.13.tar.gz based on inside “REDME” and the message from “Error compiling version 4.1.13 on Linux –using pdb.h & water.h files” in the Forum. However, when I run relion 3.1 for the CTF estimation it gave run.err as attached (which I have converted to the doc format with the content exactly copied and pasted from run.err).

In order for you to know whether the input corrected-micrographs.star was correctly got from the MotionCorrection step, here I also attached my corrected corrected-micrographs.star file (which I have converted to the doc format with the content exactly copied and pasted from corrected-micrographs.star) .
I am lookinf forward to getting your help to make the ctffind workable for my installation of relion 3.1.

Best regards.

Sincerely yours,

Smith

 

 

Alexis Fri, 2020-05-08 14:17

Hello Smith,

Sorry that you are encountering a segmentation fault when running ctffind. This is usually the result of a bug in the program.

The first thing I would ask is for you to run the latest version of ctffind, 4.1.14, which I just uploaded to the ctffind download page. This has a couple of a bug fixes relative to 4.1.13.

If possible, please just run a pre-compiled binary that you can download from that page. The ctffind-4.1.14.tar.gz archive contains the source, which you have to compile yourself, but the other .tar.gz files, for example ctffind-4.1.14-linux64.tar.gz contain pre-compiled binaries.

Also, for testing purposes, please do not run ctffind via the RELION interface, as this just complicates things. Instead, just execute the "ctffind" binary directly in your terminal and answer the questions at the terminal, providing the input image or movie, the pixel size, etc.

If, when you have done all this, you still get a segmentation fault, then please re-run the exact same thing but using the pre-compiled binary that includes debug information (the file you download has the suffix -dbg.tar.gz). This may printout a more informative error message.

Good luck, and thanks for trying ctffind!

Cheers,
Alexis

Smith_Lee Fri, 2020-05-08 21:51

 

Dear Alexis,

I have installed the ctffind-4.1.14 from ctffind-4.1.14-linux64.tar.gz, and the message given was as following (between “___-“):
_________
Input image file name [ln-mrc.mrc]                 : /root/MotionCorr/job019/relionrelated/tutorialdata/relion30_tutorial/Movies/20170629_00049_frameImage.mrc
Output diagnostic image file name [aaaaa.mrc]      : abc1.mrc
Pixel size [0.885]                                 : 0.885
Acceleration voltage [300]                         : 200
Spherical aberration [1.4]                         : 1.4
Amplitude contrast [0.1]                           : 0.1
Size of amplitude spectrum to compute [256]        : 512
Minimum resolution [30]                            : 30
Maximum resolution [5]                             : 5
Minimum defocus [5000]                             : 5000
Maximum defocus [50000]                            : 50000
Defocus search step [10]                           : 100
Do you know what astigmatism is present? [no]      : no
Slower, more exhaustive search? [yes]              : no
Use a restraint on astigmatism? [yes]              : no
Find additional phase shift? [no]                  : no
Do you want to set expert options? [yes]           : no
File name: 20170629_00049_frameImage.mrc
File type: MRC
Dimensions: X = 3710 Y = 3838 Z = 1
Number of slices: 1
Working on micrograph 1 of 1
OpenMP is not available - will not use parallel threads.

AverageRank is NaN for bin 2011

Failed Assert at ../src/programs/ctffind/ctffind.cpp:2825
bool ComputeRotationalAverageOfPowerSpectrum(Image*, CTF*, Image*, Image*, int, double*, double*, double*, double*, float*, float*)
Aborted (core dumped)
______

Then I installed ctffind-4.1.14 from ctffind-4.1.14-linux64.compat-dbg-tar.gz (I guess with this installation it will automatically replace the ctffind I installed from ctffind-4.1.14-linux64.tar.gz, am I right？)， I have processed 2 runnings with the debug installation. The first time message was as following:
_____
Input image file name
[/root/MotionCorr/job019/relionrelated/tutorialdata/relion30_tutorial/Movies/20170629_00049_frameImage.mrc] : /root/MotionCorr/job019/relionrelated/tutorialdata/relion30_tutorial/Movies/20170629_00049_frameImage.mrc
Output diagnostic image file name [fan7.mrc]       :abc2.mrc
Pixel size [0.885]                                 : 0.885
Acceleration voltage [200]                         : 200
Spherical aberration [1.4]                         : 1.4
Amplitude contrast [0.1]                           : 0.1
Size of amplitude spectrum to compute [512]        : 512
Minimum resolution [50]                            : 30
Maximum resolution [0]                             : 5
Minimum defocus [0]                                : 5000
Maximum defocus [500000]                           : 50000
Defocus search step [10]                           : 100
Do you know what astigmatism is present? [no]      : no
Slower, more exhaustive search? [no]               : no
Use a restraint on astigmatism? [no]               : no
Find additional phase shift? [no]                  : no
Do you want to set expert options? [no]            : no
File name: 20170629_00049_frameImage.mrc
File type: MRC
Dimensions: X = 3710 Y = 3838 Z = 1
Number of slices: 1
Working on micrograph 1 of 1
OpenMP is not available - will not use parallel threads.

Average is NaN for bin 904

Failed Assert at ../src/programs/ctffind/ctffind.cpp:2824
bool ComputeRotationalAverageOfPowerSpectrum(Image*, CTF*, Image*, Image*, int, double*, double*, double*, double*, float*, float*)
Aborted (core dumped)
_____

The second time message with the new debug was as following:
____
Input image file name
[/root/MotionCorr/job019/relionrelated/tutorialdata/relion30_tutorial/Movies/20170629_00049_frameImage.mrc] : /root/MotionCorr/job019/relionrelated/tutorialdata/relion30_tutorial/Movies/20170629_00049_frameImage.mrc
Output diagnostic image file name [fan6.mrc]       :abc3.mrc
Pixel size [0.885]                                 : 0.885
Acceleration voltage [200]                         : 200
Spherical aberration [1.4]                         : 1.4
Amplitude contrast [0.1]                           : 0.1
Size of amplitude spectrum to compute [512]        : 512
Minimum resolution [0]                             : 100

Error: Number outside of acceptable range!

Minimum resolution [0]                             : 60

Error: Number outside of acceptable range!

Minimum resolution [0]                             : 50
Maximum resolution [0]                             : 0
Minimum defocus [5000]                             : 0
Maximum defocus [50000]                            : 500000
Defocus search step [100]                          : 10
Do you know what astigmatism is present? [yes]     : no
Slower, more exhaustive search? [no]               : no
Use a restraint on astigmatism? [no]               : no
Find additional phase shift? [no]                  : no
Do you want to set expert options? [no]            : no
File name: 20170629_00049_frameImage.mrc
File type: MRC
Dimensions: X = 3710 Y = 3838 Z = 1
Number of slices: 1
Working on micrograph 1 of 1
OpenMP is not available - will not use parallel threads.

Segmentation fault (core dumped)
___
I m looking forward to getting your message on whether I have processed wrongly with ctffind, or there is still bug in the new version of ctffind.

Best regards.

Sincerely yours,
Smith

 

 

Alexis Sat, 2020-05-09 03:26

This looks like a bug. Could you please share the input file 20170629_00049_frameImage.mrc  with me so I can investigate further?

Smith_Lee Tue, 2020-05-12 23:52

Dear Alexis,

With the 20170629_00049_frameImage.mrc I attached, have you fixed the bug related to the cttfind 4.14, or the error message was caused by some other reason?

I am looking forward to getting a reply from you.

Smith

 

 

 

 

Alexis Wed, 2020-05-13 01:43

Hi Smith,

Thanks for uploading the image, and for the reminder.

Unfortunately, I am not able to reproduce the problem:

% tar -xvzf ctffind-4.1.14-linux64.tar.gz
bin/ctffind
bin/ctffind_plot_results.sh
% bin/ctffind

        **   Welcome to Ctffind   **

            Version : 4.1.14
           Compiled : May  8 2020
               Mode : Interactive

Input image file name
[20170629_00049_frameImage.mrc]                    :
Output diagnostic image file name
[diagnostic_output.mrc]                            :
Pixel size [0.885]                                 :
Acceleration voltage [200]                         :
Spherical aberration [1.4]                         :
Amplitude contrast [0.1]                           :
Size of amplitude spectrum to compute [512]        :
Minimum resolution [30.0]                          :
Maximum resolution [5.0]                           :
Minimum defocus [5000.0]                           :
Maximum defocus [50000.0]                          :
Defocus search step [100.0]                        :
Do you know what astigmatism is present? [no]      :
Slower, more exhaustive search? [no]               :
Use a restraint on astigmatism? [no]               :
Find additional phase shift? [no]                  :
Do you want to set expert options? [no]            :
File name: 20170629_00049_frameImage.mrc
File type: MRC
Dimensions: X = 3710 Y = 3838 Z = 1
Number of slices: 1
Working on micrograph 1 of 1

      DFMID1      DFMID2      ANGAST          CC
    13000.02    13000.02      -42.50     0.14053

Timings
 Initialization       : 00:00:00
 Spectrum computation : 00:00:01
 Parameter search     : 00:00:01
 Diagnosis            : 00:00:00
 Total                : 00:00:02

Estimated defocus values        : 13106.41 , 12787.57 Angstroms
Estimated azimuth of astigmatism: 73.29 degrees
Score                           : 0.17492
Pixel size for fitting          : 1.400 Angstroms
Thon rings with good fit up to  : 3.3 Angstroms
CTF aliasing apparent from      : 3.1 Angstroms

Summary of results                          : diagnostic_output.txt
Diagnostic images                           : diagnostic_output.mrc
Detailed results, including 1D fit profiles : diagnostic_output_avrot.txt
Use this command to plot 1D fit profiles    : ctffind_plot_results.sh diagnostic_output_avrot.txt

Can you confirm that you are definitely running ctffind 4.1.14 (it prints the version number to the terminal when you start the program)? Also, maybe could you confirm that the same binary works OK with other input files?

Cheers,
Alexis

 

Smith_Lee Wed, 2020-05-13 05:26

 

Dear Alexis,

Based on your advice, I have installed ctffind 4.1.14 from ctffind-4.1.14-linux64.tar.gz and ctffind-4.1.14-linux64-compat-dbg.tar.gz, both show Segment fault (core dumped), before input any mrc file.

Then I reinstalled ctffind 4.1.14 from ctffind-4.1.14.tar.gz, to gurantee it is ctffind 4.1.14 works. Based on your suggestion, I run 2 differenct mrc files.

The first mrc result is:
[root@localhost Movies]# /usr/local/bin/bin/ctffind

        **   Welcome to Ctffind   **

            Version : 4.1.14
           Compiled : May 13 2020
               Mode : Interactive

Input image file name [input.mrc]                  : 20170629_00049_frameImage.mrc
Output diagnostic image file name
[diagnostic_output.mrc]                            : abcoutput.mrc
Pixel size [1.0]                                   : 0.885
Acceleration voltage [300.0]                       : 200
Spherical aberration [2.70]                        : 1.4
Amplitude contrast [0.07]                          : 0.1
Size of amplitude spectrum to compute [512]        : 512
Minimum resolution [30.0]                          : 30
Maximum resolution [5.0]                           : 5
Minimum defocus [5000.0]                           : 5000
Maximum defocus [50000.0]                          : 50000
Defocus search step [100.0]                        : 100
Do you know what astigmatism is present? [no]      : no
Slower, more exhaustive search? [no]               : no
Use a restraint on astigmatism? [no]               : no
Find additional phase shift? [no]                  : no
Do you want to set expert options? [no]            : no
File name: 20170629_00049_frameImage.mrc
File type: MRC
Dimensions: X = 3710 Y = 3838 Z = 1
Number of slices: 1
Working on micrograph 1 of 1
OpenMP is not available - will not use parallel threads.

      DFMID1      DFMID2      ANGAST          CC
    13000.02    13000.02      -42.50     0.14053
Segmentation fault (core dumped)

The second mrc result is:
[root@localhost Movies]# /usr/local/bin/bin/ctffind

        **   Welcome to Ctffind   **

            Version : 4.1.14
           Compiled : May 13 2020
               Mode : Interactive

Input image file name
[20170629_00049_frameImage.mrc]                    : 20170629_00048_frameImage.mrc
Output diagnostic image file name [abcoutput.mrc]  : fanfanabc.mrc
Pixel size [0.885]                                 : 0.885
Acceleration voltage [200]                         : 200
Spherical aberration [1.4]                         : 1.4
Amplitude contrast [0.1]                           : 0.1
Size of amplitude spectrum to compute [512]        : 512
Minimum resolution [30]                            : 30
Maximum resolution [5]                             : 5
Minimum defocus [5000]                             : 5000
Maximum defocus [50000]                            : 50000
Defocus search step [100]                          : 100
Do you know what astigmatism is present? [no]      : no
Slower, more exhaustive search? [no]               : no
Use a restraint on astigmatism? [no]               : no
Find additional phase shift? [no]                  : no
Do you want to set expert options? [no]            : no
File name: 20170629_00048_frameImage.mrc
File type: MRC
Dimensions: X = 3710 Y = 3838 Z = 1
Number of slices: 1
Working on micrograph 1 of 1
OpenMP is not available - will not use parallel threads.

      DFMID1      DFMID2      ANGAST          CC
    12100.02    12100.02      -12.50     0.17350
Segmentation fault (core dumped)

After the failure of crffind 4.1.13 and 4.1.14, I then installed ctffind 4.1.10 from ctffind-4.1.10.tar.gz. Surprisingly, ctffind 4.1.10 works for me!

So in my computer it seems it lacks something the ctffind 4.1.13 and 4.1.14 need, which ctffind 4.1.10 has.

I am looking forward to getting your advice on how to make ctffind 4.14 workable in my computer.

If you need further message from me, please feel free to let me know.

Smith

 

 

Alexis Wed, 2020-05-13 12:14

Hello,

Sometimes, segfaults are due to the program exceeding some memory limit imposed by your operating system (example). There were quite a few changes between 4.1.10 and 4.1.14 which affect how much memory ctffind needs in order to run. My hypothesis is that the computer you are working on right now has a low limit.

Two ways to test this:

1. Either try to find another computer and try again

2. Or learn about the shell program "ulimit" and use it to raise any limits in place on your computer. Just running "ulimit" should tell you what limits are in place. Then you can run it with arguments to change these limits.

Hope this helps
Alexis

Grigory Sun, 2020-05-17 09:30

Hi Alexis,

I'm having exactly the same problem with ctffind 4.1.13 (I haven't tried 4.1.14 yet) on Arch Linux. I have no limits for ulimit. I have tried both binaries and compilation from sources, both fail with segfault. I compiled ctffind with debug on and ran it with gdb. I put a breakpoint at line 2820. The problem seems to be here:

2820         // Compute the rank version of the rotational average
2821         for (counter = 0; counter < number_of_bins; counter ++ ) { average_rank[counter] = average[counter]; }
2822         Renormalize1DSpectrumForFRC(number_of_bins,average_rank,average_fit,number_of_extrema_profile);
2823         for (counter = 0; counter < number_of_bins; counter ++ ) {
2824                 MyDebugAssertFalse(std::isnan(average[counter]),"Average is NaN for bin %i\n",counter);
2825                 MyDebugAssertFalse(std::isnan(average_rank[counter]),"AverageRank is NaN for bin %i\n",counter);
2826         }

The number_of_bins in my case is 363, but the counter on line 2823 continues outside of the loop and keeps going as can be seen below. This is how I get NaNs:

2825                    MyDebugAssertFalse(std::isnan(average_rank[counter]),"AverageRank is NaN for bin %i\n",counter);
(gdb) print average_rank[counter]
$57 = 1.4392132263355512e-320
(gdb) print counter
$58 = 363
(gdb) print average[counter]      
$59 = 2.1393042464925975e-321
(gdb) print number_of_bins
$60 = <optimized out>

Alexis Sun, 2020-05-17 13:25

Hi Grigory,

Thanks for taking time to investigate this - it does look like a bug. Could you try to reproduce with 4.1.14, and, assuming it reproduces, could you please send the input image and runtime parameters so that I can investigate further? I believe if you start a new forum topic you'll have the option to attach files to the initial post in the topic.

Thanks!
Alexis