Ctffind-4.0.13 with Relion 1.3 no .star output file

Submitted by mtherkel on Sun, 2015-04-19 13:51

Forums

ctffind & ctftilt

Hello,

I'm trying to run ctffind4 and relion through the command line. Although I generated clean .log files for each .mrc file there is no output file all_micrographs_ctf.star. I'm including the errors below.

Thanks,
Matt

My command:
mpirun -n 4 `which relion_run_ctffind_mpi` --i "all_micrographs.star" --o "all_micrographs_ctf.star" --ctfWin -1 --CS 2 --HT 300 --AmpCnst 0.1 --XMAG 79096 --DStep 14 --Box 512 --ResMin 100 --ResMax 7 --dFMin 5000 --dFMax 50000 --FStep 500 --dAst 0 --ctffind3_exe "/home/mtherkel/ctffind --omp-num-threads 1 --old-school-input"
echo "done"

The error file:

forrtl: No such file or directory
forrtl: No such file or directory
forrtl: No such file or directory
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: No such file or directory
forrtl: No such file or directory
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: No such file or directory
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: No such file or directory
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: No such file or directory
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: No such file or directory
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CA31F Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown

--------------------------------

this was the at the end of the output file:

Estimating CTF parameters using Niko Grigorieff's CTFFIND3 ...
2.88/2.88 hrs ............................................................~~(,_,">
ERROR: cannot find line with Final values in Micrographs/125_ctffind3.log
File: src/ctffind_runner.cpp line: 299
2.88/2.88 hrs ............................................................~~(,_,">
ERROR: cannot find line with Final values in Micrographs/125_ctffind3.log
File: src/ctffind_runner.cpp line: 299
2.89/2.89 hrs ............................................................~~(,_,">
ERROR: cannot find line with Final values in Micrographs/125_ctffind3.log
File: src/ctffind_runner.cpp line: 299
2.89/2.89 hrs ............................................................~~(,_,">
ERROR: cannot find line with Final values in Micrographs/125_ctffind3.log
File: src/ctffind_runner.cpp line: 299
done

Alexis Sun, 2015-04-19 20:43

Dear Matt,

Thanks for the error report. This sounds very similar to what several others have reported (see this thread). Recently another user narrowed the issue enough that I hope to have a fix for this in the next few days. I will let this be known on this forum, so keep an eye out.

Thanks
Alexis

mtherkel Sun, 2015-04-19 21:43

Sound good Alexis. I figured out that only 15 micrographs were affected, and opted for a quick fix - I removed them from the star file and reran relion_run_ctffind --only_make_star which worked fine.

Matt

Alexis Mon, 2015-04-20 15:04

I've just uploaded 4.0.14. Could you please give it a try and let me know whether the error has gone away?

Many thanks!
Alexis

mtherkel Wed, 2015-04-22 22:11

Hi Alexis,

Unfortunately I received the same error. Here is what I'm seeing (below). I'm using the 80S ribosome dataset available on EMPIAR (10002).

Matt

forrtl: No such file or directory
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CB18A Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown
forrtl: No such file or directory
forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind 000000000156F4FF Unknown Unknown Unknown
ctffind 00000000017CB18A Unknown Unknown Unknown
ctffind 0000000001746992 MAIN__ 372 ctffind.f90
ctffind 0000000000400566 Unknown Unknown Unknown
ctffind 00000000016C1DFB Unknown Unknown Unknown
ctffind 0000000000400429 Unknown Unknown Unknown

Output:

Estimating CTF parameters using Niko Grigorieff's CTFFIND3 ...
42.33/42.33 min ............................................................~~(,_,">
ERROR: cannot find line with Final values in Micrographs/100_ctffind3.log
File: src/ctffind_runner.cpp line: 299
done

Alexis Wed, 2015-04-22 22:46

Hi Matt,

Many thanks for trying 4.0.14 and sorry it didn't fix this problem. My guess was wrong as to exactly where this bug is.

Because I don't have an easy way to reproduce this problem here, I wonder whether you could re-run the job and reproduce the crash with the executable "ctffind_150422_dbg.exe" which is available for download at the ctffind page. This is pretty much the same as 4.0.14 except there are debug symbols included which should make the backtrace you have reported more informative. So if you could reproduce the error and copy the error message just as you did with 4.0.14, I would appreciate it very much.

Thanks
Alexis

mtherkel Mon, 2015-04-27 12:57

Hi Alexis,

Below is the error message from using dbg.exe.

Best
Matt

forrtl: No such file or directory
forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: severe (28): CLOSE error, unit 20, file "Unknown"
Image PC Routine Line Source
ctffind_150422_db 00000000017BAE4F Unknown Unknown Unknown
ctffind_150422_db 000000000043328A userinputs_MP_upd 934 user_input.f90
ctffind_150422_db 0000000000632CC9 MAIN__ 372 ctffind.f90
ctffind_150422_db 0000000000400566 Unknown Unknown Unknown
ctffind_150422_db 000000000191ECBB Unknown Unknown Unknown
ctffind_150422_db 0000000000400429 Unknown Unknown Unknown
forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

forrtl: warning (402): fort: (1): In call to IMAGES^ALLOCATE, an array temporary was created for argument #3

Alexis Mon, 2015-04-27 14:28

Hi Matt,

Thanks again for helping with debugging ctffind!

I've uploaded a new version, 4.0.15, which should solve this problem.

When you have time, I'd be grateful if you could try this new release and let me know whether it works better.

Thanks
Alexis

Alexis Mon, 2015-04-27 16:10

Oh that's great.

There were two problems:
1. during user interaction, a temporary file is created to record the answer to questions given by the user. This temporary file always had the same name, so that if several instances were running in the same directory, there were crashes when all the instances were trying to delete the same file (only the first instance successfully deleted). Version 4.0.14 fixed this by ensuring that random temp filenames were used for this file.
2. At the end of user interaction, the defaults filename (.CTFFind.dff) is overwritten with new values from the current run. This was coded with a deletion of the defaults file before writing it again. This was causing crashes in 4.0.14 for the same reason as detailled above.

In 4.0.15, there is no deletion of the defaults file, it's just overwritten and (more importantly for your usage), the updating of the defaults file is not done anyway if ctffind is running in batch mode.

Thanks again for your help in troubleshooting this.

Alexis