Estimate processing time in SEARCH mode

Submitted by rdrighetto on Wed, 2015-08-05 09:02

Forums

Frealign

Hi,

I have been running frealign_run_search (MODE 3) on a relatively large data set for six days on our cluster. While I am happy so far with the results according to the reprojections (FMATCH = T), I don't know how long it will take to complete the job (days? weeks? months?). Therefore, I have the following questions:

1) Is it possible to somehow estimate the total running time?
(e.g. accounting for the # of images, image size, target resolution, CPUs, etc)

2) What are the recommended procedures for tracking the intermediate results?

Thanks,

Ricardo Righetto

Alexis Wed, 2015-08-05 09:06

Hi Ricardo,

There should be a "scratch/" folder in your refinement directory. In there, you will see many log files - one per refinement node. You should look in these - Frealign prints out a message after the first, second, tenth, hundredth etc. particles. That should give you an accurate idea of how long the search is taking particle. You then need to multiply this by the number of particles in the dataset and divide by the number of compute nodes and that should give you the expected wall time for the complete search.

Alexis

Alexis Wed, 2015-08-05 09:53

OK great.

Here's another tip: resample your images to a bigger pixel size. For example, if you are early on in your project (not high-resolution yet), but using a pixel size of ~1A, you're probably spending too much time/money on CPUs. Instead, use the resample program distributed with Frealign, and resample your stack of images (and your 3D volume) to a pixel size of, say, ~3A. Things may then go (much) more quickly for the search.

Alexis

rdrighetto Thu, 2015-08-06 05:33

Hi Alexis,

Thanks once again. If I refine a resampled dataset like you say, and then want to refine the full-scale dataset, is there any tool to convert the FREALIGN .par files (particularly the x,y shifts)? Or how should I proceed in this case?