added options to export ctf parameters as csv and subtomogram without ctf correction

[alncat]

Dear WARP/M developer,
I have implemented two new options to control the format of subtomograms exported by WARP.
The first option is --output_ctf_csv, which will tell WARP to export ctf parameters for each subtomogram as a csv file. The name of csv file have the same prefix as the 3DCTF exported by WARP and is located at the same folder for the corresponding subtomogram.
The second option is --dont_correct_ctf_3d, which will tell WARP to export subtomogram without CTF correction. I added a condition if (options.CorrectCTF) to toggle on

ImagesFT[threadID].Multiply(CTFs[threadID]);

when the argument --dont_correct_ctf_3d is not enabled. In the backprojection step,

Projectors[threadID].BackProject(ImagesFT[threadID], CTFsAbs[threadID], !options.PrerotateParticles ? GetAngleInAllTilts(ParticlePositions) : GetParticleAngleInAllTilts(ParticlePositions, ParticleAngles), MagnificationCorrection);

the weight for each pixel in the images of tilt series is set to 1. The 3DCTF is reconstructed in a similar way.
Hence, when --dont_correct_ctf_3d is specified, WARP will export subtomogram reconstructed by backprojecting raw images in tilt series.
These options are useful for training OPUS-TOMO (https://github.com/alncat/opusTomo), which can often effectively identify target specie from initial particle picks in the initial stages of tomographic data processing. These modifications might be a useful addition to WARP's functions. Any feedbacks will be greatly appreciated!
Sincerely,
Zhenwei Luo

[alncat]

The averages of subtomograms reconstructed by relion_reconstruct using subtomograms exported by WARP with or without --dont_correct_ctf_3d are similar.
I have tested different formats of subtomograms for the ATP synthase dimer,
subtomogram average for subtomograms with --dont_correct_ctf_3d,

subtomogram average for subtomograms without --dont_correct_ctf_3d, that is, the default format as before

[alisterburt]

@alncat thanks for taking the time to propose a PR! I'm glad you've been able to use Warp outputs for your own work :-)

Doesn't the yellow map (i.e. without new functionality) look better?

Our goal when exporting particles has been compatibility with RELION. Non-CTF aware particle reconstruction introduces a way to misuse Warp for our users and I'm not sure I see the benefit right now...

I'm very open to exporting CTF metadata if this is useful for you but would prefer that export happen via its own method on TiltSeries rather than an extra argument to the GetCTFs method

[alisterburt]

@dtegunov would be good to understand your perspective on this kind of change, is this a thing we want/are okay with accepting?

[alncat]

@alisterburt @dtegunov Thank you very much for your feedbacks and considering my PR!
WARP is very intuitive to use, and my program can work with WARP in a seamless way, thus potentially speeding up the workflow.
The CTF corrected subtomogram is better at preserving information at intermediate resolution. But for initial particle filtering, since the CTF functions at different tilt angles have similar shape at low resolution, the low resolution information in raw subtomograms are well preserved.

I kept the CTF corrected subtomogram as the default format for exporting. So this PR won't break the original compatibility with other softwares.

I was aiming to export the per-particle CTF parameters, so i modified the GetCTFs method in which the CTFs are reconstructed. If you can give me a pointer to the method in TiltSeries, i am very happy to change it~

[alncat]

Hi @alisterburt, just checking in on the status of this PR. Please let me know if there's anything you need from my side. Thanks!

[alisterburt]

Hi @alncat - sorry but it's not clear to me how to proceed here, I have to leave the decision to Dimitry

[alncat]

@alisterburt Got it~

[alncat]

Hi @dtegunov , please let me know if there's anything you need from my side. Thank you for considering this PR!

[dtegunov]

Thanks for the PR, @alncat! I'll have to give the CTF correction part more thought, but GetCTFsForOneParticle etc. are not a good place to be writing out files. You can get the parameters using TiltSeries.GetCTFParamsForOneTilt from outside the TiltSeries class and then write them out.

@alisterburt I remember discussing plans for bringing back pre-Relion 5 style particle tilt series STAR files (i.e. a line for every particle tilt, including CTF parameters). Did we ever act on those plans? They would be equivalent to these CSVs and @alncat could just convert without modifying Warp.

[alncat]

@dtegunov Thank you very much for reviewing it! I will check this method TiltSeries.GetCTFParamsForOneTilt to see how i can implement similar functions using it. I also did some visual comparisons between slices in ctfs and subtomograms using different options for your reference,

[alisterburt]

@alisterburt I remember discussing plans for bringing back pre-Relion 5 style particle tilt series STAR files (i.e. a line for every particle tilt, including CTF parameters). Did we ever act on those plans? They would be equivalent to these CSVs and @alncat could just convert without modifying Warp.

nah never got done, would be nice for 3DPCA too so I guess it could be done soon...

[alncat]

@alisterburt I remember discussing plans for bringing back pre-Relion 5 style particle tilt series STAR files (i.e. a line for every particle tilt, including CTF parameters). Did we ever act on those plans? They would be equivalent to these CSVs and @alncat could just convert without modifying Warp.

nah never got done, would be nice for 3DPCA too so I guess it could be done soon...

that sounds great!