preserve input star columns during particle export

[shahpnmlab]

This PR tries to solve the issue of preserving fields in the input star file when performing particle export. I was able to inject a dummy field rlnGoodBoi

data_

loop_
_rlnCoordinateX #1
_rlnCoordinateY #2
_rlnCoordinateZ #3
_rlnMicrographName #4
_rlnAngleRot #5
_rlnAngleTilt #6
_rlnAnglePsi #7
_rlnGoodBoi #8
675.768420  280.329043  186.950708  lam13_pos01_ts_006.tomostar 47.110208   85.968355   76.758127   c69eff2b6fe606fc
767.988150  167.457476  231.917305  lam13_pos01_ts_006.tomostar 57.129510   55.046367   -167.929250 83818753e4382e7e
268.366622  5.933382    179.789018  lam13_pos01_ts_006.tomostar 97.207547   51.317759   172.148981  91d0c1859c546315
605.553549  755.948645  135.947819  lam13_pos01_ts_006.tomostar 69.082745   70.528932   16.758236   8274a3e9f3b8c8aa
467.895984  610.609762  188.689848  lam13_pos01_ts_006.tomostar 101.250318  37.970189   37.852244   4285300587041ff1
365.430149  79.323344   55.399279   lam13_pos01_ts_006.tomostar 82.793697   73.354014   44.883255   d9dd3e0c8d269573
385.810648  145.840924  87.299531   lam13_pos01_ts_006.tomostar 93.516341   71.947021   -178.476110 8cc988bf5b2773ba
519.993252  780.430362  229.882874  lam13_pos01_ts_006.tomostar 117.246614  48.588616   -161.601060 68ddbbff2ab8f0ed
568.081757  772.639943  232.340113  lam13_pos01_ts_006.tomostar 70.121706   38.357151   -24.960290  ec17c39d9001acf3
256.924264  378.316166  57.720800   lam13_pos01_ts_006.tomostar 87.363785   50.549125   166.758366  545ee563a9ebc0cc

and preserve it in the output star file after export

data_

loop_
_rlnCoordinateX #1
_rlnCoordinateY #2
_rlnCoordinateZ #3
_rlnAngleRot #4
_rlnAngleTilt #5
_rlnAnglePsi #6
_rlnMicrographName #7
_rlnMagnification #8
_rlnDetectorPixelSize #9
_rlnCtfMaxResolution #10
_rlnImageName #11
_rlnCtfImage #12
_rlnPixelSize #13
_rlnVoltage #14
_rlnSphericalAberration #15
_rlnGoodBoi #16
  132.738   84.604  41.529    46.58264   51.69937  -115.19492  lam16_pos01_ts_003.tomostar  10000.0  20.00000  6.2  subtomo/lam16_pos01_ts_003/lam16_pos01_ts_003_0000000_20.00A.mrc  subtomo/lam16_pos01_ts_003/lam16_pos01_ts_003_0000000_ctf_20.00A.mrc  20.00000  300.000  2.700  13d54f207a9afb06
  180.880    9.291  37.853   133.06715   70.52849   101.13314  lam16_pos01_ts_003.tomostar  10000.0  20.00000  6.2  subtomo/lam16_pos01_ts_003/lam16_pos01_ts_003_0000001_20.00A.mrc  subtomo/lam16_pos01_ts_003/lam16_pos01_ts_003_0000001_ctf_20.00A.mrc  20.00000  300.000  2.700  4a9551f863b62de9

[alisterburt]

thanks @shahpnmlab ! @mgflast this might be a partial solution for #429 - ask @shahpnmlab how to use it to drag specific columns across

[shahpnmlab]

@mgflast not much going on here, just provide your input star w/ w/e colIDs and it should preserve them when the new star file is written

[shahpnmlab]

Note that if you are on a networked FS like ours (LUSTRE), then the connection can timeout

Unhandled exception. System.TimeoutException: Worker process did not connect within the allotted time.
at Warp.WorkerWrapper.ListenForPort(String pipeName, Int32 timeoutMilliseconds) in /apps/warp/WarpLib/WorkerWrapper.cs:line 168
   at Warp.WorkerWrapper..ctor(Int32 deviceID, Boolean silent, Boolean attachDebugger) in /apps/warp/WarpLib/WorkerWrapper.cs:line 100
   at WarpTools.Commands.DistributedOptions.GetWorkers(Boolean attachDebugger) in /apps/warp/WarpTools/Commands/DistributedOptions.cs:line 57
   at WarpTools.Commands.ExportParticlesTiltseries.Run(Object options) in /apps/warp/WarpTools/Commands/Tiltseries/ExportParticlesTiltseries.cs:line 236
   at WarpTools.WarpTools.Run(Object options) in /apps/warp/WarpTools/Program.cs:line 30
   at Warp.Tools.CommandLineParserHelper.ParseAndRun(String[] args, Func`2 run, Type[] verbs, String appName) in /apps/warp/WarpLib/Tools/CommandLineParserHelper.cs:line 27
   at WarpTools.WarpTools.Main(String[] args) in /apps/warp/WarpTools/Program.cs:line 17
   at WarpTools.WarpTools.<Main>(String[] args)
Aborted (core dumped)

in which case you need to re-execute the command.

[alisterburt]

makes sense and good to know, thanks for reporting - proper solution here will be to write full star to disk from master, send path to worker, load + subset on worker then read back on master as is done right now

[bionanopatterning]

Hey @shahpnmlab, thanks, this is very helpful! Seems like not a lot going on maybe but it saves me from encoding arbitrary data in the orientation parameters 😅