With the new release 2.0.0 scratch.py executes the rsync command using SLURM's srun in order to provide better resource handling from the HTC cluster.
Although the module is perfectly functional using srun leads to a small problem: as the exit_code thrown by srun is automatically generated once the command is successfully executed, instead of once its finished, we are not correctly checking the exit status of the command. The workaround for this was to generate service_info.txt inside the original folder instead of the destination folder (inside /scratch/bi/). If by any reason rsync fails during its execution, although you will see this fail status in your terminal prompt, it will still report that the copy was successful.
It would be very helpful to find a solution for this, either waiting for the process to end (without using while + sleep) or some way to get the correct exit status.
With the new release 2.0.0 scratch.py executes the rsync command using SLURM's
srunin order to provide better resource handling from the HTC cluster.Although the module is perfectly functional using srun leads to a small problem: as the exit_code thrown by srun is automatically generated once the command is successfully executed, instead of once its finished, we are not correctly checking the exit status of the command. The workaround for this was to generate
service_info.txtinside the original folder instead of the destination folder (inside /scratch/bi/). If by any reason rsync fails during its execution, although you will see this fail status in your terminal prompt, it will still report that the copy was successful.It would be very helpful to find a solution for this, either waiting for the process to end (without using while + sleep) or some way to get the correct exit status.