Print informative error message when a run fails because of a bad fit. #35
Description
This fails because of a bad fir, but the error message is not easy to understand for a normal user.
2026-03-13 14:59:35 CARACal.Stimela.Image-continuum-subtraction-0 INFO: job started at 2026-03-13 14:59:35.768733
# /.singularity.d/runscript: line 5: /etc/init.d/xvfb: No such file or directory
# Virtual frame buffer not installed. You may not be able to plot with this cab
# /venv/bin/imcontsub --order 1 --segments 200.0 --sigma-clip 5.0 --fit-model spline --cont-fit-tol 0.0 --overwrite --ra-chunks 4 --nworkers 4 --output-prefix /stimela_mount/output/fil_T29R03C01_HI.image /stimela_mount/input/cubes/cube_2/fil_T29R03C01_HI.image.fits
# WARNING: FITSFixedWarning: SPECSYS3= 'TOPOCENT'
# invalid alternate code, keyword resembles SPECSYSa but isn't. [astropy.wcs.wcs]
# WARNING: FITSFixedWarning: SPECSYS3= 'TOPOCENT'
# invalid alternate code, keyword resembles SPECSYSa but isn't. [astropy.wcs.wcs]
# WARNING: FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 58497.873737 from DATE-OBS'. [astropy.wcs.wcs]
# 2026-03-13 14:59:38: Input data dimensions: ('ra', 'dec', 'spectral')
# 2026-03-13 14:59:38: Input data shape: (900, 900, 10)
# 2026-03-13 14:59:38: Loading delayed compute for iteration 1/1 of continuum modelling.
# 2026-03-13 14:59:40: Creating binary mask as requested
# 2026-03-13 14:59:40: Creating binary mask as requested
# 2026-03-13 14:59:40: Creating binary mask as requested
# 2026-03-13 14:59:40: Creating binary mask as requested
# Traceback (most recent call last):
# File "/venv/bin/imcontsub", line 8, in <module>
# sys.exit(runit())
# ^^^^^^^
# File "/venv/lib/python3.12/site-packages/click/core.py", line 1442, in __call__
# /venv/lib/python3.12/site-packages/contsub/fitfuncs.py:35: RuntimeWarning: divide by zero encountered in scalar divide
# self.imax = int(nchan / (self.velwidth//self.dv))+1
# return self.main(*args, **kwargs)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/click/core.py", line 1363, in main
# rv = self.invoke(ctx)
# ^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/click/core.py", line 1226, in invoke
# return ctx.invoke(self.callback, **ctx.params)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/click/core.py", line 794, in invoke
# return callback(*args, **kwargs)
# ^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/contsub/parser/imcontsub.py", line 157, in runit
# out_ds_cont.writeto(outcont, overwrite=opts.overwrite)
# File "/venv/lib/python3.12/site-packages/astropy/io/fits/hdu/base.py", line 412, in writeto
# hdulist.writeto(name, output_verify, overwrite=overwrite, checksum=checksum)
# File "/venv/lib/python3.12/site-packages/astropy/io/fits/hdu/hdulist.py", line 1043, in writeto
# hdu._writeto(hdulist._file)
# File "/venv/lib/python3.12/site-packages/astropy/io/fits/hdu/base.py", line 722, in _writeto
# self._writeto_internal(fileobj, inplace, copy)
# File "/venv/lib/python3.12/site-packages/astropy/io/fits/hdu/base.py", line 728, in _writeto_internal
# data_offset, data_size = self._writedata(fileobj)
# ^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/astropy/io/fits/hdu/base.py", line 659, in _writedata
# size += self._writedata_internal(fileobj)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/astropy/io/fits/hdu/image.py", line 649, in _writedata_internal
# return self._writeinternal_dask(fileobj)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/astropy/io/fits/hdu/image.py", line 739, in _writeinternal_dask
# output.store(outarr, lock=True, compute=True)
# File "/venv/lib/python3.12/site-packages/dask/array/core.py", line 1777, in store
# r = store([self], [target], **kwargs)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/array/core.py", line 1245, in store
# compute_as_if_collection(Array, store_dsk, map_keys, **kwargs)
# File "/venv/lib/python3.12/site-packages/dask/base.py", line 397, in compute_as_if_collection
# return schedule(dsk2, keys, **kwargs)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/threaded.py", line 91, in get
# results = get_async(
# ^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/local.py", line 516, in get_async
# raise_exception(exc, tb)
# File "/venv/lib/python3.12/site-packages/dask/local.py", line 324, in reraise
# raise exc
# File "/venv/lib/python3.12/site-packages/dask/local.py", line 229, in execute_task
# result = task(data)
# ^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/_task_spec.py", line 745, in __call__
# return self.func(*new_argspec)
# ^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/_task_spec.py", line 171, in _execute_subgraph
# res = execute_graph(final, keys=[outkey])
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/_task_spec.py", line 984, in execute_graph
# cache[key] = node(cache)
# ^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/_task_spec.py", line 745, in __call__
# return self.func(*new_argspec)
# ^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/_task_spec.py", line 171, in _execute_subgraph
# res = execute_graph(final, keys=[outkey])
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/_task_spec.py", line 984, in execute_graph
# cache[key] = node(cache)
# ^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/dask/_task_spec.py", line 745, in __call__
# return self.func(*new_argspec)
# ^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/contsub/utils.py", line 36, in get_automask
# cont_model = contsub.fitContinuum(xspec, cube, mask=None)
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/venv/lib/python3.12/site-packages/contsub/image_plane.py", line 52, in fitContinuum
# fitfunc.prepare(xspec)
# File "/venv/lib/python3.12/site-packages/contsub/fitfuncs.py", line 35, in prepare
# self.imax = int(nchan / (self.velwidth//self.dv))+1
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# OverflowError: cannot convert float infinity to integer
# Traceback (most recent call last):
# File "/stimela_mount/code/run.py", line 47, in <module>
# subprocess.check_call(shlex.split(_runc))
# File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
# raise CalledProcessError(retcode, cmd)
# subprocess.CalledProcessError: Command '['/venv/bin/imcontsub', '--order', '1', '--segments', '200.0', '--sigma-clip', '5.0', '--fit-model', 'spline', '--cont-fit-tol', '0.0', '--overwrite', '--ra-chunks', '4', '--nworkers', '4', '--output-prefix', '/stimela_mount/output/fil_T29R03C01_HI.image', '/stimela_mount/input/cubes/cube_2/fil_T29R03C01_HI.image.fits']' returned non-zero exit status 1.
2026-03-13 14:59:40 CARACal.Stimela.Image-continuum-subtraction-0 ERROR: /usr/bin/singularity run --workdir /scratch/usd/pserra/Astro/caracal_devel/reduction/.stimela_workdir-17734102757280178 --contain returns error code 1
2026-03-13 14:59:40 CARACal.Stimela.Image-continuum-subtraction-0 ERROR: job failed at 2026-03-13 14:59:40.665099 after 0:00:04.896366
2026-03-13 14:59:40 CARACal ERROR: Job 'Image-continuum-subtraction-0:: Image continuum subtraction for cube ' failed: /usr/bin/singularity run --workdir /scratch/usd/pserra/Astro/caracal_devel/reduction/.stimela_workdir-17734102757280178 --contain returns error code 1 [PipelineException]
2026-03-13 14:59:40 CARACal INFO: More information can be found in the logfile at somedir1/somedir2/somedir3/output_short/logs-20260313-145754/log-caracal.txt
2026-03-13 14:59:40 CARACal INFO: exiting with error code 1