Skip to content

Users see FileNotFoundError tracebacks #110

@douglatornell

Description

@douglatornell

FileNotFoundError tracebacks are not caught by the code. Instead execution ends ungracefully and gives the user a traceback. There are at least 2 situations in which this happens:

  1. When reading the first and last dataset files to calculate the set of variables to drop; traceback:
Traceback (most recent call last):
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info
    file = self._cache[self._key]
           ~~~~~~~~~~~^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/lru_cache.py", line 56, in __getitem__
    value = self._cache[key]
            ~~~~~~~~~~~^^^^^
KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('/results2/SalishSea/month-avg.202111/SalishSeaCast_1m_biol_T_20231001_20231031.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), 'b54a73f0-b76e-44f2-bf7e-ba47f19e3334']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/doug/conda_envs/reshapr-dev/bin/reshapr", line 8, in <module>
    sys.exit(reshapr())
             ^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/doug/warehouse/MOAD/Reshapr/reshapr/cli/extract.py", line 61, in extract
    reshapr.core.extract.cli_extract(config_file, start_date, end_date)
  File "/media/doug/warehouse/MOAD/Reshapr/reshapr/core/extract.py", line 106, in cli_extract
    with open_dataset(ds_paths, chunk_size, config) as ds:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/doug/warehouse/MOAD/Reshapr/reshapr/core/extract.py", line 466, in open_dataset
    with xarray.open_dataset(ds_path, chunks=chunk_size) as ds:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/api.py", line 572, in open_dataset
    backend_ds = backend.open_dataset(
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 593, in open_dataset
    store = NetCDF4DataStore.open(
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 400, in open
    return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 347, in __init__
    self.format = self.ds.data_model
                  ^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 409, in ds
    return self._acquire()
           ^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 403, in _acquire
    with self._manager.acquire_context(needs_lock) as root:
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 199, in acquire_context
    file, cached = self._acquire_with_cache_info(needs_lock)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/doug/conda_envs/reshapr-dev/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info
    file = self._opener(*self._args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/netCDF4/_netCDF4.pyx", line 2469, in netCDF4._netCDF4.Dataset.__init__
  File "src/netCDF4/_netCDF4.pyx", line 2028, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: '/results2/SalishSea/month-avg.202111/SalishSeaCast_1m_biol_T_20231001_20231031.nc'
  1. When there is a missing file in the collection of dataset file paths that xarray.open_mfdataset() is operating on; traceback:
    Screenshot 2023-11-28 at 1 29 48 PM

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions