cdms fails to load data on a specific level #446
Description
Describe the bug
Leaving this note for a record for those who may experience the similar problem. When opening a 3-dimensional variable that has plev as one of coordinates (such as ua, va, ta), it used to be possible to load the variable from one specific level (e.g., ua on 850 hPa). However, it looks like this capability has been broken, I think.
To Reproduce
import cdms2
infile = "/p/user_pub/pmp/pmp_results/pmp_v1.1.2/additional_xmls/latest/v20220903/cmip6/historical/atmos/mon/ua/cmip6.historical.ACCESS-CM2.r1i1p1f1.mon.ua.xml"
f = cdms2.open(infile)
d = f('ua', level=85000)Expected behavior
Load variable at 850 hPa
Screenshots or traceback
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Input In [11], in <cell line: 1>()
----> 1 d = f('ua', level=85000)
File ~/.conda/envs/pmp_v2.4.0/lib/python3.10/site-packages/cdms2/cudsinterface.py:34, in cuDataset.__call__(self, id, *args, **kwargs)
32 except (AttributeError, TypeError):
33 raise CDMSError("No such variable, " + id)
---> 34 return v(*args, **kwargs)
File ~/.conda/envs/pmp_v2.4.0/lib/python3.10/site-packages/cdms2/avariable.py:194, in AbstractVariable.__call__(self, *args, **kwargs)
192 s = selectors.Selector(*args, **d)
193 # get the selection
--> 194 return s.unmodified_select(self, raw=raw,
195 squeeze=squeeze,
196 order=order,
197 grid=grid)
File ~/.conda/envs/pmp_v2.4.0/lib/python3.10/site-packages/cdms2/selectors.py:191, in Selector.unmodified_select(self, variable, raw, squeeze, order, grid)
189 if _debug:
190 print('About to call subRegion:', specifications)
--> 191 fetched = result.subRegion(*specifications)
192 axismap = list(range(len(axes)))
193 for c in components:
File ~/.conda/envs/pmp_v2.4.0/lib/python3.10/site-packages/cdms2/avariable.py:969, in AbstractVariable.subRegion(self, *specs, **keys)
962 slicelist = self.reg_specs2slices(speclist, force=circulardim)
964 d = {'raw': raw,
965 'squeeze': squeeze,
966 'order': order,
967 'grid': grid,
968 }
--> 969 return self.subSlice(*slicelist, **d)
971 #
972 # get the general wrap slice (indices that are neg -> pos and vica versa)
973 #
975 wrapslice = slicelist[wrapdim]
File ~/.conda/envs/pmp_v2.4.0/lib/python3.10/site-packages/cdms2/avariable.py:732, in AbstractVariable.subSlice(self, *specs, **keys)
730 singles = None
731 slicelist = self.specs2slices(speclist, force=1)
--> 732 d = self.expertSlice(slicelist)
733 squeeze = keys.get('squeeze', 0)
734 raw = keys.get('raw', 0)
File ~/.conda/envs/pmp_v2.4.0/lib/python3.10/site-packages/cdms2/variable.py:686, in DatasetVariable.expertSlice(self, initslist)
683 # If slices with negative strides were input, apply the appropriate
684 # reversals.
685 if haveReversals:
--> 686 result = result[revlist]
688 return result
File ~/.conda/envs/pmp_v2.4.0/lib/python3.10/site-packages/numpy/ma/core.py:3222, in MaskedArray.__getitem__(self, indx)
3212 """
3213 x.__getitem__(y) <==> x[y]
3214
3215 Return the item described by i, as a masked array.
3216
3217 """
3218 # We could directly use ndarray.__getitem__ on self.
3219 # But then we would have to modify __array_finalize__ to prevent the
3220 # mask of being reshaped if it hasn't been set up properly yet
3221 # So it's easier to stick to the current version
-> 3222 dout = self.data[indx]
3223 _mask = self._mask
3225 def _is_scalar(m):
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
** Workaround **
level = 85000
deltaP = 1
d = f('ua', level=(level+deltaP, level-deltaP))
d.shape(1980, 1, 145, 192)
Desktop (please complete the following information):
- OS: Redhat
- Version: cdms2, v3.1.5
(pmp_v2.4.0) [lee1043@gates test]$ conda list cdms2
# packages in environment at /home/lee1043/.conda/envs/pmp_v2.4.0:
#
# Name Version Build Channel
cdms2 3.1.5 py310h65abc81_14 conda-forge
Environment Information
`conda info`
active environment : pmp_v2.4.0
active env location : /home/lee1043/.conda/envs/pmp_v2.4.0
shell level : 2
user config file : /home/lee1043/.condarc
populated config files : /home/lee1043/.condarc
conda version : 4.14.0
conda-build version : 3.18.8
python version : 3.7.3.final.0
virtual packages : __linux=3.10.0=0
__glibc=2.17=0
__unix=0=0
__archspec=1=x86_64
base environment : /export/lee1043/anaconda3 (writable)
conda av data dir : /export/lee1043/anaconda3/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /home/lee1043/.conda/pkgs
envs directories : /home/lee1043/.conda/envs
/export/lee1043/anaconda3/envs
platform : linux-64
user-agent : conda/4.14.0 requests/2.28.1 CPython/3.7.3 Linux/3.10.0-1160.71.1.el7.x86_64 rhel/7.9 glibc/2.17
UID:GID : 55431:1026
netrc file : None
offline mode : False
`conda config --show-sources`
==> /home/lee1043/.condarc <==
envs_dirs:
- /home/lee1043/.conda/envs
pkgs_dirs:
- /home/lee1043/.conda/pkgs
`conda list --show-channel-urls`
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
anyio 3.6.1 pyhd8ed1ab_1 conda-forge
argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py310h5764c6d_2 conda-forge
astroid 2.11.7 py310h06a4308_0 defaults
asttokens 2.0.8 pyhd8ed1ab_0 conda-forge
attrs 22.1.0 pyh71513ae_1 conda-forge
babel 2.10.3 pyhd8ed1ab_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.11.1 pyha770c72_0 conda-forge
bleach 5.0.1 pyhd8ed1ab_0 conda-forge
bokeh 2.4.3 pyhd8ed1ab_3 conda-forge
brotli 1.0.9 h166bdaf_7 conda-forge
brotli-bin 1.0.9 h166bdaf_7 conda-forge
brotlipy 0.7.0 py310h5764c6d_1004 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.18.1 h7f98852_0 conda-forge
ca-certificates 2022.07.19 h06a4308_0 defaults
cartopy 0.20.3 py310he7eef42_2 conda-forge
cdat_info 8.2.1 pyhd8ed1ab_2 conda-forge
cdms2 3.1.5 py310h65abc81_14 conda-forge
cdp 1.7.0 pyhd8ed1ab_1 conda-forge
cdtime 3.1.4 py310h1a17f1e_7 conda-forge
cdutil 8.2.1 pyhd8ed1ab_1 conda-forge
certifi 2022.6.15 py310h06a4308_0 defaults
cffi 1.15.1 py310h255011f_0 conda-forge
charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge
click 8.1.3 py310hff52083_0 conda-forge
cloudpickle 2.1.0 pyhd8ed1ab_0 conda-forge
cryptography 37.0.4 py310h597c629_0 conda-forge
curl 7.83.1 h7bff187_0 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
cytoolz 0.12.0 py310h5764c6d_0 conda-forge
dask 2022.8.1 pyhd8ed1ab_2 conda-forge
dask-core 2022.8.1 pyhd8ed1ab_0 conda-forge
debugpy 1.6.3 py310hd8f1fbe_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
dill 0.3.4 pyhd3eb1b0_0 defaults
distarray 2.12.2 pyhd8ed1ab_2 conda-forge
distributed 2022.8.1 pyhd8ed1ab_2 conda-forge
enso_metrics 1.1.1 py310hff52083_1 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
eofs 1.4.0 py_0 conda-forge
esmf 8.2.0 nompi_h1ee4d8d_2 conda-forge
esmpy 8.2.0 nompi_py310ha0140b2_1 conda-forge
executing 0.10.0 pyhd8ed1ab_0 conda-forge
expat 2.4.8 h27087fc_0 conda-forge
flit-core 3.7.1 pyhd8ed1ab_0 conda-forge
fonttools 4.37.1 py310h5764c6d_0 conda-forge
freeglut 3.2.2 h9c3ff4c_1 conda-forge
freetype 2.12.1 hca18f0e_0 conda-forge
fsspec 2022.7.1 pyhd8ed1ab_0 conda-forge
future 0.18.2 py310hff52083_5 conda-forge
g2clib 1.6.3 heb9ad7a_1 conda-forge
genutil 8.2.1 py310h96516ba_2 conda-forge
geos 3.11.0 h27087fc_0 conda-forge
hdf4 4.2.15 h9772cbc_4 conda-forge
hdf5 1.12.2 nompi_h2386368_100 conda-forge
heapdict 1.0.1 py_0 conda-forge
icu 70.1 h27087fc_0 conda-forge
idna 3.3 pyhd8ed1ab_0 conda-forge
importlib-metadata 4.11.4 py310hff52083_0 conda-forge
importlib_metadata 4.11.4 hd8ed1ab_0 conda-forge
importlib_resources 5.9.0 pyhd8ed1ab_0 conda-forge
ipykernel 6.15.1 pyh210e3f2_0 conda-forge
ipython 8.4.0 py310hff52083_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
isort 5.9.3 pyhd3eb1b0_0 defaults
jasper 2.0.33 ha77e612_0 conda-forge
jedi 0.18.1 pyhd8ed1ab_2 conda-forge
jinja2 3.1.2 pyhd8ed1ab_1 conda-forge
jpeg 9e h166bdaf_2 conda-forge
json5 0.9.5 pyh9f0ad1d_0 conda-forge
jsonschema 4.14.0 pyhd8ed1ab_0 conda-forge
jupyter_client 7.3.4 pyhd8ed1ab_0 conda-forge
jupyter_core 4.11.1 py310hff52083_0 conda-forge
jupyter_server 1.18.1 pyhd8ed1ab_0 conda-forge
jupyterlab 3.4.5 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge
jupyterlab_server 2.15.1 pyhd8ed1ab_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.4 py310hbf28c38_0 conda-forge
krb5 1.19.3 h3790be6_0 conda-forge
lazy-object-proxy 1.7.1 py310h5764c6d_1 conda-forge
lcms2 2.12 hddcbb42_0 conda-forge
ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libblas 3.9.0 16_linux64_openblas conda-forge
libbrotlicommon 1.0.9 h166bdaf_7 conda-forge
libbrotlidec 1.0.9 h166bdaf_7 conda-forge
libbrotlienc 1.0.9 h166bdaf_7 conda-forge
libcblas 3.9.0 16_linux64_openblas conda-forge
libcdms 3.1.2 h2973bc7_118 conda-forge
libcf 1.0.3 py310hbc577d2_114 conda-forge
libcurl 7.83.1 h7bff187_0 conda-forge
libdeflate 1.13 h166bdaf_0 conda-forge
libdrs 3.1.2 hf593df3_118 conda-forge
libdrs_f 3.1.2 h7e76ec7_114 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 12.1.0 h8d9b700_16 conda-forge
libgfortran-ng 12.1.0 h69a702a_16 conda-forge
libgfortran5 12.1.0 hdcd56e2_16 conda-forge
libglu 9.0.0 he1b5a44_1001 conda-forge
libgomp 12.1.0 h8d9b700_16 conda-forge
libiconv 1.16 h516909a_0 conda-forge
liblapack 3.9.0 16_linux64_openblas conda-forge
libnetcdf 4.8.1 nompi_h21705cb_104 conda-forge
libnghttp2 1.47.0 hdcd2b5c_1 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libopenblas 0.3.21 pthreads_h78a6416_2 conda-forge
libpng 1.6.37 h753d276_4 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libsqlite 3.39.2 h753d276_1 conda-forge
libssh2 1.10.0 haa6b8db_3 conda-forge
libstdcxx-ng 12.1.0 ha89aaad_16 conda-forge
libtiff 4.4.0 h0e0dad5_3 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libwebp-base 1.2.4 h166bdaf_0 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libxml2 2.9.14 h22db469_4 conda-forge
libxslt 1.1.35 h8affb1d_0 conda-forge
libzip 1.9.2 hc869a4a_1 conda-forge
libzlib 1.2.12 h166bdaf_2 conda-forge
locket 1.0.0 pyhd8ed1ab_0 conda-forge
lxml 4.9.1 py310h5764c6d_0 conda-forge
lz4 4.0.0 py310h5d5e884_2 conda-forge
lz4-c 1.9.3 h9c3ff4c_1 conda-forge
markupsafe 2.1.1 py310h5764c6d_1 conda-forge
matplotlib-base 3.5.3 py310h8d5ebf3_2 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mccabe 0.7.0 pyhd3eb1b0_0 defaults
mistune 2.0.4 pyhd8ed1ab_0 conda-forge
msgpack-python 1.0.4 py310hbf28c38_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
nb_conda 2.2.1 unix_6 conda-forge
nb_conda_kernels 2.3.1 py310hff52083_1 conda-forge
nbclassic 0.4.3 pyhd8ed1ab_0 conda-forge
nbclient 0.6.7 pyhd8ed1ab_0 conda-forge
nbconvert 7.0.0 pyhd8ed1ab_0 conda-forge
nbconvert-core 7.0.0 pyhd8ed1ab_0 conda-forge
nbconvert-pandoc 7.0.0 pyhd8ed1ab_0 conda-forge
nbformat 5.4.0 pyhd8ed1ab_0 conda-forge
ncurses 6.3 h27087fc_1 conda-forge
nest-asyncio 1.5.5 pyhd8ed1ab_0 conda-forge
netcdf-fortran 4.6.0 nompi_hc402ea5_100 conda-forge
notebook 6.4.12 pyha770c72_0 conda-forge
notebook-shim 0.1.0 pyhd8ed1ab_0 conda-forge
numpy 1.23.2 py310h53a5b5f_0 conda-forge
openblas 0.3.21 pthreads_h320a7e8_2 conda-forge
openjpeg 2.5.0 h7d73246_1 conda-forge
openssl 1.1.1q h7f8727e_0 defaults
output_viewer 1.3.3 pyhd8ed1ab_1 conda-forge
packaging 21.3 pyhd8ed1ab_0 conda-forge
pandas 1.4.3 py310h769672d_0 conda-forge
pandoc 2.19.2 ha770c72_0 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.3.0 pyhd8ed1ab_0 conda-forge
patsy 0.5.2 pyhd8ed1ab_0 conda-forge
pcmdi_metrics 2.4.0 pyhd8ed1ab_0 conda-forge
pexpect 4.8.0 pyh9f0ad1d_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.2.0 py310hbd86126_2 conda-forge
pip 22.2.2 pyhd8ed1ab_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_0 conda-forge
platformdirs 2.4.0 pyhd3eb1b0_0 defaults
proj 9.0.1 h93bde94_1 conda-forge
prometheus_client 0.14.1 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.30 pyha770c72_0 conda-forge
psutil 5.9.1 py310h5764c6d_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.13.0 pyhd8ed1ab_0 conda-forge
pylint 2.14.5 py310h06a4308_0 defaults
pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pyproj 3.3.1 py310hf94497c_1 conda-forge
pyrsistent 0.18.1 py310h5764c6d_1 conda-forge
pyshp 2.3.1 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 py310hff52083_5 conda-forge
python 3.10.6 h582c2e5_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.16.1 pyhd8ed1ab_0 conda-forge
python_abi 3.10 2_cp310 conda-forge
pytz 2022.2.1 pyhd8ed1ab_0 conda-forge
pyyaml 6.0 py310h5764c6d_4 conda-forge
pyzmq 23.2.1 py310h330234f_0 conda-forge
readline 8.1.2 h0f457ee_0 conda-forge
requests 2.28.1 pyhd8ed1ab_0 conda-forge
scipy 1.9.0 py310hdfbd76f_0 conda-forge
seaborn 0.11.2 hd8ed1ab_0 conda-forge
seaborn-base 0.11.2 pyhd8ed1ab_0 conda-forge
send2trash 1.8.0 pyhd8ed1ab_0 conda-forge
setuptools 65.3.0 py310hff52083_0 conda-forge
shapely 1.8.4 py310h5e49deb_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sniffio 1.2.0 py310hff52083_3 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge
sqlite 3.39.2 h4ff8645_1 conda-forge
stack_data 0.4.0 pyhd8ed1ab_0 conda-forge
statsmodels 0.13.2 py310hde88566_0 conda-forge
tblib 1.7.0 pyhd8ed1ab_0 conda-forge
terminado 0.15.0 py310hff52083_0 conda-forge
tinycss2 1.1.1 pyhd8ed1ab_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
tomli 2.0.1 py310h06a4308_0 defaults
tomlkit 0.11.1 py310h06a4308_0 defaults
toolz 0.12.0 pyhd8ed1ab_0 conda-forge
tornado 6.1 py310h5764c6d_3 conda-forge
traitlets 5.3.0 pyhd8ed1ab_0 conda-forge
typing_extensions 4.3.0 pyha770c72_0 conda-forge
tzdata 2022c h191b570_0 conda-forge
udunits2 2.2.28 hc3e0081_0 conda-forge
unicodedata2 14.0.0 py310h5764c6d_1 conda-forge
urllib3 1.26.11 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 py_1 conda-forge
websocket-client 1.4.0 pyhd8ed1ab_0 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
wrapt 1.14.1 py310h5eee18b_0 defaults
xorg-fixesproto 5.0 h7f98852_1002 conda-forge
xorg-inputproto 2.3.2 h7f98852_1002 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libx11 1.7.2 h7f98852_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h7f98852_1 conda-forge
xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge
xorg-libxi 1.7.10 h7f98852_0 conda-forge
xorg-xextproto 7.3.0 h7f98852_1002 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
zeromq 4.3.4 h9c3ff4c_1 conda-forge
zict 2.2.0 pyhd8ed1ab_0 conda-forge
zipp 3.8.1 pyhd8ed1ab_0 conda-forge
zlib 1.2.12 h166bdaf_2 conda-forge
zstd 1.5.2 h6239696_4 conda-forge
Additional context
Add any other context about the problem here.