python 3.7.10/3.9 conda support

[jr3cermak]

The pinned packages for conda using python 3.7.10 do not seem to be available anymore. The entire software stack needs to be updated with packages that exist at various conda repositories.

Run conda env create -q -f conda/gridTools_export-linux-64.yml
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

CondaHTTPError: HTTP 404 NOT FOUND for url <https://conda.anaconda.org/pyviz/noarch/pyct-core-0.4.8-py_0.tar.bz2>
Elapsed: 00:00.040826
CF-RAY: 7021fc646aa85cf5-IAD

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

CondaHTTPError: HTTP 404 NOT FOUND for url <https://conda.anaconda.org/pyviz/noarch/geoviews-core-1.9.1-py_0.tar.bz2>
Elapsed: 00:00.033896
CF-RAY: 7021fcc57953826c-IAD

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

[jr3cermak]

Looking like the pyviz channel is causing problems at the moment.

[jr3cermak]

Workaround: eliminate pyviz from the channel: entry for conda configuration.

$ cat conda/gridTools_no_pyviz.yml

name: gridTools
channels:
  - conda-forge
  - defaults
dependencies:
  - bottleneck
  - cartopy
  - colorama
  - colorcet
  - conda
  - coverage
  - cython
  - dask
  - datashader
  - docutils=0.16
  - geoviews
  - hvplot
  - jupyterlab
  - matplotlib
  - m2r2
  - netCDF4
  - nodejs
  - numba
  - numpy
  - owslib
  - pandas
  - panel
  - param
  - pip
  - pooch
  - pyct
  - pykdtree
  - pyproj
  - pyqt
  - pyqt5-sip
  - pytest
  - python==3.9
  - requests
  - scipy
  - sphinx
  - sphinx_rtd_theme
  - sphinxcontrib-bibtex
  - xarray
  - xesmf
  - xgcm

change directory into the gridtools source tree in the activated environment:

$ pip install -e .

The pinned requirements within the gridtools repo still seem to work for at least example 6 using python 3.9:

(gridtest) cermak@mom6node0:~/src/gridtools/examples$ python mkGridsExample06.py 
Transformation PROJ (+proj=stere +lat_0=90.000000 +lat_ts=75.000000 +ellps=WGS84)
Computing center point in meters: (lat=90.000000, lon=0.000000) to (0.000000, 0.000000)
INFO: A new grid has been created.
Attempt to save the grid to a netCDF file.
Successfully wrote netCDF file to /import/AKWATERS/jrcermakiii/configs/zOutput/STE_miniIBCAO_bad_angledx_Example6.nc
<xarray.Dataset>
Dimensions:   (nyp: 41, nxp: 41, nx: 40, ny: 40)
Dimensions without coordinates: nyp, nxp, nx, ny
Data variables:
    x         (nyp, nxp) float64 -45.0 -43.53 -41.99 ... 138.0 136.5 135.0
    y         (nyp, nxp) float64 53.82 54.66 55.48 56.27 ... 55.48 54.66 53.82
    tile      <U5 'tile1'
    dx        (nyp, nx) float64 1.336e+05 1.342e+05 ... 1.342e+05 1.336e+05
    dy        (ny, nxp) float64 1.336e+05 1.342e+05 ... 1.342e+05 1.336e+05
    angle_dx  (nyp, nxp) float64 0.7687 0.7605 0.7335 0.7051 ... 0.0 0.0 0.0 0.0
    area      (ny, nx) float64 1.793e+10 1.809e+10 ... 1.809e+10 1.793e+10
Attributes: (12/13)
    grid_version:      0.2
    code_version:      GridTools: 0.3.2+6d5241f
    history:           2022-04-26 22:12:48: created grid with GridTools library
    projection:        Stereographic
    grid_centerX:      0.0
    grid_centerY:      90.0
    ...                ...
    grid_dxUnits:      meters
    grid_dy:           5805000.0
    grid_dyUnits:      meters
    grid_tilt:         0.0
    software_version:  Cython 0.29.28; bottleneck 1.3.4; cartopy 0.20.2; cert...
    proj:              +ellps=WGS84 +proj=stere +lat_0=90.0 +lon_0=0.0 +lat_t...
Transformation PROJ (+proj=stere +lat_0=90.000000 +lat_ts=75.000000 +ellps=WGS84)
Computing center point in meters: (lat=90.000000, lon=0.000000) to (0.000000, 0.000000)
INFO: A new grid has been created.
Attempt to save the grid to a netCDF file.
Successfully wrote netCDF file to /import/AKWATERS/jrcermakiii/configs/zOutput/STE_miniIBCAO_Example6.nc
(gridtest) cermak@mom6node0:~/src/gridtools/examples$ python -V
Python 3.9.0

[jr3cermak]

The install via conda for python 3.9 continues to be broken. Following the installation tutorial, you may see this as a result of $ conda env create -f conda/gridTools.yml.

Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.11.0
  latest version: 23.1.0

Please update conda by running

    $ conda update -n base -c defaults conda



Downloading and Extracting Packages
numpy-1.23.5         | 5.6 MB    | ######################################################################################### | 100% 
....
Preparing transaction: done
Verifying transaction: \ 
SafetyError: The package for xesmf located at /home/cermak/miniconda3/pkgs/xesmf-0.7.0-pyhd8ed1ab_0
appears to be corrupted. The path 'site-packages/xesmf/frontend.py'
has an incorrect size.
  reported size: 39829 bytes
  actual size: 40488 bytes

ClobberError: This transaction has incompatible packages due to a shared path.
  packages: pyviz/noarch::pyct-core-0.5.0-py_0, pyviz/noarch::pyct-0.5.0-py_0
  path: 'bin/pyct'



done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate gridTools
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Follow with the source install of gridTools:

$ python -m pip install .

Attempting to run the mkGridInteractive.ipynb jupyter notebook produces this error:

ImportError: cannot import name 'Markup' from 'jinja2' (/home/cermak/miniconda3/envs/gridTools/lib/python3.9/site-packages/jinja2/__init__.py)

Use this to fix the jinja2 package:

$ conda install jinja2 --force-reinstall

We will see if we can develop a better pathway for the installation of gridtools for python 3.9.