Installation issue

[josenimo]

Report

The installation is not working as expected, and I am not able to run tutorial imports correctly (especifically lightning).

I think that I am not able to get a stable env going, any help for a MacOS would be super appreciated :)

If I ignore that requirement, I can continue with the tutorial until:

model = cc.tl.TRVAE.load('./tutorial_models/codex_mouse_spleen_trvae', adata, map_location='cpu')

Here I get that scarches is not installed

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[26], line 1
----> 1 model = cc.tl.TRVAE.load('./tutorial_models/codex_mouse_spleen_trvae', adata, map_location='cpu')

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/cellcharter/tl/_trvae.py:36, in TRVAE.load(cls, dir_path, adata, map_location)
     34 @classmethod
     35 def load(cls, dir_path: str, adata: Optional[AnnData] = None, map_location: Optional[str] = None):  # noqa: D102
---> 36     raise ImportError("scarches is not installed. Please install scarches to use this method.")

ImportError: scarches is not installed. Please install scarches to use this method.

If i try import scarches this happens:

import scarches error

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[27], line 1
----> 1 import scarches

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scarches/__init__.py:1
----> 1 from . import dataset, metrics, trainers, models, zenodo, plotting, utils, classifiers
      3 __author__ = ', '.join([
      4     'Mohammad Lotfollahi',
      5     'Sergei Rybakov',
      6     'Marco Wagenstetter',
      7     'Mohsen Naghipourfar',
      8 ])
     10 __email__ = ', '.join([
     11     'mohammad.lotfollahi@helmholtz-muenchen.de',
     12     'sergei.rybakov@helmholtz-muenchen.de'
     13 ])

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scarches/models/__init__.py:1
----> 1 from .trvae.trvae import trVAE
      2 from .trvae.trvae_model import TRVAE
      3 from .trvae.adaptors import Adaptor, attach_adaptors

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scarches/models/trvae/trvae.py:9
      6 import torch.nn.functional as F
      8 from .modules import Encoder, Decoder
----> 9 from .losses import mse, mmd, zinb, nb
     10 from ._utils import one_hot_encoder
     11 from ..base._base import CVAELatentsModelMixin

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scarches/models/trvae/losses.py:1
----> 1 from scvi.distributions import NegativeBinomial
      2 import torch
      3 from torch.autograd import Variable

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scvi/__init__.py:10
      7 from ._settings import settings
      9 # this import needs to come after prior imports to prevent circular import
---> 10 from . import autotune, data, model, external, utils
     12 from importlib.metadata import version
     14 package_name = "scvi-tools"

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scvi/autotune/__init__.py:1
----> 1 from ._manager import TuneAnalysis, TunerManager
      2 from ._tuner import ModelTuner
      3 from ._types import Tunable, TunableMixin

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scvi/autotune/_manager.py:22
     20 from scvi._types import AnnOrMuData
     21 from scvi.data._constants import _SETUP_ARGS_KEY, _SETUP_METHOD_NAME
---> 22 from scvi.model.base import BaseModelClass
     23 from scvi.utils import InvalidParameterError
     25 from ._defaults import COLORS, COLUMN_KWARGS, DEFAULTS, TUNABLE_TYPES

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scvi/model/__init__.py:2
      1 from . import utils
----> 2 from ._amortizedlda import AmortizedLDA
      3 from ._autozi import AUTOZI
      4 from ._condscvi import CondSCVI

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/scvi/model/_amortizedlda.py:7
      5 import numpy as np
      6 import pandas as pd
----> 7 import pyro
      8 import torch
      9 from anndata import AnnData

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/pyro/__init__.py:4
      1 # Copyright (c) 2017-2019 Uber Technologies, Inc.
      2 # SPDX-License-Identifier: Apache-2.0
----> 4 import pyro.poutine as poutine
      5 from pyro.infer.inspect import render_model
      6 from pyro.logger import log

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/pyro/poutine/__init__.py:4
      1 # Copyright (c) 2017-2019 Uber Technologies, Inc.
      2 # SPDX-License-Identifier: Apache-2.0
----> 4 from .handlers import (
      5     block,
      6     broadcast,
      7     collapse,
      8     condition,
      9     do,
     10     enum,
     11     escape,
     12     infer_config,
     13     lift,
     14     markov,
     15     mask,
     16     queue,
     17     reparam,
     18     replay,
     19     scale,
     20     seed,
     21     substitute,
     22     trace,
     23     uncondition,
     24 )
     25 from .messenger import unwrap
     26 from .runtime import NonlocalExit, get_mask

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/pyro/poutine/handlers.py:71
     68 from typing_extensions import ParamSpec
     70 from pyro.poutine import util
---> 71 from pyro.poutine.block_messenger import BlockMessenger
     72 from pyro.poutine.broadcast_messenger import BroadcastMessenger
     73 from pyro.poutine.collapse_messenger import CollapseMessenger

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/pyro/poutine/block_messenger.py:7
      4 from functools import partial
      5 from typing import TYPE_CHECKING, Callable, List, Optional
----> 7 from pyro.poutine.messenger import Messenger
      9 if TYPE_CHECKING:
     10     from pyro.poutine.runtime import Message

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/pyro/poutine/messenger.py:19
      7 from typing import (
      8     Any,
      9     Callable,
   (...)
     14     TypeVar,
     15 )
     17 from typing_extensions import ParamSpec, Self
---> 19 from pyro.poutine.runtime import _PYRO_STACK, Message
     21 _P = ParamSpec("_P")
     22 _T = TypeVar("_T")

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/pyro/poutine/runtime.py:22
     19 import torch
     20 from typing_extensions import Literal, ParamSpec, TypedDict
---> 22 from pyro.params.param_store import (  # noqa: F401
     23     _MODULE_NAMESPACE_DIVIDER,
     24     ParamStoreDict,
     25 )
     27 _P = ParamSpec("_P")
     28 _T = TypeVar("_T")

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/pyro/params/__init__.py:4
      1 # Copyright (c) 2017-2019 Uber Technologies, Inc.
      2 # SPDX-License-Identifier: Apache-2.0
----> 4 from .param_store import (
      5     module_from_param_with_module_name,
      6     param_with_module_name,
      7     user_param_name,
      8 )
     10 __all__ = [
     11     "module_from_param_with_module_name",
     12     "param_with_module_name",
     13     "user_param_name",
     14 ]

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/pyro/params/param_store.py:21
     19 import torch
     20 from torch.distributions import constraints, transform_to
---> 21 from torch.serialization import MAP_LOCATION
     22 from typing_extensions import TypedDict
     25 class StateDict(TypedDict):

ImportError: cannot import name 'MAP_LOCATION' from 'torch.serialization' (/opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3/lib/python3.8/site-packages/torch/serialization.py)

Here is my very long traceback of installing the environment:
https://gist.github.com/josenimo/b694b7edf7822ae0bbd9eac768ec294c

Version information

# packages in environment at /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v3:
#
# Name                    Version                   Build  Channel
absl-py                   2.1.0                    pypi_0    pypi
aiohappyeyeballs          2.4.3                    pypi_0    pypi
aiohttp                   3.10.11                  pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
anndata                   0.9.2                    pypi_0    pypi
anyio                     4.5.2                    pypi_0    pypi
appnope                   0.1.4                    pypi_0    pypi
argon2-cffi               23.1.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.3.0                    pypi_0    pypi
asciitree                 0.3.3                    pypi_0    pypi
asttokens                 2.4.1                    pypi_0    pypi
async-lru                 2.0.4                    pypi_0    pypi
async-timeout             5.0.1                    pypi_0    pypi
attrs                     24.2.0                   pypi_0    pypi
babel                     2.16.0                   pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
beautifulsoup4            4.12.3                   pypi_0    pypi
bleach                    6.1.0                    pypi_0    pypi
bzip2                     1.0.8                h99b78c6_7    conda-forge
ca-certificates           2024.8.30            hf0a4a13_0    conda-forge
cached-property           2.0.1                    pypi_0    pypi
cellcharter               0.1.2                    pypi_0    pypi
certifi                   2024.8.30                pypi_0    pypi
cffi                      1.17.1                   pypi_0    pypi
charset-normalizer        3.4.0                    pypi_0    pypi
chex                      0.1.7                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
cloudpickle               3.1.0                    pypi_0    pypi
comm                      0.2.2                    pypi_0    pypi
contextlib2               21.6.0                   pypi_0    pypi
contourpy                 1.1.1                    pypi_0    pypi
cycler                    0.12.1                   pypi_0    pypi
dask                      2023.5.0                 pypi_0    pypi
dask-image                2023.3.0                 pypi_0    pypi
debugpy                   1.8.8                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
dm-tree                   0.1.8                    pypi_0    pypi
docrep                    0.3.2                    pypi_0    pypi
et-xmlfile                2.0.0                    pypi_0    pypi
etils                     1.3.0                    pypi_0    pypi
exceptiongroup            1.2.2                    pypi_0    pypi
executing                 2.1.0                    pypi_0    pypi
fasteners                 0.19                     pypi_0    pypi
fastjsonschema            2.20.0                   pypi_0    pypi
filelock                  3.16.1                   pypi_0    pypi
flax                      0.7.2                    pypi_0    pypi
fonttools                 4.55.0                   pypi_0    pypi
fqdn                      1.5.1                    pypi_0    pypi
frozenlist                1.5.0                    pypi_0    pypi
fsspec                    2024.10.0                pypi_0    pypi
gdown                     5.2.0                    pypi_0    pypi
get-annotations           0.1.2                    pypi_0    pypi
h11                       0.14.0                   pypi_0    pypi
h5py                      3.11.0                   pypi_0    pypi
httpcore                  1.0.7                    pypi_0    pypi
httpx                     0.27.2                   pypi_0    pypi
idna                      3.10                     pypi_0    pypi
igraph                    0.11.8                   pypi_0    pypi
imageio                   2.35.1                   pypi_0    pypi
importlib-metadata        8.5.0                    pypi_0    pypi
importlib-resources       6.4.5                    pypi_0    pypi
inflect                   7.4.0                    pypi_0    pypi
ipykernel                 6.29.5                   pypi_0    pypi
ipython                   8.12.3                   pypi_0    pypi
ipywidgets                8.1.5                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
jax                       0.4.13                   pypi_0    pypi
jaxlib                    0.4.13                   pypi_0    pypi
jedi                      0.19.2                   pypi_0    pypi
jinja2                    3.1.4                    pypi_0    pypi
joblib                    1.4.2                    pypi_0    pypi
json5                     0.9.28                   pypi_0    pypi
jsonpointer               3.0.0                    pypi_0    pypi
jsonschema                4.23.0                   pypi_0    pypi
jsonschema-specifications 2023.12.1                pypi_0    pypi
jupyter                   1.1.1                    pypi_0    pypi
jupyter-client            8.6.3                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-core              5.7.2                    pypi_0    pypi
jupyter-events            0.10.0                   pypi_0    pypi
jupyter-lsp               2.2.5                    pypi_0    pypi
jupyter-server            2.14.2                   pypi_0    pypi
jupyter-server-terminals  0.5.3                    pypi_0    pypi
jupyterlab                4.2.6                    pypi_0    pypi
jupyterlab-pygments       0.3.0                    pypi_0    pypi
jupyterlab-server         2.27.3                   pypi_0    pypi
jupyterlab-widgets        3.0.13                   pypi_0    pypi
kiwisolver                1.4.7                    pypi_0    pypi
lazy-loader               0.4                      pypi_0    pypi
leidenalg                 0.10.2                   pypi_0    pypi
libffi                    3.4.2                h3422bc3_5    conda-forge
libsqlite                 3.47.0               hbaaea75_1    conda-forge
libzlib                   1.3.1                h8359307_2    conda-forge
lightkit                  0.5.0                    pypi_0    pypi
lightning-utilities       0.11.9                   pypi_0    pypi
llvmlite                  0.38.1                   pypi_0    pypi
locket                    1.0.0                    pypi_0    pypi
markdown-it-py            3.0.0                    pypi_0    pypi
markupsafe                2.1.5                    pypi_0    pypi
matplotlib                3.7.5                    pypi_0    pypi
matplotlib-inline         0.1.7                    pypi_0    pypi
matplotlib-scalebar       0.8.1                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mistune                   3.0.2                    pypi_0    pypi
ml-collections            0.1.1                    pypi_0    pypi
ml-dtypes                 0.2.0                    pypi_0    pypi
more-itertools            10.5.0                   pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
msgpack                   1.1.0                    pypi_0    pypi
mudata                    0.2.3                    pypi_0    pypi
multidict                 6.1.0                    pypi_0    pypi
multipledispatch          1.0.0                    pypi_0    pypi
muon                      0.1.6                    pypi_0    pypi
natsort                   8.4.0                    pypi_0    pypi
nbclient                  0.10.0                   pypi_0    pypi
nbconvert                 7.16.4                   pypi_0    pypi
nbformat                  5.10.4                   pypi_0    pypi
ncurses                   6.5                  h7bae524_1    conda-forge
nest-asyncio              1.6.0                    pypi_0    pypi
networkx                  3.1                      pypi_0    pypi
newick                    1.0.0                    pypi_0    pypi
notebook                  7.2.2                    pypi_0    pypi
notebook-shim             0.2.4                    pypi_0    pypi
numba                     0.55.2                   pypi_0    pypi
numcodecs                 0.12.1                   pypi_0    pypi
numpy                     1.22.4                   pypi_0    pypi
numpyro                   0.12.1                   pypi_0    pypi
omnipath                  1.0.8                    pypi_0    pypi
openpyxl                  3.1.5                    pypi_0    pypi
openssl                   3.4.0                h39f12f2_0    conda-forge
opt-einsum                3.4.0                    pypi_0    pypi
optax                     0.1.8                    pypi_0    pypi
orbax-checkpoint          0.2.3                    pypi_0    pypi
overrides                 7.7.0                    pypi_0    pypi
packaging                 24.2                     pypi_0    pypi
pandas                    1.5.3                    pypi_0    pypi
pandocfilters             1.5.1                    pypi_0    pypi
parso                     0.8.4                    pypi_0    pypi
partd                     1.4.1                    pypi_0    pypi
patsy                     1.0.1                    pypi_0    pypi
pexpect                   4.9.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    10.4.0                   pypi_0    pypi
pims                      0.7                      pypi_0    pypi
pip                       24.3.1             pyh8b19718_0    conda-forge
pkgutil-resolve-name      1.3.10                   pypi_0    pypi
platformdirs              4.3.6                    pypi_0    pypi
prometheus-client         0.21.0                   pypi_0    pypi
prompt-toolkit            3.0.48                   pypi_0    pypi
propcache                 0.2.0                    pypi_0    pypi
protobuf                  5.28.3                   pypi_0    pypi
psutil                    6.1.0                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.3                    pypi_0    pypi
pycave                    3.2.1                    pypi_0    pypi
pycparser                 2.22                     pypi_0    pypi
pygments                  2.18.0                   pypi_0    pypi
pynndescent               0.5.13                   pypi_0    pypi
pyparsing                 3.1.4                    pypi_0    pypi
pyro-api                  0.1.2                    pypi_0    pypi
pyro-ppl                  1.9.1                    pypi_0    pypi
pysocks                   1.7.1                    pypi_0    pypi
python                    3.8.20          h7d35d02_2_cpython    conda-forge
python-dateutil           2.9.0.post0              pypi_0    pypi
python-json-logger        2.0.7                    pypi_0    pypi
pytorch-lightning         1.9.5                    pypi_0    pypi
pytz                      2024.2                   pypi_0    pypi
pywavelets                1.4.1                    pypi_0    pypi
pyyaml                    6.0.2                    pypi_0    pypi
pyzmq                     26.2.0                   pypi_0    pypi
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.35.1                   pypi_0    pypi
requests                  2.32.3                   pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rich                      13.9.4                   pypi_0    pypi
rpds-py                   0.20.1                   pypi_0    pypi
scanpy                    1.9.8                    pypi_0    pypi
scarches                  0.6.1                    pypi_0    pypi
schpl                     1.0.5                    pypi_0    pypi
scikit-image              0.21.0                   pypi_0    pypi
scikit-learn              1.3.2                    pypi_0    pypi
scipy                     1.10.1                   pypi_0    pypi
scvi-tools                0.20.3                   pypi_0    pypi
seaborn                   0.13.2                   pypi_0    pypi
send2trash                1.8.3                    pypi_0    pypi
session-info              1.0.0                    pypi_0    pypi
setuptools                75.3.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0                   pypi_0    pypi
slicerator                1.1.0                    pypi_0    pypi
sniffio                   1.3.1                    pypi_0    pypi
soupsieve                 2.6                      pypi_0    pypi
squidpy                   1.2.3                    pypi_0    pypi
stack-data                0.6.3                    pypi_0    pypi
statsmodels               0.14.1                   pypi_0    pypi
stdlib-list               0.10.0                   pypi_0    pypi
sympy                     1.13.3                   pypi_0    pypi
tensorstore               0.1.45                   pypi_0    pypi
terminado                 0.18.1                   pypi_0    pypi
texttable                 1.7.0                    pypi_0    pypi
threadpoolctl             3.5.0                    pypi_0    pypi
tifffile                  2023.7.10                pypi_0    pypi
tinycss2                  1.4.0                    pypi_0    pypi
tk                        8.6.13               h5083fa2_1    conda-forge
tomli                     2.1.0                    pypi_0    pypi
toolz                     1.0.0                    pypi_0    pypi
torch                     1.12.1                   pypi_0    pypi
torchmetrics              0.11.4                   pypi_0    pypi
tornado                   6.4.1                    pypi_0    pypi
tqdm                      4.67.0                   pypi_0    pypi
traitlets                 5.14.3                   pypi_0    pypi
typeguard                 4.4.0                    pypi_0    pypi
types-python-dateutil     2.9.0.20241003           pypi_0    pypi
typing_extensions         4.12.2             pyha770c72_0    conda-forge
umap-learn                0.5.7                    pypi_0    pypi
uri-template              1.3.0                    pypi_0    pypi
urllib3                   2.2.3                    pypi_0    pypi
validators                0.34.0                   pypi_0    pypi
wcwidth                   0.2.13                   pypi_0    pypi
webcolors                 24.8.0                   pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.8.0                    pypi_0    pypi
wheel                     0.45.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.13                   pypi_0    pypi
wrapt                     1.16.0                   pypi_0    pypi
xarray                    2023.1.0                 pypi_0    pypi
xz                        5.2.6                h57fd34a_0    conda-forge
yarl                      1.15.2                   pypi_0    pypi
zarr                      2.16.1                   pypi_0    pypi
zipp                      3.20.2                   pypi_0    pypi

[marcovarrone]

Hi @josenimo, can you try using a more recent version of python and pytorch?
I think there may be some conflicts in the dependencies (scarches, scvi-tools or something else).
In theory, there is nothing that makes CellCharter incompatible with those versions but I think for some reason there is a misalignment between one of those dependencies and pytorch.

So, if you don't have particular reasons for which you have to install python v3.8 and pytorch v1.12, I would suggest you to install at least python v3.10 and the latest version of pytorch.
When I will have a moment I will understand what is happening and in case I will make the requirements more restrictive in the documentation.

Let me know!

[josenimo]

Hey @marcovarrone,
I performed the following:

mamba create -n cellcharter_v4 python=3.10 pytorch
pip install scarches
pip install cellcharter

At the pip install cellcharter I got the following pip ERROR:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
schpl 1.0.5 requires pandas<2,>=1.1.2, but you have pandas 2.2.3 which is incompatible.

So there is still a bit of an issue.

For the sake of trying I tried to run the tutorial commands, the lightning import worked, however it failed again at the model import (see traceback below)

model import traceback

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[8], line 2
      1 # model = cc.tl.TRVAE.load('./tutorial_models/codex_mouse_spleen_trvae', adata, map_location='cpu')
----> 2 model = cc.tl.TRVAE.load('./tutorial_models/codex_mouse_spleen_trvae', adata, map_location='cpu')

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v4/lib/python3.10/site-packages/cellcharter/tl/_trvae.py:172, in TRVAE.load(cls, dir_path, adata, map_location)
    169 elif not os.path.exists(adata_path) and load_adata:
    170     raise ValueError("Save path contains no saved anndata and no adata was passed.")
--> 172 attr_dict, model_state_dict, var_names = cls._load_params(dir_path, map_location=map_location)
    174 # Overwrite adata with new genes
    175 adata = _validate_var_names(adata, var_names)

File /opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v4/lib/python3.10/site-packages/scarches/models/base/_base.py:131, in BaseMixin._load_params(cls, dir_path, map_location)
    128 varnames_path = os.path.join(dir_path, "var_names.csv")
    130 try:
--> 131     with open(setup_dict_path, "rb") as handle:
    132         attr_dict = pickle.load(handle)
    133 # This catches the following error:
    134 # RuntimeError: Attempting to deserialize object on a CUDA device
    135 # but torch.cuda.is_available() is False.
    136 # If you are running on a CPU-only machine, please use torch.load with
    137 # map_location=torch.device('cpu') to map your storages to the CPU.

FileNotFoundError: [Errno 2] No such file or directory: './tutorial_models/codex_mouse_spleen_trvae/attr.pkl'

happy to provide more feedback if needed. I guess I need to find the right combination of packages..

For the sake of completeness here is the list of all packages:

Details

# packages in environment at [/opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v4](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/mambaforge/base/envs/cellcharter_v4):
#
# Name                    Version                   Build  Channel
absl-py                   2.1.0                    pypi_0    pypi
affine                    2.4.0                    pypi_0    pypi
aiobotocore               2.5.4                    pypi_0    pypi
aiohappyeyeballs          2.4.3                    pypi_0    pypi
aiohttp                   3.11.7                   pypi_0    pypi
aioitertools              0.12.0                   pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
anndata                   0.10.9                   pypi_0    pypi
anyio                     4.6.2.post1              pypi_0    pypi
appnope                   0.1.4                    pypi_0    pypi
argon2-cffi               23.1.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
array-api-compat          1.9.1                    pypi_0    pypi
arrow                     1.3.0                    pypi_0    pypi
asciitree                 0.3.3                    pypi_0    pypi
asttokens                 2.4.1                    pypi_0    pypi
async-lru                 2.0.4                    pypi_0    pypi
async-timeout             5.0.1                    pypi_0    pypi
attrs                     24.2.0                   pypi_0    pypi
babel                     2.16.0                   pypi_0    pypi
beautifulsoup4            4.12.3                   pypi_0    pypi
bleach                    6.2.0                    pypi_0    pypi
botocore                  1.31.17                  pypi_0    pypi
bzip2                     1.0.8                h99b78c6_7    conda-forge
ca-certificates           2024.8.30            hf0a4a13_0    conda-forge
cellcharter               0.3.2                    pypi_0    pypi
certifi                   2024.8.30                pypi_0    pypi
cffi                      1.17.1                   pypi_0    pypi
charset-normalizer        3.4.0                    pypi_0    pypi
chex                      0.1.87                   pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
click-plugins             1.1.1                    pypi_0    pypi
cligj                     0.7.2                    pypi_0    pypi
cloudpickle               3.1.0                    pypi_0    pypi
colorcet                  3.1.0                    pypi_0    pypi
comm                      0.2.2                    pypi_0    pypi
contourpy                 1.3.1                    pypi_0    pypi
cpython                   3.10.15         py310hd8ed1ab_2    conda-forge
cycler                    0.12.1                   pypi_0    pypi
dask                      2024.11.2                pypi_0    pypi
dask-expr                 1.1.19                   pypi_0    pypi
dask-image                2024.5.3                 pypi_0    pypi
datashader                0.16.3                   pypi_0    pypi
debugpy                   1.8.9                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
distributed               2024.11.2                pypi_0    pypi
docrep                    0.3.2                    pypi_0    pypi
etils                     1.10.0                   pypi_0    pypi
exceptiongroup            1.2.2                    pypi_0    pypi
executing                 2.1.0                    pypi_0    pypi
fasteners                 0.19                     pypi_0    pypi
fastjsonschema            2.20.0                   pypi_0    pypi
filelock                  3.16.1             pyhd8ed1ab_0    conda-forge
flax                      0.10.2                   pypi_0    pypi
fonttools                 4.55.0                   pypi_0    pypi
fqdn                      1.5.1                    pypi_0    pypi
frozenlist                1.5.0                    pypi_0    pypi
fsspec                    2023.6.0                 pypi_0    pypi
gdown                     5.2.0                    pypi_0    pypi
geopandas                 1.0.1                    pypi_0    pypi
gmp                       6.3.0                h7bae524_2    conda-forge
gmpy2                     2.1.5           py310heb17c8b_2    conda-forge
grpcio                    1.68.0                   pypi_0    pypi
h11                       0.14.0                   pypi_0    pypi
h5py                      3.12.1                   pypi_0    pypi
httpcore                  1.0.7                    pypi_0    pypi
httpx                     0.27.2                   pypi_0    pypi
humanize                  4.11.0                   pypi_0    pypi
idna                      3.10                     pypi_0    pypi
igraph                    0.11.8                   pypi_0    pypi
imageio                   2.36.0                   pypi_0    pypi
importlib-metadata        8.5.0                    pypi_0    pypi
importlib-resources       6.4.5                    pypi_0    pypi
inflect                   7.4.0                    pypi_0    pypi
ipykernel                 6.29.5                   pypi_0    pypi
ipython                   8.29.0                   pypi_0    pypi
ipywidgets                8.1.5                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
jax                       0.4.35                   pypi_0    pypi
jaxlib                    0.4.35                   pypi_0    pypi
jedi                      0.19.2                   pypi_0    pypi
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jmespath                  1.0.1                    pypi_0    pypi
joblib                    1.4.2                    pypi_0    pypi
json5                     0.9.28                   pypi_0    pypi
jsonpointer               3.0.0                    pypi_0    pypi
jsonschema                4.23.0                   pypi_0    pypi
jsonschema-specifications 2024.10.1                pypi_0    pypi
jupyter                   1.1.1                    pypi_0    pypi
jupyter-client            8.6.3                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-core              5.7.2                    pypi_0    pypi
jupyter-events            0.10.0                   pypi_0    pypi
jupyter-lsp               2.2.5                    pypi_0    pypi
jupyter-server            2.14.2                   pypi_0    pypi
jupyter-server-terminals  0.5.3                    pypi_0    pypi
jupyterlab                4.2.6                    pypi_0    pypi
jupyterlab-pygments       0.3.0                    pypi_0    pypi
jupyterlab-server         2.27.3                   pypi_0    pypi
jupyterlab-widgets        3.0.13                   pypi_0    pypi
kiwisolver                1.4.7                    pypi_0    pypi
lazy-loader               0.4                      pypi_0    pypi
legacy-api-wrap           1.4                      pypi_0    pypi
leidenalg                 0.10.2                   pypi_0    pypi
libabseil                 20240722.0      cxx17_hf9b8971_1    conda-forge
libblas                   3.9.0           25_osxarm64_openblas    conda-forge
libcblas                  3.9.0           25_osxarm64_openblas    conda-forge
libcxx                    19.1.4               ha82da77_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_3    conda-forge
libgfortran5              13.2.0               hf226fd6_3    conda-forge
liblapack                 3.9.0           25_osxarm64_openblas    conda-forge
libopenblas               0.3.28          openmp_hf332438_1    conda-forge
libprotobuf               5.28.2               h8f0b736_0    conda-forge
libsqlite                 3.47.0               hbaaea75_1    conda-forge
libtorch                  2.5.1           cpu_generic_h61b979d_3    conda-forge
libuv                     1.49.2               h7ab814d_0    conda-forge
libzlib                   1.3.1                h8359307_2    conda-forge
lightning                 2.4.0                    pypi_0    pypi
lightning-utilities       0.11.9                   pypi_0    pypi
llvm-openmp               19.1.4               hdb05f8b_0    conda-forge
llvmlite                  0.43.0                   pypi_0    pypi
locket                    1.0.0                    pypi_0    pypi
markdown                  3.7                      pypi_0    pypi
markdown-it-py            3.0.0                    pypi_0    pypi
markupsafe                3.0.2           py310h5799be4_0    conda-forge
matplotlib                3.8.4                    pypi_0    pypi
matplotlib-inline         0.1.7                    pypi_0    pypi
matplotlib-scalebar       0.8.1                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mistune                   3.0.2                    pypi_0    pypi
ml-collections            1.0.0                    pypi_0    pypi
ml-dtypes                 0.5.0                    pypi_0    pypi
more-itertools            10.5.0                   pypi_0    pypi
mpc                       1.3.1                h8f1351a_1    conda-forge
mpfr                      4.2.1                hb693164_3    conda-forge
mpmath                    1.3.0              pyhd8ed1ab_0    conda-forge
msgpack                   1.1.0                    pypi_0    pypi
mudata                    0.3.1                    pypi_0    pypi
multidict                 6.1.0                    pypi_0    pypi
multipledispatch          1.0.0                    pypi_0    pypi
multiscale-spatial-image  1.0.1                    pypi_0    pypi
muon                      0.1.7                    pypi_0    pypi
natsort                   8.4.0                    pypi_0    pypi
nbclient                  0.10.0                   pypi_0    pypi
nbconvert                 7.16.4                   pypi_0    pypi
nbformat                  5.10.4                   pypi_0    pypi
ncurses                   6.5                  h7bae524_1    conda-forge
nest-asyncio              1.6.0                    pypi_0    pypi
networkx                  3.4.2              pyh267e887_2    conda-forge
newick                    1.0.0                    pypi_0    pypi
nomkl                     1.0                  h5ca1d4c_0    conda-forge
notebook                  7.2.2                    pypi_0    pypi
notebook-shim             0.2.4                    pypi_0    pypi
numba                     0.60.0                   pypi_0    pypi
numcodecs                 0.13.1                   pypi_0    pypi
numpy                     1.26.4                   pypi_0    pypi
numpyro                   0.15.3                   pypi_0    pypi
ome-zarr                  0.9.0                    pypi_0    pypi
omnipath                  1.0.8                    pypi_0    pypi
openssl                   3.4.0                h39f12f2_0    conda-forge
opt-einsum                3.4.0                    pypi_0    pypi
optax                     0.2.4                    pypi_0    pypi
orbax-checkpoint          0.10.0                   pypi_0    pypi
overrides                 7.7.0                    pypi_0    pypi
packaging                 24.2                     pypi_0    pypi
pandas                    2.2.3                    pypi_0    pypi
pandocfilters             1.5.1                    pypi_0    pypi
param                     2.1.1                    pypi_0    pypi
parso                     0.8.4                    pypi_0    pypi
partd                     1.4.2                    pypi_0    pypi
patsy                     1.0.1                    pypi_0    pypi
pexpect                   4.9.0                    pypi_0    pypi
pillow                    11.0.0                   pypi_0    pypi
pims                      0.7                      pypi_0    pypi
pip                       24.3.1             pyh8b19718_0    conda-forge
platformdirs              4.3.6                    pypi_0    pypi
pooch                     1.8.2                    pypi_0    pypi
prometheus-client         0.21.0                   pypi_0    pypi
prompt-toolkit            3.0.48                   pypi_0    pypi
propcache                 0.2.0                    pypi_0    pypi
protobuf                  5.28.3                   pypi_0    pypi
psutil                    6.1.0                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.3                    pypi_0    pypi
pyarrow                   18.0.0                   pypi_0    pypi
pycparser                 2.22                     pypi_0    pypi
pyct                      0.5.0                    pypi_0    pypi
pygments                  2.18.0                   pypi_0    pypi
pynndescent               0.5.13                   pypi_0    pypi
pyogrio                   0.10.0                   pypi_0    pypi
pyparsing                 3.2.0                    pypi_0    pypi
pyproj                    3.7.0                    pypi_0    pypi
pyro-api                  0.1.2                    pypi_0    pypi
pyro-ppl                  1.9.1                    pypi_0    pypi
pysocks                   1.7.1                    pypi_0    pypi
python                    3.10.15         hdce6c4c_2_cpython    conda-forge
python-dateutil           2.9.0.post0              pypi_0    pypi
python-json-logger        2.0.7                    pypi_0    pypi
python_abi                3.10                    5_cp310    conda-forge
pytorch                   2.5.1           cpu_generic_py310hf6af900_3    conda-forge
pytorch-lightning         2.4.0                    pypi_0    pypi
pytz                      2024.2                   pypi_0    pypi
pyyaml                    6.0.2                    pypi_0    pypi
pyzmq                     26.2.0                   pypi_0    pypi
rasterio                  1.4.2                    pypi_0    pypi
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.35.1                   pypi_0    pypi
requests                  2.32.3                   pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rich                      13.9.4                   pypi_0    pypi
rpds-py                   0.21.0                   pypi_0    pypi
s3fs                      2023.6.0                 pypi_0    pypi
scanpy                    1.10.4                   pypi_0    pypi
scarches                  0.6.1                    pypi_0    pypi
schpl                     1.0.5                    pypi_0    pypi
scikit-image              0.24.0                   pypi_0    pypi
scikit-learn              1.5.2                    pypi_0    pypi
scipy                     1.14.1                   pypi_0    pypi
scvi-tools                1.2.0                    pypi_0    pypi
seaborn                   0.13.2                   pypi_0    pypi
send2trash                1.8.3                    pypi_0    pypi
session-info              1.0.0                    pypi_0    pypi
setuptools                75.6.0             pyhff2d567_0    conda-forge
shapely                   2.0.6                    pypi_0    pypi
simplejson                3.19.3                   pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
sknw                      0.15                     pypi_0    pypi
sleef                     3.7                  h8391f65_2    conda-forge
slicerator                1.1.0                    pypi_0    pypi
sniffio                   1.3.1                    pypi_0    pypi
sortedcontainers          2.4.0                    pypi_0    pypi
soupsieve                 2.6                      pypi_0    pypi
sparse                    0.15.4                   pypi_0    pypi
spatial-image             1.1.0                    pypi_0    pypi
spatialdata               0.2.5.post0              pypi_0    pypi
spatialdata-plot          0.2.7                    pypi_0    pypi
squidpy                   1.6.2                    pypi_0    pypi
stack-data                0.6.3                    pypi_0    pypi
statsmodels               0.14.4                   pypi_0    pypi
stdlib-list               0.11.0                   pypi_0    pypi
sympy                     1.13.3           pyh2585a3b_104    conda-forge
tblib                     3.0.0                    pypi_0    pypi
tensorboard               2.18.0                   pypi_0    pypi
tensorboard-data-server   0.7.2                    pypi_0    pypi
tensorstore               0.1.68                   pypi_0    pypi
terminado                 0.18.1                   pypi_0    pypi
texttable                 1.7.0                    pypi_0    pypi
threadpoolctl             3.5.0                    pypi_0    pypi
tifffile                  2024.9.20                pypi_0    pypi
tinycss2                  1.4.0                    pypi_0    pypi
tk                        8.6.13               h5083fa2_1    conda-forge
tomli                     2.1.0                    pypi_0    pypi
toolz                     1.0.0                    pypi_0    pypi
torchgmm                  0.1.2                    pypi_0    pypi
torchmetrics              1.6.0                    pypi_0    pypi
tornado                   6.4.2                    pypi_0    pypi
tqdm                      4.67.0                   pypi_0    pypi
traitlets                 5.14.3                   pypi_0    pypi
typeguard                 4.4.1                    pypi_0    pypi
types-python-dateutil     2.9.0.20241003           pypi_0    pypi
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024.2                   pypi_0    pypi
umap-learn                0.5.7                    pypi_0    pypi
uri-template              1.3.0                    pypi_0    pypi
urllib3                   1.26.20                  pypi_0    pypi
validators                0.34.0                   pypi_0    pypi
wcwidth                   0.2.13                   pypi_0    pypi
webcolors                 24.11.1                  pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.8.0                    pypi_0    pypi
werkzeug                  3.1.3                    pypi_0    pypi
wheel                     0.45.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.13                   pypi_0    pypi
wrapt                     1.17.0                   pypi_0    pypi
xarray                    2024.3.0                 pypi_0    pypi
xarray-dataclasses        1.8.0                    pypi_0    pypi
xarray-datatree           0.0.15                   pypi_0    pypi
xarray-schema             0.0.3                    pypi_0    pypi
xarray-spatial            0.4.0                    pypi_0    pypi
xz                        5.2.6                h57fd34a_0    conda-forge
yarl                      1.18.0                   pypi_0    pypi
zarr                      2.18.3                   pypi_0    pypi
zict                      3.0.0                    pypi_0    pypi
zipp                      3.21.0                   pypi_0    pypi

[marcovarrone]

Now the packages look great! The schpl warning is not a big of a deal because CellCharter doesn't use it.

Regarding the notebook error, it's my bad, I have to make the documentation clearer.
The notebook is meant to be run from the clone of the repository.
This is because I don't want to include unnecessary files (like the pre-trained trVAE model for that specific CODEX dataset) in the cellcharter package.

In alternative, the quickest solution is just to download the 3 files of the trVAE model from the GitHub page and place them in the appropriate directory.

[josenimo]

hey @marcovarrone
wonderful now it works.

I have some general questions about usage of cellcharter, where would you prefer to discuss these? Intuitively I went to image.sc but it seems there is nothing there, and also not in scverse (btw we are starting a spatial proteomics group). I guess we could also zoom :)

[marcovarrone]

Perfect @josenimo!

I enabled "Discussions" in the repository.
In this way, if someone else is wondering the same things as yours, they will be able to find the answers there.
Then, if we see that the conversation gets too complex and long, we can move it to Zoom :)

[marcovarrone]

Actually I am reopening it as a reminder to change the notebooks and documentation

[josenimo]

In that case I can report that

sq.pl.spatial_scatter(
    adata, 
    color='sample', 
    library_key='sample', 
    img=None, 
    title=['BALBc-3'],
    size=2500,
    connectivity_key='spatial_connectivities',
    edges_width=0.3,
    legend_loc=None,
    crop_coord=(900000, 900000, 1700000, 1700000 ),
    library_id=['BALBc-3']
    )

runs into an error because can't find adata.obsm['spatial_connectivities']

If you need any testers let me know :)

[marcovarrone]

Did you run sq.gr.spatial_neighbors(adata, library_key='sample', coord_type='generic', delaunay=True), the previous cell in the notebook?

[josenimo]

I did not, hehe, my bad, now it works!

[josenimo]

This is slowly becoming a tutorial troubleshooting thread 😅

My kernel crashes when reaching the

gmm.fit(adata, use_rep='X_cellcharter')

I get:

The Kernel crashed while executing code in the current cell or a previous cell. 
Please review the code in the cell(s) to identify a possible cause of the failure. 
Click [here](https://aka.ms/vscodeJupyterKernelCrash) for more info. 
View Jupyter [log](command:jupyter.viewOutput) for further details.

With a very useless jupyter log about how it failed, any tips here?

[marcovarrone]

No problem!
Usually, this happens for the machine running out of memory. Which spatial omics technology are you using?
Have you done dimensionality reduction?
Can you print adata.obsm['X_cellcharter'].shape?

[josenimo]

I did, to understand the elements and stuff, as predicted by tutorial:
adata.obsm['X_cellcharter'].shape
(707474, 40)

I am on a MacOS Sonoma, M2 Pro, 16GB Memory.