cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub'

[Jukeman9]

Hello,
i am getting this error constantly when trying to run the first code block in jupyter notebook or the gradio interface. I tried upgrading the packages separately, downgrading and installing a new env and nothing works. Could you help me out does anyone had this issue?
I also just tried installing with pinokio rather than manually i run into the same issue:

C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\transformers\utils\generic.py:441: FutureWarning: `torch.utils._pytree._register_pytree_node` is deprecated. Please use `torch.utils._pytree.register_pytree_node` instead.
  _torch_pytree._register_pytree_node(
C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\transformers\utils\generic.py:309: FutureWarning: `torch.utils._pytree._register_pytree_node` is deprecated. Please use `torch.utils._pytree.register_pytree_node` instead.
  _torch_pytree._register_pytree_node(
Traceback (most recent call last):
  File "C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\transformers\utils\import_utils.py", line 1382, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "C:\pinokio\bin\miniconda\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\transformers\generation\utils.py", line 85, in <module>
    from accelerate.hooks import AlignDevicesHook, add_hook_to_module
  File "C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\accelerate\__init__.py", line 16, in <module>
    from .accelerator import Accelerator
  File "C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\accelerate\accelerator.py", line 34, in <module>
    from huggingface_hub import split_torch_state_dict_into_shards
ImportError: cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' (C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\huggingface_hub\__init__.py)
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\pinokio\api\storydiffusion-comics.git\app\app.py", line 15, in <module>
    from utils.gradio_utils import is_torch2_available
  File "C:\pinokio\api\storydiffusion-comics.git\app\utils\__init__.py", line 1, in <module>
    from .model import PhotoMakerIDEncoder
  File "C:\pinokio\api\storydiffusion-comics.git\app\utils\model.py", line 6, in <module>
    from transformers.models.clip.modeling_clip import CLIPVisionModelWithProjection
  File "C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\transformers\models\clip\modeling_clip.py", line 28, in <module>
    from ...modeling_utils import PreTrainedModel
  File "C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\transformers\modeling_utils.py", line 42, in <module>
    from .generation import GenerationConfig, GenerationMixin
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\transformers\utils\import_utils.py", line 1372, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\transformers\utils\import_utils.py", line 1384, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' (C:\pinokio\api\storydiffusion-comics.git\app\env\lib\site-packages\huggingface_hub\__init__.py)

[gadams999]

Appears to be an incorrect version of huggingface-hub. From a git clone of this repo I used this command to update and it's pulling down models:

pip install "huggingface-hub>=0.23.0"

Note: I run in a venv and not conda. This might break things, haven't tested beyond this. Source where I found the fix.

[FoxNickel]

Huggingface-hub version 0.23.0 has solved my problem. Also, the version number should not be too high, otherwise another error will be reported.