[BUG] Unable to instantiate HuggingFace AUTOTOKENIZER for nvidia/Nemotron-4-340B-Base

[J3Chen51]

I am attempting to run Nemotron4-340b testing on a single GB300 rack configuration (64 GPU), bf16. I am running into some issues in regards to the latest build (25.10). This build added HuggingFace authentication as a requirement for Nemotron4-340b, but I am unable to get my authentication to work.

ValueError: Unable to instantiate HuggingFace AUTOTOKENIZER for nvidia/Nemotron-4-340B-Base. Exception: We couldn't connect to 'https://huggingface.co' to load the files, and couldn't find them in the cached files.
Check your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'. [huggingface.co]
Fiddle context: failed to construct or call AutoTokenizer at .tokenizer with positional arguments: (), keyword arguments: (pretrained_model_name='nvidia/Nemotron-4-340B-Base', use_fast=True).

I have created a new account and created a token with write permissions, and provided it to the install. I then proceeded to add the HF_TOKEN parameter in manually through launch.sh, and also logged in through each virtual environment. However, I am still running into this error.

I have been able to ping the huggingface IP address (3.170.185.35), as well as been able to run a curl command using the token. However, the script fails to connect.

I have reverted to the 25.08 build, which works fine for me. Do you know what could be causing this authentication error?

[sudostock]

We are currently working on a fix to download the tokenizer as part of the install process. Don't have an ETA yet.

As to what is causing the issue we've been unable to reproduce. It seems related to the number of processes trying to make the same request. Which should be resolved by pre-downloading.

[the-elder]

same issue for me。i am on an air-gap ENV，how can i avoid this issue ? @sudostock

[the-elder]

the URL appears to access a non-existent JSON file.

[sudostock]

v25.10.01 has been published which includes an update to download the tokenizers during the install process.
@J3Chen51 Please try with this update, you'll need to do a fresh install.

@the-elder We don't have true air-gap support at the moment. The installer assumes it can talk to slurm. That said if you have a system where you can run the installer the $LLMB_INSTALL folder you selected during install is self contained and can be copied into your air-gapped env. You'll also need to copy (or create a new one) the venv with llmb-run installed.