"Bad ZIP File" when downloading models

[brav0charlie]

Ever since the PHP 8.2 update, when I attempt to bring up the containers, they begin to download the models, but shortly after they're downloaded, an error is recorded in the "ocsr_socket_server_6543*.log" files (65433 and 65434):

Downloading trained model to /root/.data/DECIMER-V2
/root/.data/DECIMER-V2/DECIMER_HandDrawn_model.zip
... done downloading trained model!
Traceback (most recent call last):
  File "/var/www/app/app/Python/decimer_predictor_server.py", line 7, in <module>
    from DECIMER import predict_SMILES
  File "/usr/local/lib/python3.11/dist-packages/DECIMER/__init__.py", line 29, in <module>
    from .decimer import predict_SMILES
  File "/usr/local/lib/python3.11/dist-packages/DECIMER/decimer.py", line 71, in <module>
    tokenizer, DECIMER_V2, DECIMER_Hand_drawn = get_models(model_urls)
                                                ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/DECIMER/decimer.py", line 56, in get_models
    model_paths = utils.ensure_models(default_path=default_path, model_urls=model_urls)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/DECIMER/utils.py", line 93, in ensure_models
    config.download_trained_weights(model_url, default_path)
  File "/usr/local/lib/python3.11/dist-packages/DECIMER/config.py", line 419, in download_trained_weights
    with zipfile.ZipFile(model_path.as_posix(), "r") as zip_ref:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/zipfile.py", line 1302, in __init__
    self._RealGetContents()
  File "/usr/lib/python3.11/zipfile.py", line 1369, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

If I open Bash in the container (docker exec -it --privileged decimer_app bash), I cannot find the /root/.data directory (ls /root/.data returns directory not found). Tried the same on the decimer_supervisor container, and the directory also doesn't appear to exist there either.

I've tried adding a volume to the docker-compose.yml file for (/home/decimer/models:/root/.data) on the containers so I could verify the files are actually downloading, and I see them on my file system when they download to that added volume, but the same error pops regardless of whether I'm using that added volume or not.

[Kohulan]

Hi @brav0charlie ,

This issue typically occurs when Zenodo is overloaded with download requests. As a result, the file may not finish downloading before Python tries to read it. Since Python encounters a partially downloaded file, it raises this error. Allow some extra time for the ZIP file to download completely.

• Kohulan

[brav0charlie]

Hi @Kohulan,

Apologies for the very delayed response. Between the holidays and some priority project work I'm just getting back to this today.

I'm a little confused: I'm not performing the download or the unzip operation, the container is. How would I make the container wait before attempting to unzip the downloaded ZIP files?

I've left the container running for 3-4 days at a time, and it just keeps downloading the models (showing download progress in the logs), then failing when it attempts to extract the ZIP.

I've tried on two different machines from two different networks, and the issue persists.

[brav0charlie]

Quick follow-up here:

I extracted the URLs to download the models myself, and when I copied the ZIP files directly to /root/.data/DECIMER-V2, the code was able to extract the ZIP files without any errors.

I'm not sure what's causing the failure, but manually downloading the model ZIP files, then copying them into place where the container expects them seems to work fine.

[Kohulan]

Hi @brav0charlie

Thank you for the update! I'm still investigating why the models aren't downloading on your end and need some time to properly diagnose the issue. I'll keep this thread updated once I've identified and resolved the problem on my side. In the meantime, I'm glad you found a workaround that works for you.