Regarding server client issue to run the model

[sandipgiri576]

I was trying to run the extopt goat run with the aimnet2 example in an HPC Linux environment (in Slurm). First I run

./aimnet2server.sh 
INFO:waitress: Serving on http://127.0.0.1:8888

It initially downloads all the dependencies and model checkpoints and successfully builds them. In another login terminal, I run
<full_path>/orca H2O_goat_extclient.inp
Getting this issue
File "<fil_path>/aimnet2/.venv/lib/python3.11/site-packages/requests/models.py", line 1026, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests. exceptions.HTTPError: 403 Client Error: Forbidden for url: http://127.0.0.1:8888/calculate

Error (ORCA/EXT): external program failed with exit code 256

Is it possible to change the IP and port in the aimnet2client argument? Please recommend the best combinations to run in a Slurm-based CentOS HPC.

[cplett]

Hey,
This type of error can occur if the fairchem core isn't able to verify that you're logged in with your Hugging Face account.
If you haven't logged in yet, the easiest way is to activate the virtual environment created by the server script (source /path/to/orca-external-tools/uma/.venv/bin/activate) and then run huggingface-cli login. Make sure you are using a Hugging Face account that has access to the UMA repository.

Regarding the possibilities to change the IP and port, you can do this by starting the server using the -b flag, e.g., aimnet2server.sh -b 0.0.0.0:7777. If you do so, make sure to pass the same argument in your ORCA input using the method block ,e. g.,

%method
  progext "path/to/aimnet2client.sh"
  Ext_Params "-b 0.0.0.0:7777"
end

[sandipgiri576]

Thank you for your quick reply. I have tested that option. This is my Slurm script

< Slurm part is omitted.>
# Load ORCA module if needed
module load orca/601

# Activate aimnet2 virtualenv
source <path>/orca-external-tools/aimnet2/.venv/bin/activate

AIMNET2_SERVER_SCRIPT="<path>/orca-external-tools/aimnet2server.sh"

# Check if AIMNet2 server script exists and is executable
if [[ ! -x "$AIMNET2_SERVER_SCRIPT" ]]; then
    echo "Error: AIMNet2 server script not found or not executable: $AIMNET2_SERVER_SCRIPT"
    exit 1
fi

# Start the AIMNet2 server in the background with nohup and redirect output
nohup "$AIMNET2_SERVER_SCRIPT" -b 0.0.0.0:7777 > aimnet2server.log 2>&1 &
SERVER_PID=$!

# Ensure server is killed on script exit
trap "kill -9 $SERVER_PID 2>/dev/null" EXIT

# Wait a bit to ensure the server starts before ORCA begins
sleep 5

# Run ORCA with the extopt+GOAT input
<orca-path>/orca_6_1_0_new_y1e0/orca job.inp > orca.out

# Clean up (handled by trap)
wait $SERVER_PID 2>/dev/null || true

But still showing "Calling /orca-external-tools/aimnet2-test/aimnet2client.sh goat_EXT.extinp.tmp -b 0.0.0.0:7777 ...

Error (ORCA/EXT): external program failed with exit code 256"

and "requests.exceptions.ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=7777): Max retries exceeded with url: /calculate (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd92ab3b8d0>: Failed to establish a new connection: [Errno 111] Connection refused'))"

But it works locally in the terminal with 127.0.0.1 as the default home address.

[cplett]

This issue might be related to the cluster setup, and a few things could be causing this behavior.
I'd suggest to check whether the server started appropriately first. Maybe the aimnet2server.log file has some deeper insights.
If the server started correctly, maybe the cluster is blocking the communication via certain ports or IPs.
You could also try running a small test example using the standalone script (aimnet2.sh) without a server to verify whether the AIMNET2 environment itself is working as expected.

[sandipgiri576]

aimnet2.sh works. But for parallel calculations, server and client are a good approach. The aimnet2server log file shows a binding, but probably, there is some port issue from the client running in SLURM in our cluster system.

[cplett]

I also guess the issue might be related to your cluster environment, as I haven’t been able to reproduce the problem on our side.
There will likely be a major update to the orca-external-tools in the near future, focusing on improved code structure and more robust server handling. Hopefully, the issue will be resolved with that version.

[cplett]

Hey, I’m not sure whether you still have the issue, but you might want to try the reworked server/client architecture introduced in the recent overhaul (available since version 2.0). This change has improved the server script and may resolve the issue on your system.

[sandipgiri576]

Thanks! I will try and let you know.