Skip to content

Fix for CUDA issue #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cplett merged 5 commits into from
Jan 20, 2026
Merged

Fix for CUDA issue #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8b713f7
Use spawn multiprocessing context for CUDA calculations
kevinkong98 Dec 27, 2025
df9c3dc
Merge pull request #1 from kevinkong98/codex/fix-cuda-subprocess-init…
kevinkong98 Dec 27, 2025
58368d7
Remove offline cache warning
kevinkong98 Dec 27, 2025
7f3a43c
Merge pull request #2 from kevinkong98/codex/edit-offline-mode-checks…
kevinkong98 Dec 27, 2025
3131ee0
Improve model file detection
cplett Jan 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 14 additions & 10 deletions src/oet/calculator/uma.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,19 +303,23 @@ def calc(
or not isinstance(cache_dir, str)
):
raise RuntimeError("Problems handling input parameters.")
# Check if we have the respective models stored locally in cache
# If so, switch to offline mode
if self.check_for_model_files(basemodel=basemodel, cache_dir=cache_dir):
print("Model parameters found in cache. Switching to offline mode.")
# Check if the model files are available
model_files_available = self.check_for_model_files(basemodel=basemodel, cache_dir=cache_dir)
# If they are available, switch to offline mode.
if model_files_available:
self.switch_to_offline_mode()
# If set by the user, switch also to offline mode.
# This is a fallback, if online communication must be prevented.
if offline_mode:
print("Model files detected. Switching to offline mode.")
# If they are not available, but the user requested the offline mode to prevent online communication,
# print a warning as this will likely cause subsequent errors.
elif offline_mode:
self.switch_to_offline_mode()
if self.check_for_model_files(basemodel=basemodel, cache_dir=cache_dir):
print(
"WARNING: No model files were detected. This might lead to subsequent errors."
# Check if the model files are locally available. If not, subsequent errors will occur
# as they cannot be downloaded.
print(
"WARNING: Offline mode selected, but no model files were detected. "
"This will likely cause subsequent errors."
)
Copy link
Contributor

@cplett cplett Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would there be any downside to keeping this warning? If not, I think it would make sense to leave it in place with the corrected logic:

            if not self.check_for_model_files(basemodel=basemodel, cache_dir=cache_dir):
                print(
                    "WARNING: No model files were detected. This might lead to subsequent errors."
                )

Copy link
Contributor Author

@kevinkong98 kevinkong98 Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not really but seems counter intuitive that the check is done no matter what =)
assuming this check is basically free it should be fine either way

Copy link
Contributor

@cplett cplett Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the model file detection a little. I hope this is fine by you.


# setup calculator if not already set
# this is important as usage on a server would otherwise cause
# initialization with every call so that nothing is gained
Expand Down
7 changes: 6 additions & 1 deletion src/oet/server_client/server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
from collections import OrderedDict
from collections.abc import Mapping, Sequence
from concurrent.futures import BrokenExecutor, ProcessPoolExecutor
import multiprocessing as mp
from contextlib import redirect_stdout
from pathlib import Path
from types import FrameType
Expand Down Expand Up @@ -623,8 +624,12 @@ def main() -> None:

# Create workers
workers = args.nthreads
# Use spawn start method so CUDA can be initialized safely in worker processes
mp_ctx = mp.get_context("spawn")
# Initialize the ProcessPool
executor = ProcessPoolExecutor(max_workers=workers, initializer=worker_initializer)
executor = ProcessPoolExecutor(
max_workers=workers, initializer=worker_initializer, mp_context=mp_ctx
)

# Make a CalculatorClass getting the hooks on calculators argument parsing
# Info on calculator type is store in the object for client requests
Expand Down