Skip to content

Regresion ProcessPoolExecutor when updating to python 3.14 #473

@PabloRuizCuevas

Description

@PabloRuizCuevas

Usually I used loky like this:

with ProcessPoolExecutor(max_workers=workers) as executor:
    for _ in range(workers):
        executor.submit(optimize, study_name, storage_url, trials, search_f, objetive_dict)

But after updating to python 3.14 it fails, it seems that paths are being concatenated in a weird maner:

#notice path repeated

/Users/pablo.ruiz/Documents/trade/.venv/bin/python: can't open file '/Users/pablo.ruiz/Documents/trade/pytrade/backtester/core/bench.py/Users/pablo.ruiz/Documents/trade/.venv/bin/python': [Errno 20] Not a directory

To be honest I didn't dig much into the details, but I was able to find the solution with a bit of help from AI plus trial and error.

import multiprocessing
ctx = multiprocessing.get_context("spawn")
ctx.set_executable(sys.executable)  # ensure correct Python interpreter is used
      with ProcessPoolExecutor(max_workers=workers, context=ctx) as executor:
          for _ in range(workers):
              executor.submit(optimize, study_name, storage_url, trials, search_f, objetive_dict)

Seems that Python 3.14 changed the process spawning mechanism, causing the interpreter path to be joined incorrectly with the script path.

Anyway feel free to close this if it is a triviality, but i felt that updating python should not break the code, so best to report it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions