CPU count failing in certain environments

[mindflayer]

Hi there,
depending on which kernel I use to bootstrap my machine (Ubuntu generic works, liquorix doesn't), loky crashes while trying to obtain the CPU count when running inside a Docker container.

The file /sys/fs/cgroup/cpu.max is not present in the host machine, but empty inside the container:

$ cat /sys/fs/cgroup/cpu.max  # host machine
cat: /sys/fs/cgroup/cpu.max: No such file or directory (os error 2)

VS

# cat /sys/fs/cgroup/cpu.max  # docker container
#

That's the stacktrace for the crash:

mbp/multiprocessing.py:25: in get_pool
    return get_reusable_executor(max_workers=max_workers, timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/local/lib/python3.13/site-packages/loky/reusable_executor.py:89: in get_reusable_executor
    _executor, _ = _ReusablePoolExecutor.get_reusable_executor(
/usr/local/lib/python3.13/site-packages/loky/reusable_executor.py:182: in get_reusable_executor
    _executor = executor = cls(
/usr/local/lib/python3.13/site-packages/loky/reusable_executor.py:118: in __init__
    super().__init__(
/usr/local/lib/python3.13/site-packages/loky/process_executor.py:1158: in __init__
    self._setup_queues(job_reducers, result_reducers)
/usr/local/lib/python3.13/site-packages/loky/reusable_executor.py:290: in _setup_queues
    min_queue_size = max(cpu_count(), self._max_workers)
                         ^^^^^^^^^^^
/usr/local/lib/python3.13/site-packages/loky/backend/context.py:114: in cpu_count
    cpu_count_user = _cpu_count_user(os_cpu_count)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/local/lib/python3.13/site-packages/loky/backend/context.py:219: in _cpu_count_user
    cpu_count_cgroup = _cpu_count_cgroup(os_cpu_count)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

os_cpu_count = 8

    def _cpu_count_cgroup(os_cpu_count):
        # Cgroup CPU bandwidth limit available in Linux since 2.6 kernel
        cpu_max_fname = "/sys/fs/cgroup/cpu.max"
        cfs_quota_fname = "/sys/fs/cgroup/cpu/cpu.cfs_quota_us"
        cfs_period_fname = "/sys/fs/cgroup/cpu/cpu.cfs_period_us"
        if os.path.exists(cpu_max_fname):
            # cgroup v2
            # https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html
            with open(cpu_max_fname) as fh:
>               cpu_quota_us, cpu_period_us = fh.read().strip().split()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
E               ValueError: not enough values to unpack (expected 2, got 0)

/usr/local/lib/python3.13/site-packages/loky/backend/context.py:152: ValueError

[mindflayer]

This was the TL;DR from ChatGPT:

• cpu.max exists because the kernel supports cgroup v2
• It’s empty because the CPU controller is not enabled for that cgroup
• This commonly happens on hybrid or partially migrated hosts
• It’s normal, not a Docker bug