[torch.compile] Use Inductor Process Pool in Compilation

[eellison]

Enable parallel inductor compilation via a subprocess pool, allowing triton kernel compilation to happen asynchronously across multiple processes. Previously, vLLM hard-coded compile_threads=1, so all triton kernels were compiled sequentially in the main process.

This change auto-computes a default based on available CPUs and GPUs: min(8, cpu_count // num_gpus - 1), capped at 8 since vLLM's graph splitting typically produces only ~4 unique kernels. The default can be overridden with VLLM_COMPILE_PROCESSES.

Lifecycle:

• Pool is warmed up at the end of load_model(), before first torch.compile.
• Pool is quiesced before cudagraph capture
• After quiesce, the sidecar subprocess settles to 0% CPU within a few seconds (just a sleeping process waiting on a pipe read), so it does not interfere with inference, which is CPU-sensitive.

Pool overhead (8 processes):

warm_pool: ~124ms
quiesce: <1ms

Benchmark (facebook/opt-1.3b, 1 GPU):

Config	Graph compile	torch.compile total	Init time
threads=1	6.73s	9.40s	34.21s
threads=8	5.87s	8.45s	32.33s
Speedup	13%	10%	5.5%

Measurement scripts for pool overhead: https://gist.github.com/eellison/4137011c0cc9c9260b1e5a35522ef90b

Purpose

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request introduces parallel compilation for torch.compile using an Inductor process pool, which is a great performance improvement. The implementation correctly calculates the number of processes and manages the pool's lifecycle. I've found one critical issue related to ensuring a fallback to single-threaded compilation when parallel compilation is not supported, which could lead to hangs. My review includes a suggested fix for this.

_{Note: Security Review did not run due to the size of the PR.}

[mergify]

Hi @eellison, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @eellison.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[zou3519]

@eellison we fixed this, please rebase

[zou3519]

same here

[zou3519]

Does the other test in this file affect this test?

[zou3519]

why pytorch versions don't? If the answer is < 2.9, then we can just remove the extra check

[zou3519]

main thing is that there shouldn't be the test_moe_startup change, otherwise, this lgtm