[Bugfix] Avoid merging empty-only partitions into splitting-op subgraphs

[ZJY0516]

Purpose

Fix #28044 (comment)

Test Plan

vllm serve deepseek-ai/DeepSeek-V3 -tp 8

lm_eval --model local-completions --model_args "model=deepseek-ai/DeepSeek-V3,base_url=http://0.0.0.0:8000/v1/completions,tokenized_requests=False,tokenizer_backend=None,num_concurrent=256,timeout=5000,max_length=4096" --tasks gsm8k --num_fewshot 5

Test Result

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.9500	±	0.006
		strict-match	5	exact_match	↑	0.9492	±	0.006

Note

I don't test deepsek v32 because my machine will oom after #30515

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request correctly modifies the graph partitioning logic to prevent empty-only partitions from being merged into splitting-op subgraphs. The implementation in vllm/compilation/backends.py is sound and addresses the intended issue. The accompanying tests in tests/compile/test_graph_partition.py have been updated to reflect this new behavior. However, I've identified a redundant test case that should be removed to improve the maintainability of the test suite.

[ElizaWszola]

nit: why not just place continue at the end of the if-statement?

[mgoin]

Cc @ProExpertProg @zou3519

[LopezCastroRoberto]

I did not check DSv3.2 due to the OOM issues referenced in the description. However, I did check DSv3.2-NVFP4 because I was also working on a fix for this, and the accuracy regression now appears to be resolved.

vllm serve nvidia/DeepSeek-V3.2-NVFP4 -tp 4

python tests/evals/gsm8k/gsm8k_eval.py

BEFORE:

Results:
Accuracy: 0.024
Invalid responses: 0.115
Total latency: 80.326 s
Questions per second: 16.421
Total output tokens: 323952

AFTER:

Results:
Accuracy: 0.952
Invalid responses: 0.000
Total latency: 45.151 s
Questions per second: 29.213
Total output tokens: 118859
Output tokens per second: 2632.498

[ZJY0516]

@LopezCastroRoberto Do you see warnings like UserWarning: The CUDA Graph is empty. This usually means that the graph was attempted to be captured on wrong device or stream. (Triggered internally at /pytorch/aten/src/ATen/cuda/CUDAGraph.cpp:138.) .This is the issue that #28044 aims to address.

[LopezCastroRoberto]

@ZJY0516 Actually, no. At least with this model, CUDA graph capturing seems to work fine.

Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|████| 51/51
Capturing CUDA graphs (decode, FULL): 100%|████| 51/51
Registering 12546 cuda graph addresses
Registering 12546 cuda graph addresses
Registering 12546 cuda graph addresses
Graph capturing finished in 23 secs, took 2.43 GiB

[ProExpertProg]

Just a few nits

[ProExpertProg]

Why do we need to reconstruct this? Isn't this guaranteed to be range(0, max(node_to_subgraph.values())+1)?

[ZJY0516]

you are right

[ProExpertProg]

I think right now we might produce consecutive splitting graphs. But let's address that in a follow up

[jikunshang]

FYI. distributed-test-4-gpus case is fixed in #36628

[ZJY0516]

FYI. distributed-test-4-gpus case is fixed in #36628

Thanks. I think force merge will better because full ci needs too much time

[jikunshang]

cc @DarkLight1337 for force merge.

[elvircrn]

FYI I restarted the currently failing AMD test https://buildkite.com/organizations/vllm/pipelines/ci/builds/55484/jobs/019cd872-a630-496c-9014-b44a2c713696/log