[Issue #245] Add pipeline parallelism support

[iMmAseu]

Summary

This PR fixes a TimeoutError during vLLM initialization when Pipeline Parallelism is enabled.

Fixes #243

---

Root Cause

KVCacheManager only used tensor_parallel_size, which caused incorrect process detection in PP setups.

---

Changes

• Fix IPC synchronization using global world_size
• Compute consistent global_rank
• Rename tp_size → world_size
• Update listener rank detection

---

Test Plan

• Syntax check
• Multi GPU inference (on Qwen3 8B)
• PP inference
• sgl and support for other version vllm
• tp=2 and pp=2

[ivanium]

Just curious does this PR support enabling both TP and PP, say TP=2 and PP=2? In that case, how do we get the consistent global_rank here?

[iMmAseu]

@ivanium Thanks for pointing this out, your concern is absolutely valid. I’ve updated the implementation to correctly handle the TP and PP setup and compute a consistent global_rank across all processes. I’ve also run tests with configurations like TP=2 and PP=2, and the initialization now works seemly as expected without synchronization issues.

[cui36]

Hi @iMmAseu, thanks for the update! Let's split it into two parts, and I will go through the vllm patch.

[cui36]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for pipeline parallelism by refactoring the handling of distributed process ranks. The changes primarily involve renaming tp_size to world_size and adding pp_rank to correctly manage inter-process communication for KV cache synchronization across different pipeline stages. The implementation correctly namespaces IPC sockets by pp_rank to prevent conflicts.

My review has identified a few areas for improvement:

• There is some duplicated code for resolving distributed ranks in the SGLang integration patches.
• The use of broad except Exception clauses could mask unexpected errors.
• The docstrings for the world_size parameter are inconsistent with its actual usage, which could lead to confusion.

Overall, the changes are well-structured and address the issue described. Addressing the feedback will improve the code's maintainability and clarity.