Skip to content

[NVIDIA] Update NVIDIA GPT-OSS vLLM image from v0.15.1 to v0.16.0 #800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cquil11 wants to merge 16 commits into
base: main
Choose a base branch
from
+14 −6
Open

[NVIDIA] Update NVIDIA GPT-OSS vLLM image from v0.15.1 to v0.16.0 #800

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
7b912e8
Update NVIDIA GPT-OSS vLLM image from v0.15.1 to v0.16.0
github-actions[bot] Feb 26, 2026
d3e5c26
Update perf-changelog.yaml with new vLLM details
cquil11 Feb 26, 2026
e4863ad
update container image
ankursingh-nv Mar 6, 2026
83612e7
add stream-interval
ankursingh-nv Mar 6, 2026
8d68260
update perf-changelog
ankursingh-nv Mar 6, 2026
8c20107
add flock for h100
cquil11 Mar 6, 2026
d7c0a47
add flock for cw runners as well
cquil11 Mar 6, 2026
320bb8f
add flock for cw runners as well pt 2
cquil11 Mar 6, 2026
b13abb5
cw h100 and h200 require separate squash for users
cquil11 Mar 6, 2026
37381b7
cw update
cquil11 Mar 6, 2026
4ca13fc
cw update pt 2
cquil11 Mar 6, 2026
7e64f05
reverting hf download
cquil11 Mar 6, 2026
a0afe2f
reverting cw changes
cquil11 Mar 6, 2026
7a3406a
Merge branch 'main' into claude/issue-798-20260226-0534
cquil11 Mar 7, 2026
ebe7cb3
trigger test check
cquil11 Mar 7, 2026
390816f
b200 rm locks revert
cquil11 Mar 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3076,7 +3076,7 @@ gptoss-fp4-b200-trt:
- { tp: 8, conc-start: 4, conc-end: 4}

gptoss-fp4-b200-vllm:
image: vllm/vllm-openai:v0.15.1
image: vllm/vllm-openai:v0.16.0-cu130
model: openai/gpt-oss-120b
model-prefix: gptoss
runner: b200
Expand Down Expand Up @@ -3107,7 +3107,7 @@ gptoss-fp4-b200-vllm:
- { tp: 8, conc-start: 4, conc-end: 4 }

gptoss-fp4-h100-vllm:
image: vllm/vllm-openai:v0.15.1
image: vllm/vllm-openai:v0.16.0-cu130
model: openai/gpt-oss-120b
model-prefix: gptoss
runner: h100
Expand Down Expand Up @@ -3386,7 +3386,7 @@ gptoss-fp4-h200-trt:
- { tp: 8, ep: 8, dp-attn: false, conc-start: 4, conc-end: 8 }

gptoss-fp4-h200-vllm:
image: vllm/vllm-openai:v0.15.1
image: vllm/vllm-openai:v0.16.0-cu130
model: openai/gpt-oss-120b
model-prefix: gptoss
runner: h200
Expand Down
11 changes: 11 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -881,3 +881,14 @@
- "Expanding TP search space"
- "Adding kv-cache-fp8"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/865

- config-keys:
- gptoss-fp4-b200-vllm
- gptoss-fp4-h100-vllm
- gptoss-fp4-h200-vllm
description:
- "Update vLLM image from v0.15.1 to v0.16.0-cu130 for NVIDIA GPT-OSS configs"
- "All existing BKC flags preserved (VLLM_MXFP4_USE_MARLIN=1 on H100/H200, VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8=1 on B200)"
- "v0.16.0 key changes: async scheduling + PP (30.8% throughput), new MXFP4 backends (SM90 FlashInfer BF16, SM100 CUTLASS), MoE cold start optimization"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/800

3 changes: 0 additions & 3 deletions runners/launch_b200-nb.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,6 @@ SPEC_SUFFIX=$([[ "$SPEC_DECODING" == "mtp" ]] && printf '_mtp' || printf '')

UCX_NET_DEVICES=eth0

# Cleanup any stale enroot locks from previous runs
find /var/cache/enroot-container-images/$UID -type f -name "*.lock" | xargs rm

set -x
srun --partition=$PARTITION --gres=gpu:$TP --exclusive --job-name="$RUNNER_NAME" \
--container-image=$IMAGE \
Expand Down