Skip to content

Rebase to latest vLLM#3

Merged
wangxiyuan merged 48 commits intocosdt:apply_pluginfrom
MengqingCao:apply_plugin0120
Jan 21, 2025
Merged

Rebase to latest vLLM#3
wangxiyuan merged 48 commits intocosdt:apply_pluginfrom
MengqingCao:apply_plugin0120

Conversation

@MengqingCao
Copy link

@MengqingCao MengqingCao commented Jan 20, 2025 •
edited by github-actions bot
Loading 

* quickfix: using Torch SPDA when platform is out of tree

* Merge branch pr#11324 into apply_plugin

youkaichao and others added 30 commits January 16, 2025 19:58
@youkaichao
Support torchrun and SPMD-style offline inference (vllm-project#12071)
bf53e0c 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[core] LLM.collective_rpc interface and RLHF example (vllm-project#12084
92e793d 
)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@ywang96
[Bugfix] Fix max image feature size for Llava-one-vision (vllm-projec…
874f7c2 
…t#12104)

Signed-off-by: Roger Wang <ywang@roblox.com>
@varun-sundar-rabindranath
[misc] Add LoRA kernel micro benchmarks (vllm-project#11579)
5fd24ec
@Isotr0py
[Model] Add support for deepseek-vl2-tiny model (vllm-project#12068)
62b06ba 
Signed-off-by: Isotr0py <2037008807@qq.com>
@heheda12345
[Bugfix] Set enforce_eager automatically for mllama (vllm-project#12127)
d06e824 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
@KuntaiDu
[Bugfix] Fix a path bug in disaggregated prefill example script. (vll…
ebc73f2 
…m-project#12121)

Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
@jikunshang
[CI]add genai-perf benchmark in nightly benchmark (vllm-project#10704)
fead53b 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
@terrytangyuan
[Doc] Add instructions on using Podman when SELinux is active (vllm-p…
1475847 
…roject#12136)

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
@terrytangyuan
[Bugfix] Fix issues in CPU build Dockerfile (vllm-project#12135)
b8bfa46 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
@heheda12345
[BugFix] add more is not None check in VllmConfig.__post_init__ (vl…
d1adb9b 
…lm-project#12138)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
@Isotr0py
[Misc] Add deepseek_vl2 chat template (vllm-project#12143)
d75ab55 
Signed-off-by: Isotr0py <2037008807@qq.com>
@divakar-amd
[ROCm][MoE] moe tuning support for rocm (vllm-project#12049)
8027a72 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
@heheda12345 @comaniac
[V1] Move more control of kv cache initialization from model_executor…
69d765f 
… to EngineCore (vllm-project#11960)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
@jeejeelee
[Misc][LoRA] Improve the readability of LoRA error messages (vllm-pro…
07934cc 
…ject#12102)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
@bigPYJ1151
[CI/Build][CPU][Bugfix] Fix CPU CI (vllm-project#12150)
d4e6194 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
@youkaichao
[core] allow callable in collective_rpc (vllm-project#12151)
87a0c07 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@wallashss
[Bugfix] Fix score api for missing max_model_len validation (vllm-pro…
58fd57f 
…ject#12119)

Signed-off-by: Wallas Santos <wallashss@ibm.com>
@jikunshang
[Bugfix] Mistral tokenizer encode accept list of str (vllm-project#12149
54cacf0 
)

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
@gshtras
[AMD][FP8] Using MI300 FP8 format on ROCm for block_quant (vllm-proje…
b5b57e3 
…ct#12134)

Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
@youkaichao
[torch.compile] disable logging when cache is disabled (vllm-project#…
7b98a65 
…12043)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[misc] fix cross-node TP (vllm-project#12166)
2b83503 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@hongxiayang
[AMD][CI/Build][Bugfix] use pytorch stale wheel (vllm-project#12172)
c09503d 
Signed-off-by: hongxyan <hongxyan@amd.com>
@youkaichao
[core] further polish memory profiling (vllm-project#12126)
da02cb4 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@russellb
[Docs] Fix broken link in SECURITY.md (vllm-project#12175)
813f249 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@Isotr0py
[Model] Port deepseek-vl2 processor, remove dependency (vllm-project#…
02798ec 
…12169)

Signed-off-by: Isotr0py <2037008807@qq.com>
@youkaichao
[core] clean up executor class hierarchy between v1 and v0 (vllm-proj…
6d0e3d3 
…ect#12171)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@ice-tong
[Misc] Support register quantization method out-of-tree (vllm-project…
32eb0da 
…#11969)
@simon-mo
[V1] Collect env var for usage stats (vllm-project#12115)
7a8a48d
@madamczyk-intel
[BUGFIX] Move scores to float32 in case of running xgrammar on cpu (v…
4e94951 
…llm-project#12152)

Signed-off-by: Michal Adamczyk <madamczyk@habana.ai>
DarkLight1337 and others added 16 commits January 18, 2025 19:16
@DarkLight1337
[Bugfix] Fix multi-modal processors for transformers 4.48 (vllm-proje…
630eb5b 
…ct#12187)
@youkaichao
[torch.compile] store inductor compiled Python file (vllm-project#12182)
e66faf4 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@gujingit @ywang96
benchmark_serving support --served-model-name param (vllm-project#12109)
936db11 
Signed-off-by: zibai <zibai.gj@alibaba-inc.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
@Isotr0py
[Misc] Add BNB support to GLM4-V model (vllm-project#12184)
edaae19 
Signed-off-by: Isotr0py <2037008807@qq.com>
@ywang96 @imkero @DarkLight1337
[V1] Add V1 support of Qwen2-VL (vllm-project#12128)
81763c5 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
@MartinGleize
[Model] Support for fairseq2 Llama (vllm-project#11442)
bbe5f9d 
Signed-off-by: Martin Gleize <mgleize@meta.com>
Co-authored-by: mgleize user <mgleize@a100-st-p4de24xlarge-4.fair-a100.hpcaas>
@ShangmingCai
[Bugfix] Fix num_heads value for simple connector when tp enabled (vl…
df450aa 
…lm-project#12074)

Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
@youkaichao
[torch.compile] fix sym_tensor_indices (vllm-project#12191)
51ef828 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@hmellor
Move linting to pre-commit (vllm-project#11975)
3ea7b94 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@terrytangyuan
[DOC] Fix typo in docstring and assert message (vllm-project#12194)
c5c0620 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
@terrytangyuan
[DOC] Add missing docstring in LLMEngine.add_request() (vllm-project#…
d264312 
…12195)

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
@terrytangyuan
[Bugfix] Fix incorrect types in LayerwiseProfileResults (vllm-project…
0974c9b 
…#12196)

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
@Isotr0py
[Model] Add Qwen2 PRM model support (vllm-project#12202)
8360979 
Signed-off-by: Isotr0py <2037008807@qq.com>
@DarkLight1337
[Core] Interface for accessing model from VllmRunner (vllm-project#…
59a0192 
…10353)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao
[misc] add placeholder format.sh (vllm-project#12206)
5c89a29 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@DarkLight1337
[CI/Build] Remove dummy CI steps (vllm-project#12208)
4001ea1 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@github-actions
Copy link

github-actions bot commented Jan 20, 2025

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@MengqingCao
Merge NPU related pr to apply_plugin
bfa00ad 
    * quickfix: using Torch SPDA when platform is out of tree

    * Merge branch pr#11324 into apply_plugin

Signed-off-by: <maomaoyu870@gmail.com>
Signed-off-by: MengqingCao <cmq0113@163.com>
@wangxiyuan wangxiyuan merged this pull request into cosdt:apply_plugin Jan 21, 2025
2 checks passed
@MengqingCao MengqingCao deleted the apply_plugin0120 branch February 25, 2025 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.