Skip to content

Int8 percolumn moe new #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
yych0745 wants to merge 2,843 commits into from
Closed

Int8 percolumn moe new #8

yych0745 wants to merge 2,843 commits into from

Conversation

@yych0745
Copy link
Collaborator

@yych0745 yych0745 commented Feb 19, 2025

Motivation

Modifications

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

BBuf and others added 30 commits February 6, 2025 16:42
@BBuf
clean moe align block kernel code and add acc test (sgl-project#3332)
4d5c49b
@saienduri @HaiShaw
Add sgl-kernel to MI300 CI paths tested. (sgl-project#3335)
200d3b1 
Co-authored-by: HAI <hixiao@gmail.com>
@saienduri @HaiShaw
Add sgl-kernel to MI300 CI paths tested. (sgl-project#3335)
0822879 
Co-authored-by: HAI <hixiao@gmail.com>
@zhyncs
update pull request template (sgl-project#3337)
25ed22b
@zhyncs
update pull request template (sgl-project#3337)
677a059
@zhyncs
add AMD guide for DeepSeek-R1 (sgl-project#3338)
7348d96
@zhyncs
add AMD guide for DeepSeek-R1 (sgl-project#3338)
824da15
@ispobock
[Doc] Add optimization option guide for deepseek v3 (sgl-project#3349)
6792411
@ispobock
[Doc] Add optimization option guide for deepseek v3 (sgl-project#3349)
8b683ac
@zhyncs
fix sgl-kernel build failure on AMD (sgl-project#3352)
adeee15
@zhyncs
fix sgl-kernel build failure on AMD (sgl-project#3352)
199e7ae
@BBuf
optimize moe_align_kernel cuda (sgl-project#3347)
cdae77b
@BBuf
optimize moe_align_kernel cuda (sgl-project#3347)
89d1d87
@zhaochenyang20
Docs/CI: Enable Fake Finish for Docs Only PR (sgl-project#3350)
76ca91d
@zhaochenyang20
Docs/CI: Enable Fake Finish for Docs Only PR (sgl-project#3350)
f08040a
@lycanlancelot @shuaills
@zhaochenyang20 @zhyncs
Docs: Add deepseek usage and add multi-node, credit to lycanlancelot (s…
823148e 
…gl-project#3314)

Co-authored-by: Shi Shuai <126407087+shuaills@users.noreply.github.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@zhaochenyang20
Feature: Fix the binding error in Llama (sgl-project#3355)
36186f7
@zhaochenyang20
Feature: Fix the binding error in Llama (sgl-project#3355)
40022d0
@shuaills
Fix: Runtime error for function calling (sgl-project#3300)
6a43429
@shuaills
Fix: Runtime error for function calling (sgl-project#3300)
591e751
@lizamd
update waves_per_eu to 1 (sgl-project#3356)
ecbb69c
@lizamd
update waves_per_eu to 1 (sgl-project#3356)
e868d0b
@zhyncs
update unit test in AMD CI (sgl-project#3366)
fecf3b6
@zhyncs
update unit test in AMD CI (sgl-project#3366)
2b1808c
@zhyncs
fix undefined symbol cudaGetDriverEntryPointByVersion (sgl-project#3372)
d393657
@zhyncs
fix undefined symbol cudaGetDriverEntryPointByVersion (sgl-project#3372)
45c87e0
@zhyncs @Ying1123
support speculative decoding kernel in sgl-kernel (sgl-project#3373)
049099b 
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
@zhyncs @Ying1123
support speculative decoding kernel in sgl-kernel (sgl-project#3373)
f9905d5 
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
@zhyncs
update sgl-kernel version (sgl-project#3374)
2106897
@zhyncs
update sgl-kernel version (sgl-project#3374)
f287037
zhyncs and others added 29 commits February 18, 2025 02:06
@zhyncs
feat: support flashinfer mla with prefix cache (sgl-project#3643)
714f3e6
@zhyncs
chore: update flashinfer v0.2.1.post2 (sgl-project#3644)
b0d7b41
@zhyncs
chore: update flashinfer v0.2.1.post2 (sgl-project#3644)
75d171a
@zhyncs @pankajroark
chore: bump v0.4.3.post2 (sgl-project#3645)
77ebb60 
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
@zhyncs @pankajroark
chore: bump v0.4.3.post2 (sgl-project#3645)
a5375ad 
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
@zhyncs
use transformers 4.48.3 (sgl-project#3650)
c0ab9e4
@zhyncs
use transformers 4.48.3 (sgl-project#3650)
058d199
@whchung @HaiShaw
[ROCm] Add additional block quant GEMM tuning configs for AMD GPUs. (s…
cd9a4fd 
…gl-project#3616)

Co-authored-by: HAI <hixiao@gmail.com>
@whchung @HaiShaw
[ROCm] Add additional block quant GEMM tuning configs for AMD GPUs. (s…
2eab113 
…gl-project#3616)

Co-authored-by: HAI <hixiao@gmail.com>
@BruceXcluding
[ROCm] Optimal MOE Tuning for AMD Radeon Graphics (sgl-project#3567)
4abff47
@BruceXcluding
[ROCm] Optimal MOE Tuning for AMD Radeon Graphics (sgl-project#3567)
ddf39d3
@whybeyoung
Docs: Deploy multi-node inference (LWS method) using sglang in a K8s …
e000278 
…cluster (sgl-project#3624)
@whybeyoung
Docs: Deploy multi-node inference (LWS method) using sglang in a K8s …
c51dc2c 
…cluster (sgl-project#3624)
@saienduri
Update amd docker image. (sgl-project#3654)
3bf24be
@saienduri
Update amd docker image. (sgl-project#3654)
522e18e
@zhyncs
fix sgl-project#3654
2178631
@zhyncs
fix sgl-project#3654
07ab4d4
@Fridge003
[Feature] Apply Cublas Grouped Gemm kernel (sgl-project#3629)
ea57279
@Fridge003
[Feature] Apply Cublas Grouped Gemm kernel (sgl-project#3629)
67fc595
@zhyncs
update pr-test (sgl-project#3663)
9b775eb
@zhyncs
update pr-test (sgl-project#3663)
f983213
@ispobock
Fix draft decode max batch size (sgl-project#3676)
2a9e5c4
@ispobock
Fix draft decode max batch size (sgl-project#3676)
e5ce395
@mickqian
fix: remove dependency on latest transformers impl (sgl-project#3635)
5890002
@mickqian
fix: remove dependency on latest transformers impl (sgl-project#3635)
424848d
@fsx950223 @AMD-dteng @HaiShaw
AMD Prefill optimize (sgl-project#3665)
96af564 
Co-authored-by: AMD-dteng <dteng@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
@fsx950223 @AMD-dteng @HaiShaw
AMD Prefill optimize (sgl-project#3665)
634a356 
Co-authored-by: AMD-dteng <dteng@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
@yych0745
support moe int8-w8a8 percolumn
978cd71
@yych0745
Merge main branch
0149f77
@yych0745 yych0745 closed this Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.