Skip to content

Commit c232d6d

Browse files
committed
docs: update README and remove ulysses_size from ep_fsdp_qwen3_moe.py
- Split combined "ep/sp FSDP MoE finetuning" entry in README into separate "ep FSDP MoE finetuning" and "sp FSDP finetuning" rows for clarity - Remove ulysses_size parameter from device_mesh initialization in ep_fsdp_qwen3_moe.py as it is no longer needed for the example configuration
1 parent c365198 commit c232d6d

File tree

2 files changed

+2
-3
lines changed

2 files changed

+2
-3
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,8 @@ pip install -e .
6969
| --------------------------------- | --------------- | ------------------------------------------------- |
7070
| FSDP finetuning | transformers | [Script](cookbook/transformers/fsdp2.py) |
7171
| FSDP MoE finetuning | transformers | [Script](cookbook/transformers/fsdp2_moe.py) |
72-
| ep/sp FSDP MoE finetuning | transformers | [Script](cookbook/transformers/ep_fsdp_qwen3_moe.py) |
72+
| ep FSDP MoE finetuning | transformers | [Script](cookbook/transformers/ep_fsdp_qwen3_moe.py)|
73+
| sp FSDP finetuning | transformers | [Script](cookbook/transformers/sp_fsdp_dense.py) |
7374
| EP MoE finetuning | transformers | [Script](cookbook/transformers/ep_fsdp_qwen3_moe.py) |
7475
| pp/tp/cp finetuning | megatron | [Script](cookbook/megatron/tp.py) |
7576
| pp/tp/cp MoE finetuning | megatron | [Script](cookbook/megatron/tp_moe.py) |

cookbook/transformers/ep_fsdp_qwen3_moe.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,13 +21,11 @@
2121
# 4 gpus, dp=2, ep=2
2222
dp_size = 2
2323
ep_size = 2
24-
ulysses_size = 2
2524

2625
device_mesh = DeviceMesh(
2726
device_type=Platform.get_platform().device_prefix(),
2827
mesh=np.arange(dp_size * ep_size).reshape(dp_size, ep_size),
2928
mesh_dim_names=('dp', 'ep'),
30-
ulysses_size=ulysses_size, # enable sp
3129
)
3230

3331
twinkle.initialize(

0 commit comments

Comments
 (0)