Skip to content

Commit bbbeddd

Browse files
authored
fix_contiguous (#96)
1 parent 468c7e2 commit bbbeddd

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/twinkle/model/megatron/model/gpts/qwen3_next.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -361,7 +361,7 @@ def _gated_delta_net_forward(self, hidden_states: torch.Tensor, **kwargs):
361361
res = res[attention_mask][:, None]
362362
res = torch.concat([res, res.new_zeros(seq_len - res.shape[0], 1, res.shape[2])])
363363
else:
364-
res = res.transpose(0, 1)
364+
res = res.transpose(0, 1).contiguous()
365365
if args.sequence_parallel and args.tensor_model_parallel_size > 1:
366366
res = reduce_scatter_to_sequence_parallel_region(res) / args.tensor_model_parallel_size
367367
return res, None

0 commit comments

Comments
 (0)