Skip to content

fault in qwen3.py #69

@Along-whu

Description

@Along-whu

packed_module_mapping = {
"q_proj": ('q_proj', 'q'),
"k_proj": ('k_proj', 'k'),
"v_proj": ('v_proj', 'v'),
"gate_up": ('gate_up_proj', '0'),
"gate_down": ('gate_down_proj', '1'),
}
in class Qwen3ForCausalLM(nn.Module), I think this is fault: ('q_proj', 'q'), ('k_proj', 'k'),('v_proj', 'v').

The weight name is "qkv_projection".

packed_module_mapping = {
"q_proj": ('qkv_projection', 'q'),
"k_proj": ('qkv_projection', 'k'),
"v_proj": ('qkv_projection', 'v'),
"gate_up": ('gate_up', '0'),
"gate_down": ('gate_up', '1'),
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions