Skip to content

Phase 2: replace dict-style quant_config access with LayerQuantSpec a…#274

Draft
thpereir wants to merge 1 commit intothpereir/quark_quant_layerfrom
thpereir/quark_quant_layer_phase2
Draft

Phase 2: replace dict-style quant_config access with LayerQuantSpec a…#274
thpereir wants to merge 1 commit intothpereir/quark_quant_layerfrom
thpereir/quark_quant_layer_phase2

Conversation

@thpereir
Copy link
Contributor

@thpereir thpereir commented Mar 6, 2026

…cross all models and ops

Motivation

Following up DeepSeek R1 enablement with per layer quantization support this converts all other models to use the same structured quantization config instead of a dict.

This PR depends on both #236 and #268

Technical Details

  1. Clean type boundary — LayerQuantSpec is a frozen dataclass; impossible to accidentally mutate or misspell fields.
  2. Single resolution API — resolve(prefix) handles exclude-list, per-layer overrides, pattern matching, and global fallback in one call with documented priority. Less error prone than previous approach of accessing the dict. What was quant_config.get_layer_quant_config(f"{prefix}.fused_qkv_a_proj") becomes quant_config.resolve(prefix)
  3. Parser registry — @register_quant_parser("quark") means new quantization formats are self-contained modules. In the current approach adding a new quantization method means editing QuantizationConfig.__init__ directly.

Test Plan

Test Result

Submission Checklist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant