DSQLora: Quantization-Aware LoRA Fine-tuning Framework with Dynamic Sequence Adjustment

🎯 Project Overview

DSQ-Lora is an efficient quantization and fine-tuning framework for large language models, integrating the following advanced techniques:

• LoRA Fine-tuning: Parameter-efficient adaptation method for large models
• Quantization Technology: Model compression and inference acceleration (2-8bit)
• Dynamic Sequence Adjustment (DSQ): Gradient-based intelligent weight adjustment mechanism
• Multi-GPU Parallel Training: Support for distributed training and inference

The framework is specifically optimized for Qwen and Llama series models, supporting various benchmarks including PPL evaluation, HumanEval, Huatuo medical QA, and more.

---

🔧 System Architecture

Project Structure

dsqlora/
├── dsqlora.py              # Main training script
├── modelutils.py           # Model loading and utility functions
├── quant.py                # Quantization-related functions and classes
├── LoraPTQquantize.py      # LoRA quantization conversion module
├── gptq.py                 # GPTQ quantization algorithm
├── gradutils.py            # Gradient estimation tools
├── datautils.py            # Data loading utilities
├── alphaanlyse.py          # Alpha analysis tool
├── chechpoint.py           # Checkpoint management
├── peft/                   # PEFT library (Parameter-Efficient Fine-tuning)
│   ├── tuners/             # Various tuner implementations
│   │   ├── lora/           # LoRA implementation
│   │   ├── adalora/        # AdaLora implementation
│   │   ├── lokr/           # LoKr implementation
│   │   └── ...
│   └── utils/              # Utility functions
├── benchmark/              # Evaluation benchmarks
│   ├── runbenchmark.py     # Benchmark testing main script
│   ├── human_eval/         # HumanEval benchmark
│   ├── grade_school_math/  # Mathematical problem benchmark
│   └── huatuo.py           # Medical QA benchmark
└── datasetsutils/          # Dataset utilities
    ├── code_alpaca/        # Code Alpaca dataset
    ├── disclawsft/         # Disclaimer SFT dataset
    └── orca_math_word/     # Orca math dataset

Core Module Description

dsqlora.py - Main Training Entry

• Support for model loading (Qwen/Llama)
• Complete quantization fine-tuning pipeline implementation
• Command-line parameter configuration
• SwanLab monitoring integration

quant.py - Quantization Core

• Quantizer: Base quantizer supporting symmetric/asymmetric quantization
• Quant3Linear/Quant8Linear: Quantized Linear layers for various precisions
• Quantization functions and parameter search algorithms

LoraPTQquantize.py - Quantization Conversion

• LoRA weight to quantized weight conversion
• Support for multiple quantization configurations
• Quantized model saving and loading

peft/ - Parameter-Efficient Fine-tuning

• Multiple fine-tuning methods including LoRA, AdaLora, LoKr, etc.
• Dynamic configuration and model loading
• Integration with quantization framework

benchmark/ - Evaluation Framework

• Unified interface for multiple benchmarks
• Support for distributed inference evaluation
• Result recording and comparison

---

📦 Dependency Requirements

Core Dependencies

torch>=1.13.0
transformers==4.52.4
peft==0.10.0

GPU Acceleration Libraries

flash-attn==2.5.8
flash-mla==1.0.0.dev0
bitsandbytes==0.46.0
xformers==0.0.30

Training and Monitoring

pytorch-lightning==2.5.2
deepspeed==0.17.1
accelerate==1.6.0
swanlab

Evaluation Tools

rouge-score==0.1.2
sacrebleu==2.5.1
torchmetrics==1.7.3
deepeval==3.1.6

Other Utilities

numpy==1.24.1
pandas==2.3.0
tqdm
matplotlib

See requirements.txt file for complete dependencies.

---

🚀 Quick Start

1. Environment Setup

# Clone or copy the project
git clone <project-url>
cd dsqlora

# Create a Python virtual environment (recommended)
conda create -n dsqlora python=3.10
conda activate dsqlora

# Install dependencies
pip install -r requirements.txt

# Install CUDA extension (optional, but recommended for best performance)
# Requires CUDA toolkit and cuDNN
python setup.py build_ext --inplace

2. Basic Usage Example

# Basic training command
python dsqlora.py \
    /path/to/model \
    c4 \
    --model-type qwen \
    --act-order \
    --new-eval \
    --batchsize 8 \
    --benchmark ppl \
    --evaltype full \
    --learningrate 0.0005 \
    --lorar 64 \
    --wbits 8 \
    --evalbatchsize 128 \
    --epochs 3 \
    --lorainit loraqat \
    --nsamples 2048

3. Supported Datasets

Supported datasets:

• huatuo - Medical QA dataset
• code_alpaca - Code fine-tuning dataset
• orca_math_word - Mathematical problem dataset
• grade-school-math - Elementary school math dataset

---

💡 Advanced Usage

1. Multi-GPU Distributed Training

# Use all available GPUs
python dsqlora.py \
    /path/to/model \
    c4 \
    --model-type qwen \
    --wbits 8 \
    --batchsize 4 \
    --usedpa  # Enable BalanceDataParallel

2. Resume Training from Checkpoint

python dsqlora.py \
    /path/to/model \
    c4 \
    --model-type qwen \
    --resume_from ./outputs/checkpoint-500

3. Evaluation-Only Mode

python dsqlora.py \
    /path/to/model \
    c4 \
    --model-type qwen \
    --wbits 8 \
    --onlyeval \
    --evalmodeltype qatmodel \
    --modelweightpath ./outputs/adapter_model.safetensors \
    --evaltype full_and_quantize

4. Custom Training Parameters

python dsqlora.py \
    /path/to/model \
    c4 \
    --model-type qwen \
    --wbits 4 \
    --lorar 128 \
    --learningrate 5e-5 \
    --epochs 5 \
    --adjust_theta 0.05 \
    --lorainit loftq

---

📚 Key References

• LoRA: Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models"
• GPTQ: Frantar et al., "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers"
• LOFTQ: Li et al., "LQ-LoRA: Low-rank Quantization-Aware Training for Large Language Models"
• Flash Attention: Dao et al., "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness"

---

🤝 Contributing

We welcome issues, suggestions, and improvement proposals!

---

📄 License

Please refer to the license file in the project for detailed information.

---

📞 Contact

For questions or suggestions, please contact via:

• GitHub Issues
• Project Discussions

---

🎓 Related Resources

• PEFT Official Documentation
• Transformers Official Documentation
• PyTorch Official Documentation
• SwanLab Monitoring Platform

---

Project Status: 🚀 Actively Maintained