feat: Apple Silicon (MLX) support for query expansion fine-tuning

[dgilperez]

Summary

Port of the query expansion fine-tuning pipeline to Apple Silicon using MLX.

• SFT training with LoRA on Qwen3-1.7B
• GRPO training (reinforcement learning refinement)
• 100% local - no cloud GPU needed
• Works on M1/M2/M3/M4 Macs

Observation

When testing, we noticed the current published model outputs placeholder text for hyde (see #75):

This is an example of a hypothetical document passage that would answer the e...

Our retrained model (same dataset, same base model) produces actual contextual content:

hyde: To deploy to Kubernetes, first install the required dependencies...
lex: docker kubernetes deployment setup
vec: how to deploy an application to a Kubernetes cluster

Questions before proceeding

We wanted to raise these questions rather than assume:

1. Model weights: Should we share just the training method, or also publish trained GGUF weights?
2. Repository structure: Should finetune-mlx/ live alongside finetune/ in this repo, or would you prefer a separate repository?
3. Hyde placeholder issue: Do you know why the current published model outputs placeholder hyde text? Was this intentional, or an early checkpoint?
4. Base model: We matched your config (Qwen3-1.7B, 3000 iters, 2e-4 LR). Is there flexibility here or should we stick to this?

What's included

finetune-mlx/
├── train.py          # SFT training
├── grpo.py           # GRPO (RL) training  
├── eval.py           # Evaluation
├── reward.py         # Scoring function
├── convert.py        # GGUF conversion helper
├── demo.py           # Quick test script
├── configs/sft.yaml  # Training config
└── tests/            # Unit tests

Training artifacts (adapters/, models/, exports/) are gitignored.

Contributors

• @sujito00
• @dgilperez

[tobi]

that is pretty damn cool

[dgilperez]

Thank you 😊 !!

Just re-verified the training after syncing reward.py with upstream. 3000 iters, ~15 min on M3 Max, final loss 0.13.

For the open questions: I matched your config roughly (Qwen3-1.7B, 3000 iters, 2e-4 LR, LoRA rank 16 and batch size) and kept finetune-mlx/ alongside finetune/. Happy to publish the GGUF weights if useful.

I'm also happy to keep this as a separate repo or maintain it here if needed