Does AutoGGUF support quantization for SD3 or Flux and other DIT drawing models?

[Amazon90]

Your GUI tool is very beginner-friendly for those new to model quantization, but I’m unsure if your tool supports quantizing drawing models. I hope you can provide an answer.

https://github.com/city96/ComfyUI-GGUF/blob/main/tools/README.md

[leafspark]

It should support quantizing DIT models like SD3 and Flux. Here's how:

1. Convert your safetensors model to GGUF format:

python convert.py --src model.safetensors

2. Build and install the patched llama.cpp version and copy its binaries into a new folder in the llama_bin folder in your AutoGGUF directory.

3. Launch AutoGGUF - the new folder should appear in the backend selection.

4. Quantize normally, but:

   • Skip text model parameters
   • Use the Extra Arguments box for vision model settings

If there's prebuilt binaries in a repo you'd like to use, you could also set AUTOGGUF_BACKEND_REPO to owner/repo to pull new releases from that repository instead (although the binaries have to be in one zip file, same as the llama.cpp repo).

[Amazon90]

It should support quantizing DIT models like SD3 and Flux. Here's how:

1. Convert your safetensors model to GGUF format:

python convert.py --src model.safetensors

2. Build and install the patched llama.cpp version and copy its binaries into a new folder in the llama_bin folder in your AutoGGUF directory.

3. Launch AutoGGUF - the new folder should appear in the backend selection.

4. Quantize normally, but:

   • Skip text model parameters
   • Use the Extra Arguments box for vision model settings

If there's prebuilt binaries in a repo you'd like to use, you could also set AUTOGGUF_BACKEND_REPO to owner/repo to pull new releases from that repository instead (although the binaries have to be in one zip file, same as the llama.cpp repo).

Could you be more specific? This is my first time dealing with model quantization. Which version of lama.cpp should I download?

[xdong189]

Has it been done? I also want to know how to convert these safetensors into GGUF, preferably through a single interface...