Skip to content

Conversation

@xiaoyu-work
Copy link
Collaborator

@xiaoyu-work xiaoyu-work commented Jan 6, 2026

Describe your changes

Add ONNX conversion pass support for diffusers model. The exported model will be a CompositeModel.

The model can be exported with CLI:

olive capture-onnx-graph -m "runwayml/stable-diffusion-v1-5" -o "sd15"

or config file with OnnxConversion pass.

The exported model can be loaded with from optimum.onnxruntime import ORTStableDiffusionPipeline directly.

import onnxruntime as ort
import torch
from pathlib import Path
from optimum.onnxruntime import ORTStableDiffusionPipeline
from diffusers import DiffusionPipeline, OnnxRuntimeModel

ort.set_default_logger_severity(3)

sess_options = ort.SessionOptions()
sess_options.enable_mem_pattern = False

model_id = "runwayml/stable-diffusion-v1-5"
onnx_model_path = Path("sd15")
provider = "CUDAExecutionProvider"
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float32)

vae_encoder_session = OnnxRuntimeModel.load_model(onnx_model_path / "vae_encoder" / "model.onnx", provider=provider)
vae_decoder_session = OnnxRuntimeModel.load_model(onnx_model_path / "vae_decoder" / "model.onnx", provider=provider)
text_encoder_session = OnnxRuntimeModel.load_model(onnx_model_path / "text_encoder" / "model.onnx", provider=provider)
unet_session = OnnxRuntimeModel.load_model(onnx_model_path / "unet" / "model.onnx", provider=provider)

onnx_pipeline = ORTStableDiffusionPipeline(
    vae_encoder_session=vae_encoder_session,
    vae_decoder_session=vae_decoder_session,
    text_encoder_session=text_encoder_session,
    unet_session=unet_session,
    tokenizer=pipeline.tokenizer,
    scheduler=pipeline.scheduler,
    feature_extractor=pipeline.feature_extractor,
    config=dict(pipeline.config),
)

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

@xiaoyu-work xiaoyu-work merged commit d7ccb50 into main Jan 7, 2026
11 checks passed
@xiaoyu-work xiaoyu-work deleted the xiaoyu/sd-conv branch January 7, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants