Skip to content

Detect and report which ort execution provider is active #7

@dmwyatt

Description

@dmwyatt

Problem

When GPU features (cuda, directml, coreml) are enabled in the binary, ort may silently fall back to CPU if the GPU provider fails to initialize. There's currently no way to tell the user which provider was actually selected.

This means:

  • Users may think they're getting GPU acceleration when they're not (the original concern in #127)
  • We can't log a useful warning on fallback because we can't detect it happened

Closes the gap left by closing #147, which proposed silent CPU fallback but was rejected because it would mask the problem.

Desired behavior

Surface which execution provider ort is actually using, so we can:

  1. Warn users if GPU was requested but CPU is being used
  2. Show the active provider in grans embed --stats or similar diagnostic output

Investigation needed

  • Can ort::Session report which execution provider it bound to?
  • Is there a callback or log hook in ort that fires when a provider fails to register?
  • If ort doesn't expose this, is there an upstream issue/feature request we could track?

Related: #127

Metadata

Metadata

Assignees

No one assigned

    Labels

    prio: p2Medium priority, normal queuesize: mMedium, a few hours to a daystatus: readyReady to be worked ontype: featureNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions