-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
prio: p2Medium priority, normal queueMedium priority, normal queuesize: mMedium, a few hours to a dayMedium, a few hours to a daystatus: readyReady to be worked onReady to be worked ontype: featureNew feature or requestNew feature or request
Description
Problem
When GPU features (cuda, directml, coreml) are enabled in the binary, ort may silently fall back to CPU if the GPU provider fails to initialize. There's currently no way to tell the user which provider was actually selected.
This means:
- Users may think they're getting GPU acceleration when they're not (the original concern in #127)
- We can't log a useful warning on fallback because we can't detect it happened
Closes the gap left by closing #147, which proposed silent CPU fallback but was rejected because it would mask the problem.
Desired behavior
Surface which execution provider ort is actually using, so we can:
- Warn users if GPU was requested but CPU is being used
- Show the active provider in
grans embed --statsor similar diagnostic output
Investigation needed
- Can
ort::Sessionreport which execution provider it bound to? - Is there a callback or log hook in ort that fires when a provider fails to register?
- If ort doesn't expose this, is there an upstream issue/feature request we could track?
Related: #127
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
prio: p2Medium priority, normal queueMedium priority, normal queuesize: mMedium, a few hours to a dayMedium, a few hours to a daystatus: readyReady to be worked onReady to be worked ontype: featureNew feature or requestNew feature or request