Does this work with eGPU?

[thepythonnoobdude]

First, I appreciate your hard work on this!

Was wondering if this works with an eGPU? My PC has a gpu connected internally via standard PCIe. I have a second graphics card I could use as an eGPU. Wondering if raylight would work and use both GPUs. Thank you for your time!

[komikndr]

I’m sorry, it’s because:

Short answer: No.
Long answer:

1. I haven’t seen anyone test NCCL using a USB eGPU device connected to PCIe. I don’t even know whether a USB eGPU is counted as part of the PCIe root complex (where all PCIe devices are connected).
2. Even if it works, it will be very slow. From what I know about eGPUs, games already suffer when using them compared to direct PCIe and games aren’t even communication heavy workloads. Raylight definitely is.

[thepythonnoobdude]

Thank you for your reply! No worries! Considering giving it a try this week to hook up an egpu and see how it goes. Will comment here if I do and what the results are.

There's been a lot of redditors reporting that egpus surprisingly take minimal performance hit when doing AI Image/LLM tasks compared to gaming on an eGPU. I wouldn't think that could be true but those reddit comments were what inspired this whole question.

Thank you again for your time, these nodes are still such a huge contribution to open source AI and so awesome of you to make them!

[komikndr]

There's been a lot of redditors reporting that egpus surprisingly take minimal performance hit when doing AI Image/LLM tasks compared to gaming on an eGPU. I wouldn't think that could be true but those reddit comments were what inspired this whole question.

they are correct if 2 things are applied:

1. Only single GPU
2. Multiple GPUs but only work as data parallel inference, so it basically an independent work unit

cpu only sent like at most 1mb of latent tensor and few KB of text encoder output tensor to the GPU.

Anyway happy testing