Apple Silicon - metal, mps, mlx support, m1, m2, m3, m4

[doreilly257]

Any chance you can include the capability to run on Apple Silicon M1/M2/M3/M4 processors with metal, mps, or mlx support.

[matatonic]

Perhaps, there are a lot of 'cuda' assumptions in many models, but I'll try to make it possible.

[matatonic]

Currently device=mps is possible, and if it's available it will automatically select it. Which model are you trying to use?

[yukiarimo]

Does it works with Pixtral and other Vision models on MacOS?

[matatonic]

Does it works with Pixtral and other Vision models on MacOS?

Sorry, but I don't know, I don't have an apple device to test with.

Has anyone else tried?