Feature request: benchmark/tests

[Omar-afk801]

This is a feature most similar apps have, you can simply choose a model, choose the backend, configure the hyper parameters and the prefill and decode amount in tokens and start a test for the specific model on the specific device, it usually takes a couple minutes for larger models, simple.
After the test is done, it should display the average tokens per second (decode) and prefill tokens per second, this should work well if the device/ram detection feature which I requested before would be implemented.

[timmyy123]

This is a feature most similar apps have, you can simply choose a model, choose the backend, configure the hyper parameters and the prefill and decode amount in tokens and start a test for the specific model on the specific device, it usually takes a couple minutes for larger models, simple.
After the test is done, it should display the average tokens per second (decode) and prefill tokens per second, this should work well if the device/ram detection feature which I requested before would be implemented.

Can be done, but that would require a cloud database for storing/comparing the benchmark scores and ideally I don't want any cloud server

[Omar-afk801]

This is a feature most similar apps have, you can simply choose a model, choose the backend, configure the hyper parameters and the prefill and decode amount in tokens and start a test for the specific model on the specific device, it usually takes a couple minutes for larger models, simple.
After the test is done, it should display the average tokens per second (decode) and prefill tokens per second, this should work well if the device/ram detection feature which I requested before would be implemented.

Can be done, but that would require a cloud database for storing/comparing the benchmark scores and ideally I don't want any cloud server

My base idea was just to benchmark model performance on phones, I don't think adding some sort of leaderboard is actually needed here, it's just useful for measuring device performance in general not necessarily comparing.