Skip to content

Curious - would this run faster using some of vllm-project/vllm? #13

@salamanders

Description

@salamanders

I'm excited to try this project (it actually might run on my GPU!)
I actually found this project by bouncing through a news article, to vllm-project/vllm, and then after I didn't have the hardware for it, searching and finding this project.

Do you think this could run faster using ideas from vllm-project/vllm?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions