Skip to content

Adjust/Improve pipeline to be compatible with more models#4

Draft
yuqiannemo wants to merge 4 commits intoXtra-Computing:devfrom
yuqiannemo:dna_dev/run_more_models
Draft

Adjust/Improve pipeline to be compatible with more models#4
yuqiannemo wants to merge 4 commits intoXtra-Computing:devfrom
yuqiannemo:dna_dev/run_more_models

Conversation

@yuqiannemo
Copy link
Contributor

  1. Add more dependencies to support more models

@JerryLife
Copy link
Collaborator

Thanks for working on broader model compatibility here. Supporting more HF model families is a good direction, and the ModelLoader change is addressing a real misclassification issue.

That said, I don't think this is ready to merge as-is because the dependency changes would make the base package less portable.

Moving these packages into project.dependencies introduces hard platform requirements for every pip install llm-dna user, even if they never use those model families:

  • mlx and mlx-lm are Apple Silicon-specific — this will break installation on Linux/Windows, which is where most of our users run the package
  • optimum, compressed-tensors, etc. are architecture-specific/transitive runtime deps rather than core package requirements

I think the right direction is:

  • Keep the base install minimal
  • Move platform/model-specific packages into optional dependency groups (e.g. apple, quantization)
  • Document which extras are needed for which model families

On ModelLoader, the current fix works for openai/gpt-oss, but it's brittle because it carves out exceptions from a broad openai/gpt- rule — every new HuggingFace model under the openai/ namespace would need another exclusion. Instead, I'd suggest tightening the OpenRouter prefixes to be more specific:

openrouter_prefixes = [
    "openrouter/",
    "openrouter:",
    "anthropic/claude-",
    "deepseek/",
    "openai/gpt-3",
    "openai/gpt-4",
    "google/gemini-",
    "x-ai/grok-",
    "cohere/command",
    "perplexity/",
]

This way openai/gpt-3.5-turbo and openai/gpt-4o still route to OpenRouter, but openai/gpt-oss-* naturally falls through to HuggingFace without needing any exclusion logic.

One smaller process note: the PR description looks incomplete (2. is empty). Please expand that before merge so the packaging/dependency rationale is clear.

Net: I like the goal of the PR, but the dependency strategy needs to change before this can be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants