Skip to content

support dynamic model overrides #50

@mikestaub

Description

@mikestaub

As a user, I want to control the chat configuration dynamically.

Depending on the machine specs, the sliding_window_size and attention_sink_size may be needed to avoid a 'max window length exceeded' error for example.

Reference: https://github.com/mlc-ai/web-llm/blob/main/src/config.ts#L66

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions