Skip to content

Tournament model args#42

Merged
ianbulovic merged 2 commits intomainfrom
tournament-model-args
Jul 28, 2025
Merged

Tournament model args#42
ianbulovic merged 2 commits intomainfrom
tournament-model-args

Conversation

@ianbulovic
Copy link
Collaborator

@ianbulovic ianbulovic commented Jul 28, 2025

Adds support for passing a Model instance to the Tournament constructor instead of just a string. This allows the user to pass a model with custom model arguments. Also adds a new -M option to the tournament CLI for passing model arguments. For example with the CLI:

cnlp_llm tournament \
    path/to/data.tsv \
    path/to/prompt.prompt \
    --task ... \
    --pos-label ... \
    --model hf/meta-llama/Llama-3.1-8B-Instruct \
    --rounds 10 \
    --scheduler graph \
    -M "torch_dtype=auto" \
    -M "attn_implementation=flash_attention_2" \
    -M "trust_remote_code=True"

Or with the Tournament API:

from inspect_ai.model import get_model

model = get_model(
    "hf/meta-llama/Llama-3.1-8B-Instruct", 
    torch_dtype="auto", 
    attn_implementation="flash_attention_2",
    trust_remote_code=True,
)

tournament = Tournament(
    model=model,
    # ... other args
)

Resolves #41.

@ianbulovic ianbulovic requested a review from wonjininfo July 28, 2025 13:57
comparison_prompt = ComparisonPrompt.from_file(comparison_prompt_file)
tournament = Tournament(
model=model_name,
model=get_model(model_name, **parse_cli_args(m)),
Copy link
Member

@wonjininfo wonjininfo Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome. I didn't know that I can do this. 👍 (parse_cli_args function)

Copy link
Member

@wonjininfo wonjininfo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks a lot!!

@ianbulovic ianbulovic merged commit a91c55c into main Jul 28, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for model_args in Tournament

2 participants