Skip to content

Conversation

@simonrosenberg
Copy link
Collaborator

@simonrosenberg simonrosenberg commented Jan 28, 2026

Summary

  • require --model-name for swebench and swebenchmultimodal converters; no default, raise if missing
  • clarify docstrings with expected model identifier examples and why it matters for attribution
  • adjust tests to supply a model name and assert missing-name error

Testing

  • uv run pytest tests/test_swebenchmultimodal.py -q

Co-authored-by: openhands <openhands@all-hands.dev>
@simonrosenberg
Copy link
Collaborator Author

simonrosenberg commented Jan 28, 2026

@juanmichelini I think model_name: str = "OpenHands" is a bit misleading since the real values are llm names so I figured we could make it mandatory

Co-authored-by: openhands <openhands@all-hands.dev>
@openhands-ai
Copy link

openhands-ai bot commented Jan 28, 2026

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Pre-commit checks

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #376 at branch `feature/require-model-name`

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

@simonrosenberg simonrosenberg self-assigned this Jan 28, 2026
@simonrosenberg simonrosenberg requested a review from neubig January 28, 2026 15:06
Co-authored-by: openhands <openhands@all-hands.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants