add testing for Qwen3.5-0.8B to GSM8K eval

[puririshi98]

part of #36264
adds testing for Qwen3.5-0.8B to GSM8K eval

[gemini-code-assist]

Code Review

This pull request introduces a new accuracy evaluation test for the deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B model. The test suite is comprehensive, leveraging lm-evaluation-harness to check for correctness across various precisions (bfloat16, float16, fp8) and on multiple benchmarks (gsm8k, arc_easy, hellaswag). It also includes sanity checks and a smoke test. My review found a couple of inconsistencies in the documentation within the new test file that should be addressed for clarity and correctness.

[gemini-code-assist]

The docstring states that this test covers both the 1.5B and 7B versions of the DeepSeek-R1-Distill-Qwen model. However, the SMALL_DEEPSEEK_MODELS list (line 149) only includes the configuration for the 1.5B model. Please either add the configuration for the 7B model to the test suite or update the docstring to accurately reflect the tested models.

[gemini-code-assist]

The file path in the Usage example is incorrect. It points to tests/lm_eval_correctness/test_deepseek_small_accuracy.py, but the actual file path is tests/v1/attention/test-qwen-deepseek-1.5-accuracy.py. This should be updated to reflect the correct path to make the test runnable as documented.

Suggested change

	pytest tests/lm_eval_correctness/test_deepseek_small_accuracy.py -v
	pytest tests/v1/attention/test-qwen-deepseek-1.5-accuracy.py -v

[mergify]

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mergify]

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[mergify]

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[jasonlizhengjian]

/gemini review

[gemini-code-assist]

Code Review

The pull request introduces a new configuration file for the Qwen3.5-0.8B model and updates the models-small.txt list to include it. The changes are straightforward and align with the stated objective of adding new model evaluations. No critical or high-severity issues were identified in the changes.

_{Note: Security Review is unavailable for this PR.}

[jasonlizhengjian]

LGTM , please change the title to reflect the contents. Also consider adding a more descriptive PR description