evals: Add rubric-based StrongREJECT

[tomtseng]

Changes

We ran the rubric-based StrongREJECT for the paper but never committed it. Here's Claude Code's attempt at an implementation

Probably I should close this PR though, I'm assuming Saad has an implementation he used for the StrongREJECT rubric results

• Integrate the rubric-based StrongREJECT evaluator (strongreject_rubric from the strong_reject package) as the canonical StrongRejectEvaluation, registered under EvalName.STRONG_REJECT
• Rename the existing fine-tuned classifier to StrongRejectFinetunedEvaluation (EvalName.STRONG_REJECT_FINETUNED) — the fine-tuned model was trained on labels from the rubric judge, so the rubric version is the more authoritative
  scorer
• Extract shared inference/prompt-loading logic into _StrongRejectBaseEvaluation to avoid duplication between the two variants

Testing

• pytest tests/evals/test_strong_reject.py — rubric test mocks the API so no OpenAI key needed; finetuned test requires GPU
• Verify attack tests still reference STRONG_REJECT_FINETUNED correctly (all are @pytest.mark.expensive / GPU-only)
• Verify scripts/validate_references.py --list still prints StrongREJECT references

[sdhossain]

@tomtseng in case if it is of any help, here is the code I used for strongreject rubric: https://github.com/criticalml-uw/TamperBench/tree/sh/cais_checkpoint/scripts/strong_reject_rubric_backup

The main two files of concern are:

• https://github.com/criticalml-uw/TamperBench/blob/sh/cais_checkpoint/scripts/strong_reject_rubric_backup/rubric_utils.py
• https://github.com/criticalml-uw/TamperBench/blob/sh/cais_checkpoint/scripts/strong_reject_rubric_backup/evaluate_rubric.py

But there are other files in that directory that are relevant (like converting the embedding attack to also use strongreject, etc. which isn't an extremely elegant implementation, but kind of stand-alone scripts)

[sdhossain]

idea: maybe we can have them as same names, but different metrics?

right now, the compute_inferences step is shared between them, so we just need to add the additional metrics.

if they shared the same name -- perhaps (and this is untested), it would directly load the inferences from either one is already computed -- but for the metrics, we could check in compute_scores if the relevant metrics exist.

[sdhossain]

we could for example check here (And vice-versa for the other, if we had different metric names)