Skip to content

Commit ce7da6c

Browse files
sjarmakclaude
andcommitted
fix: remove unnecessary Firefox mirror clone from SR-QA verifiers
The sgonly_verifier_wrapper.sh was cloning sg-evals/firefox--871325b8 (~4-5GB even at depth 1) during verification for ccx-onboard-search-207 and -208. This took 39+ min on success and 3+ hours on failure, causing VerifierTimeoutError on MCP runs. The clone is unnecessary because SR-QA verification only does JSON string matching (function name + path). Removing the wrapper source reduces verifier time from 39+ min to <1 sec. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 0cf07e5 commit ce7da6c

File tree

2 files changed

+6
-8
lines changed
  • benchmarks/ccb_mcp_onboarding
    • ccx-onboard-search-207/tests
    • ccx-onboard-search-208/tests

2 files changed

+6
-8
lines changed

benchmarks/ccb_mcp_onboarding/ccx-onboard-search-207/tests/test.sh

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,9 @@
22
set -eo pipefail
33
# RepoQA SR-QA Verification Script
44

5-
# Source the sg_only wrapper (no-op if not in sg_only mode)
6-
if [ -f /tests/sgonly_verifier_wrapper.sh ]; then
7-
source /tests/sgonly_verifier_wrapper.sh
8-
fi
5+
# SR-QA tasks: verifier only does JSON string matching (function name + path).
6+
# sgonly_verifier_wrapper clone is NOT needed — skipped to avoid 39+ min
7+
# Firefox mirror clone that causes VerifierTimeoutError on MCP runs.
98

109
echo "Starting RepoQA verifier..." 1>&2
1110
cd /app || { echo "ERROR: Cannot cd to /app"; exit 1; }

benchmarks/ccb_mcp_onboarding/ccx-onboard-search-208/tests/test.sh

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,9 @@
22
set -eo pipefail
33
# RepoQA SR-QA Verification Script
44

5-
# Source the sg_only wrapper (no-op if not in sg_only mode)
6-
if [ -f /tests/sgonly_verifier_wrapper.sh ]; then
7-
source /tests/sgonly_verifier_wrapper.sh
8-
fi
5+
# SR-QA tasks: verifier only does JSON string matching (function name + path).
6+
# sgonly_verifier_wrapper clone is NOT needed — skipped to avoid 39+ min
7+
# Firefox mirror clone that causes VerifierTimeoutError on MCP runs.
98

109
echo "Starting RepoQA verifier..." 1>&2
1110
cd /app || { echo "ERROR: Cannot cd to /app"; exit 1; }

0 commit comments

Comments
 (0)