Safe Output Health Report - 2026-04-01

[github-actions[bot]]

Executive Summary

• Period: Last 24 hours (2026-04-01)
• Runs Analyzed: 100
• Workflows Active: ~25 distinct workflows
• Safe Output Jobs Executed: 12 (10 success, 2 failure, 89 skipped)
• Total Safe Output Messages Completed: 38 items
• Safe Output Job Failure Rate: 16.7% (2/12 jobs failed)
• Error Clusters Identified: 1

Both failures occurred in the Smoke Claude workflow and share the same root cause: the agent generates run-specific filenames for push tests, but the push_to_pull_request_branch config only allows .github/smoke-claude-push-test.md.

---

Safe Output Job Statistics

Job Type	Executions (jobs)	Failures	Success Rate
safe_outputs (all workflows)	12	2	83.3%
push_to_pull_request_branch	2	2	0%
All other message types	~36 messages	0	100%

Message-level breakdown (from completed safe-output-items):

Message Type	Completed
add_comment	8
create_pull_request_review_comment	6
create_discussion	4
submit_pull_request_review	3
add_labels	2
post_slack_message	2
update_pull_request	2
add_reviewer	2
create_issue	2
set_issue_type	2
dispatch_workflow	2
send_slack_message	2
remove_labels	1

---

Error Clusters

Cluster 1: push_to_pull_request_branch — Files Outside Allowed-Files List

• Count: 2 occurrences (runs §23847925954 and §23848854083)
• Affected Jobs: push_to_pull_request_branch
• Affected Workflows: Smoke Claude (both runs)
• Sample Errors:

  Cannot push to pull request branch: patch modifies files outside the allowed-files list
  (smoke-test-pr-push-23847925954.smoke). Add the files to the allowed-files configuration
  field or remove them from the patch.
  
  Cannot push to pull request branch: patch modifies files outside the allowed-files list
  (test-smoke-23848854083.json). Add the files to the allowed-files configuration field
  or remove them from the patch.
  

• Root Cause: The Smoke Claude agent generates run-specific test files (.smoke, .json) to verify push capability, but the push_to_pull_request_branch handler config restricts pushes to only allowed_files: [".github/smoke-claude-push-test.md"]. The filenames differ across runs.
• Cascading Impact: Each push_to_pull_request_branch failure cancels all subsequent safe output messages:
  • Run 23847925954: create_issue + add_labels cancelled (2 messages)
  • Run 23848854083: create_issue cancelled (1 message)

Cascading Cancellation Details

Run 23847925954 — 11 messages total, 6 succeeded before failure:

• ✅ post_slack_message
• ✅ update_pull_request
• ✅ create_pull_request_review_comment (×2)
• ✅ submit_pull_request_review
• ✅ add_reviewer
• ❌ push_to_pull_request_branch (FAILED)
• ⏭ create_issue (CANCELLED)
• ⏭ add_labels (CANCELLED)
• ✅ add_comment (×2 — processed before the push)

Run 23848854083 — 9 messages total, 7 succeeded before failure:

• ✅ add_comment
• ✅ update_pull_request
• ✅ create_pull_request_review_comment (×2)
• ✅ submit_pull_request_review
• ✅ add_reviewer
• ✅ post_slack_message
• ❌ push_to_pull_request_branch (FAILED)
• ⏭ create_issue (CANCELLED)

---

Root Cause Analysis

Config/Agent Mismatch in Smoke Claude Push Tests

The Smoke Claude workflow is designed to test the end-to-end push_to_pull_request_branch safe output flow. However, there is a mismatch between:

1. What the config allows: allowed_files: [".github/smoke-claude-push-test.md"]
2. What the agent pushes: Run-specific files like smoke-test-pr-push-<run_id>.smoke and test-smoke-<run_id>.json

The agent appears to be generating ephemeral test files with run-specific names rather than using the whitelisted fixed filename. This is a new pattern first observed today (not seen in previous audits).

Distinct from 2026-03-31 issue: The previous push_to_pull_request_branch failure was a "missing remote branch" error (fatal: couldn't find remote ref pr-branch). Today's pattern is a different enforcement — the branch exists, but the file contents are blocked by the allowed-files filter.

---

Recommendations

Critical Issues (Immediate Action Required)

None — the failures are contained to Smoke Claude test runs and do not affect production workflows. Other safe output messages continue to process successfully before the push failure.

Bug Fixes Required

Fix 1: Smoke Claude Agent Prompt — Use Whitelisted Filename for Push Tests

• Priority: Medium
• File/Location: Smoke Claude workflow prompt / agent instructions
• Problem: Agent generates run-specific filenames (smoke-test-pr-push-<run_id>.smoke, test-smoke-<run_id>.json) for push tests instead of the configured allowed file .github/smoke-claude-push-test.md
• Fix: Update the Smoke Claude agent prompt to explicitly instruct it to create .github/smoke-claude-push-test.md when testing push_to_pull_request_branch
• Affected Jobs: push_to_pull_request_branch

Fix 2 (Alternative): Update allowed_files Config to Accept Test Files

• Priority: Low
• File/Location: Smoke Claude workflow push_to_pull_request_branch handler config
• Problem: The allowed_files list is too narrow for smoke tests that generate dynamic filenames
• Fix: Add a wildcard or pattern to allowed_files to accommodate smoke test files (e.g., smoke-test-*.smoke, test-smoke-*.json), OR add a dedicated smoke test directory
• Note: This approach is less preferred as it broadens the allowed files surface area

---

Work Item Plans

Work Item 1: Fix Smoke Claude Push Test Filename

• Type: Bug Fix
• Priority: Medium
• Description: The Smoke Claude agent creates run-specific files for push tests, but the push_to_pull_request_branch config only whitelists .github/smoke-claude-push-test.md. This causes 100% failure rate on push tests.
• Acceptance Criteria:
  • Smoke Claude push_to_pull_request_branch succeeds in smoke test runs
  • Agent consistently uses .github/smoke-claude-push-test.md as the push test file, OR allowed_files is updated to include the generated pattern
  • No cascading cancellations of create_issue / add_labels due to push failures
• Technical Approach: Either update the agent prompt to specify the exact filename, or update the config to add a pattern match. Recommend updating the agent prompt as the primary fix.
• Estimated Effort: Small
• Dependencies: None

---

Historical Context

Date	Total Runs	Safe Output Job Failures	Dominant Error Pattern
2026-03-29	42	4	invalid_event_context (target=triggering on scheduled runs), protected-files-pr-block, upload-assets-missing-setup
2026-03-30	127	5	invalid_event_context (recurring), add_labels_no_issue_number
2026-03-31	67	1	push_to_pull_request_branch: missing remote branch
2026-04-01	100	2	push_to_pull_request_branch: files outside allowed-files list (new)

Trends

• Error rate trend: Improving overall (4 → 5 → 1 → 2), but a new pattern emerged today
• Recurring issues resolved: invalid_event_context and add_labels_no_issue_number have not recurred since 2026-03-30
• New pattern today: push_to_pull_request_branch failing due to filename mismatch in Smoke Claude (2 occurrences)
• Previous push issue resolved: The "missing remote branch" pattern (2026-03-31) did not recur today

---

Metrics and KPIs

• Overall Safe Output Job Success Rate: 83.3% (10/12 jobs)
• Most Reliable Job Type: add_comment, create_discussion, create_pull_request_review_comment — 100% success
• Most Problematic Job Type: push_to_pull_request_branch — 0% success (2/2 failures, both Smoke Claude)
• Messages Successfully Delivered: 38 completed items across 10 successful runs

---

Next Steps

• Investigate Smoke Claude agent prompt to fix push_to_pull_request_branch filename mismatch
• Monitor whether the "missing remote branch" issue from 2026-03-31 recurs in Smoke Update Cross-Repo PR
• Verify Smoke Create Cross-Repo PR and Smoke Gemini agent failures (out of scope, handled by other monitors)

---

References:

• §23847925954 — Smoke Claude (push_to_pull_request_branch failure rejig docs #1)
• §23848854083 — Smoke Claude (push_to_pull_request_branch failure Add workflow: githubnext/agentics/weekly-research #2)
• §23850310432 — This audit run

AI generated by Safe Output Health Monitor · history

• expires on Apr 2, 2026, 1:22 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Safe Output Health Monitor.

A newer discussion is available at Discussion #24113.