Skip to content

Commit 18c5399

Browse files
lpcoxCopilotCopilot
authored
fix: replace piped while loops with temp files to avoid pipefail SIGPIPE (#24350)
* fix: replace piped while loops with temp files to avoid pipefail SIGPIPE The token usage analyzer workflows use set -euo pipefail which causes jq | head | while read pipelines to fail with exit 141 (SIGPIPE) when head closes the pipe early, silently aborting artifact downloads. Write to temp files first, then read from them in while loops. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update .github/workflows/copilot-token-usage-analyzer.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/claude-token-usage-analyzer.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/copilot-token-usage-analyzer.lock.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/claude-token-usage-analyzer.lock.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: recompile lock files after jq slicing updates Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 1355331 commit 18c5399

4 files changed

Lines changed: 44 additions & 40 deletions

File tree

.github/workflows/claude-token-usage-analyzer.lock.yml

Lines changed: 16 additions & 16 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.github/workflows/claude-token-usage-analyzer.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -69,15 +69,16 @@ steps:
6969
mkdir -p "$ARTIFACT_DIR"
7070
7171
echo "📥 Downloading token-usage.jsonl artifacts..."
72-
jq -r '.[].databaseId' /tmp/token-analyzer-claude/claude-runs.json 2>/dev/null | head -50 | while read -r run_id; do
72+
jq -r '.[0:50] | .[].databaseId' /tmp/token-analyzer-claude/claude-runs.json > /tmp/token-analyzer-claude/run-ids.txt
73+
while read -r run_id; do
7374
run_dir="$ARTIFACT_DIR/$run_id"
7475
mkdir -p "$run_dir"
7576
gh run download "$run_id" \
7677
--repo "$GITHUB_REPOSITORY" \
7778
--name "firewall-audit-logs" \
7879
--dir "$run_dir" \
7980
2>/dev/null || true
80-
done
81+
done < /tmp/token-analyzer-claude/run-ids.txt
8182
8283
# Count how many token-usage.jsonl files we got
8384
JSONL_COUNT=$(find "$ARTIFACT_DIR" -name "token-usage.jsonl" 2>/dev/null | wc -l)
@@ -86,14 +87,15 @@ steps:
8687
# Merge all token-usage.jsonl files annotated with run_id
8788
MERGED_FILE="/tmp/token-analyzer-claude/token-usage-merged.jsonl"
8889
> "$MERGED_FILE"
89-
find "$ARTIFACT_DIR" -name "token-usage.jsonl" | while read -r f; do
90+
find "$ARTIFACT_DIR" -name "token-usage.jsonl" > /tmp/token-analyzer-claude/jsonl-files.txt 2>/dev/null || true
91+
while read -r f; do
9092
run_id=$(echo "$f" | grep -oP '(?<=/artifacts/)\d+(?=/)' || true)
9193
while IFS= read -r line; do
9294
if [ -n "$line" ]; then
9395
echo "${line}" | jq --arg run_id "$run_id" '. + {run_id: $run_id}' >> "$MERGED_FILE" 2>/dev/null || true
9496
fi
9597
done < "$f"
96-
done
98+
done < /tmp/token-analyzer-claude/jsonl-files.txt
9799
98100
RECORD_COUNT=$(wc -l < "$MERGED_FILE" 2>/dev/null || echo 0)
99101
echo "✅ Merged ${RECORD_COUNT} token usage records"

0 commit comments

Comments
 (0)