Improvements based on Sam Keen's recent article

[harrymunro]

Link to article by Sam Keen which provides great recommendations: https://alteredcraft.com/p/the-ralph-wiggum-agent-loop-is-really

[greptile-apps]

Greptile Overview

Greptile Summary

This PR significantly improves Ralph's production-readiness by adding comprehensive security, cost tracking, and quality control mechanisms.

Key Improvements:

• Added security pre-flight checks to ralph.sh that warn users when sensitive environment variables like AWS_ACCESS_KEY_ID or DATABASE_URL are set, preventing accidental credential exposure
• Implemented circuit breaker pattern in ralph.sh to skip stories that fail repeatedly (configurable via MAX_ATTEMPTS_PER_STORY, defaults to 5), preventing infinite retry loops
• Added COMPLETE signal verification to ensure Ralph doesn't falsely claim completion when stories still have passes: false
• Created comprehensive documentation: docs/SECURITY.md for sandboxing and emergency procedures, docs/COST_TRACKING.md for budget management, and docs/MONITORING.md for operational monitoring
• Enhanced quality gates in CLAUDE.md and prompt.md with verification checklists, forbidden shortcuts table (e.g., @ts-ignore, any type), and evidence-based verification requirements
• Added verificationCommands field to prd.json.example allowing projects to specify custom quality gate commands
• Created new /planning skill that enforces 5 rounds of requirement exploration before PRD creation to prevent under-specified features
• Strengthened acceptance criteria requirements in skills/ralph/SKILL.md with machine-verifiable examples and explicit forbidden terms table

Impact:
These changes address common failure modes in autonomous agent systems: credential leaks, runaway costs, infinite retry loops, and vague requirements that lead to wasted iterations. The additions are well-documented and provide clear operational guidance for users running Ralph in production environments.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk - it adds defensive safeguards and documentation without breaking existing functionality
• All changes are additive improvements to an autonomous agent system. The security pre-flight checks, circuit breakers, and verification enhancements reduce risk rather than introduce it. New documentation files provide clear operational guidance. The only minor style suggestion is about string comparison syntax in bash, which doesn't affect functionality. No breaking changes or logical errors detected.
• No files require special attention - all changes are well-structured improvements

Important Files Changed

Filename	Overview
CLAUDE.md	Added comprehensive quality gates, verification checklists, and backpressure controls to ensure autonomous agents complete stories properly
docs/COST_TRACKING.md	New comprehensive guide for tracking API costs with budget recommendations and monitoring strategies
docs/SECURITY.md	New security guide with pre-flight checklists, emergency stop procedures, and Docker sandboxing examples
ralph.sh	Added security pre-flight checks, circuit breaker for story retry limits, and COMPLETE signal verification
skills/ralph/SKILL.md	Strengthened acceptance criteria requirements with machine-verifiable examples and forbidden vague terms table

Sequence Diagram

sequenceDiagram
    participant User
    participant ralph.sh
    participant Security Check
    participant Circuit Breaker
    participant AI Agent
    participant Quality Gates
    participant prd.json

    User->>ralph.sh: Start Ralph
    ralph.sh->>Security Check: Check environment variables
    alt Credentials Detected
        Security Check-->>User: Warn about AWS_ACCESS_KEY_ID, DATABASE_URL
        User->>Security Check: Confirm or abort
    end
    Security Check->>ralph.sh: Continue
    
    loop For each iteration (up to MAX_ITERATIONS)
        ralph.sh->>prd.json: Get next incomplete story
        ralph.sh->>Circuit Breaker: Check story attempts
        alt Max attempts reached
            Circuit Breaker->>prd.json: Mark story as skipped
            Circuit Breaker->>ralph.sh: Skip to next story
        else Attempts remaining
            Circuit Breaker->>ralph.sh: Increment attempt counter
            ralph.sh->>AI Agent: Spawn fresh instance (Amp or Claude)
            AI Agent->>AI Agent: Read prd.json, progress.txt
            AI Agent->>AI Agent: Implement story
            AI Agent->>Quality Gates: Run typecheck, lint, tests
            alt Quality gates pass
                Quality Gates->>AI Agent: All checks passed
                AI Agent->>AI Agent: Commit changes
                AI Agent->>prd.json: Mark story passes: true
                AI Agent->>AI Agent: Append to progress.txt
                AI Agent-->>ralph.sh: Output COMPLETE signal (if all done)
            else Quality gates fail
                Quality Gates->>AI Agent: Checks failed
                AI Agent->>AI Agent: Leave story incomplete
            end
            ralph.sh->>ralph.sh: Check for COMPLETE signal
            alt COMPLETE claimed
                ralph.sh->>prd.json: Verify all stories passes: true
                alt Verification passes
                    ralph.sh-->>User: All tasks completed
                else Verification fails
                    ralph.sh->>ralph.sh: Continue iteration
                end
            end
        end
    end

Loading

[greptile-apps]

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Using == for string comparison in bash. While this works in [[ ]] context, consider using = for better POSIX compatibility if the script needs to run on different shells.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review.
Path: ralph.sh
Line: 204:204

Comment:
Using `==` for string comparison in bash. While this works in `[[ ]]` context, consider using `=` for better POSIX compatibility if the script needs to run on different shells.

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.