Skip to content

benchmark.sh: add missing commands for pre-release pipeline #208

@pszymkowiak

Description

@pszymkowiak

Problem

scripts/benchmark.sh currently tests 22 commands but is missing ~12 testable commands that exist in RTK. This means regressions in those filters can slip through the pre-release pipeline undetected.

Missing commands (testable)

Command Notes
rtk cargo build/test/clippy Always available (we're a Rust project)
rtk smart <file> 2-line summary, easy to test
rtk diff <f1> <f2> File diff, easy to test with temp files
rtk tree Directory tree, easy to test
rtk curl <url> HTTP fetch, test with public URL
rtk wget <url> Download, test with public URL
rtk npm run If package.json present
rtk npx If package.json present
rtk proxy <cmd> Passthrough, always available
rtk ruff check Already has Python fixture, just uses wrong $RTK test wrapper
rtk pytest Same — should use $RTK pytest not $RTK test pytest
rtk pip list If pip available
rtk go test/build Already has Go fixture, just uses wrong $RTK test wrapper
rtk golangci-lint Same — should use $RTK golangci-lint not $RTK test golangci-lint
rtk format Universal format checker

Not benchmarkable (skip)

gain, discover, learn, config, init, cc-economics, hook-audit — these are analytics/setup commands, not filters.

Current bugs in benchmark.sh

The Python and Go sections use $RTK test ruff check . and $RTK test golangci-lint run instead of the dedicated $RTK ruff check . and $RTK golangci-lint run commands. This means we're benchmarking the generic test runner instead of the specialized filters.

Expected outcome

All filter commands covered in benchmark.sh so we can catch token savings regressions before release.

# Pre-release validation
cargo build --release
bash scripts/benchmark.sh  # Should test ALL filters

Metadata

Metadata

Assignees

No one assigned

    Labels

    effort-smallQuelques heures, 1 fichierenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions