Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions .github/actions/enforce/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: "shekel — LLM Budget Enforcement"
description: "Run a Python agent script with a USD budget cap. Exits 1 if the budget is exceeded."
author: "shekel"

branding:
icon: "dollar-sign"
color: "green"

inputs:
script:
description: "Path to the Python agent script to run"
required: true
budget:
description: "Maximum spend in USD (e.g. 5 for $5)"
required: false
default: ""
warn-at:
description: "Warn fraction 0.0–1.0 of budget (e.g. 0.8 for 80%)"
required: false
default: ""
max-llm-calls:
description: "Maximum number of LLM API calls"
required: false
default: ""
max-tool-calls:
description: "Maximum number of tool invocations"
required: false
default: ""
warn-only:
description: "If true, warn but do not exit 1 on budget exceeded"
required: false
default: "false"
output:
description: "Output format: text or json"
required: false
default: "text"
budget-file:
description: "Path to a TOML budget config file (shekel.toml)"
required: false
default: ""
shekel-version:
description: "shekel package version to install (e.g. '>=0.3.0')"
required: false
default: ">=0.3.0"

outputs:
spent:
description: "Total USD spent (populated when --output json is used)"
status:
description: "Budget status: ok | warn | exceeded"

runs:
using: "composite"
steps:
- name: Install shekel
shell: bash
run: pip install "shekel[cli]${{ inputs.shekel-version }}" --quiet

- name: Run agent with budget enforcement
shell: bash
run: |
ARGS="${{ inputs.script }}"

if [ -n "${{ inputs.budget }}" ]; then
ARGS="$ARGS --budget ${{ inputs.budget }}"
fi
if [ -n "${{ inputs.warn-at }}" ]; then
ARGS="$ARGS --warn-at ${{ inputs.warn-at }}"
fi
if [ -n "${{ inputs.max-llm-calls }}" ]; then
ARGS="$ARGS --max-llm-calls ${{ inputs.max-llm-calls }}"
fi
if [ -n "${{ inputs.max-tool-calls }}" ]; then
ARGS="$ARGS --max-tool-calls ${{ inputs.max-tool-calls }}"
fi
if [ "${{ inputs.warn-only }}" = "true" ]; then
ARGS="$ARGS --warn-only"
fi
if [ -n "${{ inputs.output }}" ]; then
ARGS="$ARGS --output ${{ inputs.output }}"
fi
if [ -n "${{ inputs.budget-file }}" ]; then
ARGS="$ARGS --budget-file ${{ inputs.budget-file }}"
fi

shekel run $ARGS
85 changes: 84 additions & 1 deletion docs/cli.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# CLI Tools

Shekel provides command-line tools for cost estimation and model information.
Shekel provides command-line tools for budget enforcement, cost estimation, and model information.

## Installation

Expand All @@ -12,6 +12,88 @@ This installs the `shekel` command with Click support.

## Commands

### `shekel run`

Run a Python script with budget enforcement. Equivalent to wrapping your script
in `with budget(max_usd=N):` — zero code changes required.

#### Usage

```bash
shekel run SCRIPT [OPTIONS] [-- SCRIPT_ARGS...]
```

#### Options

| Option | Description |
|--------|-------------|
| `--budget N` | Max spend in USD. Equivalent to `AGENT_BUDGET_USD=N`. |
| `--warn-at F` | Warn fraction 0.0–1.0 (e.g. `0.8` = warn at 80% of budget). |
| `--max-llm-calls N` | Cap on LLM API calls. |
| `--max-tool-calls N` | Cap on tool invocations. |
| `--warn-only` | Warn but never exit 1 when budget exceeded. |
| `--dry-run` | Track costs only — no enforcement. Implies `--warn-only`. |
| `--output text\|json` | Output format (default: `text`). |
| `--budget-file PATH` | Path to a `shekel.toml` config file. |
| `--fallback-model M` | Cheaper model to switch to at threshold. |
| `--fallback-at F` | Fallback activation threshold (default: `0.8`). |

#### Exit codes

| Code | Meaning |
|------|---------|
| `0` | Script completed within budget |
| `1` | Budget exceeded (unless `--warn-only`) |
| `2` | Configuration error (missing script, bad TOML, etc.) |

#### Environment variables

| Variable | Description |
|----------|-------------|
| `AGENT_BUDGET_USD` | Fallback for `--budget`. Ideal for Docker/CI operator control. |

#### Examples

```bash
# Enforce a $5 cap
shekel run agent.py --budget 5

# Warn at 80%, hard-stop at $5
shekel run agent.py --budget 5 --warn-at 0.8

# Cap LLM calls instead of spend
shekel run agent.py --max-llm-calls 20

# JSON output for CI log parsing
shekel run agent.py --budget 5 --output json

# Warn but don't fail the pipeline
shekel run agent.py --budget 5 --warn-only

# Dry-run: track costs without enforcement
shekel run agent.py --budget 5 --dry-run

# Load limits from TOML file
shekel run agent.py --budget-file shekel.toml

# Set budget via env var (Docker / CI)
AGENT_BUDGET_USD=5 shekel run agent.py
```

#### `shekel.toml` format

```toml
[budget]
max_usd = 5.0
warn_at = 0.8
max_llm_calls = 50
max_tool_calls = 200
```

See [Docker & Container Guardrails](docker.md) for container-specific patterns.

---

### `shekel estimate`

Estimate API call costs without making actual requests.
Expand Down Expand Up @@ -244,6 +326,7 @@ print(f"Available models: {models}")

## Next Steps

- [Docker & Container Guardrails](docker.md) - Using `shekel run` in Docker
- [Supported Models](models.md) - Full model list with pricing
- [Installation](installation.md) - Installing CLI tools
- [Basic Usage](usage/basic-usage.md) - Using budgets in code
154 changes: 154 additions & 0 deletions docs/docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Docker & Container Guardrails

Use `shekel run` as an entrypoint wrapper to enforce LLM cost limits on any agent
running inside a Docker container — zero code changes required.

## Quick start

```dockerfile
FROM python:3.12-slim

WORKDIR /app

# Install your agent and shekel CLI
COPY requirements.txt .
RUN pip install -r requirements.txt shekel[cli]

COPY agent.py .

# shekel run becomes the entrypoint; AGENT_BUDGET_USD sets the cap at runtime
ENTRYPOINT ["shekel", "run", "agent.py"]
```

Run with a $5 cap:

```bash
docker run -e AGENT_BUDGET_USD=5 my-agent-image
```

The container exits with code 1 if the budget is exceeded, so your orchestration
layer (ECS, Kubernetes, Compose) can detect it as a failed task.

---

## Patterns

### Budget via environment variable

The `AGENT_BUDGET_USD` env var is equivalent to `--budget N`. This is the
preferred pattern for containers because the budget can be set by the operator
without rebuilding the image.

```bash
# docker run
docker run -e AGENT_BUDGET_USD=10 my-agent-image

# docker-compose
services:
agent:
image: my-agent-image
environment:
AGENT_BUDGET_USD: "10"
```

### Budget via CLI flag (baked into image)

```dockerfile
ENTRYPOINT ["shekel", "run", "agent.py", "--budget", "5"]
```

### TOML config file

Mount a `shekel.toml` at runtime for fine-grained control:

```bash
docker run -v $(pwd)/shekel.toml:/app/shekel.toml \
my-agent-image shekel run agent.py --budget-file /app/shekel.toml
```

```toml
# shekel.toml
[budget]
max_usd = 5.0
warn_at = 0.8
max_llm_calls = 50
max_tool_calls = 200
```

### Warn-only mode (log but don't kill)

```dockerfile
ENTRYPOINT ["shekel", "run", "agent.py", "--warn-only"]
```

With `--warn-only`, the container exits 0 even if the budget is exceeded.
Use this during development to observe spend without blocking the run.

### JSON output for structured logging

```bash
docker run my-agent-image shekel run agent.py --budget 5 --output json \
| tee /logs/spend.json
```

The JSON line emitted at the end:

```json
{
"spent": 1.23,
"limit": 5.0,
"calls": 12,
"tool_calls": 4,
"status": "ok",
"model": "gpt-4o"
}
```

---

## Exit codes

| Code | Meaning |
|------|---------|
| `0` | Script completed within budget (or `--warn-only` mode) |
| `1` | Budget exceeded (default mode) |
| `2` | Configuration error (missing script, bad TOML, etc.) |

---

## Shell script wrapper

For non-Docker environments (e.g. bare VMs, `.sh` CI scripts):

```bash
#!/usr/bin/env bash
set -euo pipefail

BUDGET="${AGENT_BUDGET_USD:-5}"

shekel run agent.py \
--budget "$BUDGET" \
--warn-at 0.8 \
--output json \
| tee spend.json

status=$(jq -r '.status' spend.json)
if [ "$status" = "exceeded" ]; then
echo "Budget exceeded — check spend.json for details" >&2
exit 1
fi
```

---

## GitHub Actions

See the [CLI reference](cli.md) or use the bundled composite action:

```yaml
- uses: ./.github/actions/enforce
with:
script: agent.py
budget: "5"
warn-at: "0.8"
```
6 changes: 5 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "shekel"
version = "0.2.8"
version = "0.2.9"
description = "LLM budget enforcement and cost tracking. Zero config — with budget(max_usd=1.00): run_agent(). Works with LangGraph, CrewAI, raw OpenAI/Anthropic/Gemini."
readme = "README.md"
license = { file = "LICENSE" }
Expand Down Expand Up @@ -142,6 +142,10 @@ warn_return_any = true
warn_unused_configs = true
exclude = ["/_pytest/"]

[[tool.mypy.overrides]]
module = "tomli"
ignore_missing_imports = true

[[tool.mypy.overrides]]
module = "tokencost"
ignore_missing_imports = true
Expand Down
2 changes: 1 addition & 1 deletion shekel/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from shekel._tool import tool
from shekel.exceptions import BudgetExceededError, ToolBudgetExceededError

__version__ = "0.2.8"
__version__ = "0.2.9"
__all__ = [
"budget",
"Budget",
Expand Down
Loading
Loading