feat: production-ready alpha — gateway, benchmarks, demo, CI, Docker

.github/wiki/Benchmarking-and-Diff.md

@@ -58,29 +58,46 @@ This is representative of a production codebase with meaningful cross-file depen
 
 ## Pre-Generated Results
 
-| Task | Baseline | Compressed | Reduction |
-|------|----------|------------|-----------|
-| Add Redis caching to task lookup endpoints | 12,230 tok | 7,937 tok | **35%** |
-| Add JWT authentication | 12,230 tok | 3,259 tok | **73%** |
-| Refactor database module | 12,230 tok | 1,768 tok | **86%** |
+**Focused (4 tasks, Python microservice — 12,228 baseline tokens):**
 
-See `docs/benchmarks/` in the repository for the full pre-generated reports.
+Each task was measured with a cold cache; `cache_assisted_pack` shows savings
+when summaries from the prior run are reused.
+
+| Task | Baseline | Cold (compressed_pack) | Reduction | Warm (cache_assisted) | Reduction |
+|------|----------|-----------------------|-----------|-----------------------|-----------|
+| Add Redis caching | 12,228 | 7,827 | 36% | 1,459 | 88% |
+| Add JWT authentication | 12,228 | 7,512 | 39% | 150 | 99% |
+| Refactor module | 12,228 | 7,831 | 36% | 610 | 95% |
+| Add rate limiting | 12,228 | 7,827 | 36% | 1,459 | 88% |
+| **Average** | | | **37%** | | **92%** |
+
+**Large-scale (1,000 tasks, Redcon codebase — 2.2 M baseline tokens):**
+
+| Metric | compressed_pack | cache_assisted_pack |
+|--------|----------------|---------------------|
+| Mean tokens | 2,565 | 496 |
+| Mean savings | 99.9% | 100.0% |
+| p10 savings | 99.7% | 100.0% |
+| Quality risk low | 99.8% | - |
+
+> The 2.2 M baseline includes all repo files (source, docs, wiki, examples).
+
+See `redcon-benchmarks/` in the repository for raw JSON and the reproducible runner.
 
 ---
 
 ## Reproduce Locally
 
-Run all three canonical tasks:
-
 ```bash
-python benchmarks/run_benchmarks.py
-```
+# 1,000-task benchmark (~14 min)
+python redcon-benchmarks/run_large_benchmark.py
 
-Or run a single task:
+# 100-task sample (~90 s)
+python redcon-benchmarks/run_large_benchmark.py --sample 100
 
-```bash
+# Single task
 redcon benchmark "Add Redis caching to task lookup endpoints" \
-    --repo benchmarks/dataset --max-tokens 8000
+    --repo . --max-tokens 32000
 ```
 
 ---

.github/workflows/redcon.yml

@@ -60,7 +60,7 @@ jobs:
           changed_files="${CHANGED_FILES_INPUT}"
           if [[ -z "$changed_files" ]]; then
             if [[ "${{ github.event_name }}" == "pull_request" ]]; then
-              git fetch origin "${{ github.base_ref }}" --depth=1
+              git fetch origin "${{ github.base_ref }}"
               changed_files="$(git diff --name-only "origin/${{ github.base_ref }}...${{ github.sha }}" | paste -sd, -)"
             elif [[ "${{ github.event_name }}" == "push" ]]; then
               changed_files="$(git diff --name-only "${{ github.event.before }}...${{ github.sha }}" | paste -sd, -)"

.github/workflows/tests.yml

@@ -0,0 +1,46 @@
+name: Tests
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+jobs:
+  test:
+    name: "Python ${{ matrix.python-version }} / ${{ matrix.extras }}"
+    runs-on: ubuntu-latest
+
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+        extras: ["core", "gateway", "redis"]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies (core)
+        if: matrix.extras == 'core'
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e ".[dev]"
+
+      - name: Install dependencies (gateway)
+        if: matrix.extras == 'gateway'
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e ".[dev,gateway]"
+
+      - name: Install dependencies (redis)
+        if: matrix.extras == 'redis'
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e ".[dev,redis]"
+
+      - name: Run tests
+        run: python -m pytest tests/ -x -q --tb=short

Dockerfile

@@ -1,32 +1,38 @@
-# ContextBudget CLI — Docker image
-# Provides the full contextbudget CLI and Python API.
+# Redcon — Docker image
+# Provides the full Redcon CLI, gateway server, and Python API.
 # Use this image in CI/CD pipelines or as a base for custom integrations.
 #
-# Build:   docker build -t contextbudget .
-# Run:     docker run --rm -v "$(pwd)":/repo contextbudget pack "my task" --repo /repo
+# Build:   docker build -t redcon .
+# Run CLI: docker run --rm -v "$(pwd)":/repo redcon pack "my task" --repo /repo
 #
-# To use the gateway server:
-#   docker run --rm -p 8787:8787 contextbudget gateway --host 0.0.0.0 --port 8787
+# Gateway server (no Redis):
+#   docker run --rm -p 8787:8787 redcon gateway --host 0.0.0.0 --port 8787
+#
+# Gateway server (with Redis session store):
+#   RC_GATEWAY_REDIS_URL=redis://redis:6379/0 \
+#   docker run --rm -p 8787:8787 redcon gateway --host 0.0.0.0 --port 8787
+#
+# See docker-compose.yml for the full gateway + Redis stack.
 
 FROM python:3.12-slim
 
-LABEL org.opencontainers.image.title="ContextBudget CLI"
-LABEL org.opencontainers.image.description="Optimize LLM context windows and enforce token budgets"
+LABEL org.opencontainers.image.title="Redcon"
+LABEL org.opencontainers.image.description="Deterministic context budgeting for coding-agent workflows"
 LABEL org.opencontainers.image.source="https://github.com/natiixnt/ContextBudget"
 
-# Install system dependencies for git operations (used by drift/pr-audit commands)
+# git is required for pr-audit and drift commands
 RUN apt-get update && apt-get install -y --no-install-recommends git && rm -rf /var/lib/apt/lists/*
 
 WORKDIR /app
 
-# Copy and install the package
 COPY pyproject.toml ./
-COPY contextbudget/ contextbudget/
+COPY redcon/ redcon/
 
-RUN pip install --no-cache-dir -e .
+# Install core package + gateway extras (FastAPI + uvicorn)
+RUN pip install --no-cache-dir -e ".[gateway]"
 
 # Default working directory for repository mounts
 WORKDIR /repo
 
-ENTRYPOINT ["python", "-m", "contextbudget.cli"]
+ENTRYPOINT ["python", "-m", "redcon.cli"]
 CMD ["--help"]

LICENSE-COMMERCIAL

@@ -0,0 +1,24 @@
+Redcon Commercial License
+Copyright (c) 2026 nai. All rights reserved.
+
+The following components of this repository are NOT covered by the MIT License
+and are proprietary to Redcon:
+
+  redcon/gateway/
+  redcon/control_plane/
+  redcon/agents/
+  redcon/integrations/
+  redcon/runtime/
+  redcon/telemetry/
+
+These components may not be used, copied, modified, merged, published,
+distributed, sublicensed, or sold without a valid commercial license from
+Redcon.
+
+For commercial licensing, contact: natjiks@gmail.com
+
+The remaining components of this repository (redcon/core/, redcon/compressors/,
+redcon/scanners/, redcon/scorers/, redcon/stages/, redcon/schemas/,
+redcon/plugins/, redcon/cache/, redcon/sdk/, redcon/engine.py, redcon/cli.py,
+benchmarks/, and the CLI) are available under the MIT License as stated in the
+LICENSE file.

README.md

@@ -13,6 +13,19 @@ Redcon selects, compresses, and budgets repository context for coding-agent work
 - supports local multi-repo and monorepo-package workspaces
 - exposes an adapter-ready middleware layer for external agent tools
 
+## Benchmark
+
+Focused 4-task study on a 15-file Python FastAPI service (12,228 token baseline):
+
+| Scenario | Mean tokens | Mean reduction |
+|----------|-------------|----------------|
+| compressed_pack (cold start) | 7,749 | **37%** |
+| cache_assisted_pack (warm cache) | 919 | **92%** |
+
+Cold start: first run, no prior summaries. Warm cache: second run, per-file summaries reused.
+
+Full methodology, raw JSON, and the 1,000-task large-scale results: [`redcon-benchmarks/`](redcon-benchmarks/).
+
 ## Quickstart
 
 ```bash
@@ -149,3 +162,22 @@ Detailed upgrade notes: [docs/migration.md](docs/migration.md).
 - [Migration Notes](docs/migration.md)
 
 Examples and sample outputs: [examples/README.md](examples/README.md).
+
+## License
+
+Redcon uses a dual-license model:
+
+| Component | License |
+|-----------|---------|
+| Core engine (`redcon/core/`, `redcon/compressors/`, `redcon/scanners/`, `redcon/scorers/`, `redcon/stages/`, `redcon/schemas/`, `redcon/plugins/`, `redcon/cache/`, `redcon/sdk/`, `redcon/cli.py`, `redcon/engine.py`) | [MIT](LICENSE) |
+| CLI and benchmark framework | [MIT](LICENSE) |
+| Gateway server (`redcon/gateway/`) | [Proprietary](LICENSE-COMMERCIAL) |
+| Control plane (`redcon/control_plane/`) | [Proprietary](LICENSE-COMMERCIAL) |
+| Agent middleware (`redcon/agents/`) | [Proprietary](LICENSE-COMMERCIAL) |
+| LLM integrations (`redcon/integrations/`) | [Proprietary](LICENSE-COMMERCIAL) |
+| Runtime and telemetry (`redcon/runtime/`, `redcon/telemetry/`) | [Proprietary](LICENSE-COMMERCIAL) |
+
+The open-source core builds community adoption and developer trust.
+The commercial layer is what powers Redcon Cloud and enterprise deployments.
+
+For commercial licensing: natjiks@gmail.com

demo/README.md

@@ -0,0 +1,126 @@
+# Redcon Demo
+
+End-to-end walkthrough of all core components against this repository.
+
+## Quick start
+
+```bash
+# From the repo root — no extra dependencies needed
+python demo/run_demo.py
+```
+
+For the FastAPI gateway section, install the gateway extra first:
+
+```bash
+pip install -e ".[gateway]"
+python demo/run_demo.py
+```
+
+## What the demo covers
+
+| Step | Component | What it shows |
+|------|-----------|---------------|
+| 1 | **Pack** | Scan, rank, and compress repository context under a token budget |
+| 2 | **Policy** | Evaluate token-budget policy constraints against a run artifact |
+| 3 | **Cost analytics** | Translate token savings into estimated USD savings (gpt-4o pricing) |
+| 4 | **Benchmark** | Compare packing strategies side-by-side |
+| 5 | **Gateway** | Start the FastAPI gateway in-process and send real HTTP requests |
+| 6 | **Adapters** | Instantiate the OpenAI and Anthropic wrappers with a stub LLM function |
+
+No network calls are made. The gateway runs on `127.0.0.1:18787` for the duration
+of the demo and is stopped immediately after. The adapter demo uses a stub LLM
+function so no real API keys are required.
+
+## Full stack (gateway + Redis)
+
+```bash
+# Copy and configure
+cp .env.example .env          # set RC_GATEWAY_API_KEY etc.
+
+# Start gateway + Redis
+docker compose up -d
+
+# Send a request
+curl -s http://localhost:8787/health
+
+curl -s -X POST http://localhost:8787/prepare-context \
+  -H "Content-Type: application/json" \
+  -d '{"task": "add caching to the session store", "repo": "/repos/myproject", "max_tokens": 32000}'
+```
+
+## Architecture
+
+```
+agent / CI pipeline
+      |
+      | HTTP JSON
+      v
+Redcon Gateway  (POST /prepare-context, /run-agent-step, /report-run)
+      |               |
+      |         Redis session store  (horizontal scaling)
+      |
+      v
+scan -> rank -> compress -> cache -> delta
+      |
+      v
+optimized prompt  -->  OpenAI / Anthropic / any LLM
+```
+
+## Key endpoints
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/health` | GET | Liveness check |
+| `/metrics` | GET | Request counters and uptime |
+| `/prepare-context` | POST | Stateless context optimization |
+| `/run-agent-step` | POST | Stateful multi-turn agent session |
+| `/report-run` | POST | Acknowledge LLM completion and record telemetry |
+
+## Using the adapters directly
+
+```python
+from redcon.integrations import OpenAIAgentWrapper, AnthropicAgentWrapper
+
+# OpenAI
+agent = OpenAIAgentWrapper(model="gpt-4.1", repo=".")
+result = agent.run_task("add caching to the API")
+print(result.llm_response)
+
+# Anthropic
+agent = AnthropicAgentWrapper(model="claude-sonnet-4-6", repo=".")
+result = agent.run_task("add caching to the API")
+print(result.llm_response)
+```
+
+Both wrappers automatically run the full Redcon pipeline before every LLM call
+and emit telemetry to `.redcon/observe-history.json`.
+
+## Enforcing a token-budget policy
+
+```python
+from redcon.core.policy import PolicySpec, evaluate_policy
+
+policy = PolicySpec(
+    max_estimated_input_tokens=64_000,
+    max_files_included=40,
+    max_quality_risk_level="medium",
+)
+
+from redcon.engine import RedconEngine
+artifact = RedconEngine().pack("my task", repo=".")
+result = evaluate_policy(artifact, policy)
+print(result.passed, result.violations)
+```
+
+## GitHub Action
+
+```yaml
+- uses: natiixnt/ContextBudget@v1
+  with:
+    task: "review pull request changes"
+    policy: .github/redcon-policy.toml
+    strict: "true"
+    model: gpt-4o
+```
+
+See [action.yml](../action.yml) for all inputs and outputs.