Skip to content

feat: Cost Control Center#23

Open
dreamwing wants to merge 65 commits intomasterfrom
feature/cost-control-center
Open

feat: Cost Control Center#23
dreamwing wants to merge 65 commits intomasterfrom
feature/cost-control-center

Conversation

@dreamwing
Copy link
Owner

Implementation of Cost Control Center frontend and backend as specified in the PRD. Includes diagnostic engine, optimizer service, and dashboard UI.

dreamwing added 30 commits March 1, 2026 00:18
…urity hardening, config backup, test coverage

Changes across 9 files (588 additions, 238 deletions):

## Diagnostics Engine (diagnostics.js)
- REMOVED: mock fallback data — returns zeroed struct + noData flag instead
- FIXED: D06 cache hit formula to PRD spec: cacheRead/(input+cacheRead)
- FIXED: D05 thinking savings uses actual output tokens × cost ratio
- FIXED: D09 output verbosity uses output/input > 10% threshold
- FIXED: currentMonthlyCost extrapolated by activeDays
- ADDED: _computeTokenCostRatio() for precise per-token costing
- ADDED: 60s result cache with invalidateCache() method
- ADDED: cacheHitRate in API response

## Optimizer Service (optimizer.js)
- ADDED: config backup before every optimization (data/backups/)
- ADDED: action_id whitelist validation
- ADDED: dynamic A01 model target via meta.alternative
- ADDED: diagnostics cache invalidation after apply
- FIXED: JSONL parse tolerance for corrupted lines
- FIXED: savings rounded to 2 decimal places

## Security (auth/middleware.js, auth/routes.js)
- ADDED: crypto.timingSafeEqual() for all key comparisons
- Prevents timing attacks on SECRET_KEY

## Frontend (dashboard.js)
- FIXED: duplicate savings variable declaration
- FIXED: field name alignment (totalMonthlySavings)
- ADDED: loading/disabled state on Apply button
- ADDED: meta passthrough for dynamic A01
- ADDED: savings amount in history timeline entries
- REPLACED: alert() with inline toast notifications
- ADDED: noData state handling

## Route (optimize.js)
- ADDED: meta passthrough from request body

## Tests (3 files)
- diagnostics: 5 → 9 tests (added D07, D09, empty stats, caching)
- optimizer: 4 → 13 tests (added failure, unknown action, backup, corrupted JSONL)
- integration: 3 → 6 tests (added meta, invalid action, cost fields)
…culation

Instead of binary on/off, A02 now offers multiple interval options:
- Every 30 min, 1h, 2h, 4h, or Disable completely
- Each option shows precise savings based on token difference vs current interval
- Radio selector in frontend updates savings tag dynamically on selection
- Optimizer accepts custom interval via meta.interval (not just '0m')

Changes:
- diagnostics.js: generate options array with per-interval savings calculation
- optimizer.js: A02 accepts custom interval value, dynamic title
- dashboard.js: radio selector UI with dynamic savings tag
- dashboard.css: opt-radio styling + .applying button state
- Tests: updated D02 and integration tests for new format
…anced guidance

Five UX improvements from persona analysis report:

1. Layered Copy System (all user levels)
   - plainTitle: beginner-friendly headers (e.g., 'Switch to a cheaper AI model')
   - plainSideEffect: scenario-language side effects
   - Collapsible 'Technical Details' section with codeTag + calcDetail formula
   - Removed hardcoded title overrides — titles now come from backend

2. Config Diff Preview (intermediate users)
   - Each action shows configDiff: { key, from, to }
   - Rendered as: heartbeat.every: 15m → 0m (with strikethrough + green)

3. History Undo (all user levels)
   - backupPath now stored in optimizations.jsonl entries
   - Added restoreBackup() method in optimizer.js
   - POST /api/optimizations/undo endpoint
   - 'Undo' button on most recent history entry
   - Restores all config keys from backup + logs UNDO action

4. Enhanced No-Data Guidance (beginners)
   - Updated message: 'Start chatting with your AI agent...'

5. Calculation Detail Formulas (advanced users)
   - calcDetail field shows the exact formula used for savings
   - e.g., '10K output × 40% thinking × 75% cut × .00/M × 5.0x'

Changes: 5 files, +249/-32
…te card (WIP)

Partial implementation of remaining UX persona analysis items:
- helpText tooltip field on all 6 actions
- diagnostics.config.json custom threshold loading
- ?verbose=true API param for raw data export
- Cache hit rate card render + tooltip ? icon in frontend
- All diagnostic rules now use configurable thresholds
Items implemented:

#3 — Tooltip micro-interaction
  - helpText field on all 8 actions (A01-A09)
  - ? icon with native title tooltip in opt-header

#8 — 7-day Effect Tracking
  - preOptCostSnapshot stored in optimization log entries
  - Frontend compares predicted vs actual savings for >7d entries
  - Color-coded tag: green (>80% achieved), amber (partial)

#9 — Custom Thresholds (diagnostics.config.json)
  - _getThresholds() loads from data/diagnostics.config.json
  - Falls back to built-in defaults
  - All 6 rules use configurable thresholds
  - Example file: data/diagnostics.config.json.example
  - invalidateCache() clears threshold cache

#10 — D03/D04 Diagnostic Rules
  - D03: Session Reset detection (short sessions + high frequency)
  - D04: Idle Skill detection (>3 Skills installed in workspace)
  - Both include full layered copy (plainTitle, helpText, calcDetail, configDiff)

#11 — Pricing Service (from existing pricing.json)
  - NEW: src/services/pricing.js
  - Loads from data/config/pricing.json (no remote fetch)
  - Curated KNOWN_REPLACEMENTS with dynamic savingsRatio
  - MODEL_REPLACEMENTS now dynamically built from actual prices
  - Process-lifetime cache (invalidated on demand)

#12 — Verbose API Export
  - GET /api/diagnostics?verbose=true returns _rawData
  - Includes thresholds, per-model costs, token ratios
  - Non-verbose mode auto-strips _rawData

#13 — Cache Hit Rate Card
  - HTML element in index.html optimizer panel
  - Color-coded: green (>50%), amber (>10%), red (<10%)
  - Hidden until data is available

Changes: 8 files, +619/-337
D04 now scans each Skill folder's modification time instead of just
counting total Skills. Groups results into:
- Idle (>7d): strongly recommended for removal, shown in red badges
- Quiet (>3d): listed for user review, shown in amber badges

Backend: diagnostics.js D04 rule rewritten with fs.stat() per-folder,
returns _meta.idleSkills / _meta.quietSkills arrays with name + daysSince.

Frontend: renderSkillAuditList() renders grouped badge lists inside A04.
CSS: .skill-audit-list, .skill-group, .skill-badge with idle/quiet variants.
Previously D04 was incorrectly scanning ~/.openclaw/workspace/ which
contains project folders, not Skills. Now scans the correct directories:
- ~/.openclaw/skills/ (primary, Skills installed here, may be symlinks)
- ~/.openclaw/workspace/skills/ (secondary, legacy location)

Also handles symbolic links (e.g. backlink-pilot -> ...) and deduplicates
by name when the same Skill appears in both directories.
Based on openclaw/src/agents/skills/workspace.ts loadSkillEntries():
- Skills are loaded from 6 sources with precedence order
- D04 now only scans 'managed' skills (user-installed via openclaw CLI)
- Managed dir: (OPENCLAW_STATE_DIR || ~/.openclaw)/skills
- Validates SKILL.md exists in each subfolder (not just any directory)
- Handles symlinks like OpenClaw's listChildDirectories()
- Skips node_modules and dotfiles

Previously incorrectly scanned ~/.openclaw/workspace/ (project dirs).
Integrated robust directory resolution logic from openclaw/src/infra/home-dir.ts:
- Added support for OPENCLAW_HOME environment variable.
- Added os.homedir() fallback for HOME/USERPROFILE.
- Unified resolution via _resolveHomeDir() and _resolveConfigDir() helpers.
- Applied fixed resolution to Heartbeat (D02) and Skill (D04) detection.
P1: Add MODEL_ALIAS_MAP for pricing fallback — maps provider-native
    model names (e.g. gemini-3-pro-high) to OpenRouter pricing keys.
    Prevents 20x cost underestimation when usage.cost is missing.

P2: Add OPENCLAW_HOME env var support in analyze.js path discovery,
    aligned with OpenClaw's src/infra/home-dir.ts resolution order.

P3: Add OPENCLAW_HOME env var support in config.js HOME_DIR,
    same resolution chain: OPENCLAW_HOME → HOME → USERPROFILE → os.homedir()

P4: Track session count per day in history aggregation.
    Each JSONL file = 1 session. Enables D03 session-reset detection.
Extract resolveHomeDir and resolveConfigDir from diagnostics.js, config.js,
and analyze.js into a single shared utility module (src/utils/paths.js).
This removes duplicate implementations, guarantees consistent OS-specific
path resolution across the app, and avoids triggering env-var constraints
when standalone scripts import the logic.
- Added tests for D03 (Session Reset) and D04 (Idle Skill) in diagnostics.
- Added tests for restoreBackup (Undo) and backupConfig in optimizer.
- Created new test suite for local pricing fallback and replacement generation.
…ify paths, add input validation

- Extract duplicated safeCompare() to shared src/auth/utils.js
- Sanitize backupPath in undo endpoint to prevent path traversal
- Replace os.homedir() with resolveHomeDir()/resolveConfigDir() in optimizer and openclaw_config
- Add input validation (savings, meta) on POST /api/optimize/:action_id
- D05: only trigger on explicit thinkingDefault config, skip undefined
- D01: add comment clarifying intentional single-model recommendation
- Remove unused cacheWrite from diagnostics fallback
- Add OPENCLAW_STATE_DIR override in integration tests
The bottom-nav, view-settings, and script tag were incorrectly nested
inside the external-trigger-container due to missing closing </div> tags
introduced by the Optimizer flip-card feature. This caused the bottom
tabs to only be visible on the Tokens view. Added 3 missing closing tags
to restore proper HTML structure.
- Separated external-trigger-container and flip-container as siblings
  inside view-tokens (they were incorrectly nested inside each other)
- Removed extra 'card' class from card-front that conflicted with
  the dashboard's .card styles (padding, border, background)
- Removed 'card-wrapper' class from flip-container
- Added perspective: 1500px to body for proper 3D flip animation
- Fixed maxWidth -> max-width CSS property
@dreamwing
Copy link
Owner Author

@codex review please~

@dreamwing
Copy link
Owner Author

@greptileai review thx~

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 13, 2026

Greptile Summary

This PR implements the Cost Control Center: a diagnostics engine (D01–D09), an optimizer service (A01–A09), new API routes, and a substantially reworked dashboard UI. The architecture is well-structured — the action whitelist, backup-before-modify pattern, and idempotent A09 marker are all solid — but two issues were found that need attention before merging.

  • Stored XSS in Missions tab (dashboard.js line 422): scriptPath, extracted from a cron job's payload text via regex, is inserted into div.innerHTML without escapeHtml. A cron job whose payload embeds <script> or <img onerror=…> tags in its .js path string would execute arbitrary JavaScript in the dashboard viewer's browser. Fix: wrap with escapeHtml(scriptPath).
  • 7-day effect tracking is dead code (optimizer.js line 335): logOptimization accepts a preOptCostSnapshot field that the UI in renderHistoryList relies on (gated by hist.preOptCostSnapshot && daysSince >= 7), but applyAction never captures or passes the pre-optimization monthly cost. The effect tag is therefore never rendered, even for optimizations applied weeks ago.
  • New backend services (diagnostics.js, optimizer.js, pricing.js, openclaw_config.js) use execFile with fixed argument arrays (no shell injection), properly validate action IDs against a whitelist, and protect the undo endpoint from path traversal via path.basename.
  • New safeCompare utility correctly normalises key lengths via SHA-256 before timingSafeEqual, avoiding the length-leaking shortcut that a naive implementation would have.

Confidence Score: 3/5

  • Not safe to merge as-is — one stored XSS and one silently broken feature need to be addressed first.
  • The two issues found are meaningful: the unescaped scriptPath is a stored XSS that can be triggered by a crafted cron job, and the missing preOptCostSnapshot leaves a fully implemented feature permanently broken. The rest of the new code is well-guarded (whitelist validation, timing-safe auth, path-traversal protection on the undo endpoint), which keeps the overall score from dropping lower.
  • public/js/dashboard.js (unescaped scriptPath XSS) and src/services/optimizer.js (missing preOptCostSnapshot in logOptimization call).

Important Files Changed

Filename Overview
src/services/optimizer.js New optimizer service applying A01–A09 actions; path traversal protection for A04 is solid, but preOptCostSnapshot is never passed to logOptimization, permanently disabling the 7-day effect tracking feature.
public/js/dashboard.js Major rewrite of the dashboard frontend including optimizer UI; unescaped scriptPath in fetchJobs is a stored XSS vector, and the 7-day effect tracking renders dead HTML due to the missing preOptCostSnapshot on the server side.
src/services/diagnostics.js New diagnostics engine (D01–D09) with configurable thresholds, 60s cache, and multi-interval heartbeat options; logic and savings calculations are sound.
src/services/pricing.js New pricing service loading model rates from a local JSON file; replacement ratio calculation is correct and legacy fallback is clearly documented.
src/routes/optimizations.js New undo/history routes; path traversal is properly mitigated via path.basename before resolve, and security check is defense-in-depth.
src/routes/optimize.js New optimization apply route; action_id is validated against a whitelist in the service layer, input types are validated, and auth is applied globally.
src/auth/utils.js New safeCompare utility using SHA-256 hashing for length normalisation before timingSafeEqual; timing-safe comparison is correctly implemented for both equal- and unequal-length inputs.
src/utils/paths.js New shared home/config directory resolver; correctly handles OPENCLAW_HOME tilde expansion and OPENCLAW_STATE_DIR override.
src/services/openclaw_config.js New config manager wrapping the openclaw CLI with a direct JSON file fallback; uses execFile (not exec) with a fixed argument array, avoiding shell injection.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Browser: Dashboard UI] -->|GET /api/diagnostics| B[diagnostics.js\nRunDiagnostics]
    B --> C{Cache fresh?}
    C -- Yes --> D[Return cached result]
    C -- No --> E[Load thresholds\nfrom config JSON]
    E --> F[Read token stats\nlatest.json]
    F --> G[Run D01–D09\nchecks]
    G --> H[pricingService\ngetReplacements]
    H --> G
    G --> I[Cache & return\nactions + savings]
    I --> D

    A -->|POST /api/optimize/:id| J[optimize.js route]
    J --> K[optimizerService\napplyAction]
    K --> L[backupConfig\ndata/backups/]
    L --> M{action}
    M -- A01/A02/A05/A06/A07 --> N[openclaw_config\nsetConfig via CLI]
    M -- A04 --> O[Move skill folders\nto backup dir]
    M -- A09 --> P[Append to SOUL.md\nif marker absent]
    N & O & P --> Q[logOptimization\ndata/logs/*.jsonl]
    Q --> R[invalidateCache\ndiagnosticsEngine]

    A -->|POST /api/optimizations/undo| S[optimizations.js route]
    S --> T[path.basename\ntraversal guard]
    T --> U[restoreBackup\nfrom JSON snapshot]
    U --> R
Loading

Last reviewed commit: 82f07eb

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9fbc8c16c6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +335 to +342
await this.logOptimization({
actionId,
title: details.title,
savings: details.savings,
configChanged: details.configChanged,
backupPath,
undoable: !!backupPath
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7-day effect tracking permanently broken

The logOptimization call here never passes preOptCostSnapshot, so it is always stored as null in the JSONL file. The 7-day post-optimization effect feature in renderHistoryList (dashboard.js line 969) is gated on hist.preOptCostSnapshot && …, which will always be falsy. The entire effect-tracking UI path (effectHtml) is effectively dead code.

To fix this, the current monthly cost must be captured before applying the action and passed through:

// Fetch pre-optimization cost snapshot before applying changes
let preOptCostSnapshot = null;
try {
    const diag = await diagnosticsEngine.runDiagnostics();
    preOptCostSnapshot = diag.currentMonthlyCost ?? null;
} catch (_) { /* non-blocking */ }

// ... (existing switch/case logic) ...

await this.logOptimization({
    actionId,
    title: details.title,
    savings: details.savings,
    configChanged: details.configChanged,
    backupPath,
    preOptCostSnapshot,   // ← add this
    undoable: !!backupPath
});

Comment on lines +420 to 425
if (status === 'ok') badgeClass = 'ok';
if (status === 'error' || status === 'skipped') badgeClass = 'fail';

// Deduplication Logic
const firstItem = feed.firstElementChild;
if (firstItem) {
const textSpan = firstItem.querySelector('span:last-child');
if (textSpan && textSpan.innerText === task) {
// Match found! Update time instead of adding new row.
const timeSpan = firstItem.querySelector('span:first-child');
const time = new Date(ts).toLocaleTimeString('en-US', { hour12: false });
timeSpan.innerText = time;

// Flash effect
firstItem.style.background = 'rgba(255,255,255,0.1)';
setTimeout(() => firstItem.style.background = 'transparent', 300);
return;
}
}
const div = document.createElement('div');
div.className = 'job-item';

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stored XSS via unescaped scriptPath in job list

scriptPath is extracted from the cron job's payload text with a regex, but it is interpolated directly into the pathHtml string without HTML-escaping before being assigned to div.innerHTML. The [^']+ pattern in the regex does not restrict <, >, or other HTML metacharacters, so a cron job with a payload like:

node '/path/to/<img src=x onerror=alert(document.cookie)>.js'

would cause arbitrary JavaScript execution when the Missions tab is viewed.

Use escapeHtml on scriptPath:

Suggested change
if (status === 'ok') badgeClass = 'ok';
if (status === 'error' || status === 'skipped') badgeClass = 'fail';
// Deduplication Logic
const firstItem = feed.firstElementChild;
if (firstItem) {
const textSpan = firstItem.querySelector('span:last-child');
if (textSpan && textSpan.innerText === task) {
// Match found! Update time instead of adding new row.
const timeSpan = firstItem.querySelector('span:first-child');
const time = new Date(ts).toLocaleTimeString('en-US', { hour12: false });
timeSpan.innerText = time;
// Flash effect
firstItem.style.background = 'rgba(255,255,255,0.1)';
setTimeout(() => firstItem.style.background = 'transparent', 300);
return;
}
}
const div = document.createElement('div');
div.className = 'job-item';
if (scriptPath) {
pathHtml = `<div style="font-family:monospace; font-size:10px; color:var(--text-dim); margin-top:3px; word-break:break-all; opacity:0.7;">
📄 ${escapeHtml(scriptPath)}
</div>`;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant