diff --git a/marketing/differentiators-vs-claude-code.md b/marketing/differentiators-vs-claude-code.md new file mode 100644 index 00000000..0def00c9 --- /dev/null +++ b/marketing/differentiators-vs-claude-code.md @@ -0,0 +1,13 @@ +# Codev vs Claude Code: Key Differentiators + +1. **Multi-model**: We use Gemini + Codex as reviewers alongside Claude. Each catches different classes of issues — Codex finds security edge cases, Claude catches runtime semantics, Gemini catches architecture problems. + +2. **Specs and plans are first-class citizens**: Every feature produces a specification and an implementation plan that are preserved as project artifacts. You always know WHY something was built and HOW it was designed. + +3. **Plans are enforced**: You can't start Phase 2 until the acceptance criteria for Phase 1 are met. The Porch state machine makes the process deterministic — no skipping steps. Human gates require architect approval before implementation begins, so the AI can't run off and build the wrong thing. Testing is enforced as acceptance criteria, not optional — this is why codev produces 2.9x more test coverage consistently. + +4. **Annotation over direct editing**: It's far more about annotating docs than directly editing code. Reviews, specs, and plans are documents that guide the work rather than the AI just hacking at files. + +5. **Parallel builders**: Push parallelization to its limit by having one "Architect" AI that can direct 5+ builders working simultaneously in isolated worktrees. + +6. **Whole lifecycle management**: From idea through specification, planning, implementation, review, PR, and deployment — codev manages the entire lifecycle, not just the coding step. diff --git a/marketing/logo-transparent.png b/marketing/logo-transparent.png new file mode 100644 index 00000000..1a775e3d Binary files /dev/null and b/marketing/logo-transparent.png differ diff --git a/marketing/logo.png b/marketing/logo.png new file mode 100644 index 00000000..6baa6826 Binary files /dev/null and b/marketing/logo.png differ diff --git a/marketing/real-results.md b/marketing/real-results.md new file mode 100644 index 00000000..9034491c --- /dev/null +++ b/marketing/real-results.md @@ -0,0 +1,24 @@ +# Real Results + +## Head-to-Head: Codev vs Claude Code (R4 Comparison) + +- **Overall quality score**: 7.0 vs 5.8 +- **Test coverage**: 2.9x more test lines (0.79:1 vs 0.26:1 test-to-code ratio) +- **Fewer bugs**: Fewer consensus bugs, fewer High severity issues +- **Deployment readiness**: Codev produces Dockerfile, .dockerignore, deploy README. Claude Code produces none. +- **Architecture**: Clean three-layer separation vs single-flow mixing parsing and execution + +## 14-Day Production Sprint + +- **106 PRs merged** in 14 days (53/week) +- **85% fully autonomous** — builders completed without human intervention +- **24 pre-merge bug catches**, 4 security-critical +- **$1.59 per PR** ($168.64 total), 3.4x ROI +- **66% of bugfixes** ship in under 30 minutes + +## Multi-Model Review Catches What Single-Model Misses + +- **Codex**: Security edge cases +- **Claude**: Runtime semantics +- **Gemini**: Architecture problems +- Three independent reviewers with complementary blind spots diff --git a/marketing/slides-intro.html b/marketing/slides-intro.html new file mode 100644 index 00000000..f4562294 --- /dev/null +++ b/marketing/slides-intro.html @@ -0,0 +1,246 @@ + + + + + +codev — Demo Intro + + + + + +
+
+ codev + codev +
+ +

Productive Human-AI Co-development
on Large Codebases: Demo

+ +

Waleed Kadous

+
+ + +
+

Vibe coding is not production coding

+ + +
+ + +
+

We used Codev to build Codev

+ +

~80K lines of TypeScript. 289 source files. 14 days.

+ + +
+ + +
+
+ codev + codev +
+ +

Let me show you

+
+ + + +
1 / 4
+ + + + + diff --git a/marketing/why-pay-attention.md b/marketing/why-pay-attention.md new file mode 100644 index 00000000..9b750bd1 --- /dev/null +++ b/marketing/why-pay-attention.md @@ -0,0 +1,31 @@ +# Why You Should Pay Attention + +## When Starting a New Project + +We ran the same feature spec through Claude Code and Codev four times. Codev produces far fewer bugs, higher quality code, and manages the whole lifecycle — tests, deployment artifacts, and PRs. + +| Dimension | Claude Code | Codev | Delta | +|-----------|:----------:|:-----:|:-----:| +| **Bugs** | 6.7 | 7.3 | +0.7 | +| **Code Quality** | 7.0 | 7.7 | +0.7 | +| **Maintainability** | 7.3 | 7.3 | 0.0 | +| **Tests** | 5.0 | 6.7 | +1.7 | +| **Extensibility** | 5.7 | 6.3 | +0.7 | +| **NL Interface** | 6.3 | 6.7 | +0.3 | +| **Deployment** | 2.7 | 6.7 | +4.0 | +| **Overall** | **5.8** | **7.0** | **+1.2** | + +Scored by three independent AI reviewers (Claude, Codex, Gemini). Full methodology in the R4 comparison report. + +**No free lunch**: Codev took ~56 minutes vs ~15 minutes for Claude Code, and cost 3-5x more ($14-19 vs $4-7) due to multi-model review overhead. The question is whether the quality delta is worth it for your project. + +## When Maintaining Large Codebases + +We used Codev to build Codev — an ~80K-line TypeScript codebase across 289 source files. + +- **106 PRs merged in 14 days** — 85% completed fully autonomously, no human intervention +- **20 bugs caught before merge** by multi-model review, including 1 security-critical +- **Extensive architectural documentation**, including an accessible `arch.md` that stays current as the codebase evolves +- **Allowed us to ship 26 features in two weeks** — from custom session management to full workspace orchestration +- **Equivalent throughput of 3-4 elite engineers** at $1.59 per PR ($168.64 total for the sprint) +- But with everything you'd expect of production grade: testing, PRs, clear docs, and multi-model code review on every change diff --git a/packages/codev/dashboard/__tests__/analytics.test.tsx b/packages/codev/dashboard/__tests__/analytics.test.tsx index b8c436d9..ce1c2657 100644 --- a/packages/codev/dashboard/__tests__/analytics.test.tsx +++ b/packages/codev/dashboard/__tests__/analytics.test.tsx @@ -27,10 +27,9 @@ function makeStats(overrides: Partial = {}): AnalyticsRespons timeRange: '7d', activity: { prsMerged: 12, - avgTimeToMergeHours: 3.5, + medianTimeToMergeHours: 3.5, issuesClosed: 6, - avgTimeToCloseBugsHours: 1.2, - activeBuilders: 2, + medianTimeToCloseBugsHours: 1.2, projectsByProtocol: { spir: { count: 3, avgWallClockHours: 48.2 }, bugfix: { count: 2, avgWallClockHours: 1.5 }, @@ -240,10 +239,9 @@ describe('AnalyticsView', () => { const stats = makeStats({ activity: { prsMerged: 3, - avgTimeToMergeHours: null, + medianTimeToMergeHours: null, issuesClosed: 0, - avgTimeToCloseBugsHours: null, - activeBuilders: 0, + medianTimeToCloseBugsHours: null, projectsByProtocol: {}, }, }); diff --git a/packages/codev/dashboard/src/components/AnalyticsView.tsx b/packages/codev/dashboard/src/components/AnalyticsView.tsx index 986b85d3..38b342b7 100644 --- a/packages/codev/dashboard/src/components/AnalyticsView.tsx +++ b/packages/codev/dashboard/src/components/AnalyticsView.tsx @@ -129,9 +129,8 @@ function ActivitySection({ activity, errors }: { activity: AnalyticsResponse['ac - - - + + ); diff --git a/packages/codev/dashboard/src/lib/api.ts b/packages/codev/dashboard/src/lib/api.ts index b729c72c..14f3bf0d 100644 --- a/packages/codev/dashboard/src/lib/api.ts +++ b/packages/codev/dashboard/src/lib/api.ts @@ -152,10 +152,9 @@ export interface AnalyticsResponse { timeRange: '24h' | '7d' | '30d' | 'all'; activity: { prsMerged: number; - avgTimeToMergeHours: number | null; + medianTimeToMergeHours: number | null; issuesClosed: number; - avgTimeToCloseBugsHours: number | null; - activeBuilders: number; + medianTimeToCloseBugsHours: number | null; projectsByProtocol: Record; }; consultation: { diff --git a/packages/codev/src/agent-farm/__tests__/analytics.test.ts b/packages/codev/src/agent-farm/__tests__/analytics.test.ts index e2ba2c16..1d6b31ba 100644 --- a/packages/codev/src/agent-farm/__tests__/analytics.test.ts +++ b/packages/codev/src/agent-farm/__tests__/analytics.test.ts @@ -232,14 +232,14 @@ describe('computeAnalytics', () => { ]), }); - const result = await computeAnalytics('/tmp/workspace', '7', 3); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result.timeRange).toBe('7d'); expect(result.activity.prsMerged).toBe(2); - expect(result.activity.avgTimeToMergeHours).toBeCloseTo(30); // (36+24)/2 + expect(result.activity.medianTimeToMergeHours).toBeCloseTo(30); // median of [24, 36] = 30 expect(result.activity.issuesClosed).toBe(2); - expect(result.activity.avgTimeToCloseBugsHours).toBeCloseTo(84); // 3.5 days for bug only - expect(result.activity.activeBuilders).toBe(3); + expect(result.activity.medianTimeToCloseBugsHours).toBeCloseTo(84); // 3.5 days for bug only (single item) + expect(result.activity).not.toHaveProperty('activeBuilders'); // Protocol breakdown now includes count + avgWallClockHours (no "on it" → falls back to PR times) expect(result.activity.projectsByProtocol.spir).toEqual({ count: 1, avgWallClockHours: expect.closeTo(36) }); expect(result.activity.projectsByProtocol.aspir).toEqual({ count: 1, avgWallClockHours: expect.closeTo(24) }); @@ -262,7 +262,7 @@ describe('computeAnalytics', () => { it('does not have github or builders top-level keys', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result).not.toHaveProperty('github'); expect(result).not.toHaveProperty('builders'); expect(result).toHaveProperty('activity'); @@ -270,31 +270,31 @@ describe('computeAnalytics', () => { it('does not have costByProject in consultation', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result.consultation).not.toHaveProperty('costByProject'); }); it('returns 24h label for range "1"', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - const result = await computeAnalytics('/tmp/workspace', '1', 0); + const result = await computeAnalytics('/tmp/workspace', '1'); expect(result.timeRange).toBe('24h'); }); it('returns 30d label for range "30"', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - const result = await computeAnalytics('/tmp/workspace', '30', 0); + const result = await computeAnalytics('/tmp/workspace', '30'); expect(result.timeRange).toBe('30d'); }); it('returns all label for range "all"', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - const result = await computeAnalytics('/tmp/workspace', 'all', 0); + const result = await computeAnalytics('/tmp/workspace', 'all'); expect(result.timeRange).toBe('all'); }); it('passes null since date for "all" range', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - await computeAnalytics('/tmp/workspace', 'all', 0); + await computeAnalytics('/tmp/workspace', 'all'); const prCall = execFileMock.mock.calls.find( (c: unknown[]) => (c[1] as string[]).includes('merged'), @@ -305,7 +305,7 @@ describe('computeAnalytics', () => { it('passes a date string for "7" range', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - await computeAnalytics('/tmp/workspace', '7', 0); + await computeAnalytics('/tmp/workspace', '7'); const prCall = execFileMock.mock.calls.find( (c: unknown[]) => (c[1] as string[]).includes('merged'), @@ -322,14 +322,14 @@ describe('computeAnalytics', () => { it('returns GitHub defaults and error when all GitHub calls fail', async () => { execFileMock.mockRejectedValue(new Error('gh not found')); - const result = await computeAnalytics('/tmp/workspace', '7', 2); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result.errors?.github).toBeDefined(); expect(result.activity.prsMerged).toBe(0); - expect(result.activity.avgTimeToMergeHours).toBeNull(); + expect(result.activity.medianTimeToMergeHours).toBeNull(); expect(result.activity.issuesClosed).toBe(0); - expect(result.activity.avgTimeToCloseBugsHours).toBeNull(); - expect(result.activity.activeBuilders).toBe(2); + expect(result.activity.medianTimeToCloseBugsHours).toBeNull(); + expect(result.activity).not.toHaveProperty('activeBuilders'); expect(result.activity.projectsByProtocol).toEqual({}); // Consultation still works expect(result.consultation.totalCount).toBe(5); @@ -342,7 +342,7 @@ describe('computeAnalytics', () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); mockSummary.mockImplementation(() => { throw new Error('DB file not found'); }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result.errors?.consultation).toBe('DB file not found'); expect(result.consultation.totalCount).toBe(0); @@ -365,17 +365,17 @@ describe('computeAnalytics', () => { successCount: 0, byModel: [], byType: [], byProtocol: [], }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); - expect(result.activity.avgTimeToMergeHours).toBeNull(); - expect(result.activity.avgTimeToCloseBugsHours).toBeNull(); + expect(result.activity.medianTimeToMergeHours).toBeNull(); + expect(result.activity.medianTimeToCloseBugsHours).toBeNull(); expect(result.consultation.avgLatencySeconds).toBeNull(); expect(result.consultation.successRate).toBeNull(); }); // --- Bug-only avg time to close --- - it('only counts bug-labeled issues for avgTimeToCloseBugsHours', async () => { + it('only counts bug-labeled issues for medianTimeToCloseBugsHours', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: JSON.stringify([ @@ -384,8 +384,8 @@ describe('computeAnalytics', () => { ]), }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); - expect(result.activity.avgTimeToCloseBugsHours).toBeCloseTo(24); + const result = await computeAnalytics('/tmp/workspace', '7'); + expect(result.activity.medianTimeToCloseBugsHours).toBeCloseTo(24); }); // --- costByModel derivation --- @@ -400,7 +400,7 @@ describe('computeAnalytics', () => { ], }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result.consultation.costByModel).toEqual({ codex: 3.50 }); }); @@ -417,7 +417,7 @@ describe('computeAnalytics', () => { closedIssues: '[]', }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result.activity.projectsByProtocol.spir?.count).toBe(2); expect(result.activity.projectsByProtocol.spir?.avgWallClockHours).toBeCloseTo(24); expect(result.activity.projectsByProtocol.air?.count).toBe(1); @@ -435,7 +435,7 @@ describe('computeAnalytics', () => { 42: '2026-02-10T06:00:00Z', }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); // Wall clock should be mergedAt - onIt = 30 hours (not 24 from PR createdAt) expect(result.activity.projectsByProtocol.bugfix?.avgWallClockHours).toBeCloseTo(30); }); @@ -448,7 +448,7 @@ describe('computeAnalytics', () => { closedIssues: '[]', }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); // No "on it" → uses PR createdAt → mergedAt = 24 hours expect(result.activity.projectsByProtocol.bugfix?.avgWallClockHours).toBeCloseTo(24); }); @@ -463,7 +463,7 @@ describe('computeAnalytics', () => { closedIssues: '[]', }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(Object.keys(result.activity.projectsByProtocol)).toEqual(['bugfix']); expect(result.activity.projectsByProtocol.bugfix?.count).toBe(1); }); @@ -476,14 +476,14 @@ describe('computeAnalytics', () => { closedIssues: '[]', }); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result.activity.projectsByProtocol).toEqual({}); }); it('returns empty projectsByProtocol when GitHub fails', async () => { execFileMock.mockRejectedValue(new Error('gh not found')); - const result = await computeAnalytics('/tmp/workspace', '7', 0); + const result = await computeAnalytics('/tmp/workspace', '7'); expect(result.activity.projectsByProtocol).toEqual({}); }); @@ -492,8 +492,8 @@ describe('computeAnalytics', () => { it('returns cached result on second call within TTL', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - const result1 = await computeAnalytics('/tmp/workspace', '7', 3); - const result2 = await computeAnalytics('/tmp/workspace', '7', 3); + const result1 = await computeAnalytics('/tmp/workspace', '7'); + const result2 = await computeAnalytics('/tmp/workspace', '7'); expect(result1).toBe(result2); expect(mockSummary).toHaveBeenCalledTimes(1); @@ -502,8 +502,8 @@ describe('computeAnalytics', () => { it('bypasses cache when refresh=true', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - await computeAnalytics('/tmp/workspace', '7', 3); - await computeAnalytics('/tmp/workspace', '7', 3, true); + await computeAnalytics('/tmp/workspace', '7'); + await computeAnalytics('/tmp/workspace', '7', true); expect(mockSummary).toHaveBeenCalledTimes(2); }); @@ -511,8 +511,8 @@ describe('computeAnalytics', () => { it('does not share cache between different ranges', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - await computeAnalytics('/tmp/workspace', '7', 3); - await computeAnalytics('/tmp/workspace', '30', 3); + await computeAnalytics('/tmp/workspace', '7'); + await computeAnalytics('/tmp/workspace', '30'); expect(mockSummary).toHaveBeenCalledTimes(2); }); @@ -522,7 +522,7 @@ describe('computeAnalytics', () => { it('passes workspace filter to MetricsDB.summary()', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - await computeAnalytics('/tmp/my-workspace', '7', 0); + await computeAnalytics('/tmp/my-workspace', '7'); expect(mockSummary).toHaveBeenCalledWith( expect.objectContaining({ workspace: '/tmp/my-workspace' }), @@ -532,7 +532,7 @@ describe('computeAnalytics', () => { it('passes workspace filter for all time ranges', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - await computeAnalytics('/tmp/workspace-a', 'all', 0); + await computeAnalytics('/tmp/workspace-a', 'all'); expect(mockSummary).toHaveBeenCalledWith( expect.objectContaining({ workspace: '/tmp/workspace-a' }), @@ -542,8 +542,8 @@ describe('computeAnalytics', () => { it('different workspaces get different cache entries', async () => { mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); - await computeAnalytics('/tmp/workspace-a', '7', 0); - await computeAnalytics('/tmp/workspace-b', '7', 0); + await computeAnalytics('/tmp/workspace-a', '7'); + await computeAnalytics('/tmp/workspace-b', '7'); expect(mockSummary).toHaveBeenCalledTimes(2); expect(mockSummary).toHaveBeenCalledWith( @@ -554,6 +554,65 @@ describe('computeAnalytics', () => { ); }); + // --- Regression: median instead of average (#548) --- + + it('uses median (not average) for time-to-merge with outliers', async () => { + // 3 PRs: 2h, 3h, 100h — average=35h, median=3h + mockGhOutput({ + mergedPRs: JSON.stringify([ + { number: 1, title: 'PR', createdAt: '2026-02-10T00:00:00Z', mergedAt: '2026-02-10T02:00:00Z', body: '', headRefName: 'main' }, + { number: 2, title: 'PR', createdAt: '2026-02-10T00:00:00Z', mergedAt: '2026-02-10T03:00:00Z', body: '', headRefName: 'main' }, + { number: 3, title: 'PR', createdAt: '2026-02-10T00:00:00Z', mergedAt: '2026-02-14T04:00:00Z', body: '', headRefName: 'main' }, + ]), + closedIssues: '[]', + }); + + const result = await computeAnalytics('/tmp/workspace', '7'); + // Median of [2, 3, 100] = 3 (middle value) + expect(result.activity.medianTimeToMergeHours).toBeCloseTo(3); + }); + + it('uses median (not average) for bug close time with outliers', async () => { + // 3 bugs: 1h, 2h, 200h — average=67.67h, median=2h + mockGhOutput({ + mergedPRs: '[]', + closedIssues: JSON.stringify([ + { number: 1, title: 'Bug 1', createdAt: '2026-02-10T00:00:00Z', closedAt: '2026-02-10T01:00:00Z', labels: [{ name: 'bug' }] }, + { number: 2, title: 'Bug 2', createdAt: '2026-02-10T00:00:00Z', closedAt: '2026-02-10T02:00:00Z', labels: [{ name: 'bug' }] }, + { number: 3, title: 'Bug 3', createdAt: '2026-02-10T00:00:00Z', closedAt: '2026-02-18T08:00:00Z', labels: [{ name: 'bug' }] }, + ]), + }); + + const result = await computeAnalytics('/tmp/workspace', '7'); + // Median of [1, 2, 200] = 2 (middle value) + expect(result.activity.medianTimeToCloseBugsHours).toBeCloseTo(2); + }); + + it('computes median correctly for even number of items', async () => { + // 4 PRs: 1h, 2h, 10h, 20h — median = (2+10)/2 = 6 + mockGhOutput({ + mergedPRs: JSON.stringify([ + { number: 1, title: 'PR', createdAt: '2026-02-10T00:00:00Z', mergedAt: '2026-02-10T01:00:00Z', body: '', headRefName: 'main' }, + { number: 2, title: 'PR', createdAt: '2026-02-10T00:00:00Z', mergedAt: '2026-02-10T02:00:00Z', body: '', headRefName: 'main' }, + { number: 3, title: 'PR', createdAt: '2026-02-10T00:00:00Z', mergedAt: '2026-02-10T10:00:00Z', body: '', headRefName: 'main' }, + { number: 4, title: 'PR', createdAt: '2026-02-10T00:00:00Z', mergedAt: '2026-02-10T20:00:00Z', body: '', headRefName: 'main' }, + ]), + closedIssues: '[]', + }); + + const result = await computeAnalytics('/tmp/workspace', '7'); + // Median of [1, 2, 10, 20] = (2+10)/2 = 6 + expect(result.activity.medianTimeToMergeHours).toBeCloseTo(6); + }); + + // --- Regression: activeBuilders removed (#548) --- + + it('does not include activeBuilders in response', async () => { + mockGhOutput({ mergedPRs: '[]', closedIssues: '[]' }); + const result = await computeAnalytics('/tmp/workspace', '7'); + expect(result.activity).not.toHaveProperty('activeBuilders'); + }); + }); // --------------------------------------------------------------------------- diff --git a/packages/codev/src/agent-farm/__tests__/tower-routes.test.ts b/packages/codev/src/agent-farm/__tests__/tower-routes.test.ts index ce1df3e3..e70734cb 100644 --- a/packages/codev/src/agent-farm/__tests__/tower-routes.test.ts +++ b/packages/codev/src/agent-farm/__tests__/tower-routes.test.ts @@ -1017,7 +1017,7 @@ describe('tower-routes', () => { describe('GET /api/analytics', () => { const fakeStats = { timeRange: '7d', - activity: { prsMerged: 5, avgTimeToMergeHours: 2.5, issuesClosed: 4, avgTimeToCloseBugsHours: 1.2, activeBuilders: 1, projectsByProtocol: { spir: { count: 2, avgWallClockHours: 36 }, bugfix: { count: 1, avgWallClockHours: 2.5 } } }, + activity: { prsMerged: 5, medianTimeToMergeHours: 2.5, issuesClosed: 4, medianTimeToCloseBugsHours: 1.2, projectsByProtocol: { spir: { count: 2, avgWallClockHours: 36 }, bugfix: { count: 1, avgWallClockHours: 2.5 } } }, consultation: { totalCount: 10, totalCostUsd: 0.5, costByModel: {}, avgLatencySeconds: 12, successRate: 90, byModel: [], byReviewType: {}, byProtocol: {} }, }; @@ -1034,7 +1034,7 @@ describe('tower-routes', () => { expect(statusCode()).toBe(200); const parsed = JSON.parse(body()); expect(parsed.activity.prsMerged).toBe(5); - expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', '7', 0, false); + expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', '7', false); }); it('returns 400 for invalid range', async () => { @@ -1052,7 +1052,7 @@ describe('tower-routes', () => { const { res } = makeRes(); await handleRequest(req, res, makeCtx()); - expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', '7', 0, false); + expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', '7', false); }); it('passes refresh=true when refresh=1 query param is set', async () => { @@ -1060,7 +1060,7 @@ describe('tower-routes', () => { const { res } = makeRes(); await handleRequest(req, res, makeCtx()); - expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', '30', 0, true); + expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', '30', true); }); it('returns default empty response when no workspace is available', async () => { @@ -1074,7 +1074,7 @@ describe('tower-routes', () => { const parsed = JSON.parse(body()); expect(parsed.timeRange).toBe('30d'); expect(parsed.activity.prsMerged).toBe(0); - expect(parsed.activity.activeBuilders).toBe(0); + expect(parsed.activity).not.toHaveProperty('activeBuilders'); expect(mockComputeAnalytics).not.toHaveBeenCalled(); }); @@ -1083,7 +1083,7 @@ describe('tower-routes', () => { const { res } = makeRes(); await handleRequest(req, res, makeCtx()); - expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', 'all', 0, false); + expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', 'all', false); }); it('accepts range=1 (24h)', async () => { @@ -1091,7 +1091,7 @@ describe('tower-routes', () => { const { res } = makeRes(); await handleRequest(req, res, makeCtx()); - expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', '1', 0, false); + expect(mockComputeAnalytics).toHaveBeenCalledWith('/tmp/workspace', '1', false); }); }); }); diff --git a/packages/codev/src/agent-farm/servers/analytics.ts b/packages/codev/src/agent-farm/servers/analytics.ts index fae10314..52ba3a29 100644 --- a/packages/codev/src/agent-farm/servers/analytics.ts +++ b/packages/codev/src/agent-farm/servers/analytics.ts @@ -1,10 +1,9 @@ /** * Analytics aggregation service for the dashboard Analytics tab. * - * Aggregates data from three sources: + * Aggregates data from two sources: * - GitHub CLI (merged PRs, closed issues, protocol breakdown from branch names) * - Consultation metrics DB (~/.codev/metrics.db) - * - Active builder count (passed in from tower context) * * Each data source fails independently — partial results are returned * with error messages in the `errors` field. @@ -32,10 +31,9 @@ export interface AnalyticsResponse { timeRange: '24h' | '7d' | '30d' | 'all'; activity: { prsMerged: number; - avgTimeToMergeHours: number | null; + medianTimeToMergeHours: number | null; issuesClosed: number; - avgTimeToCloseBugsHours: number | null; - activeBuilders: number; + medianTimeToCloseBugsHours: number | null; projectsByProtocol: Record; }; consultation: { @@ -108,19 +106,22 @@ function rangeToSinceDate(range: RangeParam): string | null { // GitHub metrics computation // ============================================================================= -function computeAvgHours(items: Array<{ start: string; end: string }>): number | null { +function computeMedianHours(items: Array<{ start: string; end: string }>): number | null { if (items.length === 0) return null; - const totalMs = items.reduce((sum, item) => { - return sum + (new Date(item.end).getTime() - new Date(item.start).getTime()); - }, 0); - return totalMs / items.length / (1000 * 60 * 60); + const hours = items + .map(item => (new Date(item.end).getTime() - new Date(item.start).getTime()) / (1000 * 60 * 60)) + .sort((a, b) => a - b); + const mid = Math.floor(hours.length / 2); + return hours.length % 2 === 0 + ? (hours[mid - 1] + hours[mid]) / 2 + : hours[mid]; } interface GitHubMetrics { prsMerged: number; - avgTimeToMergeHours: number | null; + medianTimeToMergeHours: number | null; issuesClosed: number; - avgTimeToCloseBugsHours: number | null; + medianTimeToCloseBugsHours: number | null; mergedPRList: MergedPR[]; } @@ -142,8 +143,8 @@ async function computeGitHubMetrics( const prs = mergedPRs ?? []; const prsMerged = prs.length; - // Average time to merge - const avgTimeToMergeHours = computeAvgHours( + // Median time to merge + const medianTimeToMergeHours = computeMedianHours( prs.filter(pr => pr.mergedAt).map(pr => ({ start: pr.createdAt, end: pr.mergedAt })), ); @@ -151,19 +152,19 @@ async function computeGitHubMetrics( const closed = closedIssues ?? []; const issuesClosed = closed.length; - // Average time to close bugs + // Median time to close bugs const closedBugs = closed.filter(i => i.labels.some(l => l.name === 'bug') && i.closedAt, ); - const avgTimeToCloseBugsHours = computeAvgHours( + const medianTimeToCloseBugsHours = computeMedianHours( closedBugs.map(i => ({ start: i.createdAt, end: i.closedAt })), ); return { prsMerged, - avgTimeToMergeHours, + medianTimeToMergeHours, issuesClosed, - avgTimeToCloseBugsHours, + medianTimeToCloseBugsHours, mergedPRList: prs, }; } @@ -337,13 +338,11 @@ async function computeProjectsByProtocol( * * @param workspaceRoot - Path to the workspace root (used as cwd for gh CLI) * @param range - Time range: '1', '7', '30', or 'all' - * @param activeBuilders - Current active builder count (from tower context) * @param refresh - If true, bypass the cache */ export async function computeAnalytics( workspaceRoot: string, range: RangeParam, - activeBuilders: number, refresh = false, ): Promise { const cacheKey = `${workspaceRoot}:${range}`; @@ -369,9 +368,9 @@ export async function computeAnalytics( errors.github = msg; githubMetrics = { prsMerged: 0, - avgTimeToMergeHours: null, + medianTimeToMergeHours: null, issuesClosed: 0, - avgTimeToCloseBugsHours: null, + medianTimeToCloseBugsHours: null, mergedPRList: [], }; } @@ -405,10 +404,9 @@ export async function computeAnalytics( timeRange: rangeToLabel(range), activity: { prsMerged: githubMetrics.prsMerged, - avgTimeToMergeHours: githubMetrics.avgTimeToMergeHours, + medianTimeToMergeHours: githubMetrics.medianTimeToMergeHours, issuesClosed: githubMetrics.issuesClosed, - avgTimeToCloseBugsHours: githubMetrics.avgTimeToCloseBugsHours, - activeBuilders, + medianTimeToCloseBugsHours: githubMetrics.medianTimeToCloseBugsHours, projectsByProtocol, }, consultation: consultMetrics, diff --git a/packages/codev/src/agent-farm/servers/tower-routes.ts b/packages/codev/src/agent-farm/servers/tower-routes.ts index 95806014..1780eadc 100644 --- a/packages/codev/src/agent-farm/servers/tower-routes.ts +++ b/packages/codev/src/agent-farm/servers/tower-routes.ts @@ -719,18 +719,13 @@ async function handleAnalytics(res: http.ServerResponse, url: URL, workspaceOver if (!workspaceRoot) { res.writeHead(200, { 'Content-Type': 'application/json' }); - res.end(JSON.stringify({ timeRange: rangeLabel, activity: { prsMerged: 0, avgTimeToMergeHours: null, issuesClosed: 0, avgTimeToCloseBugsHours: null, activeBuilders: 0, projectsByProtocol: {} }, consultation: { totalCount: 0, totalCostUsd: null, costByModel: {}, avgLatencySeconds: null, successRate: null, byModel: [], byReviewType: {}, byProtocol: {} } })); + res.end(JSON.stringify({ timeRange: rangeLabel, activity: { prsMerged: 0, medianTimeToMergeHours: null, issuesClosed: 0, medianTimeToCloseBugsHours: null, projectsByProtocol: {} }, consultation: { totalCount: 0, totalCostUsd: null, costByModel: {}, avgLatencySeconds: null, successRate: null, byModel: [], byReviewType: {}, byProtocol: {} } })); return; } const range = rangeParam as '1' | '7' | '30' | 'all'; const refresh = url.searchParams.get('refresh') === '1'; - // Get active builder count from workspace terminals - const wsTerminals = getWorkspaceTerminals(); - const entry = wsTerminals.get(normalizeWorkspacePath(workspaceRoot)); - const activeBuilders = entry?.builders.size ?? 0; - - const data = await computeAnalytics(workspaceRoot, range, activeBuilders, refresh); + const data = await computeAnalytics(workspaceRoot, range, refresh); res.writeHead(200, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(data)); } diff --git a/packages/codev/src/commands/consult/__tests__/metrics.test.ts b/packages/codev/src/commands/consult/__tests__/metrics.test.ts index 01f3468a..b98caf9e 100644 --- a/packages/codev/src/commands/consult/__tests__/metrics.test.ts +++ b/packages/codev/src/commands/consult/__tests__/metrics.test.ts @@ -390,6 +390,31 @@ describe('Workspace filtering (#545)', () => { expect(proj).toBeDefined(); expect(proj!.totalCost).toBeCloseTo(10.00); }); + + // --- Regression: prefix match for builder worktree paths (#548) --- + + it('includes builder worktree paths that are children of the workspace', () => { + db.record(sampleRecord({ workspacePath: '/projects/codev/.builders/bugfix-535-fix', costUsd: 7.00 })); + + const summary = db.summary({ workspace: '/projects/codev' }); + // Should include the worktree record via prefix match + expect(summary.totalCount).toBe(3); // 2 existing + 1 builder worktree + }); + + it('does not include unrelated workspace paths via prefix match', () => { + db.record(sampleRecord({ workspacePath: '/projects/codev-other', costUsd: 7.00 })); + + const summary = db.summary({ workspace: '/projects/codev' }); + // Should NOT include /projects/codev-other (different workspace, not a subpath) + expect(summary.totalCount).toBe(2); // only the 2 original /projects/codev records + }); + + it('handles workspace paths with trailing slash', () => { + db.record(sampleRecord({ workspacePath: '/projects/codev/.builders/spir-42', costUsd: 4.00 })); + + const summary = db.summary({ workspace: '/projects/codev/' }); + expect(summary.totalCount).toBe(3); // 2 existing + 1 builder + }); }); // Test 8: CLI flag acceptance (--protocol, --project-id) diff --git a/packages/codev/src/commands/consult/metrics.ts b/packages/codev/src/commands/consult/metrics.ts index 9cd98d8d..abb8af67 100644 --- a/packages/codev/src/commands/consult/metrics.ts +++ b/packages/codev/src/commands/consult/metrics.ts @@ -150,8 +150,12 @@ function buildWhereClause(filters: StatsFilters): { where: string; params: Recor params.filterProject = filters.project; } if (filters.workspace) { - conditions.push('workspace_path = @filterWorkspace'); - params.filterWorkspace = filters.workspace; + // Prefix match: builder worktree paths like /repo/.builders/bugfix-42 + // should match when filtering by /repo. + const ws = filters.workspace.endsWith('/') ? filters.workspace.slice(0, -1) : filters.workspace; + conditions.push("(workspace_path = @filterWorkspace OR workspace_path LIKE @filterWorkspacePrefix)"); + params.filterWorkspace = ws; + params.filterWorkspacePrefix = ws + '/%'; } const where = conditions.length > 0 ? `WHERE ${conditions.join(' AND ')}` : '';