Skip to content

Add testing standards template (standards/testing-standards.md) #9

@nakurian

Description

@nakurian

Problem

Testing guidance is currently scattered:

  • A few lines in coding-standards.md (80% coverage, use factories)
  • Quality gate in manifest.yaml (coverage minimum)
  • Testing strategy section in SPEC.md template
  • Phase 7 review checklist

There's no dedicated, comprehensive testing standard that defines how the team tests across all service types. The builder agent makes its own decisions on test structure, mocking strategy, test data management, and CI test execution.

Proposed solution

Add standards/testing-standards.md as a default template:

# Testing Standards

## Test Pyramid
- Unit tests: 70% — business logic, pure functions, utilities
- Integration tests: 20% — API endpoints, database operations, event handlers
- E2E tests: 10% — critical user journeys only

## Naming Conventions
- File: `<module>.spec.ts` or `<module>.test.ts` (choose one, be consistent)
- Describe blocks: feature or module name
- Test names: "should [expected behavior] when [condition]"

## Unit Testing
- Test business logic in services, NOT controllers
- Mock external dependencies (HTTP clients, message brokers)
- Do NOT mock the database — use test containers or in-memory DB
- Use factories/fixtures for test data (e.g., `createTestOrder()`)
- Each test must be independent — no shared mutable state

## Integration Testing
- Test API endpoints with real database (test containers)
- Test the full request → response cycle including middleware
- Verify response shape, status codes, headers
- Test authentication/authorization scenarios
- Test pagination, filtering, sorting
- Test error responses (400, 401, 403, 404, 409, 422, 500)

## Contract Testing
- Generate consumer-driven contracts from `contracts/` directory
- Provider tests verify the API matches the contract
- Consumer tests verify the client handles responses correctly
- Run contract tests in CI before deployment
- Tool: Pact / Schema validation against OpenAPI specs

## E2E Testing
- Cover only critical user journeys (3-5 per service)
- Run against a fully composed environment (docker-compose)
- Use realistic test data (seeded, not mocked)
- Include happy path + one failure scenario per journey
- Keep fast (< 5 minutes total)

## Event/Message Testing
- Test event publishing: verify payload matches contract schema
- Test event consumption: happy path, duplicate, malformed, handler failure
- Test idempotency: sending same event twice produces same result
- Test dead letter queue: failed events are routed correctly

## Test Data Management
- Use factory functions, not inline objects
- Factories should produce valid defaults that can be overridden
- Database: seed before test suite, clean after each test (transactions)
- Never use production data in tests

## Mocking Strategy
- Mock: external HTTP APIs, third-party services, message brokers
- Don't mock: database, internal service logic, framework middleware
- Use dependency injection for easy mocking
- Prefer fakes over mocks where possible (in-memory repo vs mock repo)

## CI Integration
- All tests run on every PR
- Unit tests: run first (fast feedback)
- Integration tests: run after unit (need DB container)
- Contract tests: run after integration
- E2E tests: run on merge to main (slowest)
- Coverage report uploaded as PR comment
- Failing tests block merge

## Coverage Requirements
- Minimum: from manifest.yaml quality_gates.test_coverage_minimum
- Services handling money/auth/PII: 90% minimum
- Shared libraries: 95% minimum
- UI components: focus on interaction tests, not snapshot coverage

## Performance Testing (baseline)
- Establish response time baselines for key endpoints
- Track: p50, p95, p99 latency
- Fail CI if p95 regresses by > 20% from baseline
- Load test with realistic concurrency (from NFR in PROJECT.md)

Integration with framework

  • Phase 0 setup should ask about testing preferences and generate this file
  • Phase 3 (specify) should read this when generating the Testing Strategy section of SPEC.md
  • Phase 5 (build) should follow these rules when writing tests
  • Phase 7 (review) should check compliance against these standards
  • The spec agent should reference coverage requirements per service type (standard vs money-handling vs shared-lib)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions