Skip to content

test: comprehensive testing improvements for audit commands #23960

@Mossaka

Description

@Mossaka

Parent Epic

Part of #22735

Summary

The audit feature has 12 test files (~200KB) with good unit coverage for data structures and computation, but has gaps in output rendering verification, edge case handling, and integration testing. This issue covers 4 testing improvement areas.

Area 1: Golden File Tests for Audit Output Rendering

Why: Render functions are only spot-checked with substring assertions. No snapshot testing catches unintended regressions.

Files to create:

  • pkg/cli/audit_golden_test.go
  • pkg/cli/testdata/audit_golden/*.golden

Implementation:

  • Follow pattern from pkg/workflow/wasm_golden_test.go (-update flag for regeneration)
  • Build canonical fixtures using existing createTestProcessedRun helper
  • Golden files for each render path:
    • Console render (renderConsole)
    • JSON render (renderJSON)
    • Diff markdown (renderAuditDiffMarkdown)
    • Diff pretty (renderAuditDiffPretty)
    • Diff JSON
    • Cross-run report markdown (renderCrossRunReportMarkdown)
    • Cross-run report JSON
  • Add update-audit-golden Makefile target

Area 2: Edge Case Tests for Malformed/Incomplete Data

Why: Real audit runs frequently have incomplete data (failed runs, partial uploads, GHES). Must not panic.

File to create: pkg/cli/audit_edge_cases_test.go

Test cases:

  • TestBuildAuditData_MalformedFirewallJSON — invalid JSON → graceful degradation, warning added
  • TestBuildAuditData_TruncatedLogs — log ends mid-line → parsing completes
  • TestBuildAuditData_EmptyArtifacts — directory exists, no files → MissingData populated
  • TestBuildAuditData_NilPointerSafety — nil optional fields through all render functions → no panics
  • TestComputeFirewallDiff_EmptyDomainMaps — both runs have empty maps
  • TestComputeAuditDiff_MixedNilSummaries — one run has data, other doesn't
  • TestBuildCrossRunAuditReport_SingleRunNoData — one run, no firewall artifacts

Area 3: Render Function Unit Tests

Why: renderConsole, renderAuditDiffPretty, and renderCrossRunReportMarkdown lack direct unit tests with output validation.

File to create: pkg/cli/audit_render_test.go

Test cases:

  • TestRenderConsole_AllSections — all sections populated → every section header in output
  • TestRenderConsole_EmptyData — minimal AuditData → no crash, minimum viable output
  • TestRenderAuditDiffMarkdown_EmptyDiff — verify "No changes" message
  • TestRenderAuditDiffMarkdown_AllSections — firewall, MCP, metrics sections present
  • TestRenderCrossRunReportMarkdown_AllSections — all report sections present
  • TestRenderGuardPolicySummary — block reason breakdown formatting

Area 4: Integration Test Fixtures

Why: Fills gap between unit tests (in-memory structs) and real E2E (needs API access).

Files to create:

  • pkg/cli/testdata/audit_fixtures/successful-run/aw_info.json, firewall log, agent log, policy manifest
  • pkg/cli/testdata/audit_fixtures/failed-run/ — same but with errors
  • pkg/cli/testdata/audit_fixtures/minimal-run/ — only aw_info.json
  • pkg/cli/audit_integration_test.go (//go:build integration)

Test cases:

  • TestAuditWorkflowRun_SuccessfulFixturebuildAuditData with fixture, all sections populated
  • TestAuditWorkflowRun_FailedFixture — error detection, failure analysis populated
  • TestAuditWorkflowRun_MinimalFixture — graceful handling of missing optional artifacts

Key Source Files

  • pkg/cli/audit_report.goAuditData struct
  • pkg/cli/audit_report_render.go — render functions (1045 lines)
  • pkg/cli/audit_diff.go / audit_diff_render.go — diff computation and rendering
  • pkg/cli/audit_cross_run.go / audit_cross_run_render.go — cross-run report
  • pkg/cli/audit_report_test.go — existing patterns, createTestProcessedRun helper
  • pkg/workflow/wasm_golden_test.go — golden file test pattern to follow

Verification

make test-unit                    # All new unit tests pass
go test ./pkg/cli/ -run TestAuditGolden -update  # Generate golden files
go test ./pkg/cli/ -run TestAuditGolden           # Verify golden files match
make test                         # Integration tests pass
make lint                         # No lint issues

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions