-
Notifications
You must be signed in to change notification settings - Fork 322
test: comprehensive testing improvements for audit commands #23960
Description
Parent Epic
Part of #22735
Summary
The audit feature has 12 test files (~200KB) with good unit coverage for data structures and computation, but has gaps in output rendering verification, edge case handling, and integration testing. This issue covers 4 testing improvement areas.
Area 1: Golden File Tests for Audit Output Rendering
Why: Render functions are only spot-checked with substring assertions. No snapshot testing catches unintended regressions.
Files to create:
pkg/cli/audit_golden_test.gopkg/cli/testdata/audit_golden/*.golden
Implementation:
- Follow pattern from
pkg/workflow/wasm_golden_test.go(-updateflag for regeneration) - Build canonical fixtures using existing
createTestProcessedRunhelper - Golden files for each render path:
- Console render (
renderConsole) - JSON render (
renderJSON) - Diff markdown (
renderAuditDiffMarkdown) - Diff pretty (
renderAuditDiffPretty) - Diff JSON
- Cross-run report markdown (
renderCrossRunReportMarkdown) - Cross-run report JSON
- Console render (
- Add
update-audit-goldenMakefile target
Area 2: Edge Case Tests for Malformed/Incomplete Data
Why: Real audit runs frequently have incomplete data (failed runs, partial uploads, GHES). Must not panic.
File to create: pkg/cli/audit_edge_cases_test.go
Test cases:
TestBuildAuditData_MalformedFirewallJSON— invalid JSON → graceful degradation, warning addedTestBuildAuditData_TruncatedLogs— log ends mid-line → parsing completesTestBuildAuditData_EmptyArtifacts— directory exists, no files →MissingDatapopulatedTestBuildAuditData_NilPointerSafety— nil optional fields through all render functions → no panicsTestComputeFirewallDiff_EmptyDomainMaps— both runs have empty mapsTestComputeAuditDiff_MixedNilSummaries— one run has data, other doesn'tTestBuildCrossRunAuditReport_SingleRunNoData— one run, no firewall artifacts
Area 3: Render Function Unit Tests
Why: renderConsole, renderAuditDiffPretty, and renderCrossRunReportMarkdown lack direct unit tests with output validation.
File to create: pkg/cli/audit_render_test.go
Test cases:
TestRenderConsole_AllSections— all sections populated → every section header in outputTestRenderConsole_EmptyData— minimal AuditData → no crash, minimum viable outputTestRenderAuditDiffMarkdown_EmptyDiff— verify "No changes" messageTestRenderAuditDiffMarkdown_AllSections— firewall, MCP, metrics sections presentTestRenderCrossRunReportMarkdown_AllSections— all report sections presentTestRenderGuardPolicySummary— block reason breakdown formatting
Area 4: Integration Test Fixtures
Why: Fills gap between unit tests (in-memory structs) and real E2E (needs API access).
Files to create:
pkg/cli/testdata/audit_fixtures/successful-run/—aw_info.json, firewall log, agent log, policy manifestpkg/cli/testdata/audit_fixtures/failed-run/— same but with errorspkg/cli/testdata/audit_fixtures/minimal-run/— onlyaw_info.jsonpkg/cli/audit_integration_test.go(//go:build integration)
Test cases:
TestAuditWorkflowRun_SuccessfulFixture—buildAuditDatawith fixture, all sections populatedTestAuditWorkflowRun_FailedFixture— error detection, failure analysis populatedTestAuditWorkflowRun_MinimalFixture— graceful handling of missing optional artifacts
Key Source Files
pkg/cli/audit_report.go—AuditDatastructpkg/cli/audit_report_render.go— render functions (1045 lines)pkg/cli/audit_diff.go/audit_diff_render.go— diff computation and renderingpkg/cli/audit_cross_run.go/audit_cross_run_render.go— cross-run reportpkg/cli/audit_report_test.go— existing patterns,createTestProcessedRunhelperpkg/workflow/wasm_golden_test.go— golden file test pattern to follow
Verification
make test-unit # All new unit tests pass
go test ./pkg/cli/ -run TestAuditGolden -update # Generate golden files
go test ./pkg/cli/ -run TestAuditGolden # Verify golden files match
make test # Integration tests pass
make lint # No lint issues