test: add e2e tests with localstack for all scanners, and make summarization stuff work

[PhantomInTheWire]

Summary by CodeRabbit

• New Features

  • AI-powered summarization service (batch + streaming) with grouped findings, risk scores, remediation suggestions and commands; GraphQL Scan summary surfaced.

• Infrastructure

  • Local dev now includes AI microservice and LocalStack for full E2E runs; updated containers and Python runtime for the AI service.

• Tests

  • Large E2E test suite, orchestration helpers, provisioning script, and Makefile targets covering multiple AWS services and AI demo flows.

• Chores

  • Protobuf/gRPC generation, client integration, dependency/config updates, pre-commit security-scan exclusions, and removal of stale test docs.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

Adds a Summarization protobuf/gRPC API and Python FastAPI AI microservice with LLM integration, Go gRPC client and service wiring, GraphQL schema/resolver extensions, LocalStack-based E2E infra and tests, buf tooling and generation configs, and supporting build/test automation and generated code.

Changes

Cohort / File(s)	Summary
AI service build & config backend/ai/Dockerfile, backend/ai/pyproject.toml, .pre-commit-config.yaml, .github/workflows/ci.yml	Dockerfile bumped to python:3.13-slim with uvicorn entrypoint and uv caching; pyproject adds grpc/protobuf/openai and dev tools, ruff/mypy adjustments; gosec pre-commit rule updated to also exclude G103 and exclude-dir=internal/grpc; CI security scan step removed.
Generated Python gRPC backend/ai/app/grpc_gen/__init__.py, backend/ai/app/grpc_gen/summarization_pb2.py, backend/ai/app/grpc_gen/summarization_pb2_grpc.py	New generated Python protobuf and gRPC modules for SummarizationService (messages, enums, stubs, servicer scaffolding, serializers/deserializers).
AI service application backend/ai/app/main.py, backend/ai/app/services/__init__.py, backend/ai/app/services/summarization.py	FastAPI app with lifespan that starts a background gRPC server; LLMClient with rotation, retries, and fallbacks; SummarizationServicer implementing SummarizeFindings and StreamSummarizeFindings and remediation command generation.
Proto & buf config proto/summarization.proto, buf.yaml, buf.gen.yaml	New Summarization proto (messages, enums, RPCs) and Buf config to generate Go and Python protobuf/gRPC code and apply lint/breaking policies.
Go summarization client & tests backend/api/internal/summarization/client.go, backend/api/internal/summarization/client_test.go	New gRPC client to call AI service, converters between local scanner types and protobuf, plus unit tests for conversion/response mapping.
Security service integration backend/api/internal/security/service.go, backend/api/internal/scanner/scanner.go	New Service wrapper to run scans and optionally request summarization; adds ScanSummary, FindingGroupSummary, ActionItemSummary and ScanResultWithSummary plus conversion logic.
GraphQL schema & resolvers backend/api/graph/schema.graphqls, backend/api/graph/model/models_gen.go, backend/api/graph/generated.go, backend/api/graph/schema.resolvers.go, backend/api/graph/resolver.go	Schema: added Scan.summary and new summary types; generated models/resolvers extended; Resolver gains Security and ScanResults fields; StartScan updated to run scans and expose mapped ScanSummary.
E2E infra & orchestration infra/docker-compose.yml, backend/api/e2e/setup.go, backend/api/e2e/docker-compose.yml, backend/api/Makefile	Added LocalStack service and ai-service to infra compose; LocalStack-aware client factory and lifecycle helpers; Makefile targets for E2E workflows and orchestration (including AI demo).
E2E tests & provisioning backend/api/e2e/*.go (ec2_test.go, s3_test.go, iam_test.go, lambda_test.go, dynamodb_test.go, security_service_test.go, graphql_test.go), backend/api/e2e/setup-misconfigs.sh	New LocalStack-based E2E tests for EC2/S3/IAM/Lambda/DynamoDB/security/graphql StartScan; provisioning script to create misconfigured resources and test helpers.
Go module & deps backend/api/go.mod	Added/updated Go dependencies for gRPC and protobuf generation.
GraphQL generated artifacts backend/api/graph/generated.go, backend/api/graph/model/models_gen.go	Generated GraphQL code extended with ActionItemSummary, FindingGroupSummary, ScanSummary types and complexity hooks; new resolver signature for Scan.Summary.
Docs & cleanup backend/api/TESTS_COMPLETED.md, backend/api/TEST_SUMMARY.md, README.md	Removed two generated test-summary docs; added README Development & Testing section describing E2E AI demo and commands.
Buf tooling files buf.gen.yaml, buf.yaml	Added Buf generation and linting configuration for multi-language codegen.

Sequence Diagram(s)

sequenceDiagram
    participant Client as GraphQL Client
    participant API as Backend API (GraphQL)
    participant Svc as Security Service
    participant Coord as Scanner Coordinator
    participant Scanner as Service Scanner
    participant SumCli as Summarization Client (Go)
    participant AI as AI Service (gRPC)
    participant LLM as LLM Provider

    Client->>API: StartScan(accountID, services, regions)
    API->>Svc: Scan(ctx, config)
    Svc->>Coord: Coordinator.Scan()
    Coord->>Scanner: Scanner.Scan()
    Scanner-->>Coord: []Finding
    Coord-->>Svc: ScanResult

    alt summarization enabled and failures exist
        Svc->>SumCli: SummarizeFindings(scanID, accountID, findings)
        SumCli->>AI: gRPC SummarizeFindings(Request)
        AI->>LLM: prompt -> completion
        LLM-->>AI: summary & commands
        AI-->>SumCli: SummarizeFindingsResponse
        SumCli-->>Svc: SummaryResult
        Svc->>Svc: convertSummaryResult -> attach to ScanResultWithSummary
    end

    Svc-->>API: ScanResultWithSummary
    API-->>Client: Scan { findings, summary }

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45–75 minutes

• Areas needing careful review:
  • backend/ai/app/services/summarization.py — LLMClient (model rotation, retries, fallback logic), prompt construction and error handling.
  • proto ↔ generated code and buf config: proto/summarization.proto, buf.gen.yaml, generated Python and Go artifacts.
  • gRPC client/server behavior and streaming semantics: backend/api/internal/summarization/client.go, backend/ai/app/grpc_gen/*.
  • Type conversions between summarization results and internal scanner types: backend/api/internal/security/service.go, backend/api/internal/scanner/scanner.go.
  • E2E test reliability and LocalStack orchestration, timing and cleanup in backend/api/e2e/*, infra/docker-compose.yml.
  • GraphQL resolver lifecycle and concurrent ephemeral ScanResults handling: backend/api/graph/schema.resolvers.go, backend/api/graph/resolver.go.

Poem

🐇 I nibbled proto crumbs at dawn,
grouped findings on a dewy lawn.
The LLM hummed fixes in gentle tone,
tests hopped in, infra and code grown.
A rabbit cheers — new features sown!

---

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Free

📥 Commits

Reviewing files that changed from the base of the PR and between d312157 and 10c0952.

📒 Files selected for processing (1)

• .github/workflows/ci.yml

---

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
23584316	Triggered	Generic Password	7bcee4f	backend/api/e2e/lambda_test.go	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}