Skip to content

perf: speed up CI tests with shared Docker image and more shards#161

Merged
gricha merged 3 commits intomainfrom
perf/faster-ci-tests
Feb 13, 2026
Merged

perf: speed up CI tests with shared Docker image and more shards#161
gricha merged 3 commits intomainfrom
perf/faster-ci-tests

Conversation

@gricha
Copy link
Copy Markdown
Owner

@gricha gricha commented Feb 13, 2026

Summary

CI tests are slow due to redundant Docker image builds and an unnecessarily heavy Neovim compile-from-source step. This PR fixes both.

Before (from recent successful run):

Step Shard 1 Shard 2 Shard 3
Docker build 4m 21s 3m 41s 4m 0s
Actual tests 7m 12s 4m 42s 4m 56s
Total ~12m ~9m ~9m

Changes:

  • Shared Docker image — Build once in a dedicated docker job, save as artifact, docker load in each consumer job. Saves ~4min per shard.
  • Prebuilt Neovim binary — Replace multi-stage compile-from-source (clone + cmake + build-essential + make) with official v0.11.6 prebuilt tarball (~10MB download). Shaves minutes off the Docker image build itself.
  • 5 shards instead of 3 — Better distribution of sequential test files, reducing the slowest shard bottleneck.
  • Separate test-unit job — Unit tests run in parallel without Docker overhead at all.
  • Reduced timeouts — Test/hook timeouts from 120s → 60s (still generous for Docker operations).

Test plan

  • All 5 test shards pass
  • Unit tests pass in separate job
  • E2e tests still work with loaded Docker image
  • Neovim works in workspace containers (nvim --version shows 0.11.6)
  • Compare job timing before/after

🤖 Generated with Claude Code

- Build Docker image once in a dedicated job, share via artifact
  (saves ~4min of redundant Docker builds per shard)
- Increase test shards from 3 to 5 for better distribution
- Add separate test-unit job that runs without Docker overhead
- Reduce test/hook timeouts from 120s to 60s
Comment on lines +214 to +218
- name: Download Docker image
uses: actions/download-artifact@v4
with:
name: docker-image

This comment was marked as outdated.

Neovim v0.11.6 ships official prebuilt tarballs for Linux x86_64.
Replaces the entire multi-stage build (clone + cmake + compile) with
a single curl | tar (~10MB download, seconds instead of minutes).
- Default client timeout 30s → 120s (workspace creation takes 30-60s
  with Docker operations, credential sync, and agent setup)
- Skip agent binary updates (curl install scripts) in test environments
  via SKIP_AGENT_UPDATES=true — agents are already in the Docker image
- Fixes flaky "Request timed out" failures in CI integration tests
@gricha gricha merged commit ff5746b into main Feb 13, 2026
16 checks passed
@gricha gricha deleted the perf/faster-ci-tests branch February 13, 2026 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant