Skip to content

Conversation

@chakra-guy
Copy link
Collaborator

@chakra-guy chakra-guy commented Jan 12, 2026

Summary

Adds infrastructure management for distributed load testing using DigitalOcean droplets.

What's included

CLI Commands (yarn infra)

  • create - Provision DigitalOcean droplets with Node.js, clone repo, install deps
  • destroy - Tear down all load test droplets
  • list - Show current droplets and hourly cost
  • exec - Run commands on all droplets (with --background for fire-and-forget)
  • wait - Poll for a file to exist on droplets (for async completion)
  • update - Git pull and rebuild on all droplets
  • collect - Download result files via SFTP

Results Aggregation (yarn results)

  • aggregate - Combine results from multiple droplets into a single summary

Validated

Successfully ran a 70K connection steady-state test (35k per relay server instance) (10 droplets × 7K connections, 5 min hold):

  • ✅ 100% success rate
  • ✅ 100% connection stability
  • ✅ Zero failures
  • ✅ p99 connect time: 208ms

Depends on

Screenshot 2026-01-12 at 12 31 20 Screenshot 2026-01-12 at 12 31 26 Screenshot 2026-01-12 at 12 31 47 Screenshot 2026-01-12 at 12 34 08

Note

Adds end-to-end tooling for distributed load testing across DigitalOcean droplets.

  • New infra CLI: list, create (provisions droplets, installs Node/Yarn, clones repo, builds), destroy, update (git checkout/pull/build), exec (supports --background), wait (polls for file), collect (SFTP fetch of results)
  • DigitalOcean + SSH utilities: API client (list/create/delete/waitForActive), SSH exec/SFTP download, droplet setup/status formatting, log saving, progress output, cost summary
  • Config loading: .env-based DO token and SSH settings with validation and ~ expansion
  • Results CLI: results aggregate to load JSON files and print combined connection/timing/retry statistics per droplet and totals
  • Safety/robustness: branch-name validation, shell quoting, timeouts, concurrency limits, and concise table/summary printers

Written by Cursor Bugbot for commit 2cf2124. This will update automatically on new commits. Configure here.

@chakra-guy chakra-guy force-pushed the ts/setup-load-testing branch from d833694 to 885a284 Compare January 12, 2026 12:34
@chakra-guy chakra-guy requested a review from a team January 12, 2026 15:08
ffmcgee725
ffmcgee725 previously approved these changes Jan 13, 2026
An error occurred while trying to automatically change base from ts/load-testing-scaffolding to main January 15, 2026 09:24
An error occurred while trying to automatically change base from ts/load-testing-scaffolding to main January 15, 2026 09:24
@chakra-guy chakra-guy force-pushed the ts/setup-load-testing branch from 59062fd to f75e1ab Compare January 15, 2026 09:25
Base automatically changed from ts/load-testing-scaffolding to main January 15, 2026 09:26
@chakra-guy chakra-guy dismissed ffmcgee725’s stale review January 15, 2026 09:26

The base branch was changed.

…testing

- DigitalOcean droplet provisioning (create/destroy)
- SSH-based command execution on droplets
- Parallel execution with progress tracking
- Results collection via SFTP
- Background execution for long-running tests
- Results aggregation across multiple droplets
- Fix latency aggregation bug (allLatencies was never populated)
- Rename latency → connectTime to match scaffolding PR
- Use corepack for Yarn installation instead of npm install -g yarn
- Fix SSH exit code null handling (assume failure if no exit code)
- Add concurrency limit (5) for droplet creation to avoid rate limits
- Add aggregator input validation for required fields
- Extract DROPLET_HOURLY_COST constant
- Add branch name validation to prevent shell injection (droplet.ts, infra.ts)
- Add regex escaping for name prefix to prevent incorrect droplet matching
- Add shell single-quote escaping for file paths in wait command
- Add NaN validation for timeout/interval to prevent infinite loops
- Add timeout to SFTP download to prevent hanging indefinitely
@chakra-guy chakra-guy force-pushed the ts/setup-load-testing branch from f75e1ab to fa791da Compare January 15, 2026 09:37
Add client.end() calls to error handlers in execSsh and downloadFile
functions. Previously, connection errors (e.g., ECONNREFUSED, auth
failures) would clear the timeout and reject the promise but not
explicitly close the SSH connection, potentially leaking socket
resources.
@chakra-guy chakra-guy requested a review from ffmcgee725 January 15, 2026 10:00
@chakra-guy chakra-guy merged commit 0781ff0 into main Jan 15, 2026
12 checks passed
@chakra-guy chakra-guy deleted the ts/setup-load-testing branch January 15, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants