feat(rpc): add webzockets dependency for json rpc websockets by jbuckmccready · Pull Request #1222 · Syndica/sig

jbuckmccready · 2026-02-09T18:36:41Z

Summary

Adds webzockets package to path src/rpc/webzockets. webzockets implements websockets in zig, built on top of libxev for network IO. See src/rpc/webzockets/README.md for details on the package itself.

This is only adds a websocket implementation needed to implement the json RPC websocket API. The json RPC API will be in a different PR.

Changes

Updated the libxev version used by sig in build.zig.zon to new commit (current HEAD of zig-0.14 branch): Syndica/libxev@11a00ee
This commit is on top of a slightly later version of libxev than we were previously using that is still compatible with zig 0.14.1, and the commit fixes error returns for the kqueue backend (associated pull request upstream: mitchellh/libxev#207).

Updated build.zig and build.zig.zon with webzockets dependency, and added import to src/rpc/lib.zig so webzockets is part of the build.

Ran docs/generate.py so webzockets README.md is included in the published docs.

Updated scripts/style.py to skip cache dirs.

Update 2026-02-13:

Fixed another bug in libxev where io_uring gave unexpected error on shutdown when socket already closed.
Upstream PR: mitchellh/libxev#211
Syndica fork commit: Syndica/libxev@c16162f
NOTE: for kqueue/epoll backends it could just issue std.posix.shutdown directly to avoid the event loop processing entirely, but io_uring backend actually queues the shutdown command to avoid a syscall.

Update 2026-02-14:

Added read pause/resume (0028053)

Added heuristic for compaction to avoid small TCP reads (e.g., want to avoid a full TCP read cycle through IO loop if there is only 46 bytes remaining in the read buffer but compaction would free 8KB).
Added ability for connections to pause reads to exert back pressure on the TCP connection. This is helpful for future jrpc layer to allow server handlers to not have any queue of received messages by having the control flow: receive sub/unsub request from client -> pause reads -> send sub/unsub response -> write complete -> resume reads.

Removed BufferPool (51641cc), it will not be used by the jrpc server (embedded read buffer will cover sub/unsub requests) and adds complexity.

Update 2026-02-19:

Optimized close initiated by caller to not wait for handshake for the close frame response (not required by websockets RFC) and simplified logic a bit for closing. Commit: 19cf287

Update 2026-02-22:

Added sendRaw to api to allow the caller to prepare the web socket frames (header + payload), this avoids the 2 sends per message required for zero-copy use (one for header, one for payload) without having to change the backend to support vectored writes. Jrpc broadcasting benchmarks show about 50% increase in performance with this small change.

Note even with vectored writes this API is useful as it allows the caller to batch an arbitrary number of websocket messages into a single TCP write buffer, avoids creating the frame on the IO loop, and can support the caller performing per-message deflate compression in the future (allowing for shared broadcasting across web socket clients utilizing per-message deflate).

Update 2026-02-24:

Fix missing timeout on handshake upgrade: 3a7db8f
Needed otherwise TCP socket can stay open indefinitely and not release associated resources.

- Add webzockets as a local path dependency in build.zig.zon - Wire up webzockets module in build.zig - Re-export webzockets module from src/rpc/lib.zig to include in build - Update libxev dependency with fix for error returns (compatible with zig 0.14): Syndica/libxev@11a00ee

codecov · 2026-02-09T19:32:47Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Not particularly useful and takes up a lot of space in the README.md.

- Define test_webzockets job for Linux environment - Define test_webzockets_macos job for macOS environment - Update workflows to run webzockets tests on both platforms

- add RawClient wait helpers for message type, close frames, and disconnects - make read timeout configurable and default to 2s to reduce jitter flakes - update close, fragmentation, handshake, and timeout tests to use bounded waits

src/rpc/webzockets/src/buffer.zig

src/rpc/webzockets/autobahn/client/client.zig

Sobeston · 2026-02-10T21:22:50Z

src/rpc/webzockets/src/reader.zig

+/// Manages read buffer state with automatic tier escalation:
+/// - Starts with an embedded buffer for small messages
+/// - Upgrades to pooled or dynamic buffer when needed
+/// - Retains larger buffer for subsequent messages (performance optimization)
+/// - Restores to embedded only on explicit reset()


Not convinced that we need any of this

Ties in with #1222 (comment)

Removed buffer pool, commit: 51641cc

I would tend to agree with the original comment here, I don't quite understand why we need this.

Comments have been cleaned up. We need a read buffer of some kind, what is the suggested change here?

With the merge of the 0.15 update, I think it would be worth looking at trying to use std.Io.Reader instead, and simplifying some of the requirements. My gut instinct is that most of what we actually need can be expressed through that interface.

src/rpc/webzockets/src/server/server.zig

src/rpc/webzockets/src/mask.zig

src/rpc/webzockets/src/root.zig

Also removes need for pool_buffer_size field in Reader.

- remove pending_data from client and server write state - add outstandingUserWrite helper for busy, defer, and cleanup checks - clear header_len in finishWrite before invoking onWriteComplete

Update libxev dependency hash in both root and webzockets build.zig.zon

…ertNoLeaks with check() - Rename fd_leak.zig to fd_leak_detector.zig; import is now the module directly - Replace assertNoLeaks() (panics inline) with check() returning a Result enum - Call sites use: defer std.testing.expect(fd_check.check() == .ok) catch @Panic("FD leak") - Leak details logged via std.log.err instead of embedded in panic message

@intcast

- Replace manual crypto random seed with std.testing.random_seed in stress test - Replace std.crypto.random.bytes with seeded PRNG in raw_client WebSocket key generation - Remove redundant default_read_buf_size constant, inline the value - Remove unnecessary @intcast on timeval fields

…G in e2e clients - Add RawClient.rng field seeded at connect time; use it for mask key generation - Seed TestEnv ClientMaskPRNG from std.testing.random_seed instead of std.crypto.random

- Change loop and server fields from pointers to inline values - Remove heap allocations and matching destroy calls for both fields

- Remove comptime frameStatic and calculateFrameLen utilities with all associated tests - Extract anonymous fuzz test structs into named helpers for clarity

@Splat

…d struct and @Splat - Replace required-header bitmask constants with FoundRequired packed struct in http.zig - Replace "x" ** N string literal patterns with typed [N]u8 = @Splat(...) in tests

- Add sendRaw() to both server and client connections for sending pre-built frame data as a single write with no validation - Use header_len=1 as sentinel to distinguish raw from normal deferred writes (1 is impossible for real frame headers) - Add server and client e2e tests for text, binary, 16-bit length, and batched raw sends - Update README with sendRaw API and buffer lifetime

Also avoid panic in tests so they continue running and just std.log.err for failures.

Test runner code for Zig 0.14.1 indicates it should reset std.testing.log_level between each test but the testing level holds across all tests (bug). Removed comments for behavior that may change/break in the future.

- Add upgrade_timeout_ms config to client and server with 10_000ms default - Enforce absolute handshake deadlines in client/server handshake state machines - Cancel active I/O safely before terminal callbacks to avoid completion races - Add e2e coverage for stalled and partial upgrade responses and update docs

- std.DoublyLinkedList is no longer generic; use @fieldParentPtr to recover connection from node - std.time.sleep moved to std.Thread.sleep - std.net.Address format specifier {} is now ambiguous; use {f} - std.ArrayList drops stored allocator; init() removed, use .empty; deinit/append now take allocator

jbuckmccready requested review from dnut, ultd and yewman as code owners February 9, 2026 18:36

github-project-automation bot added this to Sig Feb 9, 2026

github-project-automation bot moved this to 🏗 In progress in Sig Feb 9, 2026

jbuckmccready self-assigned this Feb 9, 2026

jbuckmccready requested a review from kprotty February 9, 2026 18:36

jbuckmccready force-pushed the jbuckmccready/webzockets branch from 87e052b to 35e2914 Compare February 9, 2026 18:40

jbuckmccready force-pushed the jbuckmccready/webzockets branch from 35e2914 to 8e6b2f6 Compare February 9, 2026 19:09

jbuckmccready requested review from InKryption and Rexicon226 February 9, 2026 19:10

jbuckmccready added 7 commits February 9, 2026 11:44

docs(webzockets): remove module dependencies from README.md

79fb4b2

Not particularly useful and takes up a lot of space in the README.md.

fix: run docs/generate.py

10c03c2

ci(webzockets): add Linux and macOS test jobs

9c3e188

- Define test_webzockets job for Linux environment - Define test_webzockets_macos job for macOS environment - Update workflows to run webzockets tests on both platforms

test(webzockets): improve null value diagnostics and fix flakey tests

94d60ef

fix(style): skip cache directories in style script

9d058e6

test(webzockets): stabilize e2e frame and close waits

12ea7ce

- add RawClient wait helpers for message type, close frames, and disconnects - make read timeout configurable and default to 2s to reduce jitter flakes - update close, fragmentation, handshake, and timeout tests to use bounded waits

test(webzockets): handle NoResponse on close frame wait

8bb390a

Sobeston reviewed Feb 10, 2026

View reviewed changes

jbuckmccready added 4 commits February 10, 2026 16:03

docs(webzockets): cleanup doc comments for autobahn client

9402c82

docs(webzockets): simplify server struct field comments

aa9cb5c

test(webzockets): use gpa allocator for autobahn client

744fc9a

refactor(webzockets): pass mask keys by value in mask api

96a343e

jbuckmccready force-pushed the jbuckmccready/webzockets branch from 83cd274 to 96a343e Compare February 11, 2026 01:18

jbuckmccready added 4 commits February 10, 2026 17:32

refactor(webzockets): remove BufferPool mutex

6a75ec7

Also removes need for pool_buffer_size field in Reader.

docs(webzockets): simplify reader struct field comments

189852a

refactor(webzockets): use header_len for outstanding writes

fa599f3

- remove pending_data from client and server write state - add outstandingUserWrite helper for busy, defer, and cleanup checks - clear header_len in finishWrite before invoking onWriteComplete

chore(deps): bump libxev to fix io_uring shutdown unexpected error

279a715

Update libxev dependency hash in both root and webzockets build.zig.zon

jbuckmccready added 7 commits February 20, 2026 21:12

cleanup: simplify and improve error stack traces in client close tests

78b06d6

cleanup: remove redundant e2e category prefix in test names

b6c2264

refactor(webzockets): replace std.crypto.random with deterministic RN…

c922088

…G in e2e clients - Add RawClient.rng field seeded at connect time; use it for mask key generation - Seed TestEnv ClientMaskPRNG from std.testing.random_seed instead of std.crypto.random

refactor(webzockets): store loop and server by value in ServerRunner

874f4ac

- Change loop and server fields from pointers to inline values - Remove heap allocations and matching destroy calls for both fields

fix(webzockets): revert removal of intCast required on MacOS

f463465

jbuckmccready force-pushed the jbuckmccready/webzockets branch from 231cbaf to f463465 Compare February 21, 2026 06:33

jbuckmccready added 15 commits February 20, 2026 22:38

docs(webzockets): add doc comment to TestEnv for expected type param

72b911c

fix(webzockets): use defer for conn.deinit in simple_client example

e6b0d91

refactor(webzockets): remove calculateFrameLen and frameStatic functions

db2c03e

- Remove comptime frameStatic and calculateFrameLen utilities with all associated tests - Extract anonymous fuzz test structs into named helpers for clarity

refactor(webzockets): replace bitmasks and repeat literals with packe…

49f5cf5

…d struct and @Splat - Replace required-header bitmask constants with FoundRequired packed struct in http.zig - Replace "x" ** N string literal patterns with typed [N]u8 = @Splat(...) in tests

refactor(webzockets): fix fd_leak_detector.zig file name

4fb8fa5

refactor(webzockets): align FdLeakDetector api with gpa allocator

d5c6d40

Also avoid panic in tests so they continue running and just std.log.err for failures.

docs(webzockets): remove comments regarding test logging

a0f1a46

Test runner code for Zig 0.14.1 indicates it should reset std.testing.log_level between each test but the testing level holds across all tests (bug). Removed comments for behavior that may change/break in the future.

docs(webzockets): remove noisy unhelpful import comments

781eaf1

refactor(webzockets): simplify mask empty payload test

2dff404

chore(webzockets): remove unused imports

3ecbd6c

style(webzockets): avoid self referential key slice

a75020d

docs(webzockets): add TODOs for critical missing timeouts for handshakes

a056c41

chore(webzockets): remove unused from summarize_reports.py

76360a9

jbuckmccready force-pushed the jbuckmccready/webzockets branch 2 times, most recently from 7e0691f to 588472a Compare February 25, 2026 17:30

Merge branch 'main' into jbuckmccready/webzockets

008a8ff

jbuckmccready force-pushed the jbuckmccready/webzockets branch from 588472a to 008a8ff Compare February 25, 2026 18:06

jbuckmccready force-pushed the jbuckmccready/webzockets branch from 1f8b9da to 2af680e Compare February 25, 2026 18:30

chore(webzockets): fix autobahn client and example client for zig 0.15

60012a1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(rpc): add webzockets dependency for json rpc websockets#1222

feat(rpc): add webzockets dependency for json rpc websockets#1222
jbuckmccready wants to merge 49 commits intomainfrom
jbuckmccready/webzockets

jbuckmccready commented Feb 9, 2026 •

edited

Loading

Uh oh!

codecov bot commented Feb 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sobeston Feb 10, 2026

Uh oh!

jbuckmccready Feb 11, 2026

Uh oh!

jbuckmccready Feb 14, 2026

Uh oh!

InKryption Feb 20, 2026

Uh oh!

jbuckmccready Feb 21, 2026

Uh oh!

InKryption Feb 25, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jbuckmccready commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Update 2026-02-13:

Update 2026-02-14:

Update 2026-02-19:

Update 2026-02-22:

Update 2026-02-24:

Uh oh!

codecov bot commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sobeston Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

jbuckmccready Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

jbuckmccready Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

InKryption Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

jbuckmccready Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

InKryption Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jbuckmccready commented Feb 9, 2026 •

edited

Loading

codecov bot commented Feb 9, 2026 •

edited

Loading