Draft
Conversation
Add a functional test that creates a large JSON-RPC batch reply (using repeated getblock verbosity=0 calls) and asserts the HTTP server logs that the large response body was copied into the libevent output buffer. This establishes baseline behavior related to bitcoin#31041 (memory pressure from large batch replies) before switching the reply path to avoid the extra copy.
Add `HTTPRequest::WriteReply(int, std::string&&)` that stores the reply body and uses `evbuffer_add_reference()` so libevent can send large responses without an extra full copy. This reduces peak memory when serving large RPC/REST replies (eg JSON-RPC batch responses implicated in bitcoin#31041). Update the functional test to assert the referenced-reply path is used for large batch replies.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes bitcoin#31041
Problem
Serving very large JSON-RPC batch responses can spike peak memory usage because the full reply body is copied into libevent’s output buffer.
This can contribute to out-of-memory termination on low-memory systems when an indexer (or other client) requests large batches.
Fix
This PR reduces peak memory by adding
HTTPRequest::WriteReply(int, std::string&&)and usingevbuffer_add_reference()so large reply bodies can be sent without an extra full copy (with a safe fallback to copying if referencing fails).It also adds a functional test that constructs a large batch reply and asserts the large-reply path switches from "copied" (baseline) to "referenced" after the change.