Skip to content

Add extranonce_manager module in channels_sv2#2098

Open
GitGab19 wants to merge 5 commits intostratum-mining:mainfrom
GitGab19:extranonce-allocation
Open

Add extranonce_manager module in channels_sv2#2098
GitGab19 wants to merge 5 commits intostratum-mining:mainfrom
GitGab19:extranonce-allocation

Conversation

@GitGab19
Copy link
Copy Markdown
Member

@GitGab19 GitGab19 commented Mar 2, 2026

This PR introduces a new extranonce_manager module in channels_sv2 centered around ExtranonceAllocator

It's a unified, reusable allocator for extranonce prefixes for both standard and extended channels.

It replaces the legacy ExtendedExtranonce factory from mining_sv2 and provides a clearer, spec‑aligned, and more configurable way to size and manage extranonce space across pools, proxies (JDC/Translator), and other applications.

Internally, the allocator uses a compact bitmap (1 bit per channel) to track local_prefix_id usage, ensuring fast allocations, deterministic reuse when channels close, and no overlap between standard and extended channel prefixes.

This PR is inspired by the @coleFD's bitvec approach (see #2049), but it doesn't add any new external dependencies to our crates.

Closes #1924

companion stratum-mining/sv2-apps#310

@GitGab19 GitGab19 force-pushed the extranonce-allocation branch 5 times, most recently from b33c8be to 834b6ce Compare March 2, 2026 20:22
Comment thread sv2/channels-sv2/src/extranonce_manager.rs Outdated
Comment thread sv2/channels-sv2/src/extranonce_manager.rs Outdated
Comment thread sv2/subprotocols/mining/src/new_mining_job.rs Outdated
@GitGab19 GitGab19 force-pushed the extranonce-allocation branch from 834b6ce to 7394b40 Compare March 3, 2026 17:50
@GitGab19 GitGab19 force-pushed the extranonce-allocation branch from 7394b40 to 6038120 Compare March 8, 2026 00:14
@plebhash plebhash self-requested a review March 19, 2026 00:23
@GitGab19 GitGab19 force-pushed the extranonce-allocation branch 4 times, most recently from 39adc06 to eb4f941 Compare March 26, 2026 09:49
@plebhash
Copy link
Copy Markdown
Member

@pavlenex had some issues while trying to connect rented hashrate from Nicehash to a tProxy (exposed over internet)

image

while reviewing/testing this PR, we should make sure nicehash use-case is not blocked

Comment thread sv2/channels-sv2/src/extranonce_manager.rs Outdated
Comment thread sv2/channels-sv2/src/extranonce_manager.rs Outdated
@plebhash
Copy link
Copy Markdown
Member

plebhash commented Apr 15, 2026

the suffix *_id is being overloaded with multiple competing meanings, and it's making me pretty confused while trying to build a mental map about this new approach

for example:

  • server_id represents raw bytes
  • local_prefix_id represents an index in the BitVector
  • local_prefix_id ALSO represents the encoded bytes written into the extranonce prefix (AFAIU local_prefix_id_len follows this interpretation)

we should establish terminology that clearly separates concerns.

I suggest we rename variables to something like:

  • server_id -> server_bytes
  • local_prefix_id for BitVector indices -> local_slot or local_index
  • local_prefix_id for encoded bytes written into the extranonce prefix -> local_slot_bytes or local_index_bytes

on the suggestion above, I'm leaving *_bytes with the exclusive meaning of raw/encoded bytes, while *_slot/*_index is used exclusively for BitVector indices

what's currently described as local_prefix could remain the same, because the suffix *_prefix would be used consistently with upstream_prefix

IMO this would bring more consistency to how the new concepts map into the code, and make the code easier to reason about and maintain on the long term.

@plebhash
Copy link
Copy Markdown
Member

aside from the suggestion above, I think I got a much better understanding of this new approach

I'll make another round of review tomorrow (my clanker found some potential edge cases that could cause problems, but I need to sanity-check them before posting here)

@GitGab19 GitGab19 force-pushed the extranonce-allocation branch 2 times, most recently from 9d90704 to e590b32 Compare April 16, 2026 11:58
@GitGab19
Copy link
Copy Markdown
Member Author

@plebhash I included all your suggestions and now we have:

  • server_id -> server_bytes
  • local_prefix_id for BitVector indexes -> local_index
  • local_prefix_id for encoded bytes written into the extranonce prefix -> local_index_bytes

@GitGab19 GitGab19 force-pushed the extranonce-allocation branch 2 times, most recently from 02e4ebc to efb3a27 Compare April 16, 2026 12:39
Comment thread sv2/channels-sv2/src/extranonce_manager.rs Outdated
@plebhash
Copy link
Copy Markdown
Member

plebhash commented Apr 16, 2026

semver concern coming from my clanker:

The dependent crate version bumps should be reconsidered.

Even if these crates do not expose the specific mining_sv2 APIs that were removed, they do expose mining_sv2 types in public signatures. Moving those signatures from mining_sv2 8.0.* types to mining_sv2 9.0.* types is still a breaking public API change by Rust type identity.

With the current patch bumps, downstream users could have something like this in their Cargo.toml files:

channels_sv2 = "^4.0.0"
mining_sv2 = "^8.0.0"
# etc

and a normal cargo update could select channels_sv2 4.0.1, which now expects mining_sv2 9.0.* message types. Their code would then fail to compile because mining_sv2 8::Foo and mining_sv2 9::Foo are different Rust types, even if the actual message struct did not change.

So crates exposing mining_sv2 message types publicly should get incompatible version bumps too, otherwise we risk semver-compatible updates causing unexpected downstream compile breaks.

I think this is less of a concern if consumers only used stratum-core, because then this repo controls the internally consistent set of mining_sv2, channels_sv2, etc. versions that get re-exported.

but if anyone imports channels_sv2 + mining_sv2 + etc as seperate dependencies, it feels like a legitimate concern (and unfortunately we cannot force people to only use stratum-core instead of separate deps)


speaking more from the human side, I empathize with the psychological urge to be conservative and avoid major bumps as much as possible, especially when due to indirect changes on the dependency chain

however, I think this is more of an undesired consequence of the way SRI core crates were structured (excessive crate fragmentation), and I've somewhat accepted the tradeoff of living with frequent major bumps (in name of avoiding forcing dependency hell on downstream consumers)

@GitGab19
Copy link
Copy Markdown
Member Author

Should we start bumping MAJOR on every dependent crate though?

@plebhash
Copy link
Copy Markdown
Member

plebhash commented Apr 16, 2026

Should we start bumping MAJOR on every dependent crate though?

hmm that feels too generic...channels_sv2 does deserve a MAJOR bump, but I'd say there's exceptions.

v0.x.y crates

Here's a table my agent to put together, which after careful examination, I agree with:

Crate Current PR bump More appropriate bump Why
parsers_sv2 0.2.2 -> 0.2.3 0.2.2 -> 0.3.0 Public enums wrap mining_sv2 message types.
handlers_sv2 0.2.2 -> 0.2.3 0.2.2 -> 0.3.0 Public traits expose mining_sv2 and parsers_sv2 types.
stratum_translation 0.1.3 -> 0.1.4 0.1.3 -> 0.2.0 Public functions expose mining_sv2 types; public error also exposes a channels_sv2 error type.
stratum-core 0.2.1 -> 0.2.2 0.2.1 -> 0.3.0 Re-exports the affected crates, so its public type graph changes.

Those crates are still on v0.x.y, so they are exceptions because a breaking change can be signaled as a MINOR bump.

dependencies that don't touch public APIs

Let's say crate B depends on crate A, which is undergoing some breaking change.

The main criterion is whether B public API has any types from A. Not whether the breaking change in A leaks over B in any significant way.

If B only uses types from A without exposure via public APIs, then a PATCH bump would be fine.

More specifically with the case of this PR:

  • it does NOT matter that the breaking changes of mining_sv2 have no effect on channels_sv2 public APIs
  • it DOES matter that there are types from mining_sv2 exposed via channels_sv2 public APIs

if there were no types from mining_sv2 exposed via channels_sv2 public APIs, a PATCH bump would have been fine.


PR #2133 is formalizing these rules into CONTRIBUTING.md so we have more clarity moving forward (both for humans and agents). It's also updating RELEASE.md, which got outdated since the repo split.

@plebhash
Copy link
Copy Markdown
Member

plebhash commented Apr 16, 2026

my final review point on this PR is somewhat subjective, and therefore it's not blocking


I feel we could have better conceptual organization by splitting into submodules

  • mod.rs retains the header Rust Docs module description, const MAX_EXTRANONCE_LEN and helper bytes_needed
  • ExtranoncePrefix goes to public extranonce_manager/prefix.rs submodule
  • ExtranonceAllocator goes to public extranonce_manager/allocator.rs submodule
  • BitVector goes to private extranonce_manager/bitvector.rs submodule

commit c89e150 implements these suggestions, and could be either cherry-picked or squashed into other commits on this PR if agreed upon

if not (especially given this could also affect stratum-mining/sv2-apps#310), I think this PR is already good to go (aside from the versioning issues detailed above)

@plebhash
Copy link
Copy Markdown
Member

plebhash commented Apr 16, 2026

about the Nicehash comment above: #2098 (comment)

I haven't had time to do some actual testing with this PR + stratum-mining/sv2-apps#310, but from what I understood of this PR on a conceptual level, I don't see any major blockers

at channels_sv2::extranonce_manager, we just need to make sure that:

  • Pool can establish arbitrary sizes for Extranonce Prefix and Full Extranonce
  • if running in aggregated mode, tProxy can establish arbitrary sizes for local Extranonce Prefix (aka extranonce1) allocation, while respecting sizes for Extranonce Prefix and Full Extranonce that were imposed by Pool

both points seem sufficiently covered with these new APIs, and any potential edge cases with Sv1 extranonce1 or extranonce2 could only arise from specific combinations of configuration parameters of Pool + tProxy


TLDR I don't see any material blockers on this PR with regards to the issues observed on Nicehash, at least on a conceptual level

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improved Extranonce Prefix Generation

3 participants