Skip to content

chore(store,topo): bound store to fixed-32#8631

Open
lidatong wants to merge 1 commit intomainfrom
chali/chore/store-mem
Open

chore(store,topo): bound store to fixed-32#8631
lidatong wants to merge 1 commit intomainfrom
chali/chore/store-mem

Conversation

@lidatong
Copy link
Member

@lidatong lidatong commented Mar 2, 2026

Closes #8222

SUMMARY
              Total Tiles: 43
      Total Memory Locked: 512197939200 bytes (477 GiB + 22 MiB + 20 KiB)
  Required Gigantic Pages: 476
      Required Huge Pages: 523
    Required Normal Pages: 921
  Required Gigantic Pages (NUMA node 0): 476
      Required Huge Pages (NUMA node 0): 523

Copilot AI review requested due to automatic review settings March 2, 2026 21:26
@lidatong lidatong requested a review from emwang-jump as a code owner March 2, 2026 21:26
@lidatong lidatong force-pushed the chali/chore/store-mem branch from 8dc015a to 9ee7ace Compare March 2, 2026 21:30
@github-actions
Copy link

github-actions bot commented Mar 2, 2026

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.061347 s 0.061206 s -0.230%
backtest mainnet-368528500-perf snapshot load 3.241 s 2.493 s -23.079%
backtest mainnet-368528500-perf total elapsed 61.347182 s 61.206398 s -0.229%
firedancer mem usage with mainnet.toml 966.37 GiB 1196.37 GiB 23.800% ⚠️

@lidatong lidatong force-pushed the chali/chore/store-mem branch from 9ee7ace to cf159f3 Compare March 2, 2026 21:30
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces memory usage by bounding the store to fixed-32 shreds per FEC set, enabling an increase in max_live_slots from 512 to 4096 (addressing issue #8222) while keeping total locked memory within the 512 GiB limit of target hardware.

Changes:

  • FD_STORE_DATA_MAX reduced from 63985 to 31840 bytes, reflecting fixed-32 FEC set sizes (32 shreds × 995 bytes payload = 31840, matching FD_SHREDDER_NORMAL_FEC_SET_PAYLOAD_SZ)
  • Store fec_max sizing formula in topology.c replaced from a temporary hardcoded * 1024UL hack with the semantically correct * FD_SHRED_BLK_MAX / FD_FEC_SHRED_CNT
  • max_live_slots default increased from 512 to 4096

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
src/disco/store/fd_store.h Reduces FD_STORE_DATA_MAX from 63985 to 31840 to reflect fixed-32 FEC set constraint
src/app/firedancer/topology.c Replaces temporary * 1024UL hack with correct * FD_SHRED_BLK_MAX / FD_FEC_SHRED_CNT formula
src/app/firedancer/config/default.toml Increases max_live_slots from 512 to 4096

67 shreds per FEC set * 955 payloads per shred = 63985 bytes max. */
32 shreds per FEC set (FD_FEC_SHRED_CNT) * 955 payloads per shred = 31840 bytes max. */

Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment block above FD_STORE_DATA_MAX still describes the old computation ("67 shreds per FEC set * 955 payloads per shred = 63985 bytes max"), which no longer matches the new value of 31840. The comment should be updated to reflect the fixed-32 basis: 32 shreds per FEC set × 995 bytes payload per shred = 31840 bytes max (matching FD_SHREDDER_NORMAL_FEC_SET_PAYLOAD_SZ in fd_shredder.h).

Copilot uses AI. Check for mistakes.
@github-actions
Copy link

github-actions bot commented Mar 2, 2026

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.061192 s 0.061107 s -0.139%
backtest mainnet-368528500-perf snapshot load 3.288 s 2.479 s -24.605%
backtest mainnet-368528500-perf total elapsed 61.191652 s 61.106743 s -0.139%
firedancer mem usage with mainnet.toml 966.37 GiB 1196.37 GiB 23.800% ⚠️

@lidatong lidatong force-pushed the chali/chore/store-mem branch from cf159f3 to 2349996 Compare March 2, 2026 21:48
Copilot AI review requested due to automatic review settings March 2, 2026 21:49
@lidatong lidatong force-pushed the chali/chore/store-mem branch from 2349996 to 35a9e6f Compare March 2, 2026 21:49
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

@github-actions
Copy link

github-actions bot commented Mar 2, 2026

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.075007 s 0.075244 s 0.316%
backtest mainnet-368528500-perf snapshot load 4.198 s 3.191 s -23.988%
backtest mainnet-368528500-perf total elapsed 75.006532 s 75.244321 s 0.317%
firedancer mem usage with mainnet.toml 964.37 GiB 1179.37 GiB 22.294% ⚠️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

increase max_live_banks to 4096

3 participants