optimize big size scan reqs

[liunyl]

Summary by CodeRabbit

• Chores

  • Updated an internal submodule reference (no functional changes).

• Bug Fixes / Performance

  • Range scans now correctly track whether an explicit end boundary was provided and reset that state after scans.
  • When an end boundary is specified, prefetch is forced to a smaller fixed value to stabilize performance near scan limits.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

Updated data_substrate submodule pointer and added an end_specified_ flag to ha_eloq; scan open paths set the flag, close clears it, and PrefetchSize() returns 256 when the flag is true.

Changes

Cohort / File(s)	Summary
Submodule Update data_substrate	Submodule pointer updated from 3e3d1821bb8323c1962e65f4f7b5e55044e19796 to f694e422f1fe433c27ce92716e20e6b0dd5da74b. No code changes.
Scan state updates storage/eloq/ha_eloq.cc	Set end_specified_ when end_key/end_index_key is provided in PkIndexScanOpen and SkIndexScanOpen; clear end_specified_ in IndexScanClose.
Flag and prefetch logic storage/eloq/ha_eloq.h	Added private boolean end_specified_; PrefetchSize() now returns 256 immediately when end_specified_ is true, otherwise retains existing logic.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20–30 minutes

• Check end_specified_ initialization and all scan-exit paths to ensure it's always cleared.
• Review thread-safety/concurrency assumptions where ha_eloq instances may be reused.
• Validate PrefetchSize() change doesn't adversely affect callers expecting previous heuristics.

Possibly related PRs

• optimize big size scan reqs #178 — modifies the same ha_eloq flag usage and PrefetchSize() behavior; likely directly related.

Suggested reviewers

• lzxddz
• thweetkomputer

Poem

🐇 I set a tiny flag at scan's first light,
A hop at open, a reset at night.
Substrate pointer nudged along the way,
Small paws, small changes — ready to play. 🥕

Pre-merge checks and finishing touches

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.
Title check	❓ Inconclusive	The title 'optimize big size scan reqs' is vague and lacks specificity about what optimization is being made or which scan requirements are addressed.	Replace with a more descriptive title that explains the specific optimization, such as 'Add end_specified flag to optimize scan prefetch size' or similar.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f488842 and b6ac5ca.

📒 Files selected for processing (3)

• data_substrate (1 hunks)
• storage/eloq/ha_eloq.cc (3 hunks)
• storage/eloq/ha_eloq.h (2 hunks)

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

storage/eloq/ha_eloq.h (1)

718-739: Aggressive prefetch when an end key is present – consider de‑magicifying 256.

Using a fixed return 256; when end_specified_ is true is straightforward and aligns with the intent to prefetch more aggressively on bounded scans, but it does hard‑code a new behavior step change. Consider:

• Defining a named constant (e.g. static constexpr uint32_t kBoundedScanPrefetch = 256;) near DEFAULT_SCAN_TUPLE_SIZE to document and centralize the tuning knob.
• Optionally revisiting the value in perf tests to ensure small bounded ranges are not over‑prefetched compared to the prior adaptive ramp‑up.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 530ad21 and f488842.

📒 Files selected for processing (3)

• data_substrate (1 hunks)
• storage/eloq/ha_eloq.cc (3 hunks)
• storage/eloq/ha_eloq.h (2 hunks)

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-11-07T07:10:40.633Z

Learnt from: githubzilla
Repo: eloqdata/eloqsql PR: 154
File: storage/eloq/ha_eloq.cc:2571-2623
Timestamp: 2025-11-07T07:10:40.633Z
Learning: Repo: eloqdata/eloqsql PR: 154
File/Area: storage/eloq/ha_eloq.cc (eloq_init_func cluster membership resolution)
Learning: In this repo’s ng_config, the local node is guaranteed to be a candidate (NodeConfig::is_candidate_ == true). Therefore, assigning native_ng_id only when is_candidate_ is true is acceptable; the key improvement is to break early from both loops once the local node is found.

Applied to files:

• storage/eloq/ha_eloq.cc
• storage/eloq/ha_eloq.h

🔇 Additional comments (3)

data_substrate (1)

1-1: Submodule update aligns with PR objectives—verify no breaking changes in data_substrate.

The data_substrate submodule pointer update to commit d2a3c09 is part of the commit "prefetch more aggressively when end key is specified," which directly supports the PR's goal of optimizing large scan requests. Ensure the new submodule commit does not introduce breaking changes to dependent code.

storage/eloq/ha_eloq.h (1)

781-787: end_specified_ scan state flag wiring looks correct.

Default initialization to false and placement next to scan_end_key_/scan_end_tx_key_ are appropriate for per‑scan state; the .cc changes set/clear it consistently on scan open/close.

storage/eloq/ha_eloq.cc (1)

4123-4166: end_specified_ lifecycle around scan open/close is coherent.

Setting end_specified_ in PkIndexScanOpen/SkIndexScanOpen based on end_key/end_index_key nullness and clearing it in IndexScanClose() cleanly scopes the flag to the lifetime of a single open scan and matches how PrefetchSize() is consulted in the scan batch paths. Given current call sites only pass a non‑null end pointer when a real end boundary has been computed, this looks functionally sound.

Also applies to: 4188-4227, 5158-5219