Skip to content

feat: Add pin node type#4747

Open
Bidon15 wants to merge 13 commits intocelestiaorg:mainfrom
Bidon15:feat/pin-node
Open

feat: Add pin node type#4747
Bidon15 wants to merge 13 commits intocelestiaorg:mainfrom
Bidon15:feat/pin-node

Conversation

@Bidon15
Copy link
Member

@Bidon15 Bidon15 commented Dec 22, 2025

Add Pin Node Type

Introduces a new Pin node type that enables namespace-filtered data storage and synchronization.

  1. It reads ODS from the core.ip like a bridge node
  2. Discards blocks which don't contain data under their namespace
  3. Stores only blobs from the namespace provided by the user

Motivation

Hosted providers need fine-grained control over data retention to price reads according to actual namespace usage from the users they serve. Different clients have different requirements:

  • 1st might need a 7-day retention policy only
  • 2nd needs full archival data for their entire namespace since block N
  • 3rd is priced more on reads due to higher blob usage
  • 4th is paying little as they are submitting close to none

The Pin node type enables this by allowing operators to selectively store and retain data only for configured namespaces, rather than requiring full network archival per each customer they serve.

Summary

  • Add Pin node type with dedicated CLI commands
  • Implement namespace filtering for ODS file storage - only store data for configured namespaces
  • Add namespace tracking store to record which namespaces are pinned
  • Add --pin.namespaces flag to configure target namespaces
  • Allow Pin nodes to optionally run with --archival flag
  • Validate minimum pruning window against sampling window

Changes

  • nodebuilder/node/type.go: New Pin node type
  • store/: Namespace filtering and tracking in ODS storage
  • nodebuilder/share/: Pin-specific module configuration
  • nodebuilder/pruner/: Support for Pin node pruning behavior
  • cmd/: CLI integration for pin node commands

@github-actions github-actions bot added the external Issues created by non node team members label Dec 22, 2025
@Bidon15 Bidon15 added area:node Node area:cli and removed external Issues created by non node team members labels Dec 22, 2025
- Add NamespaceFilter type and CreateODSWithFilter/CreateODSQ4WithFilter
- Original functions unchanged for backward compatibility
- Add TrackedNamespace and NamespaceDatastore to Parameters
- Add namespace store for caching namespace-specific data
- Store uses filter when writing ODS files if configured
- Getter uses namespace cache for fast lookups on tracked namespace
- All features opt-in, standard nodes unaffected
- Revert unnecessary corelib/corepruner renames
- Use clean dependency injection for namespace config
- Add node.Pin to core and pruner module switch cases
Restore original badger settings:
- BlockCacheSize = 0 (disable block cache)
- Compression = None

This avoids breaking changes for existing production nodes.
- Add WriteOptions and WithFilter functional option
- CreateODS and CreateODSQ4 now accept variadic options
- Remove separate WithFilter functions and if/else branches
- Store uses s.writeOpts... for clean, maintainable code
…bility

- Replace binary encoding with strconv.FormatUint for namespace store keys
- Add package-level nsStorePrefix following codebase conventions
- Rename method receiver from 'ns' to 's' to avoid variable shadowing
- Update test structs to use wrapper functions for variadic WriteOption params
- Use cmd.Flag().Changed instead of comparing against magic string '0s'
- Use type-safe GetDuration() instead of manual string parsing
- Remove unused time import
Pin nodes should be able to run in archival mode to store all blocks
for their tracked namespace from a specific starting height without
pruning.
Prevent users from setting a pruning window shorter than the sampling
window (7 days). A shorter window would cause the node to delete blocks
that the network expects it to serve for DAS.

Users must either:
- Use default or custom window >= 7 days (pruning mode)
- Use --archival flag (no pruning)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant