Proposal: Spatial-Partitioned Out-of-Core Spatial Join

[Kontinuation]

SedonaDB's SpatialJoinExec currently buffers the entire build side, builds a single in-memory R-tree, and streams the probe side. This approach works well when the build side and its index fit in memory but fails with memory reservation failures or OOM errors for large datasets.

We propose introducing a spatial-partitioned out-of-core execution path that keeps memory usage bounded by:

1. Partitioning the build side into disjoint partitions that each fit in memory.
2. Building one in-memory R-tree per partition and probing it before moving to the next partition.

The design intentionally mirrors hybrid/grace hash join but adapts it to SedonaDB’s spatial join requirements.