Commit 1ad1b90
committed
feat[arrow-ord]: suppport REE comparisons
This commit implements native comparisons on REE-encoded arrays which are
treated similarly to dictionary indirection.
This commit implements REE to scalar comparisons by operating on the physical
values only then bulk expanding the boolean result.
REE-to-REE comparisons are also optimized by computing aligned physical value
runs to minimize comparisons.
Mixed cases (REE vs flat) materialize a logical index mapping similar to
dictionaries.
This commit also supports REE<Dict>.
For comparison, here are the benchmark results with flat arrays as a reference
on my local machine:
```
eq Int32 time: [14.955 µs 15.162 µs 15.396 µs]
eq scalar Int32 time: [11.379 µs 11.418 µs 11.459 µs]
ree_comparison/eq_ree_scalar(phys=64,log=65536) time: [453.31 ns
454.88 ns 456.43 ns]
ree_comparison/eq_ree_scalar(phys=1024,log=65536) time: [4.1224 µs
4.1298 µs 4.1368 µs]
ree_comparison/eq_ree_scalar(phys=32768,log=65536) time: [93.506 µs
94.085 µs 94.993 µs]
ree_comparison/eq_ree_ree(phys=64,log=65536) time: [413.96 ns
414.82 ns 415.87 ns]
ree_comparison/eq_ree_ree(phys=1024,log=65536) time: [4.1597 µs
4.1660 µs 4.1749 µs]
ree_comparison/eq_ree_ree(phys=32768,log=65536) time: [128.74 µs
144.40 µs 161.53 µs]
```
As is expected, the more we take advantage of REE encoding, the faster the
comparisons are.
Signed-off-by: Alfonso Subiotto Marques <alfonso.subiotto@polarsignals.com>1 parent 471f6c3 commit 1ad1b90
2 files changed
+440
-31
lines changed
0 commit comments