You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Three analyses on one result set. No SQL string construction, no JSON parsing, no round-trips. The intermediate results are live objects in memory — you inspect them, branch on them, and feed them into the next stage.
54
54
55
-
> **What about memory?**`collect()` doesn't load a 50GB file into a V8 isolate. Filter pushdown already skipped irrelevant pages via min/max stats, aggregation already reduced rows to group summaries, projection already dropped unused columns. What lands in memory is the *result*, not the dataset. Operators are memory-bounded (default 32MB) and [spill to R2](/operators#memory-bounded-with-r2-spill) when they exceed budget.
55
+
> **What about memory?**`collect()` doesn't load a 50GB file into a V8 isolate. Filter pushdown already skipped irrelevant pages via min/max stats, aggregation already reduced rows to group summaries, projection already dropped unused columns. What lands in memory is the *result*, not the dataset. Operators are memory-bounded (default 32MB) and [spill to R2](/querymode/operators/#memory-bounded-with-r2-spill) when they exceed budget.
56
56
57
57
## How it actually works under the hood
58
58
@@ -64,7 +64,7 @@ Data sits in **R2** as columnar files (Parquet, Lance, Iceberg). Nothing gets re
64
64
65
65
### The operators ARE the optimizer
66
66
67
-
Every query runs through a pull-based [operator pipeline](/operators):
67
+
Every query runs through a pull-based [operator pipeline](/querymode/operators/):
@@ -102,4 +102,4 @@ Both test suites also include multi-step analyses that would be awkward with the
102
102
103
103
QueryMode doesn't eliminate transformation. It moves it from a pre-built schedule to query time. The agent decides what to query, how to transform it, and what to do with the result — all in the same code, same process. If the data is well-structured, the agent queries it directly. If it's not, the agent builds the transformation on the spot. Either way, no one had to anticipate the question in advance.
104
104
105
-
It doesn't eliminate the query optimizer either. The operators do filter pushdown, vectorized decode, memory-bounded spill — but you assemble them, you control the budget, and you can put an [ML scoring function between pipeline stages](/operators#compose-operators-directly) if you want to.
105
+
It doesn't eliminate the query optimizer either. The operators do filter pushdown, vectorized decode, memory-bounded spill — but you assemble them, you control the budget, and you can put an [ML scoring function between pipeline stages](/querymode/operators/#compose-operators-directly) if you want to.
0 commit comments