Skip to content

Commit a405963

Browse files
committed
perf: eliminate .map().join() in WindowOperator/SetOperator key construction; fix docs categorization
- WindowOperator partition key: .map().join() → direct concatenation loop - SetOperator rowKey: Object.keys().sort().map().join() → direct concatenation loop - dataframe-api docs: move toCode/toDescriptor from "Terminal methods" to new "Introspection" section (they don't execute queries)
1 parent 12069eb commit a405963

File tree

2 files changed

+26
-6
lines changed

2 files changed

+26
-6
lines changed

docs/src/content/docs/dataframe-api.mdx

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -172,8 +172,6 @@ These execute the query and return results:
172172
| `stream(batchSize)` | `AsyncGenerator<Row[]>` | Async generator over result batches |
173173
| `cursor({ batchSize })` | `AsyncIterable<Row[]>` | Async iterable over batches |
174174
| `materialize()` | `DataFrame` | Execute and re-wrap as in-memory DataFrame |
175-
| `toCode(opts?)` | `string` | Convert query to fluent builder TypeScript code |
176-
| `toDescriptor()` | `QueryDescriptor` | Serializable query plan (JSON) |
177175
| `append(rows, opts?)` | `AppendResult` | Write rows (CAS-coordinated) |
178176
| `dropTable()` | `DropResult` | Delete table and all fragments |
179177

@@ -204,6 +202,15 @@ const csv = await df.toCSV({ delimiter: "\t" })
204202

205203
These are all sugar over existing methods — `shape()` calls `explain()`, `valueCounts()` uses `groupBy().aggregate()`, `fillNull()` and `cast()` use `computed()`.
206204

205+
## Introspection
206+
207+
These methods inspect the query without executing it:
208+
209+
| Method | Returns | Description |
210+
|--------|---------|-------------|
211+
| `toCode(opts?)` | `string` | Convert query to fluent builder TypeScript code |
212+
| `toDescriptor()` | `QueryDescriptor` | Serializable query plan (JSON) |
213+
207214
## Query decompilation (toCode)
208215

209216
Convert any query back to readable, copy-pasteable fluent builder TypeScript. Useful for logging, debugging, and LLM context compression (~50x smaller than raw QueryDescriptor JSON).

src/operators.ts

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -924,9 +924,16 @@ export class WindowOperator implements Operator {
924924
// Group by partitionBy keys
925925
const partitions = new Map<string, number[]>();
926926
for (let i = 0; i < rows.length; i++) {
927-
const key = win.partitionBy.length > 0
928-
? win.partitionBy.map(c => String(rows[i][c] ?? "")).join("\x00")
929-
: "__all__";
927+
let key: string;
928+
if (win.partitionBy.length > 0) {
929+
key = "";
930+
for (let p = 0; p < win.partitionBy.length; p++) {
931+
if (p > 0) key += "\x00";
932+
key += String(rows[i][win.partitionBy[p]] ?? "");
933+
}
934+
} else {
935+
key = "__all__";
936+
}
930937
const indices = partitions.get(key);
931938
if (indices) indices.push(i);
932939
else partitions.set(key, [i]);
@@ -1199,7 +1206,13 @@ export class SetOperator implements Operator {
11991206
}
12001207

12011208
private rowKey(row: Row): string {
1202-
return Object.keys(row).sort().map(k => `${k}=${String(row[k] ?? "")}`).join("\x00");
1209+
const keys = Object.keys(row).sort();
1210+
let result = "";
1211+
for (let i = 0; i < keys.length; i++) {
1212+
if (i > 0) result += "\x00";
1213+
result += keys[i] + "=" + String(row[keys[i]] ?? "");
1214+
}
1215+
return result;
12031216
}
12041217

12051218
async next(): Promise<RowBatch | null> {

0 commit comments

Comments
 (0)