-
Notifications
You must be signed in to change notification settings - Fork 0
BufferPool: make it user-controllable via eval() API #124
Copy link
Copy link
Open
Description
Summary
Currently BufferPool is always ON, created internally in EinsumNode::eval(), and not controllable by users. For small tensors, jemalloc/tcmalloc tcache is sufficient and the pool adds unnecessary complexity. For large tensors (>4MB), the pool avoids costly mmap/munmap.
Proposal
- Make
BufferPoolpublic and acceptOption<&mut BufferPool>in the eval API:
// With pool (user-managed, reusable across calls)
let mut pool = BufferPool::new();
node.eval_with_pool(&operands, &mut pool)?;
// Without pool (allocator handles everything)
node.eval(&operands)?;- This also enables sharing a single pool across multiple einsum calls.
Benefit
- Users can benchmark ON vs OFF to determine if the pool helps their workload
- Pool lifetime is user-controlled (can persist across calls or be dropped)
- Cleaner separation of concerns
Risk
Low. The pool is already passed as an argument internally (eval_node(..., pool, ...)). This just exposes it at the public API level.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels