Skip to content

BufferPool: make it user-controllable via eval() API #124

@shinaoka

Description

@shinaoka

Summary

Currently BufferPool is always ON, created internally in EinsumNode::eval(), and not controllable by users. For small tensors, jemalloc/tcmalloc tcache is sufficient and the pool adds unnecessary complexity. For large tensors (>4MB), the pool avoids costly mmap/munmap.

Proposal

  1. Make BufferPool public and accept Option<&mut BufferPool> in the eval API:
// With pool (user-managed, reusable across calls)
let mut pool = BufferPool::new();
node.eval_with_pool(&operands, &mut pool)?;

// Without pool (allocator handles everything)
node.eval(&operands)?;
  1. This also enables sharing a single pool across multiple einsum calls.

Benefit

  • Users can benchmark ON vs OFF to determine if the pool helps their workload
  • Pool lifetime is user-controlled (can persist across calls or be dropped)
  • Cleaner separation of concerns

Risk

Low. The pool is already passed as an argument internally (eval_node(..., pool, ...)). This just exposes it at the public API level.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions