-
Notifications
You must be signed in to change notification settings - Fork 3
Optimize .mem Uploads with External JSON for Structure & Diff #8
Description
Description:
Currently, .mem tracks all incremental changes and contextual history, which is essential for local operations like replay, trace analysis, and feature mapping.
However, when uploading to a remote service (memhub), there's no need to upload the full .mem directory. It contains a lot of local state that's not required for high-level visualization or collaboration.
Problem
The .mem folder can grow large over time, especially with frequent snapshots and tracked files.
Uploading the entire .mem directory consumes unnecessary storage and bandwidth.
Most downstream features (e.g. feature graphs, prompt history, replay UI) only need:
Structural metadata (e.g. prompt → snapshot → feature mapping)
Code diffs between snapshots
Proposal
Introduce a new export format:
.mem/export/trace.json