Skip to content

Optimize .mem Uploads with External JSON for Structure & Diff #8

@crispyberry

Description

@crispyberry

Description:

Currently, .mem tracks all incremental changes and contextual history, which is essential for local operations like replay, trace analysis, and feature mapping.
However, when uploading to a remote service (memhub), there's no need to upload the full .mem directory. It contains a lot of local state that's not required for high-level visualization or collaboration.

Problem
The .mem folder can grow large over time, especially with frequent snapshots and tracked files.
Uploading the entire .mem directory consumes unnecessary storage and bandwidth.
Most downstream features (e.g. feature graphs, prompt history, replay UI) only need:
Structural metadata (e.g. prompt → snapshot → feature mapping)
Code diffs between snapshots

Proposal
Introduce a new export format:

.mem/export/trace.json

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions