Add VirtualFs: runtime-generated filesystem with pluggable sources#177
Add VirtualFs: runtime-generated filesystem with pluggable sources#177Shaar-games wants to merge 2 commits intovercel-labs:mainfrom
Conversation
Introduces a new IFileSystem implementation that generates content on-the-fly through async hooks, enabling synthetic filesystems backed by APIs, databases, or any custom data source. Key features: - VirtualFsSource interface with required readFile/readdir hooks - Optional stat/exists hooks with automatic derivation - Optional write hooks (writeFile, mkdir, rm, etc.) or EROFS rejection - defineVirtualFs() factory helper for type-safe source creation - Full test coverage (70 tests) including Bash integration - Example implementations (report DB, metrics API) Typical use case: mount dynamic content inside MountableFs so shell commands can access virtual files as if they were real (ls, cat, grep, wc, etc.). Made-with: Cursor
|
@Shaar-games is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
Apply consistent path normalization and null-byte validation to all write operations (writeFile, appendFile, mkdir, rm, cp, mv, chmod, symlink, link, utimes) before passing paths to source hooks, matching the behavior of read operations for security and consistency. Made-with: Cursor
|
Not sure if this is really useful. Folks seem to be fine with the existing interface |
Yes a lot of use cases are already covered. Where it gets awkward for me is: with InMemoryFs you can’t really “forward” writeFile / rm to something outside the process; it just updates memory. To do that you end up implementing the whole IFileSystem yourself. Same for huge trees: lazy files still need every path registered up front. If the tree is huge or comes from a DB/API, I’d rather only load what ls / cat actually touches. |
|
Can't you do this already by supplying lazy files to the InMemoryFs? |
|
I still don't get it why you don't want either lazy files OR implement a full FS. Maybe share an implementation |
Add VirtualFs: Runtime-Generated Filesystem with Pluggable Sources
Summary
Introduces a new
IFileSystemimplementation that generates content on-the-fly through async hooks, enabling synthetic filesystems backed by APIs, databases, or any custom data source.Key Features
readFile/readdirhookswriteFile,mkdir,rm, etc.) or EROFS rejectiondefineVirtualFs()factory helper for type-safe source creationUse Case
Mount dynamic content inside
MountableFsso shell commands can access virtual files as if they were real:Test Plan
examples/virtual-fs/Files Changed
src/fs/virtual-fs/virtual-fs.ts- Core implementation (273 lines)src/fs/virtual-fs/virtual-fs.test.ts- Test suite (70 tests)src/fs/virtual-fs/index.ts- Public exportssrc/index.ts- Added VirtualFs to public APIexamples/virtual-fs/- Complete working examples with READMEImplementation Details
The
VirtualFsclass delegates all operations to user-provided hooks:readFile,readdir,stat) call the source hooksEROFSstatis not provided, it's derived fromreaddir/readFiledispose()hook for cleanupThis enables powerful use cases like:
This is one of my first contributions via pull request, so please let me know if any changes are needed or if I should adjust anything in the implementation or documentation. I'm happy to iterate based on your feedback!