Skip to content

Add VirtualFs: runtime-generated filesystem with pluggable sources#177

Open
Shaar-games wants to merge 2 commits intovercel-labs:mainfrom
Shaar-games:feature/virtual-fs
Open

Add VirtualFs: runtime-generated filesystem with pluggable sources#177
Shaar-games wants to merge 2 commits intovercel-labs:mainfrom
Shaar-games:feature/virtual-fs

Conversation

@Shaar-games
Copy link
Copy Markdown

Add VirtualFs: Runtime-Generated Filesystem with Pluggable Sources

Summary

Introduces a new IFileSystem implementation that generates content on-the-fly through async hooks, enabling synthetic filesystems backed by APIs, databases, or any custom data source.

Key Features

  • VirtualFsSource interface with required readFile/readdir hooks
  • Optional stat/exists hooks with automatic derivation when not provided
  • Optional write hooks (writeFile, mkdir, rm, etc.) or EROFS rejection
  • defineVirtualFs() factory helper for type-safe source creation
  • Full test coverage: 70 tests including Bash integration
  • Example implementations: report DB simulator, metrics API simulator

Use Case

Mount dynamic content inside MountableFs so shell commands can access virtual files as if they were real:

import { defineVirtualFs, VirtualFs, MountableFs, Bash } from "just-bash";

const mySource = defineVirtualFs((opts: { userId: string }) => ({
  async readFile(path) {
    return path === "/hello.txt" ? `Hello, ${opts.userId}!` : null;
  },
  async readdir(path) {
    if (path === "/") {
      return [{ name: "hello.txt", isFile: true, isDirectory: false }];
    }
    return null;
  },
}));

const bash = new Bash({
  fs: new MountableFs({
    mounts: [
      { mountPoint: "/data", filesystem: new VirtualFs(mySource({ userId: "alice" })) },
    ],
  }),
});

await bash.exec("cat /data/hello.txt"); // → Hello, alice!

Test Plan

  • ✅ 70 tests pass successfully
  • ✅ No linter errors
  • ✅ TypeScript compiles without errors
  • ✅ Includes comprehensive example in examples/virtual-fs/

Files Changed

  • src/fs/virtual-fs/virtual-fs.ts - Core implementation (273 lines)
  • src/fs/virtual-fs/virtual-fs.test.ts - Test suite (70 tests)
  • src/fs/virtual-fs/index.ts - Public exports
  • src/index.ts - Added VirtualFs to public API
  • examples/virtual-fs/ - Complete working examples with README

Implementation Details

The VirtualFs class delegates all operations to user-provided hooks:

  • Read operations (readFile, readdir, stat) call the source hooks
  • Write operations delegate to optional source hooks or reject with EROFS
  • Automatic stat derivation: when stat is not provided, it's derived from readdir/readFile
  • Lifecycle management: optional dispose() hook for cleanup

This enables powerful use cases like:

  • Mounting API responses as files
  • Database query results as directory trees
  • Generated documentation from code
  • Dynamic configuration from remote sources

This is one of my first contributions via pull request, so please let me know if any changes are needed or if I should adjust anything in the implementation or documentation. I'm happy to iterate based on your feedback!

Introduces a new IFileSystem implementation that generates content on-the-fly
through async hooks, enabling synthetic filesystems backed by APIs, databases,
or any custom data source.

Key features:
- VirtualFsSource interface with required readFile/readdir hooks
- Optional stat/exists hooks with automatic derivation
- Optional write hooks (writeFile, mkdir, rm, etc.) or EROFS rejection
- defineVirtualFs() factory helper for type-safe source creation
- Full test coverage (70 tests) including Bash integration
- Example implementations (report DB, metrics API)

Typical use case: mount dynamic content inside MountableFs so shell commands
can access virtual files as if they were real (ls, cat, grep, wc, etc.).

Made-with: Cursor
@vercel
Copy link
Copy Markdown

vercel bot commented Apr 3, 2026

@Shaar-games is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

Apply consistent path normalization and null-byte validation to all write
operations (writeFile, appendFile, mkdir, rm, cp, mv, chmod, symlink, link,
utimes) before passing paths to source hooks, matching the behavior of read
operations for security and consistency.

Made-with: Cursor
@cramforce
Copy link
Copy Markdown
Contributor

Not sure if this is really useful. Folks seem to be fine with the existing interface

@Shaar-games
Copy link
Copy Markdown
Author

Not sure if this is really useful. Folks seem to be fine with the existing interface

Yes a lot of use cases are already covered.

Where it gets awkward for me is: with InMemoryFs you can’t really “forward” writeFile / rm to something outside the process; it just updates memory. To do that you end up implementing the whole IFileSystem yourself.

Same for huge trees: lazy files still need every path registered up front. If the tree is huge or comes from a DB/API, I’d rather only load what ls / cat actually touches.

@cramforce
Copy link
Copy Markdown
Contributor

Can't you do this already by supplying lazy files to the InMemoryFs?

@cramforce
Copy link
Copy Markdown
Contributor

I still don't get it why you don't want either lazy files OR implement a full FS. Maybe share an implementation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants