In-memory interface to make testing easier

[kvnkusch]

Adds an interface and impl for running mutations/queries/transactions backed only by memory storage. The idea is to use this for testing, specifically for permissions enforced via synced queries. Based on discussion here: https://discord.com/channels/830183651022471199/830183651022471202/1442950450503483627

Example usage:

import { test, expect, beforeEach } from "vitest";
import { MemoryZeroImpl } from "@rocicorp/zero/test-helpers";
import { schema } from "../src/zero/schema";
import { createId } from "../src/core/utils";

const z = new MemoryZeroImpl(schema);

const userId1 = createId();
const userId2 = createId();

beforeEach(() => {
  z.reset();
});

test("transaction", async () => {
  const listId = createId();
  await z.transaction(async (tx) => {
    const collectionId = createId();
    const createCollectionFn = ListCollection.defaultForUser(userId1)
    await createCollectionFn(tx, {
      id: collectionId,
    });
    const createListFn = List.create(userId1);
    await createListFn(tx, {
      id: listId,
      type: "shopping",
      collectionId,
      title: "test",
      sections: [],
      orphanItems: [],
    });
  });

  const r1 = await z.run(
    // named query
    List.details({
      ctx: {
        auth: userId1,
      },
      args: listId,
    }),
  );
  expect(r1).toBeDefined();

  const r2 = await z.run(
    List.details({
      ctx: {
        auth: userId2,
      },
      args: listId,
    }),
  );
  expect(r2).toBeUndefined();
});

[vercel]

Someone is attempting to deploy a commit to the Rocicorp Team on Vercel.

A member of the Team first needs to authorize it.

[kvnkusch]

Unsure where this should be exported from, left this as a placeholder

[kvnkusch]

If this change is not desired, would be nice to export these dependencies at least.

[aboodman]

I'm really sorry about the slow reply here - we need to get better at monitoring the PRs for user submissions. Can you say more about what the intent here is? What kind of tests are you trying to write?

[kvnkusch]

The intent is to make tests of queries as fast as possible and not require any docker setup. In my case, I have non-trivial authorization rules that I need to test (e.g. a record can be returned if a user has a grant to it or the user belongs to a group that has a grant to it).

I recognize it's possible to test this end-to-end, but I'd like to have most of my tests be unit tests if possible. I actually originally looked into using SQLite / ZQLite to run these, but stumbled upon the MemorySource and test QueryDelegate after getting pointed in the right direction. Using an in-memory instance of these means I don't even need to copy around SQLite files in test setups.

• This obviously assumes that the in-memory / SQLite / PG implementations of query running are all correct/consistent with each other, but I feel safe making that assumption.
• Still plan on using a PG instance for testing mutators

[tantaman]

Sorry, I didn't see this. Feel free to add me as a reviewer next time!

For others on the team, this is different from our existing TestZero:

The key difference is what layer of the stack is being testing:

TestZero with kvStore: 'mem' still runs the full Zero client stack:

• Connection management, sync protocol, Replicache integration
• You must manually trigger server messages (triggerPoke(), triggerPullResponse(), etc.)
• Requires understanding the Zero protocol to write test data
• Good for integration testing the sync machinery

MemoryZeroImpl bypasses all that and talks directly to the ZQL query engine:

• Uses MemorySource (BTree-backed in-memory tables) instead of Replicache
• Direct CRUD: mutate.table.insert(row) - no protocol simulation needed
• No connection state, no message passing, no mock socket
• Good for unit testing queries, mutations, and business logic in isolation

My main piece of feedback would be that MemoryZeroImpl should be renamed to MemoryZQL.

The other would be that we want to keep total code surface area down so I'd need to look into what existing test helpers we can replace with this -- given your helper is more complete than the ones we have today.

Ideally we can update zql-integration-tests/helpers/runner.ts to use this new utility to reduce duplication.

[aboodman]

The intent is to make tests of queries as fast as possible and not require any docker setup. In my case, I have non-trivial authorization rules that I need to test (e.g. a record can be returned if a user has a grant to it or the user belongs to a group that has a grant to it).

I think this is possible today just with new Zero({kvStore: "mem"}). You can insert data, run queries, etc. No postgres required.

[kvnkusch]

I think this is possible today just with new Zero({kvStore: "mem"}). You can insert data, run queries, etc. No postgres required.

Gave another pass at getting this running with new Zero({kvStore: "mem"}), here are some of the issues I ran into:

1. If you do not have legacy mutators enabled, you have to do any test setup for queries using mutators. This is doable but can be inconvenient.
2. There's no reset() function on Zero, but you can create a new instance per test.
3. If I don't pass a server value to the Zero constructor, mutations failed with No server socket origin provided. If I provide a dummy URL, tests run as expected but logs the following errors and times out:

clientID=vb4kflqogc5m95unt2 wsid=1wlHGPQWDLSEzpSx-NmBe Got unexpected socket close event {"code":1006,"reason":"Failed to connect","wasClean":false}
clientID=vb4kflqogc5m95unt2 runLoopCounter=1 Failed to connect {"name":"ClientError","message":"WebSocket connection closed abruptly","stack":"ClientError: WebSocket connection closed abruptly\n    at #onClose (/Users/kkusch/Code/cc/node_modules/@rocicorp/zero/out/zero-client/src/client/zero.js:845:30)"} Unknown Error {"lmid":0,"baseCookie":null}
✗ update, delete [5000.20ms]
  ^ this test timed out after 5000ms.

Obviously providing a valid URL corresponding to an instance of zero-cache would work, but that kind of defeats the point here.

[0xcadams]

If you pass server: null, does that solve the third problem?

[kvnkusch]

Passing server: null gets the same result as omitting it, No server socket origin provided. errors on mutations

[kvnkusch]

@tantaman I took a pass at using MemoryZQL in zql-integration-tests/helpers/runner.ts. I got all of the zql integration tests passing with MemoryZQL assertions alongside everything else, but it was a bit awkward because all of the different DBs and Delegates are grouped together, kind of the opposite of how MemoryZQL has both in one class. Let me know if it'd be helpful to push this up to this branch or not.