Skip to content

Switch to Tanstack DB#580

Open
brendanlong wants to merge 38 commits intomasterfrom
tanstack-db
Open

Switch to Tanstack DB#580
brendanlong wants to merge 38 commits intomasterfrom
tanstack-db

Conversation

@brendanlong
Copy link
Copy Markdown
Owner

@brendanlong brendanlong commented Feb 9, 2026

Summary

Migrates frontend state management from React Query cache manipulation to TanStack DB collections as the primary data layer. This creates a normalized, reactive client-side store that integrates with React Query for server communication while providing:

  • Immediate rendering from cached data with async background refreshes
  • On-demand data loading — subscriptions and entries are fetched per-view, not all upfront
  • Deduplication — single source of truth for each entity type
  • Graceful degradation — works correctly even when subscription data hasn't been loaded yet
  • Server-side prefetching preserved for fast initial page loads

Architecture

Server ──(tRPC)──> React Query ──> TanStack DB Collections
                                        │
SSE events ──────────────────────> Collection writes
                                        │
                                   Live Queries (differential dataflow)
                                        │
                                        ▼
                                   Components (useLiveSuspenseQuery)

Collections

Collection Type Purpose
subscriptions Local-only Accumulates subscriptions as sidebar sections expand
entries Local-only Global entry cache, populated from view collections
tags Query-backed (eager) All user tags, small dataset loaded upfront
counts Local-only Unread/total counts for all/starred/saved/uncategorized
entries-view-* Query-backed (on-demand) Per-route entry list with pagination
subscriptions-tag-* Query-backed (on-demand) Per-tag subscription list with pagination

Phases (original migration)

  1. Sidebar migration to live queries
  2. Entries collection with dual-write for mutations and SSE
  3. Remove React Query entry list cache updates
  4. Migrate sidebar counts to collections
  5. Remove React Query count invalidations
  6. On-demand collections with collection-driven rendering

Bug fixes (issues #571-#578, #622-#625)

Test coverage

Added 95 unit tests for collection operations:

  • writes.test.ts (76 tests): All collection write functions with real TanStack DB collections
  • count-sync.test.ts (19 tests): Count seeding from SSR prefetch and query cache subscription sync

Test plan

  • Entry list loads, scrolls, and paginates correctly
  • Mark read/starred updates entry in-place (no disappearing in filtered views)
  • Navigate away and back → fresh filtered data
  • Unread count badges appear for All, Starred, Saved in sidebar
  • Counts update when new entries arrive via SSE
  • SSE entry_state_changed reflects in list and detail views
  • Save a new article while on All — article appears in the list
  • markAllRead zeros subscription counts in sidebar
  • Keyboard navigation (j/k) and swipe gestures work
  • SSR prefetch: no redundant entries.list fetch on initial page load
  • Sidebar subscription lists load per-tag and paginate on scroll
  • pnpm typecheck passes ✅
  • pnpm vitest run passes (1680 tests) ✅

claude and others added 17 commits February 8, 2026 21:59
Install @tanstack/db, @tanstack/react-db, and @tanstack/query-db-collection.
Define four collections (subscriptions, tags, entries, counts) with types
inferred from tRPC router outputs. Create a CollectionsProvider context and
initialize collections in TRPCProvider using a vanilla tRPC client for the
queryFn bridge. No existing components read from collections yet.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TagList reads from TanStack DB tags collection via useLiveSuspenseQuery
for reactive updates. Uncategorized counts are stored in the counts
collection (populated by the tags.list fetch).

TagSubscriptionList uses tRPC useInfiniteQuery with IntersectionObserver
for on-demand pagination (standard infinite scroll). Loaded pages are
written into a local-only subscriptions collection for fast lookups
and optimistic unread count updates.

Dual-write strategy: mutations and SSE events update both the old React
Query cache and new TanStack DB collections during the transition period.

New file: src/lib/collections/writes.ts - collection write utilities
called alongside existing cache operations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Establishes the entries collection as a local-only store populated from
tRPC infinite query results. Mutations and SSE events now write to both
the existing React Query cache and the entries collection.

Key changes:
- entries collection: local-only, populated from SuspendingEntryList pages
- writes.ts: add entry write utilities (updateEntryRead/Starred/Score/Metadata)
- useEntryMutations: dual-write to collection on mutate/success/error
- event-handlers: dual-write entry_updated and entry_state_changed to collection
- EntryContent: sync entries.get data to collection on load
- Simplified collection creation (removed unused fetchEntries parameter)

This is Phase 2a of the TanStack DB migration - no behavioral changes,
just establishing the collection infrastructure that Phase 2b will use
to replace the React Query cache manipulation code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace React Query cache scanning with O(1) collection lookups:
- EntryContentFallback: use entries collection instead of findEntryInListCache
- UnifiedEntriesContent TitleFallback: use subscriptions/tags collections
  instead of findCachedSubscription and tags.list.getData()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entry list now reads mutable state (read, starred, title, etc.) from
the TanStack DB entries collection via useLiveQuery, while tRPC infinite
query pages still provide ordering and identity. This eliminates the
need for complex React Query cache manipulation on entry state changes.

Changes:
- SuspendingEntryList: merge collection state over page items via useLiveQuery
- useEntryMutations: replace cache operations with collection writes + entries.get setData
- event-handlers: replace list cache updates with collection writes
- operations.ts: remove entry state functions (handleEntriesMarkedRead, etc.)
- entry-cache.ts: strip to only findParentListPlaceholderData (~808→363 LOC)
- Update tests to remove tests for deleted functions

Net: -1227 LOC of cache manipulation code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… (Phase 4)

All count/subscription/tag state updates now flow through TanStack DB
collections only. React Query is only used for entries.get (detail view)
and entries.list (pagination). This eliminates the dual-write pattern
where every mutation wrote to both React Query caches and collections.

Changes:
- SidebarNav: replace entries.count.useSuspenseQuery with useLiveQuery(counts)
- provider.tsx: seed "all"/"starred"/"saved" counts from SSR cache
- writes.ts: add setEntriesCountInCollection, adjustEntriesCountInCollection,
  setUncategorizedUnreadInCollection
- operations.ts: rewrite to collection-only writes (718→263 LOC)
- count-cache.ts: strip to calculateTagDeltasFromSubscriptions only (468→51 LOC)
- event-handlers.ts: remove applySyncTagChanges/removeSyncTags, remove queryClient param
- Remove useQueryClient from useEntryMutations, useRealtimeUpdates,
  SubscribeContent, BrokenFeedsSettingsContent

Net: -999 LOC of React Query cache manipulation code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add refreshGlobalCounts helper that fetches counts via tRPC and writes
  to collections (replaces invalidation/subscription pattern)
- Replace all entries.count.invalidate() calls (5 sites) with either
  refreshGlobalCounts or adjustEntriesCountInCollection
- Delete entry-cache.ts (363 LOC) - replace findParentListPlaceholderData
  with direct collection lookup in EntryListFallback
- Remove useQueryClient from Sidebar.tsx (last component import)
- Update FRONTEND_STATE.md to reflect removed files

After this phase, React Query is only used for:
- entries.get (detail view cache with optimistic updates)
- entries.list (pagination via useSuspenseInfiniteQuery)
- entries.count SSR prefetch (seeds collections on first load)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ring

Replace React Query-driven pagination with TanStack DB on-demand collections
as the primary data source for entry and subscription lists. Collections now
fetch pages from the server as the user scrolls, bridging TanStack DB's
offset-based loading to cursor-based pagination via an offset-to-cursor map.

Key changes:
- Entry lists render from useLiveInfiniteQuery over per-view collections
- Display stability: entries visible when view loaded don't disappear on
  state changes (e.g., marking read in "unread only" view)
- Sidebar tag subscription lists use per-tag on-demand collections
- Entry navigation (next/prev for swipe gestures) shared via external store
- Mutations and SSE writes propagate to active view collection
- SSR prefetch seeding preserved via React Query cache check

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DESIGN.md: Add Client-Side State Management section documenting
  TanStack DB collections architecture, data flow, and key hooks.
  Update Real-time Updates to reference collection writes instead
  of React Query cache invalidation.
- frontend-data-flow.d2: Rewrite diagram to show TanStack DB
  collections as primary state store, with React Query limited to
  SSR prefetch and entry detail views. Add collection writes,
  on-demand loading, display stability, and entry navigation.
- sse-cache-updates.d2: Add entry_state_changed and tag_* event
  sources/handlers. Update cache operations to show collection
  writes instead of React Query cache helpers. Add entries router
  and tags service as event sources.
- CLAUDE.md: Add collections/, hooks/, cache/, trpc/ to project
  structure under src/lib/.
- Mark optimistic-updates-and-cache-management.md and
  query-normalization-plan.md as superseded by the collections
  migration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…b6-3449f7069dd6

Update docs for TanStack DB collections migration
Remove redundant Suspense boundaries that duplicated logic already
handled by TanStack DB collections. Title components now read directly
from collections (subscriptions, tags) instead of using useSuspenseQuery
with a fallback that did the same collection read. Entry list Suspense
replaced by dynamic() loading prop since SuspendingEntryList handles
its own loading state internally.

- Delete EntryListFallback.tsx (143 lines of duplicated filter logic)
- Replace SubscriptionTitle/TagTitle useSuspenseQuery with collection reads
- Remove TitleFallback component (merged into EntryListTitle)
- Add subscription upsert from validation query to populate collection
- Move skeleton to dynamic() import loading prop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…e1-c63344cda4ca

Simplify Suspense boundaries after TanStack DB migration
The title component was using non-reactive collection.get() calls which
return a snapshot but don't trigger re-renders when the collection
updates. On direct page load the collection is empty, so the skeleton
rendered and never updated.

Fix: extract EntryListTitle into its own component using useLiveQuery
with findOne() for reactive lookups. When the subscription/tag data
arrives in the collection (via SSR prefetch hydration or the validation
query's upsert), useLiveQuery automatically re-renders with the title.

Loaded via dynamic() with ssr: false since useLiveQuery uses
useSyncExternalStore without getServerSnapshot.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix title not loading on direct page load
The focus-existing + postMessage approach from #483 doesn't work on
Android — the app state still gets destroyed before showing the result.
Revert to always redirecting URL shares to /save, which saves the
article and auto-closes itself, providing more useful behavior.

- Remove launch_handler from PWA manifest
- Simplify service worker to redirect URL shares to /save instead of
  saving directly via API and redirecting back to the app
- Remove share result query param handling, launchQueue consumer, and
  service worker message listener from RealtimeProvider

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…64-4e306ebc05b9

Switch PWA share target back to auto-closing save page
claude and others added 5 commits February 9, 2026 06:24
…573)

The onSuccess handlers were performing two rounds of collection updates:
first through applyWinningStateToCache (timestamp-based merging), then
unconditionally again. The second round was wasted work when allComplete
was true, and could cause state flickering when false by bypassing the
timestamp-based merging logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Since TypeScript uses structural typing, the extra `_sortMs` property
on SortedEntryListItem is harmless when passed to components expecting
EntryListData. Remove the manual field-by-field mapping that stripped
`_sortMs`, and pass stableEntries directly to avoid unnecessary object
allocations on every render.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create shared event schemas at src/lib/events/schemas.ts with Zod
validation for all sync event types, replacing ~290 lines of manual
typeof checks in parseEventData with a 5-line Zod safeParse call.
The manually-defined TypeScript interfaces in event-handlers.ts are
now inferred from the Zod schemas.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…scription (#578)

The tags collection's `select` function was writing uncategorized counts
to the counts collection as a side effect. Since `select` can be called
multiple times by TanStack Query, this violated the expected contract
that `select` is a pure transformation.

Instead, use `queryClient.getQueryCache().subscribe()` to watch for
tags.list query updates and sync uncategorized counts in response.
Also seeds from SSR-prefetched data on initialization. The subscription
is cleaned up when the TRPCProvider unmounts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The createConnection function was making a fetch() request to check for
503 status before opening an EventSource to the same URL, causing two
HTTP requests per connection attempt. Now EventSource is created directly
and error handling switches to polling mode when the connection fails.

Also fixes pre-existing lint issues:
- performSyncRef updated during render instead of in useEffect
- cleanup/stopPolling calling setState synchronously in effect body
  (replaced isPollingMode state with derivation from connectionStatus)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
claude and others added 4 commits February 9, 2026 06:53
The entry_state_changed SSE event now includes subscriptionId,
previousRead, and previousStarred fields. This allows other tabs/devices
receiving the event to compute and apply count deltas for subscription,
tag, uncategorized, and global unread/starred counts.

Server changes:
- Add subscriptionId, previousRead, previousStarred to EntryStateChangedEvent
- Pass these fields from markRead and setStarred mutations
- Update SSE event parser to validate and forward new fields

Client changes:
- Add optional fields to entryStateChangedEventSchema (optional so sync
  polling events that lack previous state still validate)
- Compute and apply count deltas in the event handler when previous
  state is available

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…576)

handleSubscriptionCreated/Deleted was using unreadCount for both the
totalDelta and unreadDelta arguments to adjustEntriesCountInCollection.
When a subscription is deleted after some entries have been read,
totalCount != unreadCount, so the total entry count would be decremented
by the wrong amount.

This adds totalCount as a new field throughout the subscription data
model (service, tRPC schema, SSE events, Zod schemas, sync endpoint)
and uses it for the totalDelta parameter in both handlers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The view collection (TanStack DB query-backed collection) used
staleTime: Infinity and was never invalidated when new entries arrived
via SSE or when the user navigated. Additionally, all call sites used
utils.entries.list.invalidate() which only invalidated the React Query
cache — not the view collection's separate query cache.

Changes:
- Add invalidateActiveView() method to Collections interface
- Set it in useViewEntriesCollection when the backing query is created
- Call it on new_entry SSE events so new articles appear in the list
- Call it on subscription delete so removed entries disappear
- Replace utils.entries.list.invalidate() with invalidateActiveView()
  in SuspendingEntryList, Sidebar, useEntryMutations, FileUploadButton
- Update FRONTEND_STATE.md to reflect the new invalidation pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
useRef<T>() without an initial argument triggers TS2554 in strict mode.
Pass undefined explicitly to satisfy the type checker.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@brendanlong brendanlong deleted the tanstack-db branch February 18, 2026 04:44
@brendanlong brendanlong restored the tanstack-db branch February 18, 2026 05:45
@brendanlong brendanlong reopened this Feb 18, 2026
claude and others added 2 commits February 21, 2026 03:21
Incorporates master features into the TanStack DB branch:
- Sidebar unread-only toggle (useSidebarUnreadOnly hook)
- Best/Recently Read nav links with score-based feed
- Server sync event schema (serverSyncEventSchema)
- Exported event schemas for shared use
- Updated dependencies to latest versions

Keeps tanstack-db architectural changes:
- TanStack DB collections for reactive state
- Rich event schemas (subscriptionId, previousRead/Starred, totalCount)
- Deleted entry-cache.ts and EntryListFallback.tsx (replaced by collections)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts:
#	src/components/layout/SidebarNav.tsx
claude and others added 4 commits February 21, 2026 04:20
Move entry count seeding from TRPCProvider into createCollections and
add a query cache subscription that watches for entries.count query
updates. This mirrors the existing pattern for uncategorized counts
(via tags.list subscription) and ensures counts are synced even when
SSR-prefetched data hasn't arrived in the cache at initialization time.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Handle missing feedType in new_entry SSE events by defaulting to "web"
  so count updates aren't skipped entirely when the server omits this field
- Change tag subscriptions staleTime from Infinity to 60s so expanding a
  tag section after SSE-driven count drift will trigger a refetch that
  corrects the counts
- Document the acceptable tradeoff where tag/subscription count updates
  are skipped when subscription data isn't loaded (Goal 4)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…624)

After markAllRead, subscriptions in the TanStack DB global collection
still showed their old unread counts because the onSuccess handler only
invalidated React Query caches without updating the local collection.

Add zeroSubscriptionUnreadForMarkAllRead() to writes.ts that zeroes
unread counts based on the mutation's filter (subscriptionId, tagId,
or all subscriptions). Call it in the markAllRead onSuccess handler.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove dead `utils.entries.list.invalidate()` calls that targeted
  query keys no longer subscribed to after TanStack DB migration
- Remove unsafe `as unknown as Subscription` type cast in
  handleSubscriptionCreated (SubscriptionData is structurally compatible)
- Deduplicate EntryType definition: canonical source is now
  entries-list-input.ts, re-exported from useEntryMutations.ts
- Remove unused `utils` variable in FileUploadButton

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
claude and others added 4 commits February 21, 2026 04:42
Tests cover:
- zeroSubscriptionUnreadForMarkAllRead (#624 fix): all filter modes
  (no filter, subscriptionId, tagId), already-zero subscriptions, null
  collections, and multi-tag edge cases
- upsertEntriesInCollection: insert, update, mixed, null/empty
- updateEntryReadInCollection: read/unread, missing entries, null
- updateEntryStarredInCollection: star/unstar, missing, null
- adjustEntriesCountInCollection: delta adjustments, floor at zero
- setEntriesCountInCollection: create new and update existing records
- adjustSubscriptionUnreadInCollection: deltas, floor at zero, bulk
- set/setBulk subscription unread counts
- add/remove/upsert subscriptions
- updateEntryScoreInCollection and updateEntryMetadataInCollection
- uncategorized count adjustments (unread and feed count)
- createCollections count seeding from SSR-prefetched data (#622 fix)
- Query cache subscription syncing counts on async updates
- Cleanup function stops syncing

All tests use real TanStack DB collections (not mocks). 95 tests total.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All callers of publishNewEntry() already provide feedType. Make it
required in the server function signature, the NewEntryEvent type,
the client Zod schema, and the SSE serialization. This removes the
need for a client-side fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per-tag subscription collections (rendered by TagSubscriptionList) were
independent from the global subscriptions collection. When mutations
updated unread counts in the global collection, the per-tag collections
never received the update, so sidebar counts inside expanded tag
sections stayed stale.

Fix by tracking active tag subscription collections in a Map on the
Collections object. TagSubscriptionList registers/unregisters its
collection on mount/unmount. All subscription write functions now
propagate changes to every registered tag subscription collection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The readChanged sort mode had two issues:

1. Server returned all entries because readChangedAt defaults to NOW() on
   row creation. Unread entries the user never interacted with appeared in
   the list. Fix: add read=true filter when sortBy=readChanged.

2. Toggling read/unread didn't update the sort order because the markRead
   mutation's onSuccess didn't invalidate the active view collection. The
   client had no way to know readChangedAt changed without refetching.
   Fix: call invalidateActiveView() after markRead succeeds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
claude and others added 2 commits February 21, 2026 05:41
New entries get read_changed_at=NULL by default. It's only set when the
user explicitly changes read state (mark read, mark unread, auto-read
on open). This lets the Recently Read list filter to entries the user
has actually interacted with.

Migration backfills NULL for entries that were never explicitly toggled
(has_marked_read_on_list=false AND has_marked_unread=false AND read=false).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Now that read_changed_at is nullable, add IS NOT NULL filter when
sortBy=readChanged so only entries the user has explicitly toggled
appear in the Recently Read list.

Also update all readChangedAt idempotency checks (markRead, markAllRead,
MCP markEntriesRead, GReader mark-all-as-read) to handle NULL with
(read_changed_at IS NULL OR read_changed_at <= changedAt). NULL always
allows the update since it means this is the first explicit change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants