Skip to content

feat: Call user events endpoint when indexing 3rd party HSs#726

Draft
ok300 wants to merge 31 commits intomainfrom
feat/dx-events-by-user
Draft

feat: Call user events endpoint when indexing 3rd party HSs#726
ok300 wants to merge 31 commits intomainfrom
feat/dx-events-by-user

Conversation

@ok300
Copy link
Copy Markdown
Contributor

@ok300 ok300 commented Feb 15, 2026

This PR updates HS indexing logic to rely on the new user events endpoint on the HS.

This PR acts as an "umbrella PR" for this DX feature. It is meant to contain all relevant changes and sub-PRs. It will remain a draft until the full feature is ready for final review.

Steps:

  • Bump pubky to relevant feature branch (done in 6a06e9c)
  • Bump pubky to v0.7.x (done in c6513d7)
  • Restructure NexusWatcher::start to start 3 threads, each with own tick: index default HS, index other HSs, resolve user-to-HS mappings
  • Implement "index default HS" logic
    • Only process events of an user if that user's HS is explicitly set to the default HS (e.g. do not blindly ingest all events)
  • Implement "index other HSs" logic
    • Switch to using new HS SDK user events endpoint
  • Implement "resolve user-to-HS mappings"
  • Avoid duplicated events through event hash (e.g. when user migrates across HSs)
  • Store user cursors per HS (one idea: when user leaves HS, persist cursor in sorted set)

ok300 and others added 3 commits February 15, 2026 12:58
…HS from run_all

- NexusWatcher::start now spawns 3 parallel tokio tasks:
  1. Default homeserver processing (run_default_homeserver)
  2. Other homeservers processing (run_all, excluding default HS)
  3. Reserved for future use (TODO placeholder)
- Added run_default_homeserver to TEventProcessorRunner trait
- Modified homeservers_by_priority to exclude the default homeserver
- Added Send + Sync bounds to TEventProcessorRunner trait
- Updated all tests and rustdocs

Co-authored-by: ok300 <106775972+ok300@users.noreply.github.com>
@ok300 ok300 added this to the 2026-Q1 milestone Feb 15, 2026
@ok300 ok300 added the 🕸️ decentralization Distributed events from homeservers label Feb 15, 2026
@ok300 ok300 self-assigned this Feb 19, 2026
@ok300 ok300 mentioned this pull request Feb 27, 2026
1 task
ok300 and others added 5 commits March 2, 2026 20:26
…s` (#761)

* refactor: extract periodic task loop into reusable run_periodic_tasks

* Fix periodic task ticks to skip missed intervals

Set MissedTickBehavior::Skip on the interval timer in run_periodic_tasks
so that if a task takes longer than its tick interval, missed ticks are
dropped instead of queuing up a burst of back-to-back invocations.

Added tests verifying that slow tasks don't accumulate queued ticks and
that fast tasks still tick at the expected cadence.

* chore: rename variables for clarity

* chore: make task_runner module private

* log: NexusWatcher::start logs task outcomes on exit

* chore: cargo fmt

* chore: limit visibility of Task structs to pub(crate)

* fix: correctly set task name for panicking tasks

* fix: ensure NexusWatcher::start returns Err if any task panics

---------

Co-authored-by: Claude <noreply@anthropic.com>
* nexus-api tests: use unseeded testnet

* deps: update pubky dependency to latest 0.7.x
@ok300 ok300 mentioned this pull request Mar 12, 2026
1 task
@ok300 ok300 force-pushed the feat/dx-events-by-user branch from c6513d7 to 5ad570a Compare March 19, 2026 17:13
@ok300 ok300 force-pushed the feat/dx-events-by-user branch from c905186 to 49af0a0 Compare March 30, 2026 10:44
tipogi and others added 5 commits April 3, 2026 11:06
* ref: runners and indexers

* chore: use runner pub function

* extract_retry_event_info -> handle_error (#785)

* remove get_homeserver_id fn (#784)

* Extract common TEventProcessorRunner::run (#789)

* Add HsEventProcessor rustdoc

* Add KeyBasedEventProcessor rustdoc

* Add HsEventProcessorRunner rustdoc

* Add KeyBasedEventProcessorRunner rustdoc

* Add TODO for KeyBasedEventProcessor::run_internal

* Extract no-op default TEventProcessorRunner::post_run fn

---------

Co-authored-by: aintnostressin <aintnostressin@proton.me>
Co-authored-by: ok300 <106775972+ok300@users.noreply.github.com>
* ref: runners and indexers

* chore: use runner pub function

* extract_retry_event_info -> handle_error (#785)

* remove get_homeserver_id fn (#784)

* TEventProcessor: expand logic of handle_error

* Clarify comments

* fix: remove InternalError from is_infrastructure

* Expand logic for EventProcessorError::is_infrastructure_err

* chore: remove unused imports

---------

Co-authored-by: tipogi <tipogi@protonmail.com>
Co-authored-by: aintnostressin <aintnostressin@proton.me>
@ok300 ok300 force-pushed the feat/dx-events-by-user branch from 6d6fae4 to ec43bd7 Compare April 3, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🕸️ decentralization Distributed events from homeservers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants