Skip to content

feat: universal tags with app-specific paths#777

Open
SHAcollision wants to merge 9 commits intomainfrom
feat/universal-tags-app-specific-paths
Open

feat: universal tags with app-specific paths#777
SHAcollision wants to merge 9 commits intomainfrom
feat/universal-tags-app-specific-paths

Conversation

@SHAcollision
Copy link
Copy Markdown
Collaborator

Summary

  • Implements universal tags (Alternative B: app-specific paths) from the
    universal_tags_specs.md specification
  • Tags at /pub/<appname>/tags/TAG_ID are now indexed as generic Resource
    nodes, enabling cross-app tag aggregation without forking Nexus
  • App namespace on TAGGED relationships enables efficient feed filtering
    by source app (e.g., ?app=mapky, ?app=eventky)
  • Fully additive — zero changes to existing Post/User tag behavior

What changed

nexus-common (models + graph):

  • New resource/ module: URI normalization, BLAKE3 resource_id, classify_uri, ResourceDetails, TagResource, ResourceStream with 8 Redis sorted set indexes
  • Graph DDL: Resource node constraint + scheme index
  • Graph queries: create_resource_tag (MERGE), delete_tag extended with resource_id + orphan cleanup, resource_tags, resource_stream
  • TaggedType::Resource enum variant

nexus-watcher (event processing):

  • Second-chance intercept in processor.rs, when Event::parse_event() fails for non-pubky.app paths, universal_tag::try_handle() pattern-matches */tags/* and processes the tag
  • sync_put_resource: classifies URI -> InternalKnown delegates to existing flow, InternalUnknown/External creates Resource node
  • del_sync_resource: decrements Redis indexes, orphan cleanup via Cypher
  • Zero changes to Event struct, telemetry, retry, or serialization

nexus-webapi (API):

  • GET /v0/resource/:resource_id/tags , tag details with WoT support
  • GET /v0/resource/by-uri?uri=... , lookup by raw URI (normalizes internally)
  • GET /v0/resource/:resource_id/tags/:label/taggers , tagger list
  • GET /v0/stream/resources?app=mapky&tags=bitcoin&sorting=taggers_count , resource feed
  • GET /v0/stream/resources/ids , cursor-paginated resource IDs
  • Input validation: 32-char hex resource_id, max 5 tags, empty tag filtering

nexusd (migration):

  • ResourceNodeSetup migration applying DDL constraints + indexes

Test plan

  • 16 unit tests for URI normalization (all spec test vectors), resource_id, classify_uri, extract_scheme
  • 7 unit tests for universal_tag path parsing + event line parsing
  • 17 WebAPI integration tests against live Neo4j: resource tags, by-uri lookup, taggers, stream with app/tag/combined filters, sorting, pagination
  • 6 watcher integration tests (compile-verified): full PUT/DEL cycle, InternalKnown delegation, cross-app dedup
  • 65 unit tests pass, 304 existing integration tests pass, 0 regressions

@SHAcollision SHAcollision force-pushed the feat/universal-tags-app-specific-paths branch 2 times, most recently from f00aeb6 to 61d5705 Compare March 25, 2026 14:02
@tipogi tipogi added the 👀 watcher Nexus indexer related operations label Mar 25, 2026
@SHAcollision SHAcollision force-pushed the feat/universal-tags-app-specific-paths branch 4 times, most recently from ae50791 to e515926 Compare March 26, 2026 01:17
@SHAcollision SHAcollision force-pushed the feat/universal-tags-app-specific-paths branch from e515926 to af41082 Compare March 26, 2026 01:40
@SHAcollision SHAcollision changed the title (WIP) feat: universal tags with app-specific paths feat: universal tags with app-specific paths Mar 26, 2026
@SHAcollision SHAcollision marked this pull request as ready for review March 26, 2026 01:40
@SHAcollision SHAcollision requested a review from tipogi March 26, 2026 01:40
pagination: Pagination,
order: SortOrder,
sorting: &ResourceSorting,
tags: &Option<Vec<String>>,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: This forces callers to construct and own an Option<Vec<String>>. The function only reads the contents.

tags: Option<&[String]>,

Copy link
Copy Markdown
Collaborator Author

@SHAcollision SHAcollision Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, Option<&[String]> is more idiomatic. But the change ripples through get_resource_keys, get_resource_keys_from_graph, can_use_index, build_index_key, and the webapi caller deserialization. Tracking as follow-up to keep this diff focused on the review fixes.


/// Determines whether a query can be satisfied by a pre-computed Redis sorted set.
fn can_use_index(
_sorting: &ResourceSorting,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If _sorting isn't used at all in the function, why not remove it as an arg?

.param("tag_id", tag_id)
/// Deletes a tag relationship created by a user and retrieves relevant details about the tag's target.
/// When `app` is Some, filters by app to prevent cross-app deletion for Resource tags.
/// When `app` is None, behaves as before (Post/User tags have no app property).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When app is None, behaves as before (Post/User tags have no app property).

Judging by the query DELETE_TAG_WITHOUT_APP below, this is not true.

Providing no app as arg will end up deleting that tag of any app, because the query won't filter by app. It doesn't filter for "app must be NULL".

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This behavior is intentional. The None path is used by two callers:

  1. Standard event flow (events/mod.rs): For pubky.app tags, these TAGGED relationships have no app property, so no filter is needed.
  2. InternalKnown DEL fallback (handle_del): When a tag at /pub/mapky/tags/TAG_ID targets a Post/User, sync_put_resource delegates to the standard flow, which creates a TAGGED relationship WITHOUT app. On DEL, handle_del tries Some("mapky") first (no match), then falls back to None, correctly matching the app-less relationship.

Resource tags created via the universal flow always have app set and are deleted via the Some(app) path. There's no collision risk because tag_id = blake3(uri:label) is deterministic per URI, a Post tag and a Resource tag would have different URIs and therefore different tag_ids.

Updated the doc comment to clarify this reasoning.

}
}

fn normalize_parsed_url(parsed: &Url) -> String {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Wouldn't it make more sense to move this to pubky-app-specs and call it there too, on the uri target of a tag?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed this is the right long-term home, we need to move this to pubky-app-specs, needs release cycle and cross-repo coordination. How abou keeping in nexus-common for now; and proposing a migration to pubky-app-specs as a separate effort to ease this PR going trough?


// Second-chance: try handling as universal tag at app-specific path
if maybe_event.is_none() {
if let Some(result) = crate::events::handlers::universal_tag::try_handle(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be incorporated into Event::parse_event. Conceptually its an extension of the event parsing logic, not of the event line processing (this fn).

That would

  • remove the need for these 5-6 extra levels of nesting (i.e. hard-to-reason-about logic)
  • remove the need for parse_event_line which duplicates the event parsing steps

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree it would reduce nesting. The tradeoff: Event::parse_event lives in nexus-common and uses ParsedUri::try_from from pubky-app-specs, which only recognizes pubky.app paths. Adding universal tag fallback there would couple nexus-common to app-specific path parsing logic that currently lives cleanly in nexus-watcher. The second-chance pattern keeps the concern separated. Worth revisiting when pubky-app-specs gains native support for app-specific paths (which would make ParsedUri::try_from handle them directly).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about #782 ?

It addresses the nesting issue I raised, but avoids the coupling risk you mentioned.

@SHAcollision SHAcollision requested review from ok300 and tipogi April 3, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

👀 watcher Nexus indexer related operations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants