Cannot create transient activities in ActivityPub API

[evanp]

@ThisIsMissEm notes on the mailing list:

if the a server is receiving via client to server an object that is an activity (because activities all extend from Object), and that activity has no object ID, then this language is saying one will be allocated, which would mean via C2S you could not send transient activities.

This is true. Do we need a way to say, I want this activity to have no ID, through the API?

[trwnh]

I am inclined to say "no", as the point of the outbox is to publish activities more than it is to deliver them. When you POST to an outbox, the main responsibility of the outbox is to assign a URI and return that as the Location header in an HTTP 201 Created response. (This really puts the "Pub" in "ActivityPub".)

If you don't assign a URI to an activity, then what is the outbox supposed to do? We might assume the payload should be delivered to recipients but not persisted in the outbox collection. But then, is a 201 Created response still appropriate, or should we expect a 200 OK instead? It doesn't make sense to persist the activity but not assign it a URI.

I think the main place where a transient activity could make sense is in POST to inbox instead of outbox, as might be more common with Linked Data Notifications. Alternatively, with proposals like using Server-Sent Events to stream activities from some event source, a client might be interested in directly published activities (e.g. a Like activity as received in the inbox) or indirect transient activities (e.g. an Add activity describing the Like being added to an as:likes collection).

So, rephrased or reframed, the question of "creating transient activities" is more like "delivering activities without creating them in the outbox". If you weren't using HTTP Message Signatures or if you controlled your own private keys, you could POST to inbox directly from a client without ever passing through an outbox. But for cases where signed HTTP messages are required and the outbox holds custodial keys, you would probably need some kind of "passthrough mode" or "proxy delivery endpoint" similar to as:proxyUrl but for delivering instead of fetching.

[silverpill]

Any mention of "transient" should be removed from the specification.

1. The concept of transient activity is not defined. There is only this vague sentence in section 3.1 Object Identifiers:

(short lived activities that are not intended to be able to be looked up, such as some kinds of chat messages or game notifications).

2. It conflicts with another concept, anonymous objects, which does have a precise definition:

An ID explicitly specified as the JSON null object, which implies an anonymous object (a part of its parent context)

3. Transient objects can't be authenticated, because fetching object by its id is the primary authentication method in ActivityPub.

4. There is no practical benefit in having this feature. The cost of generating an identifier is nearly zero.

5. Transient objects are not used by implementers. I know only one implementation that uses them, and the usage is limited to a single operation, so that is probably not intentional.

[evanp]

@silverpill I think Mastodon uses a lot of non-dereferenceable non-null activity IDs, for likes and follows and so on, which is the worst of both worlds, honestly.

[ThisIsMissEm]

@evanp yes, correct, it should really use dereferenceable IDs if it uses IDs at all

[silverpill]

I don't find non-dereferenceable IDs problematic. Maybe the object doesn't exist anymore, or maybe you're not allowed to view it -- there could be many reasons to return an error.

Mastodon uses fragment IDs for some activities, that's more annoying, but better than having no id because you can still reference these activities in Undo, Accept and Reject.

[TallTed]

Given that the IDs being minted are within the control of the minter, such that the minter can guarantee that these IDs will only ever be used to reference one and only one entity, it doesn't matter whether or not they are dereferenceable.

Linked Data best practices do strongly suggest that dereferencing an ID should lead to useful information about the entity identified by that ID, but the information provided by the server is (as always) at its discretion.

[evanp]

So, I think the question we have right now is: is this an error in the text? I think that at best it's an error in the interface design; we have a type of activity (id-less) that can be sent through the federation protocol but not through the social API.

I see four ways to implement submitting transient activities:

• No id in POST body -> no id in object This seems like the most straightforward. However, it blocks the most mainline way to submit persistent activities (for which an id is generated).
• A special value for id in the POST body Something like as:NoID or rdf:nil. A null is incompatible with JSON-LD so we can't use that. This isn't compatible with the text that says specified id values will be overwritten.
• Distinguished by type. The server knows which activity types are transient and which are persistent. If it gets a Like activity, that should be persistent. If it gets a https://chat.example/type/Message or https://game.example/type/Notification type activity, that should be transient, and no id should be generated.
• A special property on the POST body. This seems like the best way forward. A flag that indicates the user's or client's intent for the activity to be transient.

I think the last one is the best option. I'm going to mark this issue as needs FEP, since this could be implemented as an extension, but also Next Version, since it seems like pretty mainline functionality.

[github-actions]

This issue has been labelled as potentially needing a FEP, and contributors are welcome to submit a FEP on the topic.
Note that issues may be closed without the FEP being created; that does not mean that the FEP is no longer needed.

[evanp]

I am irritated to note that the text around id generation is, well, crappy:

In Client to Server Interactions, we have this text:

If an Activity is submitted with a value in the id property, servers MUST ignore this and generate a new id for the Activity. Servers MUST return a 201 Created HTTP code, and unless the activity is transient, MUST include the new id in the Location header.

It calls out an exceptional case -- when the id is provided -- without describing the base case, when it is not.

It also notes that the activity can be transient, but it doesn't say how you'd define that.

[trwnh]

it's an error in the interface design; we have a type of activity (id-less) that can be sent through the federation protocol but not through the social API

this is not necessarily an error at all; it's more that the outbox is defined for publishing while delivery is optional (see #482 for more on this), while the desired behavior here is skipping the publishing but delivering nonetheless. less of an "activitypub outbox" and more of an "http POST proxy". it might be possible to make the same endpoint handle both use cases, but this would in effect overload the outbox as already defined, so it would be better (and more backwards-compatible) to define a separate endpoint for "proxy this notification using my actor's (bound, server-managed, custodial) keys to sign the delivery to inbox". perhaps this kind of proxying would also be useful for Linked Data Notifications? you can think of it as a pipeline of the following behaviors from the client's perspective:

• client submits activity input to outbox
• outbox generates activity from input, assigns it an id, returns 201 Created to client, delivers activity to discovered inboxes

but you can also separate the outbox's concerns into a storage component and a delivery component:

• outbox: generate the published activity and discover inboxes
• storage: persist the activity and assign an id
• agent: perform the http request for delivery to inbox

the logical pipeline would be more like this:

• client submits activity input to outbox (start of client's http request)
• outbox generates activity from input
• outbox persists generated activity to storage at id
• outbox returns id to client (end of client's http request)
• outbox discovers inboxes from addressed actors/collections
• http agent delivers to those inboxes

sequenceDiagram
  Client ->>+ Outbox: Activity input
  Outbox ->> Outbox: Generate activity from input
  Outbox <<->> Storage: Persist the activity at <id>
  Outbox ->>- Client: HTTP 201 Created<br>Location: <id>
  Outbox ->> HTTP Agent: Inboxes discovered from addressing properties
  loop for each inbox
    HTTP Agent ->> Inbox: Linked Data<br>Notification
  end

Loading

in monolithic deployments, the server is typically the outbox, storage, and delivering agent all at once. this doesn't have to be the case, though -- the roles are separated by concerns, so we could assign roles differently if we wanted to. if http messages are being signed, then the keys are bound to the http agent doing deliveries.

in fact, there's nothing really stopping you from having the client be the http agent, since you pass the location back to the client anyway. it's just redundant with the expectation that the outbox is already doing this for you (probably -- see #482 again). assuming signed http messages, this agent role can be fulfilled by the client (with client keys), the outbox (with custodial keys), or something else (also with custodial keys, but not accessible by the outbox). if "something else", then the client can skip the outbox and communicate directly with the http agent; since the activity was never assigned an ID, it is therefore transient (i.e. not persistent).

in this modified sequence diagram, the outbox is preserved as a generator/adapter to ensure activities match the S2S format instead of the C2S format (see #477 for discussion around "C2S partial Update" vs "S2S full Update" as an example of when this matters). but the client is now responsible for coordinating delivery with the http agent, instead of the outbox. the client is free to publish without delivering, or deliver without publishing, or both (or neither):

sequenceDiagram
  Client ->>+ Outbox: Activity input
  Outbox ->> Outbox: Generate activity from input
  Outbox ->> Storage: Persist the activity
  Storage ->> Outbox: ID assigned
  Outbox ->>- Client: HTTP 201 Created<br>Location: <id>
  Client ->>+ HTTP Agent: Payload including id + inboxes to notify
  loop for each inbox
    HTTP Agent ->> Inbox: Linked Data<br>Notification
  end
  HTTP Agent ->>- Client: HTTP 200/202/etc delivery report

Loading

if you eliminate the outbox, it would look like this instead:

sequenceDiagram
  Client ->>+ Storage: Persist the activity
  Storage ->>- Client: ID assigned
  Client ->>+ HTTP Agent: Payload including id + inboxes to notify
  loop for each inbox
    HTTP Agent ->> Inbox: Linked Data<br>Notification
  end
  HTTP Agent ->>- Client: HTTP 200/202/etc delivery report

Loading

[trwnh]

I see four ways to implement submitting transient activities:

• No id in POST body -> no id in object This seems like the most straightforward. However, it blocks the most mainline way to submit persistent activities (for which an id is generated).

-1 on this because id being assigned is in many ways the primary role of the outbox (and why it returns a 201 Created).

• A special value for id in the POST body Something like as:NoID or rdf:nil. A null is incompatible with JSON-LD so we can't use that. This isn't compatible with the text that says specified id values will be overwritten.

-1 as currently written, because null isn't the only problematic option here -- using a subject like rdf:nil or as:noID is technically making statements about those subjects, not about the activity itself. (you wouldn't say rdf:nil has an actor, would you? rdf:nil is already the empty list whose rdf:first value is rdf:nil and the rdf:rest of the list is also rdf:nil.)

now, that -1 could become a +0 or higher because there is an option that technically works here without issue, as long as the outbox understands it and doesn't ignore/overwrite it as it's currently specified that it MUST: blank node identifiers. we could say that if the outbox receives an activity whose id is a blank node identifier (serialized in JSON-LD as prefixed with _:), then the outbox MUST NOT overwrite this identifier. for example:

POST /outbox HTTP/1.1
Host: foo.example
Content-Type: application/activity+json

{
  "id": "_:activity",
  "actor": "https://foo.example/actor/",
  "summary": "This is an activity with a blank node identifier."
}

the equivalent Turtle would be like so:

@prefix as: <https://www.w3.org/ns/activitystreams#>.

_:activity as:actor <https://foo.example/actor/>.
_:activity as:summary "This is an activity with a blank node identifier.".

this concept extends beyond just the activity, as it applies to other nodes without id at the data level:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "actor": {
    "name": "Anonymous Coward"
  },
  "summary": "This is an activity with a blank node identifier."
}

@prefix as: <https://www.w3.org/ns/activitystreams#>.

_:1 as:actor _:2.
_:1 as:summary "This is an activity with a blank node identifier.".
_:2 as:name "Anonymous Coward".

• Distinguished by type. The server knows which activity types are transient and which are persistent. If it gets a Like activity, that should be persistent. If it gets a https://chat.example/type/Message or https://game.example/type/Notification type activity, that should be transient, and no id should be generated.

-1 because the outbox can never know this on its own, unless you constrain its semantics to disallow publishing certain arbitrary activity types.

• A special property on the POST body. This seems like the best way forward. A flag that indicates the user's or client's intent for the activity to be transient.

I think the last one is the best option.

0 on this, but maybe it should be in a header and not in the body? if you do a property like transient: true, it may get transmitted as part of the activity unless special-cased like bto/bcc (which imo is in need of its own reconsideration). drawing from AtomPub there is the Slug header which could maybe be used here (or something like it): https://datatracker.ietf.org/doc/html/rfc5023#section-9.7

   Slug is an HTTP entity-header whose presence in a POST to a
   Collection constitutes a request by the client to use the header's
   value as part of any URIs that would normally be used to retrieve the
   to-be-created Entry or Media Resources.

   Servers MAY use the value of the Slug header when creating the Member
   URI of the newly created Resource, for instance, by using some or all
   of the words in the value for the last URI segment.  Servers MAY also
   use the value when creating the atom:id, or as the title of a Media
   Link Entry (see Section 9.6).

   Servers MAY choose to ignore the Slug entity-header.  Servers MAY
   alter the header value before using it.  For instance, a server might
   filter out some characters or replace accented letters with non-
   accented ones, replace spaces with underscores, change case, and so
   on.

say something like Slug: @null or some other reserved keyword (preferably not something that could be confused for an actual slug...) would result in the outbox skipping id generation.

---

in summary, we arrive at the following erratum:

Problem statement: In Section 6 "Client to Server Interactions", a paragraph reads: "If an Activity is submitted with a value in the id property, servers MUST ignore this and generate a new id for the Activity. Servers MUST return a 201 Created HTTP code, and unless the activity is transient, MUST include the new id in the Location header." This paragraph alludes to transient activities, but provides no mechanism for clients to signal that an activity should be transient.

Possible correction: Modify the server requirement to allow for blank node identifiers, which would not be ignored and overwritten by the outbox. The paragraph could instead read: "If an Activity is submitted with a value in the id property that is not a blank node identifier (an identifier beginning with _:), servers MUST ignore this and generate a new id for the Activity. Servers MUST return a 201 Created HTTP code, and unless the activity is transient (specified by having a blank node identifier), MUST include the new id in the Location header."

Alternative possible correction: Modify the server requirement to skip id generation when some header is present. [I'm not writing the rest of this until/unless there's consensus on a specific mechanism, e.g. Slug: as:NoID or something at least as specific]