Conversation
Outlines the transition from a mutable document-based system to an immutable versioning model for published forms. This includes new API endpoints for draft and versioned forms, addressing limitations in the current system and enhancing caching, submission tracking, and user experience during form completion.
| | Endpoint | Description | | ||
| | --------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | ||
| | `GET /api/v3/forms/:form_id/draft` | Returns the current draft form document JSON (mutable, changes as the form creator edits) | | ||
| | `GET /api/v3/forms/:form_id/versions/:form_version` | Returns an immutable, versioned form document. Once created, this content never changes. | |
There was a problem hiding this comment.
Do we want live form documents to be actually immutable - or immutable from an end user perspective? The main issue we might encounter here is when we make internal changes to form structure such as adding/removing/changing attributes.
It would make it harder to make changes to forms-runner and hard to support users reverting changes if we don't also update the structure of historic form documents when we make breaking changes to the format.
There might be some things we can do to lessen the need for altering existing form documents. Currently when we add attributes to a form, we always back-fill the attribute on form documents. Potentially we can change our approach to this and fall back on default values when an attribute isn't present rather than updating it in the database.
There was a problem hiding this comment.
V good point — I think ultimately we want the semantic content to be immutable.
We could handle changes in schema in a few ways:
-
Include a schema version in the form document. This makes it explicit to consumers how to interpret the content. It may mean we'd need to support multiple schema versions, but in practice, making sure most changes are backwards-compatible minimises the complexity around this. (Could reserve breaking schema changes for a new API version.)
-
Weaken the immutability by keeping the semantic content the same but allowing the schema to be updated. i.e. migrate old documents to newer schemas. We wouldn't be able to cache "forever", but schema changes are probably infrequent enough that caching would still be useful.
There was a problem hiding this comment.
I'd probably lean towards 1. as it gives stronger guarantees for the API, in way that is less likely to break forms-runner.
There was a problem hiding this comment.
I've updated the ADR with a section on API changes.
| - Audit trail. The full history of published form versions is preserved and addressable. | ||
|
|
||
| ### Negative | ||
|
|
There was a problem hiding this comment.
Another negative is increased complexity for form processors; making it so that an old version of a form can be submitted after the new version has been made live means that submission processors need to be able to handle both old and new version submissions for a time.
There was a problem hiding this comment.
That's true, but I think form processors already have this problem today. Unless a form owner can update the form and their processing system at exactly the same time, there's always a window where submissions come in against a version the processor wasn't expecting. The current setup just makes that invisible.
Explicit version numbers actually make it easier to handle. A processor can see which version a submission was made against and branch accordingly (e.g. if version 2, do X, otherwise do Y), rather than having to infer what changed from the shape of the data.
There was a problem hiding this comment.
Okay, so as part of the changes for this ADR we'll expose the version information to form processors, and inform them of the change?
There was a problem hiding this comment.
Yup - added info that submissions would reference the form version. And yup, would be good to inform them.
|
|
||
| ## Decision | ||
|
|
||
| We will introduce an immutable versioning model for published forms, exposed through a new v3 API. The key changes are: |
There was a problem hiding this comment.
I think there's a way we could achieve the same aims and outcomes of this ADR but without needing a new API version; would you be interested in that or would you prefer to have a new API version?
There was a problem hiding this comment.
Ooo! Yes always interested in hearing ideas!
There was a problem hiding this comment.
I think if we added the /versions endpoints you're suggesting to the v2 API, and added a version link to the form documents, that would cover the same functionality more or less?
So for a form document that's been made live, regardless of where it's retrieved from, you'd see something like:
{
form_id: "aB123x",
name: "Test form",
...
links : {
version: "https://api.forms.service.gov.uk/api/v2/forms/aB123x/versions/9"
}
}
There was a problem hiding this comment.
Ah thats a nice idea.
I think my main concern with doing this in v2: we’d end up with a slightly confusing API shape where the old state-based endpoints still exist alongside the new version-based ones.
For example, it becomes unclear what /api/v2/forms/:id/archived is meant to represent
There was a problem hiding this comment.
I think it still represents the form document as it was when it was archived? We have that so that it's easy for forms-runner to preview the form after its been archived but changes have been made to it. With endpoints for versions we could alternatively do that by sending them to a preview for that version, which I guess what you're proposing currently, but both still seem valid to me?
It doesn't help that I've thought about this so much that it's hard to imagine looking at it for the first time, so I'm unsure about what you mean about it being unclear 😅
When I was thinking about the v2 API I thought about the state endpoints as branches/tags in a git repository, or symbolic links in a filesystem, so that's where I'm coming from.
Added a section on hard submission deadlines to ADR048, outlining the need for strict cutoff times for certain forms. This change clarifies that archiving will no longer serve as a method to prevent future submissions and suggests a new implementation approach for managing submission deadlines.
Added a section detailing how each submission will now store the `form_version` it was made against, helping proccess handle changes to the form.
Added a section detailing schema changes
3 participants
Outlines an immutable versioning model for published forms. This includes new API endpoints for draft and versioned forms, addressing limitations in the current system and enhancing caching, submission tracking, and user experience during form completion.