From 93b17a33cb038f023d097a6c5ed744b8806fd7e2 Mon Sep 17 00:00:00 2001 From: Sam Mesterton-Gibbons Date: Mon, 29 Sep 2025 16:43:20 +0100 Subject: [PATCH 1/6] adr: Add ADR for improving tag usability --- docs/README.md | 1 + docs/adr/0040-tag-usability-enhancements.md | 81 +++++++++++++++++++++ 2 files changed, 82 insertions(+) create mode 100644 docs/adr/0040-tag-usability-enhancements.md diff --git a/docs/README.md b/docs/README.md index b18e27c1..66d22989 100644 --- a/docs/README.md +++ b/docs/README.md @@ -71,5 +71,6 @@ For more information on how we use ADRs, see [here](./adr/README.md). | [0037](./adr/0037-improve-webhooks.md) | Proposal for improvements to the Webhooks endpoints | | [0038](./adr/0038-improved-storage-management.md) | Improved Storage Management | | [0039](./adr/0039-remove-pre-actions.md) | Proposal to remove pre-actions from storage allocation response | +| [0040](./adr/0040-tag-usability-enhancements.md) | Tag Usability Enhancements | \* Note: ADR 0004a was the unintended result of a number clash in the early development of TAMS which wasn't caught before publication diff --git a/docs/adr/0040-tag-usability-enhancements.md b/docs/adr/0040-tag-usability-enhancements.md new file mode 100644 index 00000000..de7c35d3 --- /dev/null +++ b/docs/adr/0040-tag-usability-enhancements.md @@ -0,0 +1,81 @@ +--- +status: "proposed" +--- +# Tag Usability Enhancements + +## Context and Problem Statement + +TAMS provides tags on Flows and Sources as an ad-hoc key-value store of data. +This enables some simple workflows without the need for an additional database of content, simply by using the TAMS API. +It also enables simpler interoperability, in cases where the tag store is enough to locate the content required in TAMS using e.g. a panel in a tool, rather than an integration with some other MAM. +Over the course of other work on TAMS (notably the exploration of fine-grained authorisation) it has become clear that some improvements to tags would be useful. + +## Considered Options + +* Option 1a: Add support for lists in tags +* Option 1b: Add support for lists in tags, require every item match +* Option 1c: Add support for lists in tags, add a query for all items matching +* Option 2: Make it possible to filter object instances on tags +* Option 3: Add tags to webhooks + +## Decision Outcome + +Chosen options: + +* Option 1a: Add support for lists in tags +* Option 2: Make it possible to filter object instances on tags +* Option 3: Add tags to webhooks + +Option 1a is chosen at present because there is no clear need for Option 1c, but it may be added in future without a breaking change to client implementations. + +Options 2 and 3 make the support and behaviour of tags uniform across Flows, Sources, Object Instances and Webhooks. + +### Implementation + +Implemented by + +## Pros and Cons of the Options + +### Option 1a: Add support for lists in tags + +Add the option that the value of a tag either be a string, or a list. +Where tags are queryable, make the `tag.{name}` query parameter accept a comma separated list of strings, and require that at least one item in the query matches at least one of the values in the list to return the resource. + +* Good, because it makes tags usable for cases when multiple options are present (e.g. authorisation attributes) +* Good, because it improves discoverability of well-tagged resources +* Good, because it allows for finding e.g. a Flow with any of a specific property in a list ("OR" queries) +* Bad, because it makes implementations more complex to handle checking lists in queries +* Bad, because it encourages more usage of TAMS as a MAM + +### Option 1b: Add support for lists in tags, require every item match + +As in Option 1a, however the `tag.{name}` query parameter requires that every item in the given list matches. + +* Good, because it enables queries finding e.g. a Flow with a number of properties ("AND" queries) +* Bad, because it requires knowing the precise list to query it using tags + +### Option 1c: Add support for lists in tags, add a query for all items matching + +As in Option 1a, with the addition of a `tag_every.{name}` query parameter where every item in the given list matches. + +* Good, because it enables both "AND" and "OR" queries +* Good, because it makes the tag query mechanism more complete +* Bad, because it adds another query parameter to implement +* Bad, because at the time of writing there is no concrete use case for "AND" queries +* Bad, because it encourages more usage of TAMS as a MAM + +### Option 2: Make it possible to filter object instances on tags + +Add `flow_tag.{name}` and `flow_tag.{exists}` to the `GET /objects/{objectId}` endpoint, to adjust the contents of the `referenced_by_flows` list down to a specific subset of Flows. + +* Good, because it avoids excessive pagination if an object is used by a very large number of Flows +* Good, because it enables the option to use tags to restrict access to Flows and associated objects +* Neutral, because it requires a change to specification and implementation +* Bad, because it encourages more usage of TAMS as a MAM + +### Option 3: Add tags to webhooks + +Add tags and tag-based querying to webhooks, in the same was as Flows and Sources. + +* Good, because it improves the ability of clients to manage large numbers of webhooks by locating them using tags +* Good, because it enables the option to use tags to restrict access to webhooks From b9b7977171ad049b395aa3494f76f4babb49541f Mon Sep 17 00:00:00 2001 From: Sam Mesterton-Gibbons Date: Mon, 29 Sep 2025 16:43:27 +0100 Subject: [PATCH 2/6] api: Make tag values optionally a list sem-ver: api-break Co-authored-by: James Sandford --- api/TimeAddressableMediaStore.yaml | 36 ++++++++++++++++++++---------- api/schemas/tags.json | 14 ++++++++++-- api/schemas/url-tag-list.json | 6 +++++ 3 files changed, 42 insertions(+), 14 deletions(-) create mode 100644 api/schemas/url-tag-list.json diff --git a/api/TimeAddressableMediaStore.yaml b/api/TimeAddressableMediaStore.yaml index cc5904e6..0f8371c7 100644 --- a/api/TimeAddressableMediaStore.yaml +++ b/api/TimeAddressableMediaStore.yaml @@ -370,10 +370,13 @@ paths: - name: tag.{name} in: query description: | - Filter on Sources that have a tag named {name} and with the given value. - {name} and the value MUST be URL encoded where special characters are present. + Filter on Sources that have a tag named {name} with a value in the given comma-seperated list of values. + The {name} and the value MUST be URL encoded where special characters are present. + Where the tag's value is a string, at least one of the given values will match. + Where the tag's value is an array, at least one value in the array will match at least one of the given values. + Partial string matches of the values are not valid. schema: - type: string + $ref: 'schemas/url-tag-list.json' - name: tag_exists.{name} in: query description: | @@ -428,10 +431,13 @@ paths: - name: tag.{name} in: query description: | - Filter on Sources that have a tag named {name} and with the given value. - {name} and the value MUST be URL encoded where special characters are present. + Filter on Sources that have a tag named {name} with a value in the given comma-seperated list of values. + The {name} and the value MUST be URL encoded where special characters are present. + Where the tag's value is a string, at least one of the given values will match. + Where the tag's value is an array, at least one value in the array will match at least one of the given values. + Partial string matches of the values are not valid. schema: - type: string + $ref: 'schemas/url-tag-list.json' - name: tag_exists.{name} in: query description: | @@ -814,10 +820,13 @@ paths: - name: tag.{name} in: query description: | - Filter on Flows that have a tag named {name} and with the given value. - {name} and the value MUST be URL encoded where special characters are present. + Filter on flows that have a tag named {name} with a value in the given comma-seperated list of values. + The {name} and the value MUST be URL encoded where special characters are present. + Where the tag's value is a string, at least one of the given values will match. + Where the tag's value is an array, at least one value in the array will match at least one of the given values. + Partial string matches of the values are not valid. schema: - type: string + $ref: 'schemas/url-tag-list.json' - name: tag_exists.{name} in: query description: | @@ -898,10 +907,13 @@ paths: - name: tag.{name} in: query description: | - Filter on Flows that have a tag named {name} and with the given value. - {name} and the value MUST be URL encoded where special characters are present. + Filter on flows that have a tag named {name} with a value in the given comma-seperated list of values. + The {name} and the value MUST be URL encoded where special characters are present. + Where the tag's value is a string, at least one of the given values will match. + Where the tag's value is an array, at least one value in the array will match at least one of the given values. + Partial string matches of the values are not valid. schema: - type: string + $ref: 'schemas/url-tag-list.json' - name: tag_exists.{name} in: query description: | diff --git a/api/schemas/tags.json b/api/schemas/tags.json index fec402b5..3276e09d 100644 --- a/api/schemas/tags.json +++ b/api/schemas/tags.json @@ -1,8 +1,18 @@ { "title": "Tags", - "description": "Key value is a freeform string.", + "description": "Key is a freeform string. Value is a freeform string, or an array of freeform strings.", "type": "object", "additionalProperties": { - "type": "string" + "anyOf": [ + { + "type": "string" + }, + { + "type": "array", + "items": { + "type": "string" + } + } + ] } } \ No newline at end of file diff --git a/api/schemas/url-tag-list.json b/api/schemas/url-tag-list.json new file mode 100644 index 00000000..7cbd6bb2 --- /dev/null +++ b/api/schemas/url-tag-list.json @@ -0,0 +1,6 @@ +{ + "title": "Query String Tag value list", + "description": "A list of tag values, formatted for use in query string parameters", + "type": "string", + "pattern": "^([^,]+(,[^,]+)*)?$" +} \ No newline at end of file From b5e2cc2598715adda3da4721b926a90dd4bf55ed Mon Sep 17 00:00:00 2001 From: Sam Mesterton-Gibbons Date: Mon, 29 Sep 2025 16:43:32 +0100 Subject: [PATCH 3/6] api: Allow filtering referenced_by_flows on tags Adds a way to filter the list in `referenced_by_flows` on `GET /objects/` based on the tags on that Flow. Co-authored-by: James Sandford --- api/TimeAddressableMediaStore.yaml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/api/TimeAddressableMediaStore.yaml b/api/TimeAddressableMediaStore.yaml index 0f8371c7..f4937555 100644 --- a/api/TimeAddressableMediaStore.yaml +++ b/api/TimeAddressableMediaStore.yaml @@ -2164,6 +2164,18 @@ paths: Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. schema: type: boolean + - name: flow_tag.{name} + in: query + description: | + Filter `referenced_by_flows` on tag values. This option is the same as the `tag.{name}` query parameter on the `/flows/` API endpoint. + schema: + type: string + - name: flow_tag_exists.{name} + in: query + description: | + Filter `referenced_by_flows` on tag names. This option is the same as the `tag_exists.{name}` query parameter on the `/flows/` API endpoint. + schema: + type: boolean - $ref: '#/components/parameters/trait_resource_paged_key' - $ref: '#/components/parameters/trait_paged_limit' responses: @@ -2249,6 +2261,18 @@ paths: Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. schema: type: boolean + - name: flow_tag.{name} + in: query + description: | + Filter `referenced_by_flows` on tag values. This option is the same as the `tag.{name}` query parameter on the `/flows/` API endpoint. + schema: + type: string + - name: flow_tag_exists.{name} + in: query + description: | + Filter `referenced_by_flows` on tag names. This option is the same as the `tag_exists.{name}` query parameter on the `/flows/` API endpoint. + schema: + type: boolean - $ref: '#/components/parameters/trait_resource_paged_key' - $ref: '#/components/parameters/trait_paged_limit' responses: From 3a352735b182a7e2a0c4b084259cce88cd7d3b52 Mon Sep 17 00:00:00 2001 From: Sam Mesterton-Gibbons Date: Mon, 29 Sep 2025 16:43:38 +0100 Subject: [PATCH 4/6] api: Add tag support to webhooks Co-authored-by: James Sandford --- api/TimeAddressableMediaStore.yaml | 40 ++++++++++++++++++++++++++++++ api/schemas/webhook.json | 3 +++ 2 files changed, 43 insertions(+) diff --git a/api/TimeAddressableMediaStore.yaml b/api/TimeAddressableMediaStore.yaml index f4937555..0d7da360 100644 --- a/api/TimeAddressableMediaStore.yaml +++ b/api/TimeAddressableMediaStore.yaml @@ -154,6 +154,26 @@ paths: tags: - Webhooks parameters: + - name: tag.{name} + in: query + description: | + Filter on webhooks that have a tag named {name} with a value in the given comma-seperated list of values. + The {name} could contain escaped characters to allow it to be used in a URL. + Where the tag's value is a string, at least one of the given values will match. + Where the tag's value is an array, at least one value in the array will match at least one of the given values. + Partial string matches of the values are not valid. + schema: + $ref: 'schemas/url-tag-list.json' + - name: tag_exists.{name} + in: query + description: | + Filter on webhooks that have a tag named {name} regardless of value. + The {name} could contain escaped characters to allow it to be used in a URL. + If set to true then the presence of the tag is filtered for. + If set to false then its absence is. + If left out then no filtering on tag presence is performed. + schema: + type: boolean - $ref: '#/components/parameters/trait_resource_paged_key' - $ref: '#/components/parameters/trait_paged_limit' responses: @@ -188,6 +208,26 @@ paths: tags: - Webhooks parameters: + - name: tag.{name} + in: query + description: | + Filter on webhooks that have a tag named {name} with a value in the given comma-seperated list of values. + The {name} could contain escaped characters to allow it to be used in a URL. + Where the tag's value is a string, at least one of the given values will match. + Where the tag's value is an array, at least one value in the array will match at least one of the given values. + Partial string matches of the values are not valid. + schema: + $ref: 'schemas/url-tag-list.json' + - name: tag_exists.{name} + in: query + description: | + Filter on webhooks that have a tag named {name} regardless of value. + The {name} could contain escaped characters to allow it to be used in a URL. + If set to true then the presence of the tag is filtered for. + If set to false then its absence is. + If left out then no filtering on tag presence is performed. + schema: + type: boolean - $ref: '#/components/parameters/trait_resource_paged_key' - $ref: '#/components/parameters/trait_paged_limit' responses: diff --git a/api/schemas/webhook.json b/api/schemas/webhook.json index 6caa2364..e64bbccb 100644 --- a/api/schemas/webhook.json +++ b/api/schemas/webhook.json @@ -81,6 +81,9 @@ "verbose_storage": { "description": "Whether to include storage metadata in the `get_urls` property in `flows/segments_added` events. This option is the same as the `verbose_storage` query parameter for the [/flows/{flowId}/segments](#/operations/GET_flows-flowId-segments) API endpoint.", "type": "boolean" + }, + "tags": { + "$ref": "tags.json" } } } From 2dc8b5901f016c50db05a893deb9de22237239a8 Mon Sep 17 00:00:00 2001 From: Sam Mesterton-Gibbons Date: Thu, 9 Oct 2025 10:39:47 +0100 Subject: [PATCH 5/6] api: Use more precise wording about URL encoding Co-authored-by: James Sandford --- api/TimeAddressableMediaStore.yaml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/api/TimeAddressableMediaStore.yaml b/api/TimeAddressableMediaStore.yaml index 0d7da360..c5284280 100644 --- a/api/TimeAddressableMediaStore.yaml +++ b/api/TimeAddressableMediaStore.yaml @@ -158,7 +158,7 @@ paths: in: query description: | Filter on webhooks that have a tag named {name} with a value in the given comma-seperated list of values. - The {name} could contain escaped characters to allow it to be used in a URL. + The {name} and the value MUST be URL encoded where special characters are present. Where the tag's value is a string, at least one of the given values will match. Where the tag's value is an array, at least one value in the array will match at least one of the given values. Partial string matches of the values are not valid. @@ -168,7 +168,7 @@ paths: in: query description: | Filter on webhooks that have a tag named {name} regardless of value. - The {name} could contain escaped characters to allow it to be used in a URL. + The {name} MUST be URL encoded where special characters are present. If set to true then the presence of the tag is filtered for. If set to false then its absence is. If left out then no filtering on tag presence is performed. @@ -212,7 +212,7 @@ paths: in: query description: | Filter on webhooks that have a tag named {name} with a value in the given comma-seperated list of values. - The {name} could contain escaped characters to allow it to be used in a URL. + The {name} and the value MUST be URL encoded where special characters are present. Where the tag's value is a string, at least one of the given values will match. Where the tag's value is an array, at least one value in the array will match at least one of the given values. Partial string matches of the values are not valid. @@ -222,7 +222,7 @@ paths: in: query description: | Filter on webhooks that have a tag named {name} regardless of value. - The {name} could contain escaped characters to allow it to be used in a URL. + The {name} MUST be URL encoded where special characters are present. If set to true then the presence of the tag is filtered for. If set to false then its absence is. If left out then no filtering on tag presence is performed. From a0b75633fe61ca45e622669477bb45d3f5e3315a Mon Sep 17 00:00:00 2001 From: Sam Mesterton-Gibbons Date: Thu, 9 Oct 2025 10:43:20 +0100 Subject: [PATCH 6/6] adr: Mark ADR0040 as accepted --- docs/adr/0040-tag-usability-enhancements.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0040-tag-usability-enhancements.md b/docs/adr/0040-tag-usability-enhancements.md index de7c35d3..94a686dc 100644 --- a/docs/adr/0040-tag-usability-enhancements.md +++ b/docs/adr/0040-tag-usability-enhancements.md @@ -1,5 +1,5 @@ --- -status: "proposed" +status: "accepted" --- # Tag Usability Enhancements