Improve usability of tags

api/TimeAddressableMediaStore.yaml

@@ -154,6 +154,26 @@ paths:
       tags:
         - Webhooks
       parameters:
+        - name: tag.{name}
+          in: query
+          description: |
+            Filter on webhooks that have a tag named {name} with a value in the given comma-seperated list of values.
+            The {name} and the value MUST be URL encoded where special characters are present.
+            Where the tag's value is a string, at least one of the given values will match.
+            Where the tag's value is an array, at least one value in the array will match at least one of the given values.
+            Partial string matches of the values are not valid.
+          schema:
+            $ref: 'schemas/url-tag-list.json'
+        - name: tag_exists.{name}
+          in: query
+          description: |
+            Filter on webhooks that have a tag named {name} regardless of value.
+            The {name} MUST be URL encoded where special characters are present.
+            If set to true then the presence of the tag is filtered for.
+            If set to false then its absence is.
+            If left out then no filtering on tag presence is performed.
+          schema:
+            type: boolean
         - $ref: '#/components/parameters/trait_resource_paged_key'
         - $ref: '#/components/parameters/trait_paged_limit'
       responses:
@@ -188,6 +208,26 @@ paths:
       tags:
         - Webhooks
       parameters:
+        - name: tag.{name}
+          in: query
+          description: |
+            Filter on webhooks that have a tag named {name} with a value in the given comma-seperated list of values.
+            The {name} and the value MUST be URL encoded where special characters are present.
+            Where the tag's value is a string, at least one of the given values will match.
+            Where the tag's value is an array, at least one value in the array will match at least one of the given values.
+            Partial string matches of the values are not valid.
+          schema:
+            $ref: 'schemas/url-tag-list.json'
+        - name: tag_exists.{name}
+          in: query
+          description: |
+            Filter on webhooks that have a tag named {name} regardless of value.
+            The {name} MUST be URL encoded where special characters are present.
+            If set to true then the presence of the tag is filtered for.
+            If set to false then its absence is.
+            If left out then no filtering on tag presence is performed.
+          schema:
+            type: boolean
         - $ref: '#/components/parameters/trait_resource_paged_key'
         - $ref: '#/components/parameters/trait_paged_limit'
       responses:
@@ -370,10 +410,13 @@ paths:
         - name: tag.{name}
           in: query
           description: |
-            Filter on Sources that have a tag named {name} and with the given value.
-            {name} and the value MUST be URL encoded where special characters are present.
+            Filter on Sources that have a tag named {name} with a value in the given comma-seperated list of values.
+            The {name} and the value MUST be URL encoded where special characters are present.
+            Where the tag's value is a string, at least one of the given values will match.
+            Where the tag's value is an array, at least one value in the array will match at least one of the given values.
+            Partial string matches of the values are not valid.
           schema:
-            type: string
+            $ref: 'schemas/url-tag-list.json'
         - name: tag_exists.{name}
           in: query
           description: |
@@ -428,10 +471,13 @@ paths:
         - name: tag.{name}
           in: query
           description: |
-            Filter on Sources that have a tag named {name} and with the given value.
-            {name} and the value MUST be URL encoded where special characters are present.
+            Filter on Sources that have a tag named {name} with a value in the given comma-seperated list of values.
+            The {name} and the value MUST be URL encoded where special characters are present.
+            Where the tag's value is a string, at least one of the given values will match.
+            Where the tag's value is an array, at least one value in the array will match at least one of the given values.
+            Partial string matches of the values are not valid.
           schema:
-            type: string
+            $ref: 'schemas/url-tag-list.json'
         - name: tag_exists.{name}
           in: query
           description: |
@@ -814,10 +860,13 @@ paths:
         - name: tag.{name}
           in: query
           description: |
-            Filter on Flows that have a tag named {name} and with the given value.
-            {name} and the value MUST be URL encoded where special characters are present.
+            Filter on flows that have a tag named {name} with a value in the given comma-seperated list of values.
+            The {name} and the value MUST be URL encoded where special characters are present.
+            Where the tag's value is a string, at least one of the given values will match.
+            Where the tag's value is an array, at least one value in the array will match at least one of the given values.
+            Partial string matches of the values are not valid.
           schema:
-            type: string
+            $ref: 'schemas/url-tag-list.json'
         - name: tag_exists.{name}
           in: query
           description: |
@@ -898,10 +947,13 @@ paths:
         - name: tag.{name}
           in: query
           description: |
-            Filter on Flows that have a tag named {name} and with the given value.
-            {name} and the value MUST be URL encoded where special characters are present.
+            Filter on flows that have a tag named {name} with a value in the given comma-seperated list of values.
+            The {name} and the value MUST be URL encoded where special characters are present.
+            Where the tag's value is a string, at least one of the given values will match.
+            Where the tag's value is an array, at least one value in the array will match at least one of the given values.
+            Partial string matches of the values are not valid.
           schema:
-            type: string
+            $ref: 'schemas/url-tag-list.json'
         - name: tag_exists.{name}
           in: query
           description: |
@@ -2152,6 +2204,18 @@ paths:
             Where multiple filter query parameters are provided, the returned `get_urls` will match all filters.
           schema:
             type: boolean
+        - name: flow_tag.{name}
+          in: query
+          description: |
+            Filter `referenced_by_flows` on tag values. This option is the same as the `tag.{name}` query parameter on the `/flows/` API endpoint.
+          schema:
+            type: string
+        - name: flow_tag_exists.{name}
+          in: query
+          description: |
+            Filter `referenced_by_flows` on tag names. This option is the same as the `tag_exists.{name}` query parameter on the `/flows/` API endpoint.
+          schema:
+            type: boolean
         - $ref: '#/components/parameters/trait_resource_paged_key'
         - $ref: '#/components/parameters/trait_paged_limit'
       responses:
@@ -2237,6 +2301,18 @@ paths:
             Where multiple filter query parameters are provided, the returned `get_urls` will match all filters.
           schema:
             type: boolean
+        - name: flow_tag.{name}
+          in: query
+          description: |
+            Filter `referenced_by_flows` on tag values. This option is the same as the `tag.{name}` query parameter on the `/flows/` API endpoint.
+          schema:
+            type: string
+        - name: flow_tag_exists.{name}
+          in: query
+          description: |
+            Filter `referenced_by_flows` on tag names. This option is the same as the `tag_exists.{name}` query parameter on the `/flows/` API endpoint.
+          schema:
+            type: boolean
         - $ref: '#/components/parameters/trait_resource_paged_key'
         - $ref: '#/components/parameters/trait_paged_limit'
       responses:

api/schemas/tags.json

@@ -1,8 +1,18 @@
 {
     "title": "Tags",
-    "description": "Key value is a freeform string.",
+    "description": "Key is a freeform string. Value is a freeform string, or an array of freeform strings.",
     "type": "object",
     "additionalProperties": {
-            "type": "string"
+        "anyOf": [
+            {
+                "type": "string"
+            },
+            {
+                "type": "array",
+                "items": {
+                    "type": "string"
+                }
+            }
+        ]
     }
 }

api/schemas/url-tag-list.json

@@ -0,0 +1,6 @@
+{
+    "title": "Query String Tag value list",
+    "description": "A list of tag values, formatted for use in query string parameters",
+    "type": "string",
+    "pattern": "^([^,]+(,[^,]+)*)?$"
+}

api/schemas/webhook.json

@@ -81,6 +81,9 @@
         "verbose_storage": {
             "description": "Whether to include storage metadata in the `get_urls` property in `flows/segments_added` events. This option is the same as the `verbose_storage` query parameter for the [/flows/{flowId}/segments](#/operations/GET_flows-flowId-segments) API endpoint.",
             "type": "boolean"
+        },
+        "tags": {
+          "$ref": "tags.json"
         }
     }
 }

docs/README.md

@@ -71,5 +71,6 @@ For more information on how we use ADRs, see [here](./adr/README.md).
 | [0037](./adr/0037-improve-webhooks.md)                             | Proposal for improvements to the Webhooks endpoints                        |
 | [0038](./adr/0038-improved-storage-management.md)                  | Improved Storage Management                                                |
 | [0039](./adr/0039-remove-pre-actions.md)                           | Proposal to remove pre-actions from storage allocation response            |
+| [0040](./adr/0040-tag-usability-enhancements.md)                   | Tag Usability Enhancements |
 
 \* Note: ADR 0004a was the unintended result of a number clash in the early development of TAMS which wasn't caught before publication

docs/adr/0040-tag-usability-enhancements.md

@@ -0,0 +1,81 @@
+---
+status: "accepted"
+---
+# Tag Usability Enhancements
+
+## Context and Problem Statement
+
+TAMS provides tags on Flows and Sources as an ad-hoc key-value store of data.
+This enables some simple workflows without the need for an additional database of content, simply by using the TAMS API.
+It also enables simpler interoperability, in cases where the tag store is enough to locate the content required in TAMS using e.g. a panel in a tool, rather than an integration with some other MAM.
+Over the course of other work on TAMS (notably the exploration of fine-grained authorisation) it has become clear that some improvements to tags would be useful.
+
+## Considered Options
+
+* Option 1a: Add support for lists in tags
+* Option 1b: Add support for lists in tags, require every item match
+* Option 1c: Add support for lists in tags, add a query for all items matching
+* Option 2: Make it possible to filter object instances on tags
+* Option 3: Add tags to webhooks
+
+## Decision Outcome
+
+Chosen options:
+
+* Option 1a: Add support for lists in tags
+* Option 2: Make it possible to filter object instances on tags
+* Option 3: Add tags to webhooks
+
+Option 1a is chosen at present because there is no clear need for Option 1c, but it may be added in future without a breaking change to client implementations.
+
+Options 2 and 3 make the support and behaviour of tags uniform across Flows, Sources, Object Instances and Webhooks.
+
+### Implementation
+
+Implemented by <https://github.com/bbc/tams/pull/153>
+
+## Pros and Cons of the Options
+
+### Option 1a: Add support for lists in tags
+
+Add the option that the value of a tag either be a string, or a list.
+Where tags are queryable, make the `tag.{name}` query parameter accept a comma separated list of strings, and require that at least one item in the query matches at least one of the values in the list to return the resource.
+
+* Good, because it makes tags usable for cases when multiple options are present (e.g. authorisation attributes)
+* Good, because it improves discoverability of well-tagged resources
+* Good, because it allows for finding e.g. a Flow with any of a specific property in a list ("OR" queries)
+* Bad, because it makes implementations more complex to handle checking lists in queries
+* Bad, because it encourages more usage of TAMS as a MAM
+
+### Option 1b: Add support for lists in tags, require every item match
+
+As in Option 1a, however the `tag.{name}` query parameter requires that every item in the given list matches.
+
+* Good, because it enables queries finding e.g. a Flow with a number of properties ("AND" queries)
+* Bad, because it requires knowing the precise list to query it using tags
+
+### Option 1c: Add support for lists in tags, add a query for all items matching
+
+As in Option 1a, with the addition of a `tag_every.{name}` query parameter where every item in the given list matches.
+
+* Good, because it enables both "AND" and "OR" queries
+* Good, because it makes the tag query mechanism more complete
+* Bad, because it adds another query parameter to implement
+* Bad, because at the time of writing there is no concrete use case for "AND" queries
+* Bad, because it encourages more usage of TAMS as a MAM
+
+### Option 2: Make it possible to filter object instances on tags
+
+Add `flow_tag.{name}` and `flow_tag.{exists}` to the `GET /objects/{objectId}` endpoint, to adjust the contents of the `referenced_by_flows` list down to a specific subset of Flows.
+
+* Good, because it avoids excessive pagination if an object is used by a very large number of Flows
+* Good, because it enables the option to use tags to restrict access to Flows and associated objects
+* Neutral, because it requires a change to specification and implementation
+* Bad, because it encourages more usage of TAMS as a MAM
+
+### Option 3: Add tags to webhooks
+
+Add tags and tag-based querying to webhooks, in the same was as Flows and Sources.
+
+* Good, because it improves the ability of clients to manage large numbers of webhooks by locating them using tags
+* Good, because it enables the option to use tags to restrict access to webhooks