Describe naming convention for role in collections

api/examples/flow-collection-get-200.json

@@ -1,11 +1,11 @@
 [
     {
         "id": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34",
-        "role": "video"
+        "role": "programme"
     },
     {
         "id": "6101df05-06bb-41b8-8af4-cf7cd33df209",
-        "role": "audio"
+        "role": "programme"
     },
     {
         "id": "e85efab4-993b-4ad6-9af3-4cd8d0d38860",

api/examples/flow-collection-put.json

@@ -1,11 +1,11 @@
 [
     {
         "id": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34",
-        "role": "video"
+        "role": "programme"
     },
     {
         "id": "6101df05-06bb-41b8-8af4-cf7cd33df209",
-        "role": "audio"
+        "role": "programme"
     },
     {
         "id": "e85efab4-993b-4ad6-9af3-4cd8d0d38860",

api/examples/flow-get-200-multi-container-map.json

@@ -11,11 +11,16 @@
     "flow_collection": [
         {
             "id": "32b75860-83e5-48c9-8d6b-de46bdbc7f80",
-            "role": "video"
+            "role": "programme",
+            "container_mapping": {
+                "mp2ts_container": {
+                    "pid": 256
+                }
+            }
         },
         {
             "id": "94996f2e-0cb5-43d3-ab6c-db5a9cf667aa",
-            "role": "L",
+            "role": "programme_L",
             "container_mapping": {
                 "track_index": 1,
                 "format_track_index": 0,
@@ -26,7 +31,7 @@
         },
         {
             "id": "9ba0523e-22a0-4c69-a598-baf4be244a79",
-            "role": "R",
+            "role": "programme_R",
             "container_mapping": {
                 "track_index": 2,
                 "format_track_index": 1,

api/examples/flow-get-200-multi.json

@@ -10,11 +10,15 @@
     "flow_collection": [
         {
             "id": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34",
-            "role": "video"
+            "role": "programme"
         },
         {
             "id": "6101df05-06bb-41b8-8af4-cf7cd33df209",
-            "role": "audio"
+            "role": "programme"
+        },
+        {
+            "id": "c8943cb3-08df-46de-8ce8-0d7d70ed204c",
+            "role": "audio_description"
         },
         {
             "id": "e85efab4-993b-4ad6-9af3-4cd8d0d38860",

api/examples/flow-put-multi.json

@@ -10,11 +10,15 @@
     "flow_collection": [
         {
             "id": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34",
-            "role": "video"
+            "role": "programme"
         },
         {
             "id": "6101df05-06bb-41b8-8af4-cf7cd33df209",
-            "role": "audio"
+            "role": "programme"
+        },
+        {
+            "id": "c8943cb3-08df-46de-8ce8-0d7d70ed204c",
+            "role": "audio_description"
         },
         {
             "id": "e85efab4-993b-4ad6-9af3-4cd8d0d38860",

api/schemas/collection-item.json

@@ -12,7 +12,7 @@
             "$ref": "uuid.json"
         },
         "role": {
-            "description": "A human-readable role of the element in this collection (e.g. 'R' to denote a right audio channel in a collection of mono audio Sources)",
+            "description": "A string describing the purpose of this element in the collection, primarily intended to be human-readable (e.g. 'audio_description' to denote the audio description track in a multi of audio and video Sources). See the [Role Names](https://github.com/bbc/tams/blob/main/docs/appnotes/0020-role-names.md) application note for suggested names.",
             "type": "string"
         }
     }

docs/README.md

@@ -26,6 +26,7 @@ For more information on how we use application notes, see [here](./appnotes/READ
 | [0017](./appnotes/0017-reuse-of-ids.md)                              | When to re-use IDs in TAMS and compatible systems               |
 | [0018](./appnotes/0018-managing-multiple-object-instances.md)        | Managing Multiple Object Instances                              |
 | [0019](./appnotes/0019-implementing-retention-management.md)         | Methods of implementing retention management                    |
+| [0020](./appnotes/0020-role-naming-convention.md)                    | Role Naming Convention                                          |
 
 ## ADRs
 
@@ -81,5 +82,6 @@ For more information on how we use ADRs, see [here](./adr/README.md).
 | [0042](./adr/0042-uncontrolled-object-instance-labels.md)          | Make `label` Mandatory for Uncontrolled Object Instances |
 | [0043](./adr/0043-signalling-retention-time.md)                    | Signalling retention time |
 | [0046](./adr/0046-governance.md)                                   | Governance |
+| [0047](./adr/0047-role-naming-convention.md)                       | Role Naming Convention |
 
 \* Note: ADR 0004a was the unintended result of a number clash in the early development of TAMS which wasn't caught before publication

docs/adr/0047-role-naming-convention.md

@@ -0,0 +1,176 @@
+---
+status: proposed
+---
+# Role Naming Convention
+
+## Context and Problem Statement
+
+Each item in the collection of a multi-essence Source or Flow has a `role`, intended as a description of that item's purpose in the collection.
+Roles are intended to be human-readable, however it is useful for clients and systems to make sense of them too.
+It would be useful to have some recommendations about how to use roles.
+
+One part of that recommendation should be the editorial purpose of an item: for example programme audio, audio description or commentary.
+Or main video, signed video or clean-feed without graphics.
+Currently this is represented in free text.
+
+## Decision Drivers
+
+* Roles must be set when creating a multi-essence collection.
+* When viewing a multi-essence Source or Flow, only the list of IDs collected and roles are available: getting formats and tags requires another query.
+* Items in a collection serve a variety of different purposes, and it is useful to have some commonality.
+* At time of writing `role` is typically `video` or `audio`, but practical applications need to represent more complex packages.
+
+## Considered Options
+
+For how to use the `role` field (options `Rn`):
+
+* Option R1: Make `role` a semi-structured field containing editorial purpose, media type and other details
+* Option R2: Replace `role` with a number of additional controlled fields
+* Option R3a: Use `role` as editorial purpose, use other queries for Flow/Source properties
+* Option R3b: Use `role` as editorial purpose, use other queries for Flow/Source properties, surface `role` as a query param
+* Option R3c: Use `role` as editorial purpose, additionally surface `format` in collection
+
+For how to represent editorial purpose (options `Pn`):
+
+* Option P1: Represent editorial purpose using DVB component descriptors
+* Option P2: Represent editorial purpose using roles from MPEG-DASH
+* Option P3: Represent editorial purpose based on descriptions in the MovieLabs Ontology for Media Creation
+* Option P4: Represent purpose of content using a list, as with tags
+
+## Decision Outcome
+
+Chosen option: Option R3a and Option P4, because it lines up with the existing common use of `role` and allows it to evolve over time.
+It also limits the changes required to the API, and while in some cases it may trigger extra API requests (e.g. to enumerate the media types of a collection), it is likely the details of those collection items will be needed anyway, so the requests would be needed.
+Furthermore the TAMS API is intended not to serve as a Media Asset Manager and provide minimal library management and discovery features: through that lens the additional request burden seems reasonable and if it becomes too onerous in a particular deployment, that may indicate a MAM is required.
+However Option R3c (surfacing `role` in collections) could be added later if necessary.
+
+Regarding editorial purposes, while Option P4 makes it harder for clients to automatically determine the purpose of each item in a collection, in many cases the role merely serves to change what the user is shown and they are left to make the decisions.
+In cases where it matters which elements of a collection are picked up (for example packaging content for distribution) the correct items should be identified by some more rigorous method anyway (for example creating a multi-essence Flow of just the items required).
+
+### Implementation
+
+Implemented by <https://github.com/bbc/tams/pull/173>
+
+## Pros and Cons of the Options - for the `role` field
+
+### Option R1: Make `role` a semi-structured field containing editorial purpose, media type and other details
+
+Write an Application Note suggesting a naming convention for the `role` field, with a structured form that captures media type, editorial purpose and additional details.
+This would be similar to the approach taken in the (now deprecated) [storage label AppNote](../appnotes/0009-storage-label-format.md).
+Clients could either read the collection items and identify both type and purpose, or query the members of the collection and consider their labels/tags directly instead.
+
+* Good, because it avoids a breaking change to the specification, instead only creating guidance.
+* Good, because it makes `role` open-ended, allowing space for future change and expansion.
+* Good, because it saves making additional API requests to get media type and other details of collection items.
+* Bad, because it does not constrain the use of `role`, and clients may have to rely on human intervention.
+* Bad, because it forces clients to handle and parse an un-enforced free-text field, which may be malformed as a result.
+
+### Option R2: Replace `role` with a number of additional controlled fields
+
+Change the specification to remove `role` and replace it with a more precise set of fields conveying the purpose of each element in the collection.
+
+* Good, because it constrains and precisely describes what each item in a collection is for.
+* Bad, because it requires a breaking change to the API, to provide a capability that can be achieved another way.
+* Bad, because it requires work to fully specify all the possible purposes an item in a collection can fulfil.
+* Bad, because it then constrains TAMS and requires a change as new purposes are identified.
+
+### Option R3a: Use `role` solely for editorial purpose, and rely on other fields for other purposes
+
+Use `role` primarily as a label to represent the editorial purpose of an item in a collection (broadly aligned with common usage).
+Where a client wants to identify the type of a collection item, they can request the full details from the API.
+For example if a collection contains multiple items of role "programme" the client could request all of those items by ID to find out one was video and one audio (or, subject to a draft ADR being accepted, make a direct query of `GET /flows?collected_by_id=...` and cross-reference the results with the collection listing).
+
+* Good, because it aligns with current common usage of `role`.
+* Good, because it avoids a breaking change to the specification.
+* Good, becuause it does not introduce a semi-structured field and force clients to handle malformed data.
+* Bad, because it forces clients to make additional requests (and possibly a somewhat complex cross-reference) to locate all the data they need.
+  In particular a UI element displaying all the Sources in a store, the editorial purposes they contain and their media types would require an additional listing request per top-level Source.
+
+### Option R3b: Use `role` as editorial purpose, use other queries for Flow/Source properties, surface `role` as a query param
+
+As Option R3a, however `role` is added as an extra query parameter to the Flow and Source listing endpoints.
+As a result, a client could request all the items of a specific role which are members of a particular collection (`GET /flows?collected_by_id=...&role=...`).
+
+* Good, because it aligns with current common usage of `role`.
+* Good, because it avoids a breaking change to the specification.
+* Good, becuause it does not introduce a semi-structured field and force clients to handle malformed data.
+* Bad, because it forces clients to make additional requests to locate all the data they need.
+  In particular a UI element displaying all the Sources in a store, the editorial purposes they contain and their media types would require an additional listing request per top-level Source.
+* Bad, because an item's role in a collection is a property of the collection, not the item, so the implementation is likely to be quite complex!
+
+### Option R3c: Use `role` as editorial purpose, additionally surface `format` in collection
+
+As Option R3a, however the `collects` property of a multi-essence Flow or Source gains an additonal property, `format` which is the `format` property of the collected item.
+For example:
+
+```json
+[
+    {
+        "id": "f59a1785-b5bd-4829-afe0-7f65f9b335dd",
+        "role": "programme",
+        "format": "urn:x-nmos:format:video"
+    },
+    {
+        "id": "7162d669-3230-4254-99d1-2c06f815d025",
+        "role": "programme",
+        "format": "urn:x-nmos:format:audio"
+    },
+    {
+        "id": "2e285f91-6dff-4117-b26b-1ae749c5f5aa",
+        "role": "audio_description",
+        "format": "urn:x-nmos:format:audio"
+    }
+]
+```
+
+A client can directly identify the members of the collection directly.
+
+* Good, because it aligns with current common usage of `role`.
+* Good, becuause it does not introduce a semi-structured field and force clients to handle malformed data.
+* Good, because it allows clients to get more of the salient information about a collection in a single request.
+  It would allow the UI element suggested above to be built from a single listing request, for example.
+* Neutral, because it changes the specification, albeit in a non-breaking way.
+
+## Pros and Cons of the Options - for representing editorial purpose
+
+### Option P1: Represent editorial purpose using DVB component descriptors
+
+DVB uses the `stream_content`, `stream_content_ext` and `component_type` fields in the `component_descriptor` to describe the type of a Service.
+These are described in [ETSI EN 300 468 pp 60-70](https://www.etsi.org/deliver/etsi_en/300400_300499/300468/01.19.01_60/en_300468v011901p.pdf).
+TAMS could use the same descriptors, or their names.
+
+* Good, because it is part of an established standard
+* Bad, because the component types are represented as hex bytes (for carriage in the SDT table) and would need human-readable names
+* Bad, because many of the types also include technical characteristics of content, making them unsuitable for use in Sources
+* Bad, because the list is focused on distribution, so cannot contain aspects such as clean-feed video
+
+### Option P2: Represent editorial purpose using MPEG-DASH roles
+
+MPEG-DASH contains a role attribute for an `AdaptationSet`, which describes the purose of that particular track.
+A number of values for that attribute are given in the specification (see ISO/IEC 23009-1:2022 section 5.8.5.5), covering the `main` content along with others such as `alternate`, `supplementary`, `commentary`, `description`, etc.
+TAMS could use these descriptors for the table directly.
+
+* Good, because it is part of an established standard
+* Neutral, because the example use cases identified above could be represented, however some would be ambiguous, such as using `alternate` for clean-feed video
+* Bad, because the list cannot be expanded beyond what is in the MPEG-DASH specification
+
+### Option P3: Represent editorial purpose based on descriptions in the MovieLabs Ontology for Media Creation
+
+The MovieLabs [Ontology for Media Creation](https://mc.movielabs.com/docs/ontology/) contains some definitions of the purpose of pieces of content.
+Unfortunately it does not appear to represent differing purposes of video content, and for audio it refers to definitions in SMPTE ST 377-41.
+It was not possible to acquire a copy of this SMPTE document at time of writing, although a version of those definitions are available in Appendix B of the [PDF version on the document](https://mc.movielabs.com/omc/Asset/Audio/ML_Ontology_Pt3C_Audio_v2.8.pdf).
+
+* Good, because it is part of a controlled specification.
+* Bad, because it relies on a document not available to the TAMS community.
+* Bad, because video is not covered in the document.
+
+### Option P4: Represent purpose of content using a list, as with tags
+
+When describing the editorial purpose of an item in a collection, draw from a list of suggested types in an Application Note.
+Allow new types to be added as they come up, in a similar process to tags.
+Seed the initial list based on a combination of other specifications (including those above).
+
+* Good, because it captures suggested names without constraining the purposes that a Flow or Source can be used for
+* Good, because inspiration can be drawn from the other documents suggested, without being constrained by tem
+* Good, because it is easy to add new items to the list
+* Bad, because the list is open-ended and uncontrolled, which may lead to content using a mix of names

docs/appnotes/0001-multi-mono-essence-flows-sources.md

@@ -62,7 +62,7 @@ In this case, Segment timestamps will be remapped so the relationship between th
 ### Ingest of SRT Stream (video + stereo audio)
 
 Stepping up to a stream containing both audio and video, packaged into a MPEG2 Transport Stream and encapsulated in SRT, three `Flow` entities are created on ingest: one for the video essence, one for the audio essence and a multi-essence `Flow` to record the association of the mono-essence `Flows` as an ingested stream.
-The multi-essence `Flow` features a `collection` attribute that lists the `Flow IDs` in the set, each annotated with a string describing its role.
+The multi-essence `Flow` features a `collection` attribute that lists the `Flow IDs` in the set, each annotated with a string describing its role (see [AppNote 0020](./0020-role-names.md) for suggested role naming).
 
 Technical metadata relating to the elementary streams is used to populate the corresponding mono-essence `Flow` properties.
 

docs/appnotes/0020-role-naming-convention.md

@@ -0,0 +1,33 @@
+# 0020: Role Naming Convention
+
+## Abstract
+
+Each item in the collection of a multi-essence Source or Flow has a `role`, intended as a description of that item's purpose in the collection.
+Roles are intended to be human-readable and uncontrolled, however a list of common editorial purposes and associated names is useful.
+
+This functions much like [Tag Names](./0003-tag-names.md) and this list will likely grow over time.
+
+## Editorial Purpose Names
+
+## Video
+
+| Name | Description |
+| ---- | ----------- |
+| `programme` | Primary, or default video for a piece of content, e.g. that will be edited or distributed on to the audience. |
+| `signed` | Primary video with a signer in-vision. Consider using a label on the collected Source/Flow with the [language code](https://github.com/bbc/tams/blob/main/docs/appnotes/0003-tag-names.md#language_code) of the signer. |
+| `cleanfeed` | Version of the video without graphics, for reversioning and re-use. |
+| `video` | In a simple A/V mux that only contains the audio and video for an asset, the `role` provides little additional information, and calling it "video" may be sufficient. |
+
+## Audio
+
+_Note that in general, audio Flows and Sources should also use a [language code](https://github.com/bbc/tams/blob/main/docs/appnotes/0003-tag-names.md#language_code) label to allow clients to identify the language used._
+
+| Name | Description |
+| ---- | ----------- |
+| `programme` | Primary, or default audio for a piece of content, e.g. that will be edited or distributed on to the audience. |
+| `audio_description` | Audio description track. |
+| `commentary` | Commentary track, where sent separately (e.g. from a sports event).  |
+| `music&effects` | Music and Effects track, where sent separately. |
+| `audio` | In a simple A/V mux that only contains the audio and video for an asset, the `role` provides little additional information, and calling it "audio" may be sufficient. |
+
+In some cases it will also be appropriate to suffix channel labels for audio: for example if a stereo mix is set as two separate Flows, it may be referred to as `programme_L` and `programme_R`.