Add Object Instance management endpoints

api/TimeAddressableMediaStore.yaml

@@ -1286,7 +1286,7 @@ paths:
           $ref: '#/components/responses/trait_resource_info_head_404'
     get:
       summary: Flow Read-Only
-      description: Returns the Flow read_only property. If set to 'true', service implementations SHOULD reject client requests to update Flow metadata (other than the read_only property), Flow Segments and Media Objects.
+      description: Returns the Flow read_only property. If set to 'true', service implementations SHOULD reject client requests to update Flow metadata (other than the read_only property), and Flow Segments. Service implementations should also reject requests to the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage) endpoint for the Flow, and requests to delete the Flow.
       operationId: GET_flows-flowId-read-only
       tags:
         - Flows
@@ -1302,7 +1302,7 @@ paths:
           description: The requested Flow does not exist.
     put:
       summary: Set Flow Read-Only
-      description: Set the read-only property. If set to 'true', service implementations SHOULD reject client requests to update Flow metadata (other than the read_only property), Flow Segments and Media Objects.
+      description: Set the read-only property. If set to 'true', service implementations SHOULD reject client requests to update Flow metadata (other than the read_only property), and Flow Segments. Service implementations should also reject requests to the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage) endpoint for the Flow, and requests to delete the Flow.
       operationId: PUT_flows-flowId-read-only
       tags:
         - Flows
@@ -1826,6 +1826,10 @@ paths:
         A service instance MAY support Media Objects that are held in external storage in another TAMS or other media storage system.
         The Flow Segment may in that case require the `get_urls` property to provide the information needed by clients to access the Media Object.
 
+        Clients MAY modify Flow Segments, but this should only be done in exceptional circumstances to correct metadata as such operations will likely break the idempotency of Segments.
+        Properties of Media Objects, such as `get_urls`, SHOULD be modified via the [`/objects`](#/operations/GET_objects) endpoints.
+        They SHOULD NOT be modified at this endpoint, and TAMS instences SHOULD reject such requests with a `400` error.
+
         If a client needs to modify a Flow Segment, e.g. to correct metadata such as the `key_frame_count` or add additional URLs to `get_urls`, then the client SHOULD first delete the existing Segment and then write a new one.
         The behaviour is undefined if the Segment exists and the service may return a 400 error response.
 
@@ -1922,9 +1926,9 @@ paths:
           $ref: 'schemas/uuid.json'
         description: The Flow identifier.
     post:
-      summary: Allocate Flow Storage
+      summary: Allocate Initial Flow Storage
       description: |
-        Allocate storage locations for writing Media Objects.
+        Allocate initial storage locations for writing Media Objects.
 
         The Storage Backend type, which is indicated in the [/service](#/operations/GET_service) resource, determines the information provided in the response.
         The examples and description below are for the "http_object_store" Storage Backend type.
@@ -1934,10 +1938,6 @@ paths:
         The client is expected to register the Flow Segment using the [/flows/{flowId}/segments](#/operations/POST_flows-flowId-segments) endpoint once the upload is complete.
         Service implementations need to handle situations where Objects were uploaded but no Flow Segment was registered successfully.
 
-        The response may include PUT URLs for creating buckets for the Media Objects.
-        These PUT URLs should be used before uploading Media Objects.
-        The object_id associated with each storage location has the bucket name as its prefix.
-
         When making requests to the provided `put_url`, clients should include credentials if the provided URL is on the same origin as the API itself, akin to the `same-origin` mode in the [WhatWG Fetch Standard](https://fetch.spec.whatwg.org/#concept-request-credentials-mode).
       operationId: POST_flows-flowId-storage
       tags:
@@ -1981,10 +1981,52 @@ paths:
       parameters:
         - name: objectId
           in: path
-          description: The Media Object identifier.
+          description: The Media Object identifier. The Object ID may include special characters such as `/` which should be URL encoded.
           required: true
           schema:
             type: string
+        - name: verbose_storage
+          in: query
+          description: |
+            Include storage metadata in `get_urls`.
+            When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`.
+          schema:
+            default: false
+            type: boolean
+        - name: accept_get_urls
+          in: query
+          description: |
+            A comma separated list of labels of media object `get_urls` to include in the response.
+            Omitting `accept_get_urls` will result in no filtering of `get_urls`.
+            An empty `accept_get_urls` results in an empty or no `get_urls` in the response.
+            Media object `get_urls` with no label or storage ID cannot be filtered for; they will only be returned if `accept_get_urls` is omitted, and `accept_storage_ids` is omitted or empty.
+            Without `get_urls`, the response from the service could be substantially faster if it is not required to
+            generate a large number of pre-signed URLs for example.
+            Where multiple filter query parameters are provided, the returned `get_urls` will match all filters.
+          schema:
+            $ref: 'schemas/url-label-list.json'
+        - name: accept_storage_ids
+          in: query
+          description: |
+            A comma separated list of `storage_id`s of media object `get_urls` to include in the response.
+            Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`.
+            Media object `get_urls` with no label or storage ID cannot be filtered for; they will only be returned if `accept_get_urls` is omitted, and `accept_storage_ids` is omitted or empty.
+            A full list of available `storage_id`s may be found at the `service/storage-backends` endpoint.
+            Where multiple filter query parameters are provided, the returned `get_urls` will match all filters.
+          schema:
+            type: string
+            pattern: ^([0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})(,[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})*$
+        - name: presigned
+          in: query
+          description: |
+            If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls`.
+            If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`.
+            If omitted, both presigned and non-presigned URLs will be returned.
+            If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to
+            generate a large number of pre-signed URLs.
+            Where multiple filter query parameters are provided, the returned `get_urls` will match all filters.
+          schema:
+            type: boolean
         - $ref: '#/components/parameters/trait_resource_paged_key'
         - $ref: '#/components/parameters/trait_paged_limit'
       responses:
@@ -2024,10 +2066,52 @@ paths:
       parameters:
         - name: objectId
           in: path
-          description: The Media Object identifier.
+          description: The Media Object identifier. The Object ID may include special characters such as `/` which should be URL encoded.
           required: true
           schema:
             type: string
+        - name: verbose_storage
+          in: query
+          description: |
+            Include storage metadata in `get_urls`.
+            When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`.
+          schema:
+            default: false
+            type: boolean
+        - name: accept_get_urls
+          in: query
+          description: |
+            A comma separated list of labels of media object `get_urls` to include in the response.
+            Omitting `accept_get_urls` will result in no filtering of `get_urls`.
+            An empty `accept_get_urls` results in an empty or no `get_urls` in the response.
+            Media object `get_urls` with no label or storage ID cannot be filtered for; they will only be returned if `accept_get_urls` is omitted, and `accept_storage_ids` is omitted or empty.
+            Without `get_urls`, the response from the service could be substantially faster if it is not required to
+            generate a large number of pre-signed URLs for example.
+            Where multiple filter query parameters are provided, the returned `get_urls` will match all filters.
+          schema:
+            $ref: 'schemas/url-label-list.json'
+        - name: accept_storage_ids
+          in: query
+          description: |
+            A comma separated list of `storage_id`s of media object `get_urls` to include in the response.
+            Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`.
+            Media object `get_urls` with no label or storage ID cannot be filtered for; they will only be returned if `accept_get_urls` is omitted, and `accept_storage_ids` is omitted or empty.
+            A full list of available `storage_id`s may be found at the `service/storage-backends` endpoint.
+            Where multiple filter query parameters are provided, the returned `get_urls` will match all filters.
+          schema:
+            type: string
+            pattern: ^([0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})(,[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})*$
+        - name: presigned
+          in: query
+          description: |
+            If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls`.
+            If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`.
+            If omitted, both presigned and non-presigned URLs will be returned.
+            If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to
+            generate a large number of pre-signed URLs.
+            Where multiple filter query parameters are provided, the returned `get_urls` will match all filters.
+          schema:
+            type: boolean
         - $ref: '#/components/parameters/trait_resource_paged_key'
         - $ref: '#/components/parameters/trait_paged_limit'
       responses:
@@ -2055,7 +2139,98 @@ paths:
         "400":
           description: Bad request. Invalid query options.
         "404":
-          description: The requested Media Object does not exist.
+          description: The requested media object does not exist.
+  /objects/{objectId}/instances:
+    post:
+      summary: Register a Media Object instance
+      description: |
+        Request the service to create an Object instance on a new Storage Backend. Or add a new uncontrolled URL to `get_urls`.
+
+        To request the duplication of the Object to a new Storage Backend, clients POST a `storage_id` to this endpoint that does not currently have an instance of the Object. The API will then:
+
+        - Allocate storage for Media Object `objectId` on Storage Backend `storage_id`
+        - Copy the Media Object from an existing location to the newly allocated storage
+        - Start advertising the new copy in `get_urls` once ready
+
+        The API instances SHOULD be capable of handling the case where the only existant instances are uncontrolled.
+
+        Where a client has written a new uncontrolled Object instance, the client is responsible for ensuring that the Object written is complete and correct before registering it with this method.
+
+        All instances of an Object MUST be identical.
+      operationId: POST_objects-instances
+      tags:
+        - Objects
+      parameters:
+        - name: objectId
+          in: path
+          description: The Media Object identifier. The Object ID may include special characters such as `/` which should be URL encoded.
+          required: true
+          schema:
+            type: string
+      requestBody:
+        content:
+          application/json:
+            examples:
+              controlled:
+                summary: Registering a controlled instance
+                value:
+                  $ref: examples/objects-instances-controlled-post.json
+              uncontrolled:
+                summary: Registering a uncontrolled instance
+                value:
+                  $ref: examples/objects-instances-uncontrolled-post.json
+            schema:
+              $ref: schemas/objects-instances-post.json
+        required: true
+      responses:
+        "201":
+          description: Object instance successfully registered.
+        "400":
+          description: Bad request. Invalid request JSON.
+        "403":
+          description: Forbidden. You do not have permission to modify this Media Object.
+        "404":
+          description: The Media Object does not exist.
+    delete:
+      summary: Delete a Media Object instance
+      description: |
+        Delete an instance of a Media Object.
+
+        One of `storage_id` or `label` MUST be specified in the query parameters. `storage_id` SHOULD be used where `controlled` is `True` for the instance.
+
+        API instances should remove the Media Object instance from the `get_urls` list and then, if the instance is controlled, delete the Object instance from storage.
+
+        API instances SHOULD prevent clients from deleting all Object instances. Additionally, API instances MAY prevent clients from deleting all controlled Object instances. Where clients wish to remove all copies of an Object from the store, they should do so by deleting all Flows or Flow Segments which reference the Object.
+      operationId: DELETE_objects-instances
+      tags:
+        - Objects
+      parameters:
+        - name: objectId
+          in: path
+          description: The Media Object identifier. The Object ID may include special characters such as `/` which should be URL encoded.
+          required: true
+          schema:
+            type: string
+        - name: storage_id
+          in: query
+          description: The storage_id identifying the Media Object instance to be deleted.
+          schema:
+            type: string
+        - name: label
+          in: query
+          description: The label identifying the Media Object instance to be deleted.
+          schema:
+            type: string
+      responses:
+        "204":
+          description: No content. The Media Object instance has been deleted.
+        "400":
+          description: Bad request. Invalid query options.
+        "403":
+          description: Forbidden. You do not have permission to modify this Media Object.
+        "404":
+          description: The requested Object ID in the path is invalid.
+
   /flow-delete-requests:
     head:
       summary: List Flow Delete Requests

api/examples/objects-get-200.json

@@ -5,5 +5,10 @@
         "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34",
         "0fde9c11-da9d-434a-a113-d3b20a2cf251"
     ],
-    "first_referenced_by_flow": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34"
-}
+    "first_referenced_by_flow": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34",
+    "get_urls": [
+        {
+            "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/846023d3-612d-5014-bc47-88f6eb2d04bb"
+        }
+    ]
+}

api/examples/objects-instances-controlled-post.json

@@ -0,0 +1,3 @@
+{
+  "storage_id": "60af2ab4-e8a5-4c65-a09b-d35983680315"
+}

api/examples/objects-instances-uncontrolled-post.json

@@ -0,0 +1,4 @@
+{
+  "url": "https://tams-b.s3.eu-west-1.amazonaws.com/35e9b447-be10-43e0-ab85-e1e9fe15d354?X-Amz-Security-Token=signature...",
+  "label": "pipeline-b"
+}

api/schemas/flow-core.json

@@ -75,7 +75,7 @@
         },
         "read_only":
         {
-            "description": "If set to 'true', service implementations SHOULD reject client requests to update Flow metadata (other than the read_only property), Flow Segments and Media Objects",
+            "description": "If set to 'true', service implementations SHOULD reject client requests to update Flow metadata (other than the read_only property), and Flow Segments. Service implementations should also reject requests to the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage) endpoint for the Flow, and requests to delete the Flow.",
             "type": "boolean"
         },
         "codec":

api/schemas/flow-segment-post.json

@@ -41,7 +41,7 @@
         },
         "get_urls":
         {
-            "description": "A list of URLs to which a GET request can be made to directly retrieve the contents of the Segment. This is required by the `http_object_store` Storage Backend type, which is the only one currently described. Clients may choose any URL in the list and treat them as identical, however service instances may sort the list such that the preferred URL is first. `get_urls` should only be used to add uncontrolled URLs. URLs for the provided object_id controlled by the service instance will be populated automatically by the service instance.",
+            "description": "A list of URLs to which a GET request can be made to directly retrieve the contents of the Media Object. This is required by the `http_object_store` Storage Backend type, which is the only one currently described. Clients may choose any URL in the list and treat them as identical, however service instances may sort the list such that the preferred URL is first. `get_urls` should only be used to add uncontrolled URLs. URLs for the provided object_id controlled by the service instance will be populated automatically by the service instance.",
             "type": "array",
             "items":
             {
@@ -54,7 +54,7 @@
                 {
                     "url":
                     {
-                        "description": "A URL to which a GET request can be made to directly retrieve the contents of the Segment. Clients should include credentials if the provide URL is on the same origin as the API endpoint",
+                        "description": "A URL to which a GET request can be made to directly retrieve the contents of the Media Object. Clients should include credentials if the provide URL is on the same origin as the API endpoint",
                         "type": "string"
                     },
                     "label":
@@ -67,7 +67,7 @@
         },
         "key_frame_count":
         {
-            "description": "The number of key frames in the Segment. This should be set greater than zero when the Segment contains key frames that serve as a stream access point",
+            "description": "The number of key frames in the Media Object. This should be set greater than zero when the Media Object contains key frames that serve as a stream access point",
             "type": "integer"
         }
     }