Design: Batch add of endpoints

[Ben-Hodgkiss]

Overview

We need to consider the designs to update the functionality of the Manage service and async to allow it to process multiple endpoints in the same run. A rough user flow would be as follows:

• User can upload multiple lines via the Import CSV functionality. Data adding via the form completion would still be limited to only one endpoint per run at first.
• The endpoints are analysed via the async Check service. Once all endpoints have been analysed, the Check results page is shown. (Nice to have would be to show the progress of the analysis via the loading page).
• The Check tool shows the results for each endpoint. We may need to consider the designs for this to decide if we would want all the results to show on one page, or be tabulated with one tab per endpoint checked.
• Configure would be available per endpoint, with that endpoint re-processed following configuration.
• User would be able to remove a particular endpoint from the batch via Check if it could not be resolved.
• User would then add data - this would create the file changes for all the endpoints potentially across multiple collections (source, endpoint, lookup, entity-org, config CSVs etc...)
• Github Action would add all endpoints in same action to same PR ready for review.

This ticket is for us to consider the designs of this process as it isn't ready for development until we understand key aspects of it. We will need to have clear designs of the following pages/processes to pass to developers:

• Confirmation of parsing data following import.CSV
• Loading page showing analysis of progress
• Check Results page with endpoint tabulation and ability to remove an endpoint from the batch
• Add Data page with multiple endpoints across multiple collections.

Once these designs are complete, a ticket should be written to pass this over into the development backlog.

[paris-dp]

Booked a meeting with @Swati-Dash to discuss how this should work

[paris-dp]

Discussed with @Swati-Dash this afternoon.

We discussed that when adding multiple endpoints, the same steps would need to be completed so it would be helpful to still go through these steps one at a time, however what does take more time is waiting for PRs to be approved.

We also talked through workflow and the complexity of having tabs for each endpoint, along with the potential of configuration for each endpoint.

We thought about a solution which would allow the user adding data to work through an endpoint (including check and configuration) with the option to add another within that batch.

This works on the assumption that the now logged in user experience could store check results/configurations and rows that would need to be added and assemble these to be added.

@Ben-Hodgkiss happy to discuss/walk this through as I know it is different from what the ticket outlines.

[paris-dp]

Discussed with @Ben-Hodgkiss today and how we could approach this. However agreed that this would be a later iteration due to the main priority being being able to add multiple endpoints per PR. Going to bring this ticket forward ahead of batch adding of endpoints