-
Notifications
You must be signed in to change notification settings - Fork 1
Description
There are new API endpoints available for the creation of model output profiles at runtime:
https://ncitest.modelevaluation.org/api/openapi.json
What do we need to do?
We need to update the client to be able to use these endpoints to create, modify and delete model outputs as needed by the benchcab workflow.
Why are we doing this?
At the moment, a user is required to provide configuration details pertaining to the exact model output needed to attach output files for analysis. This requires more comprehensive knowledge of how to configure the server-side of ME.org in order to get useful outputs.
By allowing the system to create "disposable" model outputs at runtime the user will need only provide their API credentials to the benchcab system which will do the rest based on a preconfigured workflow.
Specifics
I haven't gone into the detail of the spec yet, but initial discussions suggested the following:
- The client would create a name based on the branches being tested in benchcab (unique)
- That name would be used to create (or acquire an existing on in the case of rerunning) model output profile which is repopulated with experiment information etc.
- The profile would be used for all model output file uploads and used as the basis for triggering analyses.
Either on explicit deletion by the user, or when some other criteria is met the "disposable" model output profile will be deleted from the system.
The next piece of work will need to update benchcab to use the new functionality.