-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Want to extend ingestion endpoint to support multiple buckets/endpoints
An ingestion endpoint should consist of the following parameters when being registered with the DSA.
importProcedureName : This is a short handle to describe the process ( e.g. dcc-htan-duke or dcc-htan-duke-images)
importProcedureNameDescrtiption: Text description that's more verbose than the short name
importAssetstore --- Girder Internal MongoID that maps to internal assestore name
importPath--- This is the AWS path to use to begin the ingestion process; this can also be overridden during an actual POST
Import Folder: Girder Folder where the data is ingested to
includeRegex: As per the current ingestion endpoint
excludeRegex: As part current ingestion endpoint
onImportEnd: For now this will do nothing, but if we want to call some process on completion, this would be where the function could be stored; I'm not sure yet how we want to create/generate these
EndPoint for actual ingestion should require the importProcedureName to be specified, other parameters from above can be included if desired, but otherwise use the default that was specified during in the HTAN configuration
SECOND endpoint: This would dump the list of currently registered importProcedures and their metadata.. basically just dumps the JSON from above
