Skip to content

Modify HTAN Ingest API #1

@dgutman

Description

@dgutman

Want to extend ingestion endpoint to support multiple buckets/endpoints

An ingestion endpoint should consist of the following parameters when being registered with the DSA.

importProcedureName : This is a short handle to describe the process ( e.g. dcc-htan-duke or dcc-htan-duke-images)
importProcedureNameDescrtiption: Text description that's more verbose than the short name
importAssetstore --- Girder Internal MongoID that maps to internal assestore name
importPath--- This is the AWS path to use to begin the ingestion process; this can also be overridden during an actual POST
Import Folder: Girder Folder where the data is ingested to
includeRegex: As per the current ingestion endpoint
excludeRegex: As part current ingestion endpoint
onImportEnd: For now this will do nothing, but if we want to call some process on completion, this would be where the function could be stored; I'm not sure yet how we want to create/generate these

image

EndPoint for actual ingestion should require the importProcedureName to be specified, other parameters from above can be included if desired, but otherwise use the default that was specified during in the HTAN configuration

SECOND endpoint: This would dump the list of currently registered importProcedures and their metadata.. basically just dumps the JSON from above

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions