-
Notifications
You must be signed in to change notification settings - Fork 19
Open
Description
rapids-s3-path is currently the source of truth for computing proper paths for CI workflow artifacts.
The problem with rapids-s3-path is that it only returns S3 URIs (e.g. s3://rapids-downloads/ci/cudf/...).
There are a lot of different areas where we call rapids-s3-path and then replace s3://rapids-downloads/ with https://downloads.rapids.ai/:
gha-tools/tools/_rapids-download-from-s3
Line 18 in 71f9e9a
s3_dl_path=$(echo "${s3_dl_path}" | sed "s|s3://${RAPIDS_DOWNLOADS_BUCKET}|https://${RAPIDS_DOWNLOADS_DOMAIN}|") gha-tools/tools/rapids-upload-to-s3
Line 36 in 71f9e9a
ARTIFACTS_URL=${browsable_url/s3:\/\/${RAPIDS_DOWNLOADS_BUCKET}\//https:\/\/downloads.rapids.ai\/} ARTIFACTS_URL=$(rapids-s3-path | sed "s|s3://${RAPIDS_DOWNLOADS_BUCKET}|https://${RAPIDS_DOWNLOADS_DOMAIN}|")
To prevent this manual replacement from being necessary, we should add a rapids-http-path script that computes the https://downloads.rapids.ai path.
The scripts should ultimately be refactored to work like this:
_rapids-path- a new script that's called byrapids-s3-pathandrapids-http-pathto generate the URI path. This script is prefixed with an underscore (_) to denote that it's not intended by be used publicly (only by othergha-toolsscripts).rapids-s3-path- should be refactored to call_rapids-pathand prepend thes3://rapids-downloads/stringrapids-http-path- a new script that calls_rapids-pathand prepends thehttps://downloads.rapids.ai/string
Metadata
Metadata
Assignees
Labels
No labels