Conversation
* Add custom endpoint via `GOCACHE_AWS_URL`. * Add auth with session token via `GOCACHE_AWS_SESSION_TOKEN` * Use a default `us-east-1` region. * Fix ObjectID -> OutputID field rename * Fix TimeNanos -> Time field rename * Replace `GOCACHE_CACHE_KEY` with `GOCACHE_S3_PREFIX`. * Add compression to >8KB files on S3. * Use first 3 bytes of hash for prefix. * Fix NaN in timekeeping (and just ignore time not spent doing requests) * Update all dependencies
|
It is completely broken on Windows: golang/go#71059 RC1 also breaks if the experiment isn't enabled due to the renamed |
|
@or-shachar I was thinking, if there is a reason to (forcefully) separate on arch/platform. AFAICT the files are pretty much unique per platform anyway. We can either set it as metadata or tags (which will make it available to lifecycle filters). |
|
Thanks @klauspost ! Any chance you can pull changes from upstream, and add 1.24rc1 to the test matrix (github action files)? |
# Conflicts: # cachers/s3.go
|
Main merged and workflows updated, but I expect you need to approve them to run. |
|
mm... I was sure that I got the pull request trigger right. Can you kindly pull changes / rebase again? |
|
Merged + pushed. Seems like changes to workflows aren't being picked up before they are merged. |
|
Not sure why... I'll need to read a little more to figure it out. Will continue tomorrow. If you have an idea - let me know. |
or-shachar
left a comment
or-shachar
left a comment
There was a problem hiding this comment.
quick comment about the workflows change
or-shachar
left a comment
or-shachar
left a comment
There was a problem hiding this comment.
Still need to review the bytes operations but here is some questions I gathered.
| envVarS3AwsSessionToken = "GOCACHE_S3_SESSION_TOKEN" | ||
| envVarS3BucketName = "GOCACHE_S3_BUCKET" | ||
| envVarS3CacheKey = "GOCACHE_CACHE_KEY" | ||
| envVarS3Prefix = "GOCACHE_S3_PREFIX" |
There was a problem hiding this comment.
Yes. See discussion above. Honestly I thought I was talking to you until I had already sent in the change.
| // Copyright 2009 The Go Authors. All rights reserved. | ||
| // Use of this source code is governed by a BSD-style | ||
| // license that can be found in the LICENSE file. |
There was a problem hiding this comment.
What is this ? external source file? why is it checked in here? should we import it from somewhere else?
There was a problem hiding this comment.
Forks
bytes.Bufferto make it seekable, since non-TLS requests will include checksum.
The aws SDK needs an input that is seekable to work with HTTP endpoints. So I forked bytes.Buffer to include that.
3 participants
This breaks the placement on older versions. It will probably also break when using versions prior to 1.24 RC.
GOCACHE_S3_URL.GOCACHE_S3_SESSION_TOKENus-east-1region.GOCACHE_CACHE_KEYwithGOCACHE_S3_PREFIX.bytes.Bufferto make it seekable, since non-TLS requests will include checksum.If you want to test on our playground you can try...