Skip to content
This repository was archived by the owner on Jul 7, 2022. It is now read-only.

Parallelize S3 pull/push files + various bug fixes, take 2.#11

Open
didip wants to merge 38 commits intoblake-education:masterfrom
newrelic-forks:blake-education-pr
Open

Parallelize S3 pull/push files + various bug fixes, take 2.#11
didip wants to merge 38 commits intoblake-education:masterfrom
newrelic-forks:blake-education-pr

Conversation

@didip
Copy link

@didip didip commented Oct 16, 2014

NOTE: I am closing PR #10 and replace it with this one to fix import paths so the PR can be merged as-is.

Original Message:

This is PR is essentially a merge of changes between 2 level of forks (aldrinleal and New Relic).

The changelog of what we, New Relic, made:

  • Use go fmt on all files.
  • Use the official fsouza/go-dockerclient
  • Set GOMAXPROCS to consume all available CPU.
  • Pulling files from S3 and pushing to S3 are parallelized using fixed pool of goroutines, currently hard-coded at 50.
  • Removed the ability to set S3 prefix path. It makes debugging image name difficult.
  • Logging functionality is now standarized to using fmt library.
  • Added various code cleanup and test fixes.

@relistan
Copy link

FYI @lachie, we've integrated Dogestry support into Centurion: https://github.com/newrelic/centurion . It's currently linked to our fork, but if this is merged we'll point it here.

@lachie
Copy link
Contributor

lachie commented Oct 20, 2014

Hey @relistan I'm happy for you guys to take over dogestry.
I simply don't have the time to give it the love it deserves and I'm really happy that you do.
I just created the dogestry org and added you and @didip as owners. How about we move it there and thus remove the fork name ambiguity?
Let me know if you want to chat some more outside PR messages :)

@didip
Copy link
Author

didip commented Oct 20, 2014

I think that sounds great. Thanks for helping us out @lachie!

@relistan
Copy link

Sounds good! Thanks :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants