rewrite in node, add parallel downloading

[timoxley]

Would you consider a pull request rewriting these shell scripts in node so they can be more easily parallelized? pull-all.sh is terribly slow and doing nested async in bash is clunky.

[dfreedm]

I'd certainly like to make that faster at some point, but it's a bit lower priority that a few other things.

[sjmiles]

Ideally nobody is using pull-all.sh but core developers. Unless you need to make commits to all these repos, you should be using Bower or tarballs.

This was the rationale in the first place for allowing one zillion repositories to exist.

[dfreedm]

So, the bring the conversation back into this bug: I've been thinking about this, and I think this is actually very important. However, I have 100 other things that are more important for now.

I hope that properly explains my split opinion (one in the wrong bug 👎 ) on this.

[timoxley]

Ideally nobody is using pull-all.sh but core developers.

@sjmiles I'm maintaining a browserify-compatible version of the platform thus I need to be able to build the platform easily and regularly. Polymer is 'alpha' enough that I can't wait on official releases plus there's occasionally been bugs to fix.

[sjmiles]

1. platform is a tiny fraction of the repos we are talking about.
2. Building platform does not require the commit access that is the key advantage of having Git checkouts.
3. bower update can update huge swaths of stuff in one step.

[timoxley]

Yep, I don't need all of pull-all but I was happy to contribute a fix for the lot if I was going to be bothered improving the platform-related stuff. Not sure why you're trying to convince me to not contribute a fix for this but ok.

I'll look into using the bower build again, I recall there was something prohibitive about using it before, I think it was infrequent builds, but looks like that's resolved now.

[sjmiles]

I'm not trying to convince you not to contribute, I'm only defending this statement which seemed to be in dispute.

Ideally nobody is using pull-all.sh but core developers

[timoxley]

@sjmiles ahh the reason I can't use the bower distribution is the bundled sourcemap. Just want vanilla concatenated js via the raw build I set up here: googlearchive/platform-dev#9

[sjmiles]

It's not clear what that has to do with Bower. The only thing you can't do with the Bower distribution is push back to GitHub, otherwise the content is identical.

[timoxley]

you can't get unminified, concatenated platform source with the current code in the bower distribution.

[dfreedm]

Fixed with ed4b5e7 and bigstraw