Specs don't run on a clean checkout

[bjeanes]

Based on the output from this build (a test run for future pull request #59), it seems that it expects the config files to be set up with certain values which only get written out by setup.rb.

Are there any downsides you foresee to giving all configs sane default values, prior to a setup.rb run, @RISCfuture? Alternatively (and possibly more desirably), the tests could setup the configs as they expect them (i.e. if a test relies on authentication being set to "password", it should just set it up front for that test).

[RISCfuture]

There's already a script in the script directory that runs specs under a variety of setups. Requires MRI and JRuby.

[bjeanes]

Interesting... that's not likely to be immediately helpful in the context of Travis since it does all the combinations itself to track the results in the matrix. Do the LDAP tests need an LDAP server running to work?

I wonder if it makes sense to do what Rails does here (for example, when testing ActiveRecord against different DBs). That is, feed in these configs as environment variables and have the test suite itself do the right thing for that scenario. The all_specs.sh script can be a shim to keep existing expectations in line.

[RISCfuture]

LDAP tests are all mocked; no server is required. The main point of all_specs is to just allow me to do a quick comprehensive spec run under all major configuration possibilities before pushing code. I'm fine altering it for Travis support but, to me, Travis support (which happens after I've already pushed) is less important than having a way to verify my code works before pushing it.

[bjeanes]

Agreed. The real benefit of Travis for a project maintainer, IMO, is seeing if a pull request passes or not.