Suggestion: Add in a regression test of some sort

[cbbrowne]

It would be a good idea to add in a regression test that:

• Sets up the database from scratch
• Adds a few players
• Makes them do some things
• Returns some sort of "true" value

Several salutory effects:
a) Means that it doesn't take much work to see if "that dumb patch someone contributed" breaks stuff
b) If done carefully enough, allows using "git bisect" to figure exactly which, of 45 "dumb patches someone contributed" was to blame
c) Increases confidence in proposed changes. "Yes, the system survives that surgery!"
d) Provides examples of how to do stuff to would-be players.

[cbbrowne]

Trivial thing; just loads the schema. (Possibly badly.)
cbbrowne@fc9bfac

[Abstrct]

Sorry, I keep forgetting to reply to this thread. I think starting to develop a testing framework for the game is probably a great idea.

This looks like a great start so far. We just need to work in the perl scripts.

I have been working on a GettingStarted.sql script for a while (although it kind of sucks currently). It cannot be that difficult to run it a couple times to understand where it should be at certain portions of the round, then use these values to validate the changes. The only thing that should change would be the amount of fuel being returned during mine (although this can be forced with setseet() for testing purposes).

[cbbrowne]

Yeah, running the perl script is an apropos addition.

I'm not sure it's necessary to force completely deterministic results (e.g. - make the random bits repeatable).

The usual approach to "smoke tests", a good example being the regression test that's in the PostgreSQL build, is NOT to try to exercise every single bit of logic. Rather, to make sure that enough interesting bits are exercised that you know when things break.

For Schemaverse, it's important to know that:
a) The schema still loads properly, start to end
b) The ticker tics
c) All the functions that players use are functioning
d) A fleet script can be run

Super-comprehensive isn't necessary. Really, it would be mighty nice if the whole test runs in a few seconds. That would make it plausible to use "git bisect" to search for the stupid change that busted things. If it takes 20 minutes for the game to "warm up" before you know if the test worked, that won't work out terribly well ;-).

So, ideally, I think you'd want to:
a) Load schema
b) Add 2 players
c) Player 1 has 1 script
d) Player 2 has 2 scripts
e) Fudge parameters so that they have enough money to build 50 ships, and enhance speed so that they can start attacking each other after a single turn
f) Set up 3 fleets across the players
g) Make sure a planet has >30 ships, so some mining is expected to fail
h) Move close enough to attack, and do so
i) Repair some damage
j) Look at the events table to make sure that everything expected occurred

In effect, a successful run gives a known output that's basically a COUNT(*) query against my_events.