make a canonical scenario for testing various MCTS implementations against

[samwalls]

Currently, the only example implementation resides in tictactoe_test.go which was very useful for getting the initial implementation of the UCT algorithm working. As the tests indicate, two AIs playing against each other seems to lead to a draw every time.

However, as new implementations of MCTS get added it may be beneficial to have a canonical example of a game to test against, and measure for: complexity, absolute performance, and AI strength. The tic-tac-toe example would work for this purpose, however the search depth and branching factor are not very high at all.

The game Go seems like an ideal example to use for this purpose - I'd appreciate feedback about this, maybe there's a better example to use?

[mtfelian]

May be extend TTT to 10x10 board? It will give a much bigger tree.

Another variant is to implement checkers / some variant of chess like on 5x5 board.

Also it will be good (but not concerning this particular topic) to test it on games without full knowledge, like durak or poker.

[samwalls]

@mtfelian Expanding the size of the TTT board seems like a simple solution in the meantime.

Other aspects of the library which stick out to me as being untested right now:

• The determinization policy. The use-case for this policy are for game states where the selected action is only dependant on probability (not player choice).
• How does the branching factor of a game affect performance? (and, if it's significant, how to aleviate it?) Currently, "actions" are functions generated by the game implementer at run-time; and I have a feeling that having more actions available will affect performance somewhat.

I think poker might be a decent next step as it should give the chance to test both of the above, as well as dealing with incomplete knowledge as you mentioned. Once the library interface is slightly more stable (and documented a little) it should be easy to implement.

[mtfelian]

I'm now writing an engine for chess/checkers -like games on chess/checkers -like boards of custom size and form. Just like hobby (no, no bitmaps).

I think i will so-so finish it to the NY. So after completing the board engine, i plan to try this library with it.

I'm planning to implement something like Los Alamos Chess (mini-chess well known variant) based on my engine, hope this will work fine with it =)

Think if things will go well, i'll also contribute on some point here.