A quick warning: Note: This testing harness is experimental and needs more work. If you want to contribute, please read the future work section. Or take a look at the code.
Brane is a complicated piece of software with a lot of code. Most parts that interact with the operating system and file system can be very hard to test. To automate this testing we present: Brane-test!
The initial goal for this testing harness was not actually writing the harness
itself, but making sure that Brane is well behaved with the Docker in Docker
(DinD) environment. After some alterations this seems to be the case asnd Brane
passes the tests. Note that the alterations are not yet reviewed and are
therefore not merged into the main branch, but can be found on the automated-testing
branch.
The current state of the project is proof of concept. The harness works and an old Brane demo with multiple packages is successfully ran without any intervention. However the demo and configution is currently hardcoded. Cleaning up the code and implementing more, but smaller tests is an obvious next step.
The design of the harness consists of two binaries. A harness that creates the outer docker environment and starts the "runner" binary in a container. The runner is ran in a container to isolate the filesystem from the host system. This way your local configuration of Brane is not influencing the tests.
Then there is the already mentioend inner runner. This binary actually does the testing and basically performs the system testing tasks automatically. It reads the docker logs of the Brane instance for both errors and to see if the Brane microservices is up and running and ready for testing.
It should be possible to concurrently test multiple configurations of Brane at once. Either by from running multiple instances of Brane in the runner, or by running multiple (different) runners. The latter making most sense as it would allow for multiple different "host" environments when running Brane. This could be circumvented in the runner as well, but the chances of accidentally sharing a file is a lot higher and care should be taken to prevent this.
While the foundation of the testing framework has been laid. Brane test can use a lot of love to get to a point where is can thoroughly test all components of Brane. I would suggest one starts with implementing more test workflows and generating the network configuration on the fly using branectl. This way branectl is tested as well and more network topologies can be tested.