RFC: Integration with Terraform Test #443
Description
Is this related to an existing feature request or issue?
No response
What part of the Cloud Game Development Toolkit does this RFC relate to?
Modules
Summary
This proposal is for integration with Terraform Tests. Ideally, all local and pipeline tests will be transitioned from Terratest to Terraform Tests.
Use case
The reasoning behind this is to streamline testing for the entire project, especially when it comes to new module contributions. Currently there is some portion that is manual, and in addition the requirement of tests in Terratest having to be written in Go can be a barrier to entry for contributors. With Terraform Test, tests are written in HCL and examples that will already be written/modified, can be used as the tests themselves which simplifies the process and encourages the writing of tests more frequently. Terraform Test manages state on your behalf, and will create and destroy resources based on the tests that are defined by running a simple terraform test command.
Proposal
This proposal is for integration with Terraform Tests. Ideally, all local and pipeline tests will be transitioned from Terratest to Terraform Tests. Alongside a new module addition, a subsequent example is written. This example will be pointed at for the test. Additionally, this makes it easier for newer contributors as they can easily run local and pipelines tests to have confidence in the functionality in their PR, even when making smaller changes. This can also be leveraged by dependabot to ensure small version updates also don't inadvertently break functionality.
For configuration, we'd leverage a centralized CI account where we could have dependencies already setup/configured (such as domains, etc.) which would be used for pipeline tests.
For local tests, we can leverage abp-local and define a series of tests that can run locally with the abp-local binary and docker. This will accelerate local development, and result in less pipeline runs until there is a substantial level of confidence in the functionality of the PR. Examples of these types of configurations can be seen in the IAM Identity Center Terraform Module, specifically in the .config, .project-automation, examples, and tests directories.
Out of scope
Testing of Packer may need to be out of scope for now in terms of Terraform Test, however I'd like to add it in the future depending on how much of a focus it will continue to be. There was a a few Packer issues (#439, #440) identified that a dedicated test would help avoid in the future. The test would create the AMI, validate it's creation in the correct region, and then destroy the AMI for cleanup).
At the moment I believe Terraform Test is only scoped to Terraform (as you'd probably assume). However, there may be other ways we can automate testing of Packer templates, though this may require leveraging a combination local-exec and null resource resources to run packer CLI commands, which would need to be in the context of the related Terraform Module(s) (e.g. Perforce modules). Alternatively, we could make a dedicated module for testing of all packer templates available in CGDTK.
Potential challenges
Potential challenges may be timing, as we currently are in a sprint. Ideally this would be implemented after this sprint is completed, and before the next one begins. There are already manny large PRs in the works, so adding another one that could potentially impact all modules and the ability of PRs to be merged is risky to do at the moment.
Dependencies and Integrations
No response
Alternative solutions
No response