Skip to content

rshiyani/terraform-provider-code-generator

BranchesTags

Repository files navigation

terraform-provider-code-generator (terraninja)

Terraninja is a CLI tool which leads to the creation of Terraform Provider Code File. The data used by Terraninja for creation of Terraform Provider File is provided in the form of YAML.

Technologies Used: Jinja - Used for Templating Python - Used for CLI Creation

File Structure

Your Workplace
|
+---config    // Config folder is created by user contains the configuration specified by user in yaml form which leads to the generation of Terraform Provider Files.
|   |   provider.yml    // You have to write Configuration Required for creation of provider.go file for Terraform Provider.
|   |   provider_generated.yml    // This File is Generated by Terraninja which preprocess provider file and is used for generation by Terraninja.
|   |
|   +---client    // Folder Named client contains the go-client configuration yamls for the resource.
|   |       movies.yml    // Contains client configuration of the resource.
|   |
|   +---datasources    // Folder Named datasources contains the datasources configuration yamls for the resources, leads to generation of datasource file and test file.
|   |       resource_name.yml    // Contains datasource configuration of the resource.
|   |       other_resource_name.yml // Contains datasource configuration of the resource.
|   |
|   \---resources    // Folder Named resources contains the resources configuration yamls for the resources, leads to generation of resource file and test file.
|   |   |   resource_name.yml    // Contains resource configuration of the resource.
|   |   |   other_resource_name.yml    // Contains resource configuration of the resource.
|   |   |
|   |   \---preprocess    // This Folder is generated by Terraninja which preprocess the user configuration and is used for generation by Terraninja.
|   |          resource_name_generated.yml    // Contains preprocessed resource configuration of the resource.
|   |           other_resource_name_generated.yml    // Contains preprocessed resource configuration of the resource.
|   +---terraformer  //Folder Named terraformer contains the resource terraformer configuration yamls, leads to generation of terraformer file.
|              tenant.yml  // Contains terraformer configuration of the resource tenant
|
+---output     // Output generated will be stored here.

Steps to Use Terraninja

  1. Build the executable of terajinja(use Ctrl+Shift+B to build), then you can check by running below command :-
$./terraninja -h

usage: terraninja.exe [-h] [--generate GENERATE] [--input INPUT] --pname PNAME

TerraJinja Tool that generates files for Terraform Provider.

Made By:  █▀▀ █▀█ █▀▀ █▀ ▀█▀   █▀▄ ▄▀█ ▀█▀ ▄▀█   █▀ █▄█ █▀ ▀█▀ █▀▀ █▀▄▀█ █▀
          █▄▄ █▀▄ ██▄ ▄█ ░█░   █▄▀ █▀█ ░█░ █▀█   ▄█ ░█░ ▄█ ░█░ ██▄ █░▀░█ ▄█

optional arguments:
  -h, --help           show this help message and exit
  --generate GENERATE  Enter the type you want to generate Eg. "resource,resource_test,datasource"
  --input INPUT        Enter the resource file name you want to generate for type Eg. "contract,subnet"
  --pname PNAME        Enter the provider name Eg. "aws"
  • --pname(Provider Name): This flag is a required.
  • --generate: Here the comma seperated string is provided. By default the value of this is "all". Supported Values: resource, resource_test, datasource, datasource_test, provider, provider_test, model, client, client_test and terraformer
  • --input: Comma separated list of input files, default value is all.

👀Note: .yaml extension may not work. Prefer using .yml

Examples:

./terraninja --pname="YourProviderName"

Above command will generate all the terraform provider files for the ymls provided in config.

./terraninja --pname="YourProviderName" --input="contract,tenant"

Above command will generate all the terraform provider files for the ymls provided in config for resource contract and tenant only.

./terraninja --pname="YourProviderName" --generate="resource,resource_test"

Above command will generate the terraform provider resource and resource_test files for the ymls provided in config.

./terraninja --pname="YourProviderName" --input="tenant" --generate="datasource,datasource_test"

Above command will generate the terraform provider datasource and datasource_test files for the ymls provided for tenant resource in config.

./terraninja --pname="YourProviderName" --generate="provider,provider_test"

Above command will generate the terraform provider.go and provider_test.go file for the provider yaml provided in config.

YAML Guide:

To enable yaml validation json schemas and autosuggestions:

For resource : yaml header

# yaml-language-server: $schema=../../json_schema/resource_validate.json

yaml path

/config/resources/

For datasource : yaml header

# yaml-language-server: $schema=../../json_schema/datasource_validate.json

yaml path

/config/datasources/

For provider : yaml header

# yaml-language-server: $schema=../json_schema/provider_validate.json

yaml path

/config/provider.yml

For client : yaml header

# yaml-language-server: $schema=../../json_schema/client_validate.json

yaml path

/config/client/

For terraformer : yaml header

# yaml-language-server: $schema=../../json_schema/terraformer_validate.json

yaml path

/config/terraformer/

Resource Datasource Provider Terraformer
For generating the terraform resources,provider,datasource with their test files and terraformer, the yaml files should be kept to the above given paths.Here's the description of every attribute of why it exists and how it will be used in the generated file.

It is compulsory to add yaml-language-server, which provides you the autosuggestion while writing the yaml.

Attribute Description
provider_name string
The name of the Terraform Provider you are working on.
name string
The name of the resource for which you are writting yaml.
imports list of strings
List of import links of modules to include in go file.
unique_attribute list of strings
unique_attribute is an array of strings that must be required and not dependent on parent and is of type string, it is preferred that you only write one parameter only, Moreover it is used for parallel testing.
schemas list of objects
schemas contains the list of objects, each object represents the schema element for resource.
attribute string
attribute is basically a field for objects in schemas or schema which denotes the name of the attribute.
type string
type is basically a field for objects in schemas or schema which denotes the type of the attribute. Values which can be provided to this field are string, int, float, bool, map, list and set.
forcenew boolean(fixed:true)
forcenew is basically a field for objects in schemas or schema which denotes whether we want to enable or disable forcenew for the attribute in schema.
required boolean(fixed:true)
required is basically a field for objects in schemas or schema which denotes whether we want to enable or disable required for the attribute in schema.
computed boolean(fixed:true)
computed is basically a field for objects in schemas or schema which denotes whether we want to enable or disable computed for the attribute in schema.
optional boolean(fixed:true)
optional is basically a field for objects in schemas or schema which denotes whether we want to enable or disable optional for the attribute in schema.
sensitive boolean(fixed:true)
sensitive is basically a field for objects in schemas or schema which denotes whether we want to enable or disable sensitive for the sensitive attribute in schema.
default_func boolean(fixed:true)
default_func is basically a field for schema.EnvDefaultFunc used for provider.
envvar string
envvar is basically a field for setting schema.EnvDefaultFunc environment variable used for provider.
default (number,string)
default is basically a field for objects in schemas or schema which is used when we want to provide the default value to the attribute in schema.
description string
description is basically a field for objects in schemas or schema which denotes the description of the attribute.
max_items number
max_items is used to defines a maximum amount of items that can exist within a TypeSet or TypeList.
min_items number
max_items is used to defines a maximum amount of items that can exist within a TypeSet or TypeList.
validation object
validation is basically a field for objects in schemas or schema which actually is a object which contains func_name as a attribute, However it is used for the primitive datatypes of schema.
func_name string
Field of Validation object which is a validateDiagFunc function. If a validation function needs a params and ignore_case in case of StringInSlice validation
ignore_case boolean
Field of Validation object which is used for StringInSlice/StringNotInSlice which returns a SchemaValidateFunc which tests if the provided value is of type string and matches/not matches the value of an element in the valid slice will test with in lower case if ignoreCase is true.
params list
Field of Validation object which is used to list test parameters.
element object
element is basically a feild for objects in schemas or schema which should be provided when the type is in list,set,map. It is a object which contains the type and schema attribute.Used for Resource only.
type string
Field of element object used to describe Schema type as Resource/Schema for Elem in Type(List/Set/Map).
schema list of object(for type:resource)
Field of element with type(resource) used to describe Internal Schema Resource for Type(List/Set/Map).
schema object(for type:schema)
Field of element with type(schema) used to describe Internal Schema Schema for Type(List/Set/Map).
depends_on object
depends_on is basically a field for objects in schemas or schema which is used to describe dependent resource name with hierarchy of attributes.
resource_name string
Field of depends_on object which is used to describe dependent resource name.
attribute_name list
Field of depends_on object which is used to describe list of dependent resource name with hierarchy.Example application_profile have tenant_dn which is dependent on tenant so we here provide id in attribute_name list and tenant as resource name.
getURL string
(terraformer only) getURL is a URL used to get all resources from API.
resource_allowed_empty_attributes list of strings
(terraformer only) resource_allowed_empty_attributes is a list of attributes which is used to describe allowed empty attributes for the resource.

👀It is recommended to create yaml schemas proper as per the rules provided by the Terraform. For Example: One can't provide required and optional both simultaneously to the single attribute. Refer this links for more info:-

  1. Schema Behaviour
  2. Schema Types

Use Autosuggestions for ease of use in writing YAML 😎 One can find the example config at /config/resources/movie.yml

client
For generating the terraform go client, the yaml files will go to the /config/client directory. An example movies.yml is already provided for the reference. Here's the description of every attribute of why it exists and how it will be used in the generated file.

Attribute Description
include list of strings
Include contains all the third party packages that needs to be imported for the execution of the go file. Built in packages will be automatically imported. Listing the packages here does not essentially do anything helpful but you can simply hover over them in vs code and install them with one click. Kinda makes the work easier.
endpoint list of objects
Endpoints consists of the properties path, name, GET / POST / PUT / DELETE and params. At least one of the HTTP Method is required to be present.
endpoint.path string
path is a string that holds the path to where the specified HTTP request needs to be made.
endpoint.name string
name will be used in conjuction with the HTTP Method to generate the function name.
endpoint.params list of strings
params is an array of strings that contains the query params from the path. for example, if the path is /movies/id and id is a query param, then you can specify it in the params and it will be handled accordingly.
endpoint.overwriteBaseURL This is not ready to use yet, can be ignored.
endpoint.METHOD object
Every method will contain 2 properties. funcName and args. POST Method can contain one additional property unique. the value for unique attribute can be set the unique identifier variable name of the resource from the response.
endpoint.METHOD.funcName string
funcName is a string denoting the function name. In case you'd like to specify a name yourself, or in conflicting scenarios, such as when there exists multiple methods of the same type on endpoints who have same value for the name attribute.
endpoint.METHOD.args string
Args refers to the arguments the function created will have. it will have the following properties: name, type, send, pointer, and value. The first two are self-explanatory. send indicates if this argument is to be sent as payload when making the request, pointer indicates that this argument will be a pointer, and value can be statically defined if its a primitive variable otherwise will be a reference to one of the item from the data section. Value will be used for the testing purposes when calling the function. Other methods can also contain fromPostResponse, that will us the id of the resource acquired from POST call as value.
data list of objects
This contains all the non-primitive values for variables such as structs. This section is very much similar to args hence not explained in depth.

👀It is recommended to create endpoints in the particular order of POST -> GET -> PUT -> DELETE. This way, you can use the fromPostRequest attribute in the subsequent endpoints to perform the GET, PUT and DELETE operations on the same resource created with the POST call.

Use Autosuggestions for ease of use in writing YAML 😎

This ensures that the resource will be deleted every time and ensures that the same test can be run successfully multiple times without worrying that resource with the same value would be already existing that rather expects some unique value.

The data and endpoints part of the yaml have some inconsistencies. Such as when defining the args and data attributes. Of course this can be made more uniform, but it was a lazy decision, and the efforts were better to be put at completing the features rather than refactoring for the aesthetics.

You can find the example config at /config/client/movies.yml

endpoints:
  - path: "/movies"
    name: movie
    GET:
      funcName: GetAllMovies
   ...

Contributor Guide

Jinja Templates are contained in the templates directory. Any change for the existing resources are to be made there. You can create a new template or modify the existing ones as per the requirements. Make sure that the new changes does not break the existing resources generated using the current state of the product. For major updates, if possible maintain another branch, repo or version. Merge only after thoroughly testing and making sure that the new updates does not introduct any breaking updates.

JSON Schema for validating the YAMLs will go in the json_schema folder. If you introduce any new attribute and change any exising ones, you can modify the schema there. Comments are provided whenever seemed necessary. The same is expected for future changes, to doucment the code using comments whenever necessary and to keep the readme updated.

YourWorkplace
|   .gitignore
|   helper.py    // Contains the Helper Function for Creation of Terraform Provider Files for Terraninja
|   LICENSE
|   loading_animation.py    // Contains the Logic for Formatting Go Files and Loading Animation
|   preprocess.py    // Contains the Function which preprocess the user file and generate new file used for generation of Terraform Provider File
|   README.md    // README contains the information related to use of terraninja
|   requirements.txt    // Contains the names of the dependencies
|   terraninja.py    // Contains the Logic of the CLI. It's the main file which leads to the execution of the terraninja CLI.
|   terraninja.spec    // Autogenerated when we build the execuatable(.exe file) of the terraninja
|   utils.py    // Contains the Function for Custom JINJA Filters used in templating.
|
+---.vscode    // Contains the Workplace Settings, User Settings
|       launch.json    // File Controls the run/debug settings of your Vistual Studio Code Project
|       move_binary.sh    // Contains the script for movoing the executable generated to the path of directory where the system environment variable is set.
|       tasks.json    // Workplace or Folder Specific Tasks are configured in tasks.json file.
|
+---build    // This Folder is created automatically when you build the executable for the project
|   \---terraninja
|           Analysis-00.toc
|           base_library.zip
|           EXE-00.toc
|           PKG-00.toc
|           PYZ-00.pyz
|           PYZ-00.toc
|           terraninja.exe.manifest
|           terraninja.pkg
|           Tree-00.toc
|           Tree-01.toc
|           Tree-02.toc
|           warn-terraninja.txt
|           xref-terraninja.html
|
+---config    // Config folder is created by user contains the configuration specified by user in yaml form which leads to the generation of Terraform Provider Files.
|   |   provider.yml    // You have to write Configuration Required for creation of provider.go file for Terraform Provider.
|   |   provider_generated.yml    // This File is Generated by Terraninja which preprocess provider file and is used for generation by Terraninja.
|   |
|   +---client    // Folder Named client contains the go-client configuration yamls for the resource.
|   |       movies.yml    // Contains client configuration of the resource.
|   |
|   +---datasources    // Folder Named datasources contains the datasources configuration yamls for the resources, leads to generation of datasource file and test file.
|   |       resource_name.yml    // Contains datasource configuration of the resource.
|   |       other_resource_name.yml // Contains datasource configuration of the resource.
|   |
|   \---resources    // Folder Named resources contains the resources configuration yamls for the resources, leads to generation of resource file and test file.
|   |   |   resource_name.yml    // Contains resource configuration of the resource.
|   |   |   other_resource_name.yml    // Contains resource configuration of the resource.
|   |   |
|   |   \---preprocess    // This Folder is generated by Terraninja which preprocess the user configuration and is used for generation by Terraninja.
|   |           resource_name_generated.yml    // Contains preprocessed resource configuration of the resource.
|   |           other_resource_name_generated.yml    // Contains preprocessed resource configuration of the resource.
|   |
|   +---terraformer  //Folder Named terraformer contains the resource terraformer configuration yamls, leads to generation of terraformer file.
|              tenant.yml  // Contains terraformer configuration of the resource tenant
|
+---dist    // The Generated Executable File is stored in the dist Folder.
|
+---json_schema    // This folder contains the file which is responsible to display autocomplete suggestions for configuration files in YAML Format.
|       client_validate.json    // Contains the json_schema responsible for autocomplete suggestions for Client YAML Files.
|       datasource_validate.json    // Contains the json_schema responsible for autocomplete suggestions for Datasource YAML Files.
|       provider_validate.json    // Contains the json_schema responsible for autocomplete suggestions for Provider YAML File.
|       resource_validate.json    // Contains the json_schema responsible for autocomplete suggestions for Resource YAML Files.
|
+---output     // Ouput generated will be stored here.
|
+---templates    // Folder contains the general templates written in jinja used for generation of Terraform Provider Files.
|   |   client.j2    // File Contains the template for Client File
|   |   client_scaffold.j2    // File Contains the Static General template for CLient Scaffold File
|   |   client_test.j2    // File Contains the template for Client Test File
|   |   datasource.j2    // File Contains the template for Datasource File
|   |   datasource_test.j2    // File Contains the template for Datasource Test File
|   |   model.j2    // File Contains the template for Model File
|   |   provider.j2    // File Contains the template for Provider File
|   |   provider_test.j2    // File Contains the template for Provider Test File
|   |   resource.j2    // File contains the template for Resource File
|   |   resource_test.j2    // File contains the template for Resource Test File
|   |
|   \---helpers    // This Folder Contains the Helper Macros For the template
|           resource_crud.j2    // Contains the Helper Macros for Resource File.
|           schema.j2    // Contains the Helper Macros for Schema Generation.
|           test.j2    // Contains the Helper Macros for Resource Test and Datasource Test File.
|
\---__pycache__    // This Folder is created automatically Contains the bytecode-compiled and optimised bytecode-compiled versions of program files.
        helper.cpython-39.pyc
        loading_animation.cpython-39.pyc
        main.cpython-39.pyc
        preprocess.cpython-39.pyc
        utils.cpython-310.pyc
        utils.cpython-39.pyc

About

No description, website, or topics provided.

Resources

Readme

License

MIT license
Activity

Stars

1 star

Watchers

2 watching

Forks

0 forks
Report repository

Releases

1 tags

Packages

 
 
 

Contributors

Languages