Salt CICD

(The true source for this repository is located here)

This is collection of tools for linting Salt states and pillar files and triggering GitFS updates via the Salt API. It is intended to be used as the container image in relevant CI/CD workflows.

Container images are based on Alpine Linux and include the following tools:

• A deploy script for triggering fileserver updates via the Salt API
• A linting script that leverages these projects to check for problems in sls files
  • j2lint
  • yamllint
  • salt-lint

Example CI/CD Configurations

Example configurations for Gitlab CI/CD, Github Actions and yamllint/salt-lint configs can be found in the example-configs directory.

I have primarily used this project to only lint the files that are included in merge or pull requests. However, it can certainly be set up to evaluate all sls files in a repository upon any commit to the repository.

Disabling yamllint for individual files

Some files will not play nice with yamllint due to how Jinja templating is used with them. To disable yammlint for the entire file, add this line as the first line of the file

# yamllint disable

You can also use the following configuration in the .yamllint configuration file to ignore files or folders

ignore: |
  path/to/*/ignored/folder/
  path/to/ignored-file.sls

More Documentation on how to disable yammllint checks within files: https://yamllint.readthedocs.io/en/stable/disable_with_comments.html

CLI Examples

Linting Script

Script Arguments:

• -f takes a single argument of a path to a .sls file (only used with a linting scope of "single")
• -t takes a single argument of "jinja", "salt", or "yaml"
• -s takes a single argument of "merge", "single" or "all"
  • merge will only test the .sls files modified in the merge/pull request
  • single will test a single file (supplied by the -f option)
  • all will test all .sls files within the repository

Examples:

# Lint only the .sls files included in a merge request with salt-lint
linting -t salt -s merge

# Lint all .sls files in the repository with yamllint
linting -t yaml -s all

Considerations:

• It is suggested to run each linting type in a separate CI/CD job for several reasons:
  • This makes it easier to find what type of code fixes are needed
  • It allows for parallel execution of linting jobs to speed up workflows
  • YAML linting will introduce comment characters in front of jinja templating lines in the sls files being evaluated so it is best to keep it in a separate job. This is necessary as yamllint will throw errors about the templating lines. Comment characters are used instead of deleting the lines to preserve the line numbers of the code.

Deploy Script (Triggers gitfs updates via the Salt API)

The deploy script requires the setup of the Salt API (see example below) to allow access to the following runner modules

• fileserver.update
• git_pillar.update

Examples:

# Update the Salt Fileserver (salt-run fileserver.update) via API
deploy -u "$saltapi_user" -p "$saltapi_pass" -e "$saltapi_eauth" -s "$saltapi_server" -t states

# Update Salt Pillar (salt-run git_pillar.update) via API
deploy -u "$saltapi_user" -p "$saltapi_pass" -e "$saltapi_eauth" -s "$saltapi_server" -t pillar

Example Salt API Configuration for the Deploy Script

Ensure the salt-api package is installed on the Salt Master

This will configure the Salt API to receive connections on the Salt Master's IP on port 8443. It assumes that LDAP is already configured for the Master and the saltapi_gitlab user exists in the directory. The configuration only allows two Gitlab runner servers (10.0.0.80 and 10.0.0.81) to access the API.

# /etc/salt/master.d/master.conf

# External Authentication
external_auth:
  ldap:
    'saltapi_gitlab':
      - '@runner':
        - 'fileserver.update'
        - 'git_pillar.update'

# Enable the netapi client interfaces
netapi_enable_clients:
  - runner

# Salt API Configuration
rest_cherrypy:
  host: 0.0.0.0
  port: 8443
  ssl_crt: /etc/ssl/certs/certificate.crt
  ssl_key: /etc/ssl/private/certificate.key
  api_acl:
    users:
      saltapi_gitlab:
        - '10.0.0.80' # git-runner1.example.com
        - '10.0.0.81' # git-runner2.example.com