Self-hosted secrets manager

[shawngerrard]

We need a secrets manager to centrally manage, automate, and standardize authentication with external service providers.

There's multiple solutions out there, but some basic requirements:

• Passwords have to be encrypted at rest.
• Scalable technically and economically.
• Ideally open-source, cloud native.
• Access to secrets can be restricted.
• Support for changing secrets or secret rotation.
• Ability to automatically interact with pulumi, aws, kubernetes, etc.

First steps TBD.

[jmhbnz]

There are some chicken and egg situations we need to be careful of here. In order to deploy the infrastructure that would host our secrets manager we need some shared secrets to already exist and be accessible by team members.

As a balance and to give us a way to proceed with these initial shared secrets I suggest we enable gcp secrets manager at the asterion.digital organisation level in google cloud platform: https://console.cloud.google.com/marketplace/product/google/secretmanager.googleapis.com

We already rely on gcp for our company active directory so extending this slightly for some initial shared secrets makes sense to me.

In terms of this story for self-hosted secrets I would be in favor of developing a new app - vault stack to deploy hashicorp vault via helm: https://www.hashicorp.com/products/vault. @daljitdokal Has some good experience working with the vault helm chart and could supporting bringing this into a pulumi stack approach.