Infra

This repository contains the infrastructure code for the project. We use Terraform to manage the infrastructure on GCP.

If you want to deploy the infrastructure on your own, please create the gcs bucket for the terraform backend first. You can find the terraform backend configuration in the provider.tf file.

Cloud Architecture

Installation

Install Terraform on your local machine.

For windows users, you can download the Terraform by chocolatey. Run the following command in the terminal.

choco install terraform

For Mac users, you can download the Terraform by brew. Run the following command in the terminal.

brew install terraform

Then, you need to authenticate with GCP. You need to install the Google Cloud SDK. You can download the SDK from here.

After installing the SDK, you need to authenticate with GCP. Run the following command in the terminal.

gcloud auth application-default login

To authenticate with GCP in github actions, you need to create a service account, download the JSON key file and save it in the github secrets (GCP_CREDENTIALS_PROD).

Usage

Here are some useful Terraform commands :

terraform init                  # Initialize the Terraform and download the provider plugins
terraform fmt --recursive       # Format the Terraform configuration
terraform validate              # Validate the Terraform configuration
terraform plan --out plan.out   # Plan the infrastructure changes
terraform apply                 # Apply the infrastructure changes
terraform destroy               # Destroy the infrastructure

Environment

For now, we only have one environment, which is prod. The prod environment is defined in the prod folder and the domain is in the domain/prod folder.

There are three things you need to care about when you are working on reproducing the infrastructure.

• Domain Management: We recommend not managing domain resources via Terraform, as their lifecycle is difficult to maintain and may lead to unexpected behaviors.

• Network Endpoint Groups (NEGs): You should avoid using Terraform to create Network Endpoint Groups (NEGs) for Kubernetes. Instead, rely on Kubernetes' built-in tools, which can dynamically add or remove endpoints as needed — a capability that Terraform lacks. Please refer to the examples below:

  • terrafrom-neg
  • k8s-neg

• Secrets Management: We use Google Secret Manager to manage secrets in GCP, rather than using k8s-secret, since secrets in Kubernetes are stored in etcd and are not encrypted by default. Ensure that the corresponding secrets and service accounts are properly created in both GCP and Kubernetes. You can find reference implementations in the following files:

  • terraform
  • k8s-sa
  • k8s-secret-provider.

  Although I use secret manager in terraform, I recommend you to do it manually in GCP because it's easier to manage.

The following is the module list of the project.

• apis : Enable all the apis for the project.
• vpc : Create the vpc and subnets for the project.
• gcs : Store the frontend code (React)
• secretmanager : Create the secret manager for the project.
• gke : Create the GKE cluster for the project. (backend)
• loadbalancer : Create the load balancer to navigate the traffic to the gcs (frontend) and gke (backend).
• dns : Create the dns for the domain.