Skip to content

api-v2-new #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Hadishad wants to merge 10 commits into
base: api-v2-new
Choose a base branch
from
Open

api-v2-new #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
88280ab
Updated readme
danielfroding Feb 26, 2025
9deb671
Update demo-application.yml
Hadishad Feb 28, 2025
7edd1fb
Update demo-application.yml
Hadishad Feb 28, 2025
d975963
Update demo-application.yml
Hadishad Feb 28, 2025
54e72f1
Update demo-application.yml
Hadishad Feb 28, 2025
816dac5
Update demo-application.yml
Hadishad Feb 28, 2025
0d2ea4a
Update demo-application.yml
Hadishad Feb 28, 2025
171ba86
Update demo-application.yml
Hadishad Feb 28, 2025
a08ac8f
Update demo-application.yml
Hadishad Feb 28, 2025
2fcadaa
Merge branch 'api-v2-new' into main
Hadishad Feb 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions .github/workflows/demo-application.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,13 @@ on:
env:
PROJECT_ID: ${{ vars.PROJECT_ID }}
GAR_LOCATION: europe-north1
SERVICE: demo-${{ github.repository_owner }}
SERVICE: demo-hadishad
REGION: europe-north1
REPOSITORY: demo

jobs:
build-and-unittest:
runs-on: ubuntu-latest
runs-on: self-hosted
steps:
- name: Checkout project sources
uses: actions/checkout@v4
Expand All @@ -74,9 +74,10 @@ jobs:
distribution: 'zulu'
java-version: '17'
cache: 'gradle'
- name: Run build with Gradle Wrapper
- name: Grant Execute Permission (Windows)
if: runner.os == 'Windows'
working-directory: ./cloud-run-pipeline-demo
run: chmod a+x gradlew && ./gradlew test build
run: icacls gradlew /grant Everyone:RX

docker-build-and-push:
# Add 'id-token' with the intended permissions for workload identity federation
Expand All @@ -95,9 +96,10 @@ jobs:
distribution: 'zulu'
java-version: '17'
cache: 'gradle'
- name: Run build with Gradle Wrapper
- name: Run build with Gradle Wrapper (Linux)
working-directory: ./cloud-run-pipeline-demo
run: chmod a+x gradlew && ./gradlew test build
run: chmod +x gradlew && ./gradlew test build
shell: bash

- name: Google Auth
id: auth
Expand Down
243 changes: 71 additions & 172 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,15 @@ This course will guide you through setting up and using a pipeline GitHub Action
- Configure ssh access to GitHub by [following this instruction](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent)

4. **Fork the course Repository into your GitHub Account**

Start by creating a fork of this repository under your own GitHub account. From there you can
experiment freely
- Go to [the course reporitory](https://github.com/sodalabsab/cicdcourse.git)
- Select "Fork" to create your own disconnected version of the course code repository
- Marke sure to uncheck "Copy the main branch only" there is a branch in there we will use later
- Name the repository "cicdcourse" click on "Create fork"

5. **Download the repository localy**
5. **Download the repository locally**
- Go to the newly created repo in your github account and click on the green "<>Code" button. Copy the SSH URL and open a comand shell on your computer. Paste in this command to create a local repository (connected to the github repository)
```bash
git clone git@github.com:<your-username>/cicdcourse.git
Expand All @@ -50,13 +53,17 @@ This course will guide you through setting up and using a pipeline GitHub Action
You sould see something like: `origin git@github.com:<your usernam>/cicdcourse.git (push)`

6. **Docker (requires local admin)**
In orders to setup a local development environemt - we are using docker desktop as execution plattform. If you are not able to install docker because of local admin rights, you can still take part of lab 2-4.
- Donload and install from: [https://www.docker.com/products/docker-desktop](https://www.docker.com/products/docker-desktop)
- Ensure Docker is running and verify installation:
```bash
docker info
```
You should see lots of information about the docker environment running on your machine.
7. **Open the code in VS Code**

VS code is a great universal IDE with many plugins that helps develop applications efficiently. For this course you will only need the bare minimum, but we encurage to explore and learn as much as possible. Most of the things in booth GitHub and Google cloud can be done directly from the IDE.

- Start VS Code and open the directory by selecting "Open folder..." from the File meny

### This is the end of the first setup session
Expand All @@ -70,21 +77,38 @@ Let´s build and setup the system locally.
```bash
docker info
```
This command will show lot´s of information about the docker environment. It there is no ERROR message, then everything is good.
This command will show lot´s of information about the docker environment. It there is no ERROR message, then everything is good. If there is, fix it

2. **Build and Test locally**
Booth the demo application and the hub are built with Spring Boot 3, Java 17, and can be compiled, tested and started locally with the included Gradle wrapper:

```
./gradlew build test
java -jar build/libs/demo-0.0.3-SNAPSHOT.jar # or whatever version we're at
```

Alternatively, you can build and run it in a container:

```
docker build -t demo .
docker run --rm -p 8080:8080 demo
# make sure it's alive
curl localhost:8080
```

3. **Build the system locally**
There is a way to build and start booth components locally in a dev environment using Docker desktop.

2. **Build the applications locally**
- Use the docker-compose.yml file at the root of the repository to build booth applications at once.
To start the system use this command:
```bash
docker-compose up --build
```
This will build booth docker images (using their individual Docker files) and start them in one container each in docker desktop. It will setup a small network so the applications can talk to eachother
This will build booth docker images (using their individual Docker files) and start them in one container each in docker desktop. It will setup a small network (cloud-run-network) so the applications can talk to eachother using its service names.

3. **Verify**
- Go to http://localhost:8080 and http://localhost:8081 respectively and verify that booth the client and the hub is running.
- Try to register, unregister and set a rating and see how it updates in the HUB UI.
- Spring boot comes with swagger to describe the API. Have a look at the API documentation on the default swagger adress http://localhost:8080/swagger-ui.html
- Using the API, try to register an additional participant by klicking "Try it out" in the swagger gui. Change the default name to someting else and execute a API call. Verify that there now are two participants in the HUB UI.


# Setup 2 - move to the cloud

Expand Down Expand Up @@ -113,7 +137,8 @@ The service account need to have several rights, including:
- Cloud Run Admin
- Cloud Run Service Agent
- Service Account Token Creator
Once it is created, open it and create an access key. Select "Keys" in the top menue and "Add key". Create a new Json key and download the generated file containing the key.

Once it is created, open it and create an access key. Select "Keys" in the top menue and "Add key". Create a new Json key and download the generated file containing the key.

## 2. (Optional) Google Cloud CLI (gcloud)

Expand All @@ -124,23 +149,23 @@ Once it is created, open it and create an access key. Select "Keys" in the top m
```bash
gcloud init
```
This will guide you through the process of selecting a project (if you have multiple) and setting up your gcloud configuration.
This will guide you through the process of selecting a project (if you have multiple) and setting up your gcloud configuration.

**Log in to your Google Account:**
- **Log in to your Google Account:**

```bash
gcloud auth login
```
This will open a browser window where you can authenticate with your Google Cloud account.
This will open a browser window where you can authenticate with your Google Cloud account.

Set the active project (Important): If you've created multiple projects, you need to tell gcloud which one to use:
Set the active project (Important): If you've created multiple projects, you need to tell gcloud which one to use:

```bash
gcloud config set project YOUR_PROJECT_ID
```
Replace YOUR_PROJECT_ID with the actual ID of your Google Cloud project. You can find the Project ID in the Google Cloud Console.
Replace YOUR_PROJECT_ID with the actual ID of your Google Cloud project. You can find the Project ID in the Google Cloud Console.

## GitHub actions Setup for Google Artifact and Cloud Run Deployment
## 3. GitHub actions Setup for Google Artifact and Cloud Run Deployment

### Configure GitHub Secrets

Expand All @@ -149,198 +174,72 @@ You need to configure the following GitHub Secrets in your repository for secure
#### Steps to Configure Secrets
Go to your GitHub account and the repository you have cloned from the course repo.
- Select settings->Secret and variables->Actions.
- Add a new repository secret called GCP_CREDENTIALS and paste the contect of the key-file intor the "Secret" field.
- Add a new repository secret called GCP_CREDENTIALS and paste the contect of the key-file intor the "Secret" field. (The key-file is the one you downloaded from google cloud when you created the service account above)
- Create a variable (it is in another tab than secrets) called PROJECT_ID and as value, find the ID of the GCP project. Ex: "cicdcourse-32242
You will find the ID in GCP if you click on the projekt box at the top of the GCP dashboard.

That´s it, now you are ready for Lab 2.

---

## Lab 2 - deploy to production
- Go to your GitHub repository and click on "Actions". Enable GitHub actions. This will scan the repository and add the workflow definitions under .github/workflows
## Lab 2 - Deploy to production
Now we are finally ready to start the pipeline and deploy to production. Here we only focus on the Demo client application. The Hub has been deployed centrally (one central hub for the entire class at adress: https://ci-cd-course-hub-aeyvkm6j4q-lz.a.run.app/)

- Go to your GitHub repository and click on "Actions". Enable GitHub actions. This will scan the repository and add the workflow definitions under .github/workflows <- This is default in GitHub actions
- Select "Build, test and deploy Demo application" and click on "Run workflow" to trigger a pipeline run manually.
Follow the progress by clicking on the newly started pipeline instance and expand the logs.
If there are any errors the pipeline will stop. Otherwise, this will result in two instances of the demo applation running in your GCP environment.
If there are any errors the pipeline will stop. Otherwise, this will result in two instances of the demo applation running in your GCP environment, one for stage (test) and one production.
- In the pipeline find the “Show Output” step
this reveals the URL for your personal dashboard - go to it and register your client
- Go to GCP dashboard and the logs of the production application to moke sure it has successfully registered with the hub.
this reveals the URL for your personal dashboard - go to it and register your client with the central Hub.
- Go to GCP dashboard and look at the logs of the production application to moke sure it has successfully registered with the hub.

That´s it for lab 2!

## Lab 3 - Setup a local runner
Follow the instructions on this page to download and setup a local runner on your comupter:
https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/adding-self-hosted-runners

- Open the workflow definition for the demo application ./github/workflows/demo-application.yml
You can do it locally in VS Code or directly in the GitHub UI.
- Change
```bash
jobs:
build-and-unittest:
runs-on: ubuntu-latest
```

to:

```bash
jobs:
build-and-unittest:
runs-on: self-hosted
```
Commit and push the change to GitHub:
If you have done it locally, commit and push the change to GitHub with this command:
```bash
git commit -m "Changed to self-hosted
git add .
git commit -m "Changed to self-hosted"
git push
```
- Go to github and manually trigger another pipeline run. You will se a log in the local runner when GitHub sends a job to it for execution.

## Lab 4 - API versions
Make sure the central hub is running on the correct version.
Create a pull request in github to merge the api-v2 branch to main. Go to "Pull requests" in the menue. Select "Create pull request" and select api-v2 in the right side (going in to main).

### Build and Test

The application itself is built with Spring Boot 3, Java 17, and can be compiled, tested and started
locally with the included Gradle wrapper:

```
./gradlew build test
java -jar build/libs/demo-0.0.3-SNAPSHOT.jar # or whatever version we're at
```

Alternatively, you can build and run it in a container:

```
docker build -t demo .
docker run --rm -p 8080:8080 demo
# make sure it's alive
curl localhost:8080
```

Once running, you will find ways to interact with the application at http://localhost:8080/.

</details>

----
- Go to github and manually trigger another pipeline run. (The pipeline will not trigger automatically unless you change something in the cloud-run-pipeline-demo directory and the workflow definition is in another directory)
- Open the log for the local runnar and when the pipeline is triggered, you will se a logentry when GitHub sends a job to it for execution.
This job contain the complete context to be able to execute the build.

<details style="background-color: #303030">
<summary><span style="font-size: 1.8em; font-weight: bold">1️⃣ First Exercise</span></summary>
That´s it for lab 3!

### Create a Fork

Start by creating a fork of this repository under your own GitHub account. From there you can
experiment freely - even deploy it to our cloud (once we give you the necessary credentials)!

![image](docs/img/0_00_create-new-fork.png)

----

Select your own GitHub account as the "Owner", and click the "Create fork" button.

![image](docs/img/0_01_fork-owner.png)

### Add GCP Credentials

In order to push and subsequently deploy the containerized service, we need to add the appropriate
Google Cloud credentials. Start by going into the project settings:

![image](docs/img/1_00_project-settings.png)

----

In the left-hand menu, under "Security", find "Secrets and variables". Click it, and then "Actions"
below:

![image](docs/img/1_01_actions-secrets-and-variables.png)

----

Here, add a "New repository secret":

![image](docs/img/1_02_new-repository-secret.png)

----

The `name` must be `GCP_CREDENTIALS`.

The `secret` will be provided by today's instructor. Copy and paste that json into the text field
and hit "Add secret":

![image](docs/img/1_03_add-secret.png)

### Enable GitHub Actions Workflow

The definition of the pipeline for this project is included as code in the project itself - GitHub
reads it from the `.github/workflows/` folder. However, it is not enabled by default. To enable it,
click "Actions" in the top project menu:

![image](docs/img/2_00_actions-menu.png)

Here, you will also be informed that workflows are disabled by default. Go ahead and enable them,
and we will continue the exercise by looking at the pipeline and its steps in more detail.

![image](docs/img/2_01_enable-workflows.png)

----

Lastly, to verify that everything is set up and configured correctly, let's run the "Initial Cloud
Run Deploy" action:

![image](docs/img/3_00_verify-pipeline.png)

----

![image](docs/img/3_01_run-workflow.png)

----

Within seconds, the running workflow should appear:

![image](docs/img/3_02_initial-cloud-run-deploy-workflow.png)

Click it, and you will see the workflow details - including the status of the individual steps:

![image](docs/img/3_03_workflow-details.png)

Once all steps are green, we're ready to move on!

This pipeline is meant to demonstrate a "basic" or "typical" setup where the service is built,
unit-tested, containerized, pushed to an artifact registry and then deployed to a staging
environment. End-to-end tests are then run and, if successful, the service is deployed to
production.

The `deploy-production` job has an `output` section which gives you the publicly accessible URL of
the deployed service.

![image](docs/img/4_get-production-deploy-output.png)

</details>

❗ Create a fork of this repository under your own GitHub account
❗ Deploy your service to our cloud, find and navigate to the service in your browser, and leave some
feedback for the first exercise

----

<details style="background-color: #303030">
<summary><span style="font-size: 1.8em; font-weight: bold">2️⃣ Second Exercise</span></summary>

### Teamwork

As we have seen, this service is part of a bigger whole. It talks to a "hub" service where you can
use it to leave feedback for the completed exercises. So far, the hub has been out of your control -
but now we're going to start collaborating more and adding functionality to both services in order
to increase the value for the end-users.
## Lab 4 - API versions
There is a new feature developed in a brach called api-v2-new. This feature is split between the demo application and the hub. Our job is to release the compete feature in production, with zero downtime and no conflicts. There is a new API (v2) in the Hub and some new gui in the demo application.

In this exercise, we will add some functionality to both services - allowing additional information
to be sent to the hub, and visualized on the hub dashboard.
The way to make this happen is to deploy a new version of the Hub that service booth v1 and v2 of the api. This way, booth old and new versions of the client will work.
The course instructor will deploy a new version of the Hub.
Make sure the central hub is running on the correct version.
https://ci-cd-course-hub-aeyvkm6j4q-lz.a.run.app/swagger-ui.html <- you should be able to se v2 of the API here.

The hub only accepts a number as feedback for an exercise, a "score" or "grade" if you will. And it
interprets it as "score for the first exercise". Or, the only exercise. But as we have now reached
exercise number two, perhaps you are starting to see the dilemma. We will have to update the API of
the hub to accept (and handle) more information - but we must do it in a controlled and structured
way so that we don't break the service for clients that haven't been updated yet.
Your job is to:
- Create a pull request in github to merge the api-v2-new branch to main. Go to "Pull requests" in the menu. Select "Create pull request" and select api-v2-new in the right side (going in to main).
- Merge the pull request into main.
- This will trigger the pipeline to rebuild and deploy a new version of the demo application
- Go to the production application and verify that the new version is running there. Submit a "Happiness score"

</details>

❗ Create a PR for the "exercise 2" branch to main
❗ Sync rollout with the instructor, making sure the backend hub is upgraded first
❗ After the hub is upgraded, double-check that your service still works before deploying a new
version
❗ Merge the "exercise 2" PR, wait for it to deploy, and leave some feedback for the second exercise!