DevOps Seminar Project Setup

This README provides instructions for setting up and running the project in a production environment using Docker and Docker Compose.

Project Structure

The project consists of multiple services:

• load-balancer: Traefik v3.3 for routing and load balancing
• service-one: Custom service with its own Dockerfile
• service-two: Custom service with its own Dockerfile
• db-one: MySQL 8.4 database for service-one
• db-two: MySQL 8.4 database for service-two

Prerequisites

• Docker and Docker Compose installed
• External volumes created for databases
• Network configuration that supports the defined subnets

Setup Instructions

1. Create External Volumes

Before starting the services, create the external volumes required by the databases:

docker volume create db-one
docker volume create db-two

2. Configure Environment Files

Create environment files for each service in the infra/envs/ directory:

• .env.service-one - Environment variables for service-one
• .env.service-two - Environment variables for service-two

3. Create Dockerfiles

Each service requires a production Dockerfile. Below are examples for both services.

Example prod.Dockerfile for service-one

Create or update the file at infra/services/service-one/prod.Dockerfile:

# Build stage
FROM node:18-alpine as builder

# IMPORTANT: The WORKDIR must be set relative to the CONTEXT
# Context in compose.prod.yml is set to "./.." (parent directory of infra)
WORKDIR /app

# Copy package files from the context root (project root)
COPY package*.json ./

# Install dependencies
RUN npm ci

# Copy source code from the context root (project root)
# This is why context setting is important - it determines what "COPY . ." means
COPY . .

# Build the application
RUN npm run build:service-one

# Production stage
FROM node:18-alpine

WORKDIR /app

# Copy only necessary files from builder stage
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package*.json ./
COPY --from=builder /app/node_modules ./node_modules

# Set environment to production
ENV NODE_ENV production

# Expose the port your service runs on
EXPOSE 3000

# The command will be specified in compose.prod.yml
# Example: node dist/apps/service-one/main.js

Example prod.Dockerfile for service-two

Create or update the file at infra/services/service-two/prod.Dockerfile:

# Build stage
FROM node:18-alpine as builder

# IMPORTANT: The WORKDIR must be set relative to the CONTEXT
# Context in compose.prod.yml is set to "./.." (parent directory of infra)
WORKDIR /app

# Copy package files from the context root (project root)
COPY package*.json ./

# Install dependencies
RUN npm ci

# Copy source code from the context root (project root)
# This is why context setting is important - it determines what "COPY . ." means
COPY . .

# Build the application
RUN npm run build:service-two

# Production stage
FROM node:18-alpine

WORKDIR /app

# Copy only necessary files from builder stage
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package*.json ./
COPY --from=builder /app/node_modules ./node_modules

# Set environment to production
ENV NODE_ENV production

# Expose the port your service runs on
EXPOSE 3000

# The command will be specified in compose.prod.yml
# Example: node dist/apps/service-two/main.js

4. Update Service Commands

In compose.prod.yml, replace the placeholder commands with actual commands to run your services:

service-one:
  # ...
  command: node dist/apps/service-one/main.js

service-two:
  # ...
  command: node dist/apps/service-two/main.js

Running the Services

To start all services in production mode:

cd devops-seminar
docker-compose -f infra/compose.prod.yml up -d

To check the status of the services:

docker-compose -f infra/compose.prod.yml ps

To view logs:

# All services
docker-compose -f infra/compose.prod.yml logs

# Specific service
docker-compose -f infra/compose.prod.yml logs service-one

Scaling Services

You can scale services to run multiple instances for better performance and reliability:

# Syntax: docker-compose -f [compose-file] up -d --scale [service-name]=[number-of-instances]

# Scale service-one to 3 instances
docker-compose -f infra/compose.prod.yml up -d --scale service-one=3

# Scale service-two to 2 instances
docker-compose -f infra/compose.prod.yml up -d --scale service-two=2

# Scale multiple services at once
docker-compose -f infra/compose.prod.yml up -d --scale service-one=3 --scale service-two=2

To verify the scaled instances:

docker-compose -f infra/compose.prod.yml ps service-one

Important Notes on Scaling:

1. Ensure that port bindings are compatible with scaling. If you have fixed port mappings like 3000:3000, you should change them to 3000 (only exposing the internal port) when scaling.

2. Resource limits defined in the deploy section will apply to each instance:

   deploy:
     resources:
       limits:
         cpus: 0.5         # Each instance will be limited to 0.5 CPUs
         memory: 512M      # Each instance will be limited to 512MB RAM

3. To scale down, simply set a lower number of instances in the scale command.

Accessing Services

• Traefik Dashboard: http://localhost:8080
• service-one: http://local.domain.com/api/service-one
• service-two: http://local.domain.com/api/service-two

Setting Up Host File Configuration

To access services using local.domain.com, you'll need to modify your hosts file to map this domain to your localhost IP address (127.0.0.1). Here's how to do it on different operating systems:

Linux

# Open hosts file with sudo privileges
sudo nano /etc/hosts

# Add the following line
127.0.0.1 local.domain.com

# Save and exit (Ctrl+X, then Y, then Enter in nano)

macOS

# Open hosts file with sudo privileges
sudo nano /etc/hosts

# Add the following line
127.0.0.1 local.domain.com

# Save and exit (Ctrl+X, then Y, then Enter in nano)
# Flush DNS cache to apply changes
sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder

Windows

1. Open Notepad as Administrator (right-click Notepad and select "Run as administrator")
2. Open the file: C:\Windows\System32\drivers\etc\hosts
3. Add the following line:

   127.0.0.1 local.domain.com
   

4. Save the file
5. Flush DNS cache (open Command Prompt as administrator):

   ipconfig /flushdns
   

Note: Any time you change your dynamic service names and Traefik Host rules, remember to update your hosts file accordingly.

Important Notes About Docker Context

Understanding Context in Docker Builds

The context setting in compose.prod.yml is critical for Dockerfile operations:

build:
  context: ./..  # Points to the parent directory of infra
  dockerfile: ./infra/services/service-one/prod.Dockerfile

This context setting means:

1. All COPY commands in the Dockerfile are relative to the project root (parent of infra)
2. If your Dockerfile uses COPY . ., it will copy everything from the project root
3. You must structure your Dockerfiles with this context in mind

CRITICAL: COPY Command File Path Redirection

⚠️ EXTREME CAUTION WITH COPY COMMANDS ⚠️

When using COPY commands in your Dockerfile, you must be extremely careful about path redirections. Since your Dockerfiles are located in infra/services/service-one/ and infra/services/service-two/ but the context is set to ../.. (parent of infra), COPY paths need to be relative to the project root, not the Dockerfile location:

# WRONG: This fails because it's looking for files relative to the Dockerfile location
COPY ./package.json ./
COPY ../../../package.json ./  # Also wrong - trying to go outside context

# CORRECT: These use paths relative to the context (project root)
COPY package.json ./  # Directly from project root
COPY src/config/settings.json ./config/  # From src/config in project root

# EXAMPLE: For a file structure like this:
# /project-root/
#   package.json
#   tsconfig.json
#   /src/
#     /service-one/
#       index.ts
#   /infra/
#     /services/
#       /service-one/
#         prod.Dockerfile

# In infra/services/service-one/prod.Dockerfile, to copy files from project root:
COPY package.json ./  # Correct
COPY tsconfig.json ./  # Correct
COPY src/service-one/index.ts ./src/  # Correct

Remember: Docker cannot access files outside the build context. All COPY operations must refer to files within the context directory. The path is always relative to the context, which in this case is the project root (parent directory of infra).

Dynamic Service Names and Traefik Configuration

The service and database names in the compose.prod.yml file (service-one, service-two, db-one, db-two) can be customized to match your specific project requirements:

# These can be renamed to match your specific services
service-custom-name:
  image: custom-service-name
  # other configuration

db-custom-name:
  container_name: db-custom-name
  # other configuration

⚠️ CRITICAL: Updating Traefik Labels for Dynamic Services ⚠️

When using dynamic service names, it's crucial to update the Traefik routing labels accordingly:

service-custom-name:
  # ...
  labels:
    - 'traefik.enable=true'
    - 'traefik.http.routers.service-custom-name.rule=Host(`your-domain.com`) && PathPrefix(`/api/custom-path`)'
    - 'traefik.http.services.service-custom-name.service=service-custom-name'

Make sure to update:

• The router name (after traefik.http.routers.)
• The Host value to match your domain
• The PathPrefix to match your API route structure
• The service name (after traefik.http.services. and .service=)

When renaming services, remember to update the corresponding:

• Dockerfile paths
• Environment files
• Network configurations
• Traefik routing labels (as shown above)
• Volume references

Common Mistakes to Avoid

• Wrong paths in COPY commands: Always consider that the source path is relative to the context, not the Dockerfile location
• Incorrect WORKDIR: Set it according to your application structure
• Forgetting about .dockerignore: Create a proper .dockerignore file to avoid copying unnecessary files
• Inconsistent service naming: If you rename a service, update all references to it throughout the compose file

Troubleshooting

Database Connection Issues

If services can't connect to databases:

1. Check if volumes are created: docker volume ls
2. Verify network configuration
3. Check database health with: docker-compose -f infra/compose.prod.yml exec db-one mysqladmin ping -u root --password=12345678

Traefik Routing Issues

1. Check Traefik dashboard at http://localhost:8080
2. Verify label configuration in compose.prod.yml
3. Check if services are registered properly in Traefik