Deployment Test Scripts

This directory contains scripts and configuration files for deploying and testing Docker builds in a cloud VM environment.

Files

1. cloud-init.yaml

Cloud-init configuration file for VM initialization.

Features:

• Creates user kevin with password itsoktouseai
• Grants passwordless sudo access
• Installs and configures Docker
• Updates OS packages
• Installs Git and other development tools

Usage: Use this file when creating a new VM with any cloud provider (AWS, GCP, Azure, DigitalOcean, etc.)

2. build-docker-images.sh

Main build script that clones the frontendx repository and builds all Docker images with comprehensive disk usage monitoring.

Features:

• Clones https://github.com/dictyBase/frontendx
• Checks out the develop branch
• Builds 4 Docker images (frontpage, genomepage, publication, stock-center)
• Logs disk usage before and after each build
• Saves detailed logs to ~/frontendx-builds/docker-build-logs/

Usage:

# On the VM
./build-docker-images.sh

# With custom build state
BUILD_STATE=development ./build-docker-images.sh

3. BUILD_SCRIPT_README.md

Comprehensive documentation for the build-docker-images.sh script, including:

• Detailed feature descriptions
• Usage examples
• Troubleshooting guide
• Deployment instructions
• Performance tips

4. deploy-to-vm.sh

Helper script to easily deploy the build script to your VM from your local machine.

Usage:

# From your local machine
./deploy-to-vm.sh <vm-ip-address>

# Example
./deploy-to-vm.sh 192.168.1.100

This will:

• Copy build-docker-images.sh to the VM
• Make it executable
• Display commands to run it

5. justfile

Automation recipes using just command runner for Multipass workflows.

Usage:

# Complete quickstart (launch, wait, deploy, build)
just quickstart

# Individual steps
just launch      # Launch VM
just wait-init   # Wait for cloud-init
just deploy      # Deploy build script
just run-build   # Run the build

# Management
just info        # VM information
just list        # List all VMs
just delete      # Delete VM

# Logs
just logs-list    # List all log files
just logs-summary # View summary log
just logs-app frontpage  # View specific app log (frontpage, genomepage, publication, stock-center)
just logs-tail    # Tail logs in real-time

Install just:

# macOS
brew install just

# Linux
cargo install just  # or use your package manager

# Windows
cargo install just  # or download from releases

Using Multipass (Local Testing)

Multipass allows you to test the cloud-init configuration locally before deploying to a cloud provider.

Install Multipass

# macOS
brew install multipass

# Linux
sudo snap install multipass

# Windows
# Download from https://multipass.run/install

Launch VM with Multipass

Basic Launch:

multipass launch --name frontendx-build --cloud-init cloud-init.yaml

Recommended Launch (with custom specs):

multipass launch --name frontendx-build \
  --cpus 4 \
  --memory 8G \
  --disk 30G \
  --cloud-init cloud-init.yaml

Recommended specifications:

• CPUs: 4 cores (faster builds)
• Memory: 8GB (Docker builds are memory-intensive)
• Disk: 30GB (space for images and build cache)

Wait for Initialization

Cloud-init will take 2-5 minutes to:

• Create user kevin
• Install Docker
• Update packages
• Install Git

Check cloud-init status:

multipass exec frontendx-build -- cloud-init status --wait

Get VM Information

# List all VMs
multipass list

# View detailed VM info
multipass info frontendx-build

Access the VM

Option A: Shell as ubuntu user, then switch to kevin

multipass shell frontendx-build
# Inside VM
su - kevin
# Password: itsoktouseai

Option B: SSH directly as kevin

# Get IP address
multipass list

# SSH as kevin
ssh kevin@<vm-ip>
# Password: itsoktouseai

Deploy and Run Build Script

Using the deploy helper:

# Get VM IP from multipass list
./deploy-to-vm.sh <vm-ip>

Or transfer directly with Multipass:

multipass transfer build-docker-images.sh frontendx-build:/tmp/
multipass exec frontendx-build -- sudo mv /tmp/build-docker-images.sh /home/kevin/
multipass exec frontendx-build -- sudo chown kevin:kevin /home/kevin/build-docker-images.sh
multipass exec frontendx-build -- sudo chmod +x /home/kevin/build-docker-images.sh

# Then run
multipass exec frontendx-build -- sudo -u kevin -i ./build-docker-images.sh

Complete Multipass Workflow

Option A: Using just (recommended)

# One command does everything
just quickstart

Option B: Manual steps

# 1. Launch VM
multipass launch --name frontendx-build \
  --cpus 4 --memory 8G --disk 30G \
  --cloud-init cloud-init.yaml

# 2. Wait for cloud-init (2-5 minutes)
multipass exec frontendx-build -- cloud-init status --wait

# 3. Deploy build script
multipass transfer build-docker-images.sh frontendx-build:/tmp/
multipass exec frontendx-build -- sudo mv /tmp/build-docker-images.sh /home/kevin/
multipass exec frontendx-build -- sudo chown kevin:kevin /home/kevin/build-docker-images.sh
multipass exec frontendx-build -- sudo chmod +x /home/kevin/build-docker-images.sh

# 4. Run the build
multipass exec frontendx-build -- sudo -u kevin -i ./build-docker-images.sh

Useful Multipass Commands

# List all VMs
multipass list

# Start VM
multipass start frontendx-build

# Stop VM
multipass stop frontendx-build

# Delete VM
multipass delete frontendx-build
multipass purge

# View VM info
multipass info frontendx-build

# Mount local directory into VM
multipass mount /local/path frontendx-build:/vm/path

# Transfer files
multipass transfer file.txt frontendx-build:/home/ubuntu/

# Execute command in VM
multipass exec frontendx-build -- docker ps

# View cloud-init logs
multipass exec frontendx-build -- sudo cat /var/log/cloud-init-output.log

Verify Installation

# Check Docker installation
multipass exec frontendx-build -- docker --version

# Verify kevin user exists
multipass exec frontendx-build -- id kevin

# Check if kevin can use Docker
multipass exec frontendx-build -- sudo -u kevin docker ps

Quick Start Guide (Cloud Providers)

Step 1: Create VM with Cloud-Init

1. Create a new VM with your cloud provider
2. Use cloud-init.yaml as the user data/cloud-init configuration
3. Wait for VM to initialize (Docker installation may take a few minutes)

Step 2: Deploy Build Script

# Option A: Use the helper script
./deploy-to-vm.sh <your-vm-ip>

# Option B: Manual deployment
scp build-docker-images.sh kevin@<your-vm-ip>:~/

Step 3: Run the Build

# SSH into the VM
ssh kevin@<your-vm-ip>

# Run the build script
./build-docker-images.sh

Step 4: Review Results

Logs will be saved in ~/frontendx-builds/docker-build-logs/ on the VM.

# View summary log
cat ~/frontendx-builds/docker-build-logs/build_summary_*.log

# View specific app log
cat ~/frontendx-builds/docker-build-logs/dictybase-frontpage_*.log

System Requirements

VM Requirements

• OS: Ubuntu 20.04+ or Debian 10+ (recommended)
• CPU: 2+ cores (4+ recommended for faster builds)
• RAM: 4GB minimum (8GB+ recommended)
• Disk: 20GB+ free space
• Network: Internet connection for cloning repository and downloading dependencies

Local Machine Requirements

• SSH client installed
• SCP available (for deploying scripts)

Applications Built

The script builds Docker images for these frontendx applications:

1. dictybase-frontpage - Main front page application
2. dictybase-genomepage - Genome page application
3. dictybase-publication - Publication management application
4. dictybase-stock-center - Stock center application

Each image is tagged with:

• latest tag
• Timestamp tag (e.g., 20251218_143022)

Monitoring

Real-time Monitoring

While builds are running, you can monitor in a separate SSH session:

# Watch disk usage
watch -n 5 'df -h && echo "" && docker system df'

# Monitor Docker
watch -n 5 'docker images'

# View system resources
htop

Log Analysis

Each build generates detailed logs including:

• Disk usage before build
• Complete build output
• Disk usage after build
• Build duration
• Final image size

Troubleshooting

Multipass Issues

Cloud-init not completing:

# Check cloud-init status
multipass exec frontendx-build -- cloud-init status

# View cloud-init logs
multipass exec frontendx-build -- sudo cat /var/log/cloud-init-output.log

# Check for errors
multipass exec frontendx-build -- sudo cat /var/log/cloud-init.log

VM won't start:

# Check VM status
multipass list

# View VM logs
multipass info frontendx-build

# Restart VM
multipass restart frontendx-build

Can't connect to VM:

# Verify VM is running
multipass list

# Get VM IP
multipass info frontendx-build

# Test connectivity
ping <vm-ip>

VM Not Accessible (Cloud Providers)

• Verify security groups/firewall rules allow SSH (port 22)
• Check VM is running
• Verify IP address is correct

Docker Not Installed

• Check cloud-init logs: sudo cat /var/log/cloud-init-output.log
• Verify cloud-init completed: cloud-init status
• Manually install Docker: curl -fsSL https://get.docker.com | sh

Build Failures

• Check disk space: df -h
• Review application logs in ~/frontendx-builds/docker-build-logs/
• Verify internet connectivity: ping github.com
• Check Docker is running: docker ps

Out of Disk Space

# Clean up Docker
docker system prune -a -f

# Remove old logs
rm -rf ~/frontendx-builds/docker-build-logs/old_*

# For Multipass: Increase disk size (requires VM recreation)
multipass delete frontendx-build
multipass purge
multipass launch --name frontendx-build --cpus 4 --memory 8G --disk 50G --cloud-init cloud-init.yaml

Security Notes

Password Warning: The cloud-init configuration uses a plaintext password (itsoktouseai) for demo purposes. For production use:

• Use SSH keys instead of password authentication
• Set ssh_pwauth: false in cloud-init
• Use a strong, unique password
• Consider using secrets management

Documentation Links

• Cloud-init Documentation
• Docker Documentation
• Multipass Documentation
• just Command Runner
• frontendx Repository
• Build Script Detailed Docs

Support

For issues or questions about:

• frontendx project: Check the GitHub repository
• Cloud-init: See official documentation
• Docker: Visit Docker docs
• Multipass: Check Multipass documentation or GitHub issues