Skip to content

kokland/lekse-analyse

Repository files navigation

Lekse Sammendrag

Note: This project was 100% vibe-coded using OpenCode - an AI-powered coding assistant. The entire codebase, architecture, documentation, and deployment setup were created through natural language conversation with AI.

A scheduled .NET 10 worker service that monitors a Google Docs document for changes and sends email notifications with PDF attachments when updates are detected.

Features

  • Automated Monitoring: Checks document every hour (configurable)
  • Change Detection: SHA256 content hashing to detect actual changes
  • Email Notifications: Sends emails with PDF attachments when document is updated
  • SQLite Database: Tracks document history and changes
  • .NET Aspire: Built-in observability with OpenTelemetry and health checks
  • Structured Logging: Comprehensive logging with Serilog
  • Docker Support: Production-ready containerization with Alpine Linux
  • Health Checks: HTTP endpoint for monitoring service health
  • Security: Non-root user, secure credential management

Quick Start

Prerequisites

  • .NET 10 SDK
  • Docker (optional, for containerized deployment)
  • SMTP email credentials (Gmail, Outlook, SendGrid, etc.)
  • Publicly accessible Google Docs URL

1. Clone and Configure

# Clone the repository
git clone <repository-url>
cd lekse-sammendrag

# Copy environment template
cp .env.example .env

# Edit .env with your configuration
nano .env  # or use your preferred editor

2. Configuration

For testing with MailHog - No configuration needed! Skip to step 3.

For production with real SMTP - Edit .env file:

# Create .env from template
cp .env.example .env

# Edit with your settings
nano .env

Example .env for Gmail:

# SMTP Configuration
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USERNAME=your-email@gmail.com
SMTP_PASSWORD=your-16-digit-app-password
SMTP_ENABLE_SSL=true
FROM_EMAIL=your-email@gmail.com
FROM_NAME=Lekse Sammendrag
TO_EMAILS=recipient1@example.com,recipient2@example.com

# Document Configuration
GOOGLE_DOCS_URL=https://docs.google.com/document/d/YOUR_DOCUMENT_ID/edit
CHECK_INTERVAL_MINUTES=60

Important: For Gmail, you need to create an App Password if 2FA is enabled.

3. Run with Docker (Recommended)

Option A: Testing with MailHog (No Real SMTP Required)

The docker-compose setup includes MailHog, a fake SMTP server perfect for testing without sending real emails.

# Start everything (worker + MailHog)
docker-compose up -d

# View worker logs
docker-compose logs -f lekse-sammendrag-worker

# View MailHog web UI to see captured emails
open http://localhost:8025

# Check health status
curl http://localhost:8080/health

# Stop everything
docker-compose down

MailHog Benefits:

  • ✅ No real SMTP credentials needed
  • ✅ View all sent emails in web UI at http://localhost:8025
  • ✅ Perfect for development and testing
  • ✅ Automatically configured when no .env file exists

Option B: Production with Real SMTP

There are three ways to pass environment variables to docker-compose:

Method 1: .env file (Recommended)

# 1. Create .env file with real SMTP credentials
cp .env.example .env
nano .env  # Add your Gmail/Outlook credentials

# 2. Start the service (automatically loads .env)
docker-compose up -d

# 3. View logs
docker-compose logs -f

# 4. Stop the service
docker-compose down

Method 2: Inline environment variables

# Pass variables directly on command line
SMTP_HOST=smtp.gmail.com \
SMTP_PORT=587 \
SMTP_USERNAME=your-email@gmail.com \
SMTP_PASSWORD=your-app-password \
FROM_EMAIL=your-email@gmail.com \
TO_EMAILS=recipient@example.com \
GOOGLE_DOCS_URL="https://docs.google.com/document/d/YOUR_ID/edit" \
docker-compose up -d

Method 3: Export environment variables

# Export variables in your shell session
export SMTP_HOST=smtp.gmail.com
export SMTP_PORT=587
export SMTP_USERNAME=your-email@gmail.com
export SMTP_PASSWORD=your-app-password
export FROM_EMAIL=your-email@gmail.com
export TO_EMAILS=recipient@example.com
export GOOGLE_DOCS_URL="https://docs.google.com/document/d/YOUR_ID/edit"

# Then start normally
docker-compose up -d

Method 4: Custom .env file

# Use a different env file name
docker-compose --env-file .env.production up -d

Production without MailHog:

# Use production override to disable MailHog
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

4. Run Locally (Development)

Option A: Run with .NET Aspire Dashboard (Recommended for Development)

The easiest way to run locally with full monitoring and zero configuration:

# Run with Aspire (includes MailHog fake SMTP + monitoring dashboard)
dotnet run --project src/LekseSammendrag.AppHost

What happens when you run this:

  1. ✅ Aspire Dashboard opens automatically in your browser
  2. ✅ MailHog container starts automatically (fake SMTP server)
  3. ✅ Worker service starts with MailHog pre-configured
  4. ✅ All services are networked together automatically
  5. ✅ Real-time monitoring, logs, and traces available in dashboard

Access Points:

Aspire Dashboard Features:

  • 📊 Resources View: See all running services, containers, and their status
  • 📝 Logs: Live log streaming from all services with filtering and search
  • 📈 Metrics: Real-time performance metrics and health status
  • 🔍 Traces: Distributed tracing across service boundaries (OpenTelemetry)
  • 🏥 Health Checks: Monitor worker health status in real-time
  • 🌐 Environment Variables: View all configuration injected into services

Why Use Aspire for Development:

  • Zero Configuration: No need to set up SMTP credentials or environment variables
  • Instant Feedback: See logs and traces in real-time without digging through files
  • MailHog Integration: All emails captured and viewable in web UI (http://localhost:8025)
  • Production-Like: Mimics containerized production environment locally
  • Easy Debugging: Quickly identify issues with structured logs and traces

MailHog Benefits:

  • View all sent emails without sending real emails
  • Test email formatting, attachments, and recipients
  • No SMTP credentials or external services needed
  • Perfect for testing without spamming real inboxes

Option B: Run Worker Directly

If you want to run the Worker without Aspire:

# 1. Configure via user-secrets (recommended)
./scripts/sync-env-to-secrets.sh  # Sync from .env file
# OR set secrets manually (see "Local Development Configuration" section)

# 2. Run the worker
dotnet run --project src/LekseSammendrag.Worker

# Worker will be available at:
# - Health check: http://localhost:5000/health
# - Alive check: http://localhost:5000/alive

Option C: Using Environment Variables

# Set environment variables
export Email__SmtpHost="smtp.gmail.com"
export Email__SmtpPort="587"
export Email__Username="your-email@gmail.com"
export Email__Password="your-app-password"
export Email__FromEmail="your-email@gmail.com"
export Email__Recipients="recipient@example.com"

# Run the worker
dotnet run --project src/LekseSammendrag.Worker

Architecture

Project Structure

lekse-sammendrag/
├── src/
│   ├── LekseSammendrag.Core/          # Domain logic and services
│   │   ├── Configuration/             # Configuration models
│   │   ├── Data/                      # EF Core entities and DbContext
│   │   └── Services/                  # Business logic services
│   ├── LekseSammendrag.Worker/        # Background worker service
│   ├── LekseSammendrag.AppHost/       # Aspire orchestration
│   └── LekseSammendrag.ServiceDefaults/ # Aspire shared configuration
├── docker/
│   └── Dockerfile                     # Multi-stage Docker build
├── docker-compose.yml                 # Docker Compose configuration
└── .env.example                       # Environment template

Components

Core Services

  • DocumentDownloader: Downloads Google Docs as PDF
  • ChangeDetectionService: SHA256 hashing and database comparison
  • EmailService: SMTP email with MailKit, supports attachments
  • ApplicationDbContext: SQLite database with EF Core

Worker Service

  • DocumentCheckWorker: Background service with scheduled execution
  • DocumentCheckWorkerHealthCheck: Health monitoring endpoint
  • Runs every configured interval (default: 60 minutes)
  • Uses scoped services for database operations

Aspire Integration

  • AppHost: Orchestration and monitoring dashboard
  • ServiceDefaults: OpenTelemetry, health checks, service discovery
  • Real-time metrics and distributed tracing

How It Works

  1. Scheduled Check: Worker wakes up every hour (configurable)
  2. Download: Fetches Google Docs as PDF via export URL
  3. Hash Calculation: Computes SHA256 hash of PDF content
  4. Change Detection: Compares with latest hash in database
  5. Email Notification: If changed (or first run), sends email with PDF attachment
  6. Database Update: Stores new hash and timestamp in SQLite
  7. Logging: All operations logged with correlation IDs

Email Notification Format

  • Subject: Updated Document: Lekse Sammendrag
  • Body: Plain text with check timestamp and change information
  • Attachment: lekse-sammendrag-YYYY-MM-DD-HH-MM.pdf
  • Recipients: Multiple recipients supported (comma-separated)

Configuration Reference

Environment Variables

Variable Description Example Required
SMTP_HOST SMTP server hostname smtp.gmail.com Yes
SMTP_PORT SMTP server port 587 Yes
SMTP_USERNAME SMTP username user@gmail.com Yes
SMTP_PASSWORD SMTP password/app password abcd1234efgh5678 Yes
SMTP_ENABLE_SSL Enable SSL/TLS true No (default: true)
FROM_EMAIL Sender email address user@gmail.com Yes
FROM_NAME Sender display name Lekse Sammendrag Yes
TO_EMAILS Recipients (comma-separated) user1@test.com,user2@test.com Yes
GOOGLE_DOCS_URL Google Docs document URL https://docs.google.com/... Yes
CHECK_INTERVAL_MINUTES Check frequency in minutes 60 No (default: 60)

Configuration Methods & Precedence

The application loads configuration in this order (later sources override earlier):

  1. appsettings.json (default values)
  2. appsettings.{Environment}.json (environment-specific)
  3. Environment variables (highest priority)
  4. Docker .env file → converted to environment variables

Docker-specific notes:

  • .env file is automatically loaded by docker-compose
  • Variables in .env use format: SMTP_HOST=value
  • Docker passes these to the container as environment variables
  • Container converts them using format: Email__SmtpHost

Examples:

# Docker environment variable format (in .env file)
SMTP_HOST=smtp.gmail.com
FROM_EMAIL=user@gmail.com

# These become container environment variables
Email__SmtpHost=smtp.gmail.com
Email__FromEmail=user@gmail.com

# Which map to appsettings.json structure
{
  "Email": {
    "SmtpHost": "smtp.gmail.com",
    "FromEmail": "user@gmail.com"
  }
}

appsettings.json (Local Development)

If running locally without Docker, you can use appsettings.json:

{
  "Email": {
    "SmtpHost": "smtp.gmail.com",
    "SmtpPort": 587,
    "SmtpUsername": "your-email@gmail.com",
    "SmtpPassword": "your-app-password",
    "FromEmail": "your-email@gmail.com",
    "FromName": "Lekse Sammendrag",
    "ToEmails": "recipient@example.com",
    "EnableSsl": true
  },
  "Document": {
    "GoogleDocsUrl": "https://docs.google.com/document/d/YOUR_ID/edit",
    "CheckIntervalMinutes": 60,
    "DatabasePath": "data/lekse-sammendrag.db"
  }
}

Email Provider Configuration

MailHog (Testing/Development)

Recommended for testing - No configuration needed when using docker-compose!

# Leave these unset or use these values for MailHog
SMTP_HOST=mailhog
SMTP_PORT=1025
SMTP_ENABLE_SSL=false
SMTP_USERNAME=test
SMTP_PASSWORD=test
FROM_EMAIL=noreply@lekse-sammendrag.local
TO_EMAILS=test@example.com

Features:

  • Web UI: http://localhost:8025
  • View all captured emails in real-time
  • No emails actually sent (safe for testing)
  • Included in docker-compose setup

Gmail

SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_ENABLE_SSL=true
SMTP_USERNAME=your-email@gmail.com
SMTP_PASSWORD=your-16-digit-app-password

Setup: Enable 2FA and create App Password at https://myaccount.google.com/apppasswords

Outlook / Office 365

SMTP_HOST=smtp.office365.com
SMTP_PORT=587
SMTP_ENABLE_SSL=true
SMTP_USERNAME=your-email@outlook.com
SMTP_PASSWORD=your-password

SendGrid

SMTP_HOST=smtp.sendgrid.net
SMTP_PORT=587
SMTP_ENABLE_SSL=true
SMTP_USERNAME=apikey
SMTP_PASSWORD=your-sendgrid-api-key

Mailgun

SMTP_HOST=smtp.mailgun.org
SMTP_PORT=587
SMTP_ENABLE_SSL=true
SMTP_USERNAME=postmaster@your-domain.mailgun.org
SMTP_PASSWORD=your-mailgun-smtp-password

Monitoring & Health Checks

Health Check Endpoint

# Check service health
curl http://localhost:8080/health

# Response (healthy)
{
  "status": "Healthy",
  "totalDuration": "00:00:00.0123456"
}

# Response (degraded after 3 consecutive failures)
{
  "status": "Degraded",
  "totalDuration": "00:00:00.0123456"
}

.NET Aspire Dashboard

Run with Aspire for comprehensive real-time monitoring and development experience:

dotnet run --project src/LekseSammendrag.AppHost

Dashboard automatically opens in your browser (typically http://localhost:15888)

Key Features:

📊 Resources

  • View all services, containers, and executables
  • Real-time status indicators (Running, Stopped, Degraded)
  • Resource endpoints and network configuration
  • Environment variables inspection

📝 Structured Logs

  • Live log streaming from all services
  • Filter by severity level, service, or search terms
  • Correlation IDs for tracing requests across services
  • Automatic log aggregation from Worker and MailHog

📈 Metrics & Telemetry

  • HTTP request rates and response times
  • Health check status over time
  • Custom application metrics (OpenTelemetry)
  • Resource utilization (CPU, memory)

🔍 Distributed Tracing

  • End-to-end request tracing across services
  • Visualize service dependencies
  • Identify performance bottlenecks
  • Trace document checks from download → detection → email

🏥 Health Checks

  • Real-time health status of Worker service
  • Health check history and trends
  • Configurable health check intervals
  • Automatic degraded status on consecutive failures

🐳 Container Management

  • Automatic MailHog container lifecycle
  • Container networking and port mapping
  • View container logs and status
  • Easy restart and cleanup

Development Workflow:

  1. Start Aspire: dotnet run --project src/LekseSammendrag.AppHost
  2. Dashboard opens automatically, showing all resources
  3. Worker starts and connects to MailHog container
  4. View logs in real-time to see document checks
  5. When email is sent, view it instantly in MailHog web UI (http://localhost:8025)
  6. Check traces to see full execution flow
  7. Monitor health checks for worker status

Logs

Logs are written to:

  • Console: Structured output for Docker/systemd
  • File: logs/lekse-sammendrag-YYYY-MM-DD.log (30-day retention)

Log levels:

  • Information: Scheduled checks, email sent, changes detected
  • Warning: SMTP/download errors, retryable failures
  • Error: Unexpected errors, service failures

Example log entry:

[2026-02-13 14:30:00 INF] [abc123] Starting document check
[2026-02-13 14:30:02 INF] [abc123] Document changed, sending notification
[2026-02-13 14:30:05 INF] [abc123] Email sent successfully to 2 recipients

Database

Schema

DocumentHistory Table:

  • Id (INTEGER PRIMARY KEY)
  • CheckedAt (TEXT, UTC timestamp)
  • ContentHash (TEXT, SHA256 hex string)
  • WasChanged (INTEGER, boolean)
  • EmailSent (INTEGER, boolean)

Location

  • Docker: /app/data/lekse-sammendrag.db
  • Local: data/lekse-sammendrag.db

Backup

# Backup database
cp data/lekse-sammendrag.db data/backup-$(date +%Y%m%d).db

# Or with Docker
docker cp lekse-sammendrag-worker:/app/data/lekse-sammendrag.db ./backup.db

Deployment

Docker (Production)

# Build image
docker build -f docker/Dockerfile -t lekse-sammendrag:latest .

# Run container
docker run -d \
  --name lekse-sammendrag \
  --restart unless-stopped \
  -p 8080:8080 \
  -v $(pwd)/data:/app/data \
  -v $(pwd)/logs:/app/logs \
  --env-file .env \
  lekse-sammendrag:latest

Docker Compose (Recommended)

# Start
docker-compose up -d

# Update and restart
docker-compose pull
docker-compose up -d

# View logs
docker-compose logs -f --tail=100

# Stop
docker-compose down

systemd Service (Linux)

Create /etc/systemd/system/lekse-sammendrag.service:

[Unit]
Description=Lekse Sammendrag Document Monitor
After=network.target

[Service]
Type=notify
WorkingDirectory=/opt/lekse-sammendrag
ExecStart=/usr/bin/dotnet /opt/lekse-sammendrag/LekseSammendrag.Worker.dll
Restart=always
RestartSec=10
User=lekse-sammendrag
Environment=DOTNET_ENVIRONMENT=Production
EnvironmentFile=/opt/lekse-sammendrag/.env

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable lekse-sammendrag
sudo systemctl start lekse-sammendrag
sudo systemctl status lekse-sammendrag

Troubleshooting

Configuration validation failed

The service includes comprehensive startup validation that checks all required configuration before starting. If configuration is missing or invalid, you'll see a detailed error message:

╔════════════════════════════════════════════════════════════════════════════╗
║                    CONFIGURATION VALIDATION FAILED                         ║
╚════════════════════════════════════════════════════════════════════════════╝

Found 4 configuration error(s):

  1. Email:Username is required (SMTP authentication username)
  2. Email:Password is required (SMTP authentication password)
  3. Email:FromEmail is required (sender email address)
  4. Email:Recipients is required (comma-separated recipient email addresses)

Configuration can be provided via:
  1. Environment Variables
  2. appsettings.json
  3. Docker .env file

Solution:

  1. Review the listed errors carefully
  2. Set all required configuration values
  3. Validate email addresses are properly formatted
  4. Ensure Google Docs URL follows the expected format: https://docs.google.com/document/d/[ID]/edit...
  5. Restart the service after fixing configuration

No emails being sent

  1. Check SMTP credentials: Verify username, password, host, port
  2. Gmail App Password: Must use App Password if 2FA enabled
  3. Check logs: Look for SMTP errors in logs/ directory
  4. Test SMTP: Use telnet or online SMTP tester
  5. Firewall: Ensure outbound port 587/465 is open

Document not being checked

  1. Check URL: Must be publicly accessible Google Docs URL
  2. View logs: Check for download errors
  3. Health check: Visit http://localhost:8080/health
  4. Permissions: Ensure document has "Anyone with the link can view"

Database errors

  1. Permissions: Check write permissions on data/ directory
  2. Disk space: Ensure sufficient disk space
  3. Corruption: Restore from backup or delete database (will resend on first run)

Docker issues

  1. Build fails: Ensure .NET 10 SDK is available in base image
  2. Container stops: Check logs with docker-compose logs
  3. Permission denied: Check volume mount permissions
  4. Health check fails: Verify port 8080 is exposed and accessible

Development

Build

# Restore dependencies
dotnet restore

# Build solution
dotnet build

# Run all tests
dotnet test

Testing

The project includes comprehensive unit tests for all core services with 90+ tests covering:

  • ConfigurationValidator (18 tests) - Configuration validation logic
  • ChangeDetectionService (25 tests) - SHA256 hashing and change detection
  • DocumentDownloader (17 tests) - PDF download and URL transformation
  • EmailService (30 tests) - Email composition and SMTP sending

Test Technologies:

  • xUnit - Test framework
  • Moq - Mocking framework
  • FluentAssertions - Assertion library
  • EF Core InMemory - In-memory database for testing

Run all tests:

dotnet test

Run specific test class:

dotnet test --filter "FullyQualifiedName~ConfigurationValidatorTests"
dotnet test --filter "FullyQualifiedName~ChangeDetectionServiceTests"
dotnet test --filter "FullyQualifiedName~DocumentDownloaderTests"
dotnet test --filter "FullyQualifiedName~EmailServiceTests"

Run tests with detailed output:

dotnet test --logger "console;verbosity=detailed"

Test Coverage:

  • All core services have comprehensive test coverage
  • Tests verify happy paths, edge cases, and error handling
  • Logging behavior is verified using mock assertions
  • Database operations tested with InMemory provider

Local Development Configuration

For local development, you can use dotnet user-secrets to securely store your configuration without committing credentials to git.

Option 1: Sync from .env file (Recommended)

If you already have a .env file configured, use the provided script to automatically sync all values to user-secrets:

# On Linux/macOS
./scripts/sync-env-to-secrets.sh

# On Windows (PowerShell)
.\scripts\sync-env-to-secrets.ps1

The script will:

  • Read your .env file
  • Convert Docker environment variables to .NET configuration format
  • Set all values as user-secrets
  • Optionally clear existing secrets first

Option 2: Set secrets manually

# Navigate to worker project
cd src/LekseSammendrag.Worker

# Set email configuration
dotnet user-secrets set "Email:SmtpHost" "smtp.gmail.com"
dotnet user-secrets set "Email:SmtpPort" "587"
dotnet user-secrets set "Email:Username" "your-email@gmail.com"
dotnet user-secrets set "Email:Password" "your-app-password"
dotnet user-secrets set "Email:UseSsl" "true"
dotnet user-secrets set "Email:FromEmail" "your-email@gmail.com"
dotnet user-secrets set "Email:FromName" "Lekse Sammendrag"
dotnet user-secrets set "Email:Recipients" "recipient@example.com"

# Set document configuration
dotnet user-secrets set "Document:GoogleDocsUrl" "https://docs.google.com/document/d/YOUR_ID/edit"
dotnet user-secrets set "Document:CheckIntervalMinutes" "60"

# List all secrets
dotnet user-secrets list

# Clear all secrets
dotnet user-secrets clear

Environment variable mapping:

.env Variable user-secrets Key Description
SMTP_HOST Email:SmtpHost SMTP server hostname
SMTP_PORT Email:SmtpPort SMTP server port
SMTP_USERNAME Email:Username SMTP username
SMTP_PASSWORD Email:Password SMTP password
SMTP_ENABLE_SSL Email:UseSsl Enable SSL/TLS
FROM_EMAIL Email:FromEmail Sender email
FROM_NAME Email:FromName Sender name
TO_EMAILS Email:Recipients Recipients (comma-separated)
GOOGLE_DOCS_URL Document:GoogleDocsUrl Google Docs URL
CHECK_INTERVAL_MINUTES Document:CheckIntervalMinutes Check interval

Where are user-secrets stored?

User-secrets are stored outside your project directory in a secure location:

  • Windows: %APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json
  • Linux/macOS: ~/.microsoft/usersecrets/<user_secrets_id>/secrets.json

This keeps your credentials secure and separate from your source code.

Run Locally

# Run worker directly
dotnet run --project src/LekseSammendrag.Worker

# Run with Aspire dashboard
dotnet run --project src/LekseSammendrag.AppHost

Database Migrations

# Add migration
dotnet ef migrations add MigrationName --project src/LekseSammendrag.Core --startup-project src/LekseSammendrag.Worker

# Update database
dotnet ef database update --project src/LekseSammendrag.Core --startup-project src/LekseSammendrag.Worker

Technology Stack

  • .NET 10 - Runtime and framework
  • Entity Framework Core - SQLite ORM
  • MailKit - SMTP email client
  • Serilog - Structured logging
  • .NET Aspire - Cloud-native orchestration
  • Docker - Containerization
  • Alpine Linux - Lightweight container base

License

[Your License Here]

Support

For issues and questions, please open an issue on the repository.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages