Job Scanner Testing Suite

Comprehensive integration and E2E tests for Job Scanner application.

Overview

This testing project provides:

• Integration Tests: Test real API endpoints and file operations
• E2E Tests: Test complete user workflows with Playwright
• Real Server Testing: Tests run against actual running backend and frontend
• File Verification: Validates job_tracking.json and CSV updates

Prerequisites

Servers must be running before tests:

1. Backend: cd backend && python api.py

   • Should be accessible at http://localhost:5000
   • Health check: http://localhost:5000/api/health

2. Frontend: cd frontend && npm run dev

   • Should be accessible at http://localhost:5173
   • Should display login page

Installation

cd testing
npm install
npx playwright install

Project Structure

testing/
├── integration/              # Integration tests
│   ├── job-status-api.test.ts       # API endpoint tests
│   ├── file-persistence.test.ts     # File I/O tests
│   ├── status-transitions.test.ts   # Status transition tests
│   └── helpers/
│       ├── api-client.ts            # API client for backend
│       ├── file-helpers.ts          # File operation helpers
│       └── test-data.ts             # Test data fixtures
├── e2e/                      # E2E tests
│   ├── favorite-workflow.spec.ts    # Favorite functionality
│   ├── apply-workflow.spec.ts       # Apply functionality
│   ├── archive-workflow.spec.ts     # Archive functionality
│   ├── status-transitions.spec.ts   # Status transitions
│   └── fixtures/
│       ├── jobs.json                # Test job data
│       └── auth.json                # Test credentials
├── global-setup.ts           # Pre-test setup
├── global-teardown.ts        # Post-test cleanup
├── playwright.config.ts      # Playwright configuration
└── package.json              # Dependencies and scripts

Running Tests

Integration Tests

# Run all integration tests
npm run test:integration

# Run specific integration test
npm run test:integration integration/job-status-api.test.ts

What's tested:

• API endpoints (PUT /api/jobs/{job_key}/status)
• File updates (job_tracking.json)
• CSV exports (job_applications.csv)
• Status transitions (all ↔ favorite ↔ applied ↔ archive)

E2E Tests

# Run all E2E tests (headless)
npm run test:e2e

# Run with UI (interactive)
npm run test:e2e:ui

# Run in headed mode (see browser)
npm run test:headed

# Run specific test file
npm run test:e2e e2e/favorite-workflow.spec.ts

# Debug mode
npm run test:e2e:debug

What's tested:

• Complete user workflows
• Form interactions
• Status filtering
• Toast notifications
• Button state changes

All Tests

# Run everything (integration + E2E)
npm run test:all

Test Reports

After running E2E tests:

npm run report

This opens the Playwright HTML report with:

• Test results (passed/failed)
• Screenshots on failure
• Videos on failure
• Trace viewer for debugging

Test Coverage

Integration Tests (3 files, ~15 tests)

• ✅ API Endpoints: All status update operations
• ✅ File Operations: job_tracking.json CRUD
• ✅ CSV Export: job_applications.csv updates
• ✅ Status Transitions: All valid transitions

E2E Tests (4 files, ~10 tests)

• ✅ Favorite Workflow: Add, update, remove favorites
• ✅ Apply Workflow: Mark as applied, update details, return to opportunities
• ✅ Archive Workflow: Archive with reason, update reason
• ✅ Status Transitions: Complex multi-step workflows

Test Data

Test Users

• Username: admin
• Password: admin123

Test Jobs

Tests use these job keys:

• google|senior backend engineer
• amazon|staff engineer
• microsoft|principal engineer

Note: Test data is cleaned up after tests via global-teardown.ts

Configuration

Environment Variables

Create .env file (or copy from .env.example):

BACKEND_URL=http://localhost:5000
BACKEND_API_URL=http://localhost:5000/api
FRONTEND_URL=http://localhost:5173
TEST_USERNAME=admin
TEST_PASSWORD=admin123

Playwright Configuration

Edit playwright.config.ts to customize:

• Test timeout
• Number of retries
• Browser selection (chromium, firefox, webkit)
• Screenshot/video capture settings

Troubleshooting

"Backend server is not running"

Problem: Integration/E2E tests fail immediately

Solution:

cd backend
python api.py
# Should see: Running on http://localhost:5000

Verify: http://localhost:5000/api/health

"Frontend server is not running"

Problem: E2E tests fail with navigation errors

Solution:

cd frontend
npm run dev
# Should see: Local: http://localhost:5173

Verify: Open http://localhost:5173 in browser

Tests timeout

Problem: Tests hang or timeout

Diagnosis:

# Check if servers are responding
curl http://localhost:5000/api/health
curl http://localhost:5173/

# Check if ports are in use
lsof -i :5000
lsof -i :5173

Solution:

• Increase timeout in playwright.config.ts
• Check backend logs for slow API calls
• Verify database connections are working
• Check network tab for slow requests

File not found errors

Problem: Integration tests can't find job_tracking.json

Diagnosis:

# Check if output directory exists
ls -la ../backend/output/

# Check file paths in file-helpers.ts
cat integration/helpers/file-helpers.ts

Solution:

• Ensure backend/output/ directory exists
• Check permissions on output directory
• Run backend at least once to create files

Authentication failures

Problem: Tests fail with 401 Unauthorized

Diagnosis:

# Check test credentials
cat e2e/fixtures/auth.json

# Test login manually
curl -X POST http://localhost:5000/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"admin123"}'

Solution:

• Verify test credentials match backend users
• Check JWT token expiration settings
• Ensure auth endpoints are working

Flaky tests

Problem: Tests pass sometimes, fail other times

Common causes:

• Race conditions (files not written yet)
• Network delays
• Timing issues

Solutions:

• Increase waitForTimeout values
• Use waitForFileUpdate helper
• Add await page.waitForLoadState('networkidle')
• Use explicit waits instead of fixed timeouts

Best Practices

Writing Integration Tests

// ✅ GOOD: Wait for file updates
await apiClient.updateJobStatus(jobKey, 'favorite');
await new Promise(resolve => setTimeout(resolve, 500)); // Wait for file write
const tracking = getJobFromTracking(jobKey);
assert.ok(tracking);

// ❌ BAD: No wait, file might not be written yet
await apiClient.updateJobStatus(jobKey, 'favorite');
const tracking = getJobFromTracking(jobKey); // Might fail

Writing E2E Tests

// ✅ GOOD: Wait for element visibility
await expect(page.locator('[data-testid="form"]')).toBeVisible();
await page.fill('input[name="field"]', 'value');

// ❌ BAD: No wait, element might not be ready
await page.fill('input[name="field"]', 'value'); // Might fail

Test Independence

// ✅ GOOD: Each test is independent
test('should favorite job', async () => {
  // Setup: ensure clean state
  // Action: favorite job
  // Assert: verify favorited
});

// ❌ BAD: Tests depend on each other
test('should favorite job', async () => { /* ... */ });
test('should update favorite', async () => {
  // Assumes previous test ran!
});

CI/CD Integration

GitHub Actions Example

name: E2E Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '20'

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'

      - name: Install dependencies
        run: |
          cd backend && pip install -r requirements.txt
          cd frontend && npm install
          cd testing && npm install
          npx playwright install --with-deps

      - name: Start backend
        run: cd backend && python api.py &

      - name: Start frontend
        run: cd frontend && npm run dev &

      - name: Wait for servers
        run: |
          sleep 10
          curl http://localhost:5000/api/health
          curl http://localhost:5173/

      - name: Run tests
        run: cd testing && npm run test:all

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: playwright-report
          path: testing/playwright-report/

Development Workflow

Adding New Tests

1. Integration test:

   cd testing/integration
   # Create new-feature.test.ts
   npm run test:integration integration/new-feature.test.ts

2. E2E test:

   cd testing/e2e
   # Create new-workflow.spec.ts
   npm run test:e2e:ui  # Use UI mode for development

Debugging Tests

# E2E debugging
npm run test:e2e:debug e2e/favorite-workflow.spec.ts

# This opens Playwright Inspector:
# - Step through test
# - Inspect page elements
# - View network requests
# - See console logs

Running Specific Tests

# Run single E2E test
npx playwright test e2e/favorite-workflow.spec.ts

# Run test by name
npx playwright test -g "should add job to favorites"

# Run in headed mode (see browser)
npx playwright test --headed

# Run with specific browser
npx playwright test --project=firefox

Performance Tips

Speed up E2E tests

1. Run in parallel (default):

   // playwright.config.ts
   fullyParallel: true,
   workers: 4, // Number of parallel workers

2. Reduce timeouts for fast tests:

   test('fast test', async ({ page }) => {
     test.setTimeout(10000); // 10 seconds
     // ...
   });

3. Use test.describe.configure for independent tests:

   test.describe.configure({ mode: 'parallel' });

Speed up integration tests

1. Batch API calls:

   await Promise.all([
     apiClient.updateJobStatus(job1, 'favorite'),
     apiClient.updateJobStatus(job2, 'applied'),
     apiClient.updateJobStatus(job3, 'archive'),
   ]);

2. Reduce file wait times (if tests are stable):

   await new Promise(resolve => setTimeout(resolve, 200)); // Instead of 500

Maintenance

Updating Test Data

Edit integration/helpers/test-data.ts:

export const testJobs = {
  newCompany: {
    company: 'New Company',
    position: 'New Position',
    jobKey: 'new company|new position',
  },
};

Updating Fixtures

Edit e2e/fixtures/auth.json or e2e/fixtures/jobs.json:

{
  "newTestUser": {
    "username": "testuser",
    "password": "testpass"
  }
}

Cleaning Up Old Data

# Manual cleanup
node -e "
const fs = require('fs');
fs.writeFileSync('../backend/output/job_tracking.json', '{}');
fs.writeFileSync('../backend/output/job_applications.csv', '');
"

Known Issues

Issue: Tests fail on first run

Cause: Backend database might not be initialized

Solution: Run backend once manually before tests:

cd backend && python api.py
# Wait for startup, then Ctrl+C
cd ../testing && npm run test:all

Issue: Playwright browsers not installed

Symptom: browserType.launch: Executable doesn't exist

Solution:

npx playwright install

Issue: Port already in use

Symptom: Backend/frontend won't start

Solution:

# Find process using port
lsof -i :5000  # Backend
lsof -i :5173  # Frontend

# Kill process
kill -9 <PID>

Contributing

When adding tests:

1. Follow existing test structure
2. Use descriptive test names
3. Add comments for complex logic
4. Ensure tests are independent
5. Update this README if adding new test types

Resources

• Playwright Documentation
• Node.js Test Runner
• Axios Documentation

Summary

This testing suite provides comprehensive coverage for Job Scanner:

• 15+ integration tests verifying backend functionality
• 10+ E2E tests verifying user workflows
• Real server testing for accurate results
• File verification for data persistence
• Detailed reports with screenshots and videos

Run npm run test:all to verify everything works correctly!