Skip to content

Add app monitoring and alerts #337

@ECWireless

Description

@ECWireless

Cost: Free (Vercel Logs + BetterStack free tier)
Note: If the Vercel integration doesn't work well, we can add pino logs
Components:

  1. Vercel Logs (Built-in)
  2. BetterStack Uptime Monitors (5 monitors)
    • Homepage: Check every 15 min
    • Orchestrators page: Check every 15 min
    • Voting page: Check every 15 min
    • Treasury page: Check every 15 min
    • /api/health: Check every 15 min
  3. BetterStack Log Drain (Vercel Integration)
    • Captures all serverless function errors
    • Logs are more permanent
  4. Critical Log Alerts
    • Threshold: 3 occurrences in 1 hour
    • Channel: email or Slack or Discord or Telegram
  5. External Dependency Monitors (3 monitors)
    • The Graph subgraph health
    • Livepeer.com API status
    • Metrics server availability
  6. API Performance Monitors
    • /api/usage - Alert if >15s
    • /api/score/* - Alert if >15s
  7. Health Check Endpoint
// pages/api/health.ts (NEW FILE)
export const config = { runtime: 'edge' };

export default async function handler(req: Request) {
  const checks = await Promise.allSettled([
    fetch(SUBGRAPH_URL, { signal: AbortSignal.timeout(3000) }),
    fetch(METRICS_URL, { signal: AbortSignal.timeout(3000) }),
  ]);

  const subgraphOk = checks[0].status === 'fulfilled' && checks[0].value.ok;
  const metricsOk = checks[1].status === 'fulfilled' && checks[1].value.ok;
  
  const allHealthy = subgraphOk && metricsOk;

  const response = {
    status: allHealthy ? 'healthy' : 'degraded',
    timestamp: new Date().toISOString(),
    checks: {
      subgraph: subgraphOk ? 'ok' : 'error',
      metrics: metricsOk ? 'ok' : 'error',
    },
  };

  return new Response(JSON.stringify(response), {
    status: allHealthy ? 200 : 503,
    headers: {
      'Content-Type': 'application/json',
      'Cache-Control': 'no-store, max-age=0',
    },
  });
}

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

Status

Todo

Relationships

None yet

Development

No branches or pull requests

Issue actions