RepoLedger

GitHub Webhook Listener → SQLite Logger → Custom Event Handlers

Overview

RepoLedger listens for GitHub webhooks at /webhook, verifies each request’s HMAC SHA-256 signature (X-Hub-Signature-256), stores the event in a local SQLite database (github_events.db), and dispatches a pluggable handler for each X-GitHub-Event. You can use this as a foundation for notifications, CI/CD triggers, dashboards, or any workflow that starts from GitHub events.

Project Structure

RepoLedger/
├── .env
├── .gitignore
├── package.json
├── README.md
└── src/
    ├── app.js
    ├── server.js
    ├── config/
    │   ├── database.js
    │   └── logger.js
    ├── routes/
    │   ├── webhook.js
    │   └── events.js
    └── utils/
        ├── signatureValidator.js
        └── eventHandler.js

• .env    Environment variables (PORT, WEBHOOK_SECRET)
• .gitignore Git ignores (node_modules/, github_events.db, logs/, etc.)
• package.json Dependencies and scripts
• README.md Documentation

Within src/:

• app.js Configures Express, raw-body parsing on /webhook, and mounts /api/events/latest
• server.js Initialises SQLite, then starts Express on the configured port
• config/logger.js Winston logger setup
• config/database.js Initialises (and migrates) SQLite and exports insertEvent(...)
• routes/webhook.js Verifies signature, logs to SQLite, then invokes the custom handler
• routes/events.js Provides GET /api/events/latest to return the most recent logged event
• utils/signatureValidator.js Validates GitHub’s X-Hub-Signature-256
• utils/eventHandler.js Dispatches logic per GitHub event (e.g. push, pull_request, issues)

The SQLite file (github_events.db) is created automatically at runtime in the project root.

Prerequisites

• Node.js v16 or newer
• npm (bundled with Node.js)
• Git (to clone or manage the repo)

(No external database is required—everything runs locally with SQLite.)

Setup

1. Clone or copy the repository:

   git clone https://github.com/Blindeenlightz/RepoLedger
   cd RepoLedger

2. Install dependencies:

   npm install

3. Create a .env file in the project root with:

   PORT=3000
   WEBHOOK_SECRET=<your_github_webhook_secret>

   • PORT  — Port on which the Express server will listen (default: 3000)
   • WEBHOOK_SECRET  — The exact “Secret” you set when configuring the GitHub webhook

4. Start the service:

   npm start

   You should see logs like:

   2025-06-02 15:00:00 [info]: SQLite initialised at /path/to/repo/RepoLedger/github_events.db
   2025-06-02 15:00:00 [info]: Server listening on port 3000

Local Testing with ngrok

Because GitHub needs a public HTTPS endpoint for webhooks, you can use ngrok to expose your local server at a temporary public URL. This lets you test webhooks on your local machine.

1. In one terminal, ensure your Node server is running on port 3000:

   npm start

2. In another terminal, start ngrok to tunnel port 3000:

   ngrok http 3000

   You should see something like:

   Session Status                online
   Version                       3.22.1
   Region                        United States (California) (us-cal-1)
   Web Interface                 http://127.0.0.1:4040
   Forwarding                    https://abcd1234.ngrok-free.app → http://localhost:3000

3. Copy the HTTPS URL (e.g. https://abcd1234.ngrok-free.app).

4. In your GitHub repository settings → Settings → Webhooks → Add webhook, set:

   • Payload URL: https://abcd1234.ngrok-free.app/webhook
   • Content type: application/json
   • Secret: exactly the same as WEBHOOK_SECRET in your .env
   • Which events?: “Send me everything” or select specific events (Push, Pull request, Issues, etc.)
   • Click Add webhook.

5. Trigger an event in your repo (push a commit, open a PR, etc.). ngrok’s console will show a POST /webhook forwarding to your local server. If everything is set up correctly, your server console logs should include:

   Received POST /webhook
   Event inserted: type=push, at=2025-06-02T15:05:12.345Z
   Push in my-org/my-repo by octocat: commits abc123

Configure GitHub Webhook (Without ngrok)

If you deploy RepoLedger to a public server (or cloud VM) with a real domain (e.g. https://repoledger.example.com), configure GitHub’s Payload URL as https://repoledger.example.com/webhook and use the same WEBHOOK_SECRET.

How It Works

1. Signature Validation
   • The Express route reads req.body as a raw Buffer (configured by body-parser.raw({ type: '*/*' })).
   • It extracts the X-Hub-Signature-256 header and computes an HMAC SHA-256 over the raw body using WEBHOOK_SECRET.
   • If the computed hex digest matches the header’s value, the request is authentic; otherwise, it is rejected with a 400 status.
2. JSON Parsing & Database Logging
   • After signature validation, the raw Buffer is parsed into a JSON object.
   • insertEvent(eventType, payload) is called to store a record in SQLite, including eventType, the full JSON‐stringified payload, and a timestamp (receivedAt).
3. Event Dispatch
   • Once logged, handleEvent(eventType, payload) is invoked. It contains a switch statement that routes each event type to a custom handler function (e.g. handlePushEvent, handlePullRequestEvent, handleIssuesEvent).
   • By default, these handlers log details, but you can extend them to send emails, trigger CI pipelines, post to Slack, or perform other actions.
4. Immediate HTTP Response
   • The route responds with 200 { "received": true } as soon as the event is safely inserted into the database.
   • Custom handler logic runs asynchronously in the background (errors are caught and logged but do not block the HTTP response).
5. Read-back Endpoint
   • A GET request to /api/events/latest returns the most recently inserted event as JSON. This is useful for testing (e.g. via Postman) or building dashboards that show the latest GitHub activity.

Database Schema

The SQLite database (github_events.db) contains one table:

  CREATE TABLE IF NOT EXISTS events (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  eventType TEXT NOT NULL,
  payload TEXT NOT NULL,
  receivedAt TEXT NOT NULL
);

• id  — Auto‐incrementing integer
• eventType  — GitHub event name (string)
• payload  — Entire JSON payload, stored as a string
• receivedAt  — UTC ISO timestamp of insertion

Inspecting the Database

You can use the sqlite3 CLI to query events:

1. Enter the SQLite shell:

   sqlite3 github_events.db

2. List stored events:

   SELECT id, eventType, receivedAt FROM events;

3. View a specific payload (e.g. id=1):

   SELECT payload FROM events WHERE id=1;

4. Exit:

   .exit

Because the full JSON is in payload, you can even pipe it into jq for further inspection:

nginx



sqlite3 github_events.db "SELECT payload FROM events WHERE id=1;" | jq .

Customising eventHandler.js

Open src/utils/eventHandler.js. By default it includes examples for:

• push: Logs repository name, pusher, and commit IDs.
• pull_request: Logs PR number, action, and author.
• issues: Logs issue number, action, and sender.

To add a new handler:

1. Define an async function handleMyEvent(payload) { ... } below the existing handlers.
2. Add a case 'my_event_type': await handleMyEvent(payload); break; in the switch(eventType) block.

For example, to handle a release event:

// In src/utils/eventHandler.js

export async function handleEvent(eventType, payload) {
  switch (eventType) {
    case 'push':
      await handlePushEvent(payload);
      break;
    case 'pull_request':
      await handlePullRequestEvent(payload);
      break;
    case 'issues':
      await handleIssuesEvent(payload);
      break;
    case 'release':
      await handleReleaseEvent(payload);
      break;
    default:
      logger.info(`No handler for event type: ${eventType}`);
      break;
  }
}

async function handleReleaseEvent(payload) {
  const action = payload.action; // e.g. "published", "created"
  const tagName = payload.release.tag_name;
  const repo = payload.repository.full_name;
  const author = payload.sender.login;
  logger.info(`Release ${tagName} in ${repo} was ${action} by ${author}`);
  // …add your logic here… (e.g., notify team, deploy docs, etc.)
}

You can call third‐party APIs, post to chat services, send emails, or trigger deployments from within these handlers.

Troubleshooting

• “Invalid signature” errors
  • Ensure the GitHub webhook’s Secret exactly matches WEBHOOK_SECRET in .env.
  • Verify GitHub is sending the X-Hub-Signature-256 header (older GitHub apps may send X-Hub-Signature with SHA1 instead of SHA256). If needed, adjust signatureValidator.js to support SHA1.
• Database file not created
  • Check write permissions in the project directory.
  • Look at console logs: SQLite initialised at /path/to/repo/RepoLedger/github_events.db.
• Handler code not running
  • Confirm that handleEvent is imported and invoked in routes/webhook.js.
  • Check console logs for any errors in the handler functions.
• Port conflicts
  • If port 3000 is in use, change PORT in .env and restart.
• ngrok ERR_NGROK_8012 (cannot connect to localhost:3000)
  • Ensure your Node server is running and listening on port 3000 (npm start).
  • Restart ngrok with ngrok http 3000 only after confirming GET http://localhost:3000/health returns 200.

License

MIT License

Distributed icons created by gravisio - Flaticon