Conversation
Adjanour
commented
Mar 23, 2026
- Add invoice automation and notification system
- Add backup and recovery runbook
- Document architecture and operational flows
There was a problem hiding this comment.
Pull request overview
Adds invoice automation + email delivery, in-app operational notifications, and accompanying documentation/runbooks to support production operations and recovery.
Changes:
- Introduces a
backend-workerservice to generate weekly invoices, send automatic invoice emails, and report automation health/status. - Adds notification persistence + API routes and a frontend notifications UI (badge + dropdown/sheet).
- Adds operational docs (architecture, API reference, role guide, ops runbook) plus Postgres backup/restore scripts and new env knobs.
Reviewed changes
Copilot reviewed 48 out of 49 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/role-guide.md | New role-by-role product usage guide and operational expectations. |
| docs/ops-runbook.md | New backup/restore and incident-response runbook for VPS operations. |
| docs/architecture.md | New high-level architecture + flow documentation (runtime, data model, automation). |
| docs/api-reference.md | New API reference including auth model, core resources, invoices, notifications. |
| deploy/restore-postgres.sh | Adds scripted Postgres restore helper. |
| deploy/docker-compose.yml | Adds backend-worker service and healthcheck to production compose. |
| deploy/docker-compose.local.yml | Adds backend-worker service to local compose stack. |
| deploy/backup-postgres.sh | Adds scripted Postgres backup helper with checksum generation. |
| deploy/backend.local.env.example | Adds SMTP + invoice automation env variables for local runs. |
| deploy/backend.env.example | Adds SMTP + invoice automation env variables for production runs. |
| deploy/README.md | Documents worker service, backups/recovery docs, and new env knobs. |
| bun.lock | Adds nodemailer + types to lockfile. |
| apps/frontend/src/styles.css | Adds styles for notification trigger/badge/dropdown/sheet UI. |
| apps/frontend/src/pages/InvoicesPage.tsx | Adds automation monitor panel and “send/retry invoice email” action + email status fields. |
| apps/frontend/src/lib/types.ts | Adds types for invoice source/email status, automation status, and app notifications. |
| apps/frontend/src/lib/api.ts | Adds API client methods for automation status, invoice email send, and notifications. |
| apps/frontend/src/components/AppLayout.tsx | Adds notification polling, dropdown/sheet UI, and “mark read” flows. |
| apps/frontend/src/components/AppLayout.test.tsx | Adds jsdom tests for notifications badge, open, read, and mark-all-read. |
| apps/frontend/README.md | Replaces template README with repo-specific frontend overview/commands. |
| apps/backend/src/worker.ts | Adds the invoice automation worker loop orchestrating sweeps + email delivery + status updates. |
| apps/backend/src/worker-health.ts | Adds worker healthcheck entrypoint based on automation status evaluation. |
| apps/backend/src/routes/waybills.ts | Adds failed-delivery notification emission hook (currently placed in assign route). |
| apps/backend/src/routes/shifts.ts | Adds shift handover notification for incoming rider. |
| apps/backend/src/routes/notifications.ts | Adds notifications API routes (list/read/read-all). |
| apps/backend/src/routes/notifications.test.ts | Adds route tests for notifications endpoints. |
| apps/backend/src/routes/invoices.ts | Refactors invoice creation into lib, adds automation status endpoint + send-email endpoint, expands invoice list fields. |
| apps/backend/src/lib/notifications.ts | Implements notification creation, dedupe via eventKey, list/unread count, mark read APIs. |
| apps/backend/src/lib/mailer.ts | Implements SMTP invoice email sending with PDF attachment. |
| apps/backend/src/lib/mailer.test.ts | Adds unit test for invoice email content builder. |
| apps/backend/src/lib/invoices.ts | Extracts invoice creation/detail serialization logic and adds email-related fields. |
| apps/backend/src/lib/invoice-email.ts | Implements send-email flow + worker sweep for pending automatic invoice emails. |
| apps/backend/src/lib/invoice-automation.ts | Implements delivery scanning + weekly invoice generation + notifications for invoice-ready. |
| apps/backend/src/lib/invoice-automation.test.ts | Adds unit tests for invoice window grouping logic. |
| apps/backend/src/lib/automation-status.ts | Adds persistence + serialization for automation monitor status. |
| apps/backend/src/lib/automation-health.ts | Adds health evaluation logic (disabled/running/stale/failure). |
| apps/backend/src/lib/automation-health.test.ts | Adds unit tests for automation health evaluation. |
| apps/backend/src/index.ts | Registers notifications routes in the backend router. |
| apps/backend/src/db/schema.ts | Adds invoice source/email status fields, notifications table, automation job statuses table, and new enums/indexes. |
| apps/backend/src/config.ts | Adds SMTP + invoice automation configuration parsing and mail config assertion. |
| apps/backend/package.json | Updates build output to include worker entrypoints + adds nodemailer deps. |
| apps/backend/drizzle/meta/_journal.json | Adds migration journal entries for new schema changes. |
| apps/backend/drizzle/meta/0002_snapshot.json | Adds schema snapshot including new invoice fields (and related enums). |
| apps/backend/drizzle/0004_broad_joshua_kane.sql | Migration to create automation_job_statuses table + index. |
| apps/backend/drizzle/0003_gigantic_kylun.sql | Migration to create notifications table + enum + indexes. |
| apps/backend/drizzle/0002_fuzzy_silhouette.sql | Migration to add invoice source/email status fields + unique billing window index. |
| apps/backend/README.md | Replaces template README with repo-specific backend overview/commands. |
| README.md | Adds repo-level README describing system, architecture, and common commands. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const actor = await db.query.users.findFirst({ | ||
| where: eq(users.active, true), | ||
| orderBy: asc(users.createdAt), | ||
| }) | ||
|
|
||
| assert( | ||
| actor && (actor.role === 'admin' || actor.role === 'ops'), | ||
| new AppError( | ||
| 500, | ||
| 'automation_actor_missing', | ||
| 'Create an admin or ops account, or set AUTOMATION_ACTOR_PHONE before enabling invoice automation.', | ||
| ), | ||
| ) | ||
|
|
There was a problem hiding this comment.
resolveAutomationActor() falls back to the earliest active user without filtering by role, then asserts the role is admin/ops. If the first active account is a rider, invoice automation will hard-fail even if an active admin/ops exists. Change the fallback query to filter to active users with role in (admin,ops) (and keep the createdAt ordering).
| const existing = await findInvoiceByWindow(input.clientId, input.periodStart, input.periodEnd) | ||
| if (existing) { | ||
| if (input.onExisting === 'reuse') { | ||
| return { | ||
| created: false as const, | ||
| invoice: await getInvoiceDetail(existing.id), | ||
| } | ||
| } | ||
|
|
||
| throw new AppError( | ||
| 409, | ||
| 'invoice_exists', | ||
| 'An invoice already exists for that client and billing period.', | ||
| ) | ||
| } |
There was a problem hiding this comment.
createInvoiceForWindow() has a race between findInvoiceByWindow(...) and the later insert. With the new worker and manual creation, concurrent callers can both miss the pre-check and then hit the unique index on (clientId, periodStart, periodEnd). For onExisting: 'reuse', handle unique-constraint conflicts by fetching and returning the existing invoice, or do the insert with an UPSERT/transactional approach so the function is safely idempotent under concurrency.
| Checklist: | ||
|
|
||
| 1. Check `backend-worker` logs. | ||
| 2. Confirm `INVOICE_AUTOMATION_ENABLED=true`. | ||
| 3. Confirm `AUTOMATION_ACTOR_PHONE` is set to an active `admin` or `ops` user, or that at least one active `admin/ops` user exists. | ||
| 4. Restart the worker: | ||
|
|
||
| ```bash | ||
| docker compose --env-file deploy/compose.env -f deploy/docker-compose.yml restart backend-worker | ||
| ``` | ||
|
|
||
| 1. Recheck the automation monitor. | ||
|
|
There was a problem hiding this comment.
The checklist numbering jumps from step 4 to step 1 (“Recheck the automation monitor”), which is confusing in an ops runbook. Renumber that last step to continue the sequence (or switch the whole block to unordered bullets).
| Primary code lives in: | ||
|
|
||
| - [apps/frontend/src/pages](/home/bernard/Work/way-bills/apps/frontend/src/pages) | ||
| - [apps/frontend/src/components](/home/bernard/Work/way-bills/apps/frontend/src/components) | ||
| - [apps/frontend/src/lib](/home/bernard/Work/way-bills/apps/frontend/src/lib) | ||
|
|
There was a problem hiding this comment.
This doc uses absolute local filesystem paths in markdown links (e.g. /home/bernard/Work/way-bills/...), which will be broken for anyone reading the docs in GitHub. Convert these to repo-relative links (e.g. apps/frontend/src/pages) so they work for all readers.
| - [apps/frontend](/home/bernard/Work/way-bills/apps/frontend): React/Vite operations UI | ||
| - [apps/backend](/home/bernard/Work/way-bills/apps/backend): Hono API, worker, migrations, PDF generation, email delivery | ||
| - [deploy](/home/bernard/Work/way-bills/deploy): Docker, VPS, Caddy, backup, and deployment helpers | ||
| - [docs](/home/bernard/Work/way-bills/docs): product, architecture, and operations documentation |
There was a problem hiding this comment.
This README uses absolute local filesystem paths in links (e.g. /home/bernard/Work/way-bills/...), which will be broken for anyone not on that machine. Please convert these to repo-relative links (e.g. /apps/frontend, /docs/architecture.md) so they render correctly on GitHub.
| const [summary] = await db.select({ count: count() }).from(invoices) | ||
| const now = new Date() | ||
| const dueAt = | ||
| input.dueAt ?? new Date(now.getTime() + client.paymentTermsDays * 86400000) | ||
| const subtotalCents = pricedWaybills.reduce( | ||
| (total, item) => total + item.amountCents, | ||
| 0, | ||
| ) | ||
|
|
||
| const invoiceId = crypto.randomUUID() | ||
| const invoiceNumber = createInvoiceNumber(now, Number(summary?.count ?? 0) + 1) | ||
|
|
There was a problem hiding this comment.
Invoice numbers are generated from count(invoices)+1. Under concurrent invoice creation (now more likely with an automation worker), two inserts can compute the same sequence and violate the invoice_number unique index. Consider a database-backed sequence/serial, or retry-on-conflict generation (or using the invoice id/timestamp-based suffix) to guarantee uniqueness without relying on a non-atomic count.
| html: [ | ||
| `<p>Hello ${salutation},</p>`, | ||
| `<p>Attached is invoice <strong>${input.invoiceNumber}</strong> for deliveries completed between <strong>${input.periodStart}</strong> and <strong>${input.periodEnd}</strong>.</p>`, | ||
| `<p>Total due: <strong>${input.subtotalLabel}</strong><br/>Payment due by: <strong>${input.dueAt}</strong></p>`, | ||
| '<p>Please contact operations if anything needs review.</p>', | ||
| ].join(''), |
There was a problem hiding this comment.
buildInvoiceEmailContent() interpolates untrusted values (client/contact names, dates, amounts) directly into the HTML string. If any of these contain <, &, etc., the email HTML can break or be abused for HTML injection. Escape dynamic fields before inserting into the HTML body (or use a templating approach that auto-escapes).
| - app bootstrap: [src/main.tsx](/home/bernard/Work/way-bills/apps/frontend/src/main.tsx) | ||
| - shared shell: [src/components/AppLayout.tsx](/home/bernard/Work/way-bills/apps/frontend/src/components/AppLayout.tsx) | ||
| - main pages: [src/pages](/home/bernard/Work/way-bills/apps/frontend/src/pages) | ||
| - shared API/types: [src/lib](/home/bernard/Work/way-bills/apps/frontend/src/lib) |
There was a problem hiding this comment.
These links point to an absolute local path (/home/bernard/Work/way-bills/...), which won’t resolve in the repository. Use repo-relative links (e.g. src/main.tsx, src/components/AppLayout.tsx) so the README works on GitHub.
| - API server: [src/index.ts](/home/bernard/Work/way-bills/apps/backend/src/index.ts) | ||
| - worker: [src/worker.ts](/home/bernard/Work/way-bills/apps/backend/src/worker.ts) | ||
| - worker health: [src/worker-health.ts](/home/bernard/Work/way-bills/apps/backend/src/worker-health.ts) | ||
| - schema: [src/db/schema.ts](/home/bernard/Work/way-bills/apps/backend/src/db/schema.ts) |
There was a problem hiding this comment.
These links point to an absolute local path (/home/bernard/Work/way-bills/...), which won’t resolve in the repository. Use repo-relative links so the README works on GitHub.
| export async function sendInvoiceEmail(invoiceId: string) { | ||
| const detail = await getInvoiceDetail(invoiceId) | ||
|
|
||
| assert(detail.status !== 'void', new AppError(409, 'invoice_void', 'Voided invoices cannot be emailed.')) | ||
| assert( | ||
| detail.client.contactEmail, | ||
| new AppError(409, 'missing_client_email', 'This client does not have a billing email address yet.'), | ||
| ) | ||
|
|
||
| await db | ||
| .update(invoices) | ||
| .set({ | ||
| emailStatus: 'queued', | ||
| emailDeliveryAttempts: detail.emailDeliveryAttempts + 1, | ||
| lastEmailError: null, | ||
| }) | ||
| .where(eq(invoices.id, detail.id)) | ||
|
|
||
| try { | ||
| const pdfBytes = await buildInvoicePdf(toInvoicePdfDetail(detail)) | ||
| const content = buildInvoiceEmailContent({ | ||
| invoiceNumber: detail.invoiceNumber, | ||
| clientName: detail.client.name, | ||
| contactName: detail.client.contactName, | ||
| recipientEmail: detail.client.contactEmail, | ||
| periodStart: formatDateLabel(detail.periodStart.toISOString()), | ||
| periodEnd: formatDateLabel(detail.periodEnd.toISOString()), | ||
| subtotalLabel: formatMoneyLabel(detail.subtotalCents, detail.currency), | ||
| dueAt: formatDateLabel(detail.dueAt.toISOString()), | ||
| attachmentBytes: pdfBytes, | ||
| }) | ||
|
|
||
| await sendInvoiceEmailMessage(content) | ||
|
|
||
| await db | ||
| .update(invoices) | ||
| .set({ | ||
| emailStatus: 'sent', | ||
| emailSentAt: new Date(), | ||
| lastEmailError: null, | ||
| }) | ||
| .where(eq(invoices.id, detail.id)) | ||
| } catch (error) { | ||
| await db | ||
| .update(invoices) | ||
| .set({ | ||
| emailStatus: 'failed', | ||
| lastEmailError: error instanceof Error ? error.message : 'Unknown email delivery failure.', | ||
| }) | ||
| .where(eq(invoices.id, detail.id)) | ||
|
|
||
| await createRoleNotifications(['admin', 'ops'], { | ||
| type: 'invoice_email_failed', | ||
| title: 'Invoice email failed', | ||
| message: `Email delivery failed for ${detail.invoiceNumber}.`, | ||
| linkPath: '/ops/invoices', | ||
| eventKey: `invoice_email_failed:${detail.id}`, | ||
| }) | ||
|
|
||
| throw error | ||
| } | ||
|
|
||
| return { | ||
| invoice: serializeInvoiceDetail(await getInvoiceDetail(invoiceId)), | ||
| } | ||
| } | ||
|
|
||
| export async function deliverPendingAutomaticInvoiceEmails() { | ||
| const pending = await db.query.invoices.findMany({ | ||
| where: and( | ||
| eq(invoices.source, 'automatic'), | ||
| eq(invoices.status, 'issued'), | ||
| inArray(invoices.emailStatus, ['not_sent', 'failed']), | ||
| ), | ||
| orderBy: asc(invoices.issuedAt), | ||
| }) | ||
|
|
||
| let sentCount = 0 | ||
| let failedCount = 0 | ||
|
|
||
| for (const invoice of pending) { | ||
| try { | ||
| await sendInvoiceEmail(invoice.id) | ||
| sentCount += 1 | ||
| } catch { | ||
| failedCount += 1 | ||
| } | ||
| } | ||
|
|
||
| return { | ||
| scanned: pending.length, | ||
| sentCount, | ||
| failedCount, | ||
| } | ||
| } |
There was a problem hiding this comment.
sendInvoiceEmail() and deliverPendingAutomaticInvoiceEmails() introduce fairly involved state transitions (queued/sent/failed, attempt counting, lastEmailError, role notifications on failure). There are new unit tests for other automation helpers in this PR, but none covering these email workflows. Adding tests for success/failure cases (including void/missing email guards) would help prevent regressions.