Skip to content

Fix slow shutdown on SIGINT/SIGTERM #325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
brendanlong merged 3 commits into from
Apr 7, 2026
Merged

Fix slow shutdown on SIGINT/SIGTERM #325

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7474ae9
Fix slow shutdown on SIGINT/SIGTERM
claude Apr 7, 2026
a45daf6
Add 10s timeout for graceful shutdown
claude Apr 7, 2026
c6794d9
Make stopSession synchronous (no awaits needed)
claude Apr 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions src/instrumentation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,53 @@ export async function register() {
} catch (err) {
console.error('Error reconciling sessions:', err);
}

// Register graceful shutdown handler
registerShutdownHandler();
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In Next.js, the register function in instrumentation.ts can be executed multiple times during development (e.g., due to Hot Module Replacement). This would result in multiple SIGINT/SIGTERM listeners being attached to the process, leading to redundant cleanup operations and potential race conditions during shutdown. It is recommended to ensure the signal handlers are only registered once by using a global flag, ensuring that initialization logic is idempotent and protected against concurrent or repeated execution.

    if (!(globalThis as any)._shutdownHandlerRegistered) {
      registerShutdownHandler();
      (globalThis as any)._shutdownHandlerRegistered = true;
    }
References
  1. To prevent race conditions in methods with asynchronous initialization, set a processing/locked flag synchronously at the start of the method, before any await calls, to ensure concurrent invocations are rejected immediately.

}
}

/**
* Register signal handlers for graceful shutdown.
* Stops active Claude queries and disconnects Prisma so the process can exit cleanly.
*/
function registerShutdownHandler() {
let shuttingDown = false;

const shutdown = async (signal: string) => {
if (shuttingDown) {
console.log(`Received ${signal} again, forcing exit`);
process.exit(1);
}
shuttingDown = true;
console.log(`Received ${signal}, shutting down gracefully...`);

// Force exit after 10s if graceful shutdown hangs
// (important for SIGTERM from systemd where there's no second signal)
setTimeout(() => {
console.error('Graceful shutdown timed out, forcing exit');
process.exit(1);
}, 10_000).unref();

try {
// Stop all active Claude queries
const { stopAllSessions } = await import('@/server/services/claude-runner');
await stopAllSessions();
} catch (err) {
console.error('Error stopping sessions during shutdown:', err);
}

try {
// Disconnect Prisma
const { prisma } = await import('@/lib/prisma');
await prisma.$disconnect();
} catch (err) {
console.error('Error disconnecting Prisma during shutdown:', err);
}

process.exit(0);
};

process.on('SIGINT', () => void shutdown('SIGINT'));
process.on('SIGTERM', () => void shutdown('SIGTERM'));
}
3 changes: 2 additions & 1 deletion src/lib/rate-limiter.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,11 +212,12 @@ export class RateLimiter {
export const loginRateLimiter = new RateLimiter();

// Start cleanup interval (every hour)
// Use .unref() so this timer doesn't prevent graceful shutdown
if (typeof setInterval !== 'undefined') {
setInterval(
() => {
loginRateLimiter.cleanup();
},
60 * 60 * 1000
);
).unref();
}
13 changes: 12 additions & 1 deletion src/server/services/claude-runner.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -635,7 +635,7 @@ export async function markLastMessageAsInterrupted(sessionId: string): Promise<v
* Stop a session's Claude query and clean up state.
* Called when a session is stopped.
*/
export async function stopSession(sessionId: string): Promise<void> {
export function stopSession(sessionId: string): void {
const state = sessions.get(sessionId);
if (!state) return;

Expand All @@ -658,6 +658,17 @@ export async function stopSession(sessionId: string): Promise<void> {
sessions.delete(sessionId);
}

/**
* Stop all active Claude queries. Called during graceful shutdown.
*/
export async function stopAllSessions(): Promise<void> {
const sessionIds = [...sessions.keys()];
if (sessionIds.length === 0) return;

log.info('Stopping all active sessions for shutdown', { count: sessionIds.length });
await Promise.allSettled(sessionIds.map((id) => stopSession(id)));
}

/**
* Mark all running sessions as stopped.
* Called on server startup since all in-memory state is lost.
Expand Down
Loading