Health Checks & Graceful Shutdown
Health Check Endpoint (Express)
app.get('/health/live', (req, res) => {
res.json({ status: 'ok' }); // App process is running
});
app.get('/health/ready', async (req, res) => {
const checks = await Promise.allSettled([
checkDatabase(),
checkRedis(),
checkExternalApi(),
]);
const results = checks.map((c, i) => ({
name: ['database', 'redis', 'external-api'][i],
status: c.status === 'fulfilled' ? 'up' : 'down',
...(c.status === 'rejected' && { error: c.reason.message }),
}));
const allHealthy = results.every((r) => r.status === 'up');
res.status(allHealthy ? 200 : 503).json({ status: allHealthy ? 'ready' : 'degraded', checks: results });
});
async function checkDatabase() {
await db.$queryRaw`SELECT 1`;
}
async function checkRedis() {
await redis.ping();
}
Graceful Shutdown (Node.js)
const server = app.listen(3000);
async function shutdown(signal: string) {
console.log(`${signal} received, starting graceful shutdown`);
// 1. Stop accepting new connections
server.close();
// 2. Wait for in-flight requests (with timeout)
const timeout = setTimeout(() => process.exit(1), 30000);
try {
// 3. Close dependencies
await db.$disconnect();
await redis.quit();
await queue.close();
clearTimeout(timeout);
process.exit(0);
} catch {
process.exit(1);
}
}
process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));
Kubernetes Probes
containers:
- name: app
livenessProbe:
httpGet: { path: /health/live, port: 3000 }
initialDelaySeconds: 10
periodSeconds: 15
failureThreshold: 3
readinessProbe:
httpGet: { path: /health/ready, port: 3000 }
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 2
startupProbe:
httpGet: { path: /health/live, port: 3000 }
failureThreshold: 30
periodSeconds: 2
lifecycle:
preStop:
exec:
command: ["sh", "-c", "sleep 5"] # Allow LB to drain
Spring Boot Actuator
management:
endpoints:
web:
exposure:
include: health,info
endpoint:
health:
show-details: when_authorized
group:
readiness:
include: db,redis,diskSpace
liveness:
include: ping
health:
redis:
enabled: true
Probe Types
| Probe | Purpose | Failure Action |
|---|
| Liveness | Is the process alive? | Restart container |
| Readiness | Can it serve traffic? | Remove from load balancer |
| Startup | Has it finished initializing? | Don't check liveness yet |
Anti-Patterns
| Anti-Pattern | Fix |
|---|
| Liveness check hits database | Liveness = process alive only; readiness = dependencies |
| No graceful shutdown | Handle SIGTERM, drain connections |
| Immediate process.exit() | Wait for in-flight requests to complete |
| No startup probe | Slow-starting apps killed by liveness before ready |
| Health check returns 200 when degraded | Return 503 when dependencies are down |
Production Checklist
- Separate liveness and readiness endpoints
- Readiness checks all critical dependencies
- Graceful shutdown handles SIGTERM
- In-flight requests drained before exit
- Kubernetes probes configured with proper thresholds
- Startup probe for slow-initializing apps
