Zero-Downtime Deployment for Crypto Payment Services: Strategies and Gotchas

The Deployment Risk

Unlike a stateless web app, a crypto payment service has in-flight work at every moment — active sessions, pending sweeps, queued jobs. A hard restart during a sweep could leave a deposit address partially funded.

Graceful Shutdown

Paychainly handles SIGTERM gracefully. When it receives the signal, it:

1. Stops accepting new blocks into the pipeline
2. Waits for in-progress queue jobs to complete
3. Checkpoints the last processed block
4. Exits cleanly

This means a rolling restart with a reasonable timeout (30–60 seconds) is safe.

PM2 Zero-Downtime Reload

# ecosystem.config.js
module.exports = {
  apps: [{
    name: 'paychainly',
    script: 'dist/main.js',
    kill_timeout: 60000,  // Wait 60s for graceful shutdown
    wait_ready: true,     // Wait for app to signal ready
    listen_timeout: 30000,
  }]
};

# Deploy command
pm2 reload paychainly --update-env

Blue-Green with NGINX

Run two instances on different ports (3002 and 3003). Deploy to the idle instance, verify health, then switch NGINX upstream. Switch takes milliseconds.

# nginx.conf
upstream paychainly {
  server 127.0.0.1:3002;  # Switch between 3002/3003
}

Queue Job Continuity

BullMQ jobs are persisted in Redis. If a job is in-progress when the process restarts, BullMQ automatically retries it on the next startup. The sweep processor's idempotency (unique txHash constraint) prevents double processing.