CompanySite/.agents/devops-engineer/CI_CD_PIPELINE.md

CI/CD Pipeline — WorkRoot Website

Production deployment pipeline documentation for the WorkRoot website. Framework: Astro SSR + Express | Platform: GitHub Actions

---

Overview

The CI/CD system consists of two complementary pipelines:

Workflow	File	Purpose
E2E Test Suite	.github/workflows/e2e-tests.yml	Runs on every push/PR — 9 test jobs
Deploy to Production	.github/workflows/deploy.yml	Deploys on merge to main

---

Pipeline Architecture

Push to main
     │
     ├─── E2E Test Suite (parallel) ──────────────────────────────────┐
     │    ├── Smoke Tests (P0)                                         │
     │    ├── Critical Paths                                           │
     │    └── API Integration                                          │
     │                                                                 │
     └─── Deploy Pipeline ────────────────────────────────────────────┘
          │
          ▼
    [Job 1] Build & Verify
          │  ✓ npm ci
          │  ✓ tsc --noEmit
          │  ✓ npm run build
          │  ✓ Verify dist/ structure
          │  ✓ Upload build artifact
          │
          ▼
    [Job 2] Pre-Deploy Tests
          │  ✓ Download build artifact
          │  ✓ Start server locally
          │  ✓ Run smoke tests (Chromium)
          │  ✓ Run API integration tests
          │
          ▼
    [Job 3] Deploy (one of:)
          ├── Railway (DEPLOY_TARGET=railway)
          ├── Render  (DEPLOY_TARGET=render)
          ├── VPS/PM2 (DEPLOY_TARGET=vps)
          └── Fly.io  (DEPLOY_TARGET=fly)
          │
          ▼
    [Job 4] Post-Deploy Verification
          │  ✓ Smoke tests against live production
          │  ✓ Critical endpoint checks
          │  ✓ Upload results (14-day retention)
          │
          ▼
    [Job 5] Notify on Failure (if any stage failed)

---

Deployment Targets

Option A: Railway (Recommended for simplicity)

Railway auto-deploys from GitHub. CI/CD adds a verification layer.

Required secrets:

Secret	Value
RAILWAY_TOKEN	From Railway dashboard → Settings → Tokens

Required variables:

Variable	Value
DEPLOY_TARGET	railway

Setup:

1. Create a Railway project, connect the GitHub repo
2. Set START_COMMAND: npm run start:prod
3. Set PORT: 10000
4. Add the RAILWAY_TOKEN secret to GitHub
5. Set DEPLOY_TARGET=railway in GitHub repo variables

---

Option B: Render

Render uses deploy hooks triggered by the CI pipeline.

Required secrets:

Secret	Value
RENDER_DEPLOY_HOOK_URL	From Render dashboard → Service → Deploy Hook URL

Required variables:

Variable	Value
DEPLOY_TARGET	render

Render service configuration:

• Environment: Node
• Build command: npm ci && npm run build
• Start command: npm run start:prod
• Health check path: /api/health.json

---

Option C: VPS with PM2 (Full Control)

SSH-based deployment to any Linux VPS. Includes automatic rollback.

Required secrets:

Secret	Description
VPS_HOST	VPS IP or hostname
VPS_USER	SSH user (e.g., ubuntu, deploy)
VPS_SSH_PRIVATE_KEY	Private key (contents of ~/.ssh/id_rsa)
VPS_HOST_KEY	SSH host key fingerprint

Required variables:

Variable	Value
DEPLOY_TARGET	vps

VPS setup (one-time):

# On your VPS:
# 1. Create deployment directory
mkdir -p /var/www/workroot
chown -R deploy:www-data /var/www/workroot

# 2. Install Node.js 20 (via nvm or nodesource)
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs

# 3. Install PM2 globally
npm install -g pm2
pm2 startup  # Follow the output instructions

# 4. Create logs directory
mkdir -p /var/www/workroot/logs

# 5. Set up Nginx reverse proxy (port 80/443 → 10000)
# See Nginx config below

Nginx configuration:

server {
    listen 80;
    server_name workroot.in www.workroot.in;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name workroot.in www.workroot.in;

    ssl_certificate /etc/letsencrypt/live/workroot.in/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/workroot.in/privkey.pem;

    location / {
        proxy_pass http://localhost:10000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }
}

Deployment behavior:

1. Backs up current deployment before replacing
2. Extracts new files to /var/www/workroot/
3. Installs production dependencies only
4. Uses pm2 reload for zero-downtime restart
5. Runs health check (12 attempts × 10s = 2 minutes max)
6. Auto-rollback if health check fails

---

Option D: Fly.io

Container-based deployment with edge distribution.

Required secrets:

Secret	Value
FLY_API_TOKEN	From flyctl auth token

Required variables:

Variable	Value
DEPLOY_TARGET	fly

Setup:

# Install flyctl
curl -L https://fly.io/install.sh | sh

# Authenticate
flyctl auth login

# Launch app (first time only)
flyctl launch --name workroot-website

# Set production secrets
flyctl secrets set NODE_ENV=production PORT=10000

fly.toml (create in project root if using Fly.io):

app = "workroot-website"
primary_region = "sin"  # Singapore - closest to India

[build]
  [build.args]
    NODE_VERSION = "20"

[env]
  PORT = "10000"
  HOST = "0.0.0.0"
  NODE_ENV = "production"

[http_service]
  internal_port = 10000
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 1

  [http_service.concurrency]
    type = "connections"
    hard_limit = 25
    soft_limit = 20

[[vm]]
  cpu_kind = "shared"
  cpus = 1
  memory_mb = 512

---

GitHub Repository Setup

Required Secrets (Settings → Secrets → Actions)

Configure secrets for your chosen deployment target:

# For ALL targets:
# (none required at the base level)

# For Railway:
RAILWAY_TOKEN=<token>

# For Render:
RENDER_DEPLOY_HOOK_URL=https://api.render.com/deploy/...

# For VPS:
VPS_HOST=123.456.789.0
VPS_USER=deploy
VPS_SSH_PRIVATE_KEY=-----BEGIN OPENSSH PRIVATE KEY-----...
VPS_HOST_KEY=123.456.789.0 ssh-rsa AAAA...

# Optional notifications:
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...

Required Variables (Settings → Variables → Actions)

DEPLOY_TARGET=railway  # or: render, vps, fly

Environment Protection Rules

Configure via Settings → Environments:

1. Create environment named production
2. Enable "Required reviewers" for manual approval before deploy
3. Set allowed branches to main only

---

Trigger Conditions

Event	Build	Tests	Deploy
Push to main	✓	✓	✓
Manual dispatch	✓	✓ (unless skip_tests=true)	✓
Push to develop	via e2e-tests.yml	✓	✗
Pull Request	via e2e-tests.yml	✓	✗

---

Environment Variables in Production

All env vars must be configured in your hosting platform, not in the workflow.

Variable	Required	Default	Description
NODE_ENV	Yes	—	Must be production
HOST	Yes	—	0.0.0.0
PORT	Yes	—	10000
CONTACT_EMAIL	Yes (legacy)	—	Legacy admin recipient for contact form; kept for backward compatibility with older code paths. Mirror of MAIL_ADMIN_TO.
MAIL_ADMIN_TO	Yes	—	Address that receives "Tell us about your project" and contact form submissions.
SMTP_HOST	Yes	—	SMTP server hostname (e.g. smtp.hostinger.com).
SMTP_PORT	Yes	587	SMTP server port. Use 587 for STARTTLS, 465 for implicit TLS.
SMTP_USER	Yes	—	SMTP auth username (full email address).
SMTP_PASS	Yes	—	SMTP auth password — MUST be stored as a platform secret, never plaintext.
SMTP_FROM	Yes	—	From: address used by outbound mail (should match SMTP_USER or an allowed alias).
SMTP_FROM_NAME	Yes	—	Human-readable display name shown alongside SMTP_FROM.
SMTP_SECURE	Yes	false	true only for implicit-TLS ports (465). For port 587 keep false and rely on STARTTLS.
SMTP_STARTTLS	Yes	true	Negotiate STARTTLS on the SMTP connection.
SMTP_REQUIRE_TLS	Yes	true	Refuse to send if the server does not advertise STARTTLS — prevents silent plaintext fallback.
SMTP_AUTH	Yes	true	Authenticate with SMTP_USER / SMTP_PASS.
SMTP_CONNECTION_TIMEOUT_MS	No	10000	Socket connect timeout in milliseconds.
SMTP_GREETING_TIMEOUT_MS	No	10000	Wait time for the server greeting (EHLO/HELO) in milliseconds.
SMTP_SOCKET_TIMEOUT_MS	No	10000	Idle socket timeout in milliseconds (covers read/write while transferring the message).
SENTRY_DSN	No	—	Sentry error tracking
LOG_LEVEL	No	info	Logging verbosity

Production SMTP is provisioned on Hostinger and configured in the Default environment via the platform's env-var store (SMTP_PASS flagged as secret). After changing any SMTP_* value you MUST restart the frontend service so the new variables are picked up by the Astro SSR process.

---

Rollback Procedures

Railway / Render / Fly.io

Use the hosting platform dashboard to redeploy a previous commit or use the rollback button.

VPS (PM2)

The deployment script auto-rolls back if health checks fail. For manual rollback:

# SSH to VPS
ssh deploy@your-vps-ip

# Check what backups exist
ls -la /var/www/ | grep workroot-backup

# Rollback to most recent backup
BACKUP=$(ls -t /var/www/ | grep workroot-backup | head -1)
echo "Rolling back to: $BACKUP"

# Stop, restore, restart
pm2 stop workroot-website
cp -r /var/www/$BACKUP/. /var/www/workroot/
cd /var/www/workroot && npm ci --omit=dev
pm2 start ecosystem.config.cjs --env production
pm2 save

# Verify
curl http://localhost:10000/api/health.json

Emergency Deploy (Skip Tests)

Use workflow_dispatch with skip_tests: true only in genuine emergencies. Document the reason in the workflow run description.

---

Monitoring After Deploy

Immediate (0–5 minutes)

• Workflow shows all jobs green
• Health endpoint: https://workroot.in/api/health.json returns {"status":"ok"}
• Homepage loads correctly
• Contact form accessible

Short-term (15–60 minutes)

• No spike in error logs
• PM2 showing stable process count (VPS only)
• Nightly E2E suite runs clean

Verification Commands (VPS)

# Check PM2 status
pm2 status
pm2 logs workroot-website --lines 50

# Check Nginx logs
sudo tail -f /var/log/nginx/access.log
sudo tail -f /var/log/nginx/error.log

# Health check
curl -s http://localhost:10000/api/health.json | python3 -m json.tool

---

Pipeline Files Reference

File	Purpose
.github/workflows/deploy.yml	Main deployment pipeline
.github/workflows/e2e-tests.yml	Test suite (smoke, critical, API, chaos, cross-browser, mobile, security)
.github/workflows/sitemap-ping.yml	Pings Google/Bing after content changes
ecosystem.config.cjs	PM2 cluster configuration
server.mjs	Express wrapper for Astro SSR
.env.example	Environment variable template

---

Adding a New Deployment Target

1. Add a new job block in deploy.yml after the existing deploy jobs
2. Add the condition: vars.DEPLOY_TARGET == 'your-target'
3. Add required secrets to this document
4. Update the post-deploy-verify job's needs array to include the new job
5. Test with a manual workflow_dispatch trigger

---

Last updated: 2026-03-21 | Created by: devops-engineer agent