CompanySite/.agents/devops-engineer/MONITORING_SETUP.md

Monitoring & Uptime Alerts — WorkRoot Website

Production monitoring setup for workroot.in

---

Overview

Layer	Tool	Coverage
Active checks	GitHub Actions (every 5 min)	Uptime, response time, pages, SSL
External uptime	UptimeRobot (free tier)	HTTP 200, keyword, SSL cert
Health endpoint	/api/health.json	Server status, memory, uptime
Metrics endpoint	/api/metrics.json	Request counts, response times, error rate

---

Files Created / Modified

File	Purpose
src/pages/api/health.json.ts	Enhanced — now includes uptime, memory usage, version
src/pages/api/metrics.json.ts	New — request metrics, response time percentiles, error rate
.github/workflows/uptime-monitor.yml	New — runs every 5 min via GitHub Actions cron
scripts/setup-uptimerobot.sh	New — automates UptimeRobot monitor creation

---

Health Endpoint

URL: https://workroot.in/api/health.json

Response:

{
  "status": "ok",
  "timestamp": "2026-03-21T10:00:00.000Z",
  "uptime": 86400,
  "version": "1.0.0",
  "mode": "ssr",
  "adapter": "node-standalone",
  "domain": "workroot.in",
  "memory": {
    "heapUsedMB": 45,
    "heapTotalMB": 64,
    "rssMB": 82
  },
  "checks": {
    "server": "ok"
  }
}

Used by: CI/CD pipeline, uptime monitors, load balancers.

---

Metrics Endpoint

URL: https://workroot.in/api/metrics.json

Authentication: Optional. Set METRICS_TOKEN env var to require Authorization: Bearer <token>.

Response:

{
  "timestamp": "2026-03-21T10:00:00.000Z",
  "uptime": { "seconds": 86400, "human": "1d 0h 0m 0s" },
  "requests": { "total": 1250, "errors": 3, "errorRate": "0.24%" },
  "responseTime": { "avgMs": 145, "p50Ms": 120, "p95Ms": 380, "p99Ms": 750, "samples": 100 },
  "memory": { "heapUsedMB": 45, "heapTotalMB": 64, "externalMB": 2, "rssMB": 82 },
  "process": { "pid": 1234, "nodeVersion": "v20.0.0", "platform": "linux" }
}

---

GitHub Actions Uptime Monitor

File: .github/workflows/uptime-monitor.yml

Schedule: Every 5 minutes (*/5 * * * *)

What It Checks

Check	Threshold	Alert
Health endpoint HTTP 200	Must be 200	Failure → alert job runs
Health status field	Must be "ok"	Failure → alert job runs
Response time	< 3000ms	Slow → alert job runs
Critical pages (/, /services, /portfolio, /contact, /about)	HTTP 200	Failure → job fails
Sitemap + robots.txt	HTTP 200	Failure → job fails
SSL certificate	> 14 days remaining	Failure → alert job runs

Alert Channels

Currently configured in the workflow as commented examples. To enable:

Slack Alerts

1. Create a Slack Incoming Webhook
2. Add secret: SLACK_WEBHOOK_URL in GitHub → Settings → Secrets → Actions
3. Uncomment the Slack notification block in .github/workflows/uptime-monitor.yml

Generic Webhook (email, PagerDuty, etc.)

1. Add secret: ALERT_WEBHOOK_URL
2. Uncomment the webhook notification block in .github/workflows/uptime-monitor.yml

Viewing Results

• GitHub → Actions → "Uptime Monitor" tab shows every run
• Failed runs = site is down or degraded
• Each run summary shows response times and SSL days remaining

---

UptimeRobot Setup (External Monitoring)

UptimeRobot provides monitoring from external IPs, independent of GitHub Actions.

Quick Setup

# Set your API key (from UptimeRobot dashboard → My Settings → API Settings)
export UPTIMEROBOT_API_KEY="ur_xxxxxxxxxxxxxxxx"

# Optional: set alert email
export ALERT_EMAIL="alerts@workroot.in"

# Run setup script
bash scripts/setup-uptimerobot.sh

Manual Setup (Free Tier)

1. Sign up at https://uptimerobot.com (free)
2. Create monitors:

Monitor Name	URL	Type	Interval
WorkRoot Health	https://workroot.in/api/health.json	HTTP(s)	5 min
WorkRoot Homepage	https://workroot.in/	HTTP(s)	5 min
WorkRoot Health Keyword	https://workroot.in/api/health.json	Keyword	5 min
WorkRoot Services	https://workroot.in/services	HTTP(s)	5 min
WorkRoot Contact	https://workroot.in/contact	HTTP(s)	5 min

3. For keyword monitor: keyword = "status":"ok", type = "Exists"
4. Enable SSL monitoring on each HTTPS monitor:
   • Edit monitor → Advanced → SSL monitoring: ON
   • Alert threshold: 14 days before expiry
5. Set response time alert: Edit → Alert when response time > 3000ms
6. Configure alert contacts: Alert Contacts → Add Email/Slack/Webhook

Status Page

Create a public status page:

• UptimeRobot Dashboard → Status Pages → Create New
• Add all monitors
• Set URL: status.workroot.in (add CNAME DNS record)

---

Alert Thresholds Reference

Metric	Warning	Critical
Response time	> 2000ms	> 3000ms
SSL expiry	< 30 days	< 14 days
Memory (heap)	> 80%	> 95%
Error rate	> 1%	> 5%
Downtime	1 failed check	2+ consecutive

---

Incident Response Runbook

Site Down (HTTP non-200 or timeout)

1. Check GitHub Actions → Uptime Monitor for recent failures
2. Check UptimeRobot → Incidents for start time and location
3. SSH to server: ssh deploy@<VPS_HOST>
4. pm2 status                              # Is process running?
5. pm2 logs workroot-website --lines 50    # Check for crash errors
6. curl http://localhost:10000/api/health.json  # Direct check
7. If crashed: pm2 restart workroot-website
8. If persistent: trigger rollback (see CI/CD pipeline docs)

Slow Response (> 3s)

1. Check /api/metrics.json for memory and error rate
2. pm2 monit                               # Real-time CPU/memory
3. Check for memory leak: heapUsedMB trending up?
4. Check Nginx logs: sudo tail -f /var/log/nginx/access.log
5. If memory issue: pm2 restart workroot-website (graceful)
6. Consider scaling: increase PM2 cluster instances

SSL Certificate Expiring

1. SSH to VPS
2. Check cert: echo | openssl s_client -connect workroot.in:443 2>/dev/null | openssl x509 -noout -dates
3. Renew with Certbot: sudo certbot renew --nginx
4. Verify renewal: sudo certbot certificates
5. Reload Nginx: sudo nginx -s reload

High Error Rate

1. Check /api/metrics.json → requests.errorRate
2. pm2 logs workroot-website --err --lines 100
3. Check Sentry dashboard for exception details
4. Identify error pattern (specific endpoint? all routes?)
5. Deploy hotfix or rollback if regression

---

Environment Variables for Monitoring

Add to .env (production) or hosting platform secrets:

# Optional: protect the /api/metrics.json endpoint
METRICS_TOKEN=your-secure-random-token-here

---

Dashboard Quick Links

Resource	URL
Health endpoint	https://workroot.in/api/health.json
Metrics endpoint	https://workroot.in/api/metrics.json
GitHub Actions	https://github.com///actions/workflows/uptime-monitor.yml
UptimeRobot	https://uptimerobot.com/dashboard
UptimeRobot Status Page	https://status.workroot.in (after setup)

---

Created by: devops-engineer agent | Date: 2026-03-21