Database Monitoring: The Metrics That Actually Matter

1. Query latency (p95/p99) — the percentile matters more than the average. A 500ms average can hide 10-second outliers.

2. Connection pool utilization — at 80%+ you're at risk of connection refused errors. At 100% your app starts timing out.

3. Replication lag (if using replicas) — lag >10s means reads from replicas are stale. Lag growing over time signals a problem.

4. Slow query count — queries taking >1s per minute. Trend matters more than absolute count.

5. Disk usage % — alert at 70%, act before 85%.

6. Lock wait time — excessive lock contention is a sign of schema or query design problems.

Create a lightweight health endpoint that runs a simple query: ```python @app.route('/health/db') def db_health(): try: db.execute('SELECT 1') return {'status': 'ok'}, 200 except Exception as e: return {'status': 'error', 'detail': str(e)}, 503 ```

Monitor this endpoint on AlertsDock with a 1-minute interval.

Use a heartbeat cron monitor to verify backups run: ```bash # In your backup script: pg_dump mydb | gzip > backup.gz && \ curl -fsS https://alertsdock.com/ping/{uuid} ```

If the ping doesn't fire, the backup failed — you'll know within the configured interval.

Record p99 query latency daily. When a deploy causes a regression, you have historical data to compare against and can roll back with confidence.

Storage failures give you warning signs. Watch growth rate — if your database is growing 5% per week, you can calculate exactly when you'll hit 85% capacity and provision more storage in advance.