How we observe the Unified Mailer stack

We observe the Unified Mailer stack on dardev-vps with layered health checks, mail-specific signals, and Telegram alerts that fire only when a human must act. The stack—Twenty CRM, Listmonk campaigns, Stalwart MTA, and our sync worker—is documented in inside-unified-mailer-stack; this article explains what we measure, what we ignore, and how we avoid waking on-call for vanity metrics.

DarDev runs this architecture for our own outreach and news.dardev.net subscribe flows. Observability here is not a separate product demo—it is how we know a Hesabi CRM import did not break webhook delivery before a Monday campaign, or that Stalwart is not queuing thousands of messages behind a TLS misconfiguration.

What sits on the VPS

• Twenty (crm.dardev.net, localhost :3001): CRM source of truth, /healthz
• Listmonk (mail.dardev.net, :9002): campaigns, bounces, list hygiene
• Stalwart (mta.dardev.net / send.dardev.net, :8080): SMTP submission and outbound queue
• Sync worker (:3100): Twenty webhooks → Listmonk lists, /health and /api/news/health
• PostgreSQL + Redis: shared persistence behind Docker Compose with healthchecks

Host nginx terminates TLS; compose nginx is disabled on dardev-vps so host nginx owns TLS termination. That split matters for monitoring: public HTTPS checks and localhost probes tell different stories during partial failures.

Layer 1: container and endpoint health

Docker Compose defines healthchecks for Postgres (pg_isready), Redis (PING), and Twenty server (curl /healthz). After every deploy we run validate-all.sh over SSH—it confirms each dardev-mailer container is running, curls Twenty /healthz on 3001, Listmonk admin on 9002, sync worker /health on 3100, and Stalwart admin on 8080, then walks public HTTPS for crm, mail, and news.

The sync worker exposes two probes operators care about: GET /health returns {"ok": true} for process liveness, and GET /api/news/health validates the news subscribe API path end-to-end. verify-news.sh checks both localhost and https://news.dardev.net/api/news/health through nginx—subscribe forms fail silently if only the container health passes.

Layer 2: mail signals that matter

HTTP green does not mean mail is healthy. Before scaling outreach we review three mail-specific signals together.

1. Listmonk bounce rate: query the bounces table and campaign stats; sudden spikes after a CRM import usually mean stale addresses or consent gaps—not SMTP failure
2. Stalwart queue depth: docker logs on stalwart filtered for queue, deliver, reject; backlog growth with flat send volume indicates relay or auth problems on port 465 via host.docker.internal
3. Campaign send vs to_send: Listmonk Postgres shows campaigns stuck in running with sent ≪ to_send—often SMTP password drift or Listmonk smtp settings out of sync with.env

Scripts like check-listmonk-logs.sh and check-subscriber-lists.sh wrap these checks for operators who do not live in psql. We run them after bulk Twenty imports (OECT, Startup Tunisia SMB) and before Listmonk campaign start—not as daily cron noise.

Layer 3: metrics and Grafana

node_exporter on dardev-vps feeds Prometheus: CPU, memory, disk, and nginx access patterns. Application golden signals come from log scraping and manual campaign review rather than a custom Listmonk exporter—we accept that trade-off on a single VPS until volume justifies more instrumentation.

Dashboard layout follows grafana-dashboards-small-teams: one row for VPS saturation, one for HTTP error rate on public subdomains, one for cert expiry. Mail-specific panels are operational runbook links plus weekly exported Listmonk campaign CSV—not duplicated time series we would not page on.

Telegram alert routing

Critical alerts route to Telegram, not email—SMTP alerting during an SMTP outage is a circular dependency. We page on symptoms users feel: disk above seventy percent with four-hour runway, sustained 5xx on crm or mail through nginx, TLS cert expiring within fourteen days, and validate-all.sh FAIL after deploy.

We do not Telegram-alert on single failed healthcheck retries, one-off Stalwart reject lines, or CPU spikes during Twenty bulk import—those belong in logs and the weekly review. Alert fatigue on a two-person platform team is indistinguishable from no monitoring.

• Every alert includes a runbook link (docs/runbooks/TROUBLESHOOTING.md in the monorepo)
• Deploy annotations in Grafana mark when bounce baselines may legitimately shift
• Non-critical warnings (GraphQL 401 during key rotation) go to email digest, not Telegram

Weekly operator review

Before large outreach waves, one operator runs validate-all.sh, skim Stalwart queue lines, and open Listmonk analytics for the prior seven days. CRM segment counts in Twenty should move in the same direction as Listmonk list growth after opt-in events—divergence means webhook or PRODUCT_LIST_MAP drift.

Twenty API rate limits for bulk import are patched in Postgres, not only.env—monitoring import jobs means watching Twenty worker logs and import lock files under data/.twenty-crm-import.lock, not just HTTP uptime.

What we deliberately do not monitor

Per-recipient open tracking as an SLO, sub-second webhook latency histograms, and cross-region failover are out of scope for this VPS-class deployment. Hesabi fiscal uptime and TTN clearance live in separate product infrastructure—see logs-metrics-traces for when to add traces versus logs for API-heavy systems.

How DarDev helps

We implement observability stacks for MENA platform teams—Prometheus, Grafana, alert routing, and runbooks tied to real deploy paths. Platform overview: dardev.net/products. If you are replicating the Unified Mailer pattern, start with inside-unified-mailer-stack for architecture, then copy the health and mail-signal layers before importing prospect CSVs.