news.dardev.net architecture (static + sync worker)

news.dardev.net is a fully static news site: HTML, CSS, JavaScript, and JSON files served by host nginx on dardev-vps. There is no application server inside the dardev-news folder. The only server-side behavior—newsletter subscribe and the contact form—is handled by the Twenty→Listmonk sync worker, which nginx proxies at /api/news/*.

That split is deliberate. Articles change often; forms touch CRM and Listmonk with secrets that must never ship in static files. Content deploys as rsync’d files. Form traffic hits a small Node service on port 3100 that validates input, applies honeypots, and calls upstream APIs.

Static layer: what ships in dardev-news/

The site root holds index.html for the article list, pre-built article pages under article/{slug}.html, shared assets, and two JSON data files. site.json defines company URLs, solution links, analytics ID, and form API base path. articles.json holds every post: slug, title, date, category, excerpt, SEO fields, cover image path, and a typed body[] array.

• index.html + assets/js/home.js — category filters and article cards loaded from JSON
• article/{slug}.html — server-rendered HTML for crawlers (built by build-static-articles.py)
• data/articles.json — canonical content store after merge-draft-to-json.py
• assets/media/ — cover and inline images referenced by relative paths
• sitemap.xml, rss.xml, robots.txt — generated alongside static article pages

Legacy article.html?slug= URLs 301 to the static path so bookmarks and old links keep working. Client-side JS still hydrates related-product links from site.json when a post sets solution: hesabi or dardevops, but the HTML shell and JSON-LD Article schema are present before JavaScript runs—important for SEO and social previews.

Dynamic layer: sync worker behind nginx

Browsers POST subscribe and contact payloads to same-origin /api/news/subscribe and /api/news/contact. Host nginx (not the Docker Compose nginx service— that is disabled on dardev-vps) forwards those paths to the sync-worker container. The worker validates email format, rejects filled honeypot fields silently, and returns JSON the static forms.js expects.

Subscribe calls Listmonk’s public subscription API with the DarDev — Company News list UUID stored in NEWS_LIST_UUID. Listmonk sends double opt-in mail through Stalwart on send.dardev.net—never from @dardev.net root, which stays on Zoho for inbound staff mail. Contact creates or updates a person in Twenty CRM, optionally links a company, and attaches a note tagged news.dardev.net contact form. Contact does not add anyone to a marketing list; sales follow-up stays in CRM.

Content pipeline: catalog to published HTML

Editorial work lives under content/. catalog.json tracks 100 briefs with slug, keyword, phase, and status. Agents or humans write content/drafts/{slug}.md (YAML frontmatter) plus {slug}.blocks.json (structured body: paragraphs, headings, lists, callouts, FAQ, images). validate-article.py enforces minimum word count, FAQ presence for guides, meta description length, cover image, and blocked claims before merge.

1. Draft

   Write.md frontmatter and.blocks.json body blocks; research notes optional in content/briefs/.

2. Validate

   python scripts/validate-article.py --draft content/drafts/{slug}.md — fix errors until OK.

3. Merge

   python scripts/merge-draft-to-json.py content/drafts/{slug}.md — updates data/articles.json and copies to content/published/.

4. Build static pages

   python scripts/build-static-articles.py — writes article/*.html, sitemap, RSS. deploy-news.sh runs this on deploy.

5. Deploy

   wsl bash scripts/deploy-news.sh rsyncs dardev-news/ to /opt/dardev-mailer on dardev-vps. Content-only deploys do not restart sync-worker.

Do not run generate-articles.py on a populated articles.json—it overwrites real posts. Covers are ensured under assets/media/{slug}-cover.jpg during merge when missing.

Production layout on dardev-vps

The mailer stack runs via docker compose -f docker-compose.yml -f docker-compose.dardev-vps.yml. news.dardev.net TLS terminates on host nginx, which serves static files from the deployed dardev-news tree and proxies API routes to dardev_sync. Twenty listens on localhost:3001 (crm.dardev.net), Listmonk on :9002 (mail.dardev.net), sync-worker on :3100. Stalwart admin and SMTP hostname use mta.dardev.net; outbound campaigns and opt-in mail use send.dardev.net with SPF, DKIM, and DMARC on GoDaddy.

This news site sits beside—not inside—the Unified Mailer stack described in Inside DarDev Unified Mailer: Twenty, Listmonk, Stalwart. Same CRM and Listmonk instances power product outreach lists elsewhere; news uses one public Company News list with double opt-in, separate from per-product private lists synced only after explicit opt-in.

Subscribe vs contact: two integrations, one worker

1. Reader submits email on footer subscribe form → POST /api/news/subscribe → Listmonk public API → confirmation email → opted-in subscriber
2. Reader submits contact form with name and message → POST /api/news/contact → Twenty REST → person + note for sales triage
3. CRM-driven product list adds (Hesabi, DarDevLab) use different paths and tags—see CRM-driven product lists without spamming cold leads
4. Cold prospect imports land in Twenty only until marketing consent; never auto-subscribe from CRM bulk import

Double opt-in mechanics, GDPR-friendly consent copy, and Listmonk template expectations are covered in Double opt-in subscribe flows that respect compliance. Local development needs python -m http.server; forms fail on file:// unless you proxy /api/news/ to a running worker.

Operations checklist

• After content merge: build-static-articles.py then deploy-news.sh
• After Listmonk list changes: wire-news-forms.sh and verify-news.sh
• Monitor sync-worker logs on spam spikes; honeypot _website field catches naive bots
• IndexNow ping optional via scripts/ping-indexnow.py when publishing batches
• Keep TWENTY_API_KEY and Listmonk admin tokens out of git and static JSON

When subscribe returns 500 with mail delivery errors, fix Stalwart SMTP and Listmonk outbound settings before blaming the static site. When contact returns duplicate email, Twenty already holds the person—the API surfaces a friendly message. Rate limiting on /api/news/ at nginx is recommended if forms attract abuse.

If you operate a similar static-plus-API pattern, keep the read path boring (files + CDN or nginx) and the write path narrow (one worker, two endpoints, explicit health checks). That is the architecture we run for news.dardev.net today.