DarDev imports Tunisian schools from the public ecoles.com.tn directory into Twenty CRM for 20.tn outreach. The pipeline is CRM-only on import: no Listmonk subscription, consent marked unknown, and no marketing mail until a director opts in. It mirrors the ethics of Hesabi OECT and Startup Tunisia imports documented under crm-driven-product-lists and deployed-realities-explained.
Product: https://20.dardev.net. Ops solution id: 20tn. Scripts live in scripts/20tn-ecoles/. Shared CRM runs on dardev-vps beside the Unified Mailer stack described in inside-unified-mailer-stack.
Source inventory
Reconnaissance in June 2026 mapped ecoles.com.tn/etablissements-clubs: Drupal 11 Views, about 2,572 total listings across types, sixteen cards per page across roughly 161 paginated index pages. Listing cards expose name, level and ownership (for example Secondaire Privé, Primaire Public), street address, phone with high coverage, email on roughly half of cards, city or délégation, and a detail URL.
Detail pages add Open Graph metadata—email, phone, address, locality, region, postal code, geo position, description—when the listing card lacks email. Default strategy: scrape all listing pages; fetch detail only when the card has no email (about half the sample needed detail enrichment).
Scope defaults (version one)
- Record types — schools and foyers; sport and culture club paths excluded from ICP v1
- Ownership — public and private both included
- Outreach — CRM only; consentEmail UNKNOWN; no Listmonk on import
- Run location — operator workstation, not dardev-vps scraper
- Dedupe — ecolesSlug from URL path plus company name fallback
URL taxonomy to school level
- /etablissements/maternelles/ → MATERNELLE
- /etablissements/primaire/ → PRIMAIRE
- /etablissements/secondaire/ → SECONDAIRE
- /etablissements/superieur/ → SUPERIEUR
- /foyers/ → FOYER
CRM field mapping
Imports reuse DarDev prospect conventions. targetSolution maps to 20.tn (OPT20TN enum in Twenty). leadSource ECOLES_ANNUAIRE with leadSourceDetail ecoles-com-tn-2026. recordSegment distinguishes private school, public school, and higher establishment. regionMarket TUNISIA. consentEmail UNKNOWN. importedAt timestamp on insert.
School-specific Company and Person fields include schoolLevel, schoolOwnership, schoolGovernorate, schoolDelegation, ecolesSlug, and ecolesSourceUrl. Standard Twenty objects carry name, address, phones on Person when email exists, and generic contact naming when the directory only lists a school inbox.
Documented tag vocabulary for later workflows: lead:20tn, icp:ecole-tunisie, source:ecoles-annuaire-2026, consent:unknown. Tags are not a substitute for consent fields—Listmonk sync still waits for opt-in.
Data layout and checkpoints
Working files under data/20tn-ecoles/ (gitignored in production runs): raw per-page JSON checkpoints, scrape-state.json resume cursor, staging.json normalized batch, import-run.log append log. Checkpoints let operators pause a multi-hour scrape without restarting from page zero.
Scripts and quick start
- scripts/twenty/setup_20tn_crm_fields.py — one-time CRM fields and enum sync
- scripts/20tn-ecoles/scrape_ecoles_playwright.py — listing and optional detail scrape
- scripts/20tn-ecoles/import_ecoles_twenty.py — staging JSON to Twenty API
- scripts/20tn-ecoles/resume_ecoles_import.py — orchestrate scrape plus import with resume
- scripts/20tn-ecoles/run-ecoles-scrape.sh — Ops scraper-custom entry
Typical operator flow: dry-run scrape two pages to staging, dry-run CRM import, then full resume_ecoles_import.py --live when fields and API key are verified. TWENTY_API_KEY lives in repo root .env—never commit it. Import-only mode reuses existing staging.json without re-scraping.
Phase one results (aggregate)
Full scrape completed 161 listing pages producing on the order of nine hundred eighty staging records after filters. CRM import reported roughly nine hundred sixty-nine successful creates, ten skipped duplicates or out-of-scope rows, and three encoding errors retried with UTF-8 Python IO settings. QA spot-check against live ecoles.com.tn pages remains on the checklist before sales scales intros.
DarDev Ops wiring
Solution manifest 20tn registers connector 20tn-ecoles-scraper as scraper-custom. Pipeline ecoles-import runs on operator PC with staging path data/20tn-ecoles/staging.json. Ops console shows run history; it does not execute Playwright on dardev-vps.
Downstream phases
Phase two wires Twenty outreach workflows and platform-bus send-one—see 20tn-crm-workflow-orchestration. Phase three adds tracked mail engagement sync and reply handling. Phase four introduces subscribe forms and Listmonk marketing lists only after consent changes on the Person record.
Product vision for directors is in 20tn-vision-tunisia-schools. Sales staff read 20tn-outreach-one-school-at-a-time. Cross-cutting import ethics appear in integrations/PROSPECT-IMPORT-PIPELINES.md in the monorepo.
Privacy and publication boundaries
This article reports aggregate counts and field names only—no sample emails, phone numbers, or school-identifying rows from staging files. Public documentation explains the pipeline; PII stays in CRM and gitignored data directories accessible to authorized operators.
When sharing progress externally, cite rounded totals and phase status—not individual établissement names from staging unless the school has agreed to a public case study. Internal QA spreadsheets stay off news.dardev.net and out of git; the import-run.log on the operator machine is enough for engineering retros.
If a listing lacks email even after detail enrichment, the Company still lands in CRM for phone-first outreach later. Empty Person rows are acceptable; forcing placeholder emails would poison deliverability and violate the honest prospect model in deployed-realities-explained.
Why not scrape on dardev-vps?
Operator PC matches OECT and other Tunisia directory jobs: respectful rate limits, resume on laptop, and no unnecessary bot traffic from company IP against public sites.
Are public schools in scope?
Yes for CRM import v1. Sales ICP may prioritize private schools with email—filter in CRM views, not by deleting public records at import.
Does import add contacts to Listmonk?
No. CRM only until consent changes. Listmonk enters when send-one syncs a subscriber for tracked outreach—and still not for cold newsletter blasts.
How do we avoid duplicate schools on re-import?
Dedupe on ecolesSlug and company name. Skipped rows in the aggregate import log are usually duplicates or validation failures.
What if ecoles.com.tn changes layout?
Update Playwright selectors in scrape_ecoles_playwright.py, re-run recon on a few pages, and resume from scrape-state.json rather than blind full reimport.
