Architecture

Single source for system layout, request flow, and storage. See also Deployment for production setup.

Source Layout

src/
├── index.ts                  # Main entry — request router + scheduled handler
├── types.ts                  # All shared TypeScript interfaces
├── lead.ts                   # Lead capture handler
├── forms.ts                  # GET /forms/config, domain + API-key validation
├── formBuilder/
│   └── formScript.ts         # Dynamic form builder JS served at /form.js
├── lib/
│   ├── auth.ts               # API key authentication (Bearer + X-API-Key)
│   ├── logger.ts             # Structured JSON logger with request context
│   ├── rateLimiter.ts        # D1-backed fixed-window IP rate limiter (atomic)
│   ├── validator.ts          # Input validation and XSS sanitization
│   ├── retention.ts          # R2 storage monitoring and retention cleanup
│   └── leadDroughtAlert.ts   # Cron: check last lead; alert via webhook when drought (Settings)
│   └── webhook.ts            # Webhook sender with retry + exponential backoff + dead letter
├── replay/
│   ├── replayService.ts      # Session list + R2/D1 event retrieval for replay
│   ├── replayRoutes.ts       # GET /replay/* route handler
│   └── replayViewer.ts       # Self-contained HTML/CSS/JS replay viewer
└── tracking/
    ├── index.ts              # Tracking router (POST /track/*)
    ├── trackerScript.ts      # Frontend JS snippet served at /tracker.js
    ├── routes/
    │   ├── trackSessionStart.ts
    │   ├── trackEvent.ts
    │   ├── trackEventsBatch.ts
    │   ├── trackSessionEnd.ts
    │   └── analytics.ts      # GET /analytics/* dashboard queries
    ├── services/
    │   ├── eventService.ts
    │   ├── sessionService.ts
    │   ├── fingerprintService.ts
    │   ├── identityService.ts
    │   ├── scoringService.ts
    │   ├── storageService.ts
    │   ├── vpnDetectionService.ts
    │   ├── analyticsService.ts      # D1 analytics queries for dashboards
    │   ├── leadAnalyticsService.ts  # Lead listing/detail + linked session activity
    │   ├── settingsService.ts      # AppSettings (dashboard Settings)
    │   └── marketingApiKeysService.ts
    └── utils/
        ├── hash.ts
        ├── uuid.ts
        └── response.ts

Request Flow

Client Request
  │
  ├─ /track/*       → rate limiter → tracking/index.ts → validator → route handler → services → D1/KV/R2
  ├─ /analytics/*   → auth (admin or marketing key) → analytics.ts (role: 403/masking for marketing) → D1 / R2
  ├─ /replay        → replayViewer.ts (public HTML)
  ├─ /replay/*      → auth (API key) → replayRoutes.ts → replayService → D1 / R2
  ├─ /tracker.js    → inline JS (trackerScript.ts)
  ├─ /form.js       → inline JS form builder (formScript)
  ├─ /pulse.js      → combined tracker + form; PulseGate.init() defaults endpoint to script origin
  ├─ /health        → JSON health check
  ├─ /forms/config  → forms.ts → validate domain (SiteDomains) → return form schema JSON
  ├─ /lead.capture/crm-callback → lead.ts handleCrmCallback → auth (CRM_CALLBACK_SECRET) → store CrmLeadId
  ├─ /lead.capture/make-callback → lead.ts handleMakeCallback → auth (CRM_CALLBACK_SECRET) → store Make execution URL on MakeWebhookSends
  ├─ /webhooks/crm/lead        → lead.ts handleCrmWebhook → auth (CRM_WEBHOOK_SECRET or CRM_CALLBACK_SECRET) → LeadCrmActivity
  ├─ /lead.capture  → lead.ts → auth (X-API-Key via Integrations or domain via SiteDomains) → lead flow → D1 + webhook
  ├─ /lead | /      → lead.ts (deprecated; use /lead.capture; emits X-Deprecated + Sunset headers)
  └─ *              → 404 Not Found

Scheduled (crons: 0 3 * * *, */10 * * * *)
  ├─ 0 3 * * *     → retention.ts   → scans R2 by date prefix → deletes objects older than 90 days
  └─ */10 * * * *  → leadDroughtAlert.ts → if enabled in AppSettings: last lead age → webhook when > threshold

Authentication

Analytics and replay endpoints require an API key:

Authorization: Bearer <key>
X-API-Key: <key>

Unauthenticated requests receive 401 with WWW-Authenticate: Bearer. CORS preflight (OPTIONS) does not require auth.

Roles: See Marketing team access.

Admin — ANALYTICS_API_KEY: full access.
Marketing — Dashboard-generated key (Settings → Marketing API keys) or optional ANALYTICS_MARKETING_API_KEY: restricted pages; PII (email, phone) masked when setting is on.

Cloudflare Bindings

Binding	Type	Purpose
`DB`	D1	Leads, LeadSubmissions, Sites, Forms, Integrations, AppSettings, MarketingApiKeys, MakeWebhookSends, sessions, fingerprints, identity_network, etc.
`TRACKING_KV`	KV	Rate counters, IP/ASN tracking, session throttle (20s)
`TRACKING_R2`	R2	Raw session events, replay data
`LEAD_ANALYTICS`	Analytics Engine	Lead submission analytics (3‑month retention); optional
`ANALYTICS_API_KEY`	Secret	Admin API key
`ANALYTICS_MARKETING_API_KEY`	Secret (optional)	Legacy marketing key; prefer dashboard-generated keys
`CRM_CALLBACK_SECRET`	Secret	Callback from Make.com after CRM create
`CRM_WEBHOOK_SECRET`	Secret (optional)	Incoming CRM webhook; falls back to CRM_CALLBACK_SECRET

Storage Strategy (R2-first events)

Store	Data	Access Pattern
D1	Sessions (event_count, page_views, clicks, max_scroll_percent), fingerprints, identity_network; Leads, LeadSubmissions, AppSettings, MarketingApiKeys, MakeWebhookSends	Persistent queries; no events table in hot path
D1 (LeadSubmissions.PayloadJson)	Full request body per submission	Submission-level analytics
Workers Analytics Engine	lead_submissions (3‑month retention)	Dashboard charts via SQL API
KV	Rate counters, session throttle (20s)	Low-latency; 24h TTL
R2	Raw event logs (batches 50–200), replay	Append-only; prefix list for retrieval

Event ingest uses no D1 reads (client sends session_started_at); events written only to R2; session aggregate updates throttled to once per 20 seconds per session. Replay loads events from R2 only. See R2-first events migration.

R2 Key Layout

sessions/
  YYYY/MM/DD/
    {session_id}/
      meta.json                          # Session metadata (written once)
      events/
        {timestamp}-{event_id}.json       # One file per event/batch (append-only)

Each event write uses a unique key (no read-modify-write). Replay lists keys under the session prefix and merges chronologically.

D1 Schema (summary)

Leads, LeadSubmissions — Lead capture; CrmLeadId, engagement/suspicion scores, SiteKey, FormId, PublicId.
LeadCrmActivity — Incoming CRM webhook events per lead.
MakeWebhookSends — Payloads successfully sent to Make.com.
Sites, SiteDomains, Forms, Integrations — Multi-form and domain allowlist.
AppSettings — Dashboard Settings (Send leads to Make.com, bot detection, CRM URL template, marketing visible pages, etc.).
MarketingApiKeys — Per-key marketing API keys and visible pages.
sessions — Visitor sessions with aggregates (event_count, page_views, clicks, max_scroll_percent); raw events in R2 only.
fingerprints — Cross-session identity; VPN/fraud signals.
identity_network — Fingerprint ↔ IP/ASN mapping.
BlockedIPs, BlockedIPLeads, WhitelistedIPs, TestingLeads — IP and test-lead handling.

Full schema is additive via migrations in migrations/ (001 through 035+). Apply with scripts/apply-production-migrations.sh or scripts/apply-local-migrations.sh.

Background Processing

Heavy I/O runs in ctx.waitUntil() so the response returns immediately. Event ingest writes only to R2 and updates session aggregates (throttled). Order: critical D1 (session start), identity network, best-effort KV/VPN, R2 event storage, throttled D1 session aggregate update (max once per 20s per session).

Scheduled Jobs

Cron	Job	Description
`0 3 * * *`	R2 retention	Deletes R2 objects older than 90 days
`/10 * * *`	Lead drought alert	If enabled in Settings: check last lead; POST to webhook when no lead in threshold minutes

Architecture ​

Source Layout ​

Request Flow ​

Authentication ​

Cloudflare Bindings ​

Storage Strategy (R2-first events) ​

R2 Key Layout ​

D1 Schema (summary) ​

Background Processing ​

Scheduled Jobs ​