D1 read sources and reducing usage

Cloudflare D1 bills on rows read and rows written. If you see high read volume (e.g. millions per day) with little human traffic, the reads are coming from repeated requests that each trigger one or more D1 queries.

Where D1 reads come from

Source	What runs	Approx. reads per request
Analytics (GET /analytics/*)	Auth may call getAppSettings (or marketing key lookup). Handler calls getSettings() for masking/TTL. Then the endpoint runs its own queries (leads, moderation, engine, etc.).	Before cache: 1–2 (AppSettings) + many per endpoint (10–50+ for list/dashboard). After AppSettings cache: 0 from settings when KV hit; endpoint queries unchanged.
*Lead drought cron (*/10 * * * )	When disabled in Settings: only KV read (no D1). When enabled: getAppSettings + MAX(CreatedAt) from Leads.	0 (disabled) or ~2 (enabled).
*Retention cron (0 3 * * )	R2 scan + delete; minimal D1.	Negligible.
Tracking (POST /track/session/start, etc.)	Fingerprint (SELECT/INSERT), session (INSERT), waitUntil: identity network, fingerprint counters (D1).	Several per request (5–15+).
Forms config (GET /forms/config)	Sites, SiteDomains, Forms (several SELECTs).	3–6 per request.
Lead capture (POST /lead.capture)	AppSettings, site/form check, Integrations or SiteDomains, then lead insert and related tables.	5–15+ per request.
Replay (GET /replay/*)	Auth, then session/event queries from D1 and R2.	Depends on endpoint.

So the biggest levers are:

1. Analytics traffic — Every dashboard load (or polling script) can do many requests; each used to do 1–2 AppSettings reads plus heavy endpoint queries. AppSettings is now cached in KV for 60s, so repeated analytics requests share one D1 read per minute for settings. Use ANALYTICS_CACHE_TTL_SECONDS (default 604800 = 7 days) so GET responses are cached and D1/Engine isn’t hit every time. Use Refresh button cooldown (Settings or ANALYTICS_REFRESH_COOLDOWN_SECONDS; default 1800 = 30 min) to limit how often users can click Refresh.
2. Tracking traffic — Bots or crawlers loading pages with the tracker script cause POST /track/session/start (and events). Each does multiple D1 reads. Check Cloudflare Analytics or Worker logs (path, IP) to see if /track/* is being hit a lot.
3. Forms config — If /forms/config or form script is hit repeatedly (e.g. by bots), each request does several D1 reads. Consider caching form config in KV if the same key is requested often.

What we did to reduce reads

• AppSettings in KV: getAppSettings() reads from KV first (pg:app_settings, 60s TTL). On cache hit, no D1. When you save Settings we delete the cache key so the next read repopulates from D1.
• Lead drought when off: The */10 cron checks lead_drought:enabled in KV first; when it’s false or missing we skip D1 entirely (no AppSettings, no Leads query).

How to investigate high D1 usage

1. Cloudflare dashboard — D1 → your database → Metrics: see read/write volume over time.
2. Worker logs — Enable LOG_TRACKING or LOG_LEVEL=debug temporarily and inspect which paths are hit (e.g. /track/session/start, /analytics/overview, /forms/config).
3. Analytics — In Workers & Pages → your worker → Metrics or Logs (Real-time), filter by path to see request distribution.

If most requests are to /track/ or /forms/config, the traffic may be bots or crawlers; rate limiting (already on /track) and optional caching for form config can help. If most are to **/analytics/****, ensure analytics response cache is enabled (ANALYTICS_CACHE_TTL_SECONDS; default 7 days) and set Refresh button cooldown (ANALYTICS_REFRESH_COOLDOWN_SECONDS or Settings; default 30 min) to prevent overuse; avoid scripts that poll the dashboard.