Gate Surface tag tracking on form-scoped cookie consent

Author: 0xgautam

CLAUDE.md

@@ -34,7 +34,7 @@ cd test && ./serve.sh
 
 Open `http://localhost:8000/test/index.html` in a browser. Four test pages: popup, slideover, inline, and input-trigger. Test config (form source URL, environment ID) can be overridden via URL parameters -- see `test/config.js`.
 
-No automated tests, linter, or CI pipeline. Testing is manual and browser-based.
+Automated tests run with `pnpm test` (Vitest/jsdom), including policy parity, consent races, all embed shapes and external forms. `pnpm test:unit` runs the journey subset. CI runs typecheck, all tests and a committed-bundle freshness check. Run `pnpm build` before committing source changes.
 
 ## Source Architecture (`src/`)
 
@@ -83,19 +83,21 @@ No automated tests, linter, or CI pipeline. Testing is manual and browser-based.
 
 ### Consent (`src/consent/`)
 
-Forms whose Privacy settings put a category on "On consent" load no scripts for
-it until the host page reports the visitor's answer:
+The tag loads each registered form's public runtime configuration before optional work.
+`src/consent/policy.ts` mirrors Forms PR #5811 at `7d525ac12caf9f8a8475551e907929d46dbc4dc0`.
+See `docs/cookie-consent.md` for architecture, contract synchronization, integration limits and QA.
 
 ```js
-window.SurfaceSetConsent({ adTracking: true, surfaceAnalytics: true });
+window.SurfaceSetConsent({ cookieTracking: true, adTracking: true, surfaceAnalytics: true });
 ```
 
-`consent.ts` holds the answer in module state and notifies `src/index.ts`, which
-relays `surface:consent` to every Surface iframe (and re-sends it on each
-`SEND_DATA` handshake, for forms that mount after the banner was answered).
-Omitted categories count as not granted. The categories mirror the form-render
-gate in `surface_forms` (`lib/client/thirdParty/`) — keep the message shape in
-sync with its `hostConsent.ts`.
+Each call is a complete snapshot; omitted/non-true fields deny. Consent is in memory and
+shared with the Forms SDK using `window.__SURFACE_CONSENT__` and `surface:consent` DOM events.
+The host CMP owns persistence and must replay its saved answer each visit. Effective A/S
+both require effective C. Shared host work uses the intersection of registered forms; no
+form scope, unresolved settings, or multiple environments keep it off. No reload on withdrawal.
+Native iframes own operational identity; external HTML forms keep their scoped capability in
+memory and await it before submitting. Never put a capability in visitor cache or logs.
 
 ### Key APIs
 

README.md

@@ -68,3 +68,16 @@ See [docs](https://docs.withsurface.com/docs/surface-tag/installation) for integ
 3. Identity API call completes in <0.5s on slow 4G; no issues if blocked/failed
 4. PostMessage to iframe: query params, prefilled email, cookies, URL/origin/referrer
 5. Form loading speed on withsurface.com
+
+## Cookie consent
+
+The tag supports `SurfaceSetConsent({ cookieTracking, adTracking, surfaceAnalytics })`.
+Each call replaces all three values; only `true` grants. Configure the form's privacy
+controls to **On consent** to require an explicit CMP answer, and replay saved CMP
+choices every visit. No optional host tracking starts before an actual registered
+form's policy is resolved. Business form submission remains available when tracking
+is denied.
+
+See [cookie consent integration and rollout](docs/cookie-consent.md) for the Forms
+PR #5811 contract, purpose gates, multi-form behavior, release limits and QA harness.
+Run `pnpm test`, `pnpm typecheck` and `pnpm build` before committing tag changes.

docs/cookie-consent.md

@@ -0,0 +1,82 @@
+# Cookie consent runtime
+
+This tag implements the scripts handoff accompanying [Forms PR #5811](https://github.com/trysurface/surface_forms/pull/5811), reviewed at `7d525ac12caf9f8a8475551e907929d46dbc4dc0`. Deploy a compatible Forms API/renderer together with this bundle. This draft has automated coverage; staging browser/server QA below remains a release gate.
+
+## Customer integration
+
+Configure Cookie tracking, Ad & conversion tracking, and Surface analytics to **On consent** for Paddle's form. After both the tag and the host CMP are ready, report the CMP's current saved choices:
+
+```js
+window.SurfaceSetConsent({
+  cookieTracking: true,
+  adTracking: true,
+  surfaceAnalytics: true,
+});
+```
+
+Every call replaces the entire snapshot. Only literal `true` grants; omitted fields are false, including `cookieTracking` in older two-field integrations. Call again on changes and withdrawals. Replay saved choices every visit. Surface keeps consent in memory (`__SURFACE_CONSENT__` and a `surface:consent` DOM event); the CMP owns persistent consent and regional applicability. There is no built-in CMP or country detection. Do not reload the page on withdrawal.
+
+For three **On consent** policies:
+
+| Report | Host behavior |
+| --- | --- |
+| C false, any A/S | No optional cookie/storage reads, recognition, journey or parent advertising |
+| C only | Visitor recognition; no marketing or journey cookie snapshot |
+| C+A | Recognition, eligible marketing cookies, enabled Dreamdata identity, configured supported conversions |
+| C+S | Recognition, page journeys and Surface measurements |
+| C+A+S | All permitted purposes |
+
+Form rendering, explicit prefill, current-page URL/UTMs, validation and business submission continue independently. An operational `/lead/identify` request is expected for an external HTML form even without C: it uses a random interaction UUID and obtains a signed form-scoped capability, without a cached visitor/session/fingerprint.
+
+**Allowed** and missing modes on successfully loaded legacy settings preserve their Forms semantics. They can permit tracking without a positive reported answer. Use **On consent** to require explicit consent. An unavailable/malformed settings response denies optional work; it is never treated as legacy `{}`.
+
+## Runtime and data boundaries
+
+- `consent/runtime.ts` registers form contexts and fetches `{origin}/api/v1/public/forms/{formId}/runtime-config`, credential-free and `no-store`. It validates form/environment/settings, deduplicates concurrent loads, times out after eight seconds and revalidates on restored documents. Policy fetching never uses browser storage.
+- `store/store.ts` registers direct, dynamic and tag-created `/s/<formId>` frames on recognized origins, plus lazy embed declarations. Messages require the registered element's window and exact declared origin; detached, changed-src and unrelated windows cannot use that registration. Consent precedes store/readiness replies. Load handlers coexist with spinners. Each frame has separate explicit prefill and conversion deduplication state.
+- Shared host work uses the intersection of registered forms. Pending/error contexts deny; multiple environments/API origins deny because the existing browser keys have no tenant namespace. Per-frame snapshots are filtered again. Declared lazy forms participate before navigation.
+- Gates precede cookie getters, tracking storage and fingerprinting. Managed marketing snapshots permit only `hubspotutk`, `__hstc`, `__hssc`, `__hssrc`, `_ga`, `_ga_<alphanumeric>`, `_gid`, `_gcl_aw`, `_gcl_au`, `_fbp`, `_fbc` under C+A; only `surface_journey_id` is forwarded under C+S. `surface_recent_visit` remains internal. C alone reads neither snapshot. Explicit C replaces `trackCookie`; loaded legacy-only pages retain its cookie-map compatibility. A managed form on the page constrains legacy snapshots too.
+- Dreamdata requires `settings.dreamdata.enabled` and C+A. Source precedence is native localStorage, native cookie, `dd_aid`, compatibility localStorage, compatibility cookie. IDs are normalized and checked. The selected value travels in the renderer's `cookies.dd_anonymous_id` transport slot, never a new cookie. This transport does not add outbound Dreamdata calls. A denied `dd_aid` is removed from forwarded URL copies too.
+- Tag-created iframe URLs strip protocol-owned tracking identifiers and capabilities before first navigation. Current business query parameters/UTMs and explicit prefill stay available. Optional recognition cache includes verified form/API/environment scope, and never spreads an identify response/token into localStorage. Older unscoped cache records are ignored.
+- Native iframe renderers own their operational identity. The tag sends readiness with empty optional identity when denied, rather than creating a competing parent operational session. The tag's permitted recognition requests are form-scoped.
+- External forms allocate one in-memory interaction per document/form/API, deduplicate pending identity, await a capability, serialize writes using the returned response ID, and sample full reported consent immediately before sending. Established capabilities survive consent transitions and renew conservatively before the server's 12-hour expiry. Identity/write failures retain answers for retry and emit `surface:submit:error`. Repeated final clicks do not create another response. Tokens are never logged, cached, sent to analytics or put in URLs.
+- Journey events and external view/start measurements require C+S and carry actual form scope/full reported consent. Optional requests capture a generation and recheck before sending/applying results; revoke/regrant cannot restore obsolete results or replay denied-interval events. Functional route listeners remain active when journeys stop.
+- Registered valid conversion messages are acknowledged immediately even on denial to prevent the renderer's 250ms fallback. Provider/event configuration must match the requesting form. C/A consent-managed pages support only GA4 and Meta; X/LinkedIn/OpenAI remain off. Withdrawal sets owned GA opt-out flags and Meta revoke commands, and removes tag-owned queued commands during slow script loads. Meta conversions target the configured pixel. The tag does not add a HubSpot host loader; host-installed vendors remain the CMP's responsibility. GTM/Speed Insights restrictions inside the form belong to the renderer.
+- Withdrawal clears optional in-memory state and named Surface storage (`surfaceLeadData`, `LOCAL_SURFACE_LEAD_IDENTIFY_DATA`, journey cookies and `<formId>_pa`) at reachable host scopes. S withdrawal clears journeys while C may remain granted. Cross-origin iframe cleanup is owned by Forms. It does not clear all storage, CMP/login cookies or business state. Open-trigger config caching is now memory-only.
+
+## Release limits to resolve
+
+1. **Site-id-only pages have no form scope and therefore no optional host tracking.** An explicit declared journey/form scope requires a separate installer/API contract; the tag cannot infer a safe form from an environment.
+2. **Mixed SDK/tag pages need integration work in Forms** if page-wide policy intersection is required across SDK instances. PR #5811 shares consent but does not publish an SDK policy registry. This tag's intersection covers registered frames, lazy embeds and its external HTML forms. Do not enable overlapping SDK/tag journey producers without testing ownership and deduplication.
+3. **Direct iframe URLs must be authored cleanly.** The tag cannot undo identifiers already transmitted before it loaded. Scope validation uses the host's iframe `src` declaration; if embedded content navigates itself to another form without updating that declaration, the current protocol supplies no authenticated form-ID handshake. Use a new registered iframe for a new form, or add that handshake in Forms before supporting such navigation.
+4. **Previously sent requests cannot be recalled.** In particular, host withdrawal alone cannot cancel an already queued Dreamdata server job without a later response update. Capability invalidation/expiry also needs staging verification against the API: the scripts client never substitutes a persistent session on failure.
+5. **Separate legacy/standalone scripts are outside this bundle.** Audit existing installations of `surface_tracking.js` and `old/`; do not load an older tracking tag alongside this one. Duplicate loads of the new bundle reuse its active runtime and preserve sessions.
+6. **Settings fail closed here.** Reconcile any Forms SDK successful-fetch path that turns absent settings into `{}` before claiming identical error handling across every entry point.
+
+These limits and the customer's CMP/settings must be reviewed before claiming the Paddle installation is ready. This code alone does not establish legal compliance for the customer's full page or independently installed vendors.
+
+## Verification and rollout
+
+Run:
+
+```sh
+pnpm typecheck
+pnpm test
+pnpm test:unit
+pnpm build
+git diff --check
+cmp surface_tag.js surface_embed_v1.js
+```
+
+The journey suite was migrated from the separate node runner into Vitest, so CI now exercises its existing attribution cases plus lifecycle/race checks. Policy expectations were generated from the pinned Forms source for 27 modes × 8 reports; the actual SDK bridge is a fixture. See [fixture provenance and sync procedure](../test/fixtures/README.md).
+
+Use `test/consent.html` against the compatible Forms staging deployment. It has all eight snapshots, a second-frame control and an external HTML form harness. Use test form IDs, destinations and credentials. Before release verify:
+
+- No announcement, reject, C-only, C+A, C+S, full grant, withdrawal/regrant, reload and a policy API failure. Check cookie/storage **reads** as well as final storage and network output.
+- All embed shapes, delayed CMP readiness, direct/dynamic frames, multiple forms with conflicting settings, remounts and custom origins. Confirm no pre-navigation identifier leakage and correct consent-first handshake.
+- Slow fingerprint, identify, runtime-config and vendor loads during withdrawal. Confirm no stale cookie, identity or conversion is restored afterward.
+- Native and HTML-form answers still submit with one current interaction; inspect saved response consent and minimized metadata. Exercise routing, booking and CRM sync with test destinations. Retry a failed/expired capability without losing draft answers.
+- Verify Dreamdata source precedence and saved private claims, then the documented server-job withdrawal boundary.
+- For Paddle, verify the CMP's mapping and saved-choice replay on a staging copy of `/demo`; inventory other vendors and standalone Surface assets. No production form submission or customer settings change is part of this draft.
+
+Publish a versioned compatible tag only after those checks and coordinate Forms deployment. Rollback must keep optional tracking disabled for managed forms while retaining business submission; do not restore an old allow-all tag to recover tracking.

package.json

@@ -7,7 +7,7 @@
     "build": "esbuild src/index.ts --bundle --format=iife --outfile=surface_tag.js --target=es2020 && pnpm copyTagToEmbed",
     "copyTagToEmbed": "cp surface_tag.js surface_embed_v1.js",
     "dev": "esbuild src/index.ts --bundle --format=iife --outfile=surface_tag.js --target=es2020 --watch --sourcemap",
-    "test:unit": "node --test test/unit/*.test.js",
+    "test:unit": "vitest run src/store/user-journey.test.ts",
     "typecheck": "tsc --noEmit",
     "test": "vitest run",
     "postbuild": "pnpm copyTagToEmbed"
@@ -22,6 +22,7 @@
   "license": "ISC",
   "packageManager": "pnpm@10.32.1",
   "devDependencies": {
+    "@types/node": "^22.20.1",
     "esbuild": "^0.28.0",
     "jsdom": "^29.1.1",
     "typescript": "^6.0.2",

pnpm-lock.yaml

@@ -0,0 +1,22 @@
+/** Expiry writes do not read cookies or touch unrelated storage. */
+export function clearSurfaceTracking(formIds: string[], cookiesOnly = false): void {
+  if (!cookiesOnly) {
+    for (const key of ["LOCAL_SURFACE_LEAD_IDENTIFY_DATA", "surfaceLeadData", ...formIds.map((id) => `${id}_pa`)]) {
+      try { localStorage.removeItem(key); } catch { /* storage may be unavailable */ }
+    }
+  }
+  const parts = window.location.hostname.split(".");
+  const domains = ["", ...parts.map((_, index) => parts.slice(index).join("."))];
+  const segments = window.location.pathname.split("/").filter(Boolean);
+  const paths = new Set(["/", ...segments.flatMap((_, index) => {
+    const path = "/" + segments.slice(0, index + 1).join("/");
+    return [path, path + "/"];
+  })]);
+  for (const name of ["surface_journey_id", "surface_recent_visit", ...(!cookiesOnly ? formIds.map((id) => `${id}_pa`) : [])]) {
+    for (const domain of domains) for (const path of paths) {
+      try {
+        document.cookie = `${name}=; Max-Age=0; Path=${path}; SameSite=Lax${domain ? `; Domain=${domain}` : ""}`;
+      } catch { /* best effort for reachable host scopes */ }
+    }
+  }
+}