StreamPulse

StreamPulse is a visual incident copilot for synthetic video-streaming telemetry. Ask what happened in Portugal at 20:14 and the Trigger.dev chat agent can lead with a live, interactive Europe map: 24 fifteen-second ClickHouse buckets, three POPs, four selectable device types, traffic-sized markers, an error-rate timeline, and synchronized keyboard-accessible controls. A heatmap, bounded matching-event table, decision analytics, and a separately rendered eight-second FFmpeg incident story remain available for more specific questions.

The known demo incident is synthetic. It covers 21 July 2026, 20:14–20:19 CEST (18:14–18:19 UTC) and isolates elevated playback failures to Portugal, Tizen, Akamai, Widevine, and TLS_HANDSHAKE_FAILURE. About 20.7 million events are stored across 15–29 July so relative-date questions remain useful through the final judging day. Every 15-second bucket contains 20 baseline events for each city/device combination, plus a density-matched healthy control window on 20 July for like-for-like comparison.

Architecture

• Next.js 16 and React 19 provide the visual-first chat UI.
• Trigger.dev chat.agent() owns the conversation loop.
• @ai-sdk/groq runs the pinned, tool-tested llama-3.3-70b-versatile model.
• ClickHouse Cloud is the primary telemetry and analytics database. There is no Docker or local ClickHouse fallback.
• Postgres managed by ClickHouse stores saved incidents and operator feedback; ClickPipes CDC mirrors that workflow into ClickHouse for decision analytics.
• Zod-bounded tool inputs select fixed, parameterized ClickHouse query templates; the model never writes SQL.
• A persistent Agent tools · 8 available panel publishes the exact registered tool IDs, evidence sources, outputs, and example prompts, including relative-date incident history and city/device distribution charts.
• The preferred geographic answer is browser-rendered SVG over a checked-in Natural Earth basemap. Its structured result comes from live ClickHouse data and can be saved as immutable Postgres JSONB evidence without an R2 artifact.
• A separate Trigger.dev task queries 24 fifteen-second buckets spanning pre-incident, incident, and recovery phases and uses the official FFmpeg 7 build extension to create the MP4 and evidence frames.
• A narrowly scoped Cloudflare Worker stores the completed MP4/JPEG artifacts in R2 and serves their public URLs.
• OpenNext runs the Next.js application on a second Cloudflare Worker; Vercel is not part of the deployment.
• A shared judge code creates a signed, eight-hour HttpOnly session. The page and both Trigger token-minting actions enforce it server-side.

Hosted judge demo

Open streampulse-web.dbg.workers.dev and enter the access code supplied with the submission. No judge account, email allowlist, extension, or local setup is required.

1. Select Show on incident map. Play or scrub the geographic timeline; Lisbon/Tizen becomes critical while the control POPs and non-Tizen devices remain healthy.
2. Ask What was the incident yesterday evening in Portugal? on 22 July (the UI switches to the explicit 21 July wording later), then ask Compare the data to the previous day to see the identical 20:14–20:19 window on both days.
3. Ask Was there any incident 3 days ago? for an honest no-incident graph, then ask In which cities do we have Tizen devices? for the city/device distribution.
4. Open Agent tools · 8 available, then inspect the exact tool IDs and evidence sources exposed to the agent.
5. Save and confirm the map, then select Analyze operator decisions to see the Postgres-to-ClickHouse CDC result.
6. Use Explore video story only when you want to demonstrate the separate Trigger.dev + FFmpeg + R2 path.

See docs/JUDGE_DEMO.md for the expected evidence and access design.

Local setup

Requirements: Node.js 22.16 or newer, ClickHouse Cloud and Managed Postgres credentials, a Groq API key, Trigger.dev CLI access, and Cloudflare access for the artifact gateway.

npm install
cp .env.example .env
chmod 600 .env
npm run check:env
npm run db:migrate
npm run db:pg:migrate
npm run db:seed
npm run db:assert
npm run db:pg:assert

Keep the ClickHouse bootstrap identity only in the ignored local .env. Trigger.dev needs the runtime ClickHouse identity, Groq key, and three ARTIFACT_* values, but never the bootstrap username or password.

Run the application and Trigger development worker in separate terminals:

npm run dev
npm run dev:trigger

The R2 gateway lives in workers/artifact-gateway. Its PUT token belongs in the ignored .dev.vars file and in the Cloudflare Worker secret store; never place it in wrangler.jsonc.

See docs/POSTGRES_INTEGRATION.md for the separate migration/runtime identities and the exact ClickPipes table mappings.

Cloudflare web deployment

The root wrangler.jsonc deploys the OpenNext application as streampulse-web. Its ignored root .dev.vars contains the production Trigger secret, DEMO_ACCESS_CODE, and DEMO_SESSION_SECRET. The Worker reaches Managed Postgres through the POSTGRES_HYPERDRIVE binding with verified TLS, so no database URL is deployed as a Worker secret. ClickHouse, Groq, bootstrap, migration-owner, and artifact-upload credentials stay in Trigger.dev, local setup, or their existing service scopes.

npm run web:types
npm run web:build
npm run web:deploy

npm run web:preview exercises the same bundle locally in the Workers runtime. Rotate either demo secret by updating the ignored .dev.vars and deploying again; never commit an access code.

Verification

npm run lint
npm run typecheck
npm test
npm run build
npm run db:assert
npm run db:pg:assert
npm run smoke:groq
npm run smoke:heatmap
npm run smoke:map
npm run smoke:chat
npm run smoke:exploration
npm run smoke:chat:exploration
npm run smoke:chat:map
npm run smoke:chat:logs
npm run smoke:video
npm run smoke:feedback:render
npm run smoke:feedback:live
npm run worker:check
npx trigger deploy --dry-run

smoke:map queries the fixed three-POP/four-device source window directly from ClickHouse, validates all 24 ordered buckets, checks the payload and basemap budgets, and server-renders the component. smoke:chat:map proves a fresh production agent turn invokes exactly queryIncidentMap, returns the map before its short caption, and does not start the video renderer.

smoke:video sends a fresh chat request, requires a distinct child Trigger task run, downloads the MP4 and four JPEG evidence frames, checks content types, signatures, 1280×720 dimensions, an encoded duration between 7.5 and 8.5 seconds, 24 monotonic timeline buckets, trusted artifact origins, and a standards-compliant HTTP byte-range response for reliable seeking.

db:pg:assert rejects owner/admin credentials and proves that the runtime role has only the required table operations before exercising the rolled-back heatmap, map, and video workflow. db:map:cdc:assert is a read-only check for the permanent hosted map acceptance record. smoke:feedback:live is a mutating probe: it creates another permanent record and should run only when one is intentionally required.

See GATES.log for executed commands and real exit codes, docs/INCIDENT_PROOF.md for the locked data assertions, and docs/streampulse-package for the approved product and architecture documents.

Development status

The interactive geographic timeline, eight-tool catalog, relative-date incident history, city/device distribution, visual heatmap, matching event evidence, day-over-day comparison, real-video incident story, artifact gateway, password-gated Cloudflare-hosted judge demo, Managed Postgres workflow, ClickPipes CDC tables, and ClickHouse decision analytics are implemented. The GitHub repository remains private during development.

License

MIT. See LICENSE.