Seeking 3–5 design partners for offline RAG/agent release evaluation

[thangldw]

I’m looking for 3–5 teams already running a RAG or agent pilot and facing a recurring quality or release-regression problem.

What the pilot includes

• A 60-minute discovery session with a technical owner and workflow owner
• A small synthetic or redacted baseline/candidate evaluation set
• Offline, reproducible release-gate evidence
• A pseudonymous baseline/pilot measurement report for repeat usage, task success, time saved, critical incidents, and optional cost/ROI assumptions
• A final SCALE / HOLD / REDESIGN review

Data boundary

No production credentials, raw customer documents, names, emails, or unredacted prompts are required. Start with synthetic/redacted traces. Real observations require partner consent and pseudonymous identifiers.

Good fit

• You already have a RAG/agent pilot
• Someone currently reviews regressions manually or through notebooks
• You can identify one recurring workflow and one decision owner
• You can collect roughly two weeks or 50 eligible tasks when feasible

The pilot contract and commands are documented in the design-partner runbook. The bundled report is explicitly synthetic and is not a customer ROI claim.

If this fits your team, reply with the workflow, current release pain, and whether you can start with synthetic/redacted evidence.