Add managed Hermes to Relay to Switchyard E2E and negative-security coverage #8889

Parent Epic: #7937

Outcome

A dedicated NemoClaw live E2E proves the released, secure, normal-runtime Hermes → native Relay → Switchyard path and rejects false-green topologies.

Canonical NemoClaw scope was accepted in #7937 on 2026-08-13. This is the final implementation gate after child issues #8885-#8888; a green E2E does not establish general availability or compensate for an unreleased dependency.

Qualified topology

NemoClaw-managed OpenShell sandbox
└── normal supervised Hermes process
    └── native NeMo Relay runtime
        └── released nvidia.switchyard plugin
            └── OpenShell-secured providers
                ├── judge/classifier, when required
                ├── weak/efficient target
                └── strong/capable target

Relay intercepts and replaces Hermes downstream LLM execution with the Switchyard plugin async generator. Switchyard produces the RoutingOutcome, rewrites the request, and executes the selected target. Relay owns plugin lifecycle and correlation; Switchyard owns routing and selected-target execution. Relay and Switchyard run inside the supervised Hermes process with zero routing sidecars.

Release and evidence gate

The live target must consume the stable compatibility record from #8885. It must reject:

• pull-request heads, source checkouts, ad hoc builds, prereleases, latest URLs, or mutable tags;
• a Hermes Relay lock that differs from the selected Relay artifact;
• missing linux/amd64 or linux/arm64 plugin assets;
• artifact hashes or manifests not bound to the producer repository, source SHA, workflow run, and attempt; and
• stable/candidate selector mixtures.

The result artifact must record the exact installed Hermes, Relay, Switchyard, plugin bundle, activation manifest, deployment document, platform, and immutable digests. Version strings alone are insufficient.

Test cases

Use the existing typed NemoClaw live-target registry, fixture phases, artifact paths, redaction scan, and teardown proof. Publish one machine-readable expected-versus-observed manifest. Missing, duplicate, skipped, stale, or unteared-down cases fail the run.

Positive path

• Freshly provision the three-provider bundle from Provide secure multi-target OpenShell providers for Switchyard routing #8887 and activate the released two-file configuration from Add native Relay and Switchyard activation with fail-closed Hermes readiness #8886.
• Prove requested routing becomes active before the first provider request.
• Exercise seeded weighted-random and LLM-classifier routing across two real target identities.
• Cover buffered and streaming turns, tool calls, repeated turns, and stable Hermes session identity.
• Prove the normal supervised Hermes process remains the serving process before and after restart.
• Rebuild and reattach only when the provider identity/attestation contract can be revalidated without mutation ambiguity.
• Capture the canonical non-secret switchyard.routing.decision event and join it to the Hermes turn through Relay correlation metadata.

Stage routing, response-judging escalation, and additional routers are follow-up qualification, not implicit first-milestone support.

Negative and security path

• Invalid or old-form activation configuration.
• Missing, corrupt, wrong-architecture, wrong-digest, or incompatible plugin bundle.
• Requested-but-inactive routing and unknown Hermes route/model identity.
• Missing, partial, stale, duplicate-name, or unattestable judge/weak/strong provider bundle.
• Caller Authorization forwarding, cross-provider placeholder use, direct-provider egress, and policy bypass attempts.
• Judge denial, selected-target denial, retry exhaustion, candidate exhaustion, cancellation, and transport deadlines.
• Redirects, oversized response/error/SSE bodies, malformed streams, and post-first-stream failure.
• Canonical routing-event loss, correlation mismatch, typed-event corruption, and PII/redaction failure.
• Raw credential, placeholder, prompt, header, body, or unbounded reasoning leakage into files, environment, argv, logs, errors, telemetry, evidence, and cleanup backups.

No negative case may silently fall through to inference.local, another target, or unrouted Hermes. Any fallback must be the explicit, upstream-owned, configured fallback contract from #8888 and must remain observable.

Current external gates (2026-08-26)

• Hermes #77915 is merged and present in the Hermes Agent 0.20.6 (v2026.8.27), but the newly released version still requires Relay below 0.8 and locks Relay 0.7.2.
• Relay 0.8.0 is stable at verified tag 9d0ef4f9 / commit 812613b8. Producer run 33019578028 succeeded and published digest-bearing Linux amd64/arm64 artifacts. Hermes has not yet released a dependency migration that selects it.
• Switchyard [Ubuntu + Docker CE] unable install nemoclaw with the hardware requirement in readme #528 is still open, review-required, and moving; it targets the Relay 0.8 contract and is not a released bundle. Current reviews identify arbitrary inbound metadata-header forwarding, decode-before-ownership behavior for unmanaged routes, missing final streamed token usage, and an unproven packaging-test lane.
• Switchyard v0.2.0 has no native plugin bundle release asset.
• OpenShell docs: add Brev troubleshooting guide #2722 remains open for identity-bound provider mutation and secret-free endpoint/profile attestation; rebuild/rotation evidence cannot be claimed until that contract or an accepted equivalent is released and qualified.

The frozen V4 demo remains evidence only and must not be substituted for this released-path target.

Acceptance

• Scope and ownership were accepted in [Epic] Integrate Switchyard routing into NemoClaw-managed Hermes through native NeMo Relay #7937 on 2026-08-13.
• Every selected Hermes, Relay, Switchyard, OpenShell, and plugin artifact is a stable immutable release with verified provenance and required architecture coverage.
• Normal supervised Hermes serves every routed turn; Relay/Switchyard routing sidecar count is zero.
• Routing is active and ready before the first provider request.
• Weak and strong prompts select distinct real targets and return distinguishable real responses.
• Every asserted decision comes from the canonical switchyard.routing.decision event, not reconstructed provider logs.
• Event correlation binds the decision, Hermes turn, selected target, and installed artifact set without prompts, credentials, or unbounded reasoning.
• Buffered, streaming, tool, repeated-turn, and session behavior remain correct.
• Supervised restart passes; rebuild passes only with revalidated provider identity and attachment evidence.
• Every negative case fails at its intended boundary with actionable, bounded, non-secret diagnostics and no silent routing bypass.
• Credential, placeholder, policy, redirect, timeout, body-bound, SSE, cancellation, and post-first-stream probes pass.
• Machine-readable evidence binds expected test IDs, observed pass status, release identities/digests, architecture, topology, provider identities, route-event correlation, redaction scan, and teardown success.
• Missing, skipped, duplicate, stale, or unteared-down cases make the qualification fail.