Add a self-hosted CIMD opt-out with DCR fallback

.changeset/selfhost-cimd-dcr-fallback.md

@@ -0,0 +1,5 @@
+---
+"executor": patch
+---
+
+Allow self-hosted deployments whose CIMD document is unreachable by OAuth servers to use DCR for automatic MCP and discovered OpenAPI connections with `EXECUTOR_OAUTH_CIMD_ENABLED=false`. Unsetting the variable restores CIMD for new connections without rewriting integration settings, including legacy OpenAPI templates.

apps/docs/hosted/docker.mdx

@@ -53,22 +53,23 @@ Back it up by snapshotting that volume (or copying `/data`, primarily `data.db`)
 Everything is optional: a bare run boots a working instance. The defaults below are
 the container defaults.
 
-| Variable                            | Default                         | Purpose                                                                                         |
-| ----------------------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------- |
-| `PORT`                              | `4788`                          | HTTP port the server listens on.                                                                |
-| `EXECUTOR_HOST`                     | `0.0.0.0`                       | Bind address. The image binds all interfaces.                                                   |
-| `EXECUTOR_DATA_DIR`                 | `/data`                         | Directory holding the database and generated keys.                                              |
-| `EXECUTOR_DB_PATH`                  | `<data dir>/data.db`            | SQLite database file.                                                                           |
-| `EXECUTOR_WEB_BASE_URL`             | auto (`http://localhost:4788`)  | Public URL browsers use. Required behind a domain or TLS (see below).                           |
-| `EXECUTOR_TRUSTED_ORIGINS`          | unset                           | Comma-separated browser aliases allowed to authenticate without changing the public URL.        |
-| `BETTER_AUTH_SECRET`                | generated, persisted in `/data` | Session secret (32+ chars). Rotating it signs everyone out.                                     |
-| `EXECUTOR_SECRET_KEY`               | generated, persisted in `/data` | Master key encrypting stored secrets. Set it to manage it yourself.                             |
-| `EXECUTOR_BOOTSTRAP_ADMIN_EMAIL`    | unset                           | Pre-create the admin headlessly (with the password below); skips browser first-run.             |
-| `EXECUTOR_BOOTSTRAP_ADMIN_PASSWORD` | unset                           | Password for the bootstrap admin.                                                               |
-| `EXECUTOR_BOOTSTRAP_ADMIN_NAME`     | `Admin`                         | Display name for the bootstrap admin.                                                           |
-| `EXECUTOR_ORG_NAME`                 | `Default`                       | Display name of the single org every user joins.                                                |
-| `EXECUTOR_ORG_SLUG`                 | `default`                       | URL slug for that org.                                                                          |
-| `EXECUTOR_ALLOW_LOCAL_NETWORK`      | `false`                         | Allow sandboxed code to reach loopback / private addresses. Keep off unless you trust the code. |
+| Variable                            | Default                         | Purpose                                                                                                                                                                                                                                                                                                                                     |
+| ----------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `PORT`                              | `4788`                          | HTTP port the server listens on.                                                                                                                                                                                                                                                                                                            |
+| `EXECUTOR_HOST`                     | `0.0.0.0`                       | Bind address. The image binds all interfaces.                                                                                                                                                                                                                                                                                               |
+| `EXECUTOR_DATA_DIR`                 | `/data`                         | Directory holding the database and generated keys.                                                                                                                                                                                                                                                                                          |
+| `EXECUTOR_DB_PATH`                  | `<data dir>/data.db`            | SQLite database file.                                                                                                                                                                                                                                                                                                                       |
+| `EXECUTOR_WEB_BASE_URL`             | auto (`http://localhost:4788`)  | Public URL browsers use. Required behind a domain or TLS (see below).                                                                                                                                                                                                                                                                       |
+| `EXECUTOR_TRUSTED_ORIGINS`          | unset                           | Comma-separated browser aliases allowed to authenticate without changing the public URL.                                                                                                                                                                                                                                                    |
+| `BETTER_AUTH_SECRET`                | generated, persisted in `/data` | Session secret (32+ chars). Rotating it signs everyone out.                                                                                                                                                                                                                                                                                 |
+| `EXECUTOR_SECRET_KEY`               | generated, persisted in `/data` | Master key encrypting stored secrets. Set it to manage it yourself.                                                                                                                                                                                                                                                                         |
+| `EXECUTOR_BOOTSTRAP_ADMIN_EMAIL`    | unset                           | Pre-create the admin headlessly (with the password below); skips browser first-run.                                                                                                                                                                                                                                                         |
+| `EXECUTOR_BOOTSTRAP_ADMIN_PASSWORD` | unset                           | Password for the bootstrap admin.                                                                                                                                                                                                                                                                                                           |
+| `EXECUTOR_BOOTSTRAP_ADMIN_NAME`     | `Admin`                         | Display name for the bootstrap admin.                                                                                                                                                                                                                                                                                                       |
+| `EXECUTOR_ORG_NAME`                 | `Default`                       | Display name of the single org every user joins.                                                                                                                                                                                                                                                                                            |
+| `EXECUTOR_ORG_SLUG`                 | `default`                       | URL slug for that org.                                                                                                                                                                                                                                                                                                                      |
+| `EXECUTOR_ALLOW_LOCAL_NETWORK`      | `false`                         | Allow sandboxed code to reach loopback / private addresses. Keep off unless you trust the code.                                                                                                                                                                                                                                             |
+| `EXECUTOR_OAUTH_CIMD_ENABLED`       | `true`                          | Set `false` when upstream authorization servers cannot reach this instance's OAuth Client ID Metadata Document (CIMD); automatic connects then try Dynamic Client Registration (DCR) when available. Only exact `true` or `false` are accepted; any other value (including empty, uppercase, or whitespace-padded values) prevents startup. |
 
 Tracing is configured separately, and off unless you turn it on — see
 [Tracing](/hosted/tracing).

apps/host-selfhost/.env.example

@@ -36,6 +36,10 @@
 # default — adversarial generated code should not reach your internal network.
 # EXECUTOR_ALLOW_LOCAL_NETWORK=false
 
+# OAuth Client ID Metadata Document capability. For values and deployment
+# guidance, see ../docs/hosted/docker.mdx#environment-variables.
+# EXECUTOR_OAUTH_CIMD_ENABLED=true
+
 # --- Local stdio MCP (trusted deployments only) -------------------------------
 # Stdio MCP is disabled unless this is explicitly set to the exact string
 # "true". Enabling it lets users configure MCP servers whose commands execute

apps/host-selfhost/src/config.ts

@@ -86,6 +86,11 @@ export interface SelfHostConfig {
    * re-sync, leaving stale-marking and config revision as the only triggers.
    */
   readonly toolsSyncTtlMs: number | null | undefined;
+  /**
+   * Resolved `EXECUTOR_OAUTH_CIMD_ENABLED`; see apps/docs/hosted/docker.mdx.
+   * Passed to `ExecutorConfig.oauthClientIdMetadataDocumentEnabled`.
+   */
+  readonly oauthCimdEnabled: boolean;
 }
 
 export const resolveDataDir = (): string =>
@@ -197,6 +202,7 @@ export const loadConfig = (): SelfHostConfig => {
     sso: resolveSso(),
     mcpSessionIdleTtlMs: resolveMcpSessionIdleTtlMs(),
     toolsSyncTtlMs: resolveToolsSyncTtlMs(),
+    oauthCimdEnabled: resolveOauthCimdEnabled(),
   };
 };
 
@@ -256,6 +262,14 @@ const resolveSso = (): SsoConfig | undefined => {
   return { providerId, providerName, discoveryUrl, clientId, clientSecret, allowedDomains };
 };
 
+const resolveOauthCimdEnabled = (): boolean => {
+  const raw = process.env.EXECUTOR_OAUTH_CIMD_ENABLED;
+  if (raw === undefined || raw === "true") return true;
+  if (raw === "false") return false;
+  // oxlint-disable-next-line executor/no-try-catch-or-throw, executor/no-error-constructor -- boundary: refuse to boot on a malformed operator knob
+  throw new Error(`EXECUTOR_OAUTH_CIMD_ENABLED ${JSON.stringify(raw)} must be "true" or "false"`);
+};
+
 // A malformed value is refused rather than silently ignored: an operator who
 // sets the knob and typos it should find out at boot, not by watching a
 // runaway execution use the 5-minute default.

apps/host-selfhost/src/execution.ts

@@ -56,6 +56,7 @@ export const SelfHostHostConfig: Layer.Layer<HostConfig> = Layer.sync(HostConfig
     webBaseUrl: config.webBaseUrl,
     oauthCallbackPath: "/api/oauth/callback",
     toolsSyncTtlMs: config.toolsSyncTtlMs,
+    oauthClientIdMetadataDocumentEnabled: config.oauthCimdEnabled,
     onIntegrationChange: (event) =>
       selfHostAnalytics.record(
         event.kind === "added" ? "integration_added" : "integration_removed",

apps/host-selfhost/src/executor-config.test.ts

@@ -6,9 +6,11 @@ import executorConfig from "../executor.config";
 const ENV_NAME = "EXECUTOR_ALLOW_STDIO_MCP";
 const SECRET_ENV_NAME = "EXECUTOR_SECRET_KEY";
 const TTL_ENV_NAME = "EXECUTOR_TOOLS_SYNC_TTL_MS";
+const CIMD_ENV_NAME = "EXECUTOR_OAUTH_CIMD_ENABLED";
 const originalValue = process.env[ENV_NAME];
 const originalSecret = process.env[SECRET_ENV_NAME];
 const originalTtl = process.env[TTL_ENV_NAME];
+const originalCimd = process.env[CIMD_ENV_NAME];
 
 beforeEach(() => {
   process.env[SECRET_ENV_NAME] = originalSecret ?? "executor-config-test-secret";
@@ -30,6 +32,11 @@ afterEach(() => {
   } else {
     process.env[TTL_ENV_NAME] = originalTtl;
   }
+  if (originalCimd === undefined) {
+    delete process.env[CIMD_ENV_NAME];
+  } else {
+    process.env[CIMD_ENV_NAME] = originalCimd;
+  }
 });
 
 const allowStdio = (): boolean => {
@@ -112,3 +119,28 @@ test("a negative tools-sync TTL refuses to boot", () => {
   process.env[TTL_ENV_NAME] = "-1";
   expect(() => loadConfig()).toThrow(/must not be negative/);
 });
+
+test("CIMD serving is enabled when the knob is unset", () => {
+  delete process.env[CIMD_ENV_NAME];
+  expect(loadConfig().oauthCimdEnabled).toBe(true);
+});
+
+test("CIMD serving is disabled by false", () => {
+  process.env[CIMD_ENV_NAME] = "false";
+  expect(loadConfig().oauthCimdEnabled).toBe(false);
+});
+
+test("CIMD serving is enabled by true", () => {
+  process.env[CIMD_ENV_NAME] = "true";
+  expect(loadConfig().oauthCimdEnabled).toBe(true);
+});
+
+test.each(["disabled", "TRUE", "FALSE", "", "   ", " true", "true ", " false", "false "])(
+  "a malformed CIMD serving knob (%j) refuses to boot",
+  (raw) => {
+    process.env[CIMD_ENV_NAME] = raw;
+    expect(() => loadConfig()).toThrow(
+      `EXECUTOR_OAUTH_CIMD_ENABLED ${JSON.stringify(raw)} must be "true" or "false"`,
+    );
+  },
+);

apps/host-selfhost/src/oauth-cimd-capability.test.ts

@@ -0,0 +1,61 @@
+import { mkdtempSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { afterAll, beforeAll, expect, test } from "@effect/vitest";
+import { Effect } from "effect";
+
+import { serveOAuthTestServer } from "@executor-js/sdk/testing";
+
+// Config reads the environment, so set the knob (and allow the loopback test
+// AS through the hosted HTTP client) before importing the app graph.
+process.env.EXECUTOR_DATA_DIR = mkdtempSync(join(tmpdir(), "eh-cimd-"));
+process.env.EXECUTOR_OAUTH_CIMD_ENABLED = "false";
+process.env.EXECUTOR_ALLOW_LOCAL_NETWORK = "true";
+
+let handler!: (request: Request) => Promise<Response>;
+let dispose: () => Promise<void> = async () => {};
+
+beforeAll(async () => {
+  const { makeSelfHostTestApp, singleAdminIdentityLayer } = await import("./testing/test-app");
+  const app = await makeSelfHostTestApp({
+    identity: singleAdminIdentityLayer({
+      userId: "admin",
+      organizationId: "default-org",
+      organizationName: "Default",
+    }),
+  });
+  handler = app.handler;
+  dispose = app.dispose;
+});
+
+afterAll(() => dispose());
+
+test("POST /api/oauth/probe hides CIMD when the deployment cannot serve the document", async () => {
+  await Effect.runPromise(
+    Effect.scoped(
+      Effect.gen(function* () {
+        const server = yield* serveOAuthTestServer({
+          clientIdMetadataDocumentSupported: true,
+        });
+        const res = yield* Effect.promise(() =>
+          handler(
+            new Request("http://localhost/api/oauth/probe", {
+              method: "POST",
+              headers: { "content-type": "application/json" },
+              body: JSON.stringify({ url: server.mcpResourceUrl }),
+            }),
+          ),
+        );
+        expect(res.status).toBe(200);
+        const body = yield* Effect.promise(() => res.json());
+        expect(body).toEqual(
+          expect.objectContaining({
+            clientIdMetadataDocumentSupported: false,
+            registrationEndpoint: server.registrationEndpoint,
+          }),
+        );
+      }),
+    ),
+  );
+});

e2e/RUNNING.md

@@ -169,6 +169,59 @@ When handing results to the user, follow the evidence contract in the root
 [AGENTS.md](../AGENTS.md) (direct run links + a live instance + what to try);
 [RUNNING.md](../RUNNING.md) has the current sharing/demo mechanics.
 
+## Docker OAuth deployment switch
+
+`selfhost-docker-cimd` is an opt-in browser suite for MCP and OpenAPI CIMD/DCR
+selection. It creates isolated hosted emulator instances and restarts the same
+Docker image and data volume with `EXECUTOR_OAUTH_CIMD_ENABLED=false`, then with
+the variable absent. It checks provider registration, token exchange, and an
+authenticated tool call through Executor, and records browser traces and ledgers.
+The separate OpenAPI scenario removes a custom method while CIMD is disabled
+and checks that restarting preserves the original OAuth configuration.
+
+Provide an explicit image, the dedicated test container port, and its reachable
+web URL (the CIMD document must be reachable by the hosted authorization server):
+
+```sh
+E2E_SELFHOST_DOCKER_IMAGE=executor-selfhost:e2e \
+E2E_SELFHOST_DOCKER_PORT=42885 \
+E2E_SELFHOST_DOCKER_URL=https://your-test-instance.example \
+bunx vitest run --project selfhost-docker-cimd
+```
+
+The initial container must already be running at that URL. The suite owns and
+restarts `executor-e2e-selfhost-docker-<port>`; use a dedicated synthetic test
+instance. The hosted MCP emulator must implement the
+`mcp.oauth.clientIdMetadataDocumentSupported` seed option.
+
+For explicitly authorized local emulator verification, set `E2E_CIMD_MCP_URL`
+and `E2E_CIMD_OPENAPI_URL` to dedicated fresh emulator processes reachable from
+both Docker and the browser. This attaches to those processes instead of creating
+hosted instances; the same browser, token, and authenticated-operation assertions
+still run. Runtime `/_emulate/seed` bodies contain the service configuration
+directly, without the service-name wrapper used by startup configuration.
+
+The separate `selfhost-docker-cimd-legacy` project checks an upgrade from an
+image that stored discovered CIMD templates without `discoveryUrl`. Set
+`E2E_CIMD_LEGACY_IMAGE` to that older image and `E2E_SELFHOST_DOCKER_IMAGE` to the
+image under review, using the same port, URL, and optional local provider settings
+above. It uses the emulator's fault control to return 404 for protected-resource
+metadata while retaining issuer discovery. The old image must create the template
+and complete real CIMD authorization; the upgraded image must preserve the existing
+connection and complete another authorization on the same integration. It also
+checks persistence of the recovered URL and rejection of mismatched OAuth endpoints.
+Set `E2E_CIMD_OPENAPI_PATH_URL` to a second emulator mounted at a path-based issuer
+(e.g. `https://provider.example/tenant`) to run the same upgrade for both issuer shapes.
+
+```sh
+E2E_CIMD_LEGACY_IMAGE=executor-cimd:before \
+E2E_CIMD_OPENAPI_PATH_URL=https://provider.example/tenant \
+E2E_SELFHOST_DOCKER_IMAGE=executor-cimd:legacy-fixed \
+E2E_SELFHOST_DOCKER_PORT=42905 \
+E2E_SELFHOST_DOCKER_URL=https://your-test-instance.example \
+bunx vitest run --project selfhost-docker-cimd-legacy
+```
+
 ## Desktop targets (the app on real OSes, filmed)
 
 The packaged desktop app runs as its own targets, each landing in its own