APS-4720 Resolve trust-sign kid by matching the mounted private key

[Elson9]

Summary

• Let trust-sign resolve JWT kid by matching the mounted private key to config.keyset_name, instead of a hardcoded :0 kid.
• Keep explicit config.keyid as a rollback path; fail closed on zero or multiple matches.
• Reuse the same resolver in token-exchange via optional keyset_name without mutating plugin config.

Test plan

• Explicit keyid still wins
• Match is independent of keyset order (overlap rotation)
• Fail closed on missing / duplicate public-key matches
• Cache invalidates when the private-key fingerprint changes

[github-actions]

Playwright E2E tests failed (2) — kong-3.9.1 keycloak-26.5.3

Branch: feature/aps-4720-trust-sign-kid-resolution | Commit: a703e0d

Stats

startTime   2026-08-18T23:18:26.819Z
duration    716803.003
expected    96
skipped     0
unexpected  2
flaky       0

Failed tests (2)

• plugins/oidc/unauth_action.spec.ts - unauth_action - deny
• plugins/trust-verify-signature/key-discovery.spec.ts - missing kid does not refresh within grace period

View workflow run

[Elson9]

Playwright E2E tests failed (2) — kong-3.9.1 keycloak-26.5.3

Branch: feature/aps-4720-trust-sign-kid-resolution | Commit: a703e0d

Stats

startTime   2026-08-18T23:18:26.819Z
duration    716803.003
expected    96
skipped     0
unexpected  2
flaky       0

Failed tests (2)

• plugins/oidc/unauth_action.spec.ts - unauth_action - deny
• plugins/trust-verify-signature/key-discovery.spec.ts - missing kid does not refresh within grace period

View workflow run

Unrelated tests

[phowells]

Could you clarify how the fingerprint is expected to observe a key change at the same mounted path? private_pem comes from get_kong_key, which caches by path with ttl = 0, so subsequent calls keep hashing the original bytes until that cache or worker is restarted. If key promotion always restarts Kong, documenting and testing that assumption may be enough; if in-place rotation is intended, this cache needs a refresh path. The current test checks the cache-key shape for one value, but not invalidation after the file changes.

[Elson9]

Key promotion restarts Kong (RG rolling restart after the new public key is provisioned). get_kong_key caches the PEM with ttl = 0, so both signing and kid matching see a new key at the same path only after that restart. The kid cache fingerprints those bytes, so a restart produces a new cache key. Tests cover PEM-byte change vs. the ttl = 0 hold.

[ikethecoder]

we need to consider how this is going to rotate smoothly without a failed request. If the bootstrap job runs to create a new key pair and does a rolling restart, the public key needs to get updated in between those two events, otherwise there is risk that a token is signed with the new key, but the keyset doesn't have the public key to validate yet. Might need to look at a way for the bootstrap job to update the keyset, or have the job be a 2-step process - one that stages a new key paid, and then another that waits until the public key exists before promoting it to the real keys and doing the rolling restarts.

[Elson9]

Thanks, I agree. We need a gap between generating the new key pair and the rolling restart so the public key can land in the keyset first.

The sequence we have in the other APS-4720 PRs:

1. Stage a new key pair without restarting Kong (bootstrap.stageSecret writes {release}-client-next). See aps-devops #15.
2. Publish the new public key and keep the old one (sdx-keys.r1 operation=rotate). See api-services-portal #1546 and gwa-api #293.
3. Confirm JWKS has both kids, then promote the staged secret and rolling-restart (rotation.promote).

This plugin just matches the mounted private key against the keyset. If there's no match it fails closed, so we don't emit a token that can't verify. If rotate has already landed and we haven't promoted yet, traffic still signs with the old key.

The wait-for-JWKS step is still something we do operationally. The promote Job doesn't check for it. Happy to follow up on the chart if we want promote to wait until the new kid is actually visible.

[phowells]

Thanks, that clears up my original cache question. The code and test now make the restart boundary explicit.

[github-actions]

Playwright E2E tests failed (1) — kong-3.9.1 keycloak-26.5.3

Branch: feature/aps-4720-trust-sign-kid-resolution | Commit: ba2c025

Stats

startTime   2026-08-21T17:56:10.814Z
duration    804249.014
expected    122
skipped     0
unexpected  1
flaky       0

Failed tests (1)

• plugins/oidc/misc.spec.ts - ignore_auth_filters[yes]

View workflow run

[phowells]

Thanks, that clears up my original cache question. The code and test now make the restart boundary explicit.