http4s mtls dev toggle#2876
Open
constantine2nd wants to merge 4 commits into
Open
Conversation
Turning on the dev-only in-process mTLS termination previously meant editing five props. Reduce it to a single switch, and stop the switch from silently breaking local logins. - Http4sMtls: only mtls.enabled is required now. The four store props fall back to the checked-in dev pair (server.jks / server.trust.jks), each fallback logged at WARN naming the prop and its OBP_* environment equivalent, so a default is never silent. A missing store now fails with the resolved absolute path and the working directory in the message -- the defaults are repo-relative, so "launched from the wrong directory" is the realistic failure and a bare FileNotFoundException did not say so. - Http4sMtls.keyStoreTypeOf: load .p12/.pfx as PKCS12 and everything else as JKS, for keystore and truststore alike. The type cannot be sniffed cheaply and guessing wrong fails with an opaque parse error. This makes localhost_san_dns_ip.pfx usable -- the only local cert carrying DNS:localhost + IP:127.0.0.1, where the server.jks default has no SAN at all and relies on clients falling back to CN. - scripts/mtls_env.sh: exports the OBP_MTLS_* overrides. No new machinery was needed -- APIUtil.getPropsValue already reads OBP_<NAME> from the environment ahead of the props file, so every prop is env-overridable. Pre-exported values win. It also pins local_identity_provider. Enabling TLS means hostname must become https://, but hostname is the default for local_identity_provider (constant.scala), which is the provider column every AuthUser / ResourceUser row is keyed on -- moving the scheme alone gives existing local users a different provider string and orphans them. The pin reads the hostname via Lift's own resolution order: Lift maps Development to an EMPTY modeName, so development.default.props is never loaded and default.props is. Reading the wrong file there reintroduces exactly the orphaning this pin prevents. - scripts/java_env.sh: select a JDK >= 17 before any Maven work. The build compiles with -release 17 and a default JDK 11 fails with "'17' is not a valid choice for '-release'", which reads like a source error. An adequate JAVA_HOME is honoured; an inadequate one is replaced with an announcement. - Both build_and_run scripts: --mtls flag following the existing --clean/--online/--no-flush/--background idiom. mtls_env.sh is sourced after the build so it can never affect compilation. - Tests: extension-to-store-type mapping, and buildSslContext loading the .pfx. Verified live: plain HTTP rejected (curl 52), certless handshake rejected under client_auth=need (curl 56), 200 with a real client certificate, and a spoofed PSD2-CERT header replaced by the handshake certificate (1252-char exact match). Both scripts build and boot from a JDK 11 shell.
…ipts Both build_and_run scripts run under `set -e`, and both call mvn without guarding it. A failing build therefore aborted the script at the mvn line, so everything written to handle that failure never ran: - flushall_fast_build_and_run.sh captured BUILD_EXIT_CODE=$? and had a whole "detect incremental compilation cache issue, retry with a clean build" block keyed off it. None of it was reachable. The script exited silently with no diagnosis and no retry. - flushall_build_and_run.sh could not reach its BUILD_EXIT check, so it never printed the log tail it promises on failure. Bracket each mvn invocation with set +e / set -e so the exit code is captured and the existing handling runs. The symptom is easy to misread: the scripts pipe their output, so the exit code a caller sees is the pipeline's, not the script's -- a failed build could look like a successful run that simply produced no server. Verified by putting a failing mvn on PATH and running the real script: it now reports the failure, detects the cache-issue signature, retries with a clean build, tails the log and exits 1.
…sable
With requirePsd2Certificates=ONLINE, any request reaching passesPsd2Aisp
without a TPP-Signature-Certificate header produced
500 OBP-50000: Unknown Error.: Illegal base64 character 2d
getCertificateFromTppSignatureCertificate read the header via getHeaderValue,
which substitutes SecureRandomUtil.csprng.nextLong().toString when a header is
absent. That is a serviceable "never matches" sentinel for the string
comparisons it was written for (Digest, Signature), but the value here goes
straight into Base64.getDecoder: a negative Long renders with a leading '-',
which is 0x2d, and the decoder throws.
The bug was also flaky by construction. Only about half of calls got a negative
Long; the rest base64-decoded to garbage and failed later in getCertificatePem
with a different error. Same missing header, two symptoms.
Make getCertificateFromTppSignatureCertificate return Box[X509Certificate]
instead of throwing, and look the header up directly rather than through
getHeaderValue. Each failure mode is now a Failure: header absent, not base64,
base64 of something that is not a PEM, or a PEM that is not a certificate.
Call sites:
- APIUtil.passesPsd2ServiceProviderCommon: fail closed. passesPsd2ServiceProvider
already maps a Failure to 401 -- this is the path that produced the 500.
- BerlinGroupSigning.verifySignedRequest: checkRequestIsSigned only proves the
header is present, not usable, so an unusable one is a 401.
- BerlinGroupSigning.getOrCreateConsumer: leave the caller's result untouched.
Consumer creation is a side effect of a signed request, not the thing being
authorised; verifySignedRequest is what rejects a bad certificate.
- BerlinGroupCheck: without a certificate there is no serial number to compare
keyId.SN against, so it is an invalid signature header (400).
Found by running a real TPP (OBP-Hola) through the UK Open Banking consent
journey. Verified: the identical POST to
/open-banking/v3.1/account-access-consents now returns 401 with no "Illegal
base64" in the log; RegulatedEntityTest passes.
Note this changes only the failure mode, not the gate: a caller with no
TPP-Signature-Certificate is still rejected, just with the correct status.
…lly uses BEHAVIOUR CHANGE for UK Open Banking endpoints when requirePsd2Certificates=ONLINE. Read the note at the end before upgrading. The PSD2 gate (passesPsd2Aisp / passesPsd2Pisp) identified the TPP from the TPP-Signature-Certificate header regardless of which API standard was being served. That header is Berlin Group NextGenPSD2. UK Open Banking does not use it at all: OBIE identifies the TPP by the mTLS transport certificate, which reaches OBP as PSD2-CERT -- set by the reverse proxy that terminates mTLS, or by bootstrap.http4s.Http4sMtls in development. A correctly behaving OBIE client was therefore rejected for omitting a header its specification never mentions. BerlinGroupCheck.validate already scopes every OTHER piece of Berlin Group machinery to Berlin Group URLs -- the mandatory headers, the request-signature verification, the on-the-fly consumer creation. The PSD2 gate was the single place that crossed that boundary. tppCertificateForStandard brings it in line: UK Open Banking URLs read PSD2-CERT, everything else keeps the previous code path byte for byte. One call site changes, which covers all 10 UK enforcement sites (5 in v3.1.0, 5 in v4.0.1) and any added later. Berlin Group (37 sites) and the OBP-native endpoints (4) are untouched. Only the certificate SOURCE changes, not what is done with it: both branches feed the same regulated-entity lookup, which matches on issuer CN + serial number and works with any X509 certificate. No eIDAS QCStatement is required -- that is only needed by the CERTIFICATE mode, which is unaffected. Verified end to end against a real TPP (OBP-Hola) over mutual TLS: - before registering the entity: 401 OBP-34102 (regulated entity not found), proving the mTLS certificate is now the one being looked up, where previously the same call returned OBP-20306 (no certificate in the request at all); - after registering it: the UK v4.0.1 consent is created, Status AWAITINGAUTHORISATION, with the gate logging the matched entity and PSP_AI; - /berlin-group/v1.3/consents still rejects on missing tpp-signature-certificate, unchanged. RegulatedEntityTest and the mTLS suites pass (13 tests). No test in this repo sends TPP-Signature-Certificate to a UK endpoint -- only the Berlin Group suites reference that header. UPGRADE NOTE. Under eIDAS a TPP holds two different certificates: the QWAC, used for TLS and therefore surfacing as PSD2-CERT, and the QSEAL, used to sign and therefore carried in TPP-Signature-Certificate. They have different serial numbers and often different issuing CAs. Since regulated entities are matched on issuer CN + serial, a deployment that runs UK endpoints with requirePsd2Certificates=ONLINE and onboarded its TPPs by QSEAL will now match a different entity, or none: - no PSD2-CERT present (no mTLS proxy) -> previously working TPPs get 401; - QWAC registered as a different entity -> that entity's services decide the call. This cannot authorise an unregistered party: both paths require a CA + serial match and fail closed. Deployments on requirePsd2Certificates=NONE are unaffected because the gate does not run. Register the QWAC alongside the QSEAL to migrate.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.