Repository navigation
Report a truncated TLS session when the transport output is cancelled - #3740
Conversation
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open issues
What changed in this PR
This PR makes transport-output cancellation observable to TLS consumers by reporting inbound truncation (SessionTruncated) when buffered inbound records become unreachable due to teardown triggered by cancelling the ciphertext output.
Changes:
- Emit
SessionTruncatedon the transport-output-cancel teardown path in bothTlsGraphStageand the legacyTLSActorimplementation. - Add a dedicated TLSActor phase to defer the truncation report until plaintext output demand (and proactively cancel read-side input to avoid buffering overrun).
- Add a new spec covering multiple buffering/demand/cancellation permutations and document the contract in the stream IO guide.
| File | Description |
|---|---|
| stream/src/main/scala/org/apache/pekko/stream/impl/io/TlsGraphStage.scala | Signals inbound truncation on TransportOut cancellation before completing. |
| stream/src/main/scala/org/apache/pekko/stream/impl/io/TLSActor.scala | Introduces a truncation-signalling phase for the legacy actor and cancels inputs to prevent buffer overrun while waiting for demand. |
| stream-tests/src/test/scala/org/apache/pekko/stream/io/TlsTransportOutputCancelSpec.scala | Adds comprehensive regression tests for truncation semantics under transport-output cancellation. |
| docs/src/main/paradox/stream/stream-io.md | Documents the new observable contract when transport output is cancelled. |
Review effort: Lite
|
@He-Pin is this aimed at 2.0.0? Nothing else blocks a 2.0.0 release at the moment. If we choose to hold this to 2.1.0, then I can create the RC1 for 2.0.0. |
d3a5ce6 to
8cd70ec
Compare
pjfanning
left a comment
There was a problem hiding this comment.
Thanks for working on this. A few concerns are noted inline. The main ones:
- Silent drops remain in two paths: renegotiation in flight, and a pending plaintext element (both engines for the first).
- Behaviour change:
SessionTruncatedis now emitted on any transport-output cancellation of an established session, even with nothing buffered. closeInbound()before close_notify may invalidate theSSLSessionand break resumption.- Teardown now waits on plaintext demand, so the actor or stage can stay alive if downstream never pulls.
8cd70ec to
a01263f
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Changes are still needed because both TLS cancellation paths can treat a depleted transport input as ready, invoke closeInbound, and invalidate the session instead of emitting SessionTruncated while preserving it.
2 open issues
3 resolved since last review
Review effort: Lite
a01263f to
5ed4a10
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The change spans intricate TLS-engine interactions across cancellation, backpressure, renegotiation, and provider-specific state, so maintainer confirmation of the teardown contract is warranted despite extensive tests.
No open issues 🎉
2 resolved since last review
Review effort: Lite
pjfanning
left a comment
There was a problem hiding this comment.
Thanks for this. Some review findings are inline. The one most worth addressing is the plaintext input left open after the transport output is cancelled (TlsGraphStage onCancel). The partial-record and AwaitingClose cases are edge cases worth weighing.
One point outside the diff: Tcp's tlsWrapping (stream/src/main/scala/org/apache/pekko/stream/scaladsl/Tcp.scala, around line 99) uses collect { case sb: SessionBytes => sb.bytes }, which drops SessionTruncated. So users of Tcp().outgoingConnectionWithTls / bindWithTls still get a normal completion when the session is truncated, and the release-note advice to handle SessionTruncated doesn't reach them. That could be a follow-up (fail the ByteString flow on SessionTruncated), but the docs/release note should probably say that the signal is only visible at the TLS BidiFlow level.
5ed4a10 to
2e8364f
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The change adds interacting SSLEngine teardown state machines with deferred close_notify handling, while reentrant cancellation and provider-specific close or renegotiation behavior still require maintainer validation.
No open issues 🎉
Review effort: Lite
pjfanning
left a comment
There was a problem hiding this comment.
Thanks for the updates. I re-reviewed 2e8364f8. My earlier points are handled: the plaintext input is now cancelled on transport-output cancellation, AwaitingClose is kept, the failure is no longer re-run (isCompleted = stopped plus the idempotent pumpFinished), the predicates are shared in TlsEngineHelpers, the phase constant is reordered, and the Tcp filtering is documented. Each one has a test. I'm fine with documenting the partial-record behaviour instead of waiting for more ciphertext.
There is one new issue in the graph stage (inline).
Motivation: Cancelling a ciphertext output can make received TLS data unreachable. Completing the plaintext output silently hides the unclean shutdown. Modification: - Handle transport-output cancellation in both TLS engines, cancel the plaintext input immediately, and allow at most one final inbound round. - Report SessionTruncated for established sessions without a peer close_notify. Preserve pending plaintext and session validity, including during renegotiation, and skip depleted transport input. - Preserve an existing wait for the peer's close_notify after the local close_notify has been pushed. End that wait if both outputs are cancelled or cancellation discards a buffered close_notify. - Share cancellation predicates, preserve actor error publication order, and make actor failure cleanup safe against pump reentrancy. - Add directional tests and document the 2.0.0 behavior and the TCP ByteString APIs' filtering of SessionTruncated. Result: Direct TLS BidiFlow consumers can observe an inbound session abandoned by cancellation. The final signal obeys backpressure; incomplete input and data beyond the final inbound round can still be discarded. Tests: - JDK 17: sbt "stream-tests/Test/testOnly *TlsTransportOutputCancelSpec *TlsGraphStageDeferredCloseSpec": 119 passed, 0 failed. - The deferred-initial-close regression fails on 2e8364f and passes with the fix. Four ordinary no-demand close cases already pass on 2e8364f. - Before the pending-close guard: JDK 17 sbt "stream-tests/Test/testOnly *Tls*": 379 passed, 0 failed. - Eight upstream-cancellation and close-handshake regressions across both engines and TLS 1.2/1.3 were verified to fail before their fixes, as were the depleted-input and renegotiation regressions. - sbt +mimaReportBinaryIssues: passed on Scala 2.13.18 and 3.3.9 before the pending-close guard; the guard changes no binary shape. - sbt headerCreateAll +headerCheckAll checkCodeStyle: passed. - scalafmt diff check and git diff --check: passed. - sortImports was blocked by the existing scala.meta.package$.Input() NoSuchMethodError; unrelated partial edits were removed. - An earlier JDK 25 run was interrupted after a ScalaTest SocketReporter Broken pipe error; final local validation uses JDK 17. - docs/paradox and validatePullRequest are delegated to remote CI; an earlier local docs/paradox attempt was interrupted. References: Refs apache#3739
2e8364f to
8bd6245
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The intertwined TLS shutdown, renegotiation, cancellation, and provider-specific SSLEngine changes require remote multi-JDK validation and maintainer review beyond the provided local evidence.
No open issues 🎉
Review effort: Lite
pjfanning
left a comment
There was a problem hiding this comment.
Re-reviewed 8bd62459. The pending-close_notify case is fixed: onCancel records hadPending before clearing the slot, so it only keeps the AwaitingClose wait when our close_notify was actually pushed. The new tests cover both engines with no ciphertext demand, plus the deferred initial-output path, and the docs describe the behaviour. I have no further findings.


Motivation
Cancelling a TLS operator's ciphertext output can make received input beyond one inbound pump round unreachable while completing the plaintext output silently. The single-record immediate-loss scenario originally described in #3739 does not reproduce; the related abandonment of remaining input does.
Modification
SessionTruncatedfor an established session whose peerclose_notifyhas not been processed. Preserve pending plaintext, track established sessions across renegotiation, and skip depleted input without callingcloseInbound()solely to detect truncation.close_notifyonce the localclose_notifyhas been pushed. End that wait if both outputs are cancelled or cancellation discards a still-bufferedclose_notify. A partial record does not cause cancellation to wait for future ciphertext.Result
Direct TLS
BidiFlowconsumers can observe an inbound session abandoned by cancellation. At most one final inbound round is processed; incomplete records and input beyond that round may still be discarded. The terminal signal obeys backpressure, so consumers must consume or cancel the plaintext output;completionTimeoutcan bound the wait.The Scala and Java
Tcp.bindWithTlsandTcp.outgoingConnectionWithTlsAPIs exposeByteStringstreams and filter outSessionTruncated. This PR documents that limitation without changing those wrappers to fail on the signal.Tests
sbt "stream-tests/Test/testOnly *TlsTransportOutputCancelSpec *TlsGraphStageDeferredCloseSpec": 119 passed, 0 failed, including 84 transport-output cancellation tests and 35 deferred-close tests.2e8364f8and passes with the pending-close guard. It freezes the initial output timer to reproduce a buffered close being discarded on cancellation. The four ordinary JDK no-demand close cases already pass on2e8364f8; they provide boundary coverage, not evidence of that failure in normal JDK shutdown.sbt "stream-tests/Test/testOnly *Tls*": 379 passed, 0 failed.sbt +mimaReportBinaryIssues: passed on Scala 2.13.18 and 3.3.9 before the pending-close guard, which changes no binary shape. The latest full PR validation remains delegated to remote CI.sbt headerCreateAll +headerCheckAll checkCodeStyle, scalafmt diff check andgit diff --check: passed.sortImportswas blocked by the existingNoSuchMethodError: scala.meta.package$.Input()instream-testsandmulti-node-testkit; unrelated partial edits were removed.SocketReporterfailed withBroken pipe; final local validation uses JDK 17.docs/paradoxandvalidatePullRequestare delegated to remote CI as requested; an earlier localdocs/paradoxattempt was interrupted.References
Refs #3739