Spec: consolidate sl4jx into DataSource-Extensions (3.x minor)

[rossdanderson]

Execution spec for the consolidation planned in the wayfinder map #57. Each decision below links the
ticket holding its full detail and per-item execution checklist.

Problem Statement

Caplin ships two complementary Kotlin libraries: DataSource-Extensions (datasourcex-*, wrapping
the server-side DataSource SDK) and sl4j-extensions (sl4jx, wrapping the client-side StreamLink
SDK). Today they live in separate repositories with separate release cadences, separate build stacks,
and separate dependency-repository setups.

A developer building a Caplin integration typically uses both — a DataSource adapter and a
StreamLink client to test it against — and today must track two releases, reconcile two toolchains,
and wire two Maven-repo configurations. The libraries also duplicate the core Subject concept in
two near-identical types, use the terms subject and path inconsistently across their public
APIs, and offer no shared, docker-backed integration-test harness for proving an adapter serves data
end-to-end through a live Liberator. Maintainers pay the cost twice.

Solution

Consolidate sl4jx into the DataSource-Extensions repository as a single, co-released monorepo,
shipped as a non-breaking minor on the current 3.x line (currently 3.1.0-RC5) — not a new major.

• The StreamLink extensions become the published module sl4jx-kotlin, and the existing
  datasourcex reactive modules regroup so both SDK families sit side by side under reactive/.
• A shared, published test-support module — datasourcex-kotest — provides a Kotest
  Testcontainers Liberator plus a StreamLink client, usable by the integration tests of both
  families.
• The two Subject types are unified into one.
• A single, mechanical path/subject naming convention (recorded in ADR 0002) is applied across
  both families and the code generator.
• An example adapter gains a docker-backed integration test that proves it serves data end-to-end,
  excluded from CI (which has no Docker registry access).

The result: one artifact set, one version, one build, one dependency-repo setup, one naming
convention, one shared test harness — with no breaking change to already-released datasourcex APIs.

User Stories

1. As a Caplin integration developer, I want DataSource and StreamLink extensions released together
   under one version, so that I don't have to reconcile two independent release cadences.
2. As a Caplin integration developer, I want to add both families with one Maven-repo configuration,
   so that I don't maintain two sets of repository/credential setup.
3. As a StreamLink client developer, I want the client extensions published as sl4jx-kotlin, so
   that the coordinate signals it is the Kotlin/Flow variant and leaves room for future variants.
4. As an adapter developer, I want the datasourcex reactive modules grouped under
   reactive/datasource/, so that the repository layout makes the two SDK families obvious at a
   glance.
5. As a consumer of the already-released datasourcex-* artifacts, I want the consolidation to be
   binary-compatible, so that upgrading to the new minor does not break my build.
6. As a consumer who used a subject-named public member that becomes path, I want the old name to
   keep working with a deprecation warning, so that I can migrate at my own pace.
7. As an API user, I want a variable that holds a subject's string form to always be called path
   and one holding a Subject object to always be called subject, so that the API reads
   predictably.
8. As a developer working across both families, I want a single Subject type, so that I don't
   convert between two near-identical types.
9. As a StreamLink client author, I want query parameters in a rendered subject to sort
   deterministically, so that equivalent subjects render identically.
10. As an adapter test author, I want a Kotest extension that boots a Liberator in a container and
    hands me its DataSource port plus a StreamLink client, so that I can test my adapter end-to-end
    without standing up infrastructure myself.
11. As a StreamLink test author, I want that same extension, so that both families share one test
    harness.
12. As a maintainer, I want the shared test module to depend only on the StreamLink client (not on
    datasourcex runtime modules), so that the harness stays client-oriented and dependency-light.
13. As a CI operator, I want docker-backed integration tests excluded from the CI build, so that CI
    (which cannot reach the Docker registry) stays green.
14. As a developer, I want to run the docker-backed integration tests locally on demand, so that I
    can verify end-to-end behaviour before pushing.
15. As a spring-kotlin example user, I want a docker-backed integration test that starts the real
    adapter against a Liberator and subscribes as a client, so that the example doubles as a proof
    the starter serves data end-to-end.
16. As that test's author, I want to point the adapter at the container's mapped DataSource port via
    configuration alone, so that no production code changes just to be testable.
17. As that test's author, I want to assert the public Payload stream (incrementing version, echoed
    parameters, non-null uuid) and the container's initial 10-row bulk, so that record and container
    publishing are both proven.
18. As that test's author, I want to assert the USER_ID and persistent-session tokens are threaded
    into the payload, so that the ingress-token plumbing is proven end-to-end.
19. As a maintainer, I want the incoming modules to adopt this repo's Kotlin/Kotest/Jackson versions
    and build conventions, so that there is one toolchain to maintain.
20. As a maintainer, I want Caplin dependencies (streamlink, keymaster) pinned to exact versions
    like datasource, so that builds are reproducible and configuration-cache-friendly.
21. As a maintainer, I want Jackson and slf4j versions to come from the Spring Boot BOM, so that
    version alignment is centralised.
22. As a maintainer, I want the StreamLink publish-time POM to keep excluding transitive junit, so
    that consumers don't inherit it (preserving the PSL-889 workaround).
23. As a release manager, I want the new modules to have committed .api baselines, so that
    apiCheck protects their public surface from unintended drift.
24. As a documentation reader, I want the aggregated API docs and the repo's guidance docs to reflect
    the consolidated module set, so that the docs match the artifacts.
25. As a contributor, I want the code generator to keep emitting the path/subject naming
    convention, so that generated DSL stays consistent with hand-written code.
26. As a maintainer, I want the stale AGENTS.md claim that datasourcex-util has no DataSource SDK
    dependency corrected, so that future decisions aren't misled by it.

Implementation Decisions

Module layout and import — #59, and import mechanism

• Regroup the existing reactive modules physically into reactive/datasource/* and bring the
  StreamLink extensions in as reactive/streamlink/kotlin. Move physical folders only; keep the
  Gradle project paths unchanged by giving each module a projectDir override in
  settings.gradle.kts. Published coordinates stay pinned (datasourcex-kotlin, etc.); the new
  module publishes as sl4jx-kotlin. Existing project(":reactive:datasourcex-*") references do not
  change.
• reactive/streamlink stays a single kotlin module (no core/api split now).
• Import the sl4jx source as a plain copy in a fresh commit (history remains in the archived
  repo).
• The one place with hardcoded physical paths — the CI test-report globs — updates to the new
  reactive/datasource/<variant> locations, plus the streamlink and kotest modules.

Dependencies and repositories — #58

• No settings.gradle.kts repository change. The existing authenticated
  repository.caplin.com/caplin-release + mavenCentral() already resolve StreamLinkJava,
  KeyMasterMain, and all sl4jx transitive dependencies.

Toolchain and build conventions — #61

• Incoming modules adopt this repo's stack: Kotlin 2.2.21, Kotest 6.2.2, mockk 1.14.11,
  kotlinx-collections-immutable 0.5.1. (Note: immutable 0.5.0 renamed put→putting /
  putAll→puttingAll; migrate sl4jx call sites.)
• Jackson (tools.jackson.*, Jackson 3) and slf4j versions come from the Spring Boot BOM; drop
  sl4jx's explicit Jackson/slf4j pins and its catalog bundles. Keep kotlin-logging (a new catalog
  entry) since sl4jx logs through it.
• Add catalog entries streamlink = 8.0.7 and keymaster = 8.0.3 (exact pins, matching how
  datasource is pinned).
• reactive/streamlink/kotlin and datasourcex-kotest apply the common-library convention
  (not common-reactive-library — sl4jx is hand-written, no code generation). Drop sl4jx's
  common-dokka convention; use this repo's inline per-module dokka pattern.
• Carry the PSL-889 publish-time POM hack (strip transitive junit:junit from the published
  StreamLinkJava dependency) into the streamlink module verbatim, keeping its TODO.

Subject unification — #62

• The canonical type is the existing com.caplin.integration.datasourcex.util.Subject, left
  unchanged. sl4jx-kotlin depends on datasourcex-util and uses it; the former
  com.caplin.integration.streamlinkx.Subject (and its private impl / path extension / buildPath)
  is deleted and all references retarget.
• This is non-breaking because util.Subject shipped only in the unreleased 3.1 line and
  sl4jx-kotlin is new to this repo.
• Accepted trade-off: because datasourcex-util api-depends on the DataSource server SDK,
  sl4jx-kotlin (a client library) transitively gains that SDK plus caffeine and the Spring Boot
  BOM. Extracting an SDK-free subject module was considered and declined in favour of keeping the
  type in util.
• Query parameters render sorted (util.Subject uses a SortedMap); this changes sl4jx's former
  insertion-order rendering. No sl4jx code depends on insertion order.

Naming convention — #60, ADR 0002

• Mechanical rule: a variable/parameter/property holding a subject's String form is named path;
  one typed Subject is named subject. Applied across both families, internal code, and the
  buildSrc code generator (which already emits path: String). Method names describe an operation,
  not a variable, and are unaffected.
• The only binary-breaking public consequence is BroadcastEvent's subject: String property.
  Introduce it by deprecate-and-add: add a path member and keep subject as a @Deprecated
  alias delegating to it, so apiCheck stays green and it ships in the minor. Parameter renames
  (e.g. in AntPatternNamespace) and internal (reactive.core, spring.internal) renames do not
  touch .api.

Shared test-support module — #63

• New top-level module at kotest/, published as datasourcex-kotest, package
  com.caplin.integration.kotest (repackaged from …streamlinkx.test).
• Public surface (client-side only): LiberatorContainerExtension (with nested Config,
  Liberator, and the ContainerConfig hierarchy DefaultContainerConfig / CustomContainerConfig)
  and the StreamLinkConnection.awaitConnected helper. Liberator exposes dataSourcePort,
  httpPort, containerState, and connect(username): StreamLinkConnection. No datasource
  adapter-boot helper — tests boot their own adapter.
• api-depends on sl4jx-kotlin (StreamLink client, transitively KeyMaster), kotest-framework-engine
  (a new catalog entry), testcontainers, and kotest-extensions-testcontainers. No direct
  dependency on any datasourcex reactive/spring module. Verify the testcontainers version (prefer the
  Spring BOM) and the kotest-extensions-testcontainers group/version against Kotest 6.2.2 (it is
  versioned independently of kotest core).
• The default-liberator/{Dockerfile, liberator.sh} classpath resources move with it; base image
  pin docker-release.caplin.com/platform/core:8.0.19. The module keeps its own docker-backed IT
  (StreamLinkTest), not wired into check.

Example integration test — #64

• Add an integrationTest JvmTestSuite to examples/spring-kotlin depending on
  datasourcex-kotest, the Kotest runner, and Turbine. It is not wired into check and Kover
  instrumentation is disabled for it — so CI never runs it; developers run it explicitly.

• No production change to the example: the adapter is pointed at the container's mapped DataSource
  port purely by overriding caplin.datasource.managed.peer.outgoing at startup. The following
  shape came from the prototype in spring-kotlin docker-backed integrationTest design #64 and encodes the key decision (start-the-real-adapter-via-config

  • assert through a StreamLink client):

  val liberator = install(LiberatorContainerExtension())
  val ctx = SpringApplicationBuilder(SpringKotlinApplication::class.java)
      .properties("caplin.datasource.managed.peer.outgoing=" +
          "ws://${liberator.containerState.host}:${liberator.dataSourcePort}")
      .run()
  // then: liberator.connect("alice").getSubject<Payload>(Subject("public","stream","foo","42"))…
  //   assert first Payload: version 0, parameter1 "foo", parameter2 42, uuid non-null; next version 1
  //   container: getContainer(Subject("public","container","foo","42")) → initial Bulk of 10 rows
  //   user/session: assert injected USER_ID / session id thread into the payload

• Proving the USER_ID injection overrides the subject path (vs. the client supplying it) requires a
  Liberator object-map (%u) override supplied via DefaultContainerConfig(adapterConfig = …); the
  persistent-session assertion requires establishing a persistent session. These are validated during
  execution.

Cross-cutting wiring

• Add sl4jx-kotlin and datasourcex-kotest to settings.gradle.kts, the datasourcex-version-catalog
  module, and the aggregated api-docs Dokka set (lean: exclude datasourcex-kotest from the
  published docs as test infrastructure). Exclude datasourcex-kotest from Kover aggregation.
• Generate committed .api baselines for the two new published modules (apiDump).
• Correct the stale "No DataSource SDK dependency" claim for datasourcex-util in AGENTS.md, and
  refresh the README compatibility/module-layout sections and spring/docs/GUIDE.md for the
  consolidated module set.

Testing Decisions

Two seams; a good test asserts external behaviour (rendered subject strings, records/containers
delivered over a live connection, the published API surface) and never reaches into implementation
details.

Seam 1 — ./gradlew check (existing, highest): the per-module Kotest unit suites + apiCheck.

• The Subject-unification behaviour (path rendering, now-sorted query parameters) is asserted at the
  existing SubjectTest / AntPatternNamespaceTest seam in datasourcex-util; extend these to cover
  the behaviour sl4jx previously relied on now that it renders through util.Subject.
• BinderTest (reactive/core) and BindTest (reactive/kotlin) guard that Binder/DSL behaviour is
  unchanged by the naming/rename work.
• apiCheck (binary-compatibility-validator) is the seam that enforces the non-breaking-minor
  invariant: coordinates unchanged, new baselines for sl4jx-kotlin/datasourcex-kotest, and the
  only public .api diff being the BroadcastEvent deprecate-and-add.
• Compilation of the whole restructured graph proves the module moves, dependency rewiring, and
  catalog merge are consistent.
• Prior art: the module unit tests listed above (Kotest FunSpec/DescribeSpec, Kotest matchers,
  MockK, Turbine for Flow).

Seam 2 — the live-Liberator integration tests (new, CI-excluded).

• datasourcex-kotest's own StreamLinkTest (fetches /SYSTEM/INFO from a real Liberator) and the
  new examples/spring-kotlin integration test (starts the real adapter, subscribes as a StreamLink
  client, asserts the public stream/container payloads and the USER_ID/session token injection).
• Prior art: the example-adapter-kotlin CalendarAdapterIntegrationTest in application-apis — the
  same shape (Liberator Testcontainer → adapter against the mapped DataSource port → StreamLink client
  assertions via Turbine), adapted to this repo's build.
• These are docker-backed, so they are not wired into check and are run on demand
  (./gradlew :<module>:integrationTest).

Out of Scope

• Backporting any of this (sl4jx, the Subject unification, the naming convention) to the
  springboot-3.5.x maintenance branch, including any Jackson-2 layer sl4jx would need there — that
  branch is bugfix-only.
• Future reactive/streamlink/java-flow and reactive/streamlink/reactivestreams variants — the
  -kotlin suffix leaves room for them, but they are not built here.
• Removing the @Deprecated subject aliases introduced by the naming convention — that removal is
  breaking and reserved for a later major release.
• Extracting an SDK-free shared subject module (declined in Unify the Subject type into datasourcex-util #62); revisiting the resulting
  client→server-SDK coupling is deferred.

Further Notes

• The full decision record and per-item execution checklists live on the wayfinder map Wayfinder: consolidate sl4jx into DataSource-Extensions (3.x minor) #57 and its
  closed child issues Dependency-repo reconciliation (StreamLink/KeyMaster availability) #58–spring-kotlin docker-backed integrationTest design #64. This spec is the execution brief; those tickets are the detail.
• Suggested execution order: module regroup + plain-copy import first (mechanical, unblocks
  everything), then toolchain/catalog reconciliation, then Subject unification + naming convention,
  then the datasourcex-kotest module, then the example IT, then the cross-cutting docs/wiring and
  apiDump.
• ADR 0002-subject-path-naming-convention.md was authored during planning and is currently
  uncommitted in the working tree; commit it alongside the naming work.