Skip to content

[EXPERIMENT] build with Pekko2 - #151

Draft
pjfanning wants to merge 10 commits into
apache:mainfrom
pjfanning:pekko2
Draft

[EXPERIMENT] build with Pekko2#151
pjfanning wants to merge 10 commits into
apache:mainfrom
pjfanning:pekko2

Conversation

@pjfanning

@pjfanning pjfanning commented Apr 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Motivation

Pekko 2.0.0 is staged on repository.apache.org for its release vote. These
samples should build and run against it. Pekko 2 requires Java 17 and Scala
2.13+, so the samples can also drop older compatibility idioms.

Modification

Versions and build

  • Pekko 2.0.0 (staged). pekko-http goes to 2.0.0-M2, the latest available.
    Pekko gRPC 2.0.0-M2 and Pekko Management / Connectors Kafka 2.0.0-M1 are
    unchanged.
  • Add the Apache staging repository as a resolver to every sbt build
    (Resolver.ApacheMavenStagingRepo) and Maven pom.
  • Scala 3.3.9, sbt 1.13.1.
  • Maven poms compile with <release>17</release> instead of source/target 1.8.
  • CI and Docker images move from Java 11 to Java 17.
  • evictionErrorLevel := Level.Info where mixed milestones trip semver checks.
  • Pin pekko-pki in cluster-kubernetes-java. Maven's nearest-wins resolution
    otherwise pulls an older version and fails Pekko's mixed-version check.
  • docs-gen uses the latest Paradox plugins, so it runs on JDK 17.

Runtime

  • Add --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED to samples that
    use pekko-remote/cluster. Agrona 2.x, used by Artery, needs it. The flag is
    set on every launch path: .jvmopts, run/javaOptions,
    MultiJvm/jvmOptions, surefire argLine, .mvn/jvm.config and the Docker
    entrypoints.

API changes from Pekko 2

  • OverflowStrategy.dropNew becomes dropTail (cluster-client-grpc Java and
    Scala).
  • Http().bindAndHandle* becomes Http().newServerAt(...).bind(...).

Code modernization

  • Scala: scala.collection.JavaConverters becomes
    scala.jdk.CollectionConverters.
  • Java 17:
    • instanceof pattern matching instead of instanceof followed by a cast.
    • Stream.toList() instead of collect(Collectors.toList()).
    • List.of/Set.of/Set.copyOf/Map.copyOf instead of Arrays.asList
      and the Collections.* wrappers.
    • A switch expression for KillrWeather WeatherStation queries.

Result

All samples build against Pekko 2.0.0 on JDK 17 and use idiomatic Scala
2.13+/Java 17 code.

Tests

  • The earlier 2.0.0-M4 commits were checked locally: every sample passed
    sbt Test/compile (plus MultiJvm/compile) and mvn test-compile, and the
    sbt multi-jvm and Maven tests passed with the add-opens flag.
  • mvn test-compile passes for all 8 Maven samples with Pekko 2.0.0 and the
    Java 17 changes.
  • The Pekko 2.0.0 / pekko-http 2.0.0-M2 / Scala 3.3.9 / sbt 1.13.1 bump was
    not built locally for the sbt samples. This PR relies on CI for them.

References

None - tracks the Pekko 2.0.0 release candidate.

@pjfanning
pjfanning marked this pull request as draft April 25, 2026 12:20
pjfanning added a commit to pjfanning/incubator-pekko-samples that referenced this pull request Oct 4, 2026
@pjfanning
Upgrade to Pekko 2.0.0-M4 and fix compile issues
d64bbb4 
Motivation:
Pekko 2.0.0-M4 is the latest core milestone and removes APIs the samples
still used.

Modification:
- Bump Pekko core to 2.0.0-M4 and Pekko gRPC to 2.0.0-M2 (other modules
  stay on their latest 2.0.0-M1).
- Replace removed OverflowStrategy.dropNew with dropTail in the cluster
  client gRPC samples (Java and Scala).
- Replace removed Http().bindAndHandle/bindAndHandleAsync with
  newServerAt(...).bind(...).
- Align ScalaPB compilerplugin with pekko-grpc 2.0.0-M2 (1.0.0-alpha.5).
- Set evictionErrorLevel to Info where mixed milestones trip semver checks.
- Set scalaVersion at ThisBuild in grpc-kubernetes so subprojects do not
  fall back to Scala 2.12.

Result:
All sbt and Maven samples compile against Pekko 2.0.0-M4.

Tests:
- sbt Test/compile (+ MultiJvm/compile) and mvn test-compile: pass for all samples
- sharding-scala sbt test: pass; kafka-to-sharding-scala sbt test: pass
- cluster-client-grpc-java mvn test: fails without
  --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED (Agrona 2.x), passes with it
- cluster-client-grpc-scala multi-jvm:test: fails with the same Agrona error

References:
Refs apache#151
pjfanning added a commit to pjfanning/incubator-pekko-samples that referenced this pull request Oct 4, 2026
@pjfanning
Add jdk.internal.misc add-opens for samples using pekko-remote
6eaaa4a 
Motivation:
Pekko 2's pekko-remote depends on Agrona 2.x, which Artery uses for its
buffer pools. Agrona needs access to jdk.internal.misc.Unsafe, so any
node that starts remoting fails on JDK 17 with an IllegalAccessError.

Modification:
Pass --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED, only in
samples that use pekko-remote/cluster, through every launch path:
- .jvmopts for in-process sbt tests and non-forked sbt run
- run/javaOptions for forked runs and MultiJvm/jvmOptions for multi-jvm
- javaOptions for the forked kafka-to-sharding processor
- argLine property for Maven surefire and .mvn/jvm.config for exec:java
- start script / entrypoint of the Docker images
Docker base images move from Java 11 to eclipse-temurin:17, since
Pekko 2 requires Java 17.

Result:
Remote/cluster samples start and their tests pass on JDK 17.

Tests:
- sbt multi-jvm:test: cluster-java, cluster-scala, cluster-client-grpc-scala,
  distributed-data-java, distributed-data-scala - pass
- sbt test: distributed-workers-scala, persistence-dc-scala, sharding-java,
  sharding-scala, kafka-to-sharding-scala, cluster-kubernetes-scala - pass
- mvn test: cluster-java, cluster-client-grpc-java, persistence-dc-java,
  sharding-java - pass; cluster-kubernetes-java mvn test-compile - pass
- sbt stage for both docker-compose samples: start script contains the flag
- Docker-compose and minikube jobs: Not run

References:
Refs apache#151
pjfanning added a commit to pjfanning/incubator-pekko-samples that referenced this pull request Oct 4, 2026
@pjfanning
Pin pekko-pki in cluster-kubernetes-java
490776d 
Motivation:
The cluster-kubernetes-java pods failed at startup with Pekko's
mixed-version check: Maven's nearest-wins resolution picked pekko-pki
2.0.0-M1 from pekko-http/pekko-management while core is 2.0.0-M4.

Modification:
Add an explicit pekko-pki dependency at ${pekko.version}.

Result:
All core Pekko modules resolve to 2.0.0-M4 and the app starts.

Tests:
- Checked resolved core Pekko artifacts for every sample (sbt export
  fullClasspath / mvn dependency:list): only this sample was affected;
  sbt picks the highest version so sbt builds already got pekko-pki M4
- Ran DemoApp locally: no mixed-version error, reaches Cluster Bootstrap
- minikube job: Not run locally

References:
Refs apache#151
pjfanning added a commit to pjfanning/incubator-pekko-samples that referenced this pull request Oct 4, 2026
@pjfanning
Open java.lang for docs-gen on JDK 17
d2115b4 
Motivation:
With CI on JDK 17, `sbt paradox` in docs-gen fails: parboiled (used by
Paradox's markdown reader) reflectively calls ClassLoader.findLoadedClass,
which JDK 17 blocks without an add-opens.

Modification:
Add docs-gen/.jvmopts with --add-opens=java.base/java.lang=ALL-UNNAMED.
pekko-sbt-paradox 1.0.1 is already the latest release.

Result:
docs-gen builds on JDK 17.

Tests:
- docs-gen sbt paradox on JDK 17: fails without the file, passes with it

References:
Refs apache#151
pjfanning added a commit to pjfanning/incubator-pekko-samples that referenced this pull request Oct 4, 2026
@pjfanning
Use latest Paradox plugins in docs-gen instead of add-opens workaround
5fdf38f 
Motivation:
pekko-sbt-paradox 1.0.1 brings in sbt-paradox 0.9.2, whose parboiled
version does not work on JDK 17. The previous commit worked around this
with an add-opens in docs-gen/.jvmopts.

Modification:
- Declare sbt-paradox and sbt-paradox-theme 0.11.1 explicitly in
  docs-gen/project/plugins.sbt, as the pekko build does.
- Remove docs-gen/.jvmopts.
- Add docs-gen/.sbtopts with -J-Xmx2G: the newer plugins' ClassGraph
  scan across the 15 doc subprojects runs out of the default heap.

Result:
docs-gen builds on JDK 17 without opening java.lang.

Tests:
- docs-gen sbt paradox on JDK 17: pass, 15 sample sites generated
  (OutOfMemoryError without the heap setting)

References:
Refs apache#151
pjfanning added a commit to pjfanning/incubator-pekko-samples that referenced this pull request Oct 4, 2026
@pjfanning
Replace deprecated sbt in scoping with slash syntax
0b8d1c4 
Motivation:
sbt warns that the `key in Scope` notation is deprecated.

Modification:
Convert the remaining `in` scoping in the docker-compose and
cluster-kubernetes-scala builds to slash syntax.

Result:
No deprecated `in` scoping left in the sample builds.

Tests:
- docker-compose-java: show Docker/version, Docker/dockerExposedPorts,
  Docker/dockerEntrypoint - unchanged values
- cluster-kubernetes-scala: show Compile/run/mainClass - unchanged

References:
Refs apache#151
@He-Pin

He-Pin commented Nov 3, 2026

Copy link
Copy Markdown
Member

@pjfanning This need update too

@pjfanning
try to build with pekko 2
bb852d9
@pjfanning
java 17
3ba8f44
@pjfanning
Upgrade to Pekko 2.0.0-M4 and fix compile issues
5a47c93 
Motivation:
Pekko 2.0.0-M4 is the latest core milestone and removes APIs the samples
still used.

Modification:
- Bump Pekko core to 2.0.0-M4 and Pekko gRPC to 2.0.0-M2 (other modules
  stay on their latest 2.0.0-M1).
- Replace removed OverflowStrategy.dropNew with dropTail in the cluster
  client gRPC samples (Java and Scala).
- Replace removed Http().bindAndHandle/bindAndHandleAsync with
  newServerAt(...).bind(...).
- Align ScalaPB compilerplugin with pekko-grpc 2.0.0-M2 (1.0.0-alpha.5).
- Set evictionErrorLevel to Info where mixed milestones trip semver checks.
- Set scalaVersion at ThisBuild in grpc-kubernetes so subprojects do not
  fall back to Scala 2.12.

Result:
All sbt and Maven samples compile against Pekko 2.0.0-M4.

Tests:
- sbt Test/compile (+ MultiJvm/compile) and mvn test-compile: pass for all samples
- sharding-scala sbt test: pass; kafka-to-sharding-scala sbt test: pass
- cluster-client-grpc-java mvn test: fails without
  --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED (Agrona 2.x), passes with it
- cluster-client-grpc-scala multi-jvm:test: fails with the same Agrona error

References:
Refs apache#151
@pjfanning
Add jdk.internal.misc add-opens for samples using pekko-remote
b84a149 
Motivation:
Pekko 2's pekko-remote depends on Agrona 2.x, which Artery uses for its
buffer pools. Agrona needs access to jdk.internal.misc.Unsafe, so any
node that starts remoting fails on JDK 17 with an IllegalAccessError.

Modification:
Pass --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED, only in
samples that use pekko-remote/cluster, through every launch path:
- .jvmopts for in-process sbt tests and non-forked sbt run
- run/javaOptions for forked runs and MultiJvm/jvmOptions for multi-jvm
- javaOptions for the forked kafka-to-sharding processor
- argLine property for Maven surefire and .mvn/jvm.config for exec:java
- start script / entrypoint of the Docker images
Docker base images move from Java 11 to eclipse-temurin:17, since
Pekko 2 requires Java 17.

Result:
Remote/cluster samples start and their tests pass on JDK 17.

Tests:
- sbt multi-jvm:test: cluster-java, cluster-scala, cluster-client-grpc-scala,
  distributed-data-java, distributed-data-scala - pass
- sbt test: distributed-workers-scala, persistence-dc-scala, sharding-java,
  sharding-scala, kafka-to-sharding-scala, cluster-kubernetes-scala - pass
- mvn test: cluster-java, cluster-client-grpc-java, persistence-dc-java,
  sharding-java - pass; cluster-kubernetes-java mvn test-compile - pass
- sbt stage for both docker-compose samples: start script contains the flag
- Docker-compose and minikube jobs: Not run

References:
Refs apache#151
@pjfanning
Pin pekko-pki in cluster-kubernetes-java
d3dd1aa 
Motivation:
The cluster-kubernetes-java pods failed at startup with Pekko's
mixed-version check: Maven's nearest-wins resolution picked pekko-pki
2.0.0-M1 from pekko-http/pekko-management while core is 2.0.0-M4.

Modification:
Add an explicit pekko-pki dependency at ${pekko.version}.

Result:
All core Pekko modules resolve to 2.0.0-M4 and the app starts.

Tests:
- Checked resolved core Pekko artifacts for every sample (sbt export
  fullClasspath / mvn dependency:list): only this sample was affected;
  sbt picks the highest version so sbt builds already got pekko-pki M4
- Ran DemoApp locally: no mixed-version error, reaches Cluster Bootstrap
- minikube job: Not run locally

References:
Refs apache#151
@pjfanning
Open java.lang for docs-gen on JDK 17
505b939 
Motivation:
With CI on JDK 17, `sbt paradox` in docs-gen fails: parboiled (used by
Paradox's markdown reader) reflectively calls ClassLoader.findLoadedClass,
which JDK 17 blocks without an add-opens.

Modification:
Add docs-gen/.jvmopts with --add-opens=java.base/java.lang=ALL-UNNAMED.
pekko-sbt-paradox 1.0.1 is already the latest release.

Result:
docs-gen builds on JDK 17.

Tests:
- docs-gen sbt paradox on JDK 17: fails without the file, passes with it

References:
Refs apache#151
@pjfanning
Use latest Paradox plugins in docs-gen instead of add-opens workaround
5009efe 
Motivation:
pekko-sbt-paradox 1.0.1 brings in sbt-paradox 0.9.2, whose parboiled
version does not work on JDK 17. The previous commit worked around this
with an add-opens in docs-gen/.jvmopts.

Modification:
- Declare sbt-paradox and sbt-paradox-theme 0.11.1 explicitly in
  docs-gen/project/plugins.sbt, as the pekko build does.
- Remove docs-gen/.jvmopts.
- Add docs-gen/.sbtopts with -J-Xmx2G: the newer plugins' ClassGraph
  scan across the 15 doc subprojects runs out of the default heap.

Result:
docs-gen builds on JDK 17 without opening java.lang.

Tests:
- docs-gen sbt paradox on JDK 17: pass, 15 sample sites generated
  (OutOfMemoryError without the heap setting)

References:
Refs apache#151
@pjfanning
Upgrade to Pekko 2.0.0 and pekko-http 2.0.0-M2
b49b831 
Motivation:
Pekko 2.0.0 is staged on repository.apache.org for its release vote.

Modification:
Bump Pekko 2.0.0-M4 to 2.0.0 and pekko-http 2.0.0-M1 to 2.0.0-M2 in all
sbt and Maven builds. Add the Apache staging repository as a resolver in
every sbt build and Maven pom. Bump sbt to 1.12.15.

Result:
Samples build against the staged Pekko 2.0.0 artifacts.

Tests:
- Not run locally - relying on CI

References:
Refs apache#151
@pjfanning
Modernize samples for Scala 3.3.9, Java 17 and sbt 1.13.1
bc0fce1 
Motivation:
With Pekko 2 the minimum versions are Scala 2.13 and Java 17, so the
samples can drop compatibility idioms and use newer language features.

Modification:
- Bump Scala 3.3.7 to 3.3.9 and sbt 1.12.15 to 1.13.1.
- Replace scala.collection.JavaConverters with scala.jdk.CollectionConverters.
- Use instanceof pattern matching instead of instanceof plus cast.
- Use Stream.toList() instead of collect(Collectors.toList()).
- Use List.of/Set.of/Set.copyOf/Map.copyOf instead of Arrays.asList and
  Collections wrappers.
- Use a switch expression for WeatherStation queries.
- Remove imports that became unused.

Result:
Samples show idiomatic Scala 2.13+/Java 17 code.

Tests:
- Not run locally - relying on CI

References:
Refs apache#151
@pjfanning
Compile Maven samples with --release 17
9f9c915 
Motivation:
CI failed in cluster-client-grpc-java: the poms still compiled with
-source 1.8, which rejects the Java 17 pattern matching now used.

Modification:
Replace source/target 1.8 with release 17 in every Maven pom
(maven.compiler.release in cluster-kubernetes-java).

Result:
Maven samples compile with Java 17 language features, matching Pekko 2's
minimum.

Tests:
- mvn test-compile: pass for all 8 Maven samples

References:
Refs apache#151
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pjfanning @He-Pin