Skip to content

Report poller target on each poll and in the worker heartbeat - #3159

Open
veeral-patel wants to merge 2 commits into
mainfrom
vpatel/poller-load-balancing
Open

veeral-patel wants to merge 2 commits into
mainfrom
vpatel/poller-load-balancing

Conversation

@veeral-patel

@veeral-patel veeral-patel commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What

An autoscaling poller pool now reports its current target in two places:

  1. On each poll — PollerScalingInfo.poller_target on the workflow and activity poll requests.
  2. On the worker heartbeat — WorkerPollerInfo.poller_target, for all four poller types.

Why

Scaling suggestions ride on dispatched tasks and the matcher is FIFO, so a worker's share of suggestions tracks its share of pollers. Growth is proportional to current size, which preserves imbalance between identical workers instead of correcting it — measured at 74/12/14 percent across three identical workers, with a different winner each run.

Reporting the target gives the server the one number it needs to compare a worker against the fleet mean and withhold suggestions that would push it further from its share.

The poll-request copy is consumed inside the matching decision and discarded. Reporting it on the heartbeat too lets a follow-up read it per worker and export it as a metric, making fleet-wide poller distribution visible without instrumenting every SDK.

How

Poll path — AsyncPoller installs a target supplier on each poll task when it creates that task's PollScaleReportHandle. The poll request is rebuilt per poll, because the target moves. A default no-op on PollTaskAsync keeps pollers with nothing to report from having to implement it.

Heartbeat — PollScaleReportHandle is a local inside AsyncPoller.start() and is not reachable from Worker.buildHeartbeatCallback. PollerTracker already exists to bridge exactly that gap — its javadoc says it "bridges the gap between poll tasks (created inside start()) and heartbeat building" — so the supplier goes there, and buildPollerInfo reads it alongside the in-flight count it already reads.

Each tracker is already the right one per type: WorkflowWorker hands stickyPollerTracker to the sticky poll task and pollerTracker to the normal one, so the normal/sticky split works even though both share one PollerOptions.

Design notes

  • AsyncNexusPollTask now overrides setPollerTargetSupplier purely to feed its tracker. It had taken the interface's no-op default because the nexus poll request has no field for a target — but it does have an autoscaler and a heartbeat slot, so the heartbeat reports it.
  • Non-autoscaling workers use MultiThreadedPoller, which has no scale report handle, so no supplier is installed and the tracker reports 0 — which the proto defines as unknown.

Dependency, and the submodule bump

Needs temporalio/api#887.

The temporal-serviceclient/src/main/proto submodule moves to vpatel/poller-load-balancing-java-base on the api repo — the same two proto commits cherry-picked onto the api commit this repo already recorded, so the bump is minimal and WorkerPollerInfo is byte-identical to the PR branch.

It is pinned there rather than to the api PR branch deliberately: that branch is based on current api main, and taking it would also pull in Callback.NexusHandler.source_context, which generatePayloadLimitValidator rejects as an unclassified payload-bearing field — a failure unrelated to this change. The submodule should move to a released api version before merge.

Testing

./gradlew :temporal-sdk:compileJava and :temporal-sdk:compileTestJava pass; spotless applied. Not yet exercised end to end against a server — that has been done via the Go SDK (temporalio/sdk-go#2799), which shares the wire contract.

An autoscaling poller pool now sends its current target with every workflow and
activity poll, so the server can spread scaling suggestions across a fleet by
share rather than by whoever happens to receive the most tasks.

Suggestions ride on dispatched tasks and the matcher is FIFO, so a worker's
share of suggestions tracks its share of pollers -- growth is proportional to
current size, which preserves imbalance between identical workers instead of
correcting it. A reported target gives the server the one number it needs to
compare a worker against the fleet mean.

AsyncPoller owns the scale report handle, so it installs the target supplier on
each poll task when it creates that handle. The poll request is rebuilt per poll
because the target moves; pools with no supplier installed -- Nexus, and any
non-autoscaling pool -- send nothing, which the server reads as "no report".

The api submodule moves to a branch carrying the new proto field. It is pinned
one commit ahead of the previously recorded api commit rather than to current
api main, which would also drag in Callback.NexusHandler.source_context and
fail generatePayloadLimitValidator for reasons unrelated to this change.
@Quinn-With-Two-Ns Quinn-With-Two-Ns self-assigned this Oct 20, 2026
@veeral-patel
veeral-patel marked this pull request as ready for review October 20, 2026 20:11
@veeral-patel
veeral-patel requested a review from a team as a code owner October 20, 2026 20:11
@veeral-patel
veeral-patel force-pushed the vpatel/poller-load-balancing branch from 00456fd to ada916a Compare October 22, 2026 03:28
@veeral-patel veeral-patel changed the title Report poller target on each poll Report poller target on each poll and in the worker heartbeat Oct 22, 2026
The poller target is already sent on each poll, but matching consumes it inside
the scaling decision and discards it -- nothing persists it and nothing can
observe it. Reporting it on the heartbeat too lets the server read it per worker
and export it as a metric, making fleet-wide poller distribution visible without
instrumenting every SDK.

WorkerHeartbeat already carries four WorkerPollerInfo sub-messages, so this is
per-task-type for free, including the normal-vs-sticky workflow split.

PollScaleReportHandle is a local inside AsyncPoller.start() and is not reachable
from the heartbeat builder. PollerTracker already exists to bridge exactly that
gap -- its javadoc says so -- so the target supplier goes there, and
buildPollerInfo reads it alongside the in-flight count it already reads.

AsyncNexusPollTask now overrides setPollerTargetSupplier purely to feed its
tracker. It had taken the interface's no-op default because the nexus poll
request has no field for a target, but it does have an autoscaler and a
heartbeat slot.

Fixed-size pools run MultiThreadedPoller, which has no scale report handle, so
no supplier is installed and the tracker reports 0 -- which the proto defines as
unknown. The api submodule moves to the commit carrying the new field.
@veeral-patel
veeral-patel force-pushed the vpatel/poller-load-balancing branch from ada916a to dcb6b69 Compare October 22, 2026 03:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants