Skip to content

Backport native quota enforcement from ZooKeeper 3.8.7 (ZOOKEEPER-3301, 4219, 4377) - #165

Closed
arkmish wants to merge 1 commit into
branch-3.6from
armishra/quota-enforcement-backport-36
Closed

arkmish wants to merge 1 commit into
branch-3.6from
armishra/quota-enforcement-backport-36

Conversation

@arkmish

@arkmish arkmish commented Oct 19, 2026

Copy link
Copy Markdown

Description

Backports the native znode quota enforcement released in Apache ZooKeeper 3.8.7 onto branch-3.6. Replaces #164 and contains only the following upstream changes plus the fork-specific error-code renumbering:

  • ZOOKEEPER-3301 – hard count/byte limits (setquota -N/-B, delquota -N/-B), extensible StatsTrack wire format (old parsers ignore the new fields), QuotaExceededException, and the admission check in PrepRequestProcessor controlled by the startup-only zookeeper.enforceQuota (default false).
  • ZOOKEEPER-4219 – setData quota check uses the pending change record's data, fixing create-then-setData in multi.
  • ZOOKEEPER-4377 – unknown error codes raise an explicit IllegalArgumentException instead of an NPE.
  • Fork-specific – QUOTAEXCEEDED = -125 (matches upstream). The fork's TOTALEPHEMERALLIMITEXCEEDED moves from -125 to -128, matching the existing LinkedIn 3.8-based fork (Li dev/base 3.8.4 ephemeral change #138). The ephemeral limiter itself is unchanged.

Not included: per-namespace quota metrics / CounterSet/GaugeSet SPI (ZOOKEEPER-4211), Curator compatibility-module changes, and anything beyond released 3.8.7 behavior (no reservation-based strict enforcement).

Released 3.8 limitations are preserved: admission uses committed usage, so multi and pipelined writes can overshoot; deletes are never quota-rejected.

Tests

  • 195 tests passed, 0 failures/errors/skips (JDK 11): StatsTrackTest, QuotasTest, StringUtilTest, KeeperExceptionQuotaTest, DataTreeTest, ZooKeeperQuotaTest, ZooKeeperQuotaEnforcementTest, QuorumQuotaTest, QuorumQuotaEnforcementTest, EnforceQuotaTest, QuotaClientCompatibilityTest (old/new client in separate JVMs), ZooKeeperTest, MultiOperationTest, PrepRequestProcessorTest, ZooKeeperServerTest, ServerMetricsTest, PathUtilsTest, EphemeralNodeThrottlingTest.
  • Full reactor test-compile and uncached checkstyle:check pass.

Changes that Break Backward Compatibility (Optional)

  • Error code -125 changes meaning. Old fork clients will see quota errors as the ephemeral-limit error, and -128 is unknown to them. Migrate clients before rolling servers, even with enforcement disabled.
  • zookeeper.enforceQuota=false (default) now also disables the soft-quota warning log that 3.6 always emitted. Usage accounting continues.
  • API: StatsTrack.getCount() returns long; Quotas.quotaPath() returns the quota directory (use limitPath() for the limit znode); SetQuotaCommand.createQuota/DelQuotaCommand.delQuota take StatsTrack; DataTree.updateCountBytes → updateQuotaStat.

Documentation (Optional)

Updated zookeeperQuotas.md, zookeeperCLI.md, and zookeeperAdmin.md with the new flags, enforcement semantics, limitations, metadata ACL guidance, client migration matrix, and activation/rollback steps.


  • Local code review completed

Created with GitHub Copilot CLI.

Backport the released quota enforcement changes onto branch-3.6:

- ZOOKEEPER-3301: hard count/byte quota limits, setquota/delquota -N/-B,
  extensible StatsTrack format, QuotaExceededException, and the
  startup-only zookeeper.enforceQuota admission check (default false).
- ZOOKEEPER-4219: use the pending change record's data for setData quota
  checks so create-then-setData multi transactions work.
- ZOOKEEPER-4377: report unknown error codes explicitly instead of NPE.

Fork-specific: native QUOTAEXCEEDED uses -125 to match upstream, and the
fork's TOTALEPHEMERALLIMITEXCEEDED moves to -128 to match the LinkedIn
3.8-based fork. Clients must be migrated before servers.

Per-namespace quota metrics (ZOOKEEPER-4211) are intentionally excluded.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@arkmish

arkmish commented Oct 19, 2026

Copy link
Copy Markdown
Author

Superseded by a stack that mirrors upstream: #166 → #167 → #168

@arkmish arkmish closed this Oct 19, 2026
@arkmish
arkmish deleted the armishra/quota-enforcement-backport-36 branch October 19, 2026 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant