From 93334bacdae6b433bb2e3268b5235e7572f6eef6 Mon Sep 17 00:00:00 2001 From: yunhong <337361684@qq.com> Date: Wed, 22 Jul 2026 10:56:16 +0800 Subject: [PATCH] [hotfix][docs] Recommend 8MB for KV leader replica memory protection --- .../apache/fluss/config/ConfigOptions.java | 4 ++- website/docs/maintenance/configuration.md | 1 + .../operations/resource-protection.md | 30 +++++++++---------- 3 files changed, 19 insertions(+), 16 deletions(-) diff --git a/fluss-common/src/main/java/org/apache/fluss/config/ConfigOptions.java b/fluss-common/src/main/java/org/apache/fluss/config/ConfigOptions.java index 40d7e0bae2..ce9fca5e2b 100644 --- a/fluss-common/src/main/java/org/apache/fluss/config/ConfigOptions.java +++ b/fluss-common/src/main/java/org/apache/fluss/config/ConfigOptions.java @@ -347,7 +347,9 @@ public class ConfigOptions { + "KV leader replica capacity. This value does not reserve or " + "enforce memory on the tablet server. The default value of 0 " + "disables memory-based KV leader replica capacity control. " - + "Set a positive value to enable it."); + + "Set a positive value to enable it. When enabled, the recommended " + + "value is 8 MB, based on the measured memory consumption of an " + + "empty KV leader replica."); /** * The network address and port the server binds to for accepting connections. diff --git a/website/docs/maintenance/configuration.md b/website/docs/maintenance/configuration.md index 3ba20bad48..d959b4eed6 100644 --- a/website/docs/maintenance/configuration.md +++ b/website/docs/maintenance/configuration.md @@ -47,6 +47,7 @@ during the Fluss cluster working. | allow.create.kv.tables | Boolean | true | Whether to allow creation of kv tables (primary key tables). When set to false, attempts to create kv tables (tables with primary key) will be rejected. The default value is true. | | max.partition.num | Integer | 1000 | Limits the maximum number of partitions that can be created for a partitioned table to avoid creating too many partitions. | | max.bucket.num | Integer | 4096 | The maximum number of buckets that can be created for a non-partitioned table or for each partition of a partitioned table. The default value is 4096. This default is capped to reduce the risk that an assignment znode exceeds ZooKeeper's packet size limit. | +| kv.leader-replica.memory-reserved | MemorySize | 0 bytes | The estimated memory consumption of each KV leader replica, used by the CoordinatorServer to calculate the cluster-level KV leader replica capacity. This value does not reserve or enforce memory on the TabletServer. The default value of `0` disables memory-based KV leader replica capacity control. Set a positive value to enable it. When enabled, the recommended value is `8 MB`, based on the measured memory consumption of an empty KV leader replica. | | acl.notification.expiration-time | Duration | 15min | The duration for which ACL notifications are valid before they expire. This configuration determines the time window during which an ACL notification is considered active. After this duration, the notification will no longer be valid and will be discarded. The default value is 15 minutes. This setting is important to ensure that ACL changes are propagated in a timely manner and do not remain active longer than necessary. | | authorizer.enabled | Boolean | false | Specifies whether to enable the authorization feature. If enabled, access control is enforced based on the authorization rules defined in the configuration. If disabled, all operations and resources are accessible to all users. | | authorizer.type | String | default | Specifies the type of authorizer to be used for access control. This value corresponds to the identifier of the authorization plugin. The default value is `default`, which indicates the built-in authorizer implementation. Custom authorizers can be implemented by providing a matching plugin identifier. | diff --git a/website/docs/maintenance/operations/resource-protection.md b/website/docs/maintenance/operations/resource-protection.md index 353ae09f61..c2ad083ccd 100644 --- a/website/docs/maintenance/operations/resource-protection.md +++ b/website/docs/maintenance/operations/resource-protection.md @@ -83,8 +83,9 @@ KV leader replica capacity protection is a coordinator-side admission guard that leader replica capacity from TabletServer memory and rejects new KV table or partition creation once that capacity would be exceeded. -The feature is **disabled by default** and is considered an advanced/opt-in feature. Enable it once you have a -sense of how much memory each KV leader replica costs in your environment. +The feature is **disabled by default** and is considered an advanced/opt-in feature. When enabling it, the +recommended value is **8 MB per KV leader replica**, based on the measured memory consumption of an empty KV +leader replica. Adjust the value if measurements from your workload show a different per-replica footprint. ### When to Enable It @@ -108,13 +109,15 @@ The feature is controlled entirely by a single option, `kv.leader-replica.memory # Disabled (default) kv.leader-replica.memory-reserved: 0 -# Enabled: reserve an estimated 512MB per KV leader replica -kv.leader-replica.memory-reserved: 512mb +# Enabled with the recommended estimate of 8MB per KV leader replica +kv.leader-replica.memory-reserved: 8mb ``` - `0` (the default) disables memory-based KV leader replica capacity control entirely. - A positive value enables it, and is used as the estimated memory cost of a single KV leader replica when calculating the cluster-level capacity (see [How Capacity Is Calculated](#how-capacity-is-calculated) below). +- `8mb` is the recommended value when enabling the feature, based on the measured memory consumption of an empty + KV leader replica. Tune it if your workload measurements indicate a different per-replica memory footprint. - Negative values are invalid and are rejected at both static startup and dynamic reconfiguration time. :::tip @@ -158,12 +161,12 @@ follows: unknown servers. 5. `kvLeaderReplicaCapacity = totalEstimatedMemory / kv.leader-replica.memory-reserved`. -For example, with `kv.leader-replica.memory-reserved: 1gb` and 4 live TabletServers where 3 report 32GB each and +For example, with `kv.leader-replica.memory-reserved: 8mb` and 4 live TabletServers where 3 report 32GB each and one reports no memory: - 3 out of 4 servers report memory, so the limit is enabled (more than half). - The unknown server is assumed to have the average of the known servers: 32GB. - Total estimated memory = 32GB × 4 = 128GB. -- Capacity = 128GB / 1GB = 128 KV leader replicas. +- Capacity = 128GB / 8MB = 16,384 KV leader replicas. The capacity is recalculated on demand from the current live TabletServer set and is not persisted; it is recomputed on CoordinatorServer leadership startup and whenever a table or partition creation request is checked. @@ -181,7 +184,7 @@ Not enough KV leader replica capacity. observedKvLeaderReplicaCount=, req If you hit this error, you can: - Increase capacity by adding TabletServers or more memory per TabletServer. -- Raise `kv.leader-replica.memory-reserved` only if your existing estimate was too conservative — raising it +- Raise `kv.leader-replica.memory-reserved` only if your existing estimate was too low — raising it without cause lowers the effective capacity. - Temporarily disable the limit (set `kv.leader-replica.memory-reserved` to `0`) if you need to proceed immediately, then re-tune before re-enabling. @@ -198,20 +201,17 @@ The CoordinatorServer exposes two gauges for monitoring: Monitor `kvLeaderReplicaCount` against `kvLeaderReplicaCapacity` to anticipate when a cluster is approaching its limit before requests start failing. -### Example: Enabling, Tuning, and Disabling +### Example: Enabling and Disabling ```yaml title="conf/server.yaml" -# Step 1: Enable with an initial conservative estimate per KV leader replica -kv.leader-replica.memory-reserved: 1gb +# Step 1: Enable with the recommended estimate per KV leader replica +kv.leader-replica.memory-reserved: 8mb # Step 2 (optional): Override auto-detected resource reporting on a TabletServer # with constrained cgroup visibility tablet-server.advertised-resource.memory-size: 64gb tablet-server.advertised-resource.cpu-cores: 16 -# Step 3: Tune down once real per-replica memory usage is measured -kv.leader-replica.memory-reserved: 256mb - -# Step 4: Disable if no longer needed +# Step 3: Disable if no longer needed kv.leader-replica.memory-reserved: 0 -``` \ No newline at end of file +```