[None][chore] Stop forcing TRTLLM_DISABLE_KV_CACHE_TRANSFER_OVERLAP=1 in disagg gen_only benchmark

[dc3671]

Description

examples/disaggregated/slurm/benchmark/submit.py set TRTLLM_DISABLE_KV_CACHE_TRANSFER_OVERLAP=1 on every worker whenever benchmark_mode == "gen_only". This PR removes that.

The flag arrived with the script itself in #10712 and no rationale was recorded for it.

Why remove it:

• Setting it makes the generation worker take the blocking request_and_receive_sync KV path instead of leaving transfers in flight (PyExecutor._uses_async_disagg_gen_transfer, tensorrt_llm/_torch/pyexecutor/py_executor.py). So gen_only was measuring a KV transfer path that is not the production default.
• It is not needed to obtain gen-only decode measurements — TLLM_BENCHMARK_REQ_QUEUES_SIZE is what drives the gen-only fill loop, and that is untouched here.
• The serialized path has at least one demonstrated deadlock interaction: on a DSv4 fork with a q_b GEMM prologue fusion, a dep32 gen_only run hangs with the flag set (CTX worker spins with num_scheduled_requests=0 while KV blocks are never released) and completes normally with it removed. That is an interaction with the fusion, not proof the flag is broken on its own — the same config without the fusion completed fine with the flag set. Listing it as motivation, not as the primary justification.

The nested if role == "GEN" is collapsed into the outer condition since it became the only body.

Scope

Only the benchmark script usage is removed. The environment variable and all of its consumers stay:

• cpp/tensorrt_llm/common/envUtils.cpp
• cpp/tensorrt_llm/batch_manager/cacheTransceiver.cpp
• tensorrt_llm/_torch/pyexecutor/kv_cache_transceiver.py
• tensorrt_llm/_torch/pyexecutor/py_executor.py
• tests/integration/defs/accuracy/test_disaggregated_serving.py — test_gen_only_sync sets it deliberately to exercise the blocking path

jenkins/scripts/perf/submit.py and jenkins/scripts/perf/local/submit.py set the same flag for gen_only. They are intentionally left alone here — changing them would shift recorded CI perf numbers, which is a separate decision.

Test Coverage

No test changes. This is a SLURM benchmark launcher script that is not exercised by CI.

Known gap: removal has only been validated on a fork carrying an unrelated fusion. A stock main gen_only disagg run with and without the flag has not been done. Reviewers with a disagg SLURM setup, please confirm — happy to hold the PR if gen_only regresses without it.

PR Checklist

• PR title follows [JIRA/NVBUG/None][type] summary
• Commit is signed off (DCO)
• No API changes
• Tests added — n/a, benchmark launcher script

Dev Engineer Review

• Removed TRTLLM_DISABLE_KV_CACHE_TRANSFER_OVERLAP only from the disaggregated SLURM gen_only launcher.
• Preserved generation-worker concurrency validation and TLLM_BENCHMARK_REQ_QUEUES_SIZE.
• Preserved the environment variable, its consumers, deliberate test usage, and Jenkins performance settings.
• No API, configuration, or test-list changes were made.
• No tests were added. A stock main SLURM gen_only run with and without the flag remains unvalidated.
• Review verdict: needs follow-up for runtime validation of the launcher.

QA Engineer Review

No test changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: af533a16-fa2d-45e3-b710-c9a379cfeccd

📥 Commits

Reviewing files that changed from the base of the PR and between ba1a48b and cb797e9.

📒 Files selected for processing (1)

• examples/disaggregated/slurm/benchmark/submit.py

---

Walkthrough

The benchmark submission script now limits request-queue configuration to generation workers in gen_only mode. It no longer sets TRTLLM_DISABLE_KV_CACHE_TRANSFER_OVERLAP in the shared worker environment.

Changes

Benchmark worker environment

Layer / File(s)	Summary
Generation worker configuration examples/disaggregated/slurm/benchmark/submit.py	In gen_only mode, generation workers configure the validated request-queue size. The shared TRTLLM_DISABLE_KV_CACHE_TRANSFER_OVERLAP assignment was removed.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: bowenfu

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the benchmark script change and follows the required [None][chore] format.
Description check	✅ Passed	The description explains the motivation, scope, retained consumers, testing status, known gap, and checklist items.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[chuangz0]

Using DISABLE_OVERLAP avoids repeatedly checking the transfer states, which does introduce some overhead. As for the context worker hang, I suspect there may be an existing bug in PyExecutor. I recall that @reasonsolo encountered and fixed a similar issue before.
For now, I suggest not setting this environment variable on the context worker, while continuing to set it on the generation worker.