Skip to content

Size the KV cache to the request rather than model.config.max_seq_len - #1327

Open
risjain wants to merge 1 commit into
fishaudio:mainfrom
risjain:perf/size-kv-cache
Open

Size the KV cache to the request rather than model.config.max_seq_len#1327
risjain wants to merge 1 commit into
fishaudio:mainfrom
risjain:perf/size-kv-cache

Size the KV cache to the request rather than model.config.max_seq_len

12c006b
Select commit
Loading
Failed to load commit list.
blacksmith.sh / [code]smith skipped Aug 21, 2026

[code]smith is not active on this PR

[code]smith is not active on this PR. Ask [code]smith to autofix CI on this PR: Enable autofix

View more details on blacksmith.sh