[sclfcz]

What

--suppress_tokens defaults to -1, and decoding.py turns that into tokenizer.non_speech_tokens. That set included 「」『』:

symbols = list('"#()*+/:;<=>@[\\]^_`{|}~「」『』')

Those are ordinary Chinese and Japanese quotation marks, not speaker tags or non-speech annotations, so every CLI run (and every API call leaving suppress_tokens at its default) silently dropped them from the transcript. A sentence like

他说：「这个方案可以。」

came back as 他说：这个方案可以。 — the quotes are gone, which makes Chinese output look like it lost text.

Change

Remove the four CJK brackets from the suppression list. Everything that actually marks non-speech (♪, #, (, ), *, the speaker-tag shapes, …) is untouched.

How I checked

Appended to tests/test_tokenizer.py:

• 「 」 『 』 must not be in non_speech_tokens for a zh tokenizer;
• control: ♪, #, (, ), * must still be suppressed, so the set keeps doing its job.

Both use the real multilingual tokenizer from whisper/assets, so no model or audio is needed. I verified the before/after behaviour with a standalone harness that loads whisper/tokenizer.py directly (importing the whisper package pulls in torch, which I do not have installed locally); the tests themselves run in this repo's CI.

Related

#2878 (the other CJK issue I hit: --max_line_width counting code points instead of display columns) touches a different file.

[sclfcz]

补充一句冲突提示：#2828 也在改 suppress_tokens 相关代码（它处理的是 iterables / None / 共享列表那类问题），和这里的符号表不是同一处，但两块相邻，先合并哪个都可能需要另一个 rebase。