使用VAD + CAM++ speaker model + punctuation model 返回结果是空的 #291

Activity

1. panpeng840715-gif

   added

   questionFurther information is requestedFurther information is requested

   on Jun 10, 2026
2. panpeng840715-gif commented on Jun 10, 2026

   

   panpeng840715-gif

   on Jun 10, 2026

   Author

   More actions

   换到另一台服务器上又报错

   
3. LauraGPT commented on Jun 10, 2026

   

   LauraGPT

   on Jun 10, 2026

   Last edited by LauraGPT

   Member

   More actions

   更正（2026-09-10）：下面关于“移除 punc_model 即可解决”和“第二张截图是下载问题”的诊断缺乏证据，已撤回。实际复现、已合并修复与剩余限制请看这条更新。历史原文保留如下。

   SenseVoice 模型本身已内置标点预测，不需要额外加 punc_model="ct-punc"。加了之后可能导致 pipeline 的后处理逻辑冲突，返回空结果。

   正确用法：

   from funasr import AutoModel
   
   # SenseVoice + VAD + 说话人分离（不加 punc_model）
   model = AutoModel(
       model="iic/SenseVoiceSmall",
       vad_model="fsmn-vad",
       vad_kwargs={"max_single_segment_time": 30000},
       spk_model="cam++",
       device="cuda:0"
   )
   
   result = model.generate(input="your_audio.wav", batch_size=1)
   print(result)

   关于第二张截图的报错： 看起来是模型下载/加载问题。请确保：

   1. pip install -U funasr modelscope（升级到最新版）
   2. 网络可访问 modelscope.cn

   试试去掉 punc_model 参数后能否正常返回结果。

   😕React with 😕1Reacted by ackyjiang
4. LauraGPT commented on Jun 29, 2026

   

   LauraGPT

   on Jun 29, 2026

   Last edited by LauraGPT

   Member

   More actions

   Correction (2026-09-13): The earlier claim that removing punc_model resolves this issue is withdrawn. This issue is open. Please use the verified diagnosis, merged fix, and remaining limitations; the original empty-result report is not yet verified as resolved. Historical text follows.

   Closing this as answered.

   SenseVoice already predicts punctuation internally, so the recommended usage is SenseVoice + optional VAD / speaker model, without adding punc_model="ct-punc". Adding an external punctuation model can conflict with the SenseVoice pipeline and lead to empty or unexpected output.

   If the same issue still happens after removing punc_model and upgrading funasr / modelscope, please open a new issue with the full script, package versions, and the exact model download/loading error.
5. LauraGPT

   closed this as completedon Jun 29, 2026
6. LauraGPT

   reopened this on Aug 29, 2026
7. LauraGPT

   added

   needs feedbackWaiting for reporter feedback or retest resultsWaiting for reporter feedback or retest results

   on Aug 29, 2026
8. LauraGPT commented on Aug 29, 2026

   

   LauraGPT

   on Aug 29, 2026

   Last edited by LauraGPT

   Member

   More actions

   Correction (2026-09-13): The earlier claim that removing punc_model resolves this issue is withdrawn. This issue is open. Please use the verified diagnosis, merged fix, and remaining limitations; the original empty-result report is not yet verified as resolved. Historical text follows.

   我之前给出“移除外部 punc_model”的建议后关闭了，但没有等你确认空结果和另一台服务器的加载错误是否真正消失。现在重新打开：请用当前 FunASR/ModelScope 版本、移除 punc_model 后复测，并附完整脚本与仍存在的错误日志；确认解决后再关闭。
9. LauraGPT

   added a commit that references this issue on Sep 10, 2026

   fix: align repository SenseVoice timestamps with FunASR (#354)

   ...

   ea15219
10. LauraGPT commented on Sep 10, 2026

    

    LauraGPT

    on Sep 10, 2026

    Member

    More actions

    更正我之前的诊断：第二张截图不是模型下载失败，也不能据此判断“删掉 punc_model 就解决”。 截图实际在 VAD 时间戳聚合时执行了 int('你')。仓库旧 model.py 返回 [词元, 开始秒, 结束秒]，而该聚合路径需要毫秒数值对。

    现已通过 #354 合并修复：

    • 补齐位置编码模块的 PyTorch 初始化。
    • 修正仓库 CTC 对齐分支，返回 timestamp=[[start_ms,end_ms], ...] 和一一对应的 words；同时处理不等长 batch、一个词元展开为多个 ID 的情况。
    • 更新 README 和 demo2.py。直接消费旧三元组的代码需同步迁移。

    我在 H100、正式 FunASR 1.4.15 和固定模型版本上实测了公开的 51.66 秒双人音频：旧源码在修正初始化后，开、关标点模型都会复现同类整数转换错误；最终源码两种配置均获得覆盖完整文本的 243 组词/时间戳，以及可读取的说话人分句。另有中英文不等长批处理验证，44 项仓库测试和该提交 CI/容器构建均通过。

    请更新 仓库源码至本次提交或之后，再复测原脚本。使用 remote_code="./model.py" 时，只升级 pip 包不会更新本地 model.py。PyTorch/torchaudio 请保持与设备相匹配的一对版本，不必为了本次修复盲目升级整个环境。

    你的 test_16k.wav 和原环境版本未公开，因此我没有验证第一张截图中的空结果，也不宣称原问题已经全部解决。上面的验证也不是识别/说话人准确率评测，现有的标点重复、无 VAD 静音误识别不在这次修复范围。issue 继续保持打开；复测后若仍有空结果或异常，请补充最终 traceback 和本地源码/包版本。