[TRTLLM-13694][infra] Add benchmark refresh mappings

[Mgluhovskoi]

Description

Add a repository-owned mapping file that defines which model, GPU, and precision groups may be refreshed from the latest validated benchmark data.

The file groups repeated accelerator translations and target path rules into shared defaults, so adding a model normally requires only its source model and precision, public target model and architecture, and GPU list. It also documents how generated recipes are stored, how to add or remove models and GPUs, and how to override the defaults when necessary.

Artifact logs are discovered automatically from each record's runtime dimensions. An optional artifact_log.match_tokens override is available only when multiple complete logs remain ambiguous.

Test Coverage

• Parsed and expanded three model definitions into five source-to-target mappings with the automation schema loader.
• Verified all five mapped model/GPU targets exist in examples/configs/database/lookup.yaml.
• Ran the companion automation unit tests: 22 passed.

PR Checklist

• Please check this after reviewing the above items as appropriate for this PR.

Dev Engineer Review

• Added version 2 benchmark refresh mappings for DeepSeek-R1 and GPT-OSS-120B.
• Added B200 and H200 accelerator translations.
• Added default recipe paths and filenames.
• Documented model and GPU registration, default overrides, artifact-log matching, and removal semantics.
• Validation expanded three model definitions into five mappings and confirmed target entries in lookup.yaml.
• No public API or exported entity changes.
• 22 automation unit tests passed.

QA Engineer Review

No test changes.

[Mgluhovskoi]

/bot run

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 953a28d8-4a93-47df-a1af-84450aae6340

📥 Commits

Reviewing files that changed from the base of the PR and between 4e95cb7 and 7666c5c.

📒 Files selected for processing (1)

• examples/configs/benchmark_refresh_mappings.yaml

🚧 Files skipped from review as they are similar to previous changes (1)

• examples/configs/benchmark_refresh_mappings.yaml

---

Walkthrough

Adds a version 2 benchmark refresh mapping configuration. The configuration defines output defaults, B200/H200 mappings, and refresh targets for DeepSeek-R1 FP4/FP8 and GPT-OSS-120B FP4.

Changes

Benchmark refresh configuration

Layer / File(s)	Summary
Refresh mapping schema and targets examples/configs/benchmark_refresh_mappings.yaml	Documents model and GPU registration, artifact-log disambiguation, and removal semantics. Defines recipe output defaults, B200/H200 accelerator mappings, and refresh targets for DeepSeek-R1 and GPT-OSS-120B variants.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 7666c

This PR adds benchmark refresh mappings without any identified merge-blocking correctness, security, availability, or deployment risk; it is merge-ready after normal checks and review.

Suggested reviewers: chang-l, laikhtewari

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title follows the required ticket and type format and clearly identifies the benchmark refresh mapping change.
Description check	✅ Passed	The description explains the change, documents test coverage, and includes a completed PR checklist.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[tensorrt-cicd]

PR_Github #64709 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #64709 [ run ] completed with state SUCCESS. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #52564 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #64726 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #64726 [ run ] completed with state FAILURE. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #52578 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #65113 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #65113 [ run ] completed with state SUCCESS. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #52911 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #65155 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #65155 [ run ] completed with state FAILURE. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #52948 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #65264 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #65266 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #65264 [ run ] completed with state ABORTED. Commit: d61f6a6

Link to invocation

[tensorrt-cicd]

PR_Github #65266 [ run ] completed with state FAILURE. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #53043 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #65376 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #65376 [ run ] completed with state SUCCESS. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #53140 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #65402 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #65913 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #65913 [ run ] completed with state SUCCESS. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #53604 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #66050 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #66050 [ run ] completed with state SUCCESS. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #53733 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #66061 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #66061 [ run ] completed with state SUCCESS. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #53744 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #66155 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #66155 [ run ] completed with state FAILURE. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #53830 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #66180 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #66180 [ run ] completed with state FAILURE. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #53852 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #66280 [ run ] triggered by Bot. Commit: d61f6a6 Link to invocation

[tensorrt-cicd]

PR_Github #66280 [ run ] completed with state SUCCESS. Commit: d61f6a6
/LLM/main/L0_MergeRequest_PR pipeline #53940 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #66380 [ run ] triggered by Bot. Commit: 7666c5c Link to invocation

[tensorrt-cicd]

PR_Github #66380 [ run ] completed with state FAILURE. Commit: 7666c5c
/LLM/main/L0_MergeRequest_PR pipeline #54023 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #66406 [ run ] triggered by Bot. Commit: 7666c5c Link to invocation

[tensorrt-cicd]

PR_Github #66406 [ run ] completed with state SUCCESS. Commit: 7666c5c
/LLM/main/L0_MergeRequest_PR pipeline #54049 completed with status: 'SUCCESS'

CI Report

Link to invocation