Add nightly performance regression detection

[Amaury Levé (Evangelink)]

Summary

• add a seven-run rolling Linux baseline for PlainProcess wall-clock and CPU timings
• emit non-blocking warnings when a per-pipeline median regresses by more than 10%
• publish the comparison in the workflow summary and retain the updated baseline as an artifact
• make timing reports self-identifying and restore Scenario4 collection by fixing its generated warnings-as-errors failure
• recover automatically from missing, corrupt, or incompatible baseline artifacts

Validation

• packed the repository in Release configuration
• ran Scenario4_ClassInit_PlainProcess end to end and verified its named three-measurement report
• exercised baseline bootstrap, regression, corrupt JSON, and malformed schema recovery paths
• validated workflow YAML, embedded GitHub Script syntax, and action pins

Closes #10549

[Copilot]

Pull request overview

Adds nightly Linux performance regression detection using a seven-run rolling baseline and non-blocking warnings.

Changes:

• Adds baseline comparison, workflow summaries, and artifact persistence.
• Makes timing reports pipeline-aware.
• Restores Scenario4 collection.

Show a summary per file

File	Description
.github/workflows/perf-timing-nightly.yml	Integrates baseline retrieval, comparison, and upload.
.github/scripts/compare_perf_timings.py	Calculates medians, regressions, and updated baselines.
PlainProcess.cs	Emits named timing reports.
DotnetTestProcess.cs	Emits named timing reports.
Scenario4.cs	Addresses generated-code warning.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Balanced

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

✅ 22/22 dimensions clean — no findings.

Dimensions assessed:

• 1 Algorithmic Correctness — LGTM (Python parsing/validation logic is solid; WriteAllText correctly replaces AppendAllText for the new report-object format; GC.KeepAlive prevents JIT elision)
• 2 Threading & Concurrency — N/A (no shared mutable state in changed code)
• 3 Security & IPC — N/A (script reads/writes local artifacts; workflow downloads from own repo)
• 4 Public API — N/A (no public API changes)
• 5 Performance — N/A (this is perf tooling, not hot-path library code)
• 6 Cross-TFM — N/A (Python + YAML + perf runner)
• 7 Resource/IDisposable — N/A
• 8 Defensive Coding — N/A (no trust boundaries crossed)
• 9 Localization — N/A
• 10 Test Isolation — N/A
• 11 Assertion Quality — N/A
• 12 Flakiness — N/A
• 13 Test Completeness — LGTM (CI utility script; validated by integration through the workflow itself)
• 14 Data-Driven — N/A
• 15 Code Structure — LGTM (clean, well-organized)
• 16 Naming — LGTM
• 17 Documentation — LGTM (workflow header comment updated)
• 18 Analyzer Quality — N/A
• 19 IPC Wire — N/A
• 20 Build Infrastructure — LGTM (actions: read permission correctly added for artifact listing)
• 21 Scope & PR Discipline — LGTM (focused perf-baseline infrastructure)
• 22 PowerShell — N/A (Python script, not PowerShell)

[github-actions]

🧪 Expert test review — PR #10720

No new or modified test methods were identified in the changed regions of this PR. The changes touch performance-testing infrastructure only (.github/scripts/compare_perf_timings.py, .github/workflows/perf-timing-nightly.yml, and the MSTest.Performance.Runner scenario/process helpers in test/Performance/), none of which contain [TestMethod]-style test methods to grade.

_{Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 36 AIC · ⌖ 0.917 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🧵 Parallel-safety audit — PR #10720

Nothing audited here touches process-global state, shared filesystem paths, or
[ResourceLock] / [DoNotParallelize] declarations. Nothing to flag for
parallel-safety.

This PR only changes performance-harness/CI scripting (.github/scripts/compare_perf_timings.py, .github/workflows/perf-timing-nightly.yml) and support code in MSTest.Performance.Runner (DotnetTestProcess.cs, PlainProcess.cs, Scenario4.cs). The [assembly: Parallelize] attribute visible in Scenario4.cs lives inside a raw string template (CurrentMSTestSourceCode) used to generate a synthetic scenario project — it configures that generated project, not the MSTest.Performance.Runner assembly itself, so it is excluded per the audit rules. The only real code edit in that file is a _ = _classState; discard added to silence an unused-field warning in the generated ClassCleanup body — no lifecycle mutation, path, or lock declaration changed.

No changed [TestMethod], lifecycle member ([TestInitialize]/[ClassInitialize]/[AssemblyInitialize]/constructor/Dispose), or [ResourceLock]/[DoNotParallelize]/[Parallelize] declaration was found in this diff, and no assembly parallelization configuration changed.

_{Re-run with /parallel-audit.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 50.7 AIC · ⌖ 0.895 AIC · ⊞ 24.8K · [◷]( · ◷)

[Copilot]

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 0 new
• Review effort level: Balanced