Companion to #645 / #646 — same shape, applied to kvcache.
kvcache already computes its own aggregation in mlpstorage_py/benchmarks/kvcache.py:_aggregate_option_results (~L720): fmean for read/write bandwidth and throughput, max for storage_io_latency_ms.p95 across ranks and trials. It writes to a benchmark-owned JSON, but does not populate Result.metrics in the report generator — so the top-level results.{csv,json} restored by PR #620 has empty aggregate columns for kvcache runs.
The Phase 6 work (delivering #645 / #646) introduces a _aggregate_workload_metrics(runs, warmup_set) helper in report_generator.py that dispatches on benchmark_type via a per-metric registry (fmean, max, potentially quantile_p95 etc.). That helper is the natural home for kvcache aggregation too.
Scope: refactor _aggregate_option_results into a kvcache branch of the shared helper (or wire its existing output into Result.metrics) so kvcache aggregate rows flow into top-level results.{csv,json} alongside training/checkpointing.
Gate: PR #602 (Rules.md §6 for kvcache) should codify the aggregation math before we pin the registry's kvcache branch. Existing code precedent (max-of-P95, fmean-of-throughput) can inform §6.
Related: #645, #646, PR #602.
Companion to #645 / #646 — same shape, applied to kvcache.
kvcache already computes its own aggregation in
mlpstorage_py/benchmarks/kvcache.py:_aggregate_option_results(~L720):fmeanfor read/write bandwidth and throughput,maxforstorage_io_latency_ms.p95across ranks and trials. It writes to a benchmark-owned JSON, but does not populateResult.metricsin the report generator — so the top-levelresults.{csv,json}restored by PR #620 has empty aggregate columns for kvcache runs.The Phase 6 work (delivering #645 / #646) introduces a
_aggregate_workload_metrics(runs, warmup_set)helper inreport_generator.pythat dispatches onbenchmark_typevia a per-metric registry (fmean,max, potentiallyquantile_p95etc.). That helper is the natural home for kvcache aggregation too.Scope: refactor
_aggregate_option_resultsinto a kvcache branch of the shared helper (or wire its existing output intoResult.metrics) so kvcache aggregate rows flow into top-levelresults.{csv,json}alongside training/checkpointing.Gate: PR #602 (Rules.md §6 for kvcache) should codify the aggregation math before we pin the registry's kvcache branch. Existing code precedent (max-of-P95, fmean-of-throughput) can inform §6.
Related: #645, #646, PR #602.