Skip to content

mlpstorage: pull kvcache aggregates into top-level results.{csv,json} #660

Description

@FileSystemGuy

Companion to #645 / #646 — same shape, applied to kvcache.

kvcache already computes its own aggregation in mlpstorage_py/benchmarks/kvcache.py:_aggregate_option_results (~L720): fmean for read/write bandwidth and throughput, max for storage_io_latency_ms.p95 across ranks and trials. It writes to a benchmark-owned JSON, but does not populate Result.metrics in the report generator — so the top-level results.{csv,json} restored by PR #620 has empty aggregate columns for kvcache runs.

The Phase 6 work (delivering #645 / #646) introduces a _aggregate_workload_metrics(runs, warmup_set) helper in report_generator.py that dispatches on benchmark_type via a per-metric registry (fmean, max, potentially quantile_p95 etc.). That helper is the natural home for kvcache aggregation too.

Scope: refactor _aggregate_option_results into a kvcache branch of the shared helper (or wire its existing output into Result.metrics) so kvcache aggregate rows flow into top-level results.{csv,json} alongside training/checkpointing.

Gate: PR #602 (Rules.md §6 for kvcache) should codify the aggregation math before we pin the registry's kvcache branch. Existing code precedent (max-of-P95, fmean-of-throughput) can inform §6.

Related: #645, #646, PR #602.

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions