Skip to content

exporters/prometheus: record ExportMetrics per scrape batch instead of per metric family - #9465

Open
dashpole wants to merge 2 commits into
open-telemetry:mainfrom
dashpole:fix-prom-export-metrics-batch
Open

dashpole wants to merge 2 commits into
open-telemetry:mainfrom
dashpole:fix-prom-export-metrics-batch

Conversation

@dashpole

@dashpole dashpole commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Fixes #9452

Summary

In exporters/prometheus, collector.Collect previously called c.inst.ExportMetrics inside each per-metric-family helper (addHistogramMetric, addExponentialHistogramMetric, addSumMetric, and addGaugeMetric) rather than once for the collected ResourceMetrics batch:

  1. Any metric data points dropped before add*Metric was reached—when createResourceAttributes failed, scopeMetrics.Scope.Name == bridgeScopeName was skipped, appendScopeAttrs failed, c.metricType(m) == nil, c.getName(m) failed, or c.validateMetrics dropped a conflicting metric family—were never counted in otel.sdk.exporter.metric_data_point.inflight or recorded as failed in otel.sdk.exporter.metric_data_point.exported.
  2. During a scrape, otel.sdk.exporter.metric_data_point.inflight only reflected the data points of a single metric family at a time rather than the collected batch in flight, and invoked the synchronous counter instruments once per metric family instead of once per scrape.

This change moves c.inst.ExportMetrics(ctx, countDataPoints(metrics)) and op.End(success, exportErr) to collector.Collect (matching exporters/stdout/stdoutmetric), returns (int64, error) from the add*Metric helpers, and joins all metric/scope/resource validation errors into exportErr so dropped data points are properly accounted for.

@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Pull request dashboard status

Waiting on reviewers · refreshed 2026-11-09 20:58 UTC

Review the latest changes.

Status above doesn't look right?
  • Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
  • Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

@dashpole
dashpole force-pushed the fix-prom-export-metrics-batch branch from 7870d65 to b04dd59 Compare November 8, 2026 00:43
@dashpole
dashpole requested a balanced review from Copilot November 8, 2026 00:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@codecov

codecov Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.8%. Comparing base (945ed13) to head (08ff0a3).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #9465     +/-   ##
=======================================
- Coverage   89.9%   89.8%   -0.1%     
=======================================
  Files        357     357             
  Lines      23987   24012     +25     
=======================================
+ Hits       21565   21573      +8     
- Misses      2422    2439     +17     
Files with missing lines Coverage Δ
exporters/prometheus/exporter.go 94.8% <100.0%> (+0.3%) ⬆️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dashpole
dashpole force-pushed the fix-prom-export-metrics-batch branch from b04dd59 to 95818fc Compare November 8, 2026 00:49
@dashpole
dashpole marked this pull request as ready for review November 8, 2026 01:15
@opentelemetry-pr-dashboard
opentelemetry-pr-dashboard Bot requested a balanced review from Copilot November 8, 2026 01:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Successful metric families unnecessarily wrap prior export errors via errors.Join(exportErr, nil), causing avoidable allocations and deeply nested error trees on large scrapes.

1 open issue

Review effort: Balanced

Comment thread exporters/prometheus/exporter.go Resolved
@dashpole
dashpole force-pushed the fix-prom-export-metrics-batch branch from 95818fc to 08ff0a3 Compare November 9, 2026 20:49
@opentelemetry-pr-dashboard
opentelemetry-pr-dashboard Bot requested a balanced review from Copilot November 9, 2026 20:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

exporters/prometheus: ExportMetrics is recorded per metric family instead of per scrape batch, missing dropped metrics

2 participants