chore(ci): poll earliest S3 install script for prof-correctness by vlad-scherbich · Pull Request #20585 · DataDog/dd-trace-py

vlad-scherbich · 2026-09-25T19:17:52Z

Description

Merge after DataDog/prof-correctness#221

This PR de-flakes the prof-correctness- #20585 job, which sometimes times out due to its dependency not finishing on time: the install.sh wheel publishing job.

We now expand the pool to any of the 3 published wheels, and we take whichever one finishes first, and pass it to downstream via -f ddtrace_install_url=...:

install-manylinux2014_x86_64.sh
install-manylinux2014.sh
install.sh

Layer	Meaning	Which PR
Accept override	`workflow_dispatch` input + default URL	prof-correctness#221
Poll + pass URL	three candidates, `-f ddtrace_install_url=`	this PR

Testing

green CI, esp. prof-correctness job finishes

Risks

Depends on #221. Until it merges, any run that passes ddtrace_install_url returns error code 422.

Additional Notes

https://datadoghq.atlassian.net/browse/PROF-15511

GitLab install.sh uploads have reached ~78m; the previous 63m poll still flakes trigger-downstream before wheels land.

A 120m poll doubles the budget again after a 14m miss. 90m covers the observed ~78m GitLab upload plus slack without parking a runner for two hours.

datadog-datadog-prod-us1 · 2026-09-25T19:18:38Z

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 1 Pipeline job failed

DataDog/apm-reliability/dd-trace-py | download win_arm64 wheels

View more details · View in GitLab

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: df040aa | Docs | View more details | Give us feedback!}

cit-pr-commenter-54b7da · 2026-09-25T19:19:08Z

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

.github/CODEOWNERS                                                      @DataDog/python-guild @DataDog/apm-core-python
.github/workflows/prof-correctness.yml                                  @DataDog/python-guild @DataDog/apm-core-python

cit-pr-commenter-54b7da · 2026-09-25T19:19:11Z

Circular import analysis

⚠️ Existing circular imports

There are 1 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector

cit-pr-commenter-54b7da · 2026-09-25T19:19:25Z

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 201 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 201 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=132)
ddtrace.appsec._contrib.django -×-> ddtrace.trace  (product:appsec -> product:tracing, score=130)
ddtrace.llmobs._utils -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=130)
ddtrace.llmobs._integrations.base -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=130)
ddtrace.llmobs._integrations.langchain -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=130)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:
uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

Copilot

Copilot review overview

🟢 Approval recommended

No unresolved review comments remain.

Review effort: Lite
Findings: None

What changed in this PR

Updates the prof-correctness workflow to accommodate slower artifact publication.

Changes:

Increases the job timeout from 130 to 160 minutes.
Extends S3 polling from 3,800 to 5,400 seconds.
Adds polling status and clearer timeout diagnostics.

File	Summary
`.github/workflows/prof-correctness.yml`	Updates timeout settings and polling diagnostics.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Unsuffixed install.sh is only after upload all, so a 90m wait still 404s while install-manylinux2014_x86_64.sh is already up. Match dd-trace-doe and pass the first 200 to downstream.

Copilot

Copilot review overview

🔵 Needs a closer look

Add bounded curl timeouts so S3 polling cannot exceed its intended budget.

Review effort: Lite
Findings: None

Route CODEOWNERS reviews for .github/workflows/prof-correctness.yml to @DataDog/profiling-python, matching the other profiling GHA workflows.

Copilot

Copilot review overview

🔵 Needs a closer look

Bound each curl request and stop polling after a successful candidate.

Review effort: Lite
Findings: None

vlad-scherbich · 2026-09-28T19:06:09Z

change ownership of this workflow to profiling team?

@brettlangdon Good call, changed that in a6417ff

Keep hung TCP probes from blowing past POLL_TIMEOUT, and skip remaining install-script candidates once one returns 200.

Copilot

Copilot review overview

🟡 Changes recommended

Polling timeout excludes curl probe time, potentially exceeding the job timeout before dispatch.

Review effort: Lite
Findings: 1

Open (1)

Use wall-clock time for timeout accounting · New

Interval-only elapsed ignored curl probe time and could overrun the job timeout before dispatch.

Copilot

Copilot review overview

🔵 Needs a closer look

It depends on prof-correctness#221 and requires human validation of the downstream workflow integration.

Review effort: Lite
Findings: None

Resolved since last review (1)

Use wall-clock time for timeout accounting

Stand-alone publish-order rationale only — platform indexes land before unsuffixed install.sh.

vlad-scherbich added 2 commits September 25, 2026 14:50

chore(ci): raise S3 wheel poll timeout to 120m for prof-correctness

3faf078

GitLab install.sh uploads have reached ~78m; the previous 63m poll still flakes trigger-downstream before wheels land.

chore(ci): cap S3 wheel poll at 90m for prof-correctness

2757eda

A 120m poll doubles the budget again after a 14m miss. 90m covers the observed ~78m GitLab upload plus slack without parking a runner for two hours.

vlad-scherbich marked this pull request as ready for review September 25, 2026 19:18

vlad-scherbich requested review from a team as code owners September 25, 2026 19:18

vlad-scherbich requested review from Yun-Kim and brettlangdon and removed request for a team September 25, 2026 19:18

vlad-scherbich changed the title ~~chore(profiling): Increase prof-correctness job timeout~~ chore(ci): Increase prof-correctness job timeout Sep 25, 2026

vlad-scherbich requested review from a team and a lite review from Copilot September 25, 2026 20:11

Copilot started reviewing on behalf of vlad-scherbich September 25, 2026 20:12 View session

vlad-scherbich requested review from a team and removed request for Yun-Kim September 25, 2026 20:12

Copilot AI reviewed Sep 25, 2026

View reviewed changes

taegyunkim changed the title ~~chore(ci): Increase prof-correctness job timeout~~ chore(ci): increase prof-correctness job timeout Sep 26, 2026

taegyunkim reviewed Sep 26, 2026

View reviewed changes

Comment thread .github/workflows/prof-correctness.yml Outdated

taegyunkim added the changelog/no-changelog A changelog entry is not required for this PR. label Sep 26, 2026

chore(ci): poll earliest S3 install script for prof-correctness

b3dc993

Unsuffixed install.sh is only after upload all, so a 90m wait still 404s while install-manylinux2014_x86_64.sh is already up. Match dd-trace-doe and pass the first 200 to downstream.

vlad-scherbich requested review from taegyunkim and a lite review from Copilot September 26, 2026 13:52

Copilot started reviewing on behalf of vlad-scherbich September 26, 2026 13:53 View session

Copilot AI reviewed Sep 26, 2026

View reviewed changes

taegyunkim reviewed Sep 27, 2026

View reviewed changes

Comment thread .github/workflows/prof-correctness.yml

vlad-scherbich mentioned this pull request Sep 28, 2026

ci: optional ddtrace_install_url for downstream-python DataDog/prof-correctness#221

Merged

vlad-scherbich changed the title ~~chore(ci): increase prof-correctness job timeout~~ chore(ci): poll earliest S3 install script for prof-correctness Sep 28, 2026

vlad-scherbich requested a review from taegyunkim September 28, 2026 17:43

This comment was marked as resolved.

Sign in to view

gyuheon0h approved these changes Sep 28, 2026

View reviewed changes

vlad-scherbich requested a lite review from Copilot September 28, 2026 19:01

chore(ci): own prof-correctness workflow by profiling-python

a6417ff

Route CODEOWNERS reviews for .github/workflows/prof-correctness.yml to @DataDog/profiling-python, matching the other profiling GHA workflows.

Copilot started reviewing on behalf of vlad-scherbich September 28, 2026 19:02 View session

Copilot AI reviewed Sep 28, 2026

View reviewed changes

chore(ci): bound S3 poll curls and stop on first hit

6a4df4c

Keep hung TCP probes from blowing past POLL_TIMEOUT, and skip remaining install-script candidates once one returns 200.

vlad-scherbich requested a lite review from Copilot September 28, 2026 19:11

Copilot started reviewing on behalf of vlad-scherbich September 28, 2026 19:12 View session

Copilot AI reviewed Sep 28, 2026

View reviewed changes

Comment thread .github/workflows/prof-correctness.yml Outdated

chore(ci): use wall-clock SECONDS for S3 poll timeout

e036563

Interval-only elapsed ignored curl probe time and could overrun the job timeout before dispatch.

vlad-scherbich requested a lite review from Copilot September 28, 2026 19:44

Copilot started reviewing on behalf of vlad-scherbich September 28, 2026 19:45 View session

Copilot AI reviewed Sep 28, 2026

View reviewed changes

taegyunkim approved these changes Sep 28, 2026

View reviewed changes

juanjux approved these changes Sep 29, 2026

View reviewed changes

KowalskiThomas reviewed Sep 30, 2026

View reviewed changes

Comment thread .github/workflows/prof-correctness.yml

Comment thread .github/workflows/prof-correctness.yml Outdated

vlad-scherbich added 2 commits September 30, 2026 11:33

chore(ci): drop private dd-trace-doe refs from S3 poll comments

61a0aa4

Stand-alone publish-order rationale only — platform indexes land before unsuffixed install.sh.

Merge branch 'main' into vlad/prof-correctness-s3-poll-timeout

cc20b3d

KowalskiThomas approved these changes Oct 1, 2026

View reviewed changes

Merge branch 'main' into vlad/prof-correctness-s3-poll-timeout

df040aa

Conversation

vlad-scherbich commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Risks

Additional Notes

Uh oh!

datadog-datadog-prod-us1 Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ Errors

ℹ️ Info

Uh oh!

cit-pr-commenter-54b7da Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codeowners resolved as

Uh oh!

cit-pr-commenter-54b7da Bot commented Sep 25, 2026

Circular import analysis

⚠️ Existing circular imports

Uh oh!

cit-pr-commenter-54b7da Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency direction analysis

⚠️ Existing dependency direction violations

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟢 Approval recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

vlad-scherbich commented Sep 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

vlad-scherbich commented Sep 25, 2026 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Sep 25, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Sep 25, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Sep 25, 2026 •

edited

Loading