Skip to content

fix(openai_agents): preserve interrupted workflow spans - #20846

Open
yshapiro-57 wants to merge 1 commit into
mainfrom
yakov.shapiro/fix-openai-agents-workflow-kind
Open

yshapiro-57 wants to merge 1 commit into
mainfrom
yakov.shapiro/fix-openai-agents-workflow-kind

Conversation

@yshapiro-57

@yshapiro-57 yshapiro-57 commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Human Summary

While working on https://github.com/ddoghq/dd-source/pull/120671 I found that the automated evals that are supposed to verify that the new skill is actually working are sporadically producing this error message:

Error preparing LLMObs span event for span <Span(id=3419520971592566224,trace_id=141922899536558176561588771022095746485,parent_id=11792225431829319188,name=Agent workflow)>, missing span kind in span context.

I asked Codex and it came up with a theory that this is because our OpenAI Agents integration is setting kind="workflow" on the agent workflow spans only in on_trace_end and not in on_trace_start. If the code throws an exception before on_trace_end is called, then a span without a kind will be emitted, triggering this error. Codex's suggested fix was to redundantly set kind="workflow" on these spans in on_trace_start.

I am not familiar with the code myself and need to rely on

AI Summary

  • Preserve OpenAI Agents workflow spans when a trace is interrupted before normal end-of-trace enrichment.
  • Record the workflow classification at trace creation while retaining the existing completion-time input and output enrichment.
  • Add regression coverage and a customer-facing release note.

AI Verification

  • Full OpenAI Agents integration suite (36 passed, 2 skipped):
    scripts/run-tests --venv 1373e84
  • Targeted regression test:
    scripts/run-tests --venv 1373e84 -- tests/contrib/openai_agents/test_openai_agents_llmobs.py -k test_llmobs_workflow_kind_set_on_trace_start
  • Repository lint suite:
    scripts/lint checks

Human Verification

Install dd-trace-py from local, rerun the tests from the linked PR in dd-source, and make sure the errors are gone.

Made with Cursor

Stamp workflow kind when OpenAI Agents traces start so prematurely finished spans remain valid LLM Observability events.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/contrib/internal/openai_agents/processor.py                     @DataDog/ml-observability
ddtrace/llmobs/_integrations/openai_agents.py                           @DataDog/ml-observability
releasenotes/notes/fix-openai-agents-workflow-kind-903518e13a23134c.yaml  @DataDog/apm-python
tests/contrib/openai_agents/test_openai_agents_llmobs.py                @DataDog/ml-observability

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 1 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 201 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 201 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=132)
ddtrace.llmobs._integrations.langchain -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=130)
ddtrace.internal.opentelemetry.context -×-> ddtrace.trace  (product:opentelemetry -> product:tracing, score=130)
ddtrace.llmobs._integrations.vllm -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=130)
ddtrace.internal.ci_visibility.filters -×-> ddtrace.trace  (product:ci_visibility -> product:tracing, score=130)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Tests

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f3522d6 | Docs | View more details | Give us feedback!

@yshapiro-57
yshapiro-57 marked this pull request as ready for review October 7, 2026 17:27
@yshapiro-57
yshapiro-57 requested review from a team as code owners October 7, 2026 17:27
@yshapiro-57
yshapiro-57 requested review from ZStriker19 and removed request for a team October 7, 2026 17:27
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T17:30:21.316751Z f3522d6 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-07T17:30:46.414390Z f3522d6 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f3522d6878

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return

self._integration.trace(oai_trace=OaiTraceAdapter(trace), submit_to_llmobs=True)
self._integration.trace(oai_trace=OaiTraceAdapter(trace), submit_to_llmobs=True, kind="workflow")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update the start-time kind guidance

This establishes a new lifecycle requirement—workflow spans must stamp their kind at trace start so interrupted traces can still be exported—but .claude/skills/llmobs-integrations/references/implementation-guide.md still limits its start-time contract to kind="agent" spans. Leaving that reference unchanged means future integrations following the repository guidance can reproduce the same missing-kind drop for workflows; extend the guidance to cover this pattern.

AGENTS.md reference: AGENTS.md:L19-L19

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant