Skip to content

refactor(llmobs): abstract llmobs from llama_index integration code - #20860

Open
quinna-h wants to merge 2 commits into
emmett.butler/llmobs-contribfrom
quinna.halim/llmobs-contrib-llama-index
Open

quinna-h wants to merge 2 commits into
emmett.butler/llmobs-contribfrom
quinna.halim/llmobs-contrib-llama-index

Conversation

@quinna-h

@quinna-h quinna-h commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Description

Stacked on PR #20760 which introduced the LlmEvents span-lifecycle events and first LLMObs subscriber. This PR does the same for llama_index and generalizes the machinery so each additional integration is a ~40 line file.

Changes:

  • Shared subscriber base: The handling that lived inline in the anthropic subscribers moves to ddtrace/llmobs/_contrib/_subscribers.py as LLMObsLlmSubscriber: lazy integration construction from config.<component>, component filtering (the LlmEvents are shared by every LLM integration, so handlers must ignore other components' contexts), and the three lifecycle handlers. A concrete integration now supplies only a component name and integration class. Each subscriber still declares its own one-line on_event because Subscriber.__init_subclass__ binds on_event to the class that defines it.
  • Contrib owns its APM tags: patch.py gains MODEL_TAG/PROVIDER_TAG
    (llama_index.request.{model,provider}) and sets them on the event. Non-LLM operations
    (query, retrieval, agent) have no model or provider, so those tags are omitted rather
    than written as null.
  • Streaming: handle_streamed_response no longer takes an integration; the LlamaIndex stream handlers only need the context to dispatch the deferred ended event, so the BaseStreamHandler integration slot goes unused (None)

Testing

Existing llama_index and llmobs suites are the behavioral contract and pass unchanged.

Risks

Additional Notes

quinna-h and others added 2 commits October 6, 2026 17:22
Applies the same inversion the Anthropic integration received to LlamaIndex,
so the contrib no longer imports LLMObs.

The contrib previously constructed a LlamaIndexIntegration in patch(), stashed
it on llama_index.core._datadog_integration, and hung it off every
LlmRequestEvent, which meant ddtrace/contrib/internal/llama_index imported
ddtrace.llmobs. Now the contrib dispatches the shared LlmEvents span lifecycle
and LLMObs attaches its own subscribers on the llama_index.patch core event,
mirroring ddtrace/llmobs/_contrib/anthropic.

- patch(): drop the integration instance and _get_integration(); take
  integration_config/service from config.llama_index; own the
  llama_index.request.model and llama_index.request.provider APM tags via
  event.tags so they are set whether or not LLMObs is present; dispatch
  llama_index.patch / llama_index.unpatch.
- _streaming.py: drop the integration parameter; the stream handlers only
  need the context to dispatch the deferred ended event.
- LlamaIndexIntegration._set_base_span_tags is now a no-op: the contrib owns
  the APM tags and LlamaIndex has no per-request base_url to record.
- The LLMObs subscribers register off the patch event, with the already-patched
  fallback keyed on llama_index.core rather than the top-level package.

The dependency direction detector drops from 183 to 181 violations (both
contrib.internal.llama_index -> llmobs._integrations edges removed, no new
violations, no uncovered modules).

Testing: llmobs::llama_index (61 passed) and llmobs::anthropic (108 passed) on
py3.14; llmobs::llmobs matches the branch baseline exactly (1563 passed, same
11 pre-existing RC/sampling failures with and without this change).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against emmett.butler/llmobs-contrib using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/contrib/internal/llama_index/_streaming.py                      @DataDog/python-guild @DataDog/apm-idm-python
ddtrace/contrib/internal/llama_index/patch.py                           @DataDog/python-guild @DataDog/apm-idm-python
ddtrace/llmobs/_contrib/__init__.py                                     @DataDog/ml-observability
ddtrace/llmobs/_contrib/_subscribers.py                                 @DataDog/ml-observability
ddtrace/llmobs/_contrib/anthropic/__init__.py                           @DataDog/ml-observability
ddtrace/llmobs/_contrib/anthropic/subscribers.py                        @DataDog/ml-observability
ddtrace/llmobs/_contrib/llama_index/__init__.py                         @DataDog/ml-observability
ddtrace/llmobs/_contrib/llama_index/subscribers.py                      @DataDog/ml-observability
ddtrace/llmobs/_integrations/llama_index.py                             @DataDog/ml-observability
tests/contrib/llama_index/conftest.py                                   @DataDog/python-guild

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 1 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Dependency direction analysis

📈 Existing violations got worse

1 pre-existing violation(s) increased in severity (e.g. their target became more depended-on, or got pulled into an import cycle), though the edge itself isn't new:

ddtrace.contrib._events.llm -×-> ddtrace.llmobs._integrations.base  (contrib -> product:llmobs, score=21, +1 vs base)

⚠️ Existing dependency direction violations

There are 181 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 181 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=132)
ddtrace.internal.ci_visibility.git_client -×-> ddtrace.trace  (product:ci_visibility -> product:tracing, score=130)
ddtrace.profiling.collector.stack -×-> ddtrace.trace  (product:profiling -> product:tracing, score=130)
ddtrace.llmobs._integrations.llama_index -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=130)
ddtrace.debugging._debugger -×-> ddtrace.trace  (product:debugging -> product:tracing, score=130)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

✅ Dependency direction violations removed

2 violation(s) have been removed by this PR.

ddtrace.contrib.internal.llama_index.patch -×-> ddtrace.llmobs._integrations  (contrib -> product:llmobs, score=14)
ddtrace.contrib.internal.llama_index._streaming -×-> ddtrace.llmobs._integrations  (contrib -> product:llmobs, score=14)

@quinna-h
quinna-h marked this pull request as ready for review October 7, 2026 18:50
@quinna-h
quinna-h requested review from a team as code owners October 7, 2026 18:50
@quinna-h
quinna-h requested review from Yun-Kim and dubloom and removed request for a team October 7, 2026 18:50
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T18:53:47.949784Z f831724 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@quinna-h
quinna-h requested a review from emmettbutler October 7, 2026 18:51

@emmettbutler emmettbutler left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty much matches what we discussed. Nice work.

Comment on lines +13 to +15
class LLMObsLlamaIndexSubscriber(LLMObsLlmSubscriber):
component = LLAMA_INDEX_COMPONENT
integration_cls = LlamaIndexIntegration

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to not just compute the component and integration class on the main LLMObsLlmSubscriber class? We wouldnt need these per integration subscribers AFAIK if we did that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants