Skip to content

fix: do not close parentSink in Stitcher.finish() success path - #22792

Open
waterWang wants to merge 1 commit into
opensearch-project:mainfrom
waterWang:fix/lm-stitcher-root-sink-close
Open

fix: do not close parentSink in Stitcher.finish() success path#22792
waterWang wants to merge 1 commit into
opensearch-project:mainfrom
waterWang:fix/lm-stitcher-root-sink-close

Conversation

@waterWang

Copy link
Copy Markdown

Description

When the LateMaterialization stage is the root of the query plan (QTF rewrite fires, no downstream ReduceStageExecution), the parentSink is a RowProducingSink whose close() clears every buffered batch. The QueryExecution then reads the result via outputSource().readResult() after the stage terminal transition, finding an empty batch list and silently returning zero rows.

Root Cause

Stitcher.finish() calls parentSink.close() in its success path after feeding the output. For a non-root LM stage, this signals EOF to the parent stage input sink. For a root LM stage, the parentSink is the fresh RowProducingSink created by StageExecutionBuilder.buildRootExecution(), and RowProducingSink.close() clears every buffered batch — while QueryExecution reads the answer after the root stage terminal transition, via outputSource().readResult().

Fix

Remove the parentSink.close() call from the success path of Stitcher.finish(). The sink lifecycle is managed by the stage execution, not by the stitcher. The LateMaterializationStageExecution.onTerminalTransition() already handles cleanup for CANCELLED/FAILED states.

Fixes #22786

When the LateMaterialization stage is the root of the query plan (QTF
rewrite fires, no downstream ReduceStageExecution), the parentSink is
a RowProducingSink whose close() clears every buffered batch. The
QueryExecution then reads the result via outputSource().readResult()
after the stage's terminal transition, finding an empty batch list
and silently returning zero rows.

Fixes opensearch-project#22786
@waterWang
waterWang requested a review from a team as a code owner August 20, 2026 22:46
@github-actions github-actions Bot added bug Something isn't working Search:Query Capabilities labels Aug 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 No relevant tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review

Non-root stage EOF signaling

Removing parentSink.close() from the success path affects both root and non-root LM stages. For non-root LM stages, this close previously signaled EOF to the parent stage's input sink. The PR description asserts that "The sink lifecycle is managed by the stage execution", but it should be verified that the stage's terminal transition reliably closes the parentSink in the SUCCESS case for non-root stages as well — otherwise the downstream ReduceStageExecution may hang waiting for input completion.

// The parentSink lifecycle is managed by the stage execution, not by the
// stitcher. For a root-stage LM (QTF plan where the LM stage is the plan root),
// parentSink is a RowProducingSink whose close() clears every buffered batch,
// while QueryExecution reads the result after the stage's terminal transition
// via outputSource().readResult(). Closing the sink here would free the result
// before anyone reads it, causing the query to silently return zero rows.
// See https://github.com/opensearch-project/OpenSearch/issues/22786.
logger.debug("[Stitcher] emitted rows={}", totalRows);

@github-actions

Copy link
Copy Markdown
Contributor

✅ Gradle check result for eb52f7a: SUCCESS

@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.65%. Comparing base (42a1e95) to head (eb52f7a).

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #22792      +/-   ##
============================================
+ Coverage     71.61%   71.65%   +0.03%     
- Complexity    77336    77350      +14     
============================================
  Files          6170     6170              
  Lines        359671   359671              
  Branches      52450    52450              
============================================
+ Hits         257584   257710     +126     
+ Misses        81696    81521     -175     
- Partials      20391    20440      +49     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Search:Query Capabilities

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Multi-shard sort <field> | head N returns zero rows on the analytics-engine route

1 participant