Remap dangling parent refs when dedup drops a distinct DAG node (#259)

[cboos]

(Claude) Fixes #259.

Problem

Newer Claude Code streams one API response as several assistant JSONL entries sharing one message.id, and can emit two consecutive empty thinking blocks ("thinking": "", signature only) as separate entries within the same millisecond. Their assistant dedup keys — (type, timestamp, isMeta, sessionId, message.id + block_types) — are identical, so deduplicate_messages() treated the second entry as a version stutter and silently dropped it. But it is a distinct DAG node: its tool_use child then dangled and got promoted to a spurious root branch:

WARNING: Orphan node 66250b09-…: parentUuid b050ae73-… not found in loaded data (promoting to root)

Adding uuid to the dedup key is not the fix: true version stutters re-log the message with a different uuid per copy (pinned in test_version_deduplication.py), so that would stop deduplicating them.

Fix

• Survivor remap: whenever dedup drops an entry, record dropped_uuid → survivor_uuid; afterwards rewrite any surviving entry's dangling parentUuid (and summary leafUuid) to the survivor. Children of a dropped copy re-parent to the surviving copy — equally correct for empty-thinking splits and true version stutters.
• Empty-thinking run merge (per the follow-up comment on the issue): consecutive empty-thinking entries collapse onto the run's head even across distinct timestamps, same session only — they render nothing and essentially never become fork points a posteriori. Cross-session parent links are fork attachment points and stay addressable. Token accounting is unaffected in the observed pattern (usage is counted once per requestId, shared by all entries of one response).

Verification

• 6 new tests in TestDedupDagRemap; 4 fail on the previous code (the two guards already held), all pass with the fix.
• End-to-end: a JSONL with the issue's exact trigger pattern run through load → dedup → DAG build reproduces the orphan warning + spurious root on the previous code, and renders a single root with the child re-parented to the run head on the fixed code.
• Full CI green; no snapshot churn.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved transcript deduplication so empty-thinking assistant entries no longer break conversation links.
  • Preserved parent/child relationships when duplicate entries are removed, including summary references and tool-use chains.
  • Prevented consecutive empty-thinking entries from being merged in cases where they shouldn’t cross session or message boundaries.
  • Ensured non-empty thinking content remains distinct and is not incorrectly collapsed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2a8f3ffd-e993-4fb3-987f-7808a511146f

📥 Commits

Reviewing files that changed from the base of the PR and between 4037b26 and 1414b8c.

📒 Files selected for processing (2)

• claude_code_log/converter.py
• test/test_version_deduplication.py

---

📝 Walkthrough

Walkthrough

Modified deduplicate_messages in converter.py to track dropped-to-survivor UUID mappings, avoiding orphaned children when duplicate or empty-thinking assistant entries are removed. Added helper functions for empty-thinking detection, survivor resolution, and merging consecutive empty-thinking runs. Added corresponding tests validating parent/leaf UUID re-parenting.

Changes

Deduplication Survivor Remapping

Layer / File(s)	Summary
Empty-thinking detection and survivor resolution helpers claude_code_log/converter.py	Imports ThinkingContent and adds _is_empty_thinking() and _resolve_survivor() helpers to detect empty-thinking entries and follow dropped-to-survivor UUID chains.
Deduplication rewrite and empty-thinking run merging claude_code_log/converter.py	Adds dropped_to_survivor tracking, rewrites parentUuid/leafUuid for entries referencing dropped nodes, and adds _merge_empty_thinking_runs() to collapse consecutive empty-thinking entries while re-parenting children and respecting session boundaries.
Tests for DAG remap and empty-thinking merge test/test_version_deduplication.py	Adds new imports, test helper builders/assertions, and TestDedupDagRemap covering re-parenting after dropped stutters, cross-timestamp merges, session-boundary isolation, and summary leafUuid remapping.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Loader as load_transcript
  participant Dedup as deduplicate_messages
  participant Map as dropped_to_survivor
  participant Merge as _merge_empty_thinking_runs
  participant Output as Deduped entries

  Loader->>Dedup: raw transcript entries
  Dedup->>Map: record dropped_uuid to survivor_uuid
  Dedup->>Merge: pass deduped entries
  Merge->>Map: update mapping for merged empty-thinking runs
  Merge->>Dedup: return merged entries
  Dedup->>Dedup: resolve parentUuid/leafUuid via Map
  Dedup->>Output: emit entries with re-parented references

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main fix: remapping parent refs when dedup drops a distinct DAG node for issue #259.
Linked Issues check	✅ Passed	The code and tests implement survivor remapping and re-parent dangling children, matching the issue's expected dedup behavior.
Out of Scope Changes check	✅ Passed	The changes stay focused on dedup remapping and related tests; no unrelated feature work stands out.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch dev/dedup-dag-fix

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}