Skip to content

Refactor and improvement of AODBcRewriter#2345

Open
sawenzel wants to merge 1 commit intoAliceO2Group:masterfrom
sawenzel:swenzel/AODRewriter_new
Open

Refactor and improvement of AODBcRewriter#2345
sawenzel wants to merge 1 commit intoAliceO2Group:masterfrom
sawenzel:swenzel/AODRewriter_new

Conversation

@sawenzel
Copy link
Copy Markdown
Contributor

@sawenzel sawenzel commented May 4, 2026

Replaces the recently patched version of AODBcRewriter with a full refactor, fixing a correctness bug introduced by commit 95cc50b ("Fix concatenation for McCollisions indexed Trees").

Problem

Stage 2 was added to reorder every table carrying fIndexMcCollisions (including O2mcparticle) so that rows are grouped by their new MC-collision parent. The fIndexMcCollisions column was correctly remapped, but two other index columns inside O2mcparticle were left untouched:

• fIndexArray_Mothers — VLA of Int_t pointing to mother particles
within the same O2mcparticle table
• fIndexSlice_Daughters — fixed [2] array of Int_t giving the
[first, last] daughter slice in the same table

After reordering, these columns still held old row positions. The pointed-to rows now belonged to different MC collisions, producing structurally invalid output verified by 267,605 cross-collision violations in the example AO2D. O2Physics detected this at analysis time as:

[FATAL] MC particle N with PDG P has daughter with index M >
MC particle table size (+ offset)

Additionally, label tables (O2mctracklabel, O2mcfwdtracklabel, O2mcmfttracklabel, O2mccalolabel) carry fIndexMcParticles / fIndexArrayMcParticles pointing into O2mcparticle. After O2mcparticle is reordered those pointers became stale; they were also not remapped.

A latent memory-safety bug was also present: fIndexSlice_Daughters is stored as a fixed-size branch with leaf title fIndexSlice_Daughters[2] (8 bytes), but the generic branch I/O code allocated only 4 bytes for it (treating it as a plain scalar), causing a silent out-of-bounds write on every GetEntry call.

Changes

  1. BranchDesc gains nElems (= leaf->GetLen()), so the I/O buffer is sized nElems * elemSize for fixed-size arrays as well as scalars. This fixes the fIndexSlice_Daughters buffer underallocation.

  2. A new ExtraRemap struct and an extraRemaps parameter on rewriteTable() allow any number of additional integer columns to be remapped in-place within the same write pass. Each extra remap applies remapIdx() to every Int_t value in the branch (scalar, fixed array, or VLA) after GetEntry and before Fill, using the caller-supplied PermMap.

  3. In stage2_MCCollIndexedTables, when processing O2mcparticle:

    • The self-permutation (oldRow → newRow) is computed from rowOrder before calling rewriteTable — this is exact because rowOrder contains unique source rows in output order.
    • fIndexArray_Mothers and fIndexSlice_Daughters are passed as ExtraRemaps using this self-permutation. The stable sort by new MC-collision position preserves the original within-collision particle order, so daughter slices remain contiguous and remapping [first, last] via the same permutation is correct.
  4. processPasteJoinTables now locates the MC-particle permutation in allPerms and builds ExtraRemaps for fIndexMcParticles and fIndexArrayMcParticles in any table that carries those branches. Tables that need only index remapping (no row reordering) are processed with an identity rowOrder via rewriteTable instead of CloneTree.

  5. A new AODBcRewriterValidate(fname) function (callable as a standalone ROOT macro) checks:

    • BC table is strictly monotonic in fGlobalBC.
    • All fIndexSlice_Daughters values are in [0, nMcParticle) and point to particles sharing the same fIndexMcCollisions as the parent row.
    • All fIndexArray_Mothers values satisfy the same constraints. Returns true/false and prints [FAIL] lines for each failing DF.

Replaces the patched version with a full refactor that fixes a critical
correctness bug introduced by commit 95cc50b ("Fix concatenation for
McCollisions indexed Trees").

Problem
-------
Stage 2 was added to reorder every table carrying fIndexMcCollisions
(including O2mcparticle) so that rows are grouped by their new MC-collision
parent. The fIndexMcCollisions column was correctly remapped, but two other
index columns inside O2mcparticle were left untouched:

  • fIndexArray_Mothers  — VLA of Int_t pointing to mother particles
                          within the same O2mcparticle table
  • fIndexSlice_Daughters — fixed [2] array of Int_t giving the
                           [first, last] daughter slice in the same table

After reordering, these columns still held old row positions.  The pointed-to
rows now belonged to different MC collisions, producing structurally invalid
output verified by 267,605 cross-collision violations in the example AO2D.
O2Physics detected this at analysis time as:

  [FATAL] MC particle N with PDG P has daughter with index M >
          MC particle table size (+ offset)

Additionally, label tables (O2mctracklabel, O2mcfwdtracklabel,
O2mcmfttracklabel, O2mccalolabel) carry fIndexMcParticles /
fIndexArrayMcParticles pointing into O2mcparticle.  After O2mcparticle is
reordered those pointers became stale; they were also not remapped.

A latent memory-safety bug was also present: fIndexSlice_Daughters is stored
as a fixed-size branch with leaf title fIndexSlice_Daughters[2] (8 bytes),
but the generic branch I/O code allocated only 4 bytes for it (treating it
as a plain scalar), causing a silent out-of-bounds write on every GetEntry
call.

Changes
-------
1. BranchDesc gains nElems (= leaf->GetLen()), so the I/O buffer is sized
   nElems * elemSize for fixed-size arrays as well as scalars.  This fixes the
   fIndexSlice_Daughters buffer underallocation.

2. A new ExtraRemap struct and an extraRemaps parameter on rewriteTable()
   allow any number of additional integer columns to be remapped in-place
   within the same write pass.  Each extra remap applies remapIdx() to every
   Int_t value in the branch (scalar, fixed array, or VLA) after GetEntry
   and before Fill, using the caller-supplied PermMap.

3. In stage2_MCCollIndexedTables, when processing O2mcparticle:
   - The self-permutation (oldRow → newRow) is computed from rowOrder before
     calling rewriteTable — this is exact because rowOrder contains unique
     source rows in output order.
   - fIndexArray_Mothers and fIndexSlice_Daughters are passed as ExtraRemaps
     using this self-permutation.
   The stable sort by new MC-collision position preserves the original
   within-collision particle order, so daughter slices remain contiguous and
   remapping [first, last] via the same permutation is correct.

4. processPasteJoinTables now locates the MC-particle permutation in
   allPerms and builds ExtraRemaps for fIndexMcParticles and
   fIndexArrayMcParticles in any table that carries those branches.  Tables
   that need only index remapping (no row reordering) are processed with an
   identity rowOrder via rewriteTable instead of CloneTree.

5. A new AODBcRewriterValidate(fname) function (callable as a standalone
   ROOT macro) checks:
   - BC table is strictly monotonic in fGlobalBC.
   - All fIndexSlice_Daughters values are in [0, nMcParticle) and point to
     particles sharing the same fIndexMcCollisions as the parent row.
   - All fIndexArray_Mothers values satisfy the same constraints.
   Returns true/false and prints [FAIL] lines for each failing DF.

Verification
------------
Validated on example_AOD/AO2D_pre.root (merged, pre-rewrite):
  AODBcRewriterValidate reports PASSED — confirms input was clean.
Old rewriter output (AO2D_after.root):
  AODBcRewriterValidate reports FAILED — 267,605 cross-collision violations
  in DF_3594457012001 and DF_3594457012003.
New rewriter output:
  AODBcRewriterValidate reports PASSED (3 DFs checked) — zero violations.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
@sawenzel sawenzel requested a review from jackal1-66 as a code owner May 4, 2026 12:29
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 4, 2026

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant