Skip to content

fix[notask]: integrate tiled-image geometry and resize rounding fixes - #5489

Open
DmitryMalishev wants to merge 15 commits into
mainfrom
codex/llm-media-runtime-validation
Open

DmitryMalishev wants to merge 15 commits into
mainfrom
codex/llm-media-runtime-validation

Conversation

@DmitryMalishev

@DmitryMalishev DmitryMalishev commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Task: QVAC-27614

🎯 What problem does this PR solve?

  • Integrates the fix for Issue 2 — Cached tiled-image geometry is lost. Some valid tiled server requests fail because serialized placeholders lose their grid dimensions. The engine correction is in Fabric PR #457; this PR connects that corrected engine to QVAC's shared runtime for integration validation. No addon parser change is required.
  • Integrates and tests the fix for Issue 12 — Image resize rounding differs from the reference processor. Half-way dimensions can produce an extra image-token row or column. Fabric PR #457 corrects the sizing rule; this PR verifies the behavior through the LLM addon.
  • Issue numbers refer to the reported issue list. Issue 3 — SDK completion stop reasons, its tests and the separate test-cleanup commit are now in SDK PR #5516, which can be reviewed and merged independently.

📝 How does it solve it?

  • Pin a temporary packages/fabric vcpkg overlay to Fabric commit 4fe4ded8519d4ce4ce241fe6b73594545253025e, with its verified source checksum. Retain the shipping 11259.2.0#0 port recipe; change only the source pin/checksum, with repository JSON formatting.
  • Extend the existing public Qwen3.5 addon image test with generated BMPs at resize boundaries. Verify that 464 pixels maps to 448, 496 maps to 512, and the intervening aligned size retains its extra token row. Existing tile-mode assertions remain intact.
  • A separate test-maintenance commit restores cross-addon validation: stage the six existing sharded embedding fixture files through the existing registry model-staging action, avoiding an upstream HTTP 401; accept and verify the precise unavailable-device error in the translation GPU-backend test on CPU-only runners, while retaining all GPU assertions on GPU runners. No existing assertions or CI labels are removed.
  • This PR changes no SDK/inference implementation or public API.
  • Merge order: merge Fabric #457, publish its vcpkg port, then replace this temporary overlay with the official dependency before merging this PR. Runtime/addon version and changelog updates remain separate release PRs.

🧪 How was it tested?

  • Current head 2b6c46f47 includes main 49cb26273. The overlay source and SHA-512 were refreshed after Fabric's release-branch sync. JSON validation and formatting/lint checks passed for the test-maintenance changes.
  • Current Fabric source 4fe4ded85: all three mtmd test targets passed; 100 sequential GPU requests, 14 batched controls, four CPU tile-grid cases and two GPU resize-boundary requests passed, with no request errors or crashes.
  • Fresh Fabric overlay build and consumer checks and cross-addon desktop matrix are running. The embedding fixture staging and CPU-only translation regression still require these CI results. A local translation test attempt stopped before the test at workspace #host-addon package resolution; it is not claimed as passing.
  • Previous head 52e89a2d4: all nine Fabric prebuilds and consumer checks passed. All three LLM C++ suites and all eight LLM desktop integration jobs passed; the cross-addon matrix failed on the embedding fixture and CPU-only translation cases addressed above.
  • Earlier GPU overlay validation: the released runtime failed the 464-pixel resize boundary (318 rather than 298 prompt tokens); the overlay passed both addon tests / 14 assertions. The combined implementation also passed 13 real GPU SDK requests and all 2,516 inference tests before Issue 3 was split into fix[notask]: preserve LLM completion stop reasons in the SDK #5516. These are historical results, not fresh results for the new overlay artifact.
  • Fresh Linux overlay artifact from the current workflow was SHA-256 verified and passed all 14 addon tile-mode/resize assertions on GPU. This compatibility replay used the current multi-image addon artifact from feat[api|notask]: render ordered text and media parts as one user turn #5504, so it also checks the two changes together; the exact companion cross-addon CI matrix remains pending.
  • All nine refreshed Fabric prebuild targets have passed; consumer smoke checks and the full cross-addon matrix are still running/queued. An additional 13 GPU SDK stop-reason compatibility requests passed with the new overlay.
  • Merge remains gated on successful current validation and replacement of the temporary overlay with the published Fabric dependency.

AI assistance: Codex assisted implementation and validation; manual review is required.

@DmitryMalishev DmitryMalishev added prebuilds CI: run only prebuild/cache stage (requires verified) run-cpp-addon-tests CI: run C++ addon tests (requires verified) run-desktop-addon-tests CI: run desktop integration tests (requires verified) labels Oct 30, 2026
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

License compliance — clean

No new dependency license findings in this PR.

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

  • ./docs/website
  • ./packages/asr-ggml/benchmarks/server
  • ./packages/fabric/test/integration
  • ./packages/llm-llamacpp/benchmarks/performance
  • ./packages/llm-llamacpp/benchmarks/server
  • ./packages/inference-addon-cpp/mobile
  • ./packages/sdk-e2e
  • ./packages/embed-llamacpp/benchmarks/performance
  • ./packages/embed-llamacpp/benchmarks/server
  • ./packages/vla-ggml/sim/server
  • ./.github/actions/release-merge-guard
  • ./.github/bare-toolchain

@DmitryMalishev
DmitryMalishev marked this pull request as ready for review October 30, 2026 16:14
@DmitryMalishev
DmitryMalishev requested review from a team as code owners October 30, 2026 16:14
@github-actions

Copy link
Copy Markdown
Contributor

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

Signed-off-by: Dmitry Malishev <dmitry.malishev@tether.io>
@github-actions

Copy link
Copy Markdown
Contributor

🧪 C++ Test Coverage Report

Coverage:

📊 Detailed Coverage
Filename                                         Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
addon/FitVerdict.hpp                                  18                 0   100.00%           4                 0   100.00%          27                 0   100.00%          12                 0   100.00%
model-interface/NmtLazyInitializeBackend.cpp         100                17    83.00%          12                 1    91.67%         163                28    82.82%          67                20    70.15%
model-interface/NmtLazyInitializeBackend.hpp           2                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
model-interface/TranslationModel.cpp                 351               154    56.13%          29                 7    75.86%         586               194    66.89%         227               121    46.70%
model-interface/TranslationModel.hpp                   4                 0   100.00%           4                 0   100.00%           6                 0   100.00%           0                 0         -
model-interface/nmt.cpp                               72                22    69.44%           9                 1    88.89%         139                28    79.86%          44                16    63.64%
model-interface/nmt.hpp                               51                 4    92.16%          11                 2    81.82%          53                 4    92.45%          28                 0   100.00%
model-interface/nmt_beam_search.cpp                  116                25    78.45%          10                 3    70.00%         254                32    87.40%          76                19    75.00%
model-interface/nmt_graph_decoder.cpp                164                78    52.44%          15                 7    53.33%         540               161    70.19%         112                69    38.39%
model-interface/nmt_graph_encoder.cpp                 54                13    75.93%           3                 0   100.00%         268                33    87.69%          37                16    56.76%
model-interface/nmt_loader.cpp                       334                74    77.84%          19                 0   100.00%         835               111    86.71%         205                74    63.90%
model-interface/nmt_state_backend.cpp                188                46    75.53%          20                 0   100.00%         426                79    81.46%         121                51    57.85%
model-interface/nmt_tokenization.cpp                  88                21    76.14%           8                 0   100.00%         135                36    73.33%          61                26    57.38%
model-interface/nmt_utils.cpp                        302                38    87.42%          26                 0   100.00%         429                40    90.68%         212                50    76.42%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                               1844               492    73.32%         171                21    87.72%        3862               746    80.68%        1202               462    61.56%

@greptile-apps

greptile-apps Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] No code defect was found that blocks merging after the stated dependency replacement and CI checks.

Summary

Connects the image fixes through a temporary Fabric overlay and adds resize-boundary checks.

  • Fabric builds now use a locally pinned engine source.
  • The LLM image test checks resize rounding at half-way sizes.
  • The embedding integration job stages the sharded GTE model.
  • The translation fit test accepts matching unavailable-GPU results.

Reviews (3) · Last reviewed commit: "test: refresh Fabric overlay after relea..." · Reviewed by Greptile

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

🧪 C++ Test Coverage Report

Coverage:

📊 Detailed Coverage
Filename                                         Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
addon/FitVerdict.hpp                                  18                 0   100.00%           4                 0   100.00%          27                 0   100.00%          12                 0   100.00%
model-interface/NmtLazyInitializeBackend.cpp         100                17    83.00%          12                 1    91.67%         163                28    82.82%          67                20    70.15%
model-interface/NmtLazyInitializeBackend.hpp           2                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
model-interface/TranslationModel.cpp                 351               154    56.13%          29                 7    75.86%         586               194    66.89%         225               121    46.22%
model-interface/TranslationModel.hpp                   4                 0   100.00%           4                 0   100.00%           6                 0   100.00%           0                 0         -
model-interface/nmt.cpp                               72                22    69.44%           9                 1    88.89%         139                28    79.86%          44                16    63.64%
model-interface/nmt.hpp                               51                 4    92.16%          11                 2    81.82%          53                 4    92.45%          28                 0   100.00%
model-interface/nmt_beam_search.cpp                  116                25    78.45%          10                 3    70.00%         254                32    87.40%          76                19    75.00%
model-interface/nmt_graph_decoder.cpp                164                78    52.44%          15                 7    53.33%         540               161    70.19%         112                69    38.39%
model-interface/nmt_graph_encoder.cpp                 54                13    75.93%           3                 0   100.00%         268                33    87.69%          37                16    56.76%
model-interface/nmt_loader.cpp                       334                74    77.84%          19                 0   100.00%         835               111    86.71%         205                74    63.90%
model-interface/nmt_state_backend.cpp                188                46    75.53%          20                 0   100.00%         426                79    81.46%         121                51    57.85%
model-interface/nmt_tokenization.cpp                  88                21    76.14%           8                 0   100.00%         135                36    73.33%          59                25    57.63%
model-interface/nmt_utils.cpp                        302                38    87.42%          26                 0   100.00%         429                40    90.68%         212                50    76.42%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                               1844               492    73.32%         171                21    87.72%        3862               746    80.68%        1198               461    61.52%

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

🧪 C++ Test Coverage Report

Coverage:

📊 Detailed Coverage
Filename                                         Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
addon/FitVerdict.hpp                                  18                 0   100.00%           4                 0   100.00%          27                 0   100.00%          12                 0   100.00%
model-interface/NmtLazyInitializeBackend.cpp         100                17    83.00%          12                 1    91.67%         163                28    82.82%          67                20    70.15%
model-interface/NmtLazyInitializeBackend.hpp           2                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
model-interface/TranslationModel.cpp                 351               154    56.13%          29                 7    75.86%         586               194    66.89%         225               121    46.22%
model-interface/TranslationModel.hpp                   4                 0   100.00%           4                 0   100.00%           6                 0   100.00%           0                 0         -
model-interface/nmt.cpp                               72                22    69.44%           9                 1    88.89%         139                28    79.86%          44                16    63.64%
model-interface/nmt.hpp                               51                 4    92.16%          11                 2    81.82%          53                 4    92.45%          28                 0   100.00%
model-interface/nmt_beam_search.cpp                  116                25    78.45%          10                 3    70.00%         254                32    87.40%          76                19    75.00%
model-interface/nmt_graph_decoder.cpp                164                78    52.44%          15                 7    53.33%         540               161    70.19%         112                69    38.39%
model-interface/nmt_graph_encoder.cpp                 54                13    75.93%           3                 0   100.00%         268                33    87.69%          37                16    56.76%
model-interface/nmt_loader.cpp                       334                74    77.84%          19                 0   100.00%         835               111    86.71%         205                74    63.90%
model-interface/nmt_state_backend.cpp                188                46    75.53%          20                 0   100.00%         426                79    81.46%         121                51    57.85%
model-interface/nmt_tokenization.cpp                  88                21    76.14%           8                 0   100.00%         135                36    73.33%          59                25    57.63%
model-interface/nmt_utils.cpp                        302                38    87.42%          26                 0   100.00%         429                40    90.68%         212                50    76.42%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                               1844               492    73.32%         171                21    87.72%        3862               746    80.68%        1198               461    61.52%

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

🧪 C++ Test Coverage Report

Coverage:

📊 Detailed Coverage
Filename                                         Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
addon/FitVerdict.hpp                                  18                 0   100.00%           4                 0   100.00%          27                 0   100.00%          12                 0   100.00%
model-interface/NmtLazyInitializeBackend.cpp         100                17    83.00%          12                 1    91.67%         163                28    82.82%          67                20    70.15%
model-interface/NmtLazyInitializeBackend.hpp           2                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
model-interface/TranslationModel.cpp                 351               154    56.13%          29                 7    75.86%         586               194    66.89%         225               121    46.22%
model-interface/TranslationModel.hpp                   4                 0   100.00%           4                 0   100.00%           6                 0   100.00%           0                 0         -
model-interface/nmt.cpp                               72                22    69.44%           9                 1    88.89%         139                28    79.86%          44                16    63.64%
model-interface/nmt.hpp                               51                 4    92.16%          11                 2    81.82%          53                 4    92.45%          28                 0   100.00%
model-interface/nmt_beam_search.cpp                  116                25    78.45%          10                 3    70.00%         254                32    87.40%          76                19    75.00%
model-interface/nmt_graph_decoder.cpp                164                78    52.44%          15                 7    53.33%         540               161    70.19%         112                69    38.39%
model-interface/nmt_graph_encoder.cpp                 54                13    75.93%           3                 0   100.00%         268                33    87.69%          37                16    56.76%
model-interface/nmt_loader.cpp                       334                74    77.84%          19                 0   100.00%         835               111    86.71%         205                74    63.90%
model-interface/nmt_state_backend.cpp                188                46    75.53%          20                 0   100.00%         426                79    81.46%         121                51    57.85%
model-interface/nmt_tokenization.cpp                  88                21    76.14%           8                 0   100.00%         135                36    73.33%          59                25    57.63%
model-interface/nmt_utils.cpp                        302                38    87.42%          26                 0   100.00%         429                40    90.68%         212                50    76.42%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                               1844               492    73.32%         171                21    87.72%        3862               746    80.68%        1198               461    61.52%

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

prebuilds CI: run only prebuild/cache stage (requires verified) run-cpp-addon-tests CI: run C++ addon tests (requires verified) run-desktop-addon-tests CI: run desktop integration tests (requires verified)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant