[ruolin59]

What changes are proposed in this pull request, and why are they necessary?

This is a deliberately narrow alternative to the closed #620, which is superseded and was not reopened.

Problem. When a stored view UNIONs tables whose structs evolved differently, the fuzzy-union rewriter wraps the evolved branch in an internal generic_project call so both branches share one struct shape. coral-schema derives that branch's Avro schema from Calcite's lower-cased type names, while the other, pass-through branch keeps the case-preserved Avro names of its source (for example requestHeader.pageKey). The exact-name Avro UNION merge then fails with requestHeader is in schema ... but not in schema ..., so Avro schema inference fails for the view with forceLowercase=false.

Change. Only RelToAvroSchemaConverter changes, and only the UNION visitor's failure path:

• The existing UNION merge always runs first and is unchanged. If it succeeds, the result is exactly what it was before.
• Only if it throws a plain RuntimeException (exact class), with forceLowercase=false, a generic_project project below the UNION, and an actual source-backed name difference, the merge is retried once with the generated fields respelled like the source Avro fields they project. The retry uses the same, unchanged SchemaUtilities.mergeUnionRecordSchema.
• The respelled variant is a names-only rewrite of the schema Coral already generated. Generated record names/namespaces, nullability, defaults, docs and properties are kept. Nothing is copied from the source schema except field spelling.
• Variants are computed lazily and per conversion, so successful conversions do not pay for them. Name recovery supports the helper operand being an input column, a field of a column (unique case-insensitive match), or the argument selected by an ordinal-return function. Anything else yields no variant.
• If the attempt fails for any reason, the original exception is rethrown unchanged (the attempt is attached as a suppressed exception).

Scope. One production file (+194/-3, 197 lines); the rest is two new test files and two new test resources (5 files total). No existing file, golden resource, or SchemaUtilities method is modified, and no SchemaRexShuttle method changes. Every conversion that succeeded before still returns a byte-identical schema, and every unrelated failure keeps its exception class and message.

Deliberately not changed (pre-existing behavior, out of scope here): default/nullability/alias/logical-type handling of generated schemas, ordinary UNION matching of differently spelled physical fields, namespace handling, and anything with forceLowercase=true.

Known limitations

• Strict mode: the generated branch's synthetic namespace still differs from the source's, so strict-mode views that are repaired by names alone in non-strict mode keep failing with their original exception.
• Only adjacent UNIONs are repaired. A non-UNION node (for example a Project) between an inner generated UNION and an enclosing UNION carries no repair, and that original failure is preserved.
• Operands other than the three shapes above (for example ITEM or arbitrary UDF/CAST expressions) are not given names; they keep their current behavior.
• Acceptance of the originally affected production views through the supported Spark 3.1 and 3.5 integrations is still outstanding, so this PR does not claim that every affected view is repaired.

How was this patch tested?

New FuzzyUnionAvroCasingTests (6 tests) with an in-memory metastore/CoralCatalog fixture; every conversion runs through both ViewToAvroSchemaConverter.create(HiveMetastoreClient) and create(CoralCatalog). Views are existing VIRTUAL_VIEWs so the real fuzzy rewriter runs, with lower-cased Hive columns and case-preserved avro.schema.literal.

• 13 views that fail at the base commit only on field casing now return the exact expected schemas: both branch orders, three branches, top-level aliases, nested/array/map records, an aliased nested-struct operand, an ordinal-return operand, ordinary access above a repaired UNION, and adjacent UNION chains. Each result is reparsed with default validation.
• A frozen oracle generated from the unchanged base commit (all strict/forceLowercase combinations) asserts that every other outcome is unchanged: working views (direct/direct, generated/generated stays lower-case, lower-case and fallback/derived branches in both orders) return byte-identical schemas, and unrepaired failures (ambiguous, missing or container provenance, inconsistent source casing, opaque unions, strict namespace, intervening Project) keep their exact exception class and message.
• The generated helper's own output (standalone branches) is asserted unchanged, and repeated conversions leave source metadata untouched.

Commands (JDK 17, Gradle cleanTest), run on the head commit:

./gradlew --no-daemon --console=plain --continue :coral-schema:cleanTest :coral-hive:cleanTest :coral-spark:cleanTest :coral-schema:test :coral-hive:test :coral-spark:test

• :coral-schema:test: 187 tests, 0 failures, 0 skipped (181 existing + 6 new).
• :coral-hive:test and :coral-spark:test: pass, including FuzzyUnionTest (13/13) and FuzzyUnionViewTest (11/11).
• :coral-schema:spotlessJavaCheck: passes (run in a temporary clone because the worktree cannot run the ratcheted check).

Not run: ./gradlew clean build, :coral-spark:spark3Test, and :coral-spark-catalog:test. The saved original-view Spark 3.1/3.5 acceptance described above has also not been run.

🤖 Generated with GitHub Copilot CLI