feat(athena): Part 2 — fill the parse-path todo!() arms and the config alias by aoelvp94 · Pull Request #16367 · dbt-labs/dbt

aoelvp94 · 2026-09-18T19:49:43Z

Part of #16252. Related: #13822. Independent PR; the sequence is at the bottom.

Problem

With an athena profile parsing (#16274), dbt parse panics at the first todo!("Athena") it reaches, relation/factory.rs. Five of the nine todo!("Athena") arms are on the parse path; the other four are metadata, execution and dbt init, handled in later parts.

Solution

Fills the five parse-path arms. Each is the smallest thing that is correct for Athena, with the reason in a comment at the site:

relation/factory.rs: Athena joins the generic RelationStatic arm with the other plain-SQL adapters. Same three lines as feat(athena): AthenaDbConfig profile schema #16274; whichever merges second drops them on rebase.
adapter_impl.rs: the schema-listing column is schema_name, since Athena's information_schema is Trino's.
column/column_builder.rs (two arms): build_athena, structurally identical to build_exasol, delegating every type decision to sql_types, which already carries ATHENA_KEYS. Deliberately not routed through build_postgres_like, whose Timestamp -> "datetime" rendering that path documents as a legacy quirk.
relations/base.rs: microbatch event-time boundaries as TIMESTAMP '...' literals. The generic arm emits a bare varchar, which Trino rejects (Cannot apply operator: timestamp >= varchar); Trino's literal takes no T and no zone offset.
seed_io.rs: column-name inference is Lowercase, because the Glue catalog stores column names lowercase whatever the DDL says.

adapter.convert_type also returns dbt-athena's names for Athena: convert_text_type gives string and convert_number_type gives integer for whole numbers, with no width distinction. The macros compare against those names, so returning the DDL spellings varchar and bigint made an insert_overwrite model partitioned on a text or bigint column fail on its second run in get_partition_batches. Mapped at the convert_type boundary rather than in format_arrow_type_as_sql, which renders unit-test fixtures and seed DDL and must keep the Trino spellings and the integer widths.

Plus config_aliases: dbt-athena 1.11.0 declares _ALIASES = {"catalog": "database"} (connections_legacy.py), which canonicalises node config keys. Athena leaves the "no aliases" group and gets its own arm.

The two remaining adapter todo!("Athena") arms (MetadataAdapter construction, get_relation) are annotated, not filled; Part 5 fills them.

Verification

cargo fmt --check, cargo clippy on dbt-adapter, dbt-adapter-core, dbt-df-providers, dbt-schemas clean on the pinned toolchain.
With Parts 1, 3 and 4 applied on top, dbt parse on a 750-model project completes with 0 errors in 1.8 s; dbt ls exits 0. Without Part 4 it stops at the missing dbt-athena package, as feat(athena): AthenaDbConfig profile schema #16274 describes.

Feedback wanted: build_athena mirrors build_exasol rather than build_postgres_like; confirm that is the intended direction for Trino-family adapters.

Sequence (independent PRs, each compiles alone against main)

Part 0: feat(athena): Part 0 — map Backend::Athena to its driver entrypoint and send the driver's output-location option #16365 — ADBC driver wiring
Part 1: feat(athena): AthenaDbConfig profile schema #16274 — AthenaDbConfig profile schema (@fpiped)
Part 2: this PR — dbt-adapter core wiring
Part 3: feat(athena): Part 3 — accept partitioned_by, table_properties and force_batch as model configs #16368 — model config keys
Part 4: feat(athena): Part 4 — vendor the dbt-athena 1.11.0 macro package #16369 — dbt-athena macro package
Part 5: feat(athena): Part 5 — AthenaMetadataAdapter and athena_get_relation #16374 — AthenaMetadataAdapter and get_relation
Parts 6+7: branch ready, not yet opened — Glue / Lake Formation housekeeping shims and the table / incremental helpers (athena/parts-6-7-execution); see also feat(athena): Part 6 — AthenaAdapter methods behind the dbt-athena macros #16376
Part 8: feat(athena): Part 8 — run dbt unit tests on Athena #16370 — dbt unit tests on Athena
Part 9: feat(athena): Part 9 — dbt init profile wizard for Athena #16375 — dbt init profile wizard
Full branch for context: https://github.com/aoelvp94/dbt/tree/spike/athena-profile-parse

Checklist

I have read the contributing guide and understand what's expected of me.
I have run this code in development, and it appears to resolve the stated issue.
This PR includes tests, or tests are not required or relevant for this PR. (unit tests for the column builder and the timestamp literal are a pending follow-up on this branch)
This PR has no interface changes.

fpiped

Tested merged with the rest of the stack against a real workgroup: debug, compile --write-catalog, docs generate work, and with Part 6 the materializations do too.

One divergence that bites: convert_type returns varchar and bigint for Athena where AthenaAdapter.convert_text_type / convert_number_type return string and integer. delete_overlapping_partitions and get_partition_batches only handle integer / string / date / timestamp, so an insert_overwrite model partitioned by a text or bigint column fails with Need to add support for column type varchar on its second run. The seed macros already map string to varchar themselves, so returning dbt-athena's names is safe there.

Landing note: #16374 touches the same Athena arms in adapter_impl.rs and get_relation.rs.

…on ones Resolves five of the nine todo!("Athena") sites — the ones parse can reach — and leaves the two execution-layer sites in place with comments saying why. Filled: - relation/factory.rs: Athena joins the generic RelationStatic arm with the other plain SQL adapters. Same change dbt#16274 makes. - adapter_impl.rs: schema-listing column is `schema_name`, since Athena's information_schema is Trino's. - column_builder.rs (two sites): a `build_athena` structurally identical to `build_exasol`, delegating every adapter-specific decision to `sql_types`, which already carries ATHENA_KEYS. Deliberately NOT routed through build_postgres_like, whose Timestamp -> "datetime" rendering is a legacy quirk that path documents as broken. - relations/base.rs: microbatch event-time boundaries as explicit TIMESTAMP literals. The generic arm emits a bare varchar, which Trino rejects ("Cannot apply operator: timestamp >= varchar"); Trino's literal syntax also takes no 'T' and no zone offset. - seed_io.rs: column-name inference is Lowercase, because the Glue catalog stores column names lowercase whatever the DDL says. Left as todo!(), now annotated: - metadata/get_relation.rs: needs an athena_get_relation querying information_schema over a live connection. - adapter_impl.rs MetadataAdapter construction: needs an AthenaMetadataAdapter type against a live connection. Both are rung-3 work that parse never reaches. Guessing them would be worse than leaving an honest panic. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

dbt-athena 1.11.0 declares `_ALIASES = {"catalog": "database"}` (connections_legacy.py), which canonicalises node config keys. Athena leaves the "no aliases" group and gets its own arm. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

`adapter.convert_type` backs dbt-athena's `convert_text_type` and `convert_number_type`, which return `string` and `integer` with no width distinction. Fusion returned the DDL spellings `varchar` and `bigint`. The macros compare against dbt-athena's names: `get_partition_batches` and `delete_overlapping_partitions` handle only `integer`, `string`, `date` and `timestamp`, so an `insert_overwrite` model partitioned on a text or bigint column failed on its second run with "Need to add support for column type varchar". Mapped at the `convert_type` boundary rather than in `format_arrow_type_as_sql`, which renders DDL for unit-test fixtures and seeds and must keep the Trino spellings and the integer widths (`integer` is 32-bit there, so a collapsed name overflows bigint values). The seed macros reach the DDL spellings through `ddl_data_type`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

codescene-delta-analysis

Gates Failed
Enforce advisory code health rules (3 files with Complex Method, Large Method)

Our agent can fix these. Install it.

Gates Passed
3 Quality Gates Passed

Reason for failure

Enforce advisory code health rules	Violations	Code Health Impact
base.rs	1 advisory rule	5.94 → 5.90	Suppress
column_builder.rs	1 advisory rule	7.70 → 7.67	Suppress
adapter_impl.rs	1 advisory rule	1.95 → 1.94	Suppress

See analysis details in CodeScene

Quality Gate Profile: Clean Code Collective
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.

codescene-delta-analysis · 2026-10-06T15:00:16Z

+        // dbt-athena's `convert_text_type` returns `string` and
+        // `convert_number_type` returns `integer` for whole numbers, with no
+        // width distinction. The macros compare against those names:
+        // `get_partition_batches` and `delete_overlapping_partitions` only
+        // handle `integer` / `string` / `date` / `timestamp`, so an
+        // `insert_overwrite` model partitioned on a text or bigint column
+        // fails on its second run with "Need to add support for column type".
+        // The DDL spellings stay in `format_arrow_type_as_sql`, which the
+        // seed macros reach through `ddl_data_type`.
+        //
+        // AthenaAdapter https://github.com/dbt-labs/dbt-adapters/blob/4dc395b42dae78e895adf9c66ad6811534e879a6/dbt-athena/src/dbt/adapters/athena/impl.py#L158-L164
+        if self.adapter_type() == Athena {
+            match data_type {
+                DataType::Utf8 | DataType::Utf8View | DataType::LargeUtf8 => {
+                    return Ok("string".to_string());
+                }
+                DataType::Int8
+                | DataType::Int16
+                | DataType::Int32
+                | DataType::Int64
+                | DataType::UInt8
+                | DataType::UInt16
+                | DataType::UInt32
+                | DataType::UInt64 => return Ok("integer".to_string()),
+                _ => {}
+            }
+        }
+


❌ Getting worse: Complex Method
AdapterImpl.convert_type increases in cyclomatic complexity from 10 to 12, threshold = 9

_Suppress

codescene-delta-analysis · 2026-10-06T15:00:16Z

+            Athena => Column::new(
+                Athena,
+                name,
+                dtype,
+                char_size,
+                numeric_precision,
+                numeric_scale,
+            ),


❌ Getting worse: Large Method
ColumnBuilder.build_from_parts increases from 93 to 100 lines of code, threshold = 70

_Suppress

codescene-delta-analysis · 2026-10-06T15:00:17Z

+            // Athena is Trino, which will not coerce a varchar literal to a
+            // timestamp — `ts_col >= '2024-01-01T00:00:00+00:00'` fails with
+            // "Cannot apply operator: timestamp >= varchar". So the boundary
+            // needs an explicit TIMESTAMP literal, and Trino's literal syntax
+            // takes no 'T' separator and no zone offset (the rfc3339 boundary
+            // is always +00:00 here).
+            AdapterType::Athena => {
+                let to_athena_ts = |s: &str| {
+                    let s = s
+                        .trim_end_matches("+00:00")
+                        .trim_end_matches('Z')
+                        .replace('T', " ");
+                    format!("TIMESTAMP '{s}'")
+                };
+                (
+                    start.map(|start| format!("{event_time} >= {}", to_athena_ts(&start))),
+                    end.map(|end| format!("{event_time} < {}", to_athena_ts(&end))),
+                )
+            }


❌ Getting worse: Large Method
BaseRelation.render_with_run_filter increases from 76 to 88 lines of code, threshold = 70

_Suppress

aoelvp94 requested a review from a team as a code owner September 18, 2026 19:49

cla-bot Bot added the cla:yes label Sep 18, 2026

github-actions Bot added the community label Sep 18, 2026

This was referenced Sep 19, 2026

feat(athena): Part 6 — AthenaAdapter methods behind the dbt-athena macros #16376

Open

[Feature] Athena adapter for dbt Core 2.0 #16252

Open

fpiped reviewed Sep 19, 2026

View reviewed changes

aoelvp94 mentioned this pull request Sep 24, 2026

Release AWS Athena support #13822

Open

fpiped mentioned this pull request Sep 25, 2026

fix(athena): Part 7: Hive column quoting and case folding of quoted identifiers #16452

Open

4 tasks done

David-Banquet mentioned this pull request Oct 2, 2026

fix(athena): check contract types against the Athena column types #16571

Open

3 of 4 tasks

aoelvp94 and others added 4 commits October 6, 2026 21:58

chore(athena): changelog entry

94d9db8

aoelvp94 force-pushed the athena/part-2-core-wiring branch from 8cc7f69 to 22cce86 Compare October 6, 2026 14:58

codescene-delta-analysis Bot reviewed Oct 6, 2026

View reviewed changes

Navigation Menu

Repository navigation

feat(athena): Part 2 — fill the parse-path todo!() arms and the config alias - #16367

feat(athena): Part 2 — fill the parse-path todo!() arms and the config alias#16367
aoelvp94 wants to merge 4 commits into
dbt-labs:mainfrom
aoelvp94:athena/part-2-core-wiring

aoelvp94 commented Sep 18, 2026 •

edited

Loading

Uh oh!

fpiped left a comment

Uh oh!

codescene-delta-analysis Bot left a comment

Uh oh!

codescene-delta-analysis Bot Oct 6, 2026

Uh oh!

codescene-delta-analysis Bot Oct 6, 2026

Uh oh!

codescene-delta-analysis Bot Oct 6, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

aoelvp94 commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Verification

Sequence (independent PRs, each compiles alone against main)

Checklist

Uh oh!

fpiped left a comment

Choose a reason for hiding this comment

Uh oh!

codescene-delta-analysis Bot left a comment

Choose a reason for hiding this comment

Uh oh!

codescene-delta-analysis Bot Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

codescene-delta-analysis Bot Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

codescene-delta-analysis Bot Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aoelvp94 commented Sep 18, 2026 •

edited

Loading