Skip to content

feat(rust/ffi): support ADBC 1.1.0 rich errors from exported streams#4475

Open
fornwall wants to merge 1 commit into
apache:mainfrom
fornwall:rust-ffi-stream-errno-status
Open

feat(rust/ffi): support ADBC 1.1.0 rich errors from exported streams#4475
fornwall wants to merge 1 commit into
apache:mainfrom
fornwall:rust-ffi-stream-errno-status

Conversation

@fornwall

@fornwall fornwall commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Disclaimer: This PR description and code was AI generated. I have reviewed the code myself. Let me know if I should dig into any details or make modifications.

The driver exporter exported result readers via arrow-rs's FFI_ArrowArrayStream::new, which reduces every mid-stream failure to an errno plus a message string: rich adbc_core errors surfaced through
ArrowError::ExternalError landed in the catch-all EINVAL, and the ADBC 1.1.0 ErrorFromArrayStream slot in the driver table was left as a TODO.

Replace the export machinery (adapted from arrow-rs's ffi_stream.rs) with one that stashes the full adbc_core::error::Error found in the error's source chain (or synthesized from the ArrowError variant):

  • get_next/get_schema return the errno for the error's ADBC status via a port of the canonical InternalAdbcStatusCodeToErrno table, so e.g. cancelled operations report ECANCELED as the validation suite expects.
  • ErrorFromArrayStream is now implemented, handing consumers the stream-owned FFI_AdbcError (message, SQLSTATE, vendor code, details) together with its status code, and NULL for foreign streams. Own streams are recognized via a registry of live private-data addresses, since Rust does not guarantee the function-pointer identity the C/C++ drivers rely on (Miri fails such comparisons).

Tested by unit tests, a new end-to-end test driving the exported C driver table against a failing dummy-driver stream, and a clean run of the adbc_ffi suite under Miri.

@fornwall fornwall requested a review from wjones127 as a code owner July 7, 2026 20:12
@fornwall fornwall changed the title feat(rust/ffi): report ECANCELED through exported streams for cancell… feat(rust/ffi): report ECANCELED through exported streams for cancelled operations Jul 7, 2026
@lidavidm

lidavidm commented Jul 7, 2026

Copy link
Copy Markdown
Member

I'm not sure I want to do this; if I were to do this, it would be to implement the 1.1.0 API functions around rich errors from ArrayStreams, and not just this on its own.

@fornwall fornwall force-pushed the rust-ffi-stream-errno-status branch from d91e7b2 to e20fa42 Compare July 8, 2026 00:30
@fornwall fornwall changed the title feat(rust/ffi): report ECANCELED through exported streams for cancelled operations feat(rust/ffi): support ADBC 1.1.0 rich errors from exported streams Jul 8, 2026
The driver exporter exported result readers via arrow-rs's
FFI_ArrowArrayStream::new, which reduces every mid-stream failure to an
errno plus a message string: rich adbc_core errors surfaced through
ArrowError::ExternalError landed in the catch-all EINVAL, and the ADBC
1.1.0 ErrorFromArrayStream slot in the driver table was left as a TODO.

Replace the export machinery (adapted from arrow-rs's ffi_stream.rs)
with one that stashes the full adbc_core::error::Error found in the
error's source chain (or synthesized from the ArrowError variant):

- get_next/get_schema return the errno for the error's ADBC status via
  a port of the canonical InternalAdbcStatusCodeToErrno table, so e.g.
  cancelled operations report ECANCELED as the validation suite expects.
- ErrorFromArrayStream is now implemented, handing consumers the
  stream-owned FFI_AdbcError (message, SQLSTATE, vendor code, details)
  together with its status code, and NULL for foreign streams. Own
  streams are recognized via a registry of live private-data addresses,
  since Rust does not guarantee the function-pointer identity the C/C++
  drivers rely on (Miri fails such comparisons).

Tested by unit tests, a new end-to-end test driving the exported C
driver table against a failing dummy-driver stream, and a clean run of
the adbc_ffi suite under Miri.

Signed-off-by: Fredrik Fornwall <fredrik@fornwall.net>
@fornwall fornwall force-pushed the rust-ffi-stream-errno-status branch from e20fa42 to 3ec70f5 Compare July 8, 2026 00:43
@fornwall

fornwall commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

I'm not sure I want to do this; if I were to do this, it would be to implement the 1.1.0 API functions around rich errors from ArrayStreams, and not just this on its own.

@lidavidm Thanks! I've updated the PR to implement that now, instead of the limited initial scope.

@fornwall

fornwall commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

AI generated description for the gain for python clients:

The Python DBAPI layer wraps every result set (cursor.fetchall(),
fetch_arrow_table(), fetch_df(), iteration) in AdbcRecordBatchReader
(python/adbc_driver_manager/adbc_driver_manager/_reader.pyx), whose fetch methods catch any
mid-stream failure and call AdbcErrorFromArrayStream to upgrade the exception to a proper
ADBC error. For Rust-exported drivers that upgrade path is currently dead: the driver leaves
the ErrorFromArrayStream slot unset, so the C driver manager doesn't wrap the stream and
AdbcErrorFromArrayStream returns NULL.

The result today: every mid-stream failure from a Rust driver reaches Python as
pyarrow.lib.ArrowInvalid (the exporter's blanket EINVAL, mapped by Arrow C++'s stream
import) carrying only a message string. It is not part of the DBAPI exception hierarchy, so
except adbc_driver_manager.dbapi.OperationalError: handlers never fire; there is no status
code, SQLSTATE, vendor code, or error details; and a query cancelled via
cursor.adbc_cancel() is indistinguishable from malformed data.

With this PR the driver advertises ErrorFromArrayStream, the driver manager wraps its
streams, and Python's existing machinery (convert_error) does the rest: the same failure
now raises the documented typed exception — e.g. OperationalError with
status_code=CANCELLED for a cancelled query, ProgrammingError for NOT_FOUND/UNAUTHORIZED,
IntegrityError for constraint violations — with sqlstate, vendor_code, and structured
details populated from the driver's rich error. No Python-side changes needed; the client
support has been there since ADBC 1.1.0, waiting on the driver.

@lidavidm

lidavidm commented Jul 8, 2026

Copy link
Copy Markdown
Member

Sorry - this is something that I'd want to spend actual time on

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants