Skip to content

fix: Use backoff-provided exception instead of frame introspection - #607

Merged
edgarrmondragon merged 4 commits into
mainfrom
fix/backoff-handler-exception-details
Sep 2, 2026
Merged

fix: Use backoff-provided exception instead of frame introspection#607
edgarrmondragon merged 4 commits into
mainfrom
fix/backoff-handler-exception-details

Conversation

@edgarrmondragon

@edgarrmondragon edgarrmondragon commented Sep 1, 2026

Copy link
Copy Markdown
Member

Summary

  • CI logs (error.log) showed 3 flaky test failures: KeyError: "local variable ''e'' is not defined", all from backoff_handler in tap_github/client.py.
  • backoff_handler used a frame-introspection hack (walking f_back.f_back.f_locals["e"]) to retrieve the exception object, working around litl/backoff#158 since older backoff versions didn't pass the exception to on_backoff handlers.
  • backoff isn't a direct dependency of tap-github — it's pulled in transitively via singer-sdk, and the current lockfile happens to resolve it to python-backoff==3.0.0, whose internal call stack layout changed enough that the frame two levels up no longer has a local named e, causing the crash.
  • The upstream issue is actually resolved: backoff 3.0's Details TypedDict now includes an exception key populated directly on on_backoff/on_giveup calls, so the introspection hack is no longer needed.

Changes

  • backoff_handler now reads exc = details.get("exception") instead of frame-walking, and narrows it with isinstance(exc, RetriableAPIError) before accessing .response (avoiding an unsafe cast).
  • Removed the now-unused inspect import and FrameType type-only import.

Test plan

  • ruff check / ruff format --check pass on the changed file
  • Re-ran the three previously-failing tests (test_last_state_message_is_valid, test_get_a_repository_in_repo_list_mode[True/False]) locally — the KeyError no longer occurs (they now only fail on rate limiting due to no local GITHUB_TOKEN, unrelated to this fix)

…ction

python-backoff 3.0 changed its internal call stack, so the frame-walking
hack in backoff_handler (a workaround for litl/backoff#158) started
raising KeyError: "local variable ''e'' is not defined" and crashing
retries. backoff 3.0 already resolved that upstream issue by passing the
exception directly via details["exception"], so read it from there
instead.
@edgarrmondragon
edgarrmondragon requested a review from a team as a code owner September 1, 2026 15:31
Avoids an unsafe cast on details["exception"] by narrowing it with
isinstance(exc, RetriableAPIError) before accessing .response.
@edgarrmondragon edgarrmondragon changed the title fix(client): use backoff-provided exception instead of frame introspection fix: Use backoff-provided exception instead of frame introspection Sep 1, 2026
@edgarrmondragon
edgarrmondragon merged commit 65b0e31 into main Sep 2, 2026
8 of 9 checks passed
@edgarrmondragon
edgarrmondragon deleted the fix/backoff-handler-exception-details branch September 2, 2026 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant