Skip to content

Web search: configured API providers lose their fallback in DuckDuckGo-unreachable networks — tail the chain with Bing instead? #6746

Description

@asto18089

Summary

For a configured API search provider (tavily, bocha, metaso, …), the backend chain ends in a DuckDuckGo tail. DuckDuckGo's internal Bing fallback only covers empty results / bot challenges — it does not cover connection failure. In networks where DuckDuckGo is unreachable (common behind the GFW and some corporate egresses), the whole chain ends in ToolError::not_available("web search backends unavailable: <id>, duckduckgo") even though Bing itself is reachable, and the provider's legitimate results are thrown away.

Evidence (current main)

  • crates/tui/src/tools/web/backend.rs SearchBackendChain::from_context (~114–130): chain is [native?] → configured → SearchProvider::DuckDuckGo tail.
  • crates/tui/src/tools/web_search.rs run_scrape_search_with_endpoints (~1560+): the DDG GET's .send() error returns immediately; the internal Bing fallback only runs after a successful response with empty results/challenge. So a DDG connection failure never reaches Bing.
  • The tool description still documents "visibly degrade through DuckDuckGo then Bing", which today only holds for the empty/challenge case.

Proposal (from the Pinvou fork, commit ff299f94b)

For the configured-API-provider chain, push the keyless tail from DuckDuckGo down to Bing (the same tail native/explicit chains already use), keeping DuckDuckGo in the chain wherever it is genuinely selectable. This changes deliberately documented fallback behavior, so we're opening a discussion first rather than a PR — alternatives we're happy to consider:

  • make the tail provider configurable ([search] fallback_provider);
  • try DDG first but fall through to Bing on connection-class errors only;
  • keep status quo and document the limitation.

We have a working patch (~10-line core + a chain-shape test per provider) and measured it restoring results in a DDG-blocked network. Happy to send it whichever way maintainers prefer.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageNew external report awaiting maintainer triage; repro, logs and version output help

    Projects

    • Status
      Backlog

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions