Post-routing request transformation based on the selected cluster

[leseb]

Problem

Praxis request-body filters using StreamBuffer run before normal on_request filters. Routing and load balancing happen later during on_request.

This supports routing based on information extracted from the body, but not the reverse: a filter cannot transform the request body based on the selected upstream.

For example, Praxis AI may receive an OpenAI Responses request and route it to either:

• A backend that natively supports the Responses API.
• A backend that only supports Chat Completions and requires translation.

We want to support both through one Responses pipeline. The response store, validation, rehydration, and agentic-loop state should not need to be duplicated based on the eventual backend protocol.

Today the translation filter must run before it knows which cluster was selected.

Proposal

Add a supported lifecycle point for transforming a buffered request after routing and load balancing, but before the upstream request is sent:

request body processing
→ routing
→ load balancing
→ post-routing request transformation
→ upstream

Clusters would also carry an optional application-level protocol identifier:

- filter: load_balancer
  clusters:
    - name: native
      application_protocol: openai_responses
      endpoints:
        - "native.example.com:443"

    - name: legacy
      application_protocol: openai_chat_completions
      endpoints:
        - "legacy.example.com:443"

All endpoints in a cluster would use the same application protocol.

Praxis core would expose the selected value through HttpFilterContext. Core would treat it as an opaque identifier; application-specific filters would interpret supported values.

A Responses translation filter could then:

• Pass the request through for openai_responses.
• Translate it for openai_chat_completions.
• Fail closed when the selected protocol is missing or unsupported.

iterative_request_router should execute the same phase for each internal exchange so agentic-loop inference requests behave like normal proxy requests.

The exact API is open for discussion. This could be a dedicated request-body callback or a more general post-upstream-selection hook with mutable access to the buffered body.

Alternatives

Two protocol-specific pipelines work when routing is static, but duplicate the Responses lifecycle and its stateful filters.

Moving all routing before body processing would make upstream metadata available earlier, but would interfere with routing decisions derived from request bodies.

Adding this behavior specifically to an AI routing filter would couple translation to that router and would not work with the standard router or other cluster selectors.

Example

OpenAI Responses request
        ↓
shared validation, store and agentic loop
        ↓
select cluster
        ├─ native Responses → pass through
        └─ Chat Completions → translate

Does a generic post-routing transformation capability belong in Praxis core? If so, should it be modeled as a dedicated body phase or as a broader post-upstream-selection hook?

Related

• Responses to Chat Completions filter
• API translation for non-Anthropic providers
• Provider fallback for same-format inference endpoints
• Pipeline Continuations

[leseb]

@shaneutt @alexsnaps

[shaneutt]

+1 let's go at it

[leseb]

praxis-proxy/enhancements#13