[Bug] Provider Returns 'messages field is required' - Empty Request Body

[easonLiangWorldedtech]

Bug: Provider Returns "messages field is required" - Empty Request Body (LiteLLM & LM Studio)

Problem Description

When using LiteLLM or LM Studio (OpenAI Compatible) provider in Roomote, some inference requests fail with error:

The provider returned an error: Bad Request: 'messages' field is required

This affects both LiteLLM and LM Studio providers — the issue is not limited to one provider type.

Root Cause Analysis

Issue 1: URL Path Duplication (Intermittent)

• LITELLM_BASE_URL default value in docker-compose.fixed.yml: http://roomote-proxy:4000/v1 (includes trailing /v1)
• Roomote inference gateway appends upstreamPath (v1/chat/completions) to base URL
• Result: Some requests go to http://roomote-proxy:4000/v1v1/chat/completions

Issue 2: Empty Request Body (Critical)

Proxy logs show that some /v1/chat/completions requests have empty body:

[PROXY] POST /v1/chat/completions
[PROXY] Headers: Content-Type=application/json, Authorization=True
[PROXY] Request body keys: []
[PROXY] WARNING: 'messages' field missing! Full body: {}
[PROXY] LM Studio error 400: {"error":"'messages' field is required"}

Proxy Log Evidence (Three Patterns)

Pattern 1: Correct URL + Body OK → 200 ✓

[PROXY] POST /v1/chat/completions
[PROXY] Request body keys: ['model', 'messages', 'max_tokens']
"POST /v1/chat/completions HTTP/1.1" 200 -

Pattern 2: URL Duplication + Body OK → 200 ✓ (LM Studio accepts it)

[PROXY] POST /v1/v1/chat/completions
[PROXY] Request body keys: ['model', 'messages', 'max_tokens']
"POST /v1/v1/chat/completions HTTP/1.1" 200 -

Pattern 3: Correct URL + Empty Body → 400 ✗ (FAILS)

[PROXY] POST /v1/chat/completions
[PROXY] Request body keys: []
[PROXY] WARNING: 'messages' field missing! Full body: {}
"POST /v1/chat/completions HTTP/1.1" 400 -

Fix Applied (Partial)

Changed .env file:

• Before: LITELLM_BASE_URL=http://roomote-proxy:4000/v1
• After: LITELLM_BASE_URL=http://roomote-proxy:4000 (removed trailing /v1)

Current Status

After restarting services, proxy still shows Pattern 3 failures. The root cause of empty request body is unknown — likely a bug in Roomote's inference gateway when using litellm or LM Studio provider.

Important: This issue affects both LiteLLM and LM Studio (OpenAI Compatible) providers, not just LiteLLM.

Expected Behavior

• All requests should have ['model', 'messages', ...] fields in request body
• Response status should be 200

Environment

• Self-hosted Roomote via Docker Compose
• Source: /tmp/roomote-setup-latest/ (commit 222a1cb)
• Deploy config: /tmp/roomote-setup/deploy/compose/docker-compose.fixed.yml
• LM Studio: http://host.docker.internal:1234
• LiteLLM proxy container: roomote-roomote-proxy-1 (port 4000)

[roomote-roomote]

I've analyzed this issue against current develop and here's my proposed implementation plan.

Triage summary

Two symptoms were reported:

1. URL /v1 duplication (…/v1/v1/chat/completions)
2. Empty JSON body {} on some POST /v1/chat/completions calls → upstream messages field is required

What the code already does

Inference traffic for LiteLLM goes:

sandbox OpenCode → Roomote inference gateway (/api/inference/litellm/…) → LITELLM_BASE_URL upstream → your LiteLLM/LM Studio stack

Relevant paths:

Area	Location
Gateway proxy + raw body forward	apps/api/src/handlers/inference/index.ts
Upstream base URL normalize (strips trailing /v1)	apps/api/src/handlers/inference/registry.ts (validateDynamicUpstreamBaseUrl / stripDynamicEndpointVersionSuffix)
LiteLLM provider registration	packages/types/src/inference-gateway.ts
Sandbox OpenCode baseURL rewrite onto gateway	apps/worker/src/run-task/agent-home.ts (rebaseProviderOntoGateway)
Model discover/qualify (control plane → endpoint directly)	apps/web/src/trpc/commands/task-models/local-provider-discovery.ts
Docs	apps/docs/providers/inference/litellm.mdx, apps/docs/environment-variables.mdx
Tests	apps/api/src/handlers/inference/__tests__/inference-gateway.test.ts

Issue 1 (URL join) looks already fixed on current develop:

• LITELLM_BASE_URL values like http://host:4000/v1 or …/v1/ are normalized by stripping a trailing /v1 before appending the sandbox path (/v1/chat/completions).
• Covered by the LiteLLM gateway test that asserts
  http://litellm.internal:4000/v1/ + request → http://litellm.internal:4000/v1/chat/completions.
• So removing /v1 from the env value is a reasonable operator workaround, but Roomote should accept both forms.

docker-compose.fixed.yml / roomote-proxy are not part of this repository’s official Compose (operators typically add LiteLLM as a sibling service). Trailing-/v1 guidance should live in docs + env validation, not a repo compose default that doesn’t exist here.

Issue 2 (empty body) is the remaining failure mode of concern:

• For non-Copilot providers (including LiteLLM), the gateway forwards c.req.raw.body as a stream with duplex: 'half', and strips hop-by-hop headers including content-length / transfer-encoding.
• It does not parse or rebuild the JSON payload for LiteLLM.
• Pattern 3 (Content-Type: application/json + body {}) is consistent with either:
  • the gateway occasionally forwarding a missing/consumed stream as an empty payload, or
  • a client/upstream call that legitimately POSTs {} (less common for successful chat work).
• There is currently no LiteLLM-specific regression that asserts a full messages payload is still present on the upstream fetch body.

Proposed plan

1. Harden request-body forwarding in the inference gateway (apps/api/src/handlers/inference/index.ts)

   • For POST (and other body-carrying methods) on the gateway, prefer buffering the inbound body (arrayBuffer / text) and forwarding that concrete body—same reliability posture already used for GitHub Copilot vision inspection—rather than relying solely on undici streaming of c.req.raw.body.
   • Keep streaming response paths unchanged (SSE still streams back).
   • On upstream 4xx responses for chat-completion-like paths, log a short, non-secret diagnostic: method, upstream URL host+path, inbound content-length (if any), and forwarded body byte length (not the message contents).

2. Keep / harden /v1 base URL normalization (apps/api/src/handlers/inference/registry.ts)

   • Retain stripDynamicEndpointVersionSuffix for LiteLLM / vLLM / Ollama / openai-compatible dynamic bases.
   • Add/extend focused unit coverage for: with /v1, with /v1/, without /v1, and odd trailing-slash mixes so regression is impossible.

3. Regression tests (apps/api/src/handlers/inference/__tests__/inference-gateway.test.ts)

   • LiteLLM (and ideally one other OpenAI-compatible provider): POST a real-shaped body
     { model, messages: [{ role: 'user', content: '…' }], max_tokens }
     and assert the exact upstream URL and that init.body parses back to the same payload (including non-empty messages).
   • Optional: simulate a stream/Request body path if the test harness can express it, so buffering logic is what gates the fix.

4. Docs / operator guidance (apps/docs/providers/inference/litellm.mdx, apps/docs/environment-variables.mdx)

   • Document that both http://litellm:4000 and http://litellm:4000/v1 are accepted because Roomote normalizes the suffix.
   • Add a “Common issues” entry for messages field is required pointing at empty upstream bodies and asking operators to check Roomote API inference-gateway logs for forwarded body length next to proxy logs.
   • Note that LiteLLM must be reachable from the API service that hosts /api/inference (not only from the browser, and not via container localhost unless co-located).

5. Out of scope unless proven necessary

   • Changing LM Studio / custom roomote-proxy implementations.
   • Reworking OpenCode client request construction (only pursue if gateway-side body hardening does not eliminate Pattern 3).

Why this should fix it

• Pattern 1 already shows good end-to-end behavior when body and path are correct.
• Pattern 2’s /v1/v1 is already addressed on current gateway URL join + tests; documenting that removes operator folk-fixes that can some masking the real failure.
• Pattern 3 is the user-visible outage. Making the gateway always forward a fully-read JSON body removes the main Roomote-side failure class that can produce Content-Type: application/json with {}/empty payload while still reaching the proxy. Diagnostics make any remaining client-origin empty posts visible in API logs.

Validation (when implemented)

• Targeted:
  pnpm exec dotenvx run -f .env.test -- pnpm --filter @roomote/api exec vitest run src/handlers/inference/__tests__/inference-gateway.test.ts
• Types/lint for touched packages as usual.

---

Please tag @roomote-roomote if you'd like this plan implemented, or reply with feedback.

Any of the following would also help confirm Pattern 3 after a fix ships (optional, not blockers for the plan above):

• whether empty-body 400s occur during task agent runs, Settings model qualify/discover, or both
• a short pair of Roomote API inference-gateway logs next to the proxy lines for one failing request
• Roomote image tag / git SHA if it is not effectively current develop

[daniel-lxs]

@roomote-roomote go for it

[roomote-roomote]

I'm on it. I started a task for this issue, and I'll keep updates here.

See task

[roomote-roomote]

Implemented the inference request forwarding fix and opened draft PR #669: #669

It buffers payloads before proxying, preserves Copilot vision detection, adds LiteLLM/LM Studio regression coverage, and clarifies accepted base URLs. I’m doing a self-review on GitHub and will share what I find.

[daniel-lxs]

We tested this against current develop and stock LiteLLM to isolate the request-body behavior.

• A normal fixed-length POST /v1/chat/completions succeeded with messages intact.
• An explicitly chunked request with the same JSON also succeeded with messages intact.
• The full current Roomote inference path (Roomote gateway → stock LiteLLM → a mock OpenAI-compatible upstream) returned 200 and preserved the complete payload.

We also confirmed that roomote-proxy and docker-compose.fixed.yml are not part of this repository or its official Compose files. Roomote has unrelated preview-proxy and docker-proxy services, but no built-in inference proxy by that name.

Could you please confirm whether roomote-proxy is stock LiteLLM or a custom proxy, and share the following public-safe details?

• exact image name/tag or digest
• sanitized Compose service definition
• whether Roomote works when connected directly to LiteLLM or LM Studio, bypassing roomote-proxy

Please redact credentials, customer information, and private hostnames. These results suggest the empty body may be introduced by the additional proxy layer or its configuration rather than by Roomote or stock LiteLLM. Once we know what that service is, we can reproduce its fixed-length versus chunked handling directly.