Kokoro speech synthesis on PyTorch XPU and Whisper recognition on OpenVINO GenAI, isolated into separate service processes. OpenAI-style audio endpoints work with OpenWebUI and other local assistant clients.
Measured hardware: Intel Arc Pro B50, 16 GB. Other Intel Arc cards have not been validated in this project. This is an independent speech project, not a CUDA Speaches fork and not an LLM runner.
| Start here | What it covers |
|---|---|
| Setup guide | Host prerequisites, exact pins, model preparation and startup |
| Measurements | October 18 results, cold/warm distinctions and limits |
| Runtime options | Custom API versus Speaches; tested OpenVINO-only TTS candidate |
| OpenWebUI integration | TTS/STT URLs, voices, language behavior and HTTPS |
| Troubleshooting | Device selection, alignment workaround, language and memory |
Linux/Docker Compose, an Intel GPU supported by the host kernel and compute runtime, and Python 3.12+ for the download/smoke scripts are required. Check your exact GPU device nodes and ownership before editing the example.
git clone https://github.com/mkhomutskyi/speech-intel-arc.git
cd speech-intel-arc
cp .env.example .env
# Set actual CARD_DEVICE, RENDER_DEVICE, RENDER_GID and device selectors.
# Export your own .env values so the smoke client also sees an optional API key.
set -a; . ./.env; set +a
python3 scripts/download_models.py
docker compose config --quiet
docker compose build stt
docker compose up -d --wait --wait-timeout 300
curl -fsS http://127.0.0.1:8880/health
python3 scripts/smoke.py --output evidence/first-roundtripdownload_models.py explicitly downloads pinned public Kokoro/Whisper weights,
records SHA256 hashes and verifies available upstream size/LFS hashes. Review
model licenses before running it. Weights are not part of this repository.
Both inference services mount them read-only and run with HF offline mode.
Startup uses Compose --wait --wait-timeout 300 and must succeed before smoke.
If readiness fails, inspect docker compose ps -a and
docker compose logs --tail=100 stt speech, fix the cause and retry.
See setup for prerequisites and device mapping.
The API binds localhost by default; only the proxy exposes a host port.
The working B50 implementation and longer utterance/format tests were verified October 18, 2026. This October 19 packaging generalizes model paths/device settings, adds optional bearer authentication and forwards empty STT language as automatic detection. See the setup guide for the separate clean-package validation status. The new image passed isolated B50 TTS/STT acceptance. It does not silently modify an existing speech deployment.
The enabled TTS voice is English af_sky; Russian/Ukrainian TTS is not claimed. Whisper passed six natural-speaker recordings across English, Russian and Ukrainian, with both automatic detection and explicit language selection. See the validation record for errors and sample limits. One owner-assisted English microphone and audible-playback check passed on October 19, 2026. Readers still need to validate their own microphone/client. Six public clips do not establish general accent/noise quality. Models stay loaded in their worker processes for low latency; concurrent image, LLM and audio-generation jobs can contend for the same GPU.
A later October 29 publication rehearsal records a cache-storage failure, its packaging correction, and separate B50 model-load/first/warm measurements.
GET /health— selected TTS device, Torch version and model revision.GET /v1/models— speech model IDs.POST /v1/audio/speech— JSON input, voice af_sky, pcm/wav/mp3 output.POST /v1/audio/transcriptions— multipart audio upload, optional language, JSON transcript. 25 MiB/five-minute limits; STT decoding is greedy.
Original glue/scripts/documentation: Apache-2.0. Upstream packages and weights retain their licenses; NOTICE identifies sources. Read SECURITY.md.
Separate projects: Illama2 for LLMs, ComfyUI on Arc for images.
OpenWebUI integration guide connects chat, images and speech with persistent data and shared family access.