Local TTS service (tts_server_runtime)

Want Qwen3-TTS (3-second voice cloning / CustomVoice / VoiceDesign) without writing your own inference? The host ships tts-server.exe (qwentts.cpp, MIT), speaking OpenAI-compatible HTTP. Your plugin only starts the process, sends requests and manages voices.

The runtime is bundled with the installer — users never click "Download runtime" and need no network.

---

1) Declaring and locating it

{
  "requires_runtime": [
    { "id": "tts_server_runtime", "reason": "tts-server runtime (qwentts.cpp, bundled with the installer)" }
  ]
}

from pathlib import Path

rt = self.host.runtime_dir("tts_server_runtime")   # the directory when ready, else ""
if not rt:
    return False, "model_not_ready", "The TTS-Server runtime is missing; it is bundled in 0.2.0+, please reinstall"

exe = Path(rt) / "tts-server.exe"

Do not download, unpack or bundle this binary yourself, and do not list it in model_artifacts. Besides tts-server.exe the directory holds MinGW runtimes (libgomp-1.0.dll, libstdc++-6.dll, libgcc_s_seh-1.dll, libwinpthread-1.dll) and vulkan-1.dll — use the whole directory; copying only the exe will fail to start on other machines.

Old form (0.1.x only): back then the carrier plugin workshop.common_tts_server provided it, so host.plugin_dir("workshop.common_tts_server") / "tts-server" / "tts-server.exe" worked at the time. 0.2.0 has no such carrier plugin — that form returns an empty path. New plugins always use runtime_dir().

---

2) Startup flags

args = [
    str(exe),
    "--model", str(talker_gguf),     # required: the talker LM (qwen-talker-*.gguf)
    "--codec", str(tokenizer_gguf),  # required: the codec (qwen-tokenizer-*.gguf)
    "--device", "Vulkan0",           # CUDA0 / Vulkan0 / ROCm0 / MTL0 / CPU
    "--host", "127.0.0.1",
    "--port", str(port),
    "--alias", "Qwen3-TTS-1.7B",     # shows up in /v1/models, used to confirm "the service on this port is mine"
]

Flag	Description
--model / --codec	Both GGUFs are required; give --alias a stable name
--device	CUDA0 / Vulkan0 / ROCm0 / MTL0 / CPU (not cuda / vulkan, see §5)
--port	Prefer a high 18xxx port: 8080/8081 are permanently held by Steam's steamwebhelper
--no-fa	Disable flash attention (troubleshooting)
--codec-chunk-dur	Codec decode chunk duration for wav responses (seconds, default 24)

Poll GET /health until it returns 200. The first load takes on the order of 10 s (1.7B Q8) — do not declare failure after a 3-second timeout.

---

3) HTTP contract (measured on v1.0.0)

3.1 GET /health → {"status":"ok"}

Use it for readiness; do not substitute "the port accepts connections" (the service listens while loading weights).

3.2 GET /v1/models → used to claim the port

{"object":"list","data":[{"id":"Qwen3-TTS-1.7B","object":"model","owned_by":"local"}]}

id is the --alias you passed at startup. Other plugins' services may be running on the same host, so you must confirm "the service on this port is the one I started" — otherwise you will reuse someone else's.

3.3 POST /v1/audio/speech

{
  "model": "Qwen3-TTS-1.7B",
  "input": "text to synthesize",
  "voice": "voice name or empty",
  "response_format": "wav",
  "max_new_tokens": 400,
  "instructions": "(voicedesign only) a young woman, gentle"
}

Field	Description
input	Text to synthesize
voice	base variant: a registered clone-voice name (a file path is rejected with 400 unknown voice); customvoice variant: a built-in speaker name; empty = the server default
response_format	wav (whole clip at once) / pcm (24 kHz mono s16le raw, combine with stream=true to emit while generating)
max_new_tokens	Strongly recommended as an upper bound: at a 12 Hz code rate, tokens ≈ 2.5–3.5 × character count. Without a bound, ICL cloning with mismatched reference text/audio degrades into never stopping (measured: 7 characters produced 164 seconds of audio), and the synthesis worker is single-threaded → the whole read-aloud pipeline stalls
instructions	voicedesign variant only: describe the voice in words

Response: 200 plus audio bytes (wav is RIFF, pcm is raw s16le). Errors are 4xx/5xx + JSON.

3.4 Clone-voice registry (in memory only)

Method	Path	Description
GET	/v1/audio/voices	→ {"voices":[{"name":"x","kind":"registered"}]}
POST	/v1/audio/voices	body {"name":"x","ref_text":"text of the reference audio","wav_b64":"<base64 WAV>"}
DELETE	/v1/audio/voices/<name>	Delete

Note: GET /v1/voices does not exist (404) — do not copy that from older OpenAI docs.

⚠️ The tts-server voice registry does not persist: a process restart clears it. Your plugin must persist it itself and re-register after the service becomes ready (see Local-service engine contract §7).

---

4) Model variants (pick one)

From the same repo Serveurperso/Qwen3-TTS-GGUF:

Variant	Talker file	Capability
base	qwen-talker-1.7b-base-Q4_K_M.gguf	Voice cloning from reference audio (no built-in speakers)
customvoice	qwen-talker-1.7b-customvoice-Q4_K_M.gguf	Built-in preset voices, picked through voice
voicedesign	qwen-talker-1.7b-voicedesign-Q4_K_M.gguf	Generate a voice from a text description

All three share qwen-tokenizer-12hz-Q8_0.gguf. Declare each variant as its own model_artifacts entry and put the currently effective one first in model_artifacts() (see Model artifacts).

---

5) Compute device: --device takes a device name, not cuda / vulkan

def _device_arg(device: str) -> str:
    dev = (device or "vulkan").strip().lower()
    if dev == "cpu":
        return "CPU"
    if dev in ("cuda", "cuda0", "cuda13"):
        return "CUDA0"
    if dev in ("rocm", "hip", "rocm0"):
        return "ROCm0"
    if dev in ("metal", "mtl", "mtl0"):
        return "MTL0"
    return "Vulkan0"

⚠️ The bundled build is Vulkan-only; there is no separate CUDA build

tts-server.exe --help mentions CUDA0, but that is only generic help text. Measured on the binary's symbols: 461 embedded ggml_vk symbols, 0 ggml_cuda / cuda_backend. Passing --device CUDA0 answers directly:

--device CUDA0 not found. Available: Vulkan0 Vulkan1 CPU

So expose only vulkan / cpu in your plugin config; never put cuda in choices — the user would pick it and the service would not start.

Why not force CUDA: Vulkan runs this model at RTF 0.18–0.26 (see §6), still 4–5× away from the realtime line of 1.0, so the CUDA gain is not observable in realtime read-aloud. Real CUDA would require rebuilding qwentts.cpp with buildcuda.

Dynamically loading ggml-cuda.dll from nv-pack via GGML_BACKEND_PATH was measured not to work: it loads, initialises a CUDA device, but crashes right after [Pipeline] Ready once the weights are loaded (ggml ABI mismatch: tts-server is 0.26.x, nv-pack is 0.27.x).

---

6) Performance reference (RTX 4070 Ti, 1.7B Q4_K_M talker + Q8 codec)

Scenario	Result
Cold start to /health 200	~10 s
Non-streaming wav, 4 s of audio	1.0–1.3 s (RTF 0.26–0.31)
pcm streaming, first chunk	80–130 ms
pcm streaming, 5–6 s of audio	1.0–1.8 s (RTF 0.21–0.30)

With is_streaming_capable = True the host plays "900 ms pre-buffer, then one chunk every 0.24 s", starting earlier than waiting for the whole wav.

---

Reference implementation

Directory	Description
extensions/workshop_common_qwen3_tts/	tts-server + clone-voice persistence + three variants (complete example)

---

Related

• Host runtimes requires_runtime
• Local-service engine contract
• Model artifacts
• tts type overview