tts — text-to-speech

Wire up a cloud TTS or a local synthesis engine. Engines are currently mutually exclusive. Supports voice lists / voice cloning / streaming / speed.

Declaration

json

{
  "id": "myteam.tts.my_voice",
  "name": "My Voice TTS",
  "version": "1.0.0",
  "target_app_version": "0.2",
  "type": "tts",
  "entrypoint": "my_tts.py:MyTTSProvider",
  "tags": ["cloud"],
  "requires_hf": false,
  "capabilities": {
    "voices": true,
    "voice_clone": true,
    "voice_remove": true,
    "speed": false
  },
  "config_schema": [
    { "key": "api_key", "label": "API Key", "type": "password", "is_secret": true }
  ]
}

What capabilities means, and the two equivalent ways of writing it ("declarative" vs class attributes), are covered in Capabilities & read-only contract.

Minimal implementation

python

import numpy as np
from transbox.plugins.base import BaseTTSProvider


class CustomTTSProvider(BaseTTSProvider):
    supports_voice_clone = True   # True → the settings page shows the "Register cloned voice" panel (including in-app recording)
    supports_speed = False        # False → the host does pitch-preserving time-stretching for you; you do nothing

    @property
    def sample_rate(self) -> int:
        return 24000

    @property
    def is_streaming_capable(self) -> bool:
        return True

    def synthesize(self, text, voice_id="", voice=None, language="zh"):
        # returns (float32 mono PCM, sample rate)
        return np.zeros(24000, dtype=np.float32), self.sample_rate

    def stream_tts(self, text, voice_id="", voice=None, language="zh"):
        """Optional: yield chunk by chunk (the host plays them chunk by chunk using its prebuffer, so playback starts earlier).
        If not implemented, the host degrades to whole-clip synthesize."""
        yield self.synthesize(text, voice_id, voice, language)[0]

Parameter conventions:

voice takes priority over voice_id (the host may pass only one of them).
Return (float32 mono PCM, sample rate).
Voice config key: the host writes the voice the user selected into the voice key of the plugin config (when split per language there are also voice_en / voice_other, falling back to voice when absent). In your implementation just read self._config.get("voice").

Voice list

python

def list_voices(self) -> list[dict]:
    return [
        {"id": "vivian", "name": "薇薇安", "lang": "zh", "kind": "speaker"},
        {"id": "my-clone", "name": "我的音色", "lang": "zh", "kind": "registered"},
    ]

kind="speaker" = a voice built into the model; kind="registered" = a cloned voice registered by the user.
Only voices with kind="registered" show a "Delete voice" action.
Return only voices that are really available (e.g. the base variant has no built-in speaker, so return only the registered cloned voices).
Read-only method: when the service is not running return the cached / locally saved voice list, or an empty list; never start the service just to list voices (see Capabilities).

Voice cloning

python

def register_voice(self, voice_name, audio_samples, reference_text="", sample_rate=24000) -> bool:
    """audio_samples: float32 mono numpy, 1~30 seconds."""
    ...

def remove_voice(self, voice_name: str) -> bool:
    ...

The reference audio is provided by the host: the user picks a file, or records directly on the settings page (the host handles microphone capture, writing the WAV and reading it back as float32). Your plugin does not need to touch microphones, file dialogs or audio format conversion.
You are only responsible for transcoding / uploading / persisting, and returning True/False.

Persistence of cloned voices (mandatory)

If your voices are registered only in server-side memory (lost when the process restarts), you must persist them yourself:

Save the reference audio and ref_text into the plugin's private data directory (self.get_data_dir());
Re-register automatically once the service is ready: compare against the server's GET /v1/audio/voices and register whatever is missing; run this only once per service instance (re-run after the service gets a new instance / restarts / changes device / changes port);
remove_voice() deletes the local copy at the same time;
list_voices() returns locally saved voices (kind="registered") when the service is not running, and must not start the service.

Otherwise users have to re-register their voices after every app restart. Measured reference implementation: extensions/workshop_common_qwen3_tts (_persist_voice / _restore_voices / _read_voice_index).

Speed

Case	Host behaviour	What you do
`supports_speed = False` (default)	Applies pitch-preserving time-stretching after synthesis (host built-in DSP)	Nothing
`supports_speed = True`	Passes `speed=` (1.0 = original speed, range 0.5~2.0) to you	Implement it inside your model

Streaming × speed: when speed != 1.0 and the plugin does not declare supports_speed, the host will not call stream_tts (time-stretching is only practical on a whole clip).

Multi-capability models (variants)

When one model repository holds several mutually exclusive variants (e.g. base for voice cloning / customvoice for built-in voices / voicedesign for text-described voices), declare them as multiple model_artifacts and, in model_artifacts(), sort the currently effective one to first place — the host model card only shows/downloads the first artifact with the same category, so users are neither forced to download every ~1.1 GB variant nor falsely told the model is "not ready".

python

def model_artifacts(self) -> list[dict]:
    arts = [...]                       # all your variants
    active = self._resolve_variant()   # pick the effective one from config / what is downloaded
    arts.sort(key=lambda a: 0 if a["id"] == active else 1)
    return arts

See Model artifacts § Multi-capability models.

Local-service TTS (recommended: audio.cpp + `BaseLocalTTSProvider`)

If you start a local service yourself, or use GGUF speech models, build it on the host runtime plus the standard base classes:

Declare the host runtime dependency (mod.json):
json
```
{
  "requires_runtime": [
    { "id": "audiocpp_runtime", "reason": "audio.cpp local inference runtime (CUDA 13 / Vulkan / CPU)" }
  ]
}
```
For Qwen3-TTS use the other bundled runtime: tts_server_runtime (tts-server.exe, OpenAI-compatible /v1/audio/speech + a clone-voice registry). Startup flags and the HTTP contract live in Local TTS service.
Subclass BaseLocalTTSProvider:
- Never start the service in initialize(); start it lazily from ensure_loaded() or the synthesis entry point;
- You inherit GPU/CPU hot-switching (CUDA 13 / Vulkan / CPU), port-conflict cleanup, and full child-process / VRAM reclamation via a Windows JobObject;
- Full reference code: Local-service engine contract § Example 2.
Lightweight CPU-only option:
- If the model is a standard VITS/ONNX graph (e.g. MeloTTS), load it in-process through sherpa-onnx — no external service process needed.

Reference implementations

Directory	Type
`extensions/community_cloud_mimo_tts/`	Cloud API
`extensions/workshop_openai_tts/`	OpenAI-compatible (kokoro-fastapi / GPT-SoVITS / fish-speech…)
`extensions/workshop_common_qwen3_tts/`	Local service type + voice cloning + variants (complete example)

tts — text-to-speech ​

Declaration ​

Minimal implementation ​

Voice list ​

Voice cloning ​

Persistence of cloned voices (mandatory) ​

Speed ​

Multi-capability models (variants) ​

Local-service TTS (recommended: audio.cpp + BaseLocalTTSProvider) ​

Reference implementations ​

Related ​

tts — text-to-speech

Declaration

Minimal implementation

Voice list

Voice cloning

Persistence of cloned voices (mandatory)

Speed

Multi-capability models (variants)

Local-service TTS (recommended: audio.cpp + `BaseLocalTTSProvider`)

Reference implementations

Related