Local-service engine contract (mandatory — ASR / MT / TTS)

If your plugin starts a local inference service (llama-server / tts-server / your own HTTP child process), you must follow the contract below. Breaking it turns straight into user-visible bugs (typical complaints: "I didn't enable any track — why is it loading models just because I switched to the settings page?", "The port is taken, the service won't start").

0) Strongly recommended: subclass the standard base classes (zero boilerplate)

Since TransBox v0.2.0 the framework ships base classes in transbox.plugins built specifically for local model engines:

BaseLocalServicePlugin: generic local-service base
BaseLocalASRProvider: local speech-recognition engine base
BaseLocalMTProvider: local machine-translation engine base
BaseLocalTTSProvider: local text-to-speech engine base

Subclass them and you write no device-switching, thread-restart or lifecycle boilerplate at all — you get:

Automatic lazy startup: initialize() never starts the service; ensure_loaded() or the inference call starts it on demand.
Device normalisation for free: cuda, cuda13, vulkan, rocm, cpu, auto are mapped and GPU-probed automatically.
Hot device switching with events: if the service is running when the config changes, it emits device.switching (the UI greys out and spins), restarts the service on a background thread, and emits device.ready when done; if the service is not running it only records the setting and uses no VRAM.
Hot reload of active tracks: if the model is currently serving an enabled track, the host hot-reloads that pipeline — seamless for the user.
Leftover port cleanup: self.kill_port_conflict(port) terminates orphan processes.
VRAM release and graceful exit: the host calls stop_service() on shutdown(), releasing the child process and its VRAM.

Example: a minimal third-party ASR plugin

python

from typing import Optional, Callable
import numpy as np
from transbox.plugins import BaseLocalASRProvider, Transcript
from transbox.plugins.manifest import ModManifest

class MyCustomLocalASR(BaseLocalASRProvider):
    def __init__(self, manifest: ModManifest) -> None:
        super().__init__(manifest)
        self._port = 18090  # pick a port that will not clash

    # 1. Start the child process (only called when inference is actually needed)
    def start_service(self, device: Optional[str] = None) -> None:
        backend = self.normalize_device(device)  # normalised to cuda13 / vulkan / cpu
        self.kill_port_conflict(self._port)      # clear orphan port holders
        # start your child process, keep it in self._proc
        self._proc = ...

    # 2. Stop the child process and release VRAM
    def stop_service(self) -> None:
        # super().stop_service() terminates/kills self._proc for you
        super().stop_service()

    # 3. Inference entry point
    def transcribe(
        self,
        audio: np.ndarray,
        sample_rate: int = 16000,
        language: str = "auto",
        **kwargs,
    ) -> Transcript:
        self.ensure_loaded()  # make sure the service is ready
        # send the request and return the result
        return Transcript(text="...")

Example 2: third-party local TTS plugin (BaseLocalTTSProvider + audio.cpp)

python

from typing import Optional, Tuple
from pathlib import Path
import io, wave, requests
import numpy as np
from transbox.plugins import BaseLocalTTSProvider
from transbox.plugins.manifest import ModManifest


class MyCustomLocalTTS(BaseLocalTTSProvider):
    def __init__(self, manifest: ModManifest) -> None:
        super().__init__(manifest)
        self._port = 18320

    # 1. Start the child process (only called when inference is actually needed)
    def start_service(self, device: Optional[str] = None) -> None:
        backend = self.normalize_device(device)  # cuda13 / vulkan / cpu
        self.kill_port_conflict(self._port)
        exe = Path(self.host.runtime_dir("audiocpp_runtime")) / "audiocpp_server.exe"
        model_path = Path(self.host.model_dir("my_tts_model")) / "model.gguf"

        # start the audio.cpp service (OpenAI-compatible /v1/audio/speech included)
        cmd = [str(exe), "--model", str(model_path), "--port", str(self._port), "--backend", backend]
        # self._proc = subprocess.Popen(...)

    # 2. Synthesis entry point
    def synthesize(
        self,
        text: str,
        voice_id: str = "",
        voice: Optional[str] = None,
        language: str = "zh",
        speed: float = 1.0,
    ) -> Tuple[np.ndarray, int]:
        self.ensure_loaded()
        resp = requests.post(
            f"http://127.0.0.1:{self._port}/v1/audio/speech",
            json={"input": text, "voice": voice or voice_id, "speed": speed, "response_format": "wav"},
            timeout=30.0,
        )
        resp.raise_for_status()
        with io.BytesIO(resp.content) as buf, wave.open(buf, "rb") as wf:
            sr = wf.getframerate()
            samples = np.frombuffer(wf.readframes(wf.getnframes()), dtype=np.int16).astype(np.float32) / 32767.0
        return samples, sr

1) Lazy startup: `initialize()` must not start the service

Hook	What it should do	What it must not do
`initialize(config)`	Record config, resolve the model directory, check whether the weights are on disk, report status	Start a child process / load weights / block until ready
`ensure_loaded()`	Start the service and wait for readiness when it is actually needed (the host calls it when a track is enabled, the pipeline starts, or Test Connection is clicked)	Nothing forbidden (blocking until ready is fine: the host already calls it from a background thread)
`check_ready()`	Decide "can it be used": runtime present + weights complete (+ key configured)	Treating "service not started" as not ready (the host would then refuse to enable the track)
`list_voices()` and other read-only methods	Return the existing cache / an empty list	Start the service

Why: the host's read-only queries (settings-page status, capability probing, voice list, model card checks) never create an instance; but as soon as any path creates one (a prerequisite plugin / the host resolving an engine / a plugin action), initialize() runs. If it starts the service on the side, a user who "just opens the settings page or switches to that engine tab" waits seconds for nothing and burns several GB of VRAM.

Every other entry point must cover itself: make sure the service is ready at the entry of transcribe() / translate() / synthesize() / stream_tts() / register_voice() / remove_voice() (zero cost when it is already ready), so everything still works when the host does not call ensure_loaded() first.

python

def initialize(self, config=None) -> None:
    super().initialize(config)
    self._resolve_model_dir()      # checks / bookkeeping only
    self._ensure_running = False

def ensure_loaded(self, progress_callback=None) -> None:
    """Called by the host when the engine is really about to be used; an external plugin's initialize runs on a background thread, so wait for it to settle first."""
    self.wait_async_init()
    self._start_server_if_needed()

def _start_server_if_needed(self) -> None:
    if self._health_ok():          # idempotent: already ready, just return
        return
    self._ensure_server(...)

def check_ready(self) -> tuple[bool, str, str]:
    if not self._weights_present():
        return False, "model_not_ready", "模型未就绪，请在设置页下载"
    return True, "", ""            # a stopped service still counts as "usable"

def synthesize(self, text, **kw):
    self._start_server_if_needed()  # cover the request entry point
    ...

2) Service port: configurable + automatic fallback when taken

json

{
  "key": "port",
  "label": "服务端口",
  "type": "number",
  "default": 18080,
  "description": "本地服务监听端口。默认取高位不常用端口段；被别的程序占用时会自动顺延。"
}

Do not default to 8080 / 8081 / 8082: in practice they are often taken by Steam's steamwebhelper and various dev tools, and users see "the service won't start".
Suggested resolution order: the configured value → if another program holds it, fall back within your own candidate range (e.g. 18080~18084) → reuse a leftover service of this plugin that is already listening.
Give each plugin a non-overlapping port range (e.g. ASR 18080~18084 / MT 18090~18094 / TTS 18100~18104) so plugins never fight over ports.
Non-numeric / out of range (1024~65535) → fall back to the default and write a log line (never silently use an illegal value).
Probe the port before starting the service; when an outside program holds it, return a clear error (including a netstat -ano | findstr :<port> hint) instead of failing silently.
Changing the port is a "service behaviour change": restart only while the service is running; when it is not, just record it so the next start picks it up.

3) Compute device (`device`)

json

{ "key": "device", "label": "计算设备", "type": "select",
  "choices": ["auto", "vulkan", "cpu"], "default": "auto" }

choices may only declare backends your runtime really supports. (A llama.cpp runtime with Vulkan+CPU only must not offer cuda, otherwise users pick it and crash. Standard identifiers and host auto-labelling: config_schema § Compute device.)
Device changes → restart only while the service is running; when it is not, just record it and emit an event so the next start picks it up.

4) Status events (the host surfaces them as hints / toasts)

python

self.emit_event("device.switching", {"device": dev, "reason": f"计算设备变更为 {dev}"})
# …restart / wait for ready…
self.emit_event("device.ready", {"device": dev})

The host prefers showing reason (falling back to "Switching: compute device changed to X"), and meanwhile marks the settings-page device row as "switching" and disables the dropdown.

5) Prerequisite runtime: locate it with `host.plugin_dir()`, never hard-code directory names

A workshop-installed plugin folder is named after the plugin id (data/workshop/<id>), not after the folder inside the zip. To locate a sibling plugin (such as the llama runtime) you must use:

python

from pathlib import Path

dep_dir = self.host.plugin_dir("workshop.common_llama")   # returns "" when not installed
exe = Path(dep_dir) / "llama" / "llama-server.exe" if dep_dir else None

and declare requires_plugins in mod.json so the host blocks early and guides the installation. Never ship large native runtimes inside the package — use requires_runtime.

See Prerequisite plugins / Host runtimes.

6) Child-process lifecycle

shutdown() must kill the child process (the host calls it on exit, when "Release VRAM" is clicked, when the plugin is disabled and on config changes), otherwise it leaves an orphan process holding a port and VRAM.

python

def shutdown(self) -> None:
    self._stop_server()          # terminate → wait(timeout) → kill as a fallback
    super().shutdown()

Leave one log line when stopping (已停止 llama-server PID=…): the host's "Release VRAM" reports honestly whether you actually stopped.

Ownership check: remember the PID you started; when the handle is invalid (killed externally), verify ownership by port before cleaning up leftovers, and never adopt an outside leftover service as "your own" (otherwise the UI says cpu while the process still runs with vulkan parameters).

Concurrency safety: use one RLock to serialise the "async initialize thread" and the "host thread" when they touch the service. Concurrent access can set self._proc to None, which makes polling dereference crash and bubble up into the host's Bridge() construction (reproduced in practice).

7) State that lives in the service must be persisted yourself

A local inference service's "registry" often lives in memory only (gone after a restart). If your service supports registering voices / custom entries at runtime, store the original assets (e.g. reference audio + ref_text) in the plugin's private data directory, and re-register them automatically once the service is ready.

Otherwise users complain "I have to configure it again after every restart" (measured on Qwen3-TTS's tts-server: register, restart → GET /v1/audio/voices returns an empty list).

Read-only methods (list_voices() etc.) should return what is stored locally while the service is not running, not start the service.

Self-test checklist

[ ] Open the settings page / switch to this plugin's tab → no llama-server / tts-server in Task Manager
[ ] The service starts only after a track is enabled; when the UI says "Ready" the service really listens on its port
[ ] Change "Service port" to 1808x / change "Compute device" to cpu → the service restarts with the new parameters and the UI shows a "switching" hint
[ ] Change port/device while the service is not running → no process is spawned
[ ] Rename the plugin directory / remove a prerequisite plugin → the card says "Requires plugins" with a plain-language reason
[ ] Click "Release VRAM" → the log shows 已停止 <server> PID=… and the UI reports that this engine stopped
[ ] After quitting the app no orphan process holds a port / VRAM

Reference implementations

Directory	Description
`extensions/workshop_common_qwen3_asr/`	llama.cpp service (lazy + port fallback)
`extensions/workshop_common_hy_mt/`	llama.cpp service
`extensions/workshop_common_qwen3_tts/`	tts-server + voice persistence + variants

Local-service engine contract (mandatory — ASR / MT / TTS) ​

0) Strongly recommended: subclass the standard base classes (zero boilerplate) ​

Example: a minimal third-party ASR plugin ​

Example 2: third-party local TTS plugin (BaseLocalTTSProvider + audio.cpp) ​

1) Lazy startup: initialize() must not start the service ​

2) Service port: configurable + automatic fallback when taken ​

3) Compute device (device) ​

4) Status events (the host surfaces them as hints / toasts) ​

5) Prerequisite runtime: locate it with host.plugin_dir(), never hard-code directory names ​

6) Child-process lifecycle ​

7) State that lives in the service must be persisted yourself ​

Self-test checklist ​

Reference implementations ​

Related ​

Local-service engine contract (mandatory — ASR / MT / TTS)

0) Strongly recommended: subclass the standard base classes (zero boilerplate)

Example: a minimal third-party ASR plugin

Example 2: third-party local TTS plugin (BaseLocalTTSProvider + audio.cpp)

1) Lazy startup: `initialize()` must not start the service

2) Service port: configurable + automatic fallback when taken

3) Compute device (`device`)

4) Status events (the host surfaces them as hints / toasts)

5) Prerequisite runtime: locate it with `host.plugin_dir()`, never hard-code directory names

6) Child-process lifecycle

7) State that lives in the service must be persisted yourself

Self-test checklist

Reference implementations

Related