GigaSTT Workbook

Scenario-driven recipes for gigastt, the local Russian speech-to-text server powered by GigaAM v3. Each chapter follows the same shape: scenario → prerequisites → recipe → verifying the result → common pitfalls → links.

This book is a cookbook, not a reference. The canonical references stay in docs/ — the workbook links to them instead of duplicating them.

Documented against gigastt 2.14.x. Prefer resolving the latest release tag in install scripts (gh api …/releases/latest) rather than hard-coding older minors.

I want to…

Chapters

1. Getting started — install (macOS/Linux/Windows/Docker/air-gap), first transcription. Beginner · ~5–15 min
2. CLI and batch processing — CLI, batch, and watch-mode recipes for audio files. Beginner · ~15–30 min
3. Telephony & VoIP — G.711/G.722/Opus, PBX recordings, stereo split, and diarization. Intermediate · ~20 min
4. Streaming over WebSocket — real-time transcription over WebSocket (partials, VAD endpointing, session caps). Intermediate · ~30 min
5. Desktop & embedded — Swift/SPM, sidecar, Electron, UniFFI. Intermediate · ~30–60 min
6. Deployment & ops — production deployment, monitoring, upgrades, admin reload. Ops · ~45 min
7. Models and backends — model variants, quantization, execution providers, punctuation/ITN, hotwords. Intermediate · ~20 min

Appendices

• A — Error codes — REST/WS/close code jump table
• B — Offline checklist — air-gapped operator list

The Russian version mirrors this book chapter by chapter.

Documentation map

Full inventory of the documentation in this repository: what each file is and where it lives.

References (canonical — never duplicated in the workbook)

Guides (current)

Historical (archived)

Completed design/plan documents kept for archaeology in docs/archive/:

Rules for contributors

• The workbook holds recipes; docs/api.md, docs/cli.md, and the AsyncAPI/OpenAPI schemas remain the canonical references. Link to them — do not copy their content.
• Every command and example in a chapter must be verified before merge.
• Inside the book (chapter ↔ chapter, chapter ↔ intro) use relative .md links — they work both on GitHub and in the rendered book. Links from the book to repository files (docs/, crates/, …) must be absolute GitHub URLs — relative ones 404 on the published site. No mdBook-specific templating.
• New chapters follow the _template.md structure.
• English is canonical. The Russian book (docs/workbook/ru/) mirrors this one with identical file names; both versions are updated in the same PR.
• When a feature changes the documented surface (CLI flags, error codes, audio formats), update the chapter, the book SUMMARY.md, and the canonical references in the same PR — and keep the docs-drift gate green: python3 scripts/check-docs-drift.py (advisory in CI; it compares CLI flags, WS error codes, audio formats, mdBook TOCs, EN/RU parity, relative links, OpenAPI/SECURITY/crate pins, and workbook version currency + required recipe tokens against the code).

Getting started

Scenario

You have never run gigastt and want a working local transcription in about five minutes: install the binary, download the GigaAM v3 model, transcribe a first audio file — on macOS, Linux, or in Docker. This chapter is the whole path; you should not need any other document to get here.

Prerequisites

• Disk: ~1.5 GB free (model + tools). The lean --prequantized path needs only ~250 MB during setup.
• RAM: ~800 MB free at the default --pool-size 2 (~400 MB per session).
• Network (unless you follow the air-gapped recipe): reach either huggingface.co (full model) or github.com (pre-quantized bundle).
• An audio file to transcribe — WAV, M4A, MP3, OGG, or FLAC. Any short recording of Russian speech works.
• Only for cargo install (build from source): Rust 1.88+ and protoc on PATH (brew install protobuf / apt install protobuf-compiler).

Pick one recipe below — macOS, Linux, Windows, Docker, or air-gapped — then read Choosing the recognition head and The expensive first run once.

Recipe: macOS (Homebrew)

Homebrew is the fastest path on Apple Silicon (the tap ships a CoreML-enabled binary). On an Intel Mac use cargo install gigastt instead — see the Linux recipe for the protoc prerequisite.

brew tap ekhodzitsky/gigastt https://github.com/ekhodzitsky/gigastt
brew install gigastt

# Fetch the model (~850 MB FP32 from HuggingFace, then a one-time ~2 min
# INT8 quantization pass — see "The expensive first run" below):
gigastt download

# Transcribe your first file:
gigastt transcribe recording.wav

Verify: the last command prints the recognized text on stdout, e.g.

$ gigastt transcribe recording.wav
Привет, как дела?

and ls ~/.gigastt/models/ shows v3_rnnt_encoder_int8.onnx, v3_rnnt_decoder.onnx, v3_rnnt_joint.onnx, and v3_vocab.txt.

Recipe: Linux (prebuilt binary or cargo)

Option A — prebuilt binary (no Rust toolchain, no protoc). Each release publishes tarballs for x86_64-unknown-linux-gnu and aarch64-unknown-linux-gnu:

# Resolve the latest release tag (or set TAG=v2.14.1 by hand):
TAG=$(curl -fsSL https://api.github.com/repos/ekhodzitsky/gigastt/releases/latest \
      | sed -n 's/.*"tag_name": *"\([^"]*\)".*/\1/p')
VER=${TAG#v}

curl -fLO "https://github.com/ekhodzitsky/gigastt/releases/download/${TAG}/gigastt-${VER}-x86_64-unknown-linux-gnu.tar.gz"
curl -fLO "https://github.com/ekhodzitsky/gigastt/releases/download/${TAG}/gigastt-${VER}-x86_64-unknown-linux-gnu.tar.gz.sha256"
sha256sum -c "gigastt-${VER}-x86_64-unknown-linux-gnu.tar.gz.sha256"

tar xf "gigastt-${VER}-x86_64-unknown-linux-gnu.tar.gz"
sudo install -m 0755 gigastt /usr/local/bin/gigastt

(On ARM64 replace x86_64-unknown-linux-gnu with aarch64-unknown-linux-gnu. Homebrew on Linux x86_64 — brew install gigastt after the tap from the macOS recipe — works too.)

Option B — cargo (any platform, needs Rust 1.88+ and protoc):

sudo apt install protobuf-compiler   # Debian/Ubuntu; skip if protoc exists
cargo install gigastt

Then fetch the model — the lean way, a ~225 MB pre-quantized INT8 bundle from the pinned GitHub Release (no ~850 MB FP32 download, no ~2-minute on-device quantization; also handy when HuggingFace is unreachable but GitHub is not):

gigastt download --prequantized
gigastt transcribe recording.wav

Verify: gigastt transcribe recording.wav prints the recognized text on stdout, and ls ~/.gigastt/models/ shows the v3_rnnt_* model files.

Recipe: Windows (prebuilt binary)

Every release publishes x86_64-pc-windows-msvc tarballs (CPU). PowerShell 5.1+ / Windows 10+ includes tar and curl:

$rel = Invoke-RestMethod https://api.github.com/repos/ekhodzitsky/gigastt/releases/latest
$TAG = $rel.tag_name          # e.g. v2.14.1
$VER = $TAG.TrimStart('v')
$asset = "gigastt-$VER-x86_64-pc-windows-msvc.tar.gz"
$base = "https://github.com/ekhodzitsky/gigastt/releases/download/$TAG"

Invoke-WebRequest "$base/$asset" -OutFile $asset
Invoke-WebRequest "$base/$asset.sha256" -OutFile "$asset.sha256"
# Optional integrity check (hash file is "HASH  filename"):
$expected = (Get-Content "$asset.sha256").Split()[0]
$actual = (Get-FileHash $asset -Algorithm SHA256).Hash.ToLower()
if ($actual -ne $expected.ToLower()) { throw "SHA-256 mismatch" }

tar xf $asset
# Put gigastt.exe on PATH, or call it by full path:
.\gigastt.exe download --prequantized
.\gigastt.exe transcribe recording.wav

Model directory on Windows defaults under the user profile (%USERPROFILE%\.gigastt\models\). First serve still binds loopback only (127.0.0.1:9876) until you pass --bind-all.

Verify: .\gigastt.exe transcribe recording.wav prints text; health after .\gigastt.exe serve answers at http://127.0.0.1:9876/health.

Recipe: Docker

Prebuilt multi-arch images (amd64 + arm64) are published to GHCR for every release; -cuda tags carry the CUDA variant:

docker pull ghcr.io/ekhodzitsky/gigastt:latest   # pin :<version> in production

docker run -d --name gigastt \
  -p 127.0.0.1:9876:9876 \
  -v gigastt-models:/home/gigastt/.gigastt/models \
  ghcr.io/ekhodzitsky/gigastt:latest

The named volume keeps the model across container restarts; without it the container re-downloads ~850 MB on every recreation. On first start the container downloads the model and quantizes it — the port binds immediately, but inference is only up when /ready turns green:

# Wait until the model is loaded (503 while initializing):
until curl -sf http://127.0.0.1:9876/ready > /dev/null; do sleep 5; done

curl http://127.0.0.1:9876/health

Then transcribe a file from the host (the file path is on the host — curl reads it, not the container):

curl -F file=@recording.wav http://127.0.0.1:9876/v1/transcribe

Verify: /health returns

{"status":"ok","model":"gigaam-v3-rnnt","variant":"rnnt","version":"2.14.1","punctuation":true,"itn":true}

(the version field reflects the image you pulled), and the POST returns a JSON transcript:

{"text":"Привет, как дела?","words":[{"word":"привет","start":0.0,"end":0.4,"confidence":0.99}],"duration":1.2}

Recipe: air-gapped (offline bundle)

For hosts with no internet access, every release publishes a self-contained offline bundle per Linux target — binary + pre-quantized INT8 rnnt model + punctuation model + systemd unit + installer — plus two Debian packages with the same content. Download them on a connected machine, carry them over, install.

Tarball flow (any distro):

# On a connected machine (see the Linux recipe for resolving TAG/VER):
curl -fLO "https://github.com/ekhodzitsky/gigastt/releases/download/${TAG}/gigastt-${VER}-offline-x86_64-unknown-linux-gnu.tar.gz"
curl -fLO "https://github.com/ekhodzitsky/gigastt/releases/download/${TAG}/gigastt-${VER}-offline-x86_64-unknown-linux-gnu.tar.gz.sha256"
sha256sum -c "gigastt-${VER}-offline-x86_64-unknown-linux-gnu.tar.gz.sha256"

# On the target machine:
tar xf "gigastt-${VER}-offline-x86_64-unknown-linux-gnu.tar.gz"
cd "gigastt-${VER}-offline-x86_64-unknown-linux-gnu"
sudo ./install.sh                      # verifies SHA256SUMS, installs binary + model + unit
sudo systemctl enable --now gigastt

Debian flow: install gigastt_<ver>_amd64.deb (binary + unit) together with gigastt-model-int8_<ver>_all.deb (the same model set), then sudo systemctl enable --now gigastt.

The bundle deliberately omits optional pieces — speaker diarization and the e2e_rnnt / ml_ctc heads. The installed unit runs with GIGASTT_OFFLINE=1, so a missing optional model is a fast, instructive error naming the exact path to fill (fetch it on a connected machine with gigastt download and copy the file over), never a network timeout. The full contents list and signature verification are in packaging/offline/README-OFFLINE.md.

Verify: curl http://127.0.0.1:9876/health returns {"status":"ok",...} with "model":"gigaam-v3-rnnt", and gigastt transcribe sample.wav --model-dir /usr/share/gigastt/models prints text (the flag is only needed when running uninstalled — the systemd unit already points at the installed model).

Choosing the recognition head

gigastt ships four recognition heads; --model-variant picks one at download / serve / transcribe time. When you omit the flag, an existing model directory is used as-is (auto-detect), and a fresh install defaults to rnnt.

The ml_ctc* heads download pre-quantized INT8 directly, so there is no quantization step for them. Switching heads after install:

gigastt download --model-variant e2e_rnnt   # fetch another head
gigastt serve --model-variant e2e_rnnt      # and serve it explicitly

WER/RTF numbers per head are in docs/benchmarks.md; the deeper model/backend tour is in Models and backends.

The expensive first run

The very first gigastt download (or first gigastt serve, which auto-downloads a missing model) does two one-time things:

1. Downloads ~850 MB of FP32 ONNX files from HuggingFace (SHA-256-verified, staged to .partial, atomically renamed).
2. Quantizes the encoder to INT8 (~2 minutes, one-time), producing the ~225 MB encoder the engine actually loads. Later runs reuse it.

Three levers change what you pay:

• gigastt download --prequantized — the recommended shortcut: fetch the ~225 MB pre-quantized INT8 bundle from the pinned GitHub Release. No FP32 download, no local quantization, no protoc. Note it pulls from github.com, not huggingface.co — useful when one of the two is blocked.
• gigastt download --skip-quantize (or GIGASTT_SKIP_QUANTIZE=1 on serve) — keep the FP32 encoder and skip quantization. The engine then loads FP32: slower inference and ~4× the model RAM. Only for debugging.
• Nothing — just let the first serve do it. The port binds immediately; /health answers 200 with "model":"loading" and /ready returns 503 {"reason":"initializing"} until the model is usable, so clients should gate on /ready, never on the process being alive.

Verifying the result

End-to-end checklist that works after any of the recipes above:

# 1. Model files are in place:
ls ~/.gigastt/models/
#   v3_rnnt_encoder_int8.onnx  v3_rnnt_decoder.onnx  v3_rnnt_joint.onnx  v3_vocab.txt  ...

# 2. Offline transcription works (no server needed):
gigastt transcribe recording.wav
#   → prints the recognized text on stdout

# 3. The server comes up and reports the loaded head:
gigastt serve &                      # Ctrl-C to stop; default http://127.0.0.1:9876
curl http://127.0.0.1:9876/ready     # 200 once the model is loaded
curl http://127.0.0.1:9876/health
#   {"status":"ok","model":"gigaam-v3-rnnt","variant":"rnnt","version":"...","punctuation":true,"itn":true}

# 4. REST transcription works:
curl -F file=@recording.wav http://127.0.0.1:9876/v1/transcribe
#   → {"text":"...","words":[...],"duration":N}

Common pitfalls

• protoc not found during cargo install or a source build — install the Protocol Buffers compiler (brew install protobuf / apt install protobuf-compiler), or skip the toolchain entirely with the prebuilt binary / Homebrew.
• First serve sits there for minutes — that is the one-time model download + INT8 quantization, not a hang: /health returns {"model":"loading"} meanwhile. Pre-seed with gigastt download --prequantized and gate clients on /ready.
• Address already in use on port 9876 — find the holder with lsof -nP -tiTCP:9876 -sTCP:LISTEN; confirm it is gigastt (ps -p <pid> -o command=), then kill <pid> (SIGTERM drains cleanly), or start on another port with --port.
• Model download fails or hangs (proxy, firewall, HuggingFace unreachable) — retry gigastt download; the resume-safe staging file makes it idempotent, and exit codes distinguish causes (65 = checksum, 69 = network, 74 = disk). If huggingface.co is blocked but github.com is not, use gigastt download --prequantized; in a fully closed contour use the air-gapped bundle. Check ~/.gigastt/models/ permissions on disk errors.
• OOM or heavy swap on startup — each pool session loads its own encoder copy (~400 MB resident with INT8); the default --pool-size 2 peaks around 790 MB. On small machines run with --pool-size 1.

The full symptom → cause → fix table lives in docs/troubleshooting.md.

Links

• docs/cli.md — canonical CLI reference (every flag and env var)
• docs/api.md — REST / SSE / WebSocket API reference
• docs/benchmarks.md — per-head WER / RTF numbers
• docs/troubleshooting.md — symptom → cause → fix
• docs/deployment.md — Docker details, reverse proxy, systemd, offline installs
• docs/verifying-releases.md — checksums, minisign, SLSA provenance for release artifacts
• packaging/offline/README-OFFLINE.md — offline bundle contents and installer options
• CLI and batch processing — the next chapter: batch, watch mode, export formats
• Models and backends — heads, quantization, execution providers in depth

CLI and batch processing

Turn a folder of recordings into transcripts: one-off sweeps with transcribe-batch, a continuously watched drop folder with watch, and an asynchronous queue with the jobs API. Every recipe is copy-paste runnable and ends with a check that it worked.

Scenario

You have a directory of audio — call-center recordings, podcast episodes, an archive of voice notes — and you want text files out the other side. Sometimes it is a one-time archive conversion; sometimes new recordings keep arriving and the pipeline must run unattended, retry failures, and never get stuck on a half-copied file.

Prerequisites

• gigastt installed and the model downloaded (gigastt download) — see Getting started.
• A folder of audio files: WAV, MP3, M4A, OGG, FLAC (subfolders are scanned recursively).
• Nothing else: transcribe, transcribe-batch, and watch are offline commands — no server, no network.

One thing to know before scripting: every CLI invocation loads the model (~1–2 s warm). transcribe-batch amortizes that cost across the whole folder, so prefer it over a shell for loop around single-file transcribe calls.

Recipe: one-off folder run — transcribe-batch

The workhorse. Point it at an input directory and an output directory:

gigastt transcribe-batch calls/ transcripts/

It scans calls/ recursively, transcribes every supported audio file with --pool-size workers (default 2), and writes transcripts/<name>.txt and transcripts/<name>.json per input (default --format txt,json).

A production-shaped run — more formats, more workers, and a source policy:

gigastt transcribe-batch calls/ transcripts/ \
  --format txt,json,srt \
  --pool-size 4 \
  --retries 2 \
  --move-to calls/done/

• --format — comma-separated list of txt,json,md,srt,vtt; one output file per format per input.
• --pool-size — parallel workers; each costs ~0.4 GB RAM (INT8 encoder), so scale with memory, not just cores (see the throughput recipe below).
• --retries — extra attempts per file with a short backoff (200 ms, 400 ms, …). Default 0 for batch, 2 for watch.
• --move-to — move each successfully transcribed source into the given directory. Failed files are always left in place. The move-to directory is excluded from the scan, so putting it inside the input folder (calls/done/) is safe and is the recommended layout.
• --delete-source — alternative to --move-to: delete sources after success. Mutually exclusive with --move-to.

Reading the run report. Each file logs a line, and the run ends with a summary:

INFO gigastt::batch: done /calls/alpha.wav processed=1 failed=0
WARN gigastt::batch: failed /calls/broken.mp3 error=invalid audio: Unsupported audio format: ...
INFO gigastt: batch finished processed=12 failed=1 skipped=0

Exit codes (script on these, not on the log text):

An empty input folder is not an error: it logs no audio files found and exits 0.

Verify the result

gigastt transcribe-batch calls/ transcripts/ --move-to calls/done/
echo "exit code: $?"        # 0 = clean sweep, 1 = some files failed
ls transcripts/             # one <name>.txt + <name>.json per source
ls calls/done/              # sources that succeeded
ls calls/*.wav 2>/dev/null  # anything still here failed — check the WARN lines

Recipe: a live folder — watch

watch polls a directory and transcribes files as they arrive:

gigastt watch inbox/ transcripts/ --format txt,json --move-to inbox/done/

How it differs from transcribe-batch:

• Backlog is skipped. Files already present at startup are registered but not transcribed (watching /inbox backlog=3 poll_ms=1000). Sweep the existing pile with transcribe-batch first (see the wrapper recipe), then let watch handle new arrivals.
• Settle protection. A file is scheduled only after its size + mtime are unchanged for --settle-polls consecutive polls (default 2) spaced --poll-interval-ms apart (default 1000 ms). A recording still being copied or written is never picked up half-finished. Slow network mounts → raise both.
• Changes are picked up. Overwriting a file resets the settle counter and the new version is transcribed (a change mid-transcription re-queues it after the current run finishes).
• Failures are sticky. A file that exhausts its retries (default 2 for watch) is marked failed and left alone until its content changes.
• Graceful stop. Ctrl-C stops scheduling new files, waits for the ones in flight, prints watch stopped processed=N failed=M, and exits 0 (1 if anything failed).

Verify the result

# terminal 1
gigastt watch inbox/ transcripts/ --move-to inbox/done/

# terminal 2
cp ~/recordings/sample.wav inbox/
# wait settle-polls x poll-interval plus transcription time, then:
ls transcripts/sample.txt        # appeared
ls inbox/done/sample.wav         # source archived
# back in terminal 1: Ctrl-C → "watch stopped processed=1 failed=0"

Recipe: inbox pipeline wrapper (shell)

The standard “drop folder” service: audio lands in inbox/, transcripts come out, successes are archived to done/, failures are collected in failed/ and automatically retried on the next sweep. Save as transcribe-inbox.sh:

#!/usr/bin/env bash
# Usage: transcribe-inbox.sh [INBOX] [OUT]
set -uo pipefail

INBOX="${1:-inbox}"
OUT="${2:-transcripts}"
DONE="$INBOX/done"
FAILED="$INBOX/failed"
mkdir -p "$OUT" "$DONE" "$FAILED"

# Requeue previous failures for another attempt.
find "$FAILED" -maxdepth 1 -type f \
  \( -name '*.wav' -o -name '*.mp3' -o -name '*.m4a' -o -name '*.ogg' -o -name '*.flac' \) \
  -exec mv -n {} "$INBOX/" \;

gigastt transcribe-batch "$INBOX" "$OUT" --format txt,json --move-to "$DONE"
rc=$?

# Successes were moved to done/; whatever audio remains at the inbox top
# level failed all retries — collect it for inspection and future requeue.
if [ "$rc" -eq 1 ]; then
  find "$INBOX" -maxdepth 1 -type f \
    \( -name '*.wav' -o -name '*.mp3' -o -name '*.m4a' -o -name '*.ogg' -o -name '*.flac' \) \
    -exec mv -n {} "$FAILED/" \;
  echo "some files failed — collected in $FAILED" >&2
fi
exit "$rc"

Run it on a schedule. A cron entry is enough for most inboxes:

*/15 * * * * /usr/local/bin/transcribe-inbox.sh /srv/stt/inbox /srv/stt/transcripts >> /var/log/stt-batch.log 2>&1

For a systemd timer + service unit instead of cron, see Deployment & ops.

Watch + batch catch-up. The two commands compose into an always-on pipeline: watch handles trickling arrivals with low latency, a periodic transcribe-batch drains the startup backlog and anything the watcher marked failed. Both honor the same --move-to exclusion, so they do not double-process the backlog. One caveat: a catch-up sweep started while the watcher is live can pick up a file the watcher has just scheduled but not yet moved — schedule sweeps for quiet hours, or accept that a file is occasionally transcribed twice (its outputs are simply overwritten).

# once, and then periodically (quiet hours): drain backlog + retry failures
./transcribe-inbox.sh /srv/stt/inbox /srv/stt/transcripts

# always on: new arrivals
gigastt watch /srv/stt/inbox /srv/stt/transcripts \
  --format txt,json --move-to /srv/stt/inbox/done/

Verify the result

chmod +x transcribe-inbox.sh
cp ~/recordings/*.wav inbox/ && printf 'junk' > inbox/broken.mp3
./transcribe-inbox.sh inbox transcripts; echo "exit: $?"   # 1 — broken.mp3 failed
ls transcripts/   # transcripts for the good files
ls inbox/done/    # good sources archived
ls inbox/failed/  # broken.mp3 collected here
./transcribe-inbox.sh inbox transcripts   # re-runs the failure, exits 1 again

Recipe: queued pipeline — the jobs API

watch covers one machine with a shared folder. Switch to the jobs API when producers are on other machines, when files are long enough that holding a synchronous HTTP request is awkward, or when you need progress reporting and cancellation. It is the same engine, fronted by an in-memory FIFO queue inside gigastt serve.

Jobs are off by default. Enable them (and reserve inference slots so queued work cannot starve WebSocket/REST streaming):

gigastt serve --enable-jobs --batch-pool-size 1

Submit → poll → fetch:

# submit (accepts the same query params as /v1/transcribe, e.g. ?format=srt)
curl -s -X POST http://127.0.0.1:9876/v1/jobs \
  --data-binary @episode.wav
# {"job_id":"019f858a-...","status":"queued","created_at":1784651881.9}

# poll status
curl -s http://127.0.0.1:9876/v1/jobs/019f858a-...
# {"job_id":"...","status":"processing","processed_seconds":12.5,"percent":42}

# fetch the result once status is "done"
curl -s http://127.0.0.1:9876/v1/jobs/019f858a-.../result
# {"text":"...","words":[...],"duration":3512.4}

status walks queued → processing → done | failed | cancelled. Fetching /result before done returns 409 job_not_finished. Other endpoints: DELETE /v1/jobs/{id} cancels a queued/processing job (204), and GET /v1/jobs/{id}/events streams SSE progress (data: {"type":"progress","percent":42,...} then done/failed).

A minimal driver script:

#!/usr/bin/env bash
# submit-and-wait.sh AUDIO_FILE — submit a job and print its transcript.
set -euo pipefail
BASE="${GIGASTT_BASE:-http://127.0.0.1:9876}"

job=$(curl -sf -X POST "$BASE/v1/jobs" --data-binary "@$1" \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["job_id"])')
echo "job: $job" >&2

while true; do
  status=$(curl -sf "$BASE/v1/jobs/$job" \
           | python3 -c 'import json,sys; print(json.load(sys.stdin)["status"])')
  case "$status" in
    done)               break ;;
    failed|cancelled)   echo "job $status" >&2; exit 1 ;;
  esac
  sleep 2
done

curl -sf "$BASE/v1/jobs/$job/result" | python3 -c 'import json,sys; print(json.load(sys.stdin)["text"])'

Queue behavior to plan around:

• --jobs-retry (default 3) — retries only transient failures: inference timeouts and worker panics. A file that cannot be decoded fails immediately, no retries.
• --jobs-max (default 100) — when the store is full, submit returns 429 queue_full with Retry-After. Back off and resubmit.
• --jobs-ttl-secs (default 3600) — finished/failed/cancelled jobs are evicted after the TTL. Fetch and persist results promptly — the store is in-memory, so a server restart drops queued jobs and unfetched results.

Verify the result

curl -s http://127.0.0.1:9876/ready     # {"status":"ready",...} before submitting
./submit-and-wait.sh episode.wav        # prints the transcript text
# disabled API check: without --enable-jobs every /v1/jobs call returns 404

Recipe: choosing output formats

Five formats, one file per format per input. Pick by the consumer:

gigastt transcribe-batch episodes/ out/ --format txt,json        # default pair
gigastt transcribe recording.wav -f srt -o recording.srt         # single file

Subtitle shaping (SRT/VTT): --max-chars-per-line (default 80) and --max-words-per-line (default 14) control cue grouping; 0 disables the limit. Broadcast-style captions usually want shorter lines:

gigastt transcribe recording.wav -f vtt --max-chars-per-line 42 -o recording.vtt

Markdown extras: --word-timestamps appends a per-word table with timings and confidence — handy for manual review, noisy for archives.

Scripting caveat for single-file transcribe: logs go to stdout at the default info level, mixed with the transcript. Use -o to write the transcript to a file, or silence the logs with the global flag placed before the subcommand:

gigastt --log-level error transcribe recording.wav          # stdout = transcript only

Extracting text from a folder of JSON results:

jq -r '.text' transcripts/*.json

Verify the result

gigastt transcribe recording.wav -f srt -o /tmp/check.srt
head -4 /tmp/check.srt
# 1
# 00:00:00,480 --> 00:00:02,160
# Привет, как дела?
jq -r '.duration' transcripts/episode.json    # JSON parses and has fields

Recipe: unusual inputs — telephony WAV, Opus, raw streams

G.711 / G.722 inside WAV — just works. A-law/μ-law (8 kHz telephony exports) and G.722 ADPCM (Asterisk/Cisco/Teams, format tags 0x0064/0x028F) are decoded automatically; the batch walker picks them up like any other .wav.

OGG/Opus and .opus (Telegram voice notes, browser MediaRecorder). The container is sniffed from content, so single-file transcription works as-is:

gigastt transcribe voice.opus

But the batch/watch walkers scan by extension (wav,mp3,m4a,ogg,flac) and do not pick up .opus files. Rename them to .ogg before sweeping — the content is already an OGG container, so a plain rename suffices:

for f in inbox/*.opus; do mv "$f" "${f%.opus}.ogg"; done
gigastt transcribe-batch inbox/ transcripts/

Raw headerless streams (RTP dumps, Asterisk Monitor raw) carry no container to sniff — declare the codec and rate explicitly:

gigastt transcribe call.ulaw --codec pcmu --sample-rate 8000
gigastt transcribe call.alaw --codec pcma --sample-rate 8000
gigastt transcribe call.g722 --codec g722 --sample-rate 8000   # 16000 also accepted

--codec accepts pcmu (alias ulaw), pcma (alias alaw), g722, and requires --sample-rate. Anything else — WebM, AMR, MP4 video, a corrupt file — fails with invalid audio: Unsupported audio format: ... (REST: 422 invalid_audio).

Verify the result

file recording.wav                 # confirms the container type
gigastt --log-level error transcribe call.ulaw --codec pcmu --sample-rate 8000
echo "exit: $?"                    # 0 = decoded and transcribed
gigastt transcribe call.ulaw --codec pcmu 2>&1 | head -2
# error: the following required arguments were not provided: --sample-rate

Recipe: throughput and memory

The rnnt head on INT8 runs at RTF ≈ 0.10 on an M1 CPU — one worker processes an hour of audio in about 6 minutes. A 100-hour archive at --pool-size 4 finishes in roughly 100 h × 0.10 / 4 ≈ 2.5 h of wall time. Full measurements, other hardware, and WER numbers: docs/benchmarks.md.

Budget memory before raising --pool-size:

• Each worker loads its own encoder copy: ~0.4 GB resident with the default INT8 encoder, ~1.7 GB with FP32. Default pool of 2 ≈ 790 MB RSS.
• The engine refuses to let the pool eat more than half of total RAM: an oversized --pool-size is clamped with a warning at load, so check the log instead of assuming you got the parallelism you asked for.
• Stay on INT8 (the default after the first-run auto-quantization): the encoder shrinks 844 MB → 215 MB on disk with ~0% WER degradation, and FP32 quadruples the per-worker memory cost for no batch-speed win.
• On CPU builds, --encoder-intra-threads defaults to logical CPUs divided across the pool — the right value for a dedicated batch box; tune only for shared machines.

Verify the result

# clamp warning, if any, appears at load time:
gigastt transcribe-batch calls/ transcripts/ --pool-size 8 2>&1 | grep -i "pool" | head -3
# per-file throughput in the log: "transcribe complete audio_s=... wall_s=... rtf=0.129"
time gigastt transcribe-batch calls/ transcripts/ --pool-size 4 --move-to calls/done/

Common pitfalls

• Half-copied files. transcribe-batch transcribes whatever is in the folder now, including a file still being copied — you get a decode failure or a truncated transcript. Producers should write to a temp name and mv into the inbox (rename is atomic on the same filesystem). watch protects itself with settle polls; batch relies on a quiet folder.
• Accidental reprocessing. Without --move-to/--delete-source, every rerun redoes the whole folder. Always set a source policy for repeated sweeps. Related: --move-to flattens subdirectories — a/week1/call.wav and a/week2/call.wav collide as one done/call.wav (and the run warns duplicate output ... inputs with equal file stems overwrite each other for the transcripts). Keep source filenames unique.
• Expecting parallelism you did not get. --pool-size 16 on 8 GB RAM is silently clamped at load (warning logged). Check the startup log, and remember FP32 quadruples per-worker memory.
• invalid audio / 422 on an unsupported container. WebM, AMR, MP4 video, or a corrupt upload fails decoding. Convert first (ffmpeg -i in.webm -ar 16000 -ac 1 out.wav) or, for raw telephony streams, declare --codec + --sample-rate. A .opus file is decodable but invisible to batch/watch — rename to .ogg.
• Watch “forgets” failures. A file that failed all retries is not retried until its content changes, and restarting the watcher registers it as backlog (never processed). Fix or replace the file, or point transcribe-batch at it — that is what the catch-up sweep in the wrapper recipe is for.
• Job results evaporate. The jobs store is in-memory with a 1 h TTL on terminal jobs: a restart loses queued jobs, and unfetched results are evicted. Persist results client-side; treat 429 queue_full as backpressure, not an error worth alerting on.

Links

• docs/cli.md — full flag reference for transcribe, transcribe-batch, watch
• docs/api.md — jobs API endpoints, query parameters, error codes
• docs/benchmarks.md — RTF, memory, and WER measurements
• docs/troubleshooting.md — decode and format error catalog
• Getting started — install and model download
• Deployment & ops — systemd units and timers for always-on pipelines

Telephony & VoIP: G.711, G.722, Opus, and PBX recordings

Scenario

You run a call center or integrate a PBX (Asterisk, FreeSWITCH, Cisco, Teams), and call recordings are your main source of audio. The folder in front of you contains a mix of wav49 files, G.711/G.722 WAVs, headerless .ulaw dumps pulled from RTP captures, and the odd Telegram voice note — and you want transcripts without manually converting every file.

gigastt decodes most telephony formats natively: G.711 A-law/μ-law in WAV, G.722 in WAV (both registered format tags), OGG/Opus, and headerless raw streams via an explicit codec hint. This chapter gets you from “a folder of weird files” to working transcripts, including per-channel speaker split for stereo recordings.

Prerequisites

• gigastt installed and the model downloaded — see Getting started.
• A running server for the REST recipes (gigastt serve, default http://127.0.0.1:9876). The CLI recipes work offline, no server needed.
• ffprobe/ffmpeg — only to identify formats and for the two conversion fallbacks (wav49, G.729). Not needed for the supported formats.

Recipe

Step 0 — identify what you actually have

PBX exports rarely say what they are. Two commands tell you everything:

file recording.wav
ffprobe -v error -show_entries stream=codec_name,codec_tag_string,sample_rate,channels \
  -of default=noprint_wrappers=1 recording.wav

Match the output against this table:

The two G.722 tags are the same codec from different writers: 0x0064 comes from SBC/Asterisk-style exports, 0x028F from ffmpeg-based tooling. gigastt accepts both.

Asterisk Monitor recordings (wav49, G.722 WAV, raw streams)

What Monitor()/MixMonitor writes depends on the configured format:

• wav — plain PCM16 WAV. Upload as-is.

• wav49 — GSM 06.10 in a WAV container (codec_name=gsm_ms). gigastt does not decode GSM, so convert once to PCM16 WAV:

  ffmpeg -y -i call.wav -ar 16000 -ac 1 -c:a pcm_s16le call_16k.wav
  

  Verify: ffprobe call_16k.wav reports codec_name=pcm_s16le, sample_rate=16000. Then transcribe call_16k.wav like any WAV.

• ulaw / alaw / g722 — raw headerless streams (no container to sniff). Declare the codec explicitly:

  # REST — codec aliases: pcmu=ulaw, pcma=alaw
  curl -X POST "http://127.0.0.1:9876/v1/transcribe?codec=pcmu&sample_rate=8000" \
    -H "Content-Type: application/octet-stream" --data-binary @call.ulaw
  
  # CLI
  gigastt transcribe call.ulaw --codec pcmu --sample-rate 8000
  

  Verify: HTTP 200 and a transcript that matches the call. If you pick the wrong companding law (A-law vs μ-law), the request still returns 200 but the text is garbage — swap the codec name and retry.

Cisco and Teams exports (G.722 WAV)

Cisco phone systems and Teams-adjacent tooling typically hand you G.722 in a WAV container — codec_name=adpcm_g722, tag 0x0064 or 0x028F. The container declares the codec, so no flags are needed:

curl -X POST http://127.0.0.1:9876/v1/transcribe \
  -H "Content-Type: application/octet-stream" --data-binary @call_g722.wav

gigastt transcribe call_g722.wav

Verify: HTTP 200 with JSON text, or the transcript on stdout for the CLI.

The typical failure here is 422 — the server could not decode the upload. It means the file is not what the extension claims (a G.729 payload, a vendor-proprietary wrapper) or the file is truncated. Go back to Step 0 and check what ffprobe actually says.

Telegram and WhatsApp voice notes (Opus)

Telegram voice messages (Bot API voice.oga), WhatsApp voice notes (.ogg), and browser MediaRecorder captures (.opus) are all Opus in an Ogg container. Upload them as-is — the server sniffs the container from the bytes, so the file extension does not matter:

curl -X POST http://127.0.0.1:9876/v1/transcribe \
  -H "Content-Type: application/octet-stream" --data-binary @voice.ogg

gigastt transcribe voice.opus

Verify: HTTP 200 with the transcript. Opus decodes at its native 48 kHz and is resampled internally; mono and stereo are supported (stereo is mixed to mono unless you use channel split below), multistream (>2 channel) OGG/Opus is rejected.

A stereo call → two speakers (channels=split)

Many PBXs record one party per channel: left = one speaker, right = the other. Channel split transcribes each channel as its own speaker instead of mixing to mono: channel 0 (left) becomes speaker_0, channel 1 (right) becomes speaker_1.

# REST
curl -X POST "http://127.0.0.1:9876/v1/transcribe?channels=split" \
  -H "Content-Type: application/octet-stream" --data-binary @call.wav

# CLI — speaker-aware SRT with [SPEAKER_0] / [SPEAKER_1] cue prefixes
gigastt transcribe call.wav --stereo-speakers -f srt -o call.srt

In the JSON every word carries a speaker field, ordered by start time:

{
  "text": "…",
  "words": [
    {"word": "покажи", "start": 0.08, "end": 0.48, "confidence": 0.95, "speaker": 1},
    {"word": "шестьдесят", "start": 0.52, "end": 1.08, "confidence": 0.96, "speaker": 0}
  ],
  "duration": 3.43
}

For reports, group words by speaker to get per-party text and talk time (agent vs customer). Which party sits on which channel is your PBX’s convention — calibrate once on a call with known speakers.

Verify: both labels appear —

curl -s -X POST "http://127.0.0.1:9876/v1/transcribe?channels=split" \
  -H "Content-Type: application/octet-stream" --data-binary @call.wav \
  | python3 -c "import json,sys; d=json.load(sys.stdin); print(sorted({w.get('speaker') for w in d['words']}))"
# [0, 1]

Fallbacks to know about: if the file is mono, has more than two channels, or is dual-mono (both channels nearly identical — some PBXs mix the call into both), gigastt falls back to a plain mono transcript with no speaker fields and logs a falling back to mono transcription warning. Channel split is mutually exclusive with diarization: channels=split&diarization=true returns 400 conflicting_modes.

Mono meeting recording → speakers via diarization

When you have a single mono mix (or dual-mono that would fall back above) and still need speaker labels, use ML diarization instead of channel split:

# Speaker model is fetched with the default download path
# (skip with: gigastt download --skip-diarization)
gigastt serve   # default build includes the diarization feature

# Opt-in per request — plain /v1/transcribe never attaches speaker labels
curl -s -X POST "http://127.0.0.1:9876/v1/transcribe?diarization=true" \
  -H "Content-Type: application/octet-stream" --data-binary @meeting.wav \
  | python3 -c "import json,sys; d=json.load(sys.stdin); print(sorted({w.get('speaker') for w in d.get('words',[]) if 'speaker' in w}))"
# e.g. [0, 1] when two speakers were separated

# Live sessions: enable after ready, before the first audio frame
# {"type":"configure","diarization":true}

Verify: GET /v1/models (or WS ready) shows "diarization": true only when wespeaker_resnet34.onnx is loaded. Without the model file, the request succeeds but words have no speaker field. Offline diarization matches each word’s midpoint to a turn; streaming assigns the latest turn to new words (coarser). Full contract: docs/api.md and the project README speaker row.

An RTP dump without a container

An RTP capture stripped to payload bytes has no header to sniff, so the codec must be declared. Export payload-only from your tooling (in Wireshark: Telephony → RTP → Stream Analysis → save payload; SBCs have similar payload exports) — the 12-byte RTP headers must not be in the file.

# G.711 A-law capture at 8 kHz
curl -X POST "http://127.0.0.1:9876/v1/transcribe?codec=pcma&sample_rate=8000" \
  -H "Content-Type: application/octet-stream" --data-binary @dump.alaw

# G.722 capture — see the SDP clock-rate note below
curl -X POST "http://127.0.0.1:9876/v1/transcribe?codec=g722&sample_rate=8000" \
  -H "Content-Type: application/octet-stream" --data-binary @dump.g722

# CLI equivalent
gigastt transcribe dump.g722 --codec g722 --sample-rate 8000

The G.722 SDP quirk: for historical reasons SDP/RTP announces G.722 with an 8000 Hz clock rate, while the stream actually decodes to 16 kHz. gigastt accepts both 8000 and 16000 for g722 and always decodes to 16 kHz, so either value works. For raw G.711 (pcmu/pcma) any rate in 8000–48000 Hz is accepted and resampled.

Verify: HTTP 200 and sensible text. Parameter errors fail fast, before any inference: codec without sample_rate → 400 invalid_sample_rate (“sample_rate is required when codec is set”); an unknown codec → 400 unsupported_codec (“Unsupported codec. Supported: pcmu (ulaw), pcma (alaw), g722”).

Lossy captures: the stream is decoded as-is. Packet-loss gaps and reordered audio are not repaired — de-jitter or re-capture if the transcript has holes.

A folder of recordings (batch)

gigastt transcribe-batch scans files with wav, mp3, m4a, ogg, and flac extensions — which covers G.711/G.722 WAV and OGG/Opus (.ogg) out of the box:

gigastt transcribe-batch recordings/ out/ --format txt,json

Two kinds of telephony files are not scanned:

• .opus files — rename them to .ogg. The content is an Ogg container and is sniffed from bytes, so the rename is safe:

  for f in recordings/*.opus; do mv "$f" "${f%.opus}.ogg"; done
  

• raw .ulaw / .alaw / .g722 streams — wrap them into WAV first (batch has no --codec flag), then run batch on the wrapped files:

  mkdir -p wav
  for f in recordings/*.ulaw; do
    ffmpeg -y -v error -f mulaw -ar 8000 -ac 1 -i "$f" \
      -ar 16000 -ac 1 -c:a pcm_s16le "wav/$(basename "${f%.ulaw}").wav"
  done
  gigastt transcribe-batch wav/ out/ --format txt,json
  

  For A-law use -f alaw, for G.722 -f g722 (G.722 needs no input -ar).

Verify: out/ contains one .txt/.json per source file and the command exits 0. For an inbox that keeps receiving new recordings, gigastt watch does the same continuously — see CLI and batch processing for batch/watch details and long recordings.

Format cheat sheet

Applies everywhere: uploads are capped at 30 minutes of decoded audio, and ?codec= / ?sample_rate= work on /v1/transcribe, /v1/transcribe/stream, and /v1/jobs alike. A Deepgram-compatible /v1/listen endpoint that accepts the same telephony inputs is in progress.

Verifying the result

The repository ships telephony fixtures used by its own end-to-end tests — 4 seconds of Russian speech («шестьдесят тысяч тенге сколько будет стоить») transcoded into every supported telephony format. You need the downloaded model and a running server:

# raw path
curl -s -X POST "http://127.0.0.1:9876/v1/transcribe?codec=pcmu&sample_rate=8000" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @crates/gigastt/tests/fixtures/telephony/speech.ulaw

# container path — no parameters
curl -s -X POST http://127.0.0.1:9876/v1/transcribe \
  -H "Content-Type: application/octet-stream" \
  --data-binary @crates/gigastt/tests/fixtures/telephony/speech_g722.wav

Both return HTTP 200 with a transcript mentioning «тенге» and «стоить» — with the server’s default punctuation and ITN enabled the text comes out as «60000 тенге, сколько будет стоить?». If your own file fails while these fixtures pass, the problem is the file, not the server — go back to Step 0.

Common pitfalls

• 422 — “Check audio format” (invalid_audio / transcription_error). The bytes were probed as a container and decoding failed. Usual causes: a headerless raw stream posted without codec=, a wav49 (GSM) or G.729 file, or a truncated/corrupt upload. Identify the file with Step 0.

• G.729 is not supported. A raw upload with ?codec=g729 returns 400 unsupported_codec; G.729-in-WAV fails with 422. Convert with ffmpeg and upload the result:

  ffmpeg -y -i call_g729.wav -ar 16000 -ac 1 -c:a pcm_s16le call_16k.wav
  

• ?codec= on a container file. The parameter overrides container sniffing entirely: a WAV posted with ?codec=pcmu decodes the WAV header as μ-law noise and returns 200 with garbage text. Use codec= only for headerless streams.

• RTP dump with headers or jitter. ?codec= expects payload bytes only. Leftover 12-byte RTP headers decode as periodic clicks, and loss/reorder gaps become holes in the transcript — nothing is repaired server-side. Export payload-only and de-jitter before uploading.

• “8-bit WAV” is not broken. G.711 in WAV legitimately reports bits_per_sample=8 (companded samples) — upload it as-is. The real trap is the reverse: when converting files yourself, always write 16-bit PCM (pcm_s16le); 8-bit linear PCM (pcm_u8) destroys recognition accuracy.

• Mono/stereo confusion. channels=split on a mono file, a >2-channel file, or dual-mono stereo (some PBXs record the mixed call into both channels) silently falls back to a plain mono transcript — no speaker fields, just a falling back to mono transcription warning in the log. If your PBX records mixed mono, no flag can split the speakers after the fact; record stereo or use diarization=true instead (mutually exclusive with channels=split).

• “Audio file too long”. A single upload is capped at 30 minutes of decoded audio (“Maximum supported: 1800s”). Split longer recordings — for example per call leg — before uploading.

• A-law/μ-law swapped. Both laws decode “successfully”, so the wrong choice returns 200 with garbage instead of an error. If a raw stream transcribes to noise, retry with the other codec name.

Links

• CLI and batch processing — batch and watch mode, export formats, long recordings.
• Getting started — installation and model download.
• Introduction — documentation map.
• docs/api.md — Audio formats and telephony codecs — the canonical format table and all query parameters.
• docs/cli.md — transcribe, transcribe-batch, and watch flag reference.
• crates/gigastt-core/src/inference/audio.rs — decoder internals (G.722 sniffing, the Opus path, dual-mono detection).
• Telephony and Opus test fixtures with their generators generate_telephony_fixtures.sh and generate_opus_fixtures.sh.

Streaming over WebSocket

Live, partial-results transcription: a microphone feed, a call leg, or a browser capture goes in as raw PCM16, and text comes out within about a second of speech. This chapter is the recipe book for that integration — the field-by-field protocol reference stays in docs/api.md and the machine-readable schema in docs/asyncapi.yaml; we link to them instead of repeating them.

Scenario

You are building a real-time integration — live captions for a meeting tool, a voice bot on a phone line, or a “dictaphone with instant text” feature. Audio arrives continuously; users expect to watch the transcript grow while they speak, see each utterance finalized cleanly, and never lose trailing words when the stream ends. Some sessions run for hours, some setups capture two audio sources at once (mic + system audio), and the client must survive pool saturation and network drops without babysitting.

Prerequisites

• A running server with the model, per Getting started: gigastt serve (first run downloads ~850 MB and quantizes — wait for readiness, don’t kill it):

  until curl -sf http://127.0.0.1:9876/ready; do sleep 2; done
  # {"status":"ready","pool_available":2,"pool_total":2}
  

• A WebSocket client stack for your language: pip install websockets for the Python recipes, Node.js ≥ 22 (global WebSocket, no dependencies) for the JavaScript one, Go 1.23+ for the SDK one.

• A PCM16 mono source. For the copy-paste checks below, the repository ships a 4-second Russian speech fixture at exactly the right format (16 kHz mono Int16): crates/gigastt/tests/fixtures/golos_00.wav. For a real microphone, ffmpeg turns any capture device into raw PCM16 on stdout.

Recipe

The session shape every recipe builds on — connect, read ready, optionally configure, stream binary PCM16, stop to finalize:

Client                            Server
  |-------- connect --------------> |
  | <------- ready ----------------- |  limits + supported rates (read these!)
  |------- configure (optional) --> |  before the first audio frame
  |-------- binary PCM16 ---------> |
  | <------- partial / final ------ |
  |--------- stop ----------------> |
  | <------- final ----------------- |  trailing words flushed, then close

Recipe 1: connect, negotiate, and stream the microphone

1. Connect and read ready first. The server’s first message is always ready. It carries everything you must not hardcode: the protocol version, the accepted supported_rates, and the session caps (max_session_secs, idle_timeout_secs) that Recipe 4 plans around:

   {
     "type": "ready",
     "model": "gigaam-v3-rnnt",
     "sample_rate": 48000,
     "version": "1.0",
     "supported_rates": [8000, 16000, 24000, 44100, 48000],
     "max_session_secs": 3600,
     "idle_timeout_secs": 300
   }
   

   sample_rate (48000 by default) is what the server expects if you never send configure. One caveat to know before debugging “silent connect”: on a saturated pool the server answers with an error message instead of ready — see Recipe 5.

2. Send configure before the first audio frame. Pick the rate your capture pipeline actually emits — it must be one of ready.supported_rates. 16 kHz is the sweet spot when you control the source (the model runs at 16 kHz internally, so no resampling work); a browser capture at 48 kHz can also be sent as-is. An unsupported rate is not fatal: you get an invalid_sample_rate error and the session keeps the previous rate. A configure that arrives after the first audio frame is rejected with configure_too_late and the previous settings stay — so always send it right after ready:

   {"type": "configure", "sample_rate": 16000}
   

3. Send binary frames of PCM16 (signed 16-bit little-endian, mono) at the negotiated rate. Sensible chunking: 100–500 ms per frame (3,200–16,000 bytes at 16 kHz). Partials are produced on a decode stride of roughly 0.8 s of new audio, so sub-second chunks keep the preview feeling live without per-frame overhead. The hard cap is --ws-frame-max-bytes (default 512 KiB) — a larger frame closes the socket with 1009. Odd byte counts are fine (the trailing byte is carried into the next frame), and occasional empty frames are tolerated. Stereo sources must be mixed down to mono first (-ac 1 in ffmpeg).

4. Put it together — microphone to live text. This pipes the default mic through ffmpeg into a minimal Python client:

   # macOS (-f avfoundation); Linux: -f alsa -i default; Windows: -f dshow -i audio="..."
   ffmpeg -hide_banner -loglevel error -f avfoundation -i ":default" \
     -ac 1 -ar 16000 -f s16le - | python3 mic_stream.py
   

   mic_stream.py — the complete reference loop this chapter uses:

   #!/usr/bin/env python3
   """Stream raw PCM16 from stdin to gigastt and print live transcripts.
   
   Usage: ffmpeg ... -f s16le - | python3 mic_stream.py [label] [server]
   """
   import asyncio
   import json
   import sys
   
   import websockets
   
   LABEL = sys.argv[1] if len(sys.argv) > 1 else "mic"
   SERVER = sys.argv[2] if len(sys.argv) > 2 else "ws://127.0.0.1:9876/v1/ws"
   RATE = 16000
   CHUNK = RATE * 2 // 5  # 400 ms of PCM16
   
   
   async def main() -> None:
       async with websockets.connect(SERVER) as ws:
           ready = json.loads(await ws.recv())
           assert ready["type"] == "ready", ready
           assert RATE in ready.get("supported_rates", [ready["sample_rate"]])
           await ws.send(json.dumps({"type": "configure", "sample_rate": RATE}))
           print(f"{LABEL}: connected to {ready['model']}", file=sys.stderr)
   
           async def receive() -> None:
               async for raw in ws:
                   msg = json.loads(raw)
                   if msg["type"] == "partial":
                       print(f"\r{LABEL} ... {msg['text']}   ", end="", flush=True)
                   elif msg["type"] == "final":
                       conf = msg.get("confidence")
                       suffix = f" ({conf:.2f})" if conf is not None else ""
                       print(f"\r{LABEL} >>> {msg['text']}{suffix}   ")
                   elif msg["type"] == "error":
                       print(f"\n{LABEL} ERR {msg['code']}: {msg['message']}")
   
           receiver = asyncio.create_task(receive())
           try:
               while data := await asyncio.to_thread(sys.stdin.buffer.read, CHUNK):
                   await ws.send(data)
               # Capture ended: finalize — never close before the trailing final.
               await ws.send(json.dumps({"type": "stop"}))
               await receiver
           except websockets.ConnectionClosed:
               pass  # server closed after the trailing final (or an error we printed)
   
   
   asyncio.run(main())
   

   Blocking stdin reads run in a thread (asyncio.to_thread) so the receiver task keeps printing while you speak.

Check: while you speak, ... lines refresh about once a second; a pause of roughly half a second produces a >>> final. Closing ffmpeg (Ctrl+C) triggers one last final and a clean close — nothing is cut off.

Recipe 2: show partials live, commit finals

The two transcript message types carry the same payload but play different roles in the UI — treat them differently.

• partial is a preview. Always the raw decoder hypothesis: lowercase, no punctuation, and it may change as more audio arrives. A new partial lands roughly every 0.8 s of speech (the decode stride):

  {
    "type": "partial",
    "text": "привет как",
    "timestamp": 1712700000.123,
    "is_final": false,
    "confidence": 0.93,
    "words": [
      {"word": "привет", "start": 0.0, "end": 0.4, "confidence": 0.97},
      {"word": "как", "start": 0.5, "end": 0.7, "confidence": 0.89}
    ]
  }
  

• final is the committed line. Enriched at the finalization boundary: inverse text normalization (number-words → digits), then punctuation and casing restoration. The rnnt head needs the punctuation model attached (server --punctuation auto default; check GET /health → "punctuation":true); the e2e_rnnt head punctuates by itself — see Models and backends. words[] always keep the raw decoder output in both message types; only the joined text is rewritten:

  {
    "type": "final",
    "text": "Привет, как дела?",
    "timestamp": 1712700001.456,
    "is_final": true,
    "confidence": 0.95,
    "words": [
      {"word": "привет", "start": 0.0, "end": 0.4, "confidence": 0.97},
      {"word": "как", "start": 0.5, "end": 0.7, "confidence": 0.93},
      {"word": "дела", "start": 0.8, "end": 1.1, "confidence": 0.95}
    ]
  }
  

The UI rule of thumb: render the latest partial in a “preview” style (grey/italic, replaceable), and when a final for the utterance lands, replace the preview with final.text and append it to the committed transcript. Never persist partial text.

A final fires when an utterance ends (speech_final: true, optional endpoint_reason). Endpointing ownership:

The ~2.5 s encoder window cap never ends an utterance — it commits a stable prefix and keeps emitting partials so monologues and long commands stay one turn. (Before this fix, cap-as-final made assistants like Irene fire mid-phrase.)

# Voice assistant (Irene): VAD owns the end of the command
gigastt serve --vad --vad-min-silence-ms 1200 --endpoint-mode assistant

# Dictation / captions with longer silence before commit
gigastt serve --vad --vad-min-silence-ms 900

Per-session overrides (before first audio):

{"type":"configure","sample_rate":16000,"endpoint_mode":"assistant","min_silence_ms":1200}

Per-session post-processing overrides compose with the same configure message (finals only; partials always stay raw). Asking for punctuation: true on a server without the model is a graceful no-op, not an error:

{"type": "configure", "sample_rate": 16000, "punctuation": false, "itn": false}

Check: say “привет как дела” with short pauses. The preview shows привет как дела in lowercase; the committed line arrives as Привет, как дела? (when the punctuation model is attached — verify with curl -s http://127.0.0.1:9876/health showing "punctuation":true). With --vad --vad-min-silence-ms 900, a deliberate ~0.7 s pause between words should not force a premature final.

Recipe 3: end the session cleanly — stop is the drain

The only correct end-of-stream pattern:

1. Send {"type": "stop"}.
2. Wait for the final — the server decodes whatever is still buffered since the last partial (trailing words are not lost) and emits one last final, possibly with empty text if nothing was pending.
3. Then close the socket (or let the server close it — it ends the session right after that final).

Do not close immediately after the last audio frame (the sub-stride remainder would be dropped) and do not insert a fixed sleep drain — the final after stop is the explicit, lossless end marker. The mic_stream.py loop in Recipe 1 already implements this.

Keepalive needs no code on your side: the server pings every 30 s and closes the connection after two consecutive pings with no inbound frame in between (≈ 90 s detection of a half-open peer). Any inbound frame — a pong, binary audio, or text — resets the counter, and standard WebSocket clients answer pings automatically. Only raw-socket implementations must reply to pings themselves.

Check: say one word and send stop immediately, before any partial arrives. The word still shows up in the trailing final. Then watch the close handshake: it happens after that final, not before.

Recipe 4: keep long sessions alive (idle timeout, session cap)

Two server-side clocks govern every session; both are announced in ready so you can plan instead of being surprised by a close frame.

• Idle timeout (idle_timeout_secs, default 300): any frame — audio, pong, or text — resets it. Pauses in speech only count as idle if the client stops sending; streaming quiet PCM keeps the session alive (silence is still audio). When it trips: an idle_timeout error and close code 1001. If your app mutes capture during pauses, send silence frames instead of nothing.
• Session cap (max_session_secs, default 3600, 0 = disabled): wall-clock, starts at connect. When it trips: the server sends a max_session_duration_exceeded error, flushes a final first, then closes with 1008 — nothing already recognized is lost, so reconnecting is safe.

For recordings longer than an hour you have two options, and combining them is fine:

1. Operator side: raise or disable the cap — gigastt serve --max-session-secs 0 (every limit is a CLI flag; see docs/cli.md).

2. Client side: rotate on a schedule. Read the cap from ready, and at ~90 % of it finalize with stop and open a fresh session — a planned reconnect, not an abrupt cut:

   # rotate_sessions.py — keep transcription running across the server cap.
   import asyncio
   import json
   
   import websockets
   
   SERVER = "ws://127.0.0.1:9876/v1/ws"
   
   
   def handle_message(msg: dict) -> None:
       if msg["type"] == "final":
           print(">>>", msg["text"])
       elif msg["type"] == "error":
           print("ERR", msg["code"], msg["message"])
   
   
   async def run_session(ws, stream_audio, rotate_at: float | None) -> None:
       async def rotate() -> None:
           await asyncio.sleep(rotate_at)
           await ws.send(json.dumps({"type": "stop"}))  # flush now, reconnect fresh
   
       tasks = [asyncio.create_task(stream_audio(ws))]
       if rotate_at:
           tasks.append(asyncio.create_task(rotate()))
       try:
           async for raw in ws:  # ends when the server closes after the final
               handle_message(json.loads(raw))
       finally:
           for task in tasks:
               task.cancel()
   
   
   async def run_shift(stream_audio) -> None:
       """Rotate before the cap; back off on transient drops."""
       backoff = 0.25
       while True:
           try:
               async with websockets.connect(SERVER) as ws:
                   ready = json.loads(await ws.recv())
                   cap = ready["max_session_secs"]  # always sent; 0 = no cap
                   await run_session(ws, stream_audio, rotate_at=cap * 0.9 or None)
                   backoff = 0.25  # clean stop → final → close: rotate at once
           except (OSError, websockets.ConnectionClosed):
               # 1006 drop, a proxy-injected 1011, the 1008 cap close, network...
               await asyncio.sleep(backoff)
               backoff = min(backoff * 2, 30)
   

   stream_audio(ws) is your capture loop from Recipe 1. Note what this pattern does not do: it never retries fatal, fix-your-client errors (unsupported_protocol_version, a rate outside supported_rates) — those fail fast at handshake and must be fixed, not retried.

Check: start the server with a tiny cap to see the whole lifecycle in a minute: gigastt serve --max-session-secs 30 --idle-timeout-secs 10. With audio flowing you observe the max_session_duration_exceeded error, a flushed final, close 1008, and the rotator rejoining without losing text. Stop sending frames entirely and the idle_timeout error with close 1001 fires after 10 s.

Recipe 5: confidence thresholds and backpressure

Confidence. Every transcript segment carries an optional confidence — the duration-weighted mean of its words[].confidence (each word’s is the mean softmax score over its BPE tokens). It is an average of softmax scores, not a calibrated probability, and it is omitted when the segment has no words. As starting thresholds to tune on your own data: highlight words below ~0.7 for human review, flag segments below ~0.8:

const unsure = (msg.words ?? []).filter((w) => w.confidence < 0.7);
if (unsure.length) console.log("review:", unsure.map((w) => w.word).join(" "));

Backpressure. Inference slots come from a pool (--pool-size, default 2), and a WebSocket session holds its slot for its whole lifetime. When all slots are busy, a new connection waits up to 30 s for one — and if none frees up, the server answers instead of ready with a timeout error carrying retry_after_ms, then closes. Honor the hint exactly instead of guessing a delay:

{"type": "error", "message": "Server busy, try again later", "code": "timeout", "retry_after_ms": 30000}

first = json.loads(await ws.recv())          # may be an error, not ready
if first["type"] == "error" and first["code"] == "timeout":
    await asyncio.sleep(first["retry_after_ms"] / 1000)  # then reconnect

For everything else that closes the socket unexpectedly — a 1006 abnormal drop, a proxy-injected code such as 1011, a network blip — reconnect with exponential backoff (start ~250 ms, double up to a few seconds, cap the attempts), as in run_shift from Recipe 4. Never retry the fatal handshake errors (unsupported_protocol_version). Both official SDKs implement exactly this policy — configure-first handshake, retry_after_ms honored on pool saturation, exponential backoff otherwise — so prefer them over hand-rolling: sdks/go, sdks/js.

Check: start with --pool-size 1 and connect two clients at once. The second sits ~30 s, then receives {"code":"timeout","retry_after_ms":30000}; after sleeping it off, it connects once the first session ends. For confidence: stream a clean speech file and confirm finals carry confidence close to 1.0, then try quiet/noisy audio and watch words drop below 0.7.

Recipe 6: two channels — mic + system audio

The meeting-assistant pattern (your mic + the far end’s system audio, each labeled) maps onto two independent WebSocket sessions — the server does not tag sources, so label them client-side. The constraint to design around is the pool: each session holds one inference slot for its entire lifetime, so the default --pool-size 2 fits exactly two channels and nothing else. Give the server headroom if other clients also connect — each extra slot costs roughly 0.4 GB RAM with the INT8 encoder (the server caps the pool to available RAM at load):

gigastt serve --pool-size 4

Then run one capture pipeline per source, each with its own label (the mic_stream.py from Recipe 1 takes the label as its first argument):

# terminal 1 — your microphone (macOS example; Linux: -f alsa -i default)
ffmpeg -hide_banner -loglevel error -f avfoundation -i ":default" \
  -ac 1 -ar 16000 -f s16le - | python3 mic_stream.py mic

# terminal 2 — system audio (Linux Pulse/PipeWire monitor source;
# macOS: a virtual device such as BlackHole selected via avfoundation)
ffmpeg -hide_banner -loglevel error -f pulse -i default.monitor \
  -ac 1 -ar 16000 -f s16le - | python3 mic_stream.py system

Two lighter-weight alternatives when speaker-labeled channels are not required: mix both sources into one stream client-side (one pool slot, one session), or use per-word speaker labels from diarization — a server built with --features diarization advertises "diarization": true in ready, and {"type":"configure","diarization":true} then adds a speaker field to each word (see the reference in docs/api.md).

Check: both terminals print with their distinct mic / system labels; while both run, curl -s http://127.0.0.1:9876/ready shows "pool_available":0,"pool_total":2 (with the default pool), and a third client gets the timeout + retry_after_ms treatment from Recipe 5.

Recipe 7: client skeletons (Python, Node.js, Go)

Minimal but complete loops to start from — each does the full ready → configure → stream → stop → wait-for-final → close cycle with error handling. More clients (Bun, Kotlin, Rust) live in examples/.

Python — streams a 16 kHz mono PCM16 WAV, honors retry_after_ms on pool saturation:

#!/usr/bin/env python3
"""Stream a 16 kHz mono PCM16 WAV to gigastt, honoring server backpressure.

Usage: python3 stream_wav.py <audio.wav> [ws://host:port]
"""
import asyncio
import json
import sys
import wave

import websockets


class PoolBusy(Exception):
    """Pool saturated: the server sent retry_after_ms instead of ready."""


async def session(path: str, server: str) -> None:
    async with websockets.connect(server) as ws:
        first = json.loads(await ws.recv())
        if first["type"] == "error":  # pool busy: error + close instead of ready
            raise PoolBusy(first.get("retry_after_ms", 30000))
        assert first["type"] == "ready", first
        await ws.send(json.dumps({"type": "configure", "sample_rate": 16000}))

        with wave.open(path, "rb") as wav:
            assert wav.getnchannels() == 1 and wav.getsampwidth() == 2, "mono PCM16 WAV"
            pcm = wav.readframes(wav.getnframes())

        async def receive() -> None:
            async for raw in ws:
                msg = json.loads(raw)
                if msg["type"] == "partial":
                    print(f"\r... {msg['text']}   ", end="", flush=True)
                elif msg["type"] == "final":
                    print(f"\r>>> {msg['text']}   ")
                elif msg["type"] == "error":
                    print(f"\nERR {msg['code']}: {msg['message']}")

        receiver = asyncio.create_task(receive())
        try:
            for off in range(0, len(pcm), 16000):  # 0.5 s of PCM16 at 16 kHz
                await ws.send(pcm[off : off + 16000])
                await asyncio.sleep(0.1)  # pace it like a live feed
            await ws.send(json.dumps({"type": "stop"}))
            await receiver  # trailing final, then the server closes
        except websockets.ConnectionClosed:
            pass


async def main() -> None:
    server = sys.argv[2] if len(sys.argv) > 2 else "ws://127.0.0.1:9876/v1/ws"
    while True:
        try:
            await session(sys.argv[1], server)
            return
        except PoolBusy as busy:
            wait = busy.args[0] / 1000
            print(f"pool busy, retrying in {wait:.0f}s")
            await asyncio.sleep(wait)  # honor retry_after_ms exactly


asyncio.run(main())

Node.js — no dependencies (global WebSocket, Node ≥ 22), same WAV file contract:

// stream_wav.mjs — stream a 16 kHz mono PCM16 WAV to gigastt (Node.js ≥ 22).
import { readFile } from "node:fs/promises";

const [wavPath, server = "ws://127.0.0.1:9876/v1/ws"] = process.argv.slice(2);
if (!wavPath) {
  console.error("usage: node stream_wav.mjs <audio.wav> [ws://host:port]");
  process.exit(1);
}

const pcm = (await readFile(wavPath)).subarray(44); // skip the WAV header
const ws = new WebSocket(server);
ws.binaryType = "arraybuffer";

let stopped = false;
const done = new Promise((resolve, reject) => {
  ws.onmessage = async (event) => {
    const msg = JSON.parse(event.data);
    if (msg.type === "ready") {
      ws.send(JSON.stringify({ type: "configure", sample_rate: 16000 }));
      for (let off = 0; off < pcm.byteLength; off += 16000) { // 0.5 s at 16 kHz
        ws.send(pcm.subarray(off, off + 16000));
        await new Promise((r) => setTimeout(r, 100)); // pace like a live feed
      }
      stopped = true;
      ws.send(JSON.stringify({ type: "stop" })); // finalize — do NOT close yet
    } else if (msg.type === "partial") {
      process.stdout.write(`\r... ${msg.text}   `);
    } else if (msg.type === "final") {
      console.log(`\r>>> ${msg.text}   `);
      if (stopped) { ws.close(); resolve(); } // trailing final: safe to close
    } else if (msg.type === "error") {
      const hint = msg.retry_after_ms ? ` (retry in ${msg.retry_after_ms} ms)` : "";
      reject(new Error(`${msg.code}: ${msg.message}${hint}`));
    }
  };
  ws.onerror = () => reject(new Error("websocket transport error"));
});
await done;

For anything beyond a skeleton, the typed SDK adds the reconnect policy from Recipe 5 out of the box: npm install @gigastt/client — see sdks/js.

Go — on the official SDK (go get github.com/ekhodzitsky/gigastt/sdks/go@latest), which pins the protocol version, sends configure first, and retries backpressure with the server’s retry_after_ms:

// stream_wav.go — stream a 16 kHz mono PCM16 WAV to gigastt via the Go SDK.
package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"time"

	gigastt "github.com/ekhodzitsky/gigastt/sdks/go"
)

func main() {
	if len(os.Args) < 2 {
		log.Fatal("usage: go run stream_wav.go <audio.wav>")
	}
	done := make(chan struct{})

	client, err := gigastt.Dial(context.Background(), gigastt.DefaultURL,
		gigastt.WithSampleRate(16000), // must be in the server's supported_rates
		gigastt.WithReconnect(250*time.Millisecond, 5*time.Second, 10),
		gigastt.WithHandlers(gigastt.Handlers{
			OnPartial: func(t gigastt.Transcript) { fmt.Printf("\r... %s   ", t.Text) },
			OnFinal:   func(t gigastt.Transcript) { fmt.Printf("\r>>> %s   \n", t.Text) },
			OnError:   func(e *gigastt.ServerError) { log.Printf("server error: %v", e) },
			OnClose: func(err error) {
				if err != nil {
					log.Printf("connection closed: %v", err)
				}
				close(done)
			},
		}),
	)
	if err != nil {
		log.Fatal(err) // e.g. *gigastt.ServerError unsupported_protocol_version
	}
	defer client.Close()

	wav, err := os.ReadFile(os.Args[1])
	if err != nil {
		log.Fatal(err)
	}
	for off := 44; off < len(wav); off += 16000 { // skip WAV header; 0.5 s chunks
		if err := client.SendPCM(wav[off:min(off+16000, len(wav))]); err != nil {
			log.Fatal(err) // ErrReconnecting: drop or retry the frame yourself
		}
		time.Sleep(100 * time.Millisecond) // pace like a live feed
	}
	if err := client.Stop(); err != nil { // finalize; server closes after the final
		log.Fatal(err)
	}
	<-done
}

Check: point any of the three at the repository fixture — python3 stream_wav.py crates/gigastt/tests/fixtures/golos_00.wav — and you get a few ... partials followed by >>> finals of Russian speech, then a clean exit after the trailing final.

Verifying the result

End-to-end sanity in two terminals:

# terminal 1
gigastt serve

# terminal 2 — wait for readiness, then stream the bundled speech fixture
until curl -sf http://127.0.0.1:9876/ready; do sleep 2; done
python3 stream_wav.py crates/gigastt/tests/fixtures/golos_00.wav

You have a healthy streaming integration when all of these hold:

• The client’s first message is ready with version: "1.0", and your configured rate is one of its supported_rates.
• ... partials appear about once a second while audio flows and are lowercase/raw; >>> finals commit enriched text after each pause and carry words[] with timings and (usually) confidence.
• Sending stop yields exactly one more final (possibly empty) and the socket closes only after it — stopping mid-word still recognizes that word.
• ready.max_session_secs / ready.idle_timeout_secs match the values you passed to gigastt serve, and your rotation/backoff logic (Recipe 4/5) survives a forced cap: gigastt serve --max-session-secs 30.

Common pitfalls

Symptom → cause, with a pointer to the fix — the cookbook jump table is Appendix A — Error codes; the full operator table lives in docs/troubleshooting.md and the field-level tables in docs/api.md.

A Deepgram-compatible WebSocket mode (a drop-in endpoint for Deepgram clients) is in progress; this chapter covers the native protocol only.

Links

• Reference (canonical, not duplicated here): docs/api.md — WebSocket protocol, docs/asyncapi.yaml, docs/troubleshooting.md, docs/cli.md
• Client code: examples/ (Python, Bun/TypeScript, Go, Kotlin, Rust), sdks/go, sdks/js
• In this book: Getting started for install and first run, Models and backends for heads and punctuation/ITN behavior, CLI and batch processing for REST/jobs alternatives to live streaming, Deployment & ops for running serve in production, Appendix A — Error codes for REST/WS/close jump table.

Desktop & embedded: Swift/SPM, sidecar, Electron, UniFFI

Scenario

You are building a desktop or mobile app with on-device Russian speech-to-text: a macOS dictation app in Swift, an Electron meeting recorder, or a Kotlin / Python tool. The model must run locally (no cloud), and you have two ways to ship the engine — embed it in-process via a native binding, or run gigastt serve as a sidecar subprocess and talk to it over the loopback HTTP/WS API. This chapter helps you choose, then walks each path to the first transcript.

Choosing a path: embedded vs sidecar

Rules of thumb:

• Embedded when your app owns its whole audio pipeline (a dictation tool, a recorder) and is the only consumer. On iOS it is the only option — apps cannot spawn subprocesses.
• Sidecar when one engine is shared by several clients, when the app must survive an engine crash, or when you want to upgrade the engine without redeploying the app. Both confirmed external desktop integrations use this pattern.

Prerequisites

• A model directory — every path below assumes one. Fetch the pre-quantized INT8 bundle once (~215 MB, no FP32 download, no on-device quantization):

  gigastt download --prequantized          # -> ~/.gigastt/models
  

• Embedded: the toolchain for your binding — Xcode 15+ (Swift), Node.js (npm package), Python 3 (wheel), or an Android SDK/NDK (Kotlin).

• Sidecar: the gigastt binary — bundled inside your app’s resources, or installed (brew tap ekhodzitsky/gigastt https://github.com/ekhodzitsky/gigastt && brew install gigastt, release tarball, or cargo install gigastt) — plus curl for probing.

Recipe — Swift/SPM (iOS + macOS)

The GigaSTT Swift package wraps the C ABI in a safe Swift interface. The native code ships as a prebuilt GigasttFFI.xcframework (iOS device arm64, simulator arm64/x86_64, macOS arm64) with ONNX Runtime statically linked — there is no separate runtime to bundle. Requires iOS 15 / macOS 13 (Apple Silicon only — there is no Intel macOS slice) and Xcode 15+.

1. Add the package. Xcode → File → Add Package Dependencies… → enter the mirror repository URL https://github.com/ekhodzitsky/gigastt-swift and add the GigaSTT product to your target. The mirror is the canonical remote source (SwiftPM needs Package.swift at the repository root, so the monorepo subdirectory packaging/swift cannot be consumed via URL — use it only as a local path dependency for development).

2. Bundle the model. Copy ~/.gigastt/models into your app target as a folder reference (the blue folder — it preserves the directory layout; a yellow “group” flattens the files and the engine will not find them). Alternatively download the model on first launch and cache it.

3. Load the engine and transcribe:

   import GigaSTT
   
   guard let modelDir = Bundle.main.url(
       forResource: "models", withExtension: nil
   )?.path else {
       fatalError("bundle the model directory as a folder reference")
   }
   
   // poolSize: 1 keeps RAM around ~350 MB, recommended on device.
   let engine = try Engine(modelDir: modelDir, poolSize: 1)
   
   // Path is relative to the current working directory; absolute paths and
   // ".." are rejected by the engine.
   let text = try engine.transcribeFile(path: "audio.wav")
   print(text)
   

4. Stream little-endian mono PCM16 chunks at your capture rate (resampled to 16 kHz internally):

   let stream = try Stream(engine: engine)
   
   // pcm16: Data of little-endian Int16 mono samples at 48 kHz.
   for segment in try stream.processChunk(pcm16, sampleRate: 48000) {
       print(segment.text, segment.isFinal)
   }
   
   // Drain the tail at end-of-stream.
   for segment in try stream.flush() {
       print(segment.text)
   }
   

   processChunk and flush return [TranscriptSegment]; each segment carries text, words (per-word word/start/end/confidence and an optional speaker), isFinal, and a timestamp.

5. Handle errors. The wrapper throws GigasttError: engineLoadFailed(modelDir:) (missing/unreadable model directory), streamCreationFailed (pool checkout failed), inferenceFailed, and decodingFailed(underlying:). The C ABI signals failure with NULL returns, so a case tells you where it failed rather than carrying an engine-side message.

Verify: run the app, transcribe a known WAV, and confirm the expected text is printed. If the engine throws engineLoadFailed at boot, the model directory is not where Bundle.main.url(forResource: "models", withExtension: nil) points — see Common pitfalls.

Recipe — sidecar server (macOS / Electron)

Run gigastt serve as a managed child process. The full lifecycle: discover the binary, pre-stage the model, spawn, gate on readiness, transcribe, stop gracefully.

1. Discover the binary, in order: an env override (e.g. MYAPP_GIGASTT_BIN) → a copy bundled in your app’s resources → /opt/homebrew/bin/gigastt → /usr/local/bin/gigastt → PATH. Record which one you picked for logging.

2. Pre-stage the model at install time or first launch, with machine-readable progress for your UI:

   gigastt download --prequantized --progress json
   

   stdout carries one NDJSON event per line ({"phase":"download","file":...,"bytes_done":N,"bytes_total":M}, then verify, then done) and exit codes distinguish network/disk/checksum failures — see docs/cli.md. --prequantized skips the ~2-minute on-device INT8 pass, so the first serve starts in seconds, not minutes.

3. Pick a port. Today: a fixed high port (e.g. 49876). Ephemeral auto-selection (--port 0 with a machine-readable LISTENING line), --die-with-parent, and --log-file are planned server additions — they require a gigastt version that ships them, so do not rely on their syntax yet; use a fixed port and the lifecycle below.

4. Spawn and capture logs. Never send the sidecar’s output to /dev/null — pipe stdout/stderr to a log file (if you connect pipes instead, keep draining them: a full pipe buffer can deadlock the child). Example in an Electron main process / Node:

   import { spawn } from 'node:child_process';
   import fs from 'node:fs';
   
   const PORT = 49876;
   const BASE = `http://127.0.0.1:${PORT}`;
   
   const log = fs.createWriteStream('gigastt-sidecar.log', { flags: 'a' });
   const server = spawn(gigasttBin, [
     'serve',
     '--port', String(PORT),
     '--pool-size', '1',
     '--model-dir', modelDir,
   ], { stdio: ['ignore', log, log] });
   
   async function waitForReady(timeoutMs = 120_000) {
     const deadline = Date.now() + timeoutMs;
     for (;;) {
       try {
         const res = await fetch(`${BASE}/ready`);
         // 200 {"status":"ready","pool_available":N,"pool_total":M}
         if (res.ok) return;
         // 503 {"status":"not_ready","reason":"initializing"} — keep waiting.
       } catch {
         // Connection refused: the listener is not up (yet) — or the process
         // is gone. Distinguish by the child exit code, not by killing it.
         if (server.exitCode !== null) {
           throw new Error(`gigastt exited with code ${server.exitCode}`);
         }
       }
       if (Date.now() > deadline) throw new Error('readiness timeout');
       await new Promise((resolve) => setTimeout(resolve, 500));
     }
   }
   

5. Gate on /ready, not on TCP connect. The server binds the port immediately and answers probes from a bootstrap responder while the model loads, so “port listening” does not mean “ready”. Poll GET /ready until it returns 200; a 503 body carries {"status":"not_ready","reason":"initializing"|"pool_exhausted"|"shutting_down"}. Connection-refused means the process is dead or not yet spawned — not that it is stuck.

6. Version handshake over HTTP. GET /health returns 200 in both phases — {"status":"ok","model":"loading","version":"2.14.1"} during bootstrap, then {"status":"ok","model":"gigaam-v3-rnnt","variant":"rnnt","version":"2.14.1","punctuation":true,"itn":true}. Gate your minimum engine version on the version field instead of running gigastt --version in a subprocess.

7. Transcribe. For live partial results open a WebSocket session on /v1/ws (patterns in Streaming over WebSocket); for whole files POST to /v1/transcribe (recipes in CLI and batch processing). On pool saturation the server answers 503 + Retry-After (REST) or an error with retry_after_ms (WS) — honor it instead of inventing a backoff. Before closing a WS session, send {"type":"stop"} so the server flushes trailing words into a final segment.

8. Stop gracefully. Send SIGTERM: the server drains in-flight sessions for up to --shutdown-drain-secs (default 10), flushing a final and closing WS clients with code 1001. Escalate to SIGKILL only after the drain window. Track the child pid and terminate it when your app exits — an orphaned sidecar holds ~1 GB RSS and the port.

Verify:

gigastt serve --port 49876 --pool-size 1 &
curl -s http://127.0.0.1:49876/ready    # -> {"status":"ready","pool_available":1,"pool_total":1}
curl -s http://127.0.0.1:49876/health   # -> {"status":"ok","model":"gigaam-v3-rnnt",...,"version":"..."}
kill %1                                  # SIGTERM — process exits within the drain window

Recipe — Node/Electron in-process

The gigastt npm package (napi-rs) embeds the engine inside your Node/Electron process — no sidecar, no port, no version gate. Inference runs on a libuv worker thread, so calls return Promises and never block the event loop; onnxruntime is statically linked, so the .node addon is self-contained.

1. Install: npm install gigastt. The postinstall downloads exactly one prebuilt binary (gigastt.<platform>.node, ~47 MB) for the install platform from the GitHub release. Prebuilt platforms: darwin-arm64, linux-x64-gnu, linux-arm64-gnu, win32-x64-msvc (no Intel macOS).

2. Side-load the model — it is not bundled: gigastt download --prequantized → ~/.gigastt/models, or set GIGASTT_MODEL_DIR.

3. Use it:

   const { Engine, Stream } = require('gigastt');
   
   const engine = new Engine('/path/to/gigastt/models'); // new Engine(modelDir, poolSize?)
   const t = await engine.transcribeFile('recording.wav');
   console.log(t.text, t.durationS);
   for (const w of t.words) console.log(w.text, w.startS, w.endS, w.confidence);
   
   // streaming — await each chunk before sending the next to keep order
   const s = new Stream(engine);
   for (const seg of await s.processChunk(pcm16, 16000)) console.log(seg.text);
   console.log((await s.flush()).map((seg) => seg.text));
   
   // errors are thrown JS Errors whose message starts with a stable code
   try { new Engine('/no/such/dir'); } catch (e) { /* e.message starts "ModelNotFound:" */ }
   

4. Electron layout: construct the Engine in the main process (never in a renderer) and keep one Stream per audio channel (e.g. mic + system). Each Stream holds one pool session for its lifetime, so poolSize must cover the number of live streams — a third Stream on a pool of 2 throws PoolExhausted. The full dual-channel pattern with IPC handlers is in examples/electron_main.mjs.

5. Size the thread pool. Inference runs on libuv’s worker pool (default 4 threads, shared with fs/crypto). For N channels transcribing simultaneously, start with UV_THREADPOOL_SIZE >= N (e.g. UV_THREADPOOL_SIZE=4 electron .) and a matching poolSize.

6. Packaging (asar): keep the model directory and the native .node addon out of the asar archive — native code memory-maps the weights and dlopens the addon, neither of which works from asar’s virtual filesystem. With electron-builder, use asarUnpack for **/*.node and ship the model via extraResources.

Verify: in a checkout, GIGASTT_MODEL_DIR=~/.gigastt/models npm run smoke (from crates/gigastt-node) transcribes a test fixture; in your app, await engine.transcribeFile on a known WAV and check the text.

Recipe — Kotlin and Python (UniFFI)

crates/gigastt-uniffi generates idiomatic Swift, Kotlin, and Python bindings from one Rust source with UniFFI. They wrap the synchronous lean core (models side-loaded, no tokio runtime), surface typed GigasttFfiError errors (ModelNotFound, InvalidAudio, PoolExhausted, Inference, InvalidArgument), and manage objects by reference counting.

Shared surface (casing follows each language’s idiom):

• Engine(model_dir) / Engine.new_with_pool_size(model_dir, pool_size) · transcribe_file(path) -> Transcript { text, words, duration_s }
• Stream(engine) · process_chunk(pcm16, sample_rate) -> [TranscriptSegment] · flush() -> [TranscriptSegment]
• TranscriptSegment { text, words, is_final } · Word { text, start_s, end_s, confidence, speaker }

Python — published. pip install gigastt installs a self-contained prebuilt wheel (py3-none-<platform>; onnxruntime statically linked):

import gigastt_uniffi as g

engine = g.Engine("/path/to/gigastt/models")        # side-loaded model dir
t = engine.transcribe_file("recording.wav")
print(t.text)
for w in t.words:
    print(w.text, w.start_s, w.end_s, w.confidence)

# streaming
s = g.Stream(engine)
for seg in s.process_chunk(pcm16_bytes, 16000):
    print(seg.text)
print([seg.text for seg in s.flush()])

# typed errors
try:
    g.Engine("/no/such/dir")
except g.GigasttFfiError.ModelNotFound as e:
    ...

Kotlin — experimental AAR. The Rust cross-build is proven (CI cross-compiles libgigastt_uniffi.so per ABI via cargo-ndk), but the Gradle/Maven AAR assembly and publish have not been validated end-to-end yet. To build locally:

# native libs per ABI
cargo ndk -t arm64-v8a -t armeabi-v7a -t x86_64 \
  -o packaging/android/gigastt/src/main/jniLibs build --release -p gigastt-uniffi
# Kotlin bindings (from a host build of the cdylib; metadata is arch-independent)
cargo build --release -p gigastt-uniffi
cargo run --release -p gigastt-uniffi --bin uniffi-bindgen -- generate \
  --library target/release/libgigastt_uniffi.* --language kotlin \
  --out-dir packaging/android/gigastt/src/main/kotlin
# assemble
cd packaging/android && gradle :gigastt:assembleRelease

Swift (UniFFI): the bindings generate (--language swift), but the shipped SwiftPM package uses the handwritten C-ABI wrapper from the Swift recipe above — prefer it for app development.

To regenerate any binding from a checkout:

cargo build -p gigastt-uniffi
LIB=target/debug/libgigastt_uniffi.dylib   # .so on Linux

cargo run -p gigastt-uniffi --bin uniffi-bindgen -- generate --library "$LIB" --language python --out-dir bindings/python
cargo run -p gigastt-uniffi --bin uniffi-bindgen -- generate --library "$LIB" --language swift  --out-dir bindings/swift
cargo run -p gigastt-uniffi --bin uniffi-bindgen -- generate --library "$LIB" --language kotlin --out-dir bindings/kotlin

Generated bindings are build artifacts (bindings/ is git-ignored). Status and packaging details: crates/gigastt-uniffi/README.md, packaging/android/README.md.

Verify: the Python snippet above prints the transcript of a known file; for Kotlin, assemble the AAR and transcribe from an instrumented test or a debug screen.

Verifying the result

End-to-end checklist, whichever path you took:

• First transcript: your app prints the expected text for a known WAV (any Russian speech file, e.g. a Golos sample).
• Model on disk: ls ~/.gigastt/models shows v3_rnnt_* files (the --prequantized bundle includes v3_rnnt_encoder_int8.onnx).
• Memory: RSS stays in the expected band — roughly 350–400 MB per pool session; a poolSize/pool-size of 1 is the right default on-device.
• Sidecar health: curl -s http://127.0.0.1:<port>/ready returns {"status":"ready",...} and /health reports the version you gate on; SIGTERM shuts the process down within the drain window with a clean log tail.
• Streaming order: partial segments (isFinal == false) may be revised; persist only finals, and call flush() (or send {"type":"stop"} on WS) before tearing down a session so the tail is not lost.

Common pitfalls

• Model not bundled as a folder reference (Swift). Added as a yellow “group”, Xcode flattens the directory and the engine fails at boot with engineLoadFailed. Fix: add models as a folder reference (blue folder) and assert Bundle.main.url(forResource: "models", withExtension: nil) is non-nil in a debug run.
• Undefined std::__1::* symbols at link time (Swift). ONNX Runtime is C++; the current package already declares .linkedLibrary("c++") — this error means you are consuming an old or hand-rolled xcframework. Use the released package from the mirror (or current packaging/swift).
• Blocking call on the UI thread. Engine/Stream calls are synchronous and the handles are not thread-safe. Run all engine work on a dedicated background queue/actor (Swift) and serialize access; never transcribe on the main thread. In Node the calls already run on the libuv pool — just await them.
• Readiness timeouts on the first run (sidecar). A plain gigastt download leaves a ~2-minute on-device INT8 quantization pass to the first serve, and the punctuation model fetches lazily on first start — a client with a 10–30 s boot timeout gives up too early. Fix: pre-stage with gigastt download --prequantized, keep the readiness timeout generous, and branch on the /ready reason instead of killing the process.
• Killing the server on 503. During model load the port is bound by the bootstrap responder and every API answers 503 initializing — that is progress, not a hang. Only connection-refused plus a dead child process means failure.
• Hardcoded port collision. A fixed port already in use (often by an orphaned gigastt from a previous run) surfaces as a silent timeout. Check /health → version to learn whether the listener is your server, and terminate your child on app exit (a --die-with-parent flag is planned).
• Sidecar logs to /dev/null. You will need them the first time the model directory is corrupt. Pipe stdout/stderr to a file — and if you use pipes, keep draining them to avoid a pipe-buffer deadlock.
• require('gigastt') fails after npm install --ignore-scripts. The postinstall that downloads the native binary was skipped; run node install.js inside the package.
• Model or addon inside asar (Electron). Native code memory-maps the weights and dlopens the .node — both need real files. asarUnpack the addon and ship the model via extraResources.
• Absolute path to transcribeFile (Swift). The C ABI accepts only relative paths inside the working directory — point the working directory at the file’s location first (or copy the file there).

Links

In this book:

• Getting started — install and model download
• Streaming over WebSocket — WebSocket protocol patterns for the sidecar
• CLI and batch processing — REST / SSE / jobs recipes

Reference material (do not duplicate — read here):

• packaging/swift/README.md — Swift package API reference
• crates/gigastt-node/README.md — npm package API + in-process vs sidecar trade-offs
• crates/gigastt-uniffi/README.md — UniFFI bindings and generation
• packaging/android/README.md — Kotlin/AAR build status
• examples/electron_main.mjs — full Electron main-process pattern
• docs/quickstarts.md — per-binding quickstarts and availability table
• docs/embedding-packaging.md — onnxruntime static vs dynamic linking
• docs/api.md — /health, /ready, REST/WS/SSE reference
• docs/cli.md — serve and download flags (lifecycle knobs used above)
• Appendix A — Error codes — HTTP/WS codes when the sidecar misbehaves

macOS distribution note. Shipping a notarized .app / Developer ID build is out of band for this workbook: use Apple’s notarization tooling on your signed app + the sidecar or embedded framework you ship. The engine itself is an unsigned open-source binary from GitHub Releases — gate version with /health → version, not with code-signature assumptions.

Deployment & ops

Scenario

You are the admin who runs gigastt on a server, not a laptop. The path of this chapter: install → restrict → observe → upgrade — one supervised service (systemd or Docker), metrics flowing into Prometheus/Grafana, alerts for the failure modes that matter, and an upgrade routine that does not cut live transcription sessions.

Every recipe ends with a Verify step. Flags are checked against gigastt serve --help; the full flag reference lives in docs/cli.md and is not repeated here.

Prerequisites

• gigastt installed (binary, package, or image) — see Getting started.
• A Linux host with 4+ GB RAM: the default --pool-size 2 with the INT8 encoder sits at ~790 MiB RSS; leave headroom for the OS and request peaks.
• For the systemd path: systemd 241 or newer (any modern distro, including Astra Linux, RED OS, ALT) and root access.
• For the Docker path: Docker 20.10+; the NVIDIA Container Toolkit only for the CUDA variant.
• The model either downloadable once (~850 MB FP32, auto-quantized to INT8 on first start) or pre-installed from the offline bundle / model deb.

Recipe

Docker

Each tagged release publishes multi-arch images to GHCR — prefer pulling over building:

docker pull ghcr.io/ekhodzitsky/gigastt:2.14.1        # CPU, linux/amd64 + linux/arm64
docker pull ghcr.io/ekhodzitsky/gigastt:2.14.1-cuda   # CUDA, linux/amd64

Pin a concrete tag for reproducible deploys; :latest / :cuda float.

Run with a named volume so the ~850 MB model (and the auto-generated INT8 encoder) survives container replacement:

docker run -d --name gigastt \
  -p 127.0.0.1:9876:9876 \
  -v gigastt-models:/home/gigastt/.gigastt/models \
  ghcr.io/ekhodzitsky/gigastt:2.14.1

Notes:

• The image’s default command is serve --port 9876 --host 0.0.0.0 --bind-all (container networking needs it); -p 127.0.0.1:9876:9876 keeps the host-side exposure on loopback. Put your TLS proxy in front exactly as with a bare install.
• The container runs as the unprivileged gigastt user; the model directory inside is /home/gigastt/.gigastt/models — that is the volume mount point.
• The image carries a HEALTHCHECK on /health, so docker ps shows healthy as soon as the port serves. During the first-run model download and INT8 quantization (~2 min) /health answers 200 with model:"loading" while /ready answers 503 {"status":"not_ready","reason":"initializing"} — gate traffic on /ready, not /health.
• Baked image (zero cold start, +~850 MB): build locally with the model inside — docker build --build-arg GIGASTT_BAKE_MODEL=1 -t gigastt:baked .
• CUDA: docker run --gpus all -p 127.0.0.1:9876:9876 ghcr.io/ekhodzitsky/gigastt:2.14.1-cuda (requires the NVIDIA Container Toolkit; the binary falls back to CPU when no GPU is present).

Verify:

curl -s http://127.0.0.1:9876/ready
# {"status":"ready","pool_available":2,"pool_total":2}
curl -s http://127.0.0.1:9876/health
# {"status":"ok","model":"gigaam-v3-rnnt","variant":"rnnt","version":"2.14.1","punctuation":true,"itn":true}

Air-gapped / offline installation

For hosts with no internet access, each release ships a self-contained tarball per Linux target — binary + pre-quantized INT8 rnnt model + punctuation model + systemd unit + installer — and two Debian packages (gigastt_<ver>_<arch>.deb + gigastt-model-int8_<ver>_all.deb). The full bundle inventory lives in README-OFFLINE.md and is not repeated here.

On a connected machine, download and verify before carrying the files over (the why and the threat model: docs/verifying-releases.md):

gh release download v2.14.1 -R ekhodzitsky/gigastt \
    -p 'gigastt-2.14.1-offline-x86_64-unknown-linux-gnu.tar.gz' \
    -p 'gigastt-2.14.1-offline-x86_64-unknown-linux-gnu.tar.gz.sha256' \
    -p 'gigastt-2.14.1-offline-x86_64-unknown-linux-gnu.tar.gz.minisig'
sha256sum -c gigastt-2.14.1-offline-x86_64-unknown-linux-gnu.tar.gz.sha256
minisign -Vm gigastt-2.14.1-offline-x86_64-unknown-linux-gnu.tar.gz -p gigastt.pub
gh attestation verify gigastt-2.14.1-offline-x86_64-unknown-linux-gnu.tar.gz \
    --repo ekhodzitsky/gigastt

On the target host:

tar xf gigastt-2.14.1-offline-x86_64-unknown-linux-gnu.tar.gz
cd gigastt-2.14.1-offline
sudo ./install.sh    # verifies SHA256SUMS.txt, then installs binary + models + unit
sudo systemctl enable --now gigastt

Debian-family alternative:

sudo dpkg -i gigastt_2.14.1_amd64.deb gigastt-model-int8_2.14.1_all.deb
sudo systemctl enable --now gigastt

The model is already INT8 — no download, no quantization step, no network. The installed unit sets GIGASTT_OFFLINE=1 via /etc/gigastt/gigastt.env, so any code path that would fetch a model (enabling --vad, diarization, an alternative recognition head) fails fast with an error naming the file to provide instead of hanging on a connect timeout. To add optional models later, run gigastt download on a connected machine and copy the files into /usr/share/gigastt/models/.

Typical failures:

• install.sh aborts with sha256sum: WARNING: 1 computed checksum did NOT match — the tarball was corrupted on its way to the air-gapped host. Re-verify the outer .sha256, re-copy, re-run; nothing is installed partially.
• An offline-mode error naming a missing file (e.g. the VAD model after enabling --vad) — that model is not in the bundle; fetch it on a connected machine and copy it over.

Verify:

systemctl is-active gigastt
# active
curl -s http://127.0.0.1:9876/health
# {"status":"ok",...} — served immediately, the model is pre-installed

systemd service

The hardened unit ships in packaging/systemd/ and is installed by both the deb and the offline bundle. Key properties (the unit itself is short and commented — read it for the full list):

• Runs as the unprivileged gigastt user; models under /usr/share/gigastt/models are read-only to it.
• Loopback bind only (127.0.0.1:9876); expose the API through a reverse proxy.
• Restart=on-failure, RestartSec=5 — a crash is restarted, a clean systemctl stop is not.
• Hardening set compatible with systemd 241 (ProtectSystem=strict, NoNewPrivileges, PrivateTmp, …), so the unit works unmodified on Astra Linux, RED OS and ALT.
• Overrides live in /etc/gigastt/gigastt.env (GIGASTT_* variables, RUST_LOG), loaded via EnvironmentFile.

Logs go to the journal:

journalctl -u gigastt -f          # follow
journalctl -u gigastt -n 100      # recent

Change log verbosity by editing /etc/gigastt/gigastt.env (RUST_LOG=gigastt=debug), then sudo systemctl restart gigastt.

Change flags with a drop-in — never edit the shipped unit (package upgrades replace it). ExecStart must be cleared before being re-set:

# sudo systemctl edit gigastt
[Service]
ExecStart=
ExecStart=/usr/bin/gigastt serve --model-dir /usr/share/gigastt/models --punct-model-dir /usr/share/gigastt/models/punct --metrics

systemctl restart gigastt sends SIGTERM; the server drains live WebSocket/SSE sessions — each client receives a Final frame + Close(1001 Going Away) — for up to --shutdown-drain-secs (default 10 s), well inside systemd’s default 90 s stop timeout. How to use this for version bumps: Upgrades and rollback below.

Verify:

systemctl status gigastt --no-pager
curl -s http://127.0.0.1:9876/health

Observability

Metrics are opt-in and served on a separate listener — never on the API port, so they sit outside the CORS allowlist and the per-IP rate limiter:

gigastt serve --metrics                                  # http://127.0.0.1:9090/metrics
gigastt serve --metrics --metrics-listen 127.0.0.1:9100  # custom port

Keep the listener on loopback unless your Prometheus runs on another host — and even then bind it to a trusted interface, never a public one.

Minimal Prometheus wiring (prometheus.yml):

scrape_configs:
  - job_name: gigastt
    static_configs:
      - targets: ["127.0.0.1:9090"]

rule_files:
  - /etc/prometheus/rules/gigastt-alerts.yml   # copy of docs/observability/alerts.yml

The metrics that matter (all prefixed gigastt_):

Ready-made assets — import, don’t reinvent:

• docs/observability/alerts.yml — Prometheus rules: 5xx above 5%, gigastt_pool_available == 0 for 1m, p95 above 10 s, sustained pool timeouts, health probe down (blackbox exporter).
• docs/observability/dashboard.json — Grafana dashboard (Dashboards → Import): request rate, latency, 5xx, pool availability, active WebSockets, inference duration, rate-limit rejections.

What to alert on in practice: pool saturation (gigastt_pool_available == 0 sustained — clients are getting 503s), 5xx ratio, and RAM at the node level (gigastt exports no self-RSS metric; use node_exporter or cAdvisor).

Logs: tracing env-filter via RUST_LOG (default gigastt=info; gigastt=debug for triage). Logs carry request metadata — durations, word counts — never transcript text (docs/privacy.md).

Verify:

curl -s http://127.0.0.1:9876/ready > /dev/null   # samples the pool gauges once
curl -s http://127.0.0.1:9090/metrics | grep '^gigastt_pool_available'
# gigastt_pool_available 2

Secure by default

The defaults are already the hardened posture — this recipe is mostly about not weakening them:

• Loopback bind. serve refuses non-loopback addresses unless --bind-all / GIGASTT_ALLOW_BIND_ANY=1 is set. Remote access = a TLS-terminating reverse proxy on the same host (Caddy/nginx configs: docs/deployment.md).
• Origin allowlist. Loopback origins are always allowed; every other Origin must be listed via --allow-origin (repeatable, exact match). --cors-allow-any is development-only. Disallowed origins get 403.
• Request limits. --body-limit-bytes (default 50 MiB), --ws-frame-max-bytes (512 KiB), --idle-timeout-secs (300), --max-session-secs (3600), --inference-timeout-secs (600), --pool-checkout-timeout-secs (30) — 503 + Retry-After on pool saturation.
• Rate limiting (opt-in): --rate-limit-per-minute N with --rate-limit-burst → 429 + Retry-After. Behind a proxy it works per-client only if the proxy overwrites X-Forwarded-For and --trust-proxy is set — copy the proxy snippets in docs/deployment.md verbatim.
• Model integrity. Downloads are SHA-256-verified and atomically renamed (.partial → final); a corrupt file is never promoted.
• Release verification. Every release asset has a .sha256 sidecar + SHA256SUMS.txt, a minisign signature, a CycloneDX SBOM, and SLSA build provenance. Verify before installing anything — docs/verifying-releases.md. Minimum routine:

minisign -Vm gigastt-2.14.1-x86_64-unknown-linux-gnu.tar.gz -p gigastt.pub
gh attestation verify gigastt-2.14.1-x86_64-unknown-linux-gnu.tar.gz \
    --repo ekhodzitsky/gigastt

• Privacy. No telemetry, no outbound calls after the one-time model download, transcripts never logged (docs/privacy.md).

Verify:

ss -ltnp | grep 9876
# tcp LISTEN 0 ... 127.0.0.1:9876 ...   (loopback only)
curl -s -o /dev/null -w '%{http_code}\n' \
    -H 'Origin: https://attacker.example' http://127.0.0.1:9876/v1/models
# 403

Hot-reload the model without restart

When you replace model files on disk (new INT8 encoder, switched head files, refreshed punct model) you can rebuild the engine in place without stopping serve:

# Must be called from loopback — non-loopback peers get 403 loopback_only
# even with --bind-all.
curl -s -X POST http://127.0.0.1:9876/v1/admin/reload
# {"reloaded":true,"variant":"rnnt","encoder":"int8"}

The server rebuilds from the boot recipe (model dir, pool sizes, punct / ITN / VAD / hotwords), warms the new engine, then atomically swaps it in. In-flight requests finish on the old engine. A build failure leaves the previous model serving (503 reload_failed). Concurrent reloads return 409 reload_in_progress. Full contract: docs/api.md — Admin reload.

Verify:

curl -s -X POST http://127.0.0.1:9876/v1/admin/reload | tee /tmp/reload.json
python3 -c "import json; d=json.load(open('/tmp/reload.json')); assert d['reloaded'] is True"
# From a non-loopback bind (only if you deliberately opened one): expect 403.

Upgrades and rollback

Pin what you deploy (image tag, deb version) so an upgrade is a deliberate, reversible step. The model directory is state: it persists across upgrades, the engine auto-detects the installed recognition head, and no silent re-download happens when you bump the binary. Prefer resolving the latest release when scripting installs:

TAG=$(gh api repos/ekhodzitsky/gigastt/releases/latest -q .tag_name)   # e.g. v2.14.1
VER=${TAG#v}

Docker (upgrade to the pin you chose — here 2.14.1):

docker pull ghcr.io/ekhodzitsky/gigastt:2.14.1
docker stop --time 15 gigastt && docker rm gigastt
docker run -d --name gigastt \
  -p 127.0.0.1:9876:9876 \
  -v gigastt-models:/home/gigastt/.gigastt/models \
  ghcr.io/ekhodzitsky/gigastt:2.14.1

docker stop sends SIGTERM; --time 15 gives the drain window (--shutdown-drain-secs, default 10 s) room to finish before SIGKILL — Docker’s default of 10 s races the drain. Clients receive Final + Close(1001) and reconnect; short REST uploads in flight may need a retry.

systemd / deb:

sudo dpkg -i gigastt_2.14.1_amd64.deb
sudo systemctl restart gigastt
journalctl -u gigastt -f    # expect a clean drain, no "Drain window expired"

On Kubernetes the same rule applies from the orchestrator side: terminationGracePeriodSeconds ≥ shutdown_drain_secs + 5 (full manifest: docs/deployment.md).

Rollback. Re-deploy the previous tag or package — the on-disk model set is unchanged, so the old binary starts against the same files:

docker run -d --name gigastt \
  -p 127.0.0.1:9876:9876 \
  -v gigastt-models:/home/gigastt/.gigastt/models \
  ghcr.io/ekhodzitsky/gigastt:2.14.0
# or: sudo dpkg -i gigastt_2.14.0_amd64.deb && sudo systemctl restart gigastt

If a drain-related regression breaks your WebSocket clients after an upgrade, the escape hatch is --shutdown-drain-secs 0 (clamped to 1 s) — see docs/runbook.md for the full symptom table.

Verify (after every upgrade or rollback):

curl -s http://127.0.0.1:9876/health
# "version" is the release you deployed; "model"/"variant" are unchanged
curl -s http://127.0.0.1:9876/ready

Verifying the result

End-to-end smoke after any recipe above:

systemctl is-active gigastt || docker ps --filter name=gigastt --format '{{.Status}}'
curl -s http://127.0.0.1:9876/health     # status ok, expected version
curl -s http://127.0.0.1:9876/ready      # ready, pool_available >= 1
curl -s http://127.0.0.1:9090/metrics | grep '^gigastt_pool_available'

Then transcribe one short file through the API you actually expose (REST recipes: CLI and batch processing; CLI check: Getting started).

Common pitfalls

• OOM — container or service killed. RSS scales with --pool-size: the INT8 encoder is ~400 MiB per triplet, ~790 MiB at the default pool of 2; the FP32 encoder is ~4× larger (never pass --skip-quantize in production). On a 4 GB box keep --pool-size at 1–2; --pool-min-size 1 lets the server boot on a degraded pool instead of crashing when memory is tight. If Kubernetes reports OOMKilled, lower the pool or raise the pod limit — details in docs/runbook.md.
• 503 timeout under load. Every triplet is busy and a caller waited out --pool-checkout-timeout-secs (30 s): REST gets 503 + Retry-After, WebSocket gets an error with retry_after_ms. This is backpressure, not a bug — raise --pool-size, split batch work off with --batch-pool-size, and watch gigastt_pool_waiters / gigastt_pool_timeouts_total.
• /metrics unreachable from the Prometheus host. By design: the listener defaults to 127.0.0.1:9090. Point the scraper at the gigastt host itself, or deliberately re-bind with --metrics-listen on a trusted interface — never a public one. A scraper still aimed at :9876/metrics gets 404: metrics moved off the API port.
• Readiness probes flapping on first start. The first run downloads ~850 MB and quantizes (~2 min). /health returns 200 with model:"loading" the whole time, but /ready returns 503 initializing. If your load balancer routes on /health, early clients get 503s — probe /ready, or pre-install / bake the model so the window disappears.
• Rate limiter punishing everyone behind a proxy. Without --trust-proxy — and a proxy that overwrites rather than appends X-Forwarded-For — all clients share one bucket keyed on the proxy’s address. Symptoms and the exact proxy configuration: docs/deployment.md.

Links

• Getting started — install and first transcription
• CLI and batch processing — REST / SSE / jobs recipes
• Streaming over WebSocket — WebSocket protocol patterns
• Appendix A — Error codes — HTTP/WS/close jump table
• Appendix B — Offline checklist — air-gapped operator list
• docs/deployment.md — reverse proxy, TLS, Kubernetes manifests
• docs/runbook.md — symptom → cause → escape hatch
• docs/cli.md — full serve flag reference
• docs/observability/alerts.yml — Prometheus alerting rules
• docs/observability/dashboard.json — Grafana dashboard
• docs/verifying-releases.md — minisign, SBOM, SLSA provenance
• docs/privacy.md — what data moves where
• packaging/systemd/ — unit + env file
• packaging/offline/README-OFFLINE.md — offline bundle contents

Models and backends

Scenario

You have gigastt running with the default rnnt head on the CPU backend, and now you need to make an informed change: a different recognition head (punctuation baked in, or languages beyond Russian), a leaner model download, a faster execution provider for your hardware, or a bigger session pool. This chapter answers four questions with checkable recipes: which head, INT8 or FP32, which backend, and how much RAM the pool needs.

WER and RTF numbers are not duplicated here — the canonical, CI-annotated tables live in docs/benchmarks.md; this chapter links into them. Flags are checked against gigastt <command> --help; the full flag reference lives in docs/cli.md.

Prerequisites

• gigastt installed (binary, package, or image) — see Getting started.
• Disk: ~1.1 GB free for the default FP32-download path (FP32 set + the generated INT8 encoder), or ~250 MB for the lean pre-quantized path.
• For building a non-default backend from source: Rust 1.88+ and protoc on PATH (build requirements: docs/architecture.md).

Recipe

Choosing a recognition head

The recognition head is selected with --model-variant (env GIGASTT_MODEL_VARIANT) on serve / download / transcribe. All heads share the mel frontend and the 16 kHz mono input contract; they differ in ONNX files, vocabulary, and decoding.

Two hard constraints the table implies:

• rnnt / e2e_rnnt have a Cyrillic-only vocabulary — they cannot transcribe English at all. For English (or kk/ky/uz) the Multilingual heads are the only option.
• The Multilingual heads are encoder-only CTC: they always ship and run as pre-quantized INT8 — there is no FP32 download and no on-device quantization step for them.

Download and run a different head:

gigastt download --model-variant e2e_rnnt
gigastt serve --model-variant e2e_rnnt

Auto-detection rules (from crates/gigastt-core/src/model/mod.rs):

• No --model-variant → the engine detects the installed head from the files in the model directory and uses a complete install as-is (no network request at all).
• An explicit --model-variant that differs from the installed head → the requested set is downloaded alongside; variants are never mixed within one inference, and a warning is logged. When several heads’ files coexist and no variant is requested, auto-detect prefers rnnt.
• An empty model directory + no flag → the default rnnt is downloaded.

Verify:

curl -s http://127.0.0.1:9876/health
# {"status":"ok","model":"gigaam-v3-e2e-rnnt","variant":"e2e_rnnt",...}
curl -s http://127.0.0.1:9876/v1/models
# .id / .name reflect the loaded head; .encoder reports int8 vs fp32

Readable text: punctuation, ITN, and hotwords

The default rnnt head emits bare lowercase. Readable Russian needs a post-pass (or the e2e_rnnt head, which bakes punctuation/casing/ITN in).

# Explicit readable pipeline on the default head
gigastt serve --punctuation on --itn on
curl -s http://127.0.0.1:9876/health
# "punctuation":true,"itn":true

# Per-request override (409 punctuation_not_available if model missing and forced on)
curl -s -X POST "http://127.0.0.1:9876/v1/transcribe?punctuation=true&itn=true" \
  -H "Content-Type: application/octet-stream" --data-binary @clip.wav

Hotword bias lifts brand names and domain terms that the model otherwise misses. One phrase per line; optional built-in Russian brand/acronym lexicon:

cat > /tmp/hotwords.txt <<'EOF'
гигачат
сбер
еком
EOF

gigastt serve --hotwords-file /tmp/hotwords.txt --hotwords-default \
  --hotwords-boost 5.0
# Same flags work on `transcribe` / `transcribe-batch` / `watch` offline.

Verify: transcribe a clip that says a listed brand; without the file the hypothesis often mangled the token, with hotwords the spelling matches the list. Boost too high can invent brands from noise — start at the default 5.0 and only raise if needed. Flags: docs/cli.md.

INT8 or FP32

Short answer: INT8, always, unless you are debugging the model itself. The INT8 encoder runs as true integer compute (DynamicQuantizeLinear + MatMulInteger/ConvInteger kernels), shrinks the encoder 844 MB → 215 MB (~3.9×), and measures ~0% WER degradation — numbers and methodology: docs/benchmarks.md.

What happens on each path (RNN-T heads):

• Default — gigastt download (or the first serve / transcribe) fetches the FP32 set from HuggingFace (istupakov/gigaam-v3-onnx), then runs the native-Rust quantization pass once (~2 min) and writes v3_rnnt_encoder_int8.onnx next to it. The engine prefers the INT8 encoder whenever the file is present.
• Lean — gigastt download --prequantized fetches the pre-quantized INT8 bundle (INT8 encoder + decoder + joiner + vocab) from the pinned GitHub Release instead: no ~844 MB FP32 download, no ~2-minute quantization. Also the fallback when HuggingFace is unreachable but GitHub is not.
• Multilingual — ml_ctc / ml_ctc_large download istupakov’s pre-quantized INT8 encoder directly from HuggingFace; --prequantized is a no-op refinement for them (there is no separate bundle).
• Manual — gigastt quantize [--force] re-runs the quantization on the head detected in --model-dir (e.g. after replacing the FP32 encoder with your own fine-tune).
• Opt out — --skip-quantize (env GIGASTT_SKIP_QUANTIZE=1) skips the quantization step; the engine then loads the FP32 encoder at ~4× the RAM per pool slot and slower CPU kernels. Keep this for debugging only.

Verify:

ls ~/.gigastt/models/
# v3_rnnt_encoder_int8.onnx present alongside decoder/joint/vocab
gigastt transcribe sample.wav 2>&1 | grep 'transcribe complete'
# ... encoder=int8/cpu ... rtf=0.1xx

The encoder=int8/<backend> field in the completion log line is the ground truth for which encoder file was loaded.

Execution provider for your hardware

The backend is a compile-time Cargo feature, not a runtime flag — the provider is baked into the binary you install or build:

Build the one that matches:

cargo build --release                      # CPU, any platform
cargo build --release --features coreml    # macOS ARM64
cargo build --release --features cuda      # Linux x86_64 + NVIDIA (CUDA 12+)
cargo build --release --features nnapi     # Android / ARM64

Exclusivity is enforced at compile time: coreml and cuda are mutually exclusive; ane conflicts with coreml/cuda/nnapi/candle; candle conflicts with coreml/cuda. A compile_error! fires on a bad combination.

Runtime behavior worth knowing:

• CPU fallback is deliberate, never a crash. The CoreML build runs a ~1 s warmup probe at startup; on failure it logs falling back to CPU execution provider and rebuilds the sessions on CPU. The CUDA binary likewise runs on CPU when no GPU is visible (e.g. a container started without --gpus all).
• CUDA packaging. There is no prebuilt CUDA tarball — the release matrix ships CPU binaries for Linux (x86_64, aarch64), Windows, and a CoreML-enabled macOS ARM64 binary. For GPU use the published ghcr.io/ekhodzitsky/gigastt:<ver>-cuda image (recipes: Deployment & ops) or build with Dockerfile.cuda.
• Native ANE backend (--features ane, macOS ARM64): runs the rnnt encoder on the Neural Engine via per-bucket fixed-shape Core ML packages, ~10× warm end-to-end over the CPU build (decode-bound). Fetch the packages with gigastt download --ane (into ~/.gigastt/models/ane/); an e2e_rnnt model transparently falls back to the ort encoder, and streaming windows always take the CPU path — ANE is a file-mode accelerator. Full design and honest numbers: docs/ane-backend.md.
• Candle backend (--features candle, experimental): pure-Rust inference on the Metal GPU, byte-for-byte parity with ort, rnnt head only, FP32 weights converted once with scripts/convert_gigaam_candle.py. Details: docs/candle-backend.md.

Verify:

gigastt transcribe sample.wav 2>&1 | grep 'transcribe complete'
# encoder=int8/coreml  → the CoreML build is active (int8/cuda, int8/cpu, ...)
# encoder=int8/ane     → the native ANE backend handled the encoder

On a CoreML build, also watch startup: the absence of falling back to CPU execution provider means the warmup probe passed.

Model files in an offline perimeter

Everything the engine needs lives in one directory — ~/.gigastt/models/ by default, overridable with --model-dir on serve / download / transcribe / quantize. That makes the model set a plain file artifact you can stage and move around:

# On a connected machine:
gigastt download --prequantized --model-dir /srv/gigastt-models

# Copy to the target host any way you like (rsync, USB, artifact store):
rsync -a /srv/gigastt-models/ offline-host:/srv/gigastt-models/

# On the offline host — refuse every network fetch, fail fast instead of hanging:
gigastt --offline serve --model-dir /srv/gigastt-models

Properties that make this safe:

• Integrity is verified by the binary, not by you. Every download is SHA-256-checked against checksums pinned in the code, staged as .partial, and atomically renamed into place; a corrupt file is removed, never promoted. A checksum failure exits with code 65 (network 69, disk 74, Ctrl-C 130) — scriptable.
• A complete model set means zero network. With a full head’s files present (encoder — INT8 or FP32 — plus decoder/joiner/vocab for the RNN-T heads, or INT8 encoder + vocab for the Multilingual ones), startup performs no network request, even without --offline. --offline / GIGASTT_OFFLINE=1 turns any missing optional model (punctuation, VAD, diarization) into an immediate error naming the file to provide.
• The head is auto-detected from the files — a copied directory needs no --model-variant flag on the target host.
• When copying, include the *_int8.onnx encoder (or copy the FP32 encoder too and run gigastt quantize --model-dir … on the target), the vocab, and — for rnnt/e2e_rnnt — the decoder and joiner.

For a fully packaged offline install (tarball with binary + INT8 model + punctuation model + systemd unit, or the two-deb variant), use the release offline bundle — recipe and verification steps in Deployment & ops; inventory in README-OFFLINE.md.

Verify:

gigastt --offline transcribe sample.wav --model-dir /srv/gigastt-models
# transcribes with no network; a missing file errors immediately, naming the file

Sizing the session pool (RAM)

Every pool slot deserializes its own encoder copy, so RSS scales linearly with --pool-size (default 2). The engine budgets each slot at roughly 2 × encoder-file-size resident (measured ~1.9× on the INT8 rnnt encoder, CPU provider, release build):

Two safety nets are built in:

• RAM auto-cap. At load, the requested pool is clamped so the pooled encoders stay under half of total RAM — a Capping pool size N -> M warning is logged when it fires. The cap never raises your request, and never goes below 1.
• Degraded boot. --pool-min-size 1 (the default) lets the server start on a partially loaded pool instead of crashing when memory runs out mid-load.

Rule of thumb: RAM ≥ pool_size × per-slot + ~1 GB for the OS and request peaks. On a 4 GB box that means --pool-size 1–2 with the INT8 encoder — the same conclusion as the OOM pitfall in Deployment & ops.

Verify:

# no "Capping pool size" warning at startup, and:
curl -s http://127.0.0.1:9876/ready
# {"status":"ready","pool_available":2,"pool_total":2}

Verifying the result

End-to-end smoke after any change in this chapter:

ls ~/.gigastt/models/                  # the head's full file set, incl. *_int8.onnx + vocab
gigastt transcribe sample.wav 2>&1 | grep 'transcribe complete'
# encoder=<int8|fp32>/<cpu|coreml|cuda|ane|candle>, rtf well under 1.0
curl -s http://127.0.0.1:9876/health   # "model"/"variant" match the head you picked
curl -s http://127.0.0.1:9876/ready    # ready, pool_available >= 1

Then confirm the accuracy expectation against docs/benchmarks.md rather than re-measuring casually — the harness, manifests, and normalization matter more than the stopwatch.

Common pitfalls

• English audio comes out as garbage with the default head. Not a bug: rnnt / e2e_rnnt have a Cyrillic-only vocabulary and cannot emit Latin text at all. Switch to --model-variant ml_ctc (or ml_ctc_large).
• --model-variant seems to be ignored after the first download. The engine auto-detects the head from the model directory; an explicit variant that differs from the installed one triggers a second download alongside (with a variants are never mixed warning), and a later flag-less start prefers rnnt again. Delete the unused head’s files to keep the directory unambiguous and reclaim disk.
• First serve looks hung. That is the one-time ~850 MB FP32 download + ~2 min quantization; /health answers 200 with model:"loading" while /ready stays 503 initializing. Skip the window with gigastt download --prequantized, and gate clients on /ready, never on /health.
• OOM after switching to ml_ctc_large. Each slot now costs ~1.2 GB. Lower --pool-size, keep --pool-min-size 1 so a tight host boots degraded, and watch for the Capping pool size warning at startup.
• The CoreML build is no faster than CPU. Look for falling back to CPU execution provider in the startup log — the warmup probe failed and the engine is (deliberately) running on CPU. The completion log’s encoder=int8/cpu field confirms it.
• The CUDA container runs at CPU speed. The GPU is not visible: the container needs the NVIDIA Container Toolkit and --gpus all. The binary falls back to CPU silently — check encoder=int8/cuda in the completion log.
• error: ane and coreml are mutually exclusive (or similar) at build time. Backend features conflict by design; build exactly one of coreml / cuda / ane / candle (nnapi is the exception).
• SHA-256 mismatch during gigastt download. The staged download was corrupt or tampered with; it is removed, not promoted, and the CLI exits 65. Re-run the command — do not hand-rename a .partial file into place.

Links

• Getting started — install and first transcription
• CLI and batch processing — throughput/memory recipe for the offline CLI
• Deployment & ops — offline bundle, systemd, OOM runbook entries
• docs/benchmarks.md — canonical WER / RTF / footprint tables
• docs/architecture.md — pipeline, providers, quantization internals
• docs/cli.md — full flag reference
• docs/ane-backend.md — native Apple Neural Engine backend
• docs/candle-backend.md — Candle/Metal backend
• docs/verifying-releases.md — release artifact verification
• packaging/offline/README-OFFLINE.md — offline bundle inventory

Appendix A — Error codes and close codes

Jump table from symptom → code → what to do. Full field-level contracts stay in docs/api.md and docs/troubleshooting.md; this page is the cookbook index.

Before you dig into codes

Gate liveness on /health, readiness on /ready — never on “TCP port is open”.

REST / SSE / jobs

WebSocket error codes

WebSocket close codes

Links

• docs/api.md — canonical tables
• docs/troubleshooting.md — long symptom list
• docs/runbook.md — operator escapes
• Streaming · Deployment · CLI/jobs

Appendix B — Offline / air-gapped checklist

Use this when the host must not reach HuggingFace or GitHub at runtime. Detailed install recipes live in Getting started and Deployment & ops; this page is the operator checklist.

On a connected machine

• Resolve release tag: TAG=$(gh api repos/ekhodzitsky/gigastt/releases/latest -q .tag_name) (or pin v2.14.1)
• Download offline bundle for the target arch
  (gigastt-${VER}-offline-x86_64-unknown-linux-gnu.tar.gz or aarch64)
  or deb pair: gigastt_…_amd64.deb + gigastt-model-int8_…_all.deb
• Download .sha256 (+ .minisig if you verify with minisign)
• Verify checksums (sha256sum -c) and optional docs/verifying-releases.md
• If you need speaker diarization, also fetch wespeaker_resnet34.onnx (not always in the lean offline bundle) on a connected host and copy it into the model dir later — or accept mono transcripts without labels
• If you need Silero VAD on the air-gapped host, fetch the VAD model the same way (--vad would otherwise try the network)

On the air-gapped host

• Install binary + model (installer script, dpkg -i, or unpack tarball)
• Confirm model files under ~/.gigastt/models/ (or your --model-dir)
• Run with offline guard when scripting: GIGASTT_OFFLINE=1 or global --offline so a missing file fails fast instead of hanging on DNS
• Smoke: gigastt transcribe sample.wav (no network)
• Server: gigastt serve → curl http://127.0.0.1:9876/ready
• systemd: enable unit from the bundle; bind remains loopback unless you explicitly set --bind-all / GIGASTT_ALLOW_BIND_ANY=1

Runtime rules that stay true offline

After copy-in of new model files

# No restart required if serve is already up:
curl -s -X POST http://127.0.0.1:9876/v1/admin/reload
# {"reloaded":true,"variant":"rnnt","encoder":"int8"}

See Hot-reload.

Links

• Getting started — air-gapped
• Deployment — offline install
• packaging/offline/README-OFFLINE.md
• docs/privacy.md — runtime vs build-time network