Repository navigation
Production streaming TTS + Dictation API, called as documented - #279
Conversation
…oices The TTS gateway is retiring the bare `voice=<name>` query parameter in favour of `voice_id` (preset jane is `aai-jane`, library voices `aai-<name>`). Once that release reaches sandbox000, `assembly speak` / `assembly dub` — and the `live` cascade, the `--show-code` cascade script, and the `agent-cascade` init template, all of which build the same socket URL — would fail with a 3006 invalid_parameter. - `tts/session.py`: `SpeakConfig.query_params` now encodes the voice as `voice_id=aai-<name>` via the new `tts.voices.voice_id` (idempotent, so a copied `aai-jane` works too); the one place the param is built, so speak, dub, and live move together. `voice_name` strips the prefix so `live --voice aai-jane` still validates against the catalog. - `code_gen/agent_cascade.py` and the `agent_cascade` template send `voice_id` as well (the template's knob is renamed `VOICE_ID = "aai-jane"`). - New `tts/voice_list.py` fetches `GET /v1/voices` from the TTS host (raw-key auth like the socket; auth/HTTP/shape failures normalized to CLIErrors) and renders one line per voice. `--list-voices` on `speak`, `dub`, and `live` prints it (`--json` dumps the raw records). The flag is one shared eager option (`options.list_voices_option`) that exits in its callback, so `dub --list-voices` needs no `--lang`; `live --list-voices` therefore now needs `--sandbox` + a key instead of printing the static catalog. - `--tts-config voice_id=…` is reserved like `voice=` (it would be silently overridden by `--voice`). - The static `tts/voices.py` catalog keeps driving language defaults, rotation, validation, and completion. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NoQ5jA9AJQ8wPWZbqbLuNn
Streaming TTS is now in production, and both voice APIs are called exactly as documented. Streaming TTS (https://www.assemblyai.com/docs/tts): - Production gets a TTS host, wss://tts.us.assemblyai.com/v1/ws, so speak, dub, and live run without --sandbox (the [sandbox] help tag and the sandbox-only copy are gone). - The catalog is the 11 documented built-in voices, each of which speaks all six languages; the old 26-voice single-language catalog listed 10 voices the service doesn't have (incl. juergen, the German default) and mislabeled others. `language` is sent as the lowercase English name (`de` -> `german`) and `encoding=pcm_s16le` is pinned. A non-built-in --voice (UUID, cloned voice slug) passes through unprefixed. - Long text goes out as bounded Generate+Flush requests (<=3 in flight) on one socket instead of a socket per chunk (which hits 3009), and sessions end Terminate -> Termination. Unknown frames are ignored; is_final is gone. - --list-voices shows the built-ins plus the account's own voices from GET /v1/voices (records keyed by slug, paginated via next_cursor). - The agent-cascade template stopped waiting on is_final (the service never sends it), which hung each sentence until the idle timeout; it now ends on FlushDone. Dictation (https://www.assemblyai.com/docs/dictation): - `assembly dictate` moves from the Sync STT API to the Dictation API: POST dictation.assemblyai.com/v1/transcribe/live with the config part first (audio-first is a 400), language_codes/stt_prompt/keyterms_prompt. - It prints the send-ready cleanup rewrite by default (falling back to the verbatim transcript, with a warning, when the rewrite fails); --verbatim, --instruction, and --rewrite-mode are new; --keyterm replaces --word-boost (kept as a hidden deprecated alias). --json adds llm_response/llm_error. Models: live defaults to claude-haiku-4-5-20251001 (kimi-k2.5 is not served by the production gateway); retired ids drop out of the known-models list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| pending = list(texts) | ||
| in_flight = 0 | ||
| while pending and in_flight < _MAX_IN_FLIGHT: | ||
| _send_request(ws, pending.pop(0)) |
There was a problem hiding this comment.
Removing each request with pending.pop(0) shifts all remaining chunks, causing quadratic work for long inputs. Use an index or deque to consume chunks without repeated shifting.
Details
✨ AI Reasoning
Long inputs are split into a number of chunks that grows with input size. Each request removes the first pending item, which shifts all remaining items; repeated removals therefore add avoidable work as the document grows.
🔧 How do I fix it?
Move constant work outside loops. Use StringBuilder instead of string concatenation in loops. Cache compiled regex patterns. Use hash-based lookups instead of nested loops. Batch database operations instead of N+1 queries.
Reply on this comment to give feedback or ignore the issue.
More info
There was a problem hiding this comment.
Agreed — the fix (collections.deque + popleft(), O(1) per chunk) is ready, but this PR entered the merge queue before it could be pushed, so it will land as a small follow-up rather than in this PR.
Upgrades only the 16 locked packages with open alerts, each to (at least) its first patched release: aiohttp 3.14.5, anyio 4.14.2, cryptography 50.0.2, httpcore2/httpx2 2.13.1, langgraph-sdk 0.4.7, langsmith 0.14.9, mcp 1.30.0, pyasn1 0.6.4, pydantic-settings 2.16.0, pyjwt 2.15.1, pypdf 6.20.1, starlette 1.8.0, urllib3 2.8.0, virtualenv 21.14.5, yt-dlp 2026.8.19. mcp is held on the patched 1.x line (1.30.0) rather than jumping to 2.x: langchain-mcp-adapters, which backs `live --mcp-config`, doesn't bound it. yt-dlp's parse_duration now returns an int for whole seconds, so _section_timestamp coerces to float to keep its declared contract (and the `--show-code` download_range_func rendering) stable across versions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to #279, addressing [Aikido's review comment](#279 (comment)). #279 entered the merge queue before this could be pushed. The multi-request TTS loop in `tts/session.py` took chunks with `pending.pop(0)`, which shifts every remaining chunk, so a long `--url`/PDF text did quadratic work queueing its requests. `pending` is now a `collections.deque` consumed with `popleft()`, which is O(1) per chunk. Behavior is unchanged: still at most 3 requests in flight, sent in order. `./scripts/check.sh`: **All checks passed**. The existing multi-request tests in `tests/test_tts_session_multi.py` cover the ordering and in-flight bound. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Alex Kroman <alex@assemblyai.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Brings the voice APIs the CLI calls in line with the AssemblyAI docs and makes them work in production: streaming TTS (
speak,dub,live) now runs without--sandbox, anddictatemoves from the Sync STT API to the Dictation API.Streaming TTS (docs)
wss://tts.us.assemblyai.com/v1/ws.speak/dub/livedrop the[sandbox]help tag and sandbox-only copy, so they show in everyone's--help.session.require_availablestays as a backstop for an environment with no TTS host.juergen, the German default, which gives3012 voice_not_found. It also had wrong languages forlola,rafaelandmarius. A non-built-in--voice(a UUID or a cloned-voice slug) now passes through without anaai-prefix.voice_id,languageas the lowercase English name (de→german),sample_rate, and a pinnedencoding=pcm_s16le.WordBoundariesare ignored, and the undocumentedis_finalis gone.--list-voices: shows the built-ins plus the account's own voices fromGET /v1/voices. Records are keyed byslugand paginated vianext_cursor. A shared eager option on all three commands meansdub --list-voicesneeds no--lang.agent-cascadetemplate fix: its loop waited foris_final, which the service never sends, so each sentence hung until the 2-minute idle timeout. It now ends onFlushDone.Dictation (docs)
dictatenow callsPOST https://dictation.assemblyai.com/v1/transcribe/live(core/sync_stt.py→core/dictation.py). The config part goes first, since audio-first is a 400, and fields use the documented names:language_codes,stt_prompt,keyterms_prompt.--verbatim,--instruction(llm_instruction) and--rewrite-mode fast|smart.--keytermreplaces--word-boost, which stays as a hidden deprecated alias that warns.--jsonaddsllm_responseandllm_errornext to the verbatimtext.Models
livenow defaults toclaude-haiku-4-5-20251001.kimi-k2.5isn't served by the production gateway; Haiku 4.5 tied for the fastest time to first token with tools bound in my measurements.kimi-k2.5, thegemini-*-previews) drop out of the known-models list. Every model id the CLI references is on the production gateway's model list.Dependencies (separate commit)
Resolves all 75 open Dependabot alerts (2 critical, 40 high, 30 moderate, 3 low). All of them are in
uv.lock, across 16 packages, and each is bumped to at least its first patched release: aiohttp, anyio, cryptography, httpcore2/httpx2, langgraph-sdk, langsmith, mcp, pyasn1, pydantic-settings, pyjwt, pypdf, starlette, urllib3, virtualenv, yt-dlp. The alerts are raised againstmain, so they close when this merges.mcpstays on the patched 1.x line (1.30.0) rather than 2.x, becauselangchain-mcp-adapters(behindlive --mcp-config) doesn't set an upper bound on it.yt-dlpreturns an int fromparse_durationfor whole seconds._section_timestampnow coerces it to float so--download-sectionsranges stay floats as declared.6329ebb), so it can be cherry-picked into a dedicated PR if preferred.Verification
./scripts/check.sh: All checks passed, including 100% patch coverage and the mutation gate.transcribe(file and URL),transcripts list/get,stream,llm,agent,clip,caption,eval,doctor.speak: default voice,--language es --voice anna, a ~3,350-character text on one socket, a two-speaker dialogue, and a cloned-voice slug.dub -l de,--list-voices,live --show-code.--instruction, and JSON output.sessions,balance,usage,limits,keys,audit) andlogin/onboard.dictate,control).Follow-ups (not in this PR)
live, its--show-codescript and theagent-cascadetemplate still open a TTS socket per spoken clause. The documented pattern is one long-lived session that usesCancelfor barge-in.livewith a file input cuts off the agent's reply when the clip ends. This bug predates this PR and reproduces on the base commit.https://claude.ai/code/session_01NoQ5jA9AJQ8wPWZbqbLuNn
🤖 Generated with Claude Code