# AWS walkthrough starts agent speech earlier with Kokoro on SageMaker AI

- Source: MechNext — https://mechnext.eu/ru/article/49-aws-walkthrough-starts-agent-speech-earlier-with-kokoro-on-sagemaker-ai
- Published: 2026-10-23
- Updated: 2026-10-23
- Category: Industry News
- Language: ru (Markdown in other languages: ?lang=en, ?lang=de, ?lang=sk, ?lang=pl, ?lang=zh)

> AWS streams completed agent sentences into speech synthesis before the full reply is ready. Its walkthrough reports faster first audio and details transport, deployment and measurement limits.

An AWS walkthrough sends completed agent sentences to Kokoro-82M before the full response has been generated, allowing speech synthesis to start earlier. The 82-million-parameter model runs in a custom container on a SageMaker AI real-time endpoint using an ml.g5.xlarge GPU instance. The implementation findings and performance figures are reported by the walkthrough's authors.

### Multiple sentences, one connection

POST /invocations streams output for one complete input, while WS /invocations-bidirectional-stream supports multiple messages over one bidirectional connection. After returning audio and a synthesis_complete control frame, the container waits for another message. It closes the session only after an explicit client close message.

Each handler synthesizes and transmits messages sequentially: it does not generate the next sentence's audio while transmitting the previous sentence. The reported benefit comes from starting the first sentence sooner and avoiding a new connection for every sentence.

The sentence chunker accepts complete strings or streamed fragments. Its feed() method releases newly completed sentences and retains trailing text; flush() releases any remaining partial sentence when input ends. The authors report checking three sentence inputs over one live session, with each returned audio chunk matching its text.

SageMaker AI delivers client inputs to the container as binary WebSocket frames. A Starlette handler expecting text frames can raise a KeyError for the text field on every request while the endpoint remains InService. The sample handles both frame types, decodes binary input as UTF-8 and recommends testing with binary frames.

### What the measurements cover

To isolate TTS transport timing, the authors used a 267-character, four-sentence block and ran five live-endpoint trials for each approach: sending the entire block or only its first sentence. They report mean time to first audio byte falling from 329.3 to 156.7 milliseconds, a 52.4% relative reduction in this comparison.

The end-to-end example uses a Strands agent backed by Amazon Bedrock and hosted in AgentCore. Separate connections serve browser-facing SSE, Bedrock text generation and Kokoro bidirectional synthesis. Bidi mode concurrently consumes text and audio through a shared event queue; batch mode waits for the entire response before synthesis.

Using the same prompt, live Bedrock model and endpoint, the authors report first audio arriving 2.4 times faster with bidi in a local container and 3.5 times faster in deployed AgentCore. The reproduction test times the first audio_chunk event relative to agent_start. Both modes base64-encode caller-facing audio because BedrockAgentCoreApp yields JSON-serializable dictionaries that its SDK converts into SSE frames.

### Transport and deployment considerations

The authors examined transport costs across 24 clips, compared MP3 with AAC and report using Amazon Transcribe to confirm no quality difference across encodings. Base64 increased payload size by approximately 33%. In one response, four logical MP3 frames arrived in six delivery chunks, two ending mid-frame. Receivers therefore need buffering and reassembly instead of treating delivery chunks as complete audio units.

A model-facing alternative multiplexes raw binary audio and JSON control frames over the existing bidirectional session, avoiding base64 and SSE on that leg. A chunk is treated as control data only if it decodes as a complete JSON object; otherwise it is audio. Checking the whole chunk avoids mistaking PCM for JSON merely because it starts with an opening brace. The authors recommend keeping base64 over SSE for an established SSE frontend, and considering unified transport for a new voice path or when latency takes priority.

Prerequisites include permissions for SageMaker AI, AgentCore, IAM and ECR; bootstrapped AWS CDK v2; Python 3.12 or later, uv and Docker; Boto3 support for invoke_endpoint_with_bidirectional_stream; WebSocket and ASGI familiarity; and the referenced code. The underlying HTTP/2 client provides prebuilt wheels only for Python 3.12 onward. The container must declare com.amazonaws.sagemaker.capabilities.bidirectional-streaming=true.

speech-infra deploy creates endpoint infrastructure, IAM roles, an S3 weight cache, the container image, model and endpoint configuration. agent-infra deploy then creates the AgentCore runtime. Cleanup runs in reverse order; destroy commands retain the S3 bucket and ECR repository, requiring separate removal to stop their storage charges. The post also warns that the sample pins an older PyTorch affected by PYSEC-2026-2286. It identifies torch 2.10.0 as fixed and calls for reviewing and upgrading the pin before use beyond a proof of value.