An AWS article describes a search agent that makes choices over multiple interaction rounds. The agent autonomously selects what to search for, which retrieval strategy to use, and when to stop. The training approach therefore considers a complete sequence of decisions rather than a single isolated answer. Each choice belongs to a trajectory that ultimately produces retrieved documents, so the article frames agent training around the overall search process.

The article presents Amazon SageMaker AI MTRL, a multi-turn reinforcement learning approach for training the agent. It describes multi-turn rollouts that cover the full sequence of decisions and policy-gradient algorithms. MTRL supports custom rewards, tool loops, and multi-turn conversation shapes, according to the article. The described service also includes serverless execution and asynchronous collection of rollouts and trajectories. Listed algorithm options include PPO, CISPO, and importance-sampling losses. Other listed features include resumable training, trajectory observability, and evaluation jobs.

Decisions across a search trajectory

In the described setup, an agent can choose another search action or decide to stop. The example uses different retrieval approaches for different kinds of queries: BM25 lexical search for exact terms or identifiers, and vector search for semantic or conceptual queries. This distinction is relevant to a multi-step agent because its choices include selecting a retrieval strategy as well as deciding whether another step is needed. The article describes the choices as part of a sequence rather than treating each search in isolation.

The training reward was trajectory-level nDCG@10, based on how well the final retrieved documents matched ground truth. This means the stated reward reflects the quality of the documents at the end of the search trajectory. The metric ties the training objective to final retrieval quality in the reported setup. The account does not imply that every step receives the same value, or that either BM25 or vector search is suitable for every query; it specifically assigns them to exact-term and semantic or conceptual searches, respectively.

Training limits and reported results

The article says that reaching the maximum turn count or the per-turn sampling-token limit resulted in a reward of -1. It also lists changed hyperparameters for the described training: max_epochs was 1, global_batch_size was 128, and rollout_max_concurrency was 32. These details describe the configuration in the article. They do not establish that the same limits or settings are appropriate for every agent or task.

On four held-out benchmarks, the article reports improved nDCG@10 on three. BrowseComp-Plus gained 23.7%, and WixQA gained 18.4%. Wands showed a smaller gain, while FreshStack had a slight regression. For BrowseComp-Plus, the reported failure rate fell from 22.89% to 0.68%. Taken together, these figures show different outcomes across the listed benchmarks: the report includes both gains and a regression. They are results from the described evaluation and do not establish that all search tasks will show the same pattern.

A practical workflow

The article recommends preparing the dataset in the required format, defining a reward function based on the task-specific metric, starting with default configurations, and evaluating and iterating. This sequence connects the training setup to the outcome a team intends to assess. If the task is judged by retrieved documents, the reward should reflect the relevant task metric. Evaluation and iteration then provide a way to respond to observed results, including differences such as the gains on BrowseComp-Plus and WixQA and the slight regression on FreshStack.

The practical framing is to treat search as a sequence of actions and align training and evaluation with that sequence. As described, MTRL provides support for custom rewards, tool interactions, and trajectory collection; the benchmark results show that measured changes can vary by evaluation set. The article does not claim that every agent or dataset will achieve the same gains. Instead, its recommendations emphasize dataset preparation, a reward connected to the task metric, and repeated evaluation. Together, these points explain how the described multi-turn setup relates the agent’s search decisions to the quality of its final retrieved documents.