Baseten launches RL-trained search subagents
Baseten is offering customers small, fast search models trained with reinforcement learning on their own knowledge bases, which it says retrieve more reliably than standard RAG in regulated industries.
Baseten announced a Search Subagents training product. It is available now to current Baseten customers, and the company says it will "shortly" come to its wider platform. Baseten says the idea follows Windsurf's SWE-grep, which uses small, fast subagents to search code on behalf of a larger coding model. Baseten applies the same approach to enterprise knowledge bases, such as insurance policies, clinical guidelines and regulatory documents.
The company says it tried several search methods and settled on TF-IDF, a keyword-weighting technique. It sits between exact-match tools like grep and semantic embeddings. Baseten argues that TF-IDF's deterministic scoring gives stable reward signals during RL training, while embeddings are too noisy. It reports searches under 100ms, even on very large knowledge bases.
Training data is fully synthetic. Baseten splits a customer's markdown documents into chunks and filters them for quality. Frontier models then write domain-style questions for each chunk. A question is kept only if the search tool returns its target chunk among the top results. No human annotation is required.
Each model is trained per customer using a token-level Advantage-Indexed Policy Optimization (AIPO) objective. The reward gives 90% weight to answer correctness and 10% to partial credit for retrieving the right chunk in any search call. The model samples eight rollouts per prompt, using leave-one-out advantages. Baseten recommends non-RL methods such as iSFT and RGT for writing the final answer.
The post is labelled Part 1. It gives no benchmark results, model sizes or pricing.