All stories

Cloudflare adds web search to AI Gateway

Cloudflare launched a Web Search API in AI Gateway with Ceramic.ai, Exa and Linkup, so agents can pull live web results without guessing URLs, provided search partners follow Cloudflare's crawler rules.

Cloudflare has announced a Web Search API delivered through AI Gateway. The launch partners are the web search providers Ceramic.ai, Exa and Linkup. The service returns structured snippets from the live web that go straight into a model's context.

The company pitches it as a fix for a common agent failure. Agents often guess the URL of a page and fetch it with curl, and a wrong guess comes back as a 404.

Partners have agreed to Cloudflare's crawler standards. Their crawlers must meet the company's published "Verified bots" requirements, which include identifying themselves and respecting robots.txt. Every search result must also link to the location of the crawled content. Cloudflare says the rules give site owners transparency and control over how their content is used.

Inside AI Gateway, search calls appear in the usual observability logs and are paid for from the account's AI Gateway credit balance. Admins can control which users have access to which providers. Cloudflare says it charges partners' list API prices with no added markup, supports bring-your-own-key, and will flag which partners offer Zero Data Retention.

Developers can call the API through a REST endpoint by naming a provider and a result limit, or with a binding inside Cloudflare Workers (env.AI.websearch). Cloudflare is also building native "Server Tools" into AI Gateway and says web search will be one of the first. It gave no release date.

The announcement does not include pricing figures or performance comparisons between the providers.

Sources