ElevenLabs previews Interaction Engine and fine-tuning service
At its New York summit, ElevenLabs previewed a model-routing Interaction Engine, customer fine-tuning, and agent memory, part of its push to grow from a voice-model company into enterprise infrastructure for conversational agents.
ElevenLabs announced a set of model, agent and deployment updates at its New York Summit. CEO Mati Staniszewski said conversational agents will become "as essential as cloud or payments" for enterprises. Most of the items were previews, and the company did not give release dates or benchmark results.
Domain models and fine-tuning. The company highlighted Scribe v3 Medical, a healthcare transcription model. ElevenLabs says it recognizes thousands of medical terms more accurately and can tell apart similar drug names such as cycloserine and cyclosporine. A model for financial services is in development. Separately, ElevenLabs said that "in the coming months" it will help businesses fine-tune certain open-source language models and its own voice models. Customers would use data from live deployments and work toward metrics they choose, such as CSAT. Staniszewski compared this to each company having its own reinforcement-learning stack.
Interaction Engine. This previewed system handles turn-taking and interruptions. It also routes each part of a conversation to a different LLM, so a cheaper model can handle triage and a frontier model can step in for harder tasks.
ElevenAgents memory. A coming Memory feature lets agents recognize returning authenticated customers and keep details from past conversations, such as language preference or availability, alongside the CRM. ElevenLabs named Yahoo, Westgate and KPN as existing ElevenAgents customers.
The post also mentions agent routines, an ElevenAI app, a McKinsey alliance and 15 MW of NVIDIA Vera Rubin compute. The excerpt reviewed for this note did not include details on those items.