If you still think of ElevenLabs as "that AI voice company," yesterday's New York Summit should change your mind. On November 11, the company unveiled 13 new products and upgrades that turn it from a voice toolmaker into a full enterprise AI platform — custom models trained on your own data, voice agents that remember your customers, and a brand-new app that sits in your meetings and takes notes for you.
Custom Models Trained on Your Own Business
The biggest announcement is model customization. In the coming months, selected enterprises will work directly with ElevenLabs researchers to fine-tune open-source language models and ElevenLabs' own
voice models using data from their own live deployments. Instead of a generic model that knows nothing about your customers, products, or rules, you get one tuned for the metrics you actually care about — customer satisfaction, reliability, or anything else.
ElevenLabs is already doing this in narrower form: Scribe v3 Medical, its speech-to-text model for healthcare, understands specialized medical terminology, and a financial-services model is in development. The new program generalizes that idea to any industry. The catch: it rolls out to selected enterprises first, pricing hasn't been disclosed, and it's a roadmap item — not something every customer can sign up for today.
ElevenAI — An AI Teammate That Listens to Meetings
For employees rather than customers, CEO Mati Staniszewski previewed ElevenAI, a new app that listens to spoken conversations — including in-person meetings — and turns them into notes, summaries, and action items. It can also assist live during a call and pass context between people and agents. Free early access opened with the announcement.
Dictation and
meeting notes run on Scribe v3, ElevenLabs' transcription model covering 102 languages. The company says the app takes notes with participants' consent and includes notice and consent mechanisms — a detail that matters, since an app that listens to real meetings will face obvious questions about privacy and workplace trust.
Voice Agents That Remember You
ElevenAgents, the platform for deploying voice and chat agents across sales, support, recruitment, and operations, is getting two upgrades that push agents from answering questions toward doing work. Memory will recognize returning customers and distill each conversation into a structured record of preferences and constraints — so a customer who prefers Spanish and can only talk after 5 p.m. won't have to repeat that on the next call. Routines will automate multi-step work before, during, and after conversations, like pre-screening a mortgage applicant or reminding patients about appointments.
Alongside that, the new Interaction Engine orchestrates conversations — managing turn-taking and interruptions, and assigning cheaper models to routine parts of a call while bringing in a frontier model only when things get complicated. That routing is as much about cost as quality.
The NVIDIA Deal Behind It All
The infrastructure news may prove the most durable part of the day. ElevenLabs and NVIDIA expanded their partnership, bringing NVIDIA's Nemotron 3.5 Super VL model into ElevenAgents and adding 15 MW of NVIDIA Vera Rubin compute capacity on top of existing Blackwell Ultra systems. ElevenLabs says Nemotron 3.5 Super VL delivers leading task-completion accuracy — including strong performance in financial calculations and billing analysis — at 70% lower cost than other leading open models.
The collaboration also covers a sovereign deployment: an on-premises voice agent for Poland's national e-Health platform, which handles 40 million appointments a year.
Why It Matters
Staniszewski's message was blunt: conversational agents will become "critical infrastructure for every enterprise, as essential as cloud or payments" within a few years. With 550+ partners across 44 countries, a new Marketplace for partner-built apps, and an alliance with McKinsey, ElevenLabs is betting the winners in voice AI will own the whole stack — not just the best-sounding voice.
The honest caveat: several of the biggest pieces are previews or early access. The vision is coherent, but the next few months will show how much of it actually ships to ordinary customers.
Comments
Post a Comment