Together AI's New Compiler Lets AI Agents Make AI Models Faster Overnight

 

Together AI announced Together Compiler on November 4, 2026 — a system where teams of AI agents rewrite the GPU code behind the open-source models the company serves, so models get faster overnight with no extra work from customers.

What Is Together Compiler?

Together Compiler is the first project of a larger initiative Together AI calls Polymorphic Inference. The idea is simple to state and hard to do: inference infrastructure that keeps retuning itself as models, traffic and hardware change. Instead of engineers hand-optimizing GPU kernels for each new model release, agent teams generate candidate optimizations, test them, and put the winners into production. Before any candidate can serve real traffic, it passes verification tests for KV-cache contamination and output quality.

How Fast Is It, Really?

In Together AI's tests, the compiler delivered 1.3x to 3x the tokens per second of highly optimized open-source baselines in under 24 hours, across tests involving Kimi K3, GLM 5.3, MiniMax M3.1-Flash and DeepSeek v4.1-Flash. The company also reports its startup comparison dropped from 467 seconds to serving down to 9.0 seconds with the compiler's setup. The system already runs across many of Together AI's top production inference workloads and some popular serverless endpoints, and the company says it serves trillions of tokens per day.

Why It Matters

Model releases are accelerating, and every new architecture used to mean weeks of manual tuning before it ran efficiently. If optimization becomes automatic and continuous, smaller teams can serve frontier-class open models at competitive speed and cost — and speed is cost in the inference business. Customers can request early access through a limited beta for dedicated endpoints, with self-serve optimization coming soon.

The Catch

These are the company's own numbers, and independent verification is still pending — the baselines were leading open configurations as of October 30, 2026, with parallelism strategy held fixed. Still, the direction is hard to argue with: the bottleneck in AI is shifting from training models to serving them cheaply, and now AI itself is doing that optimization work.

AI that makes AI faster is the quietest kind of self-improvement — no sci-fi drama, just cheaper, quicker answers for everyone building on open models.