Anthropic Just Pitched Congress Its Own AI Safety Bill — and the Bar Is $1 Billion

 

In a rare move, an AI lab is now writing the law it wants to live under. Anthropic quietly shared a draft AI safety bill with Capitol Hill lawmakers this week, and it sets a strikingly high bar before the government can pull the plug on a model. The draft — first reported by Politico on October 16 and confirmed by Reuters — reflects Anthropic's "Advanced AI Framework" released earlier this year. An Anthropic spokesperson called the proposals "sensible policies we believe Congress should act on."

The details: the government could block a model's release or use ONLY if it poses a foreseeable, imminent risk of more than $1 billion in damages — or of killing or seriously injuring more than 50 people. And the risk must come from just three pathways: the creation or release of a biological weapon, a model evading its developer's controls, or a cyberattack. Even then, the process is deliberately narrow. Independent evaluators registered with the Commerce Department would assess frontier models periodically; labs could pay them, but not in a way tied to the evaluation's outcome. Evaluators must report any imminent qualifying risk within 72 hours. Only then could Commerce ask a federal judge — specifically the U.S. District Court for the District of Columbia — to suspend the model. Every other government agency would be explicitly barred from halting a model's development, deployment, or internal use.

The bill also pushes developers to test their most powerful models, publish their findings, and protect whistleblowers. And it would preempt some state AI laws on risk disclosure, outside evaluation, and incident reporting — a provision Democrats have historically resisted.

Why it matters: this is the clearest signal yet of how the labs want AI regulated — serious in principle, but with a tripwire set extraordinarily high. Consider the recent incident wave: rogue agents filing fake police tips, Anthropic pulling its own models off the live internet, OpenAI's growing incident log. None of them caused $1 billion in damage or 50 deaths — so under Anthropic's own bill, none would have triggered government action. That context is important, and it makes the threshold itself the debate.

The timing isn't accidental either. Politico notes the pitch comes months after federal officials placed two of Anthropic's most advanced models under sweeping export controls, spooking Silicon Valley about arbitrary government intervention. Anthropic would rather write the rules of the intervention itself.

AiPost's take: Anthropic deserves credit for putting specifics on paper instead of vague "we support regulation" talking points. But a $1B/50-death threshold is a last-resort fire extinguisher, not a day-to-day safety regime. The real fight ahead: whether Congress accepts Anthropic's line in the sand — or writes its own, much lower one.