Anthropic/AI: safety first mandate does not eclipse unusual set-up

Unlock the Editor’s Digest for free

Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.

Silicon Valley doomers worry that generative artificial intelligence will supercharge global risk. OpenAI rival Anthropic positions itself as an ultra-safe, responsible AI start-up. That does not mean it will escape regulator scrutiny.

Anthropic is run by siblings and former OpenAI employees Dario and Daniela Amodei, chief executive and president respectively. Based in San Francisco, not far from OpenAI’s headquarters, it seeks to create AI models that follow a set of guiding principles. This month, it published research that examined bias in its AI-powered chatbot Claude. Not to be outdone, OpenAI has released its own safety plan assessing potential catastrophes. 

This will please Washington worrywarts. But the US Federal Trade Commission’s Lina Khan has more prosaic concerns. She is interested in the ways in which Big Tech companies are investing in AI start-ups like Anthropic, in effect concentrating power in a new sector.

Deals struck between large companies and AI start-ups tend to include revenue-boosting cloud agreements. Microsoft’s $13bn commitment to OpenAI involved OpenAI being trained on Microsoft’s cloud platform. Anthropic’s backers include Google and Amazon. It is a cloud customer of both.

Conditional deals that mingle investment and infrastructure spending are not unusual for cloud computing companies. But AI has shone a spotlight on their structure and the way in which they can leave start-ups overly reliant on large tech companies.

For those companies, Anthropic’s appeal is less connected to its safety-first attitude than its role as a rival to OpenAI and a challenge to Microsoft’s dominance in generative AI. Although Anthropic’s reported recurring revenue is a tenth the size of OpenAI’s, it has already created a Pro subscription service that charges $20 per month.

One point that investors may be newly wary of is Anthropic’s unusual set-up. In addition to a board of directors it has a “Long-Term Benefit Trust” that oversees some seats on the board and does not answer to investors. OpenAI’s recent governance troubles could push Anthropic investors to demand change.