
Among the millions of documents filed in the Securities and Exchange Commission’s EDGAR database since 2001, the registration statement just submitted by the AI company Anthropic in anticipation of its initial public offering is the first in which a company uses the phrase “existential risks to humanity” in reference to its product.
Anthropic’s management does not explicitly say that it shares the concern about such risks. Instead, the filing states: “Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.”
The company wants to have it both ways. It uses the registration statement to warn potential investors about the worst-case scenario, yet it gives no impression that those extraordinary risks are slowing its work on AI, which it depicts as “one of the most consequential technologies in human history.” In fact, Anthropic insists that it needs to proceed at a breakneck pace, stating: “Our success depends in large part on our ability to continue to develop the most advanced and capable AI models, and any failure to do so could adversely affect our business.”
While acknowledging that AI could cause enormous harm, Anthropic wants us to believe it has no choice but to forge ahead in order to maintain its competitive position and its revenue growth. The message is clear: the success of the company must take precedence over the potentially negative consequences of its products.
This is nothing new. Large corporations ranging from cigarette manufacturers to fossil fuel producers have long promoted dangerous products to serve their pecuniary aims. The difference here is that Anthropic is admitting that it doesn’t know the full extent of the risks involved, because it is not sure to what extent it will be able to continue controlling its creation.
The filing states: “Our AI models may themselves cause harm by producing undesirable outputs or behaving in unintended, misaligned, or autonomous ways, including by taking unintended or harmful actions that no user or third party directed.”
Given this possibility, one might expect Anthropic to be willing to accept the strictest forms of government oversight. Yet, here too it sends conflicting signals. It has to acknowledge that many are calling for such guardrails, but Anthropic then reverts to a more typical corporate position by warning that the controls could “increase our compliance burdens, limit our product development or deployment flexibility, or otherwise slow our growth relative to competitors.”
The format of a registration statement requires Anthropic to describe risks of all kinds, but the fact that the company is proceeding with what is expected to be the biggest initial public offering in stock market history shows that, in the end, its business imperatives will take precedence over all else. That’s why the ultimate decisions about the future of AI need to be made not by the industry but by society at large.









You must be logged in to post a comment.