Anthropic, the frontier artificial intelligence (AI) lab behind Claude, plans to allocate 5% of its shares to a new Public Benefit Fund, as the company seeks to expand donations for the public good amid growing concerns about the risks of advanced AI.
The proposed fund will be established as a wholly owned limited liability company and could receive an additional 2% of Anthropic’s shares for every 50% increase in the company’s value above a baseline yet to be disclosed. The additional allocations would be subject to the board’s discretion.
Led by Dario Amodei, Anthropic is also preparing for a potential initial public offering (IPO) as a public benefit corporation, a legal structure that requires management to consider broader interests alongside generating profits for investors.
The planned fund comes amid mounting concerns about AI safety, following security breaches involving AI systems and warnings from industry leaders, including Amodei, OpenAI CEO Sam Altman, and Elon Musk, about the potential risks of uncontrolled AI development.
Anthropic, which has positioned itself as a safety-focused AI developer, has warned that AI could pose “catastrophic or existential risks to humanity.” The company devoted roughly 80 pages of the 261-page main body of its prospectus to outlining risk factors.
In a blog post, Anthropic said the Public Benefit Fund could support nonprofit donations and direct distributions to specific populations and initiatives addressing AI’s impact on the labour market.
The company said that, combined with its co-founders’ pledge to donate 80% of their equity and an employee donation-matching program, roughly 20% of its shares would be dedicated to public purposes, excluding any additional shares allocated in the future.
Anthropic plans to disclose further details about the fund’s governance, structure, and valuation thresholds in the coming months. It also intends to publish a public-benefit report outlining its progress towards its mission.
The announcement drew support from Musk, CEO of Anthropic rival SpaceXSI, who suggested distributing shares directly to the public.
Responding on X, formerly Twitter, Musk said he recommended “giving the shares directly to the public.”
“Next year, I will give one of my SpaceX shares to every kid in America. This will be the first of many,” Musk wrote in a post early on Monday.
Musk held about 6.42 billion shares in the company as of June 2023.
Anthropic has also identified labor-market disruption as a priority for the proposed fund, building on an earlier commitment of $200 million to study and address the effects of AI on employment.
Companies worldwide have been cutting thousands of jobs since generative AI gained traction, with many businesses, particularly technology firms, citing efficiency gains from rapid adoption of the technology.
International Monetary Fund Managing Director Kristalina Georgieva said last week that research by the organisation indicated that about 40% of jobs could be augmented, downgraded, or eliminated by AI, with the share reaching 60% in advanced economies.
Beyond the Public Benefit Fund, Anthropic is seeking greater government oversight of advanced AI systems in the United States.
According to Politico, company representatives have privately presented Capitol Hill lawmakers with a draft bill aimed at regulating frontier AI safety and security.
The proposed legislation would allow the Donald Trump administration and future governments to block a model’s release or restrict its use if it posed a foreseeable, imminent risk of more than $1 billion in damages or of killing or seriously injuring more than 50 people.
Politico reported that the specified risks would have to arise from the creation or release of a biological weapon, an AI model evading its developer’s controls, or a cyberattack.
An Anthropic spokesperson confirmed to Reuters that the draft reflected proposals outlined in the company’s Advanced AI Framework, released earlier this year.
“The bill would grant the government the ability to block or deter dangerous deployments of the most powerful models and also requires developers to test frontier AI models, be transparent about findings, and get them independently evaluated,” the spokesperson said.
Under the draft, independent evaluators registered with the Commerce Department would periodically assess AI developers and their frontier models. Companies could pay for these evaluations, provided compensation was not tied to the outcome.
If evaluators identified an imminent risk covered by the proposed legislation, they would have to report their findings to the Commerce Department within 72 hours of completing the assessment, Politico reported.
The Commerce Department could then ask a judge to halt a model after determining that it posed the specified risk. The draft explicitly prohibits other federal departments, agencies, and officers from suspending a model’s development, deployment, or internal use.
A federal judge for the District Court for the District of Columbia could order the suspension of a model.
The proposals reflect Anthropic’s efforts to combine its stated public-benefit commitments with a regulatory framework for the risks posed by increasingly capable AI systems.