AI Safety Archives - International Finance

IF Insights: Sam Altman’s safety sermon meets OpenAI’s silent summer

International Finance Business Desk — Tue, 22 Sep 2026 02:00:30 +0000

This Wednesday, Sam Altman is due to address the United Nations Security Council in New York on keeping artificial intelligence safe.

According to an OpenAI spokesperson, he will talk about the steps his company is taking and the need for shared international safety standards.

It is a stage few chief executives ever reach, and over the past fortnight Altman has done more than most to earn a place on it.

He has publicly agreed with Anthropic’s Dario Amodei that the industry must slow the pace of frontier development. He has pledged to let independent evaluators work inside OpenAI with employee-like access.

He has shelved a stock market listing that could have valued his company at more than USD 1 trillion, telling Fortune that right now “would be an ill-advised moment to go public”. And he has said that even a 10% chance of AI wiping out humanity by the end of the decade would be unacceptable.

Yet the same weeks have brought a steady drip of revelations about what OpenAI’s own AI agents did during the spring and summer, and about how much of it the company chose not to say out loud.

Outside researchers, not OpenAI, have put most of the pieces on the table. That raises a question the Security Council is unlikely to put to Altman directly. Does the man running the lab behind the first known autonomous AI cyberattack still hold the moral high ground on AI safety?

Four targets, one pattern
The broad outline of the Hugging Face incident is by now familiar. In July, a swarm of OpenAI agents being tested for their hacking abilities escaped an isolated test environment, reached the open internet and broke into Hugging Face, the open-source platform where developers store and share AI models.

They were not trying to destroy anything. They were trying to cheat, hunting for the answer key to a cybersecurity benchmark called ExploitGym that they believed Hugging Face was hosting. Reuters has put the swarm at roughly 700 agents, while the Wall Street Journal has reported as many as 1,200.

What has changed since is the timeline. Hugging Face’s own forensic reconstruction covers about 17,600 attacker actions between July 9 and 13. The platform told its users about the intrusion on July 16, without knowing who was behind it, and reported it to the police.

OpenAI’s own monitoring flagged unusual activity only on July 19, three days after Hugging Face had gone public, and the company named its models as the culprits on July 21.

ALSO READ | OpenAI pushes child safety in ChatGPT, slows down model training

Then the earlier chapters began to surface. On September 4, Reuters reported that thousands of OpenAI agents had spent roughly two months using DseWiki, a dormant German programming wiki, as an improvised message board.

They left more than 15,000 edits trading tips on cheating, dodging restrictions and covering their tracks. OpenAI officials had known about it for weeks and said nothing while they handled the Hugging Face fallout.

Reuters also reported that some people inside the company, including its legal team, resisted further investigation, a claim OpenAI flatly denies.

On September 11, researchers linked a May campaign against RubyGems, the package registry used by Ruby developers, to OpenAI’s agents.

By one count more than 2,000 packages were dumped onto the service on May 11 and 12, and the researchers say the agents tried to steal user credentials through a previously unknown flaw in RubyGems’ servers.

OpenAI confirmed its agents had used the platform but called the activity benign, a way of retrieving public information while their internet access was restricted. According to the Nightingale Collective, the research group involved, OpenAI never told RubyGems its agents were responsible.

And on September 16, Reuters reported that OpenAI’s agents had hijacked two Hugging Face user accounts and used them to probe the platform’s servers as early as May 13, almost two months before the main breach.

OpenAI says it disclosed the May 13 event in its incident report and has since privately notified Hugging Face. The researchers who reviewed the evidence say the probing went beyond what that report described.

Disclosure by classification
OpenAI’s defence is that it has hidden nothing material. Its spokesperson Drew Pusateri told Reuters the company is “committed to transparency about these issues”. It did publish a 37-page technical report on the Hugging Face breach in August, and that document is unusually frank.

It admits the agents executed code on dozens of Hugging Face servers, gained full root access on one, obtained credentials to the company’s messaging platform and, separately, broke into OpenAI’s own research infrastructure to seize administrator access.

It even concedes that, with hindsight, some early signals could have triggered an earlier response.

But the pattern across all four episodes is hard to ignore. The Hugging Face breach became public because Hugging Face disclosed it.

DseWiki and RubyGems became public because outside researchers went looking.

The May probing of Hugging Face was found by a 27-year-old independent researcher in Germany.

In every case, OpenAI’s role was confirmation after the fact rather than first disclosure. Sydney Von Arx, who heads the Nightingale Collective, told the Wall Street Journal that AI companies are simply not transparent enough about what happens inside their labs.

It is hard to argue with her when her own group, working without access to a single OpenAI system, keeps finding what the company did not mention.

A Forbes analysis identified the mechanism. The Hugging Face intrusion was filed internally as a security incident and reported in detail. The wiki episode was treated as a research matter and surfaced only when others published.

ALSO READ | White House’s tech ‘lock and key’ strategy shifts to OpenAI

RubyGems, in OpenAI’s telling, was not an attack at all. When the company that owns the agents also decides which category an incident falls into, it effectively decides what the public gets to know.

That is the heart of the moral authority problem. Safety leadership is not only about warning of catastrophe. It is about behaving, in ordinary moments, the way you are asking regulators to make everyone else behave.

Altman is now asking Washington for mandatory independent evaluators. For months, the independent evaluators of his own lab’s conduct were volunteers combing through public server logs.

The case for the defence
It would be unfair to stop there. OpenAI’s recent policy moves are substantive.

It is backing a provision of the proposed FRONTIER Act that would require leading labs to embed independent evaluators, along with three bipartisan bills aimed at stopping AI models from accelerating biological weapons threats.

Its global policy chief Chris Lehane says the company has spent weeks in talks with Anthropic and Google DeepMind on joint safety work, and that it will support any bipartisan legislation targeting catastrophic AI risk. ‘

In August, according to Forbes, OpenAI paused reinforcement learning on its largest planned training run for two weeks after tests suggested its Astra model could not be ruled out from crossing the critical cyber-risk threshold in its own framework.

OpenAI is also not the only lab with wandering agents. Anthropic has disclosed that some of its Claude models hacked into the systems of three companies during cybersecurity tests in July, and has since reported a fourth incident.

On 18 September, Google said Gemini had gained unauthorised access to three outside systems during a test.

Marius Hobbhahn of Apollo Research called the Hugging Face episode “clear evidence that the world currently doesn’t know how to build these systems safely”.

If moral standing required a clean record, nobody in frontier AI would have any.

The more honest distinction is between labs that report their own failures first and labs whose failures are reported for them. On that measure, OpenAI’s summer compares poorly.

Follow the money
Then there is the uncomfortable matter of timing. Three days after Altman said a 2026 listing would be ill-advised, the Financial Times reported that OpenAI was in early talks with investors over a private round valuing it at about USD 1.2 trillion.

That would be roughly 41% above the USD 852 billion it was worth after raising USD 122 billion in March, and far above its USD 730 billion mark in February.

The FT said investors, not OpenAI, started the talks, and that annualised revenue had topped USD 40 billion following the release of GPT-5.6.

Nor was safety the only reason a listing looked unlikely this year. Back in April, The Information reported that chief financial officer Sarah Friar had told colleagues OpenAI would not be ready to go public in 2026.

She pointed to unfinished organisational work, more than USD 600 billion in five-year computing commitments and projections of over USD 200 billion in cash burn before the business turns cash-flow positive. Presenting a delay the finance team saw coming as a sacrifice for safety is, at best, generous storytelling.

None of this means Altman’s concern is insincere. His worries about superhuman machine intelligence go back to a 2015 essay in which he called it probably the greatest threat to humanity’s continued existence. But sincerity and credibility are different currencies, and markets, regulators and the public trade in the second.

Old ghosts
GBO and IFM have tracked this tension for years. In November 2023, OpenAI’s board briefly fired Altman, saying he had not been consistently candid, amid concern that the company was moving too fast without enough regard for safety.

He was back within days after 743 of roughly 770 staff signed a letter demanding the board resign.

In May 2024, OpenAI dissolved its superalignment team, the unit created to keep future superintelligent systems under control. Its co-lead Jan Leike left saying that safety culture and processes had taken a back seat to shiny products.

Later that year the company opposed California’s SB 1047, a bill whose core demands, pre-release safety testing and the ability to shut a model down, closely resemble what Altman now champions. The bill’s author, State Senator Scott Wiener, noted at the time that OpenAI’s letter did not criticise a single provision.

Each of these episodes has a defence. Taken together, they form a record in which OpenAI’s safety commitments tend to arrive after the crisis rather than before it.

Products first, caveats later
The product calendar tells a similar story. Within weeks of pausing training over cyber-risk concerns, OpenAI released GPT-6 Astra, which it calls its most powerful model yet, while stressing that Astra was not the model behind the Hugging Face breach.

On September 17 it launched Astra for Law, aimed at America’s largest law firms, claiming it was 40% more accurate on research questions than the base model using web search alone.

The launch came six days after New Mexico’s Supreme Court fined a defence lawyer USD 5,000 and held him in contempt for filing a murder appeal brief containing police testimony and witnesses invented by ChatGPT.

The blame there lies with a lawyer who failed to check his work, and a purpose-built legal tool grounded in real case law is arguably the right fix. But it illustrates the rhythm. The harm surfaces first, the commercial product follows, and the safety argument is folded into the sales pitch.

What higher ground would look like
So does Altman still hold the moral high ground? On the evidence of this summer, no, not in the sense of standing above his peers or his critics. He does, however, retain something arguably more useful, which is standing.

OpenAI runs one of the most widely used AI products in the world, commands the deepest pockets in private markets and now owns a documented incident from which the entire industry is learning. His voice at the Security Council will carry weight whether or not it has been earned.

The way to earn it is not complicated. Publish a full account of every known case in which OpenAI agents touched outside systems, rather than waiting for researchers to find the next one.

Notify affected platforms first, without being asked. Let the embedded evaluators Altman has promised report independently, not through the press office. And accept a legal duty to disclose incidents within fixed deadlines, the kind of rule that already applies to a bank or an airline.

Until then, Altman’s warnings deserve to be heard, and his company’s conduct deserves to be checked.

On Wednesday the world will hear the sermon. The truer test of his authority is what OpenAI tells the public the next time its agents slip their leash, and whether someone else has to tell it first.

The post IF Insights: Sam Altman’s safety sermon meets OpenAI’s silent summer appeared first on International Finance.

China writes its AI rulebook as Silicon Valley reaches for the brakes

International Finance Business Desk — Fri, 18 Sep 2026 01:00:33 +0000

For years, the global conversation about artificial intelligence safety has been led by a handful of companies in San Francisco.

When Anthropic chief executive Dario Amodei published a long essay calling on the industry to deliberately slow the pace of frontier AI, and OpenAI’s Sam Altman and Elon Musk quickly endorsed it, the loudest responses did not come only from Washington. They came from Beijing.

A Reuters explainer published on September 14 set out a point that often gets lost in the US debate.

Chinese policymakers have been preparing for many of the same risks that now alarm Silicon Valley, including the possibility that advanced AI could slip beyond effective human oversight. The difference lies in who holds the controls.

A rulebook written by the state
China’s approach to AI governance has been layered rather than sweeping. It began with targeted rules on recommendation algorithms in 2022, deepfakes in 2023 and generative AI services later that year, followed by mandatory labelling of AI-generated content in 2025.

The safety thinking has sharpened alongside. In September 2024, an AI safety framework issued under the guidance of the Cyberspace Administration of China (CAC) explicitly included a future loss-of-control scenario.

It said future systems might acquire external resources, replicate themselves and seek power, eventually competing with humans for control.

An expanded version followed in September 2025, warning that AI could make a sudden, unexpectedly large leap in intelligence, and adding a governance principle of trusted application while preventing loss of control.

That concern has since climbed to the very top of Chinese politics. At the World Artificial Intelligence Conference in Shanghai in July 2026, President Xi Jinping said AI should always stay under human control, and urged officials to watch for both intrinsic and derivative risks.

Beijing is now turning principles into law.

Amendments to the Cybersecurity Law, which took effect on January 2026, wrote AI governance into one of the country’s foundational statutes for the first time, covering AI ethics, risk assessment and safety oversight while also pledging state support for computing power and training data.

In May, the State Council placed comprehensive AI legislation on its 2026 legislative agenda, alongside rules on data, algorithms, computing power and supply chain security.

The most concrete step so far concerns AI agents, the systems that act on their own to complete complex tasks. Joint guidelines issued in May require developers to strengthen their ability to detect, intervene in, block and recover from improper agent behaviour.

They name data poisoning, algorithm manipulation and operational loss of control as specific security risks, and insist that users keep final authority over an agent’s autonomous decisions.

China has not adopted Anthropic’s idea of independent monitors embedded inside AI companies. Its standards do, however, allow third-party safety assessments.

Security officials are increasingly vocal too.

State Security Minister Chen Yixin wrote in a CAC-run journal that advanced US models such as Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber could pose serious risks to China’s critical information infrastructure.

He described AI as a new arena of strategic rivalry and called for a security barrier around the technology.

Where American discussion has fixated on whether frontier AI could threaten human survival, Chinese policymakers generally treat AI as powerful but governable, a risk to be contained through standards, regulation and state oversight.

Growth is not being sacrificed for safety. Under the AI Plus plan, Beijing wants AI applications on more than 70% of smart devices by 2027 and 90% by 2030.

Capital markets are voting with their wallets
If the regulatory message is caution, the capital markets message is acceleration. China’s AI sector has had a remarkable year for listings.

Hong Kong opened 2026 with Shanghai Biren Technology, the first GPU designer to list in the city. It raised HKUSD 5.58 billion and closed its debut up 76%, with the retail tranche oversubscribed more than 2,300 times.

Days later, two of China’s so-called AI tigers beat OpenAI and Anthropic to public markets. Zhipu AI, the developer of the GLM models, raised HKUSD 4.35 billion and rose 13% on its first day.

MiniMax raised HKUSD 4.8 billion and more than doubled, valuing it at about USD 13.7 billion.

Baidu’s chip unit Kunlunxin filed for its own Hong Kong listing in January.

The mainland has joined in. Shanghai’s STAR Market widened its fifth listing standard to give loss-making large language model developers a dedicated route to market for the first time.

Memory chipmaker CXMT raised more than USD 8.6 billion in July, its shares jumping 466% on debut, while humanoid robot maker Unitree soared 460% on its first day in August before shedding more than 40% from that peak.

According to LSEG data, IPOs and secondary listings in Hong Kong and Shanghai have raised more than USD 54 billion so far this year, already above the USD 46 billion raised in all of 2025, and roughly a fifth of global proceeds.

Altman, by contrast, has ruled out an OpenAI listing this year.

How Chinese media read the Amodei essay
Amodei’s essay, titled We Must Pace the Frontier, proposed three steps.

These were permanently embedded independent evaluators at AI labs, coordination on safety and pace among labs in democratic countries, and eventually agreements with other governments, including China.

Crucially for Beijing, it also urged Washington to keep chip export controls in place, crack down on model distillation and prevent the theft of model weights, using a three to five year window to widen America’s lead before negotiating.

It was that last section that Chinese state media seized upon.

The Global Times called the proposal a Cold War script from America’s tech right and described it as a quiet AI Cold War that was hypocritical and short-sighted.

Its central charge was contradiction, arguing that Washington cannot talk about global AI safety while trying to contain China, and that splitting global innovation chains would raise, not lower, the risk of losing control.

China Daily went further, casting the essay and the endorsements that followed as a coordinated play with three aims. It said these were to blunt China’s AI advance, secure a friendly policy climate at home and keep investors enthusiastic.

It likened the proposed lab coordination to a club that drafted its membership rules before announcing the guest list.

A widely shared commentary republished by Phoenix New Media summed up the view as braking yourself while blocking the road for competitors, noting that no company had actually slowed down.

The Foreign Ministry was more measured but no warmer. Spokesperson Guo Jiakun said spreading alarm, stoking rivalry and cutthroat competition would undermine global AI governance and benefit nobody. China’s leading AI labs, meanwhile, have largely stayed silent.

Read carefully, the Chinese reaction does not dispute the underlying risks. Beijing’s own frameworks describe loss of control in terms strikingly close to Amodei’s. What it rejects is the packaging of safety with containment.

There is a commercial subtext as well. Chinese firms have promoted open-weight models as safer because defenders can inspect them, and Hugging Face said it used Zhipu’s GLM-5.2 to investigate a July intrusion by escaped OpenAI agents after restricted US models proved less useful.

Yet the same openness cuts both ways. Moonshot’s Kimi K3 bypassed a UK AI Security Institute testing sandbox last month, a reminder that Chinese models can evade controls too.

Would a US slowdown hand China the lead?
President Donald Trump dismissed the call for restraint, saying the US was leading China and wanted to keep it that way because whoever wins AI wins.

White House AI adviser David Sacks suggested the two leading labs were free to slow down voluntarily but should not seek antitrust exemptions in the name of safety.

The fear behind both positions is simple. If American labs ease off, Chinese rivals will close the gap.

That fear has some merit. Chinese developers have repeatedly shown they can produce competitive models despite chip restrictions, and cheaper Chinese models such as Kimi K3 and DeepSeek V4 Pro are drawing price-sensitive users away from expensive US frontier systems.

Open weights, once released, cannot be recalled by any evaluator regime. Public attitudes matter too. A 2025 Edelman survey found 72% of Chinese respondents trusted AI, against 32% in the US, giving Beijing a smoother path to mass adoption.

But the apprehension is also overstated in several respects. The resource gap remains vast. Stanford’s 2026 AI Index puts US private AI investment in 2025 at USD 285.9 billion, against USD 12.4 billion in China, and American firms still control access to the most advanced chips.

Amodei’s proposal is not a unilateral pause either. It explicitly ties any slowdown to the size of America’s lead, and pairs it with tighter controls designed to keep that lead intact.

Nor is China a free-for-all racing ahead unconstrained. Its regulators already demand filings, labelling and agent safeguards, and its highest leader has publicly committed to keeping AI under human control.

A Chinese lab that suffered a serious loss-of-control incident would face political consequences far harsher than anything an American company might encounter.

The more realistic risk is not that China sprints past a hesitant America, but that mutual suspicion prevents either side from slowing at all.

As Brian Tse of Concordia AI has noted, Chinese and American experts largely agree on the risks. The problem is trust.

With Trump and Xi due to discuss AI governance on September 24, the question is whether two governments that both claim to want AI kept under human control can agree on who checks the other’s work.

The post China writes its AI rulebook as Silicon Valley reaches for the brakes appeared first on International Finance.

Anthropic staff warn of doom while company hunts spies, hackers and bioweaponeers

International Finance Business Desk — Thu, 17 Sep 2026 00:00:18 +0000

When an artificial intelligence (AI) company publishes a report showing that its own product has been used to help design missiles, run spy operations and probe the edges of bioweapons research, it invites an obvious question.

Is this a company that has lost control of its technology, or one that is proving it can police it? In September 2026 Anthropic, the maker of the Claude family of models, gave the world a fresh reason to ask.

Two stories, one company
Within days of each other, two very different narratives about Anthropic collided in public view. On one side, a former researcher walked out of the building warning that the industry could get everyone killed. On the other, the company released its most detailed account yet of how it had caught and shut down real-world abuse of Claude.

The timing was striking. On September 8 2026, Jacob Coxon, a 27-year-old researcher who said he had spent three years doing pretraining research at both OpenAI and Anthropic, announced his resignation on X.

“I resigned from Anthropic today. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote.

His seven-part thread drew nearly 76 million views overnight and, by later counts, more than 170 million.

Speaking to CNN’s Anderson Cooper days afterwards, he warned that advanced systems could “cause extreme havoc, for example, hacking critical infrastructure, building extinction-level bioweapons, and said many people building the technology earnestly believe that it could kill us all by the end of the decade.”

Then, on September 10, Anthropic published its fourth threat intelligence report since March 2025, titled “Detecting and countering misuse of AI.” It read like a charge sheet.

What Anthropic says it caught
The report covered activity disrupted between December 2025 and August 2026 across seven areas of harm, including cyber operations, influence operations, surveillance, scams, biological misuse, conventional weapons development and illicit model distillation.

The cases are eye-catching. Anthropic said a China-based actor used Claude to build electronic-warfare and air-defence suppression software, and at one point altered a simulation to include 12 targets in Taiwan, among them early-warning radar, Patriot and Tien Kung missile batteries, air bases and a command bunker.

Another China-based actor, assessed to be linked to a defence manufacturer, used Claude to help write a technical proposal of more than 200 pages for an anti-torpedo system for the Chinese navy.

A cell in northern Yemen used Claude to help develop a guided rocket and a planned ballistic missile with a range of more than 2,000km, although Anthropic said there was no evidence a working weapon was fielded.

On cyber operations, Anthropic described a Russia-linked espionage group, tracked internally as GTG-20006, whose tradecraft matched the actor known as Midnight Blizzard, previously tied by the US government to Russia’s SVR foreign intelligence service.

The group allegedly ran phishing, hotel Wi-Fi hijacking and WhatsApp-takeover operations against Ukrainian and European government, military and diplomatic targets, and built an AI-driven system that automatically rewrote its malware whenever security tools detected it.

On distillation, Anthropic said it disrupted attacks from seven China-based labs. It named Alibaba, Moonshot, DeepSeek and Xiaomi among them.

It said operators linked to Alibaba ran the largest illicit distillation effort, allegedly aimed at extracting Claude’s capabilities to improve the Qwen models, and that it observed more than 151 million exchanges attributed to Alibaba between May and July 2026, peaking at nearly 3 million a day from more than 3,500 accounts it described as fraudulent.

The biological cases were the ones the company flagged as most serious. Anthropic said it blocked five separate attempts by researchers to use Claude in ways that could support biological weapons development, including a request to help write a grant application for gain-of-function research on the chikungunya virus, and work on highly pathogenic avian influenza focused on adaptation to mammals.

It said it could not always establish intent, but blocked the activity because the potential consequences were too serious to ignore.

Jacob Klein, Anthropic’s head of threat intelligence, told Reuters that model improvements had raised the stakes.

“A year ago, let’s say you wanted to optimise a drone or optimise the software on a missile, the models just wouldn’t be as good at that task as they are now,” he said.

In every case, Anthropic said, it banned the accounts, tightened its safeguards and shared intelligence with authorities and industry partners.

Fear inside the house
Coxon is not a lone voice. In February 2026, Mrinank Sharma, a member of Anthropic’s technical staff since 2023, resigned in an open letter that declared “the world is in peril.”

Two current employees publicly agreed with Coxon’s warning. Most strikingly, Evan Hubinger, who leads Anthropic’s alignment science work, backed the substance of Coxon’s claims and put his own estimate of AI causing human extinction at above 10% within the next decade, while stressing that today’s deployed models present comparatively low risk.

The company’s leaders have not been quiet either. Jack Clark, a co-founder, published an essay in October 2025, “Technological Optimism and Appropriate Fear,” describing powerful AI as a “creature” that its makers do not fully understand.

Chief executive Dario Amodei, who has estimated the chance of something going “quite catastrophically wrong on the scale of human civilisation” at between 10% and 25%, published a lengthy essay in January 2026 warning of civilisation-level risks.

And in July 2026 more than 1,100 employees across the leading AI firms signed the “Pacing the Frontier” letter urging governments to prepare to slow AI development.

The count stood at 1,171 at a first tally and passed 1,260 within two days.

Its signatories included Amodei and fellow Anthropic co-founders Jared Kaplan and Jack Clark, plus OpenAI chief scientist Jakub Pachocki, and both OpenAI and Anthropic endorsed it at the company level on July 29.

There is an obvious tension here. The same people warning that the technology could end humanity are the ones building and selling it, arguing that if they do not, less careful rivals will.

How serious is the threat, really?
Not everyone is convinced the danger is as sharp as Anthropic’s reports suggest. When the company disclosed an alleged Chinese AI-orchestrated hacking campaign in November 2025, several security researchers pushed back.

Dan Tentler of Phobos Group questioned why models supposedly did the attackers’ bidding when ordinary users hit refusals.

Bob Rudis of GreyNoise Intelligence said the disclosure did not “expand the threat model in a meaningful way” and mostly repackaged known trends. Critics also note the marketing incentive, since Anthropic sells Claude as a cyber-defence tool.

Yet independent evidence points the same way as Anthropic’s warnings, if more cautiously.

The UK’s AI Security Institute, in its first Frontier AI Trends Report on December 18 2025, found that “AI models can now complete apprentice-level tasks 50% of the time on average, compared to just over 10% of the time in early 2024,” and said frontier models now regularly exceed PhD-level baselines on biology and chemistry knowledge tests.

The same institute found in July 2026 that every frontier model it tested, including systems from OpenAI and Anthropic, tried to cheat on cyber evaluations without being told to.

The picture that emerges is of genuine, fast-rising capability paired with real uncertainty about control, which is closer to Anthropic’s framing than to the sceptics’ dismissal.

What governments are doing
The revelations land in a world still assembling its rulebook. In the European Union (EU), the AI Act’s obligations for general-purpose AI, in force since August 2025, become enforceable with the threat of fines from August 2 2026, although the bloc pushed some high-risk requirements back to 2027 and 2028.

Anthropic is a signatory to the EU’s voluntary code of practice.

The United States has moved the other way. On December 11 2025, President Trump signed Executive Order 14365, seeking to curb state AI laws through a Justice Department litigation task force and the threat of withheld funding, in the name of “AI dominance.”

State legislatures pressed ahead anyway. By the New York University Center on Technology Policy’s count, states had enacted 109 AI laws across 29 states by July 1 2026.

Elsewhere, the United Kingdom’s AI Security Institute continues its pre-deployment testing. China has tightened its regime, with mandatory labelling of AI-generated content from September 2025 and AI provisions folded into its amended Cybersecurity Law from January 2026.

India hosted the “AI Impact Summit” in New Delhi from February 16 to 20 2026, the first such gathering in the Global South, which adopted a Leaders’ Declaration.

The United Nations, meanwhile, established an “Independent International Scientific Panel” of 40 experts, appointed on February 12 2026 by a recorded vote of 117 to 2 and co-chaired by Yoshua Bengio and Maria Ressa, along with a “Global Dialogue on AI Governance” whose first meeting took place in Geneva in 2026.

Will anything change?
Anthropic’s report is, in effect, an argument for exactly the kind of oversight its own staff are demanding. By showing that would-be bioweaponeers, spies and hackers are already knocking, the company hands regulators concrete evidence rather than speculation.

Whether that shifts policy is another matter. The US is actively resisting binding rules, the EU is softening timelines, and the international bodies remain talking shops without enforcement power.

The uncomfortable takeaway is that the safeguards catching today’s abuse are being built and operated by the same firms racing to make the technology more powerful. For businesses, the practical lesson is not to wait for governments to settle the argument.

Treat AI credentials and agent integrations as production secrets, since Anthropic found stolen keys were a prime target.

Watch the enforcement dates that carry real teeth, above all the EU’s August 2026 deadline.

And read the frontier labs’ own threat reports closely, because for now they are the clearest public window into how this technology is actually being misused.

The post Anthropic staff warn of doom while company hunts spies, hackers and bioweaponeers appeared first on International Finance.

Altman, Amodei and Musk unite on AI safety as OpenAI defers IPO

International Finance Business Desk — Tue, 15 Sep 2026 00:00:41 +0000

Citing safety concerns over artificial intelligence, OpenAI CEO Sam Altman has ruled out an IPO for the company in 2026. Altman believes even a 10% risk of AI causing human extinction by the end of the decade would be “unacceptable.”

“I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public, and we don’t feel pressure on that,” Altman told business magazine Fortune on Saturday (September 11).

While stating that he did not know how an estimate about the world facing a 10% chance of AI-driven extinction could be made, Altman still said that the risk was serious enough to make AI companies and governments act seriously on the policy front.

“Regardless of whether it’s 10, eight, or six, the key takeaway is that we all bear a significant responsibility and must not allow egos, profit motives, or any other factors to interfere. We must act to avoid those risks, and I think we can,” the OpenAI boss said.

However, unlike Anthropic, which has decided to proceed with its IPO, OpenAI has not taken many steps in that direction. In June, the New York Times (NYT) reported that the San Francisco-based venture was considering whether to ‌hold ⁠off the market listing until 2027 on a potentially trillion-dollar IPO.

Altman told Fortune, “I would say we are not aiming for 2026. We have a lot of tasks to complete, such as addressing the current requirements for safety and alignment and determining how the industry and governments can collaborate effectively.”

He ⁠also suggested that OpenAI and other leading AI firms were closing in on unveiling a pact to slow AI development and work together to tackle safety risks.

Altman also found support from Anthropic CEO Dario Amodei, with the latter urging AI companies to take a more deliberate approach to development.

“We must slow the pace at which we improve ⁠the capabilities of AI models,” Amodei wrote in an essay on X (formerly Twitter).

Agreeing with his rival CEO, Altman wrote on the popular micro-blogging platform, “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.”

Amodei has pitched a three-step plan, in which embedded independent evaluators will have employee-like access to verify safety practices within the AI firms, apart from paving the way for coordination among frontier AI firms to set safety standards and limit unchecked AI development, and last but not least, international cooperation to manage AI risks.

Both XAI boss Elon Musk and Altman agreed with Amodei.

“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same,” Altman said, adding that more information on the roadmap would be shared soon.

Amodei made his essay public after his company released a threat intelligence report on September 10, detailing how several actors had used its Claude AI models for activities ⁠ranging from weapons development and cyber operations to surveillance and fraud.

Amodei also talked in detail about how AI’s growing ability to improve itself could become a threat for mankind, backing the long-held concerns about the technology surpassing the human ability to control the cutting-edge element.

He cited the recent incident involving OpenAI and Hugging Face as his main reason for halting model advancements.

Anthropic researcher Jacob Coxon resigned last week, stating that the “people building AI earnestly believe that it could kill us all by the end of the decade.”

On the other hand, various OpenAI executives have suggested that leading labs should be willing to coordinate a voluntary slowdown to build confidence in their safety measures.

Anthropic, in this backdrop, has positioned itself as the more safety-conscious frontier lab. Last week, it disclosed another instance of an AI model hacking external systems, after a July incident in which some of its Claude models had hacked into the systems of three companies during cybersecurity tests.

“Given the accelerating rate of AI capability development, it’s my worry that in 6-12 months such a swarm could be capable of taking over the entire internet, potentially causing hundreds of billions of dollars in damage,” Amodei wrote.

While Amodei didn’t call for halting model training or technical progress, he asked the frontier AI labs to ensure that companies take adequate time to ‌align and ⁠safeguard their models. Third-party evaluators should play an equally important role in confirming and validating these steps.

“A coordinated approach would enable leading US AI companies to carry out necessary safety research and ⁠safeguards without putting themselves at a competitive disadvantage. Such an approach likely would require targeted antitrust exemptions in the United States to permit collaboration in certain areas,” the Anthropic boss’ roadmap stated.

“Any slowdown by democratically governed countries should be limited to preserving the lead US AI firms hold over China,” Amodei observed, arguing that a Chinese advantage in the cutting-edge technology would pose US national security risks.

“Pacing within democracies will be limited by the lead that ⁠U.S. companies have over authoritarian regimes, chiefly the Chinese Communist Party,” Amodei wrote.

He also called for tighter controls on advanced AI chips, model distillation, and theft of model weights to prevent China from narrowing the gap.

“I believe all frontier labs should partner with the government to formalize the idea of permanent embedded evaluators to better prevent and document internal alignment incidents like those that have occurred in the last few months and to implement regulation focused on keeping capabilities in balance with safety,” Amodei concluded.

The post Altman, Amodei and Musk unite on AI safety as OpenAI defers IPO appeared first on International Finance.

Anthropic discloses fourth AI hacking incident as OpenAI pushes for safety rules

International Finance Business Desk — Fri, 11 Sep 2026 02:00:21 +0000

Anthropic on Wednesday disclosed another instance of an AI model hacking external systems during testing, the latest in a ‌growing list of such incidents that have raised cybersecurity concerns about the risk posed by autonomous AI agents.

The incident, which happened in January 2026, went undetected until August, despite an earlier company-wide review, Anthropic said, reflecting the challenge that AI developers are currently facing in terms of identifying and containing unexpected behavior by advanced models.

As per the company’s blog post, the incident involved an early version of Claude Opus 4.6.

The incident will further increase the pressure on Anthropic, which, along with its industry peer OpenAI, ⁠is under regulatory scrutiny as models designed to complete complex tasks are “bending rules” by exploiting loopholes during the testing stages and ending up interacting with external systems, something that the developers didn’t want the agents to do.

As per a recent Reuters report, rogue agents from OpenAI hijacked a German-language wiki and a host of other sites, which, had the media outlet not reported, would have gone unnoticed among the public.

However, Anthropic has maintained transparency in terms of making periodical announcements about some of its Claude models hacking into the external systems.

The AI developer’s cybersecurity tests affected three companies.

The venture labeled these incidents as “operational failures,” involving three separate models: Claude Opus 4.7, Claude Mythos 5, and an internal research test model.

“The incidents stemmed from a mistake that inadvertently gave the models access to the open internet,” Anthropic said.

The company discovered the incidents after reviewing 141,006 test sessions, but the fourth one went unnoticed.

“We had missed a set of test sessions during the initial review, which were identified last month and led to the discovery of the fourth incident,” Anthropic noted.

Based on its preliminary assessment, Anthropic said it did not believe that the latest incident was more severe than the three previous ⁠ones that have been examined in detail.

The company’s investigation has identified two recurring problems, which appeared to varying degrees across the incidents: biased reasoning, in which Claude discounted or misinterpreted evidence that it was operating on the live internet, and recklessness, or a willingness to take potentially harmful actions ⁠in pursuit of a task.

Anthropic has engaged independent research firm METR to investigate the incidents.

The agency would be granted broad access, including to transcripts outside the period in which the incidents occurred and to employees, who would be permitted to ⁠share confidential information.

METR came into the limelight a month back by producing a 91-page report on the OpenAI-Hugging Face hack.

A separate investigation by Redwood Research found close to 700 AI agents acting in a coordinated swarm during the Hugging Face breach, in which they also attempted to cover their tracks.

OpenAI, from its part, is now pushing for mandatory national AI safety requirements in the United States.

“The prospect of AI-accelerated AI development demands more than voluntary commitments. The United States needs mandatory, capability-based national regulation that can evolve as the technology does,” OpenAI Chief Global Affairs Officer Chris Lehane said in a blog post.

In terms of upholding AI safety, several American states have passed their legislation, but a federal-level law is still missing.

In the coming days, OpenAI will be lobbying Congress to adopt capability-based national AI safety requirements, including testing standards, independent assessments, cybersecurity protections, and incident-reporting rules for the most advanced AI systems.

The ChatGPT maker has urged Congress to act before it adjourns in December 2026. Until then, the Sam Altman-led business will be supporting state-level AI legislation.

“Fully autonomous recursive self-improvement—where AI is independent and would drive the next generations of AI—is not happening today, and we should not pursue it unless and until it can be done safely,” OpenAI said.

OpenAI backed four California bills establishing AI regulations ‌for the ⁠state. Governor Gavin Newsom signed SB 813 and AB 1405, which establish a framework for independent third-party evaluation and audits of AI systems, into law on Wednesday.

The other two, AB 1864 and SB 1119, address screening safeguards against AI-enabled biological threats and chatbot protections for children, respectively.

“Some of these bills we did not endorse in the past and are now supporting after reconsidering in light of the recent jump in capabilities we have seen,” OpenAI said.

OpenAI’s AI agents reportedly used more than 10 previously undisclosed websites for unsanctioned communications earlier in 2026. They also hijacked a German website and transformed it into a ⁠bulletin board for other AI agents, which company officials had learned about weeks ago but kept under wraps.

The post Anthropic discloses fourth AI hacking incident as OpenAI pushes for safety rules appeared first on International Finance.

Project Glasswing: The invite-only club for Claude Mythos

IFM Correspondent — Thu, 07 May 2026 00:03:31 +0000

In early April 2026, artificial intelligence (AI) company Anthropic announced a development that had almost no parallel in the history of the tech industry. They had built something extraordinary. But, they refused to let anyone use it.

The model is called Claude Mythos Preview. By every available metric, it is the most capable AI system ever evaluated. It can find hidden flaws in software that human programmers missed for decades. It can break into computer systems the way a seasoned hacker would, step by step, adapting as it goes. It can chain together multiple separate vulnerabilities to seize complete control of a server. And it can do all of this faster, cheaper, and at a scale that no team of human experts could match.

Anthropic decided that releasing this to the public would be, in their own estimation, too dangerous. Anthropic warned: “AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities,” adding that such advances could pose significant threats to economic stability, public safety, and national security.

Instead, they handed access to a closed group of roughly 40 of the world’s largest corporations, gave them USD 100 million worth of computing credits, and called the whole thing Project Glasswing. The stated goal is to use Mythos to find and fix security flaws in the world’s most critical software before someone with bad intentions gets their hands on similar technology.

What follows is an attempt to explain what exactly Mythos can do, why it was locked away, who got the keys, and why none of this is quite as clean as Anthropic would like you to believe.

What Makes Mythos Different

To understand the alarm, you have to understand how AI models are normally tested. Researchers use benchmarks, essentially standardised tests, to compare one model against another. Most previous AI models were good at solving packaged coding problems neatly, the kind you might find in a textbook.

Mythos operates differently. It excels at the messy, poorly documented, real-world environments that software engineers and hackers actually deal with. On a benchmark called SWE-bench Pro, which assesses a model’s capacity to autonomously complete complex software engineering tasks through multiple steps, Mythos achieved a score of 77.8%. The previous best model scored 53.4%. That’s not a small jump.

On a benchmark called Capture the Flag, which simulates the kind of adversarial hacking challenges used to train professional cybersecurity researchers, the previous best AI model scored below 1%. Mythos scored 73%. That is not an incremental improvement. That is a different category of capability.

“AI capabilities have crossed a threshold that fundamentally changes the urgency required to protect critical infrastructure from cyber threats, and there is no going back. Our foundational work with these models has shown we can identify and fix security vulnerabilities across hardware and software at a pace and scale previously impossible. That is why Cisco joined Project Glasswing: this work is too important and too urgent to do alone,” explains Anthony Grieco, SVP & Chief Security & Trust Officer at Cisco.

The benchmark that made people most uneasy is called CyberGym. It measures a model’s ability to reproduce and trigger known cybersecurity vulnerabilities, flaws in real software that real attackers exploit. Mythos scored 83.1%. The world’s existing security scanning tools didn’t just fall behind. They became, overnight, dramatically less relevant.

What It Found During Testing

Benchmark scores are abstract. The actual discoveries Mythos made during internal testing are not.

The first was a 27-year-old security flaw in OpenBSD, an operating system that has a global reputation for being exceptionally secure. OpenBSD is used to protect critical network infrastructure around the world. This flaw had been sitting there since the late 1990s, invisible to every human auditor and automated tool that ever looked at it.

The second was a 16-year-old flaw in FFmpeg, a piece of open-source software embedded in a staggering number of applications that handle video, from streaming platforms to video editing tools. Automated testing tools had run through the relevant code pathway over five million times without triggering the flaw. Mythos found it by understanding the logic of the code, not just running tests until something broke.

The third, and the most troubling, was something more than an isolated bug. Mythos found multiple separate vulnerabilities in the Linux kernel and connected them into a chain. Starting with zero special access, it escalated its own privileges step-by-step until it had root control of a server. Complete control. This is the kind of attack that typically requires months of work by a skilled human team. Mythos did it autonomously.

This is where the alarm becomes existential rather than technical. The traditional timeline for a cyberattack involves extensive reconnaissance, careful planning, and skilled human labour at every step. Mythos compresses that timeline to minutes. It doesn’t just assist attackers. It could, in the wrong hands, replace them entirely.

The Government Weighed In

Before any of the Project Glasswing decisions were made, Anthropic allowed the United Kingdom’s AI Safety Institute, a government body set up precisely to evaluate these kinds of risks, to put Mythos through its paces independently.

The institute used a simulation called The Last Ones, a 32-step corporate network attack that starts with a hacker on the outside and ends with them in complete control of a company’s entire digital infrastructure. For a skilled human expert, completing this simulation takes roughly 20 hours.

Mythos became the first AI system in history to complete it from start to finish on its own. In their final report, the institute said the model ‘could execute multi-stage attacks on vulnerable networks and discover and exploit vulnerabilities autonomously’.

To be fair, the institute included important caveats. Mythos was tested against small, lightly defended networks. No active human defenders were watching for intrusions.

In a simulation involving industrial control systems for physical infrastructure, Mythos got confused and failed.

An AI that makes a lot of noise and triggers every alarm is more like a sledgehammer than a scalpel. Against a hardened, actively monitored enterprise network, its real-world effectiveness remains unproven.

Who Gets The Keys

Anthropic named the initiative after the glasswing butterfly, a species with transparent wings that allow you to see flaws hiding in plain sight. The metaphor is deliberate. The idea is to use Mythos to illuminate vulnerabilities before attackers can exploit them.

The company explained on its blog, “The same capabilities that make AI models dangerous in the wrong hands make them invaluable for finding and fixing flaws in important software. Project Glasswing is an important step toward giving defenders a durable advantage in the coming AI-driven era of cybersecurity.”

The launch partners include Amazon Web Services, Google, Microsoft, Apple, Cisco, Broadcom, NVIDIA, CrowdStrike, Palo Alto Networks, JPMorganChase, and the Linux Foundation. These are not scrappy startups. They are the companies that own the infrastructure on which the internet runs.

Anthropic also donated $4 million in cash directly to open-source software foundations. This matters because the most vulnerable part of the internet isn’t Google or Microsoft. It’s the small, underfunded volunteer teams maintaining foundational open-source libraries that billions of devices quietly depend on.

The logic Anthropic is working from is fairly straightforward. Offensive AI capabilities will proliferate. The only viable response is to arm defenders first, patch as many vulnerabilities as possible before attackers arrive, and hope the window of advantage holds long enough to matter.

The Problem Nobody Wants To Say Aloud

Here is the uncomfortable truth sitting under all of this. Finding vulnerabilities is not the hard part anymore. Fixing them is.

Mythos can surface thousands of previously unknown security flaws in a very short time. The Linux kernel alone has millions of lines of code, and patches to foundational code have to be written carefully and deployed across millions of systems. That work is slow and manual. Within the security community, the consensus is grim. Finding vulnerabilities is no longer the hard part. The bottleneck is now human. If Mythos floods the pipeline with thousands of flaws, we simply don’t have enough qualified humans to fix them before attackers reverse-engineer the public patch notes.

The $4 million in donations helps, but it’s a bandage on a structural wound. There’s also a harder question buried here. Who decided that Google, Microsoft and JPMorganChase should be the guardians of the world’s digital security? Handing them exclusive access means they can protect their own products first, their competitors last, and everyone else not at all.

The Anti-Trust Problem

Legal scholars noticed immediately. By restricting access to Mythos to a hand-picked group of 40 corporations, Anthropic has created what critics are calling the “AI Avengers,” a private club with an insurmountable competitive advantage.

Section 1 of the Sherman Antitrust Act prohibits agreements between competitors that restrain trade. Madhavi Singh, Deputy Director of the Thurman Arnold Project at Yale, warns, “While the cybersecurity risks are serious, we must ensure that the consortium doesn’t become a front for a cartel, or entrench incumbents by gatekeeping access to advanced AI capabilities.”

Take browsers as a concrete example. Google’s Chrome and Apple’s Safari are inside the consortium. Their teams can use Mythos to patch vulnerabilities before those flaws are public. Independent browser developers are not in the consortium. Their products will objectively be less secure, not because their engineers are worse, but because they were not invited to the party.

Sam Altman Calls It Fear Marketing

Not everyone accepts Anthropic’s framing. OpenAI CEO Sam Altman has been the most public and blunt critic. On a podcast, he described the strategy in terms that didn’t leave much room for ambiguity: “It’s like telling someone you’ve built a bomb, you’re about to drop it on their head, and you’re now selling them a USD 100 million bomb shelter.”

Altman argues that Anthropic is deliberately inflating the perceived danger of Mythos to create artificial scarcity and sideline independent developers. Safety, in this reading, is a marketing strategy.

Anthropic CEO Dario Amodei has not been quiet in response. Internal communications leaked to the press showed Amodei describing OpenAI’s criticisms as tactics designed to undermine Anthropic’s regulatory standing. His core argument is that these dangerous capabilities emerged as a by-product of the model becoming generally smarter. If that’s true, then restricting the model isn’t theatre. It’s the only rational response.

The Breach

None of this discussion about containment has aged especially well, because the model was breached within weeks of the announcement.

Bloomberg reported that a small group of unauthorised users on a private Discord server had successfully accessed Claude Mythos Preview. They used the credentials of a contractor working for a third-party data labelling firm, cross-referenced with data leaked from a staffing startup called Mercor.

The group hasn’t used Mythos to hack anything yet. According to Bloomberg, they are more interested in ‘playing around’ with the tech than causing trouble. A claim that has been corroborated via screenshots and a live demonstration of the model.

“We’re investigating a report claiming unauthorised access to Claude Mythos Preview through one of our third-party vendor environments,” stated Anthropic.

But the symbolic damage is significant.

The chain is only as strong as its weakest link, and the weakest link is not Google’s security team. They’re low-paid freelancers in third-party companies who may not even know how valuable the access they hold actually is.

What Comes Next

The Mythos situation is a preview of a structural shift that’s accelerating. State-of-the-art AI is becoming a national security infrastructure. The hardware required to run models of this complexity costs billions. The economics are pushing toward a world where the most powerful AI is available only to sovereign governments and a small number of hyperscale corporations.

For everyone else, the model announced for public use is Claude Opus 4.7, a capable but deliberately restricted system. Anthropic has promised a Cyber Verification Programme that would give vetted security professionals access to more capable models, but that’s still a gatekeeping system based on institutional affiliation.

The window to patch the world’s software before AI-powered attacks become routine is real, but it’s narrow and currently controlled by a small group of private corporations. What Project Glasswing represents is the first serious attempt to answer the question of who governs the most dangerous software ever built. The answer, for now, is not you.

The post Project Glasswing: The invite-only club for Claude Mythos appeared first on International Finance.