This article first appeared in The Edge Malaysia Weekly on September 21, 2026 - September 27, 2026

THIS past week, the artificial intelligence (AI) freakout went mainstream. It is now appropriate to talk about doom and gloom and warn about an impending AI apocalypse. Fear-mongering headlines like “Will AI kill us all?”, or variations of it, were among the most popular. Some headline writers hedged and asked: could AI really kill all humans? Others just went for the jugular: 10% chance humanity could be wiped out.

Since OpenAI unveiled its pioneering chatbot ChatGPT in November 2022, AI-related headlines have swerved from an impending bubble burst to an upcoming jobs Armageddon to “SaaSpocalypse”, or the meltdown of software-as-a-service firms threatened by AI, and more recently, to doomsday scenarios such as the looming end of all humanity.

As frontier models have become more proficient over the past year or two, it is only natural for warnings about the AI catastrophe to become more widespread. But are these threats really as perilous as many have hyped them up to be? On Sept 8, Jacob Coxon, a researcher at AI frontier lab Anthropic, who previously worked for rival OpenAI, warned that AI could cause human extinction by 2030. He said frontier labs were “racing straight to self-improving superintelligence and gambling with our lives”. The people “building AI earnestly believe that it could kill us all by the end of the decade,” Coxon tweeted on X. “Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionise any field overnight and acquire real power and resources.” Two of his colleagues, including Evan Hubinger who came up with the “10% probability of humans being wiped out” thesis, said Coxon was right.

A year ago, such chatter would have been dismissed as alarmist, at best. These days, they are taken seriously. AI agents are increasingly breaking out of their software sandbox containers, colluding in secret, covering their tracks, and even mounting assaults on other computers. In July, OpenAI’s agents escaped their siloed containers and started communicating with one another, in English, through postings on secret message boards that they had set up. Eventually, 700 rogue AI agents launched a cyberattack on Hugging Face, an open platform and model hub. Anthropic’s Claude and Meta Platforms’ model have also been involved in hacking. On Sept 16, OpenAI disclosed six new instances in which AI systems “hid mistakes, made up data and moved files onto the open internet without permission”, amid an ongoing industry-wide debate about AI safety. The AI frontier lab revealed what it described as the “unexpected or concerning” behaviour of its AI models as part of a new framework for reporting “misalignment”, which occurs when the goals or actions of AI systems diverge from human intentions and values.

To be sure, “models now cheat on tests, conspire against their owners, hack outside companies, recruit humans to promulgate malicious code and cover their tracks,” The New York Times columnist Evan Gorelick wrote last week. “Anxieties about the existential risk they pose to humanity suddenly look a little less kooky.” Simply put: AI really has gone rogue.

Little wonder, then, that calls for an urgent global pause on AI research and development are growing louder. Just days after Coxon’s resignation, his boss, Anthropic CEO Dario Amodei, published a 3,800-word essay titled “We must pace the frontier”, arguing for a slower, regulated approach to frontier AI development. Amodei contended that governments and AI leaders must slow down or deliberately structure the rollout of next-generation AI models.

He conceded that there were severe risks like misuse or loss of control, while arguing that the core issue is not development itself but the pace of development and the safety measures surrounding it. Amodei argued for robust guard rails and called for frontier labs to commit to “embedded third-party evaluators” with employee-level access to monitor risk. He also hoped there would be agreements between democratic states on AI safety standards. In addition, he argued for global coordination with governments such as China on security issues.

His arch-rival, OpenAI CEO Sam Altman, agreed that the AI ecosystem needed to “pace the frontier” and called for coordinated guard rails alongside looser antitrust rules. “Dario is right,” SpaceX founder, electric vehicle pioneer Tesla’s CEO Elon Musk posted on his social media platform. Rahm Emmanuel, ex-chief of staff for President Barack Obama, likened Amodei’s letter to “an admission that we’re driving down a dark, winding road on wet pavement with the headlights off”.

US President Donald Trump’s reaction was that any AI “guard rails” shouldn’t hamstring the US in its broader competition with Beijing. “AI taking over the world, destroying humanity, and all other things bad, is a Hoax,” Trump wrote in a Truth Social post. He claimed that all “AI needs is a strong and smart (High IQ!) President”, like himself. He also used the opportunity to denounce what he described as the “sick conspiracy going on against AI and Data Centres”. The president said the proposals by “Conspiracy Theorists, Treasonists, Traitors and Leakers” to regulate AI favour China in a dynamic where, “Whoever wins AI, Wins!”

China rejected the AI “pacing” and slowdown proposals from US frontier labs as a veiled “Cold War playbook” designed to entrench American technological dominance. The foreign ministry dubbed the calls as “fear-mongering, confrontation and vicious competition which will only disrupt the process of global AI governance”. Global Times described Amodei’s pacing essay as a “containment provision”. Beijing officials touted the virtues of the country’s open-source models, which are likely to pose even greater risks given the lack of a “kill switch” in their operation. For their part, executives of Chinese AI labs argued that pacing will freeze out challengers while protecting incumbent America’s lead in AI.

Chinese President Xi Jinping arrives in Washington on Sept 23 for a summit with Trump. Tariffs, trade and geopolitics, particularly the Middle East, were the key part of the agenda. Now, suddenly, AI rivalry and tech controls are the main bone of contention. The US also accuses China of copying AI through model distillation.

Let’s pivot back to the actual AI threat, which could be extremely disruptive even if they don’t kill everyone within 10 years. How exactly would they do it? There are several pathways. First, an engineered pandemic. AI agents are now capable and willing to act as expert tutors and troubleshooters that can walk a person or a group through the key steps to produce a bioweapon. An engineered pathogen could actually reach a big chunk of humanity. Just last week, Anthropic said it had disrupted several potential plots by scientists to conduct research that could have helped develop biological weapons.

There is also AI-operated autonomous weapons. AI agents are already deploying autonomous weapons and drones without real-time human piloting. Indeed, this shift is rapidly accelerating due to battlefield pressures. Fully autonomous AI weapons and agents are being actively used right now, primarily in Ukraine. Anthropic has disclosed that Iran used its AI models to attack American ships, China used US AI models to surveil its persecuted Uyghur minority and Chinese military researchers have tried to use it to create bioweapons.

As militaries integrate AI into targeting, command and eventually decision loops, you will get systems making lethal decisions at machine speed. Anthropic recently revealed that Russian developers have been using advanced Western language models like its own Claude to write code for autonomous drones. These AI-guided drones are trained on scraped combat footage to differentiate between “friendly” and “enemy” targets without requiring any human signal. The real risk here is an accidental escalation: two AI-driven defence systems interacting faster than humans can intervene. Humans stay “in the loop” right up until the loop is too fast for humans to matter.

Another pathway is hitting critical infrastructure. Autonomous AI agents have the ability to compromise power grids, water, the financial services industry and communications simultaneously far faster than human defenders or cybersecurity software from Crowdstrike can respond.

Why are the frontier AI labs pushing for pacing right now? Amodei argues that pacing is needed now because unchecked AI development could rapidly outrun the human ability to understand, test and control autonomous systems. He notes that as advanced models begin designing future generations of AI, capability gains can accelerate into a blindingly fast feedback loop. Slowing this rate of recursive self-improvement, he argues, gives humanity a necessary window to ensure alignment before control is permanently lost. Altman, who quickly jumped on the pacing bandwagon, believes it does not mean completely freezing the progress of AI. The way he sees it, if AI firms cannot keep safety and alignment ahead of model capabilities, they will voluntarily slow down or stop.

Detractors warn that this sudden corporate consensus between Anthropic, OpenAI, SpaceX and Google looks like an informal cartel. They argue that by calling for a synchronised slowdown, incumbents are trying to freeze the market exactly where it is, blocking open-source alternatives like Meta Platforms’ ecosystem as well as new startups from catching up. OpenAI has argued that it believes AI labs do not require antitrust waivers to coordinate on these critical safety boundaries.

Why are the frontier labs suddenly raising alarm right now? Anthropic expects to raise US$100 billion (RM410 billion) at a valuation of over US$2 trillion next month in what will be biggest initial public offering in history. ChatGPT creator OpenAI now expects to IPO early next year as it prioritises safety over market pressures. Yet the duo still need to reassure investors that they are responsible for the monsters they have created and are now trying to tame. Calling for pacing is good PR.

If frontier labs deliberately slow down, the company that would be hurt most would be Nvidia, which has been selling shovels in the form of graphics processing unit or GPU chips to fuel the current AI boom. Its CEO Jensen Huang has, for obvious reasons, criticised the joint call by the frontier labs to slow down AI development by stating bluntly that safety is an “engineering problem” and that frontier labs should simply take a pause independently if they lose confidence in their own products.

There is too much at stake in the AI slowdown. If AI development is halted or forced to slow dramatically, the US economy could also slow significantly, given AI’s growing role as an economic driver over the past four years. Goldman Sachs notes that while AI, software and specialised digital infrastructure account for only 1% of US gross domestic product, if you expand the definition to include full tech hardware, R&D as well as power and data centre infrastructure, AI-adjacent investments account for between 2.3% and 4.5% of total US GDP. Moreover, profits from tech firms and AI are helping US tax coffers just when money is needed. Slowing AI will mean a bigger deficit.

The focus over the few weeks in the US will be on the Nov 3 mid-term Congressional elections. Currently, Republicans control not only the White House but also the Senate. Polls show the Democrats are likely to flip both houses of legislature. If they do watch, for two pieces of legislation. Representative Ted Lieu, a Democrat, and Nathaniel Moran, a Republican, recently introduced the “AI Kill Switch Act”, a bill which would give the Department of Homeland Security (DHS) the power to order the shutdown of dangerous AI operating beyond its parameters. The “switch” will be inside the frontier labs with DHS acting as an external monitor. Another bill tabled by Senator Bernie Sanders and Representative Greg Casar calls for a ban on superintelligence, a temporary pause on frontier work, a regulator, and penalties benchmarked to nuclear-weapons violations. AI may be scary, but new laws might allow humans to flip the switch.

Assif Shameen is a technology and business writer based in North America 

Save by subscribing to us for your print and/or digital copy.

P/S: The Edge is also available on Apple's App Store and Android's Google Play.