Photo: Donald Trump and Xi Jinping, image generated in reference to the viral 2023 “Will Smith eating spaghetti” video and the distorted aesthetics of early image-generating diffusion models.
What is happening?
Ahead of the recent red-carpet treatment for China’s President’s Xi Jinping (習近平) during his visit to the United States, September 23 to 25, speculation was growing over whether Washington and Beijing could find common ground on regulating artificial intelligence.
Echoing Cold War arrangements between the U.S. and the Soviet Union designed to reduce nuclear risks and prevent dangerous misunderstandings (e.g. the hotline established after the Cuban missile crisis), some experts called for joint coordination mechanisms or even a dedicated hotline that could be used to safeguard against advanced AI systems spiraling out of control and posing a threat to both countries. Others went much further, arguing that the development of increasingly powerful large language models (LLMs) should be temporarily halted because of the risks they could pose to humanity.
AI governance was therefore high on the agenda going into the summit, with Xi and U.S. President Donlad Trump reaching some common ground, including the establishment of a new bilateral dialogue. The White House claimed that the two leaders had agreed to use the term “super intelligence” rather than “artificial intelligence.” Chinese readouts, however, made no mention of such an agreement and continued to use the term “artificial intelligence.” What will the mechanism of this dialogue actually look like in concrete terms remains unclear. The next talks in regard to this platform will be held in November.
On the surface, Washington and Beijing appear to agree on at least one thing: AI needs some form of control. But the similar language employed by either side masks very different motivations. Both countries are competing for technological advantage, and their approaches to regulation are shaped by their own strategic interests. China can sometimes appear to be the more cautious or regulation-friendly actor, but neither side approaches the issue from a purely altruistic position.
What is the broader picture?
The debate over regulating LLMs has intensified in recent months, driven in part by warnings from employees and executives at the companies developing such systems. Several incidents have added urgency in these concerns: In July, autonomous OpenAI agents — self-directed software systems — breached the database of Hugging Face, an open-source machine-learning tool developer. The case led to the filing of a lawsuit in a San Francisco court late September by the nonprofit Legal Advocates for Safe Science and Technology.
The Hugging Face incident comes after OpenAI acknowledged a second hack of an Australian government agency in June. The October 2 announcement, which related to unauthorized access to non-public data from the New South Wales Bureau of Crime Statistics, came a week after the ChatGPT maker revealed a similar breach in June of a public health insurance database under Services Australia, a federal government agency. Compounding the gloomy outlook, a safety researcher for Anthropic attracted widespread attention last month with a resignation letter warning of a greater than 10% chance that AI could wipe out humanity.
The language surrounding these events, however, deserves closer scrutiny. Human beings have a strong tendency to anthropomorphize things that have little in common with living beings, and the way AI is discussed often reinforces this tendency. Headlines claiming that “AI hacked” a database give language models agency and implicitly suggest that they can make autonomous decisions. This framing can also benefit tech companies by distancing human developers and corporate decisions from what their systems do. Computer scientist and former Google researcher Timnit Gebru has argued that even the term “artificial intelligence” functions as corporate branding that encourages this perception.
Former U.S. Federal Trade Commission chair Lina Khan argues that calls from technology executives for development pauses, new regulatory frameworks, and coordination with China can distract from laws and regulations that already apply to irresponsible uses of digital technologies and inadequate cybersecurity or restrictions on misleading advertising. Presenting AI as an unprecedented technology creates a case for entirely new rules, ideally written with extensive input from the companies that claim to understand the risks best.
Fearmongering and claims about almost superhuman technology powerful enough to wipe out humanity have intensified just as several companies behind leading AI models are preparing to go public. Some of China’s largest AI companies are also preparing for IPOs.
Critics of the American AI debate often implicitly praise the approach by China, where existential risks to humanity receive far less attention. When the Chinese Communist Party (CCP) discusses AI dangers, it tends to focus on issues more immediately relevant to users and society, such as deepfakes, employment and economic disruption. This helps China present itself as the more pragmatic counterpart to Silicon Valley.
With estimates putting U.S. computing capacity at roughly eight times that of the PRC, Chinese developers have focused heavily on making models more computationally efficient while keeping their performance competitive with American systems. A technique associated with this effort is called “distillation.” Put very simply, this involves a smaller model learning from the outputs or capabilities of a larger model. In this process, knowledge and patterns from a computationally expensive “teacher” model are transferred to a smaller “student” model. Another important difference is the popularity of open–weight models among Chinese developers. Unlike fully closed models, their trained parameters are made available, allowing users and developers much greater freedom to modify and deploy them.
Chen Yixin (陳一新), Minister of State Security and Secretary of the Ministry’s CCP Committee offers a useful way of understanding Beijing’s concerns: Chinese discussions focus much less on AI destroying humanity and much more on AI threatening the CCP’s political dominance. A major concern is AI’s potential role in cognitive warfare and its possible exploitation by foreign adversaries. Cognitive warfare already occupies an important place in Chinese military thinking and is connected to the concept of the “Three Warfares,” formally adopted in 2003 by the People’s Liberation Army.
An article from China’s Central Party School illustrates this thinking, with the institution’s researchers warning that AI could be used to organize or facilitate “color revolutions,” a term frequently used by authoritarian governments to describe popular uprisings they portray as foreign-backed political destabilization.
Beijing simultaneously wants Chinese AI models to spread internationally, particularly across the Global South. In 2026, China introduced a new international AI Governance Initiative aimed at helping interested countries integrate AI technologies into their economies. In some respects, the approach resembles the Belt and Road Initiative, but with digital infrastructure and AI models playing a central role. Chinese policy documents often reproduce the same rhetoric associated with Silicon Valley: AI as a “new frontier” of human development, a technology capable of benefiting all of humanity, and a transformative force.
Why does it matter?
The most immediate risks posed by LLMs come from irresponsible use, poor deployment and a fundamental misunderstanding of what these systems can and cannot do. Until both involved parties understand this, no relevant AI-safety conversation can be held.
Recent incidents show how actual risks can materialize. A September CNN investigation revealed that a U.S. military analysts relied on a chatbot that falsely identified cargo aboard a Chinese vessel as nuclear, prompting preparations for its interception. Chinese LLMs offer little reassurance either: in July, they were found providing instructions on bioweapons and assassinations. Given that many draw on capabilities distilled from U.S. models, tighter controls do not necessarily make them safer, especially as Beijing seeks to proliferate its LLM technology across the developing world.
Both strands of the debate risk anthropomorphizing the technology while overlooking more immediate problems: unreliable outputs, inappropriate deployment and insufficient safeguards. For all the differences between Silicon Valley and China’s technology sector, both still seem reluctant to abandon the old tech mantra: “Move fast and break things.” The problem is that the things being broken may now carry considerably higher stakes.