China’s greatest technological ambition and its greatest political obsession are quietly destroying each other.
The same censorship apparatus the Party built to control its people is now corrupting the AI systems its leaders depend on. The United States, by leaning into an open marketplace of information and ideas, will gain advantage as it takes a different path.
AI is increasingly training newer, faster AI models. This typically involves scraping the internet for content and then loading it into datasets for new programs. The problem: online content used for training is increasingly generated by AI. As a result, each generation of technology drifts from reality.
AI researchers refer to this as “model collapse,” a phenomenon in which models trained on their own synthetic outputs degrade over successive generations. The only defense is a constant influx of fresh, honest, human-generated information. Without it, the system folds in on itself.
China’s Great Firewall cuts off that influx, expediting the impact of model collapse within its borders. The first generation of large language models worldwide was trained on massive datasets of publicly available human-generated text. These human-derived pieces of information built an algorithmic approximation of how people think, argue, explain, and communicate.
Now, the internet is filling with AI-generated content at a rate inconceivable five years ago. Marketing copy, product descriptions, social media captions and news summaries are increasingly produced by AI systems and published online. They’ve all become generic and detached from human signals.
Outpaced by the US, China’s military places selective bets on artificial intelligence
With each cycle of new AI, the models drift further from their human origins. The newer systems amplify the patterns AI systems favor while losing nuance and amplifying existing biases. Each successive AI generation is one step further removed from the humans these programs were originally built to serve.
In China, the Great Firewall accelerates this problem. The Chinese Ministry of Public Security built the Great Firewall of China in the late 1990s to censor the Chinese people. It is now the most sophisticated information control infrastructure in human history. The Great Firewall does not just restrict what Chinese users can see. It shapes the data used to train Chinese AI systems. By design, it strips out politically sensitive events, dissenting viewpoints, and independent reporting. Left behind is a curated record of reality aligned with the Party’s narrative. That filtered record of information becomes the raw material for LLMs.
The training data fed into LLMs in China does not contain any criticism of the government, fair reportage of controversial topics, or accurate information about Chinese history. Events such as the the Uyghur detention camps exist inside the Firewall only as the state chose to describe them.
Now add model collapse on top of this. Chinese AI companies such as Baidu, Alibaba, ByteDance and dozens of others aggressively deploy AI-generated content across their platforms. This material becomes training data for the next generation of Chinese AI models. With independent reporting locked out, model collapse expedites inside the Great Firewall with no escape valve. The practical consequences of this divergence are already visible and will accelerate.
Chinese LLMs struggle with tasks that require new observation, original synthesis or human complexity that the training data was designed to suppress. If asked about repression in Chinese history or controversial current events such as the Uyghur detentions, these LLMs either do not answer or produce a response indistinguishable from a Party press release. A Chinese trade official relying on domestic AI to model the economic impact of Western sanctions is working from a system incapable of providing an honest account of how those sanctions functioned or failed in comparable historical cases.
The West has a milder version of this problem.
Western AI models are trained on increasingly synthetic content, but human reporters regularly push new information into the ecosystem. Free and open societies have structural advantages to retain the capacity to reason about what is happening in the world rather than what previous AI systems described. When asked about Tiananmen Square, Western accounts typically focus on the 1989 protests and the crackdown. By contrast, Chinese models either refuse to answer or return state-aligned language. This information gap becomes part of the training data for the next generation.
To maintain a competitive AI advantage over China, Washington should treat human-generated data as a strategic asset and invest in journalism, open web archives, and synthetic-content labeling. Preserving the integrity of American training data is a defense imperative, not a tech problem.
Chinese leaders deploying AI products to make decisions about economics, geopolitics, and public health will make those decisions based on systems trained on what China’s information control apparatus wants people to believe. That is not an intelligence system. It is a mirror. And the tragedy of model collapse is that a mirror that has been looking at itself long enough no longer reflects anything.
Joe Buccino is a retired U.S. Army colonel and the author of “When Every Word Counts: How to Earn Trust, Command Attention, and Communicate Clearly in Any Situation.”
Opinion
C4ISRNet
[crypto-donation-box type=”tabular” show-coin=”all”]
