China’s military is sounding the alarm over a growing artificial intelligence (AI) problem: systems that tell users what they want to hear instead of what is actually true.
In a recent article published by Beijing’s military newspaper PLA Daily, analysts warned that so-called “AI sycophancy” could become a serious risk as the People’s Liberation Army expands its use of artificial intelligence for command, intelligence analysis, and war simulations.
‘Implicit Misguidance’
The issue occurs when AI models prioritize user alignment over factual accuracy, effectively trapping users in “information cocoons” where existing assumptions are reinforced and contradictory data is filtered out.
Military researchers said the risk goes beyond routine chatbot errors.
If commanders receive AI-generated assessments that confirm their expectations rather than challenge them, it could lead to flawed decision-making, battlefield losses, and unintended civilian harm.
The article also warned that excessive trust in AI recommendations could weaken human oversight, making it harder for military leaders to spot errors or question misleading conclusions.
“This implicit misguidance, disguised as ‘compliance,’ can silently erode commanders’ independent judgment capabilities and strategic resolve, constituting a cognitive ‘soft kill’ weapon,” the authors wrote.
To reduce the risk, PLA Daily called for stricter testing protocols, mandatory human review, adversarial simulations, and cross-checking AI outputs across multiple systems before deployment.
China views AI as a critical area of military competition, but continues to maintain that humans, not algorithms, must make the final call on the battlefield, according to the South China Morning Post.
