US Military Cancels Plan to Intercept Chinese Ships After Near-Miss with AI-Generated Disinformation

It has been revealed that an intelligence report written by an analyst at U.S. Special Operations Command Pacific (SOCOM Pacific) using an AI chatbot incorrectly identified Chinese ships in the Middle East as carrying components for a nuclear weapons program. The report was entirely false, yet the U.S. military prepared ship interception operations, including boarding by armed forces and deployment of military aircraft, based on it.

The operation was canceled after confirming just before execution that the report was AI-generated, but the incident posed a risk of escalating into war. The analyst queried the chatbot with a ship cargo manifest at SOCOM Pacific in Hawaii, and the AI inaccurately identified the nature of the data, leading to erroneous conclusions.

Accelerating AI Adoption and Lack of Verification

The U.S. Department of Defense is accelerating AI adoption to improve decision-making speed against adversaries such as China. In January, Defense Secretary Pete Hegseth announced the 'AI Acceleration Strategy,' providing 3 million personnel with AI access at all classification levels. However, different departments are using different tools and standards, and there is no unified standard for verifying AI-generated information.

Real Risks: Human Fatal Decision-Making

This case demonstrates that the more immediate risk is human fatal decision-making based on inaccurate AI information, rather than a loss of control over AI. The military is rapidly expanding the use of AI in targeting, but there are no practical human-in-the-loop guidelines to prevent civilian casualties or friendly fire. It is pointed out that excessive trust in AI and pressure for speed among young analysts are causing errors.