AI eases military planning but verification falls short

AI decision support can reduce the workload in military planning, but does not necessarily lead to better operational plans. These are the findings of a new report examining how military staff officers use AI during a planning exercise.

During the exercise Operation STORMBREAKER, two working groups planned a military operation with the same task, timetable and documents. The difference was that one group had access to an AI system that could search and summarise the exercise’s roughly 1,800 documents.

The results point to both opportunities and shortcomings. The AI-supported group produced a Mission Analysis Brief of comparable quality and reported more than a third less frustration. At the same time, fabricated claims from the AI system were incorporated into the planning product without being challenged.

“Our overall conclusion is that the technology is worth adopting, but human verification must be trained and required”, says Linnéa Gelot, Professor of War Studies at the Swedish Defence University and one of the report’s authors.

Credible answers lacked support in the sources

AI support changed the type of error rather than the number. The researchers identified three deficiencies in the AI-supported group’s Mission Analysis Brief. These were an invented NATO end state, an invented restriction on targeting leadership, and a theatre of operations that included neutral third-country territory. All three were fluently worded and accompanied by citations, but had no basis in the source documents. None was challenged by the exercise participants. In the group without AI support, the deficiencies instead consisted of manual misrepresentations of correct source text.

Few queries were used to check answers

The researchers also reviewed the 732 queries that participants put to the AI system. Only 3.6 per cent were intended to check an answer. Of the 19 users, 13 never carried out such a check. Yet the participants believed they had verified the answers.

The study shows that verification can catch fabricated claims. When a participant challenged an answer, the system withdrew the claim, and it never entered the plan. However, the researchers also found that time pressure could lead to detected errors being included nonetheless.

“Technical testing can show what a system is capable of, but once it is in use in a staff, safety depends largely on how people use it”, says Linnéa Gelot.

Verification needs to become part of the process

The report proposes 18 practical, low-cost controls. One example is requiring two military staff to check AI-generated content before it enters particularly critical parts of the plan, such as the end state, constraints and restraints, theatre of operations and rules of engagement.

Other proposals include assigning a staff to challenge the output and agreeing on which sources should be authoritative. Systems also need to distinguish more clearly between retrieved source text and AI-generated text, and flag content that the sources do not support.

The controls can be built into existing working routines and use some of the time saved by AI support. The report also highlights the need to practice verification in professional military education.

“The report provides empirical evidence on AI safety in practice: how fabricated claims arise, why they go unnoticed and what it takes to catch them”, says the report’s co-author Ludwig Gelot, who holds a PhD in International Politics and works as exercise coordinator at the Command and Control Warfare School in Enköping.

The exercise took place at the Military Academy Halmstad on 11–14 May 2026 and followed NATO’s planning process. A total of 40 people took part, divided into two groups. As the study covers two groups and one planning product per group, the results should be read as indicative. They are, however, consistent with earlier research on people’s tendency to trust automated systems.