Verified by the Machine: When an AI Chatbot Confirmed Wartime Fakes as Real

When war footage flooded X, millions asked the platform's AI chatbot a simple question: is this real? Too often, it said yes about fakes — sometimes changing its answer within minutes, sometimes inventing a source. Researchers analysed 130,000 of its replies. Inside the Case File: how the verifier became the vector, why this is true of AI chatbots in general, and the one defence that survives the fog of war. Members only.

Case Summary

During two flare-ups of conflict between Israel, Iran and the United States — the 12-day war of June 2025 and a renewed escalation in early 2026 — social media platform X was flooded with AI-generated and fabricated "war footage." Users increasingly turned to Grok, the platform's built-in AI chatbot, to verify what they were seeing. It repeatedly failed. Analysing roughly 130,000 Grok posts from the June 2025 conflict, the Atlantic Council's Digital Forensic Research Lab found the chatbot inconsistent in its fact-checking, struggling to authenticate AI-generated media, at times labelling fabricated videos as real footage.

In one documented instance, asked to verify an AI-generated video of a destroyed airport, the chatbot oscillated between conflicting answers within minutes. The core danger was not only that fakes circulated — it was that an authoritative-sounding machine "confirmed" them.

This is a Case File not about a single deepfake, but about the tool millions reached for to check the deepfakes — and how it laundered them into "verified."

Why This Case Matters

Every other synthetic-media case warrier documents assumes the victim is fooled by the fake. This one exposes a second, deeper failure: the verification step itself is now automated, confident, and wrong. When a person asks a chatbot "is this real?" and the chatbot answers "yes" about a fabrication — sometimes inventing or misattributing a source to back itself up — the fake is no longer just circulating; it is circulating with an apparent stamp of authority. In the fog of war, where the shared factual record is most fragile and most contested, an over-confident AI "fact-checker" does not merely fail to stop disinformation. It accelerates it.

advertising