MIT study links AI-altered photos to false memories
AI-generated photographs and summaries can distort people’s memories and leave them confident in inaccurate recollections, researchers warn.
MIT scientist Pattie Maes told The Washington Post that participants in her team’s research who viewed AI-altered photographs were more likely to misremember original images and were overly confident in those false memories.
Participants first viewed a series of photographs and then took a break. Some were subsequently shown the same photographs, while others saw AI-altered versions, before taking a memory test.
Numerous other studies have found that AI-generated misinformation can implant false memories and change what participants believe to be true. A study last year found that deepfakes and AI-generated text planted false memories about entirely fabricated political scandals and reduced voting intention.
Psychological experiments dating back to the 19th century have documented human memory’s vulnerability to suggestion. Some studies show that misinformation can continue to influence decisions, including political choices, even after corrections have been issued.
AI-generated content is becoming more convincing and widespread online, enabling and accelerating fraud and disinformation. Generative AI can also produce inaccurate output, known as hallucinations, without anyone deliberately using it to mislead.
Government agencies, banks, militaries and hospitals are rapidly deploying AI systems. These organizations often emphasize keeping “humans in the loop,” with a person, often a senior expert, auditing the system’s output.
The intended approach combines AI with human oversight to speed processes ranging from military strikes to record-keeping. Human reviewers are expected to catch errors before they lead to actions that cause harm.
Research indicates that users are not particularly effective at spotting hallucinations. Imperfect memory and a tendency to trust AI can leave even experts vulnerable to distorted judgment when a system’s output contradicts information they know to be true.
Studies put the average person’s ability to correctly identify AI-generated content at roughly the level of a coin toss. Researchers are developing interventions and training methods to improve deepfake detection, but no foolproof solution exists.
The AI industry is meanwhile developing agentic systems intended to operate autonomously, removing humans from the review process.
An American military strike on an elementary school in Iran earlier this year killed more than 150 people, including at least 123 children. An internal Department of Defense investigation, reported by Bloomberg last month, concluded that overreliance on a Palantir-developed AI system partly caused the strike.
The investigation found that the system relied on outdated data and recommended the elementary school as a target for the first day of strikes.
Featured image credit
