New research has identified a potential reliability problem in the memory-saving techniques used by DeepSeek’s latest models. The study found that the same information can be significantly easier or harder for a model to retrieve depending on where it appears within a long input.
The paper, led by researchers from ByteDance Seed with co-authors affiliated with Princeton University, Stanford University and the University of California, Berkeley, examines what the researchers call “phase sensitivity.” The study was posted in late September and has not been peer-reviewed.
DeepSeek uses compressed key-value caches to reduce the memory and computing requirements of processing long contexts. The technique groups consecutive tokens into windows and compresses them into fewer cache entries. That can substantially lower the infrastructure required to operate large models, but the researchers found that it also creates periodic weak points in information retrieval.
In tests using contexts of about 128,000 tokens, retrieval accuracy in some DeepSeek-V4 models varied by as much as 40 percentage points depending on the position of the target information relative to the compression window. The researchers found the pattern in DeepSeek-V4, DeepSeek-V4.1 and every compressed model they pretrained for the study.
The effect can appear even when the underlying task does not meaningfully change. In one code-completion experiment, researchers altered only irrelevant text placed before the same piece of code. The model’s preferred answer changed periodically as the added text shifted the code’s position within the compression structure. DeepSeek-V4.1-Flash showed a similar pattern with a different interval corresponding to its own compression settings.
The researchers also trained smaller transformer models from scratch with different KV-cache compression methods. Every tested model using chunk-based compression developed similar positional sensitivity, while full-attention models did not show the same periodic pattern.
The findings are relevant to DeepSeek because reducing the cost of long-context inference has become a central part of its model architecture. DeepSeek-V4 uses compressed attention techniques to lower memory requirements, while the newer V4.1-Flash further reduces its KV-cache footprint.
According to DeepSeek, V4.1-Flash requires one-quarter of the high-bandwidth memory and one-eighth of the persistent SSD cache storage used by the previous V4-Flash model.
The paper does not conclude that compressed models are broadly inaccurate. Instead, it warns that average benchmark results can conceal systematic weaknesses tied to the position of information in a prompt. A model can therefore perform well on average while remaining significantly less reliable for information placed at specific positions.
The researchers argue that evaluations of models using chunked KV-cache compression should test the same information at multiple positions rather than relying only on average long-context scores. Whether the effect causes significant problems in everyday applications beyond controlled retrieval and code-completion tests remains an open question.
Featured image credit
