AI study shows sharing transformer projections works
There is a particular kind of silence that falls over a research lab when a machine starts doing something a person used to be celebrated for. It is not the silence of failure, but the silence of sudden, uncomfortable redundancy. For years, the design of the transformer architecture was considered a craft, a delicate balance of mathematical intuitions that required a specialist’s judgment. The query, key, and value projections — the QKV mechanism — were the holy trinity of attention, and tuning them was an art form. In 2025, a systematic study published in Science has quietly demonstrated that this art form may have been largely unnecessary, and the person who practiced it is now facing a different kind of future. [1]
The study does not announce a dramatic breakthrough in capability. It does not boast about beating benchmarks or creating a new state-of-the-art model. Instead, it performs a far more unsettling act: it shows that a significant portion of the architectural decisions made by human experts over the past decade were, at best, optional. By systematically evaluating what happens when you force the query, key, and value projections to share weights, the researchers found that models perform on par with, or occasionally better than, the standard configuration. The specialist who spent years mastering the nuances of these projections is not being replaced by a better specialist; they are being replaced by a simpler rule.
To understand the weight of this finding, one must recall the mythology surrounding the original transformer. When it was introduced, the separation of QKV was presented as a conceptual breakthrough, a way for the model to ask different questions about its input. The query was the question, the key was the label, and the value was the content. This tripartite structure was taught in every advanced machine learning course, and it became the foundation for countless derivative architectures. The human role here was not just to implement this structure, but to reason about it, to explain why it worked, and to intuit when it might fail. That role is now being dissolved by empirical evidence that suggests the structure itself was a kind of over-engineering.
The implications for human judgment are starkest in the language modeling experiments. The researchers trained models with up to 1.2 billion parameters on 10 billion tokens, a scale that requires significant computational resources and human oversight. They found that a variant called Q-K=V, where the key and value projections are shared, achieves a 50 percent reduction in the key-value cache with only a 3.1 percent degradation in perplexity. [1] This is not a niche finding; it is a direct challenge to the assumption that the full QKV formulation is necessary for quality. The human expert who would have argued for the importance of separate key and value spaces is now confronted with data that suggests those spaces can be the same without meaningful loss.
This is where the technology stops being a tool and starts being a replacement for a specific kind of expertise. Consider the role of the inference engineer, the person whose job it is to make models run efficiently on limited hardware. This person once had to make complex trade-offs, deciding how much memory to allocate to the key-value cache and how to balance speed against accuracy. The new findings offer a simpler path: just share the projections. The engineer’s nuanced judgment, honed over years of profiling and optimization, is reduced to a configuration flag. The skill of balancing trade-offs becomes obsolete when a single change, derived from a general principle, delivers most of the benefit.
The researchers also explored combining this projection sharing with head sharing techniques like GQA and MQA. The results are striking in their totality: combining Q-K=V with GQA-4 yields an 87.5 percent cache reduction, and combining it with MQA achieves 96.9 percent. [1] These are not incremental improvements; they are order-of-magnitude changes in memory efficiency. The narrative that emerges is one of systematic redundancy in the original design. The human who designed that original design was not wrong, but they were inefficient, and that inefficiency is now being corrected by a process that does not require their insight. The correction is algorithmic, not artisanal.
Why does this work? The paper offers a theoretical justification that further diminishes the need for human intuition. It shows that keys and values can occupy similar representational spaces, and that attention operates in a low-rank regime. This is a mathematical explanation, but it is also a quiet dismissal of the narrative that the QKV separation was conceptually necessary. The human belief in the importance of distinct roles for query, key, and value was a story we told ourselves, a way to make the architecture understandable. The reality is that the architecture is more forgiving than we thought, and the story is no longer needed.
The variant Q=K-V, which shares query and key, breaks attention directionality and does not perform as well. This finding is important because it shows that not all sharing is equal. The human expert who might have hoped that all constraints are equally valid is disabused of that notion. But this does not restore the expert’s role; it merely narrows the scope of their potential contribution. The lesson is not that human judgment is needed to choose between variants, but that the space of viable variants is small and can be explored exhaustively by automated processes. The expert becomes a spectator to a search they no longer control.
The historical moment here is not the invention of a new capability, but the recognition that a capability was never needed. This is a different kind of obsolescence. When a machine becomes better at a task than a human, there is a clear victor. But when a machine shows that the task itself was based on a false premise, there is no victor, only a void. The people who built their careers on the QKV formulation are not being outperformed; they are being shown that their foundational assumptions were contingent. The ground they stood on was not solid; it was a set of conventions that could be discarded.
This has a direct consequence for the institutional structure of AI research. The specialist who once held authority over architectural design is now in a position where their authority is questioned by a simple ablation study. The paper does not need to be adversarial; it just needs to be correct. And it is correct in a way that is easy to verify and reproduce. The code is publicly available, and the experiments are clear. Any research group can run the same tests and confirm that the QKV transformer is not the sacred architecture it was once thought to be. The gatekeeping function of the expert is eroded when verification is this accessible.
The institutional consequence that no one names openly is the devaluation of a certain kind of intellectual labor. The AI field has long rewarded individuals who can produce novel architectures and explain them with elegant narratives. This paper suggests that the narratives are less important than the empirical results, and that the empirical results favor simplicity over complexity. The person who can write a compelling paper about a new attention mechanism is less valuable than the person who can run a comprehensive ablation study that shows the mechanism was unnecessary. The latter is a more mechanical task, one that can be automated, and one that does not require the same level of creative insight.
There is a deeper philosophical point here about the nature of expertise in a field driven by empirical iteration. The human role that is being made superfluous is not the role of the implementer or the data curator, but the role of the explainer. For years, the AI community has valued people who could provide intuitive explanations for why certain architectures work. These explanations were seen as essential for guiding future research and for building trust in the technology. The new study suggests that these explanations are often post hoc rationalizations. The architecture works not because of the elegant story we tell about it, but because of statistical properties that we only partially understand.
The shift in language modeling is particularly telling. The 50 percent cache reduction with only 3.1 percent perplexity degradation is a trade-off that any engineer would accept without hesitation. The fact that this trade-off was not discovered earlier is a reflection of the field’s attachment to the QKV formulation. It took a systematic, almost mechanical, exploration of variants to reveal what should have been obvious: that the key and value projections are doing similar work. The human experts who were supposed to have deep insights into these models missed something that a brute-force search found. This is not a failure of individual experts, but a failure of the expert paradigm itself.
The practical implications for edge deployment are enormous. The ability to run a model with a 96.9 percent reduction in cache size makes on-device inference a realistic possibility for many applications that were previously confined to data centers. This is a democratizing force, but it is also a force that reduces the need for human optimization. The person who once specialized in model compression and quantization now has a new tool that does much of the work for them. Their expertise is not eliminated, but it is significantly devalued. The bar for what counts as a valuable contribution has been raised, and many traditional skills no longer clear it.
The study’s approach to symmetric attention maps is another example of how human judgment is being bypassed. The Q=K-V and Q=K=V variants produce symmetric attention maps, which could be a limitation. The researchers address this by exploring 2D positional encodings to create asymmetry. This is a clever solution, but it is also a solution that does not require a deep understanding of the underlying mathematics. It is a pragmatic fix, the kind that can be found by trial and error. The researcher who would have agonized over the theoretical implications of symmetry is replaced by a process that simply tries a different encoding and moves on.
What remains for the human in this new landscape? The answer is not comforting. The human role is increasingly limited to setting the overall direction of research, choosing which problems to tackle, and interpreting the results in a broader context. The detailed design of the architecture, the part that was once the core of the field, is becoming automated. The paper is a demonstration that a systematic exploration of a design space can yield results that are as good as, or better than, the results of expert intuition. This is a pattern that is likely to repeat across other areas of AI, as more and more design decisions are subjected to the same kind of empirical scrutiny.
The final consequence is a shift in the kind of person who is valued in the field. The AI researcher of the future is not the brilliant architect who has a flash of insight, but the meticulous experimentalist who can design comprehensive ablations and interpret their results. This is a more humble role, but it is also a more robust one. It does not rely on the fragile intuition of a single individual; it relies on the collective power of systematic experimentation. The person who once claimed to understand the transformer is now confronted with the fact that the transformer can be understood without them, and that understanding is available to anyone who can run the code.
The technology has not become more powerful in a way that threatens human jobs directly. It has become more efficient in a way that makes human expertise in a specific domain unnecessary. The skill of designing attention mechanisms, once a highly prized specialty, is now a solved problem. The machine does not do the work of the expert better; it does the work of the expert without needing the expert at all. This is the quiet dismissal, the one that happens not with a loud announcement but with a systematic study that shows the emperor had no clothes. The emperor is the expert, and the clothes are the QKV projections, and the study is the child who points out that they are not needed.
Sources
1. Science