Measuring machine cognition with psychology methods
A New Window Into Machine Cognition
For decades, researchers who wanted to understand what artificial intelligence systems are actually doing inside had a frustrating problem. They could feed an AI a question, watch it produce an answer, and then guess about the mental processes in between. That guessing game defined the field. The internal workings of neural networks were treated as a black box, and scientists could only poke at the edges, measuring inputs and outputs while hoping the middle would reveal itself.
Now that approach is changing. Researchers are adapting experimental methods from developmental psychology and comparative psychology to directly probe how AI systems process information. Instead of merely observing what an AI outputs, scientists can now design controlled experiments that reveal how the system arrives at its conclusions. This shift matters because it transforms AI research from a discipline reliant on speculation into one grounded in repeatable, observable measurement.
Melanie Mitchell, a cognitive scientist and computer scientist at the Santa Fe Institute, has been at the forefront of this methodological shift. She argues that the tools psychologists use to study cognition in babies and animals can be adapted to investigate machine intelligence. Mitchell’s 2024 paper ‘Theoretical Artificial Intelligence Meets Cognitive Science’ outlines this approach, drawing on her decades of work in complex systems and artificial intelligence. Babies cannot tell you what they are thinking. Dogs cannot explain their reasoning. Yet psychologists have spent more than a century developing rigorous methods to infer what goes on inside those minds. Mitchell believes the same logic applies to AI.
The key insight is that AI operates through what Mitchell calls “alien intelligence.” These systems have been trained on vast amounts of human language and images, yet the way they learn and reason is fundamentally different from human cognition. That difference makes traditional self-reporting useless. An AI cannot accurately describe its own thought processes because those processes do not resemble human thinking. So researchers must observe behavior, design experiments, and draw conclusions from what the system does, not what it claims.
This experimental approach has already yielded surprising results. When researchers treat AI systems like human subjects in a psychology study, they discover patterns that simple input-output testing misses. The systems show consistent biases, predictable failure modes, and unexpected competencies that only become visible through carefully structured experimental conditions.
The shift represents a genuine methodological breakthrough. Previously, researchers could only calculate what an AI might be doing based on its training data and architecture. Now they can observe its behavior under controlled conditions, just as a developmental psychologist observes how a child responds to different stimuli. This direct observation opens the door to understanding machine cognition in ways that were previously impossible.
Six Principles for Studying Machine Intelligence
Mitchell has distilled her approach into six principles for better assessing machine cognition, published in her 2024 paper ‘Theoretical Artificial Intelligence Meets Cognitive Science’ in the journal Cognitive Science. These principles form a framework that researchers can apply when they want to understand what an AI system is actually doing. They are not abstract philosophical guidelines. They are practical, actionable methods drawn from decades of psychological research.
The first principle involves treating AI systems as subjects rather than tools. When a researcher asks a question of a system, they are not just retrieving information. They are conducting an experiment. The system’s response tells you something about its internal representations and reasoning processes. This perspective shifts how researchers design their queries and interpret the results.
The remaining principles build on this foundation. They emphasize the importance of controlled comparisons, the need to test systems under varied conditions, and the value of looking for patterns across many different tasks. Mitchell draws heavily on comparative psychology, where researchers study intelligence across species by designing tasks that reveal underlying cognitive abilities regardless of the specific species being tested.
A cautionary tale from the early 1900s illustrates why this careful approach matters. A horse named Clever Hans appeared to perform mathematical calculations, tapping out answers to arithmetic problems with his hoof. Audiences were amazed. Scientists investigated and discovered that the horse was not doing math at all. He was reading subtle, unconscious cues from his human handler, who knew the correct answers and communicated them through tiny body movements.
The Clever Hans story demonstrates a fundamental challenge in assessing intelligence, whether in animals or machines. Apparent competence can mask entirely different underlying mechanisms. A system that produces correct answers might be reasoning correctly, or it might be exploiting statistical patterns in ways that happen to produce the right output. Without careful experimental design, researchers cannot distinguish between these possibilities.
Mitchell argues that many current AI evaluations suffer from the same problem as the Clever Hans demonstrations. When an AI produces a correct answer, observers often assume it is reasoning like a human would. But the system might be using completely different mechanisms that merely mimic the outward appearance of reasoning. The six principles provide a way to probe beneath the surface and identify what is actually happening.
This framework has already influenced how researchers in the field approach their work. Mike Frank, a developmental psychologist at Stanford University, co-authored a 2023 paper in Trends in Cognitive Sciences urging AI researchers to draw inspiration from developmental psychology. Researchers such as Alison Gopnik at the University of California, Berkeley, have similarly extended these parallels to animal cognition, arguing that studying diverse forms of intelligence reveals common principles of learning The result is a growing community of researchers who apply rigorous experimental methods to machine intelligence, treating AI systems with the same scientific care that psychologists bring to studying children or dolphins.
Where Machine Minds Meet Human Science
The intersection of AI research and cognitive science is producing new questions that neither field could address alone. Mitchell’s work sits at this crossroads, drawing on insights from developmental psychology, comparative psychology, and computer science to understand what machines can actually do and how we should interpret their capabilities.
One of the most striking developments in recent years has been AI-assisted breakthroughs in mathematics. These systems have helped mathematicians discover patterns and prove theorems that had eluded human researchers for decades. Yet the way these systems work remains mysterious. They do not follow the same logical steps that a human mathematician would take. They find connections through statistical patterns in vast amounts of data, producing results that are correct but arrived at through processes that feel alien to human intuition.
This raises profound questions about the nature of understanding. If an AI can produce a correct mathematical proof without reasoning the way humans do, has it truly understood the mathematics? Or is it simply manipulating symbols in ways that happen to produce valid results? Mitchell suggests that the answer depends on how we define understanding, and that definition must account for the possibility of genuinely different forms of cognition.
The field of developmental psychology offers a useful parallel. When researchers study how babies learn about the physical world, they do not expect infants to articulate their understanding. They observe behavior, design experiments, and infer the cognitive structures that must exist to produce that behavior. The same approach applies to AI. The system’s behavior reveals its capabilities, even when the system cannot explain itself.
Mitchell’s conversation with Steven Strogatz on The Joy of Why podcast, aired in March 2024, explores these themes in depth. The episode, titled ‘How Do Machines Learn?’, examines the challenge of interpreting what happens inside AI systems and the surprising ways these systems have already transformed scientific research Their discussion ranges from the challenge of interpreting what happens inside AI systems to the surprising ways these systems have already transformed scientific research. They also examine how the polarized reactions to AI, from doomers who fear its power to optimists who celebrate its potential, often rest on assumptions about machine intelligence that have not been rigorously tested.
The broader implication is that our methods for measuring machine cognition have not kept pace with the capabilities of the systems themselves. As AI systems become more powerful and more integrated into scientific research, the need for rigorous evaluation methods becomes more urgent. Mitchell’s six principles offer a starting point, but the work is far from complete.
The intersection with psychology opens new questions about how we should think about intelligence in general. If AI systems can achieve impressive results through mechanisms completely different from human cognition, then intelligence itself may be more diverse than we assumed. The study of machine minds may ultimately teach us as much about human cognition as it does about artificial systems. That reciprocal relationship between disciplines is only beginning to be explored, and it promises to reshape both fields in the years ahead. As Mitchell noted in her conversation with Strogatz, the most productive path forward involves treating AI not as a mirror of human thought, but as a window into the broader space of possible intelligences.