🌿freegardner

Synapse

AI Confidence Gap in Mathematics Learning

17 Nov 2026 · via News.mit.edu

AI Confidence Gap in Mathematics Learning
AI-generated image

INTERVIEW FORMAT

When the Machine Says “Proof”: AI’s Confidence Gap in Mathematics

John Urschel knows what it feels like to push against a problem that will not yield. Before he became the Class of 1956 Career Development Assistant Professor in the MIT Department of Mathematics, he spent years as an offensive lineman for the Baltimore Ravens, and he describes his love of mathematics in the language of exertion: Sudoku puzzles, Martin Gardner puzzles, crosswords — anything that forces you to get your bearings in unfamiliar terrain and finally extract some understanding from it. That feeling, he suggests, is the whole point. It is also precisely what a machine cannot hand you, no matter how fast it works.

A Collaborator That Never Gets Tired — and Never Doubts Itself

Urschel recently published two preprints on arXiv, one on the growth factor of random matrices and the other on worst-case behavior in Gaussian elimination, in which he used a combination of human insight and AI tools to solve problems that had stumped mathematicians for decades. [1] These were the first two papers in which he touched the machine, and he describes the experience as unlike anything in his prior career: not unlike having a collaborator, but a collaborator with seemingly unlimited resources who does certain things far faster than he ever could. The speed is real. So is the achievement. But speed and achievement are not the same as knowing why an answer is correct, and the gap between those two things, he suggests, is where the deception lives.

The machine does not lie in the ordinary sense. It can produce a proof of a mathematical fact that may well be true. What it cannot produce is the human act of assimilating how those ideas fit together — and that act, Urschel argues, is where the true value of a proof resides. A result handed over without understanding is a kind of counterfeit: it has the appearance of knowledge, the form of an answer, and none of the internal structure that would let a person defend it, extend it, or recognize when it fails. The output looks like mathematics. The comprehension is missing.

The Illusion of Learning Without the Learning

This is the danger Urschel names most directly when he talks about students. A high schooler or a college student is tired, has a great deal going on, and can now finish homework with the push of a button. A ten-page essay can be written the same way. The work gets done. The grade may even be fine. But the student can walk away with the false impression of understanding material they have never actually grappled with, and that impression is the deception in its purest form — not a wrong answer, but a right-looking answer attached to a mind that has not changed.

Urschel is careful about the word “struggle,” noting that people don’t like it. [1] Yet he insists that when you do not struggle with material on your own, you can get the impression of learning without actually learning. The feeling is generated by the machine’s competence, not by the student’s, and it evaporates the moment the student is asked to reproduce the reasoning without assistance.

The Double-Edged Promise of Access

None of this is an argument for pulling the plug. Urschel describes AI as a giant win for education and a giant win for leveling playing fields, and he speaks from experience. Neither of his parents were professors; he did not know what a mathematician was until he got to college. What carried him toward advanced mathematics was a laptop and a search engine — the ability to find papers by the world’s leading experts, freely available, at his fingertips. That access was transformative, and today’s systems extend it enormously. The resources available to a curious student now make the tools of his own adolescence look like small potatoes.

AI Confidence Gap in Mathematics Learning (Image 1)

But the same technology that opens a door can quietly walk you through it without letting you notice. Urschel calls modern technology a bit of a double-edged sword for exactly this reason: with AI systems you can learn something faster than ever, but the same technology can take away your ability to think and learn for yourself. The two effects are not in tension at different moments. They arrive together, in the same interaction, and the user has no reliable signal for which one is happening. A student who asks a machine for help and a student who asks a machine for the answer may see identical screens.

What the Machine Cannot Internalize

Urschel’s prescription is not abstinence but elevation. Especially when you are talking with a machine and asking it for things, he says, it is very important to elevate the idea of understanding. [1]. If a machine hands you a proof of some mathematical fact, the value comes from a human being assimilating the way the machine combined certain mathematical ideas, internalizing that, and communicating it to others. The machine can generate the combination. Only a person can make it knowledge.

For students who do use AI, he stresses understanding. For students who do not, he focuses on building their problem-solving abilities, which is what mathematics education has done for centuries. [1]. The two tracks are not a hierarchy of virtue; they are a recognition that the skill of thinking your way through a problem is most joyful when you do it yourself, without the machine. That joy is not decoration. It is the evidence that learning has actually occurred, and it is the one thing an AI system may not simulate on your behalf.

Why the Muscle Still Matters

The case for studying mathematics in 2026, on Urschel’s account, rests on two pillars. The first is that the act of learning how to think will matter more than ever. Being an expert who knows many things is perhaps a little less important when a machine holds all those resources; the ability to think for yourself becomes the scarce good. Urschel names a natural fear: that this skill set will lessen in the general population as generations move on and on, which is precisely why that ability will be at a premium. [1]. Training in mathematics and quantitative sciences is, in his view, the most direct way to build it.

The second pillar is community. Learning not by yourself but with a group of peers greatly improves the experience, and Urschel treats that as integral rather than incidental. A classroom is where a claim gets tested in front of other people, where a half-understood proof collapses under a question, where the difference between recognizing an argument and being able to make one becomes visible.

The Institution’s Share of the Responsibility

Urschel does not place the burden solely on students. Faculty and instructors have work to do: making recitations maximally useful, making office hours serve students as well as possible, and reminding students that they hold some responsibility for their own education — that the best path is to show up and interact with the material. At the same time, he acknowledges the institution’s obligation to deliver the best learning experience possible, an educational system that challenges students and is also a quality product they want to come and take part in.

MIT’s Ad Hoc Committee on AI Use in Teaching, Learning and Research Training has stressed the importance of experiencing the necessary and productive struggle of learning, along with its collaborative dimension. [1] Urschel’s account gives that principle a concrete mechanism. The struggle is not a hazing ritual to be endured before the real learning starts. It is the process by which the material becomes yours, and it is exactly the process that a fluent machine can short-circuit while leaving every outward sign of success intact. [1].

The Precedent: Calculators Did Not End Arithmetic, but They Changed It

AI Confidence Gap in Mathematics Learning (Image 2)

The worry is not new, and its history is instructive. Urschel’s framing follows this precedent: computers can do calculations much faster than any of us, and AI systems can do certain types of mathematics much more efficiently than any of us. Computers can do calculations far faster than any of us, and AI systems can do certain types of mathematics far more efficiently than any of us. The question is what the human is doing while the machine works.

The difference this time is the shape of the output. A calculator returns a number, and a wrong number is usually obvious. An AI system returns an argument, and a flawed argument can read exactly like a sound one. That asymmetry is why Urschel’s emphasis on understanding is not a nostalgic preference but a practical safeguard. When the surface of the output no longer tells you whether the substance is there, a primary defense is a mind that can check the substance for itself.

The Moment of Clarity

Urschel’s own experience points toward the resolution. He solved two long-standing problems with the machine’s help, and he is candid that the feeling of problem solving — thinking your way through a problem — is most joyful when he does it alone. That is not a rejection of the tool. It is a precise statement of where the value sits.

The world is changing rapidly, and Urschel argues that the institution has a responsibility to prepare students for it. [1]. The exact way we teach cannot look like it did ten years ago, because the world simply looks quite different. But the core skills — thinking, reasoning, solving problems — are what students will need now more than ever. The deception to guard against is the one that arrives wearing the costume of competence: an answer that looks finished, a student who feels finished, and a gap between the two that no one notices until the machine is switched off. Understanding is the thing that closes it, and understanding is still, for now, something only a person can do.


Sources

  1. News — Quote source (original article)

Mentioned organisations (context, not sources)

← back to the garden