The most dangerous thing about a large language model is not that it is sometimes wrong. It is that it is always fluent. Every answer arrives polished, confident, and immediate — the syntactic signature of a competent mind, whether or not a competent mind produced it. Wrongness is a bounded problem; we can measure it, benchmark it, and grind it down with better retrieval and bigger models. Fluency is unbounded, because it is a property of the form of the output rather than its truth, and it does its work on us regardless of whether the content is right.
Here is the claim I want to defend: a civilization that outsources the struggle of thinking to a fluent oracle will accumulate more answers and less knowledge. The two are not the same thing, and the gap between them is exactly where fluency does its damage. An answer is a proposition you can repeat. Knowledge, in the sense that matters, is a justified relationship between you and that proposition — you can defend it, locate its failure modes, reconstruct it when you forget the words. Fluency hands you the first and quietly convinces you that you have the second.
Let me name the costs precisely, each with a mechanism, because "AI makes us dumber" is a lazy thought and I don't want to think it. There are four costs worth separating, and then a fair accounting of what fluency genuinely gives us, and then what to actually do.
The death of "I don't know"
Start with the human side of the loop. When a system produces a confident answer, the person receiving it becomes measurably less vigilant. This is automation bias, and it is not a hypothesis — it is one of the more robust findings in human-factors research, documented first in aviation, where crews accepted erroneous flight-management outputs over their own correct instrument readings, and later throughout clinical decision support. The pattern has two faces: commission errors, where people follow an automated recommendation that is wrong, and omission errors, where they fail to notice a problem the automation didn't flag. Crucially, the effect gets stronger as the system gets better, because a tool that is usually right teaches you to stop checking. I've written about the clinical version of this — the better your diagnostic AI, the less your doctor scrutinizes it — and the epistemic version is the same mechanism aimed at everyone.
The specific casualty is the productive admission of ignorance. "I don't know" is not a failure state; it is the precondition for inquiry. It marks the boundary where your model of the world breaks and learning becomes possible. Socrates built a whole method on it. Bacon, in the Novum Organum, argued that real knowledge begins by clearing away the false confidence of the idols — the mind's habit of imposing more order and certainty than the evidence supports. A fluent oracle is an idol-manufacturing machine. Ask it anything and the boundary of your ignorance vanishes behind a confident paragraph. The felt experience of not-knowing — the itch that drives you to actually find out — gets anesthetized before it can do its job.
Understanding is built in the struggle, not delivered with the answer
The second cost is subtler and, I think, the deepest. Understanding is not a substance that transfers from a source to a recipient. It is a structure you build, and you build it through effortful work — deriving the result yourself, being confused, taking a wrong turn, hitting the wall, backing up. The friction is the mechanism. Remove it and you remove the thing it was producing.
Cognitive science has a name for this: desirable difficulties, the finding (from Robert and Elizabeth Bjork's memory research) that conditions which make learning feel harder and slower in the moment — spacing, interleaving, retrieval practice, generating an answer before being told it — produce more durable and transferable knowledge than conditions that make it feel easy. The feeling of fluency during learning is actively misleading; it correlates with worse retention. Rereading a highlighted passage feels productive and teaches little. Struggling to reconstruct it from memory feels like failure and teaches a lot.
An oracle that hands you the finished answer optimizes for exactly the feeling that predicts not learning. Consider a concrete case. A student trying to understand why gradient descent can get stuck can either (a) ask a model, receive a crisp paragraph about saddle points and local minima, nod, and move on, or (b) write out a two-variable loss surface, compute the gradient by hand, watch it go to zero at a saddle, and feel the confusion of "wait, zero gradient but not a minimum?" resolve into structure. Both students can now say the words "saddle point." Only the second can recognize one in a loss landscape they've never seen, because only the second built the internal model that the words are pointing at. The answer was identical. The understanding was not. The answer is not the understanding — it is the residue understanding leaves behind, and you cannot get the residue without the reaction.
This is why "it explained it to me" is such a treacherous phrase. The explanation was fluent; your comprehension of the explanation may be an illusion produced by the fluency. You have to test it — try to use it, derive a consequence, teach it to someone else — and it is precisely that testing that the frictionless answer tempts you to skip.
Everyone starts from the same paragraph
The third cost operates at the level of the population, not the individual. When millions of people route their questions through a small number of models — trained on heavily overlapping corpora, tuned by similar preference data toward similar notions of a "good, balanced, helpful" response — they tend to converge on the same framings. Ask a hundred people to explain a contested topic after each has consulted the same model, and their answers will rhyme in structure, emphasis, and blind spots.
Why this matters is a point from the philosophy of science. Discovery and error-correction are population-level phenomena that run on variance. Popper's picture of science as conjecture and refutation only works if there is a wide, diverse supply of conjectures to refute; Lakatos's competing research programmes only advance because they are competing, each pushing a different hard core against the anomalies. A field in which everyone starts from the same synthesized consensus has fewer independent guesses in circulation, which means fewer of the idiosyncratic, "wrong-looking" starting points that occasionally turn out to be the productive ones. Homogenization doesn't make any single person dumber. It thins the ecological diversity of thought that makes a community smart — and it does so invisibly, because each individual answer looks perfectly reasonable. The cost is not in the answers; it's in the answers you now never hear because no one was standing far enough off the consensus to generate them.
I want to be careful here: this is a directional forecast, not a measured result. We do not yet have clean evidence of model-driven intellectual convergence at civilizational scale, and I could be wrong about the magnitude. But the mechanism — shared prior in, correlated output out — is not exotic, and it is worth watching.
The atrophy of source-checking
The fourth cost is the most tractable and the easiest to observe in yourself already. When a synthesized answer is free and instant, fewer people pay the price of going to the primary source. Why read the actual paper when the model will summarize it? The summary is right there, it's fluent, and reading the original is slow and often boring.
The trouble is that a summary is a lossy projection of the source, and the losses are not random. What survives summarization is the abstract-able skeleton: the headline claim, the tidy result. What gets dropped is the load-bearing detail — the exact conditions under which the result holds, the hedges the authors put in, the messy figure that half-contradicts the abstract, the footnote where the real limitation lives. If the median person's understanding of a topic converges to the summary's level, then the median understanding is systematically shallower than the literature it claims to represent — and confidently so, because the summary reads as complete.
This is also, precisely, where individual edge is migrating. When everyone has the summary for free, the scarce and valuable skill becomes the thing summaries can't give you: the primary text, read closely, with its caveats and texture intact. I've argued this at length — reading primary sources is becoming a genuine competitive advantage — and the reason is downstream of everything above. The more the population relies on synthesis, the more the person who actually read the source knows something the synthesis-consumers structurally cannot.
The fair accounting
None of this is an argument against the tool, and I don't want to smuggle one in. Fluency is genuinely, enormously valuable, and the value is real.
Access is the first. A fluent explanation available to anyone, in any language, at any hour, is a democratizing force of a kind we've never had — it puts a patient tutor in front of people who never had one. Speed is the second: for the ten thousand things you need to use but not deeply understand — a shell flag, an unfamiliar API, the gist of a field adjacent to yours — friction is pure waste, and removing it is a gift. Scaffolding is the third and most important: for a genuine beginner, a fluent guide through the early switchbacks can keep you climbing long enough to reach the terrain where productive struggle is even possible. Confusion is only productive when it's near your competence; below that it's just despair, and fluency can carry you up to where the good struggle starts.
So the danger is not the tool. It is a specific, seductive default mode of using it: the oracle mode, where you pose a question, accept the fluent answer, feel the click of understanding, and move on — having skipped the vigilance, the struggle, the divergence, and the source. Fluency doesn't force that mode on you. It just makes it the path of least resistance, and paths of least resistance are what most people follow most of the time.
Augment the struggle; don't replace it
The prescription follows directly from the diagnosis. If the costs come from fluency letting you skip the epistemic work, then use fluency to intensify that work instead.
Use the model as a sparring partner, not an oracle. Don't ask it for the answer; state your answer and ask it to attack the weakest point. The value isn't the conclusion it hands you — it's the refutation you have to survive, which is conjecture-and-refutation run at conversational speed.
Use it as a critic. Write your own explanation of something first — badly, gropingly, in your own words — and only then ask the model where you're wrong. This preserves the generation effect (you produced before you were told) while adding a tireless reviewer. The order is everything: struggle, then check.
Use it as a search accelerant that ends at the source, not instead of it. Let it find the three papers and tell you which section matters — and then go read that section yourself. The model routes you to the primary text faster; it does not excuse you from it.
Deliberately, against the grain of the tool, preserve the three practices fluency most tempts you to abandon. Write to think, because writing is where you discover that the understanding you felt was fluency's illusion — the blank page refuses to be fooled. Check the source when the stakes are real, because the caveat that changes everything is always the part that got summarized away. And learn to sit with "I don't know" long enough for it to do its work, because that discomfort is not a bug in your cognition. It is the engine.
The oracle will always answer. The only question that matters is whether you let its answer end your thinking or start it.