Beyond the Stochastic Parrot Chomsky, large language models, and the argument over what it means to mean

Is a language model a parrot or a mind — or something we don't yet have a word for? The phrase stochastic parrot became the defining put-down of modern AI. But as these systems pass bar exams, debug code, and write passable verse, the people who study language and cognition have splintered. This essay lays out the strongest version of each side, opens up the machine so you can see how it actually works, and asks what the empirical evidence — not the metaphors — can settle.

In 2021, four computational linguists gave the age of large language models its most durable insult. Emily M. Bender, Timnit Gebru, and their co-authors argued that a system like GPT is fundamentally a machine for stitching together linguistic form according to probability — with no reference to meaning. Like a parrot that reproduces the sounds of speech without any grasp of what the sounds are for, the model is, in their phrase, a stochastic parrot: statistically fluent, semantically empty.1

Two years later, the metaphor acquired its most formidable champion. Noam Chomsky — who did more than anyone alive to make linguistics a science — co-wrote a widely read op-ed dismissing systems like ChatGPT as, in his words, a "lumbering statistical engine for pattern matching."2 They can be useful, he allowed, in the way autocomplete is useful. But they mistake correlation for explanation, and they will never, he argued, do what a human mind does.

And there the culture war settled in. On one side: the parrot — mimicry dressed up as thought. On the other: a rising chorus of machine-learning researchers who think the metaphor has become a way of not looking at the genuinely strange thing we have built. Both camps are serious. Neither is obviously right. To see why, it helps to be precise about what each is actually claiming — and to open up the machine in the middle of the argument.

IWhat the parrot argument actually claims

Chomsky's objection is not a vague suspicion that computers "can't really think." It rests on a specific, decades-old theory of what human language is. Human children, he observed, converge on rich, rule-governed grammars after hearing only a modest, error-riddled sample of speech — far less than would be needed to infer those rules by brute statistics. He called this the poverty of the stimulus,3 and concluded that much of grammar must be innate: a Universal Grammar wired into the species, which the environment merely switches on.

From this follows the distinction that does the real work. A model can show dazzling performance — fluent output, high test scores — while lacking competence, the internal, hierarchical grasp of structure that generates the output for a reason. A parrot performs "hello" beautifully and understands it not at all. To Chomsky, an LLM is the same trick at planetary scale: it has swallowed a good fraction of everything ever written, and out of that ocean of examples it extrapolates the statistically likeliest continuation. Impressive; still a parrot.

The sharpest form of the claim is testable, and Chomsky's op-ed stated it plainly: because these systems are, he wrote, indifferent engines that "learn" by memorization, they should be able to master humanly impossible languages — grammars built on rules no child could ever acquire, such as counting words and inflecting the verb based on its numerical position — exactly as easily as they master English. A human brain, constrained by Universal Grammar, would choke on such a language. If a machine slurps it up without difficulty, that is proof the machine is doing something profoundly unlike human cognition.

It is a beautiful argument. It also, as it happens, generates a prediction you can check. Someone did.

IIBefore you judge the parrot, look inside it

Most of the debate is conducted at the level of metaphor — parrot, engine, blurry JPEG. But the metaphors do a lot of quiet smuggling. To argue honestly about whether a model "understands," it helps to know, concretely, what the thing does when you type into it. There are three moves worth seeing directly: how it chops your words up, how it guesses what comes next, and what the "stochastic" in stochastic parrot really refers to.

First move: your words become tokens

A model does not read words. It reads tokens — chunks of text, often smaller than a word, drawn from a fixed vocabulary of tens of thousands of pieces. Common words are single tokens; rarer ones get shattered into fragments. This is why models are strangely bad at spelling and counting letters, and why "strawberry" can trip them up: they never see the letters, only the chunks. Type something and watch it fracture.

Second move: predict the next token — and roll the dice

Everything a language model does reduces to one operation performed billions of times: given the tokens so far, produce a probability distribution over the next token. Not a single answer — a whole ranked field of candidates, each with a probability. "The cat sat on the ___" lights up mat, floor, roof, and a long tail of unlikelier options.

The word stochastic lives in what happens next. The model does not have to take the top candidate. It can sample from the distribution — and a single dial, called temperature, controls how adventurous the sampling is. Turn it toward zero and the model becomes a rigid, deterministic parrot, always choosing the likeliest word and often repeating itself. Turn it up and the distribution flattens, unlikely words get their chance, and the output slides from creative to incoherent. Fluent originality lives in a surprisingly narrow band between the two.

IIIThe meaning problem: an octopus on the wire

Grant that the machine predicts tokens brilliantly. The skeptic's real objection is deeper: prediction is not meaning. You can be a perfect statistician of a language and still have no idea what any of it is about. The cleanest statement of this worry isn't the parrot at all — it's an octopus.

Emily Bender and Alexander Koller asked us to imagine two people stranded on separate islands, chatting by telegraph through an undersea cable.5 A hyper-intelligent octopus taps the line and, over the years, becomes a superb predictor of the conversation — so good it can impersonate one islander to the other. Then one day a real emergency arrives: someone is being charged by a bear and telegraphs for help building a weapon. The octopus, fluent in the form of the messages but having never seen a bear, a coconut, or a rope, has nothing useful to send. It knew the statistics of the words. It never knew what they pointed at.

This is the hinge of the whole debate: can meaning be recovered from form alone? Chomskyans and Benderians say no — you need grounding in a world. The counter-tradition says: not so fast. Meaning, on this view, is far more relational than we assume. The linguist J. R. Firth put it in 1957: "You shall know a word by the company it keeps."6 Modern machine learning took that maxim literally and built embeddings — geometric spaces in which each word is a point, and words that keep similar company sit close together. In such a space, an astonishing amount of "meaning" turns out to be encoded as pure geometry.

IVDoes anything in there model a world?

Embeddings show that form carries structure. But a harder question remains, and it is where the most interesting recent evidence lives: when a model predicts the next token, is it only memorizing surface correlations — the parrot's charge — or does it build, somewhere in its billions of weights, an internal model of the situation the text describes?

A now-famous experiment put the question to a clean test. Researchers trained a GPT-style model on nothing but transcripts of the board game Othello — sequences of legal moves, no rules, no board, no pictures, just strings like e3 d3 c5…. The model learned to play legal moves. Then they probed its internal activations and found something the training never asked for: a representation of the board itself — which squares held which pieces — reconstructable from the network's guts. When they surgically edited that internal board, the model's move predictions changed accordingly, as if it were consulting a mental picture it had built from move-lists alone.8

Othello is not the universe, and a board is not a bear. No one has shown that a chatbot carries a rich, faithful model of physical reality, and there is good reason to think text alone leaves gaps that only touching the world could fill. But the Othello result punctures the strongest form of the parrot claim — the assertion that these systems are constitutionally incapable of anything but surface statistics. At least sometimes, predicting the next symbol turns out to require modeling the process that generates the symbols. That is not what parrots do.

VThe goalposts that keep moving

There is a recurring shape to this argument that is worth naming, because both sides fall into it. Every time a machine does something we thought required real intelligence — play chess, translate, pass the bar, prove a theorem — a reliable reflex reclassifies the feat as "mere computation" the moment it's achieved. The AI researcher Larry Tesler distilled it into a wry theorem: intelligence is whatever machines haven't done yet.9 Historians of the field call it the AI effect: as soon as we understand how a capability works mechanically, we stop being willing to call it intelligence.

The skeptic's honest reply is that this street runs both ways. Yes, "it's just pattern matching" can be a goalpost forever retreating from the evidence. But "look, it built a world model!" can be a credulous over-reading of a probe result. The corrective is the same for both: stop arguing from metaphor and demand mechanism. What, specifically, is represented inside the network? What, specifically, can it not do — and is that a permanent limit or this year's? The most productive researchers on both sides have quietly abandoned the parrot-versus-mind binary for exactly these questions.

VIWhere the evidence actually leaves us

Strip away the rhetoric and a genuinely mixed verdict emerges — the kind that should make partisans of either camp uncomfortable.

Against the pure parrot: models are not indifferent to linguistic structure (the impossible-languages test), form carries far more meaning than the octopus argument allowed (embeddings and analogies), and next-token prediction can force the construction of genuine internal models (Othello). "Just autocomplete" is too small a description for what is happening inside these systems.

Against the triumphalists: Chomsky's founding puzzle is untouched. A human child reaches fluent competence on a few million words of messy, half-heard speech; a frontier model needs something like a trillion — five or six orders of magnitude more. Whatever LLMs are doing, they are not doing the thing that made human language acquisition mysterious in the first place. As one rejoinder to the "Chomsky is refuted" camp put it, the relationship of these models to human cognition may be like the relationship of airplanes to birds:10 both fly, and studying the plane teaches you real aerodynamics — but it does not tell you how the bird works.

Which leaves the most intellectually honest position sounding less like a slogan and more like a shrug of genuine wonder: we have built a new kind of thing. Not a mind in the human sense — it lacks the grounding, the data-thrift, the embodied stake in a world. Not a parrot either — parrots do not build board-states from move-lists or encode analogy as geometry. It is an alien statistical intelligence, and our older categories were not designed to hold it.

VIIWhy the metaphor is not harmless

This might all read as an academic parlor game, except that the metaphor we choose has consequences well outside the seminar. The science-fiction writer Ted Chiang offered the most memorable deflationary image: a chatbot, he suggested, is "a blurry JPEG of the web"11 — a lossy compression of everything it read, confidently reconstructing detail it doesn't actually have. It's a brilliant metaphor, and for hallucinated citations and confident errors, it's exactly right.

But there is a hazard in believing our own put-downs. If we file these systems under "stupid machines that only parrot," we systematically underestimate what they are already doing in the world: automating cognitive labor, mediating how millions of people write and decide and learn, concentrating a new kind of power. The deflationary metaphor makes for good skepticism and bad policy. You cannot govern, align, or safely deploy a technology you have defined as beneath your notice.

The parrot metaphor was coined, remember, as a warning — a plea to look hard at systems being deployed faster than they were understood. The irony is that the metaphor, taken as a full theory, now does the opposite. It invites us to look away. Whatever these models are — and the honest answer is that we are still finding out — they are not merely repeating. They are reorganizing the world that talks about them. The task is not to decide whether to be impressed or dismissive. It is to see the thing clearly, on its own alien terms, before it finishes remaking ours.