We tend to think of language, perception and thought as representing the world, pointing towards and mapping reality. But AI’s large language models suggest that this isn’t how language works, argues cognitive scientist Elan Barenholtz. These models prove that language and imagery can produce coherent continuations without ever grounding themselves in external reality. This “autogenerative” capacity was always present in language, awaiting discovery. And, Barenholtz argues, it is now the best explanation we have of how language and perception work in humans. Meaning arises not by establishing facts about the world, but rather through language’s generative role in producing further language, imagery, and, ultimately, coordinated human action.

Imagine that archaeologists unearth clay tablets from an ancient civilization, long lost to the world. There are no bilingual texts, no known descendants of the civilization, nothing to anchor a translation of the tablets. They seem to display nothing more than rows of arbitrary squiggles. Now imagine that someone claims to have decoded the squiggles. “These patterns,” they assure us, “are self-predicting. The sequence of symbols in one part of the tablet is mathematically sufficient to derive what will appear in another part.” And, indeed, they produce an algorithm which correctly predicts the text on the right side of each tablet based on the text on the left side.

The finding that the symbols contain this predictive structure would be an extraordinary insight. But we still might ask, “what do the symbols mean?” Now replace the tablets with the digital corpus we call the internet, and the algorithm with a large language model. The civilization is ours. And the question of meaning is ours too.

The technical and social impact of LLMs have been much discussed, but they also represent a major scientific and philosophical breakthrough. In trying to replicate human speech, engineers uncovered something fundamental about the nature of language. What these systems suggest is that natural language, which we use to communicate and think and write, is self-predicting, just like the symbols on the tablets. At the mechanical level, the process is simple. An LLM takes in linguistic subcomponents called tokens and generates the next one. That token is appended to the sequence and fed back in to produce the next, and so on. And out comes language.

To the model, words are arbitrary marks, no different from the patterns on those ancient tablets. The model has no exposure to the kinds of information we typically associate with meaning. It knows nothing about the redness of “red” or the spatial extension of “far.” It is blind, deaf, and unembodied. Instead, it knows where “red” tends to fall in relation to every other word: its proximity to “orange,” “blood,” “firetruck,” “angry,” “stop.” It encodes these relations by giving each word an address in a high-dimensional space, called an embedding. Each point is defined only by its position relative to all the others. There is no content at the address. No meaning stored inside. Pure relations. And yet, from this manipulation of empty symbols, an LLM can learn to talk.

___

We begin sentences without knowing how they will end. We commit to grammatical paths and backtrack when they fail. We are sometimes carried somewhere unexpected by the logic of what we were saying.

___

This is astonishing. We tend to think of language as referential, pointing at a world we come to know through sensory experience and behavior. When someone tells us something like “there is a chair in the living room,” the words feel like they inform us about something—establishing facts about the world itself, or some inner model of it. From that something, everything else follows. If someone subsequently asks you a question: “Where can I sit?” or “Is there any furniture in the living room?” you consult this inner model to answer, not the words themselves. Language in → model updated or consulted → language out. And that is where real thinking and reasoning are assumed to live, generating further facts from those previously established that can then be expressed linguistically. This intermediate process is what is sometimes called “grounding”: words refer to something outside themselves. The intuition is so natural that we barely notice we are having it. But language models cut out this middleman. They do not, and cannot, maintain anything like a conventional model of the world, some stable inner scene to which words are mapped. They do not establish facts about anything outside themselves. They are patterns of squiggles that predict more patterns of squiggles. To quote my colleague William Hahn: it’s the thoughts that do the thinking.

I call this the autogenerative property of language: the internal structure of language alone is sufficient to generate any continuation of itself. Critically, the models did not invent this property. They found it. What LLMs are, fundamentally, is an expression of a remarkable and surprising capacity already present in language. And now that we know this property is in language, it is difficult to avoid the conclusion that this is also how language operates in us. It’s not that the brain is literally an LLM. Rather, the proposal is that it too may exploit the structure we now know is already present in language, generating words based on the predictive structure of previous words. To suggest otherwise is to propose that this intricate autogenerative structure, capable on its own of generating language, is merely an accidental byproduct of some other generative process. Occam would blush. The more parsimonious conclusion is that the brain also takes advantage of the autogenerative structure of language, generating utterances through prediction, one token at a time. And our own behavior is actually very consistent with this. We begin sentences without knowing how they will end. We commit to grammatical paths and backtrack when they fail. We are sometimes carried somewhere unexpected by the logic of what we were saying. It seems like, in us too, it may be the thoughts that do the thinking.

If human language operates autogeneratively, this dissolves the old referential notion of linguistic meaning, along with many of the old conundrums about how language works. Philosophers have long sensed the trouble. Wittgenstein noted that no definition of “chair” captures all chairs and excludes all non-chairs. Quine showed that the reference of any term is radically underdetermined by behavior. And the inverted qualia problem shows that two people could have systematically different inner experiences while agreeing perfectly on every linguistic judgment. LLMs do not solve these grounding problems. They suggest that there was never any such grounding in the first place. The relational structure alone is sufficient.

SUGGESTED VIEWING ChatGPT doesn't understand anything with Michael Wooldridge With Michael Wooldridge

But of course, the old referentialist picture is not so easy to dismiss. Unlike LLMs, language in people does more than generate more language. If someone tells you “there is a chair in the living room,” you not only can answer questions based on that statement; you can imagine the chair sitting in the room. You can go to the room and sit in the chair. Doesn’t this imply that while LLM language may lack grounding, human language has it? Two broad current schools of thought about LLMs yield two different answers to this question: What we might call the “minimalists,” such as Emily Bender, Noam Chomsky and Gary Marcus, argue that LLM language is a kind of superficial imitation of the human variety, lacking the kind of true meaning that human language is endowed with. Conversely, “maximalists” like Max Tegmark and Ilya Sustkever argue that LLMs must be doing more than encoding word relations through emergent reasoning capabilities.

I believe that both these extremes are incorrect. Instead, human and LLM language may both just be autoregressive next token prediction. What the minimalists and maximalists are missing is that the generative process can extend beyond language to perception, imagery and, ultimately, behavior. The other major development in today’s AI—multimodal models—points the way. These models can take a text description and generate a photorealistic image, or take an image and produce a detailed verbal account. This might seem to revive the referential picture. But multimodal training does not add a fixed mapping between words and pixels. Instead, these models learn to project images into linguistic space, functioning more like prompts that push the language-generation process in certain directions. Just as there is no direct word-to-sentence mapping in the LLMs, there is no direct word-to-pixel mapping in image generators. The process is exquisitely contextual and extremely flexible, just like language generation.

To illustrate this, let’s return to our example from earlier. “There is a chair in the living room” does not have a fixed continuation in an LLM, nor in us. If the prompt is “I need to clear out all the furniture; there is a chair in the living room,” this will lead to a very different follow-up than “I am tired; there is a chair in the living room.” There is no unique continuation. It depends. The same holds when we use the phrase as a prompt to an image generator. “A miniature dollhouse; there is a chair in the living room” produces a very different image than “An opulent villa; there is a chair in the living room.” A phrase does not map to any specific image any more than it maps to any specific linguistic continuation. It depends. Inversely, a multimodal model can look at wildly different images and answer the question: “is there a chair in the living room?” The magic of language is that we can compose arbitrary conditions, and all agree whether they are met or not. Not just “there is a chair in the living room” but “there is a red chair in Steve’s Miami penthouse living room with a pink cushion and a broken leg.” The multimodal models demonstrate that such agreement is possible purely generatively, without any hard and fast mapping between words and pixels. If generation is enough to allow us to agree with the models, it is sufficient to allow us to agree with each other.

___

Language doesn’t mean; it does.

___

This kind of agreement is what ultimately makes language functional as a coordinative tool between people. It allows us to specify to others what conditions of the world hold—or should hold—and behavior follows. This does not mean that language directly generates muscle movements the way it generates more language or imagery. In general, we don’t generate movement for its own sake. We move to achieve some outcome, either in the world or in our relation to it. We walk to enter a room; we reach to pick up a cup. Motor behavior unfolds in continuous coupling with perception, each movement shaped in real time by what perception reports back: room entered, cup held. The role of language in this loop is to provide the condition (chair needed in living room) that behavior can satisfy based on perceptual confirmation (chair delivered to living room).

This is not so different from other animals that behave to satisfy a condition. A bird builds a nest until it’s complete. A predator hunts its prey until it’s caught. But these conditions are highly rigid, either innate or learned on a case-by-case basis. Human language breaks this kind of process wide open, allowing us to generate arbitrary conditions on the fly and agree on whether they have been met. And because language is autogenerative, conditions beget conditions: “There is a chair in the living room; Stacy is vacuuming in there; she likes the room empty when she vacuums; therefore, I should move it to the kitchen.” Pure language production that leads to a new condition for behavior to satisfy and perception to confirm. And this same compositional power scales to ever more expansive “conditions”—degree obtained, family established, treaty signed—all built out of the same autogenerative chaining that gets the chair into the kitchen.

This, then, is a new way of conceiving what it means for language to be “about” something: not by establishing transcendent facts about the world but by playing its generative role. The meaning of “There is a chair in the living room” simply is its potential to generate contextually appropriate language, imagery and perceptual confirmation. That’s all. But that’s all that’s needed to coordinate people across time, place and circumstance. Language, in this view, is not a communication system so much as an autogenerative operating system: a mechanism to produce language, imagery and ultimately action that is responsive to the complexities of the world and society. The marks on the ancient tablets do not mean anything in isolation. They only come to life based on their shifting role in a larger ecosystem of people using them to make things happen in other people. Language doesn’t mean; it does.

This new kind of dynamic grounding might seem alarming. It deconstructs not just the conventional sense of meaning in language but also the notion of a unified self that perceives, thinks, speaks, and acts. Instead, we are left with parallel interacting processes that don’t really understand one another: a blind and deaf linguistic system, a mute perceptual system, a motor system carrying out simple operations, each with no real understanding of the others but somehow coordinating to produce coherent, complex behavior. Which one of these processes is you? The one that is currently visually scanning these words, the one hearing an inner voice reading them, or maybe the one who decided to do all of that in the first place? The answer, of course, is that “you” are not any one of those but the emergent whole. A whole whose parts coordinate so well that the whole doesn’t see them as parts. But if you find yourself thinking “I know what I am! I’m a single unified conscious being,” just ask yourself: who exactly is saying that?

Artificial Intelligence