Philosophers for AI

Charles Lutwidge Dodgson lectured in mathematics at Christ Church, Oxford, for twenty-six years, and by most accounts he was not very good at it. He was shy, he stammered, his students found him dull. His textbooks on determinants sold badly. He is remembered instead for two books about a girl who falls down a hole, and this is usually presented as one of those charming accidents of literary history — the dry don who had one strange dream and got lucky.

I think that reading is exactly backwards. Dodgson was a logician who understood something that most logicians of his century did not: that the deepest problems in logic are not solved by more notation, they are exposed by a well-built joke. So he smuggled them. He put a paradox about self-reference into a conversation with an egg. He put the problem of naming into a knight’s song that has a name, and a name for its name, and a name that the name is called. And in two places — one of them a fantasy novel almost nobody finishes, the other a three-page dialogue in the journal Mind — he set down the two hardest problems in world modeling, forty years before anyone had a machine that would suffer from them.

I want to take those two problems seriously here, because I think they cut straight through the middle of how we are currently building AI agents, and because between them they explain two of the most persistent failure modes in the field. The first is about what a model must throw away. The second is about what a model can never contain.

The nation that made a perfect map

The first problem appears in Sylvie and Bruno Concluded, published in 1893, in the mouth of a character called Mein Herr, who comes from somewhere else and is describing the cartographic achievements of his homeland. They started with maps at six inches to the mile. Then they tried six yards. Then a hundred yards. And finally they made the greatest map of all, at a scale of a mile to the mile.

It has never been spread out. The farmers objected: it would cover the whole country and shut out the sunlight. So, Mein Herr explains, the country is now used as its own map, and he assures his listener that it does nearly as well.

Borges would tell a colder, grander version of this fifty-three years later. Korzybski would give us the slogan about the map and the territory around forty years later. Carroll got there first, and got there funnier, and — this matters — got there with the farmers in the story. Borges’s map decays in the desert, admired by animals and beggars. Carroll’s map is stopped by agriculture. Somebody has to grow wheat under it. The objection is not aesthetic or metaphysical; it is that the perfect representation occupies the space where the represented thing has to live and act.

That is the theorem hiding inside the joke, and I want to state it plainly, because it is the founding principle of everything I have written about world models: a model that loses nothing does no work. Fidelity is not the goal. Fidelity is the failure mode. The entire value of a model is precisely equal to the value of what it discards, and the whole craft of building one is the craft of choosing the discards well.

Long context is a mile-to-the-mile map

Now look at what we have been building.

The dominant instinct in agent engineering for the last three years has been accumulation. Bigger context windows. Longer histories. Retrieve more chunks, retrieve them from more places, stuff them all in. Log every tool call, every observation, every intermediate thought, and keep it, because who knows what will turn out to matter. Vector-store everything. The implicit theory is that an agent’s competence rises monotonically with the quantity of territory you can hold in front of its eyes.

This is the map at a mile to the mile, and the farmers are already complaining. We know the symptoms. Attention dilutes across the irrelevant. The model becomes suggestible to whatever happens to sit near the edges of the window. Retrieval that returns forty passably-similar passages produces worse answers than retrieval that returns three right ones, and the reason is not a bug in the ranker. The reason is structural. Similarity search is a map with no scale. It has the same grain of detail everywhere, because it has no notion of what the agent is for. Cosine distance does not know which mile matters.

An agent that remembers everything does not have a memory. It has a warehouse. Memory, in anything that deserves the word, is constituted by forgetting — by a selection principle that runs continuously and throws away almost all of what arrives, on the basis of a stable sense of what this creature needs in order to act. Take the forgetting out and you have not built a superior memory, you have built the country and called it a map.

So the first Carrollian question to put to any agent architecture is embarrassingly simple, and I have watched it embarrass a lot of architectures: what is your scale? Not how much can you store. At what grain do you represent, and what licensed you to choose that grain?

The answer, I argue, has to come from the action side. A map’s scale is set by the journey. A world model’s grain is set by the agent’s affordances — by the set of things it can actually do and the distinctions those doings are sensitive to. An agent that books travel needs cities, times, prices and cancellation policies; it does not need aircraft metallurgy, and a “more complete” model that includes metallurgy is a strictly worse model for that agent, not a richer one. Resolution is not a free parameter you turn up when compute allows. It is a commitment about what counts as a difference. Choosing it is not preparation for the modeling. It is the modeling.

What the Tortoise said

The second problem is shorter and much nastier.

In 1895, Carroll published three pages in Mind called “What the Tortoise Said to Achilles.” The Tortoise, having finally been caught, proposes a puzzle. Take a simple argument. A: things equal to the same are equal to each other. B: the two sides of this triangle are equal to the same. Therefore Z: the two sides are equal to each other.

The Tortoise says he accepts A. He accepts B. He does not accept Z, and he asks Achilles to compel him. Achilles, patient, writes down a further premise. C: if A and B are true, Z must be true. The Tortoise reads it, considers it, and accepts C as well. And still refuses Z. Because now the argument runs from A, B, and C to Z, and what he would need in order to be forced is a further premise D: if A and B and C are true, then Z must be true. Achilles writes D. The Tortoise accepts D. He does not accept Z.

The dialogue ends with Achilles’ notebook filling up and the reader realising the notebook will never stop filling.

What Carroll had found is that inference cannot be reduced to premises. The step from premises to conclusion is not itself a premise, and any attempt to make it one produces a new gap of exactly the same kind. A rule of inference is not a proposition you hold. It is something you do. You can describe it, name it, write it on the page — and the description will sit there inertly beside the other descriptions, participating in nothing, until something outside the page enacts it.

Alignment by notebook

I do not think we have absorbed this. I think we are, at industrial scale, filling Achilles’ notebook.

Consider what we do when an agent behaves badly. It fabricates a citation, so we add a line to the system prompt: always verify sources before asserting. It skips a check, so we add: never proceed without confirming the state. It over-claims, so we add a paragraph on epistemic humility. Each addition is a proposition placed in the context window, which is to say, a premise. And we are surprised, again and again, when the twelfth instruction has less force than the third, when the model reads the rule and does not thereby follow the rule, when a policy document of nine hundred words produces a system that can quote its own policy while violating it.

That is the Tortoise. The instruction is a C. It is accepted and it changes nothing, because the thing we actually wanted — the disposition to act on it — is not the kind of thing that can be added as another item to the list of things present. Behaviour lives in the weights, in the training signal, in the control loop, in the scaffolding that mechanically refuses to let the next step happen until the check has run. It lives in the doing. Nothing you write in the notebook becomes a doing by virtue of being written.

The same lesson lands hard on knowledge representation, which is where I usually live. You can encode rules in a graph. OWL axioms, SWRL clauses, SHACL shapes, rule nodes with beautiful clean typing — you can put all of it inside the structure, and people do, and there is a persistent feeling that a sufficiently expressive formalism will eventually contain its own inference. It will not. It cannot. A graph containing a rule is a graph containing a description of a rule. The reasoner is not in the graph. The reasoner is a process that reads the graph and acts, and the moment you try to move it inside — to make the graph self-applying — you have started writing D, and E, and F.

This is not a reason to give up on rich representation. It is a reason to stop expecting representation to animate itself, and to take seriously that a world model is only half a system. The other half is the loop.

Two halves of the same claim

Put the two Carrollian results side by side and they say something more than either says alone.

The map result says: your world model must be lossy, and its losses must be principled, chosen against what the agent can do. The regress result says: your world model cannot contain the thing that makes it operative. Together they push toward a definition I have been arguing for in a lot of different vocabularies, and which I will state here as bluntly as I can.

A world model is not a description an agent consults. It is a compressed, action-scaled state representation plus a transition dynamics plus the loop that runs predictions against arrivals and repairs the difference. Take away the compression and you have Mein Herr’s map. Take away the loop and you have Achilles’ notebook. Most of what gets called a world model in current practice is missing one or the other, and usually the second.

This is exactly why I keep returning to Mark Burgess’s Semantic Spacetime, and specifically to the relation type LEADS TO. Of his four relations, that is the one that carries time. NEAR gives you similarity, CONTAINS gives you mereology, EXPRESSES PROPERTY gives you attribution — and all three of those are picture-making relations. They describe a state of affairs. LEADS TO is the one that makes a graph into a dynamics rather than a photograph, because it is the relation over which prediction runs. It is what lets you ask a counterfactual: if I do this here, what arrives next? An agent that cannot ask that question has a very good map and no idea which way it is walking.

That distinction — between the graph as a picture of what is and the graph as machinery for anticipating what follows — is the spine of my book World Models for AI Agents. It is a book about state, grain of resolution, transition structure, and the discipline of prediction and correction; about why an agent needs a model that is smaller than the world and pointed at its own actions, and about how to actually build one that stays honest as the world moves under it. If this article has landed anywhere near a problem you are currently having, that is the long version of the argument, and you can find it with my other books on Leanpub at leanpub.com/u/vpavlyshyn.

A note on brillig

There is a third Carroll thread I will only touch, because it deserves its own essay.

Jabberwocky is a perfectly well-formed English sentence structure with no referents in it. It is grammatically impeccable and semantically empty, and — this is the eerie part — you can parse it. You know that slithy toves gyre, that gyring is something toves do, that the mome raths are in bad shape. Syntax alone gets you a surprisingly long way toward the feeling of understanding.

I do not need to belabour where that lands. Fluent, structurally flawless, confidently delivered, unanchored. The difference between a sentence and a claim about the world is not a property of the sentence. It is whether something behind the sentence is tracking the world and can be wrong. That is another way of saying what the world model is for.

The farmers

I keep coming back to the farmers in Mein Herr’s country, because they are the only characters in either story who behave sensibly.

They are not making an epistemological objection. They have no position on representation. They simply have wheat in the ground and a thing has been proposed that would put the whole country in shadow, and so they say no, and the greatest map ever made stays rolled up forever. Everyone else in Carroll’s cartography — the surveyors, the officials, the man who is proud of a mile to the mile — has lost track of what maps were ever for. It takes someone who has to act in the place to notice that the perfect model of the place has become an obstruction in it.

That is the disposition I want in the people building agents, and I do not see much of it. We have surveyors in abundance. We are extremely good at making the map bigger. What is scarce is the person standing in the field, asking what this thing is supposed to help someone do, and refusing anything that shuts out the light.

Alice, lost, asks the Cat which way she ought to go from here. He tells her that depends a good deal on where she wants to get to. She says she doesn’t much care. Then it doesn’t much matter which way you go, says the Cat — and that is the whole of it. Without a destination there is no such thing as a good map, only a large one. Scale is not a property of the model. It is a property of the journey, and if you cannot say what the journey is, no amount of resolution will save you.

Decide where you are going. Then throw away everything that is not the road.


This is part of the Philosophers for AI series. If you want the full argument — state, resolution, transition dynamics, and the prediction-correction loop that turns a graph into a world model — my book World Models for AI Agents is on Leanpub, along with my work on metagraphs, promise graphs, and edge AI knowledge graphs: leanpub.com/u/vpavlyshyn

Originally published on Sovereign Agentic AI on Substack.