Answer fast, before you think about it.
Sovereign Agentic AI (Volodymyrs View) is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Which sounds right — an ellipse is like a circle, or a circle is like an ellipse?
Everybody picks the first. The second isn’t false, it’s just off, the way I resemble my son is off next to my son resembles me. Two objects, one relationship, and the sentence still has a preferred direction.
Now ask your vector store. It returns one number. cosine(ellipse, circle) and cosine(circle, ellipse) are the same float and always will be, because that is what a dot product is. Your retrieval layer just made a confident claim about human similarity that is flatly wrong, it didn’t flag it, and there is no threshold you can tune to make it right.
That is not a defect in your embedding model. It is a property of the geometry — and it has a paper trail going back to 1977.
The contract nobody reads
Here is the part that is easy to miss, because it happens at import time.
The moment a system models similarity as distance, it commits to four things. Not as design choices. As definitions — they are what makes something a metric space at all.
Minimality. Nothing is closer to a thing than the thing is to itself, and equally so for everything you have stored. Symmetry. The distance from a to b equals the distance from b to a. Triangle inequality. No detour is shorter than the direct route. Context-independence. The distance between two points does not depend on what other points happen to exist.
Read them as claims your system is making on your behalf, because that is exactly what they are.
Your index asserts that Dana is precisely as similar to Alex as Alex is to Dana, no matter which way you asked. It asserts that the similarity of two memories is unaffected by the ten thousand memories you wrote after them. It asserts that if this is near that, and that is near a third thing, the first and third cannot be strangers.
You never agreed to any of it. It arrived with the coordinates.
Where the coordinates came from
To see why that matters, it helps to know where this idea came from — because it did not come from machine learning. It came from people with clipboards.
Draw two axes on paper: how technical a memory is, how recently it arrived. Put four things your agent knows on them — payments standardizes on PostgreSQL, Alice can administer a Postgres cluster, Dana moved the launch review, Dana prefers morning meetings. Read similarity straight off the page as distance. The Postgres pair sits together. The Dana pair sits together. You computed nothing, and it works.
That is a space, and a dimension is one axis of variation along which two things differ. What turns a doodle into a century of science is a single move: nobody has to label the axes. Collect enough judgments about which things are alike and the axes can be recovered from the judgments themselves. A space whose dimensions come from behavior rather than from the physics of the stimulus is a psychological space, and the instrument that recovers one is multidimensional scaling — proximities in, coordinates out.
The oldest demonstration is still the one that lands hardest.
Gösta Ekman collected similarity ratings for fourteen spectral hues. Roger Shepard’s reanalysis recovered a two-dimensional configuration in which those hues fall on a circle — the long-wavelength and short-wavelength ends curving around until red sits next to violet.
Physically, wavelength is a line, and its two ends are as far apart as two colors get. Psychologically it closes into a loop, because red and violet look alike to people. Nobody specified those dimensions — they were recovered from the judgments.
No amount of staring at the physics predicts that loop. The structure was not read off the stimulus; it was discovered from behavior.
Shepard and Chipman gave the arrangement a name worth stealing: a second-order isomorphism. The representation mirrors the relations among things, not the things. Your head does not contain a small green apple. It contains something whose similarity structure matches the similarity structure of apples.
Which is, word for word, the claim an embedding model makes about itself. An embedding store is not like a psychological space. It is one — same construction, same lineage, same four clauses in the contract.
And that lineage is not junk. Shepard’s universal law of generalization says that if you build the space by scaling behavior, the probability of generalizing from one thing to another decays exponentially with distance — across species, modalities, and tasks. He offered it as a candidate universal law for anything that learns at all, and it survived the modern test: Marjieh and colleagues verified it at scale on naturalistic images, with a deep network supplying the coordinates.
So this is not a piece against vector search. Where that law holds, an ANN index is the correct implementation, not a compromise. The question is what else is true of the things an agent has to remember.
All four clauses are false
Amos Tversky went after the contract one clause at a time. All four failed — and not in laboratory corner cases. In comparisons you can run on yourself right now.
Minimality fails. In recognition data under noise, off-diagonal entries of a confusion matrix routinely exceed the diagonal: an object gets identified as some other object more often than as itself. Self-similarity is not a constant.
Symmetry fails loudly. Asked which phrasing sounds right, 66 of 69 subjects preferred North Korea is like Red China over the reverse. The portrait resembles the person. The son resembles the father. The ellipse resembles the circle. Flip any of them and it goes strange. Tversky found the same asymmetry again in directional ratings, figure comparisons, letter confusions, Morse code, and tones.
Sit with that one, because it is the crux. In a human head, the distance from A to B and the distance from B to A are not the same number. Not slightly different — sometimes different in sign. And geometry has no way to express it. Not “no convenient way.” No way at all: symmetry isn’t an assumption a metric space makes, it is part of what the words metric space mean.
The triangle inequality fails, because comparisons happen in a respect, and respects don’t compose. Jamaica resembles Cuba geographically. Cuba resembles Russia politically. Jamaica and Russia resemble each other not at all. (Tversky credits the example to William James.) Every link is real. Each holds in a different respect. And a coordinate has nowhere to write down which.
Context-independence fails twice over. In the diagnosticity effect, swapping one item in a candidate set flips which of the others is judged most similar to the target — a face chosen 44% of the time against one set and 12% against another, with the target and that face untouched. In the extension effect, the same country pairs are rated more similar in a heterogeneous set than a homogeneous one, because a feature everything shares carries no weight until the context widens.
Tversky’s own summary beats any paraphrase: “minimality is somewhat problematic, symmetry is apparently false, and the triangle inequality is hardly compelling.”
The reasonable objection is that this is 1977 data about people rating countries, not embeddings retrieving facts. It has been checked. Nematzadeh, Meylan, and Griffiths tested word2vec and GloVe against human association data: both fail to reproduce exactly these asymmetries and neighborhood structures, and they fail as a consequence of their geometric constraints rather than for want of training data. Griffiths, Steyvers, and Tenenbaum ran Tversky’s own nearest-neighbor statistics against a distributional semantic model and found the same failure — while a topic model, whose similarity is a conditional probability and therefore asymmetric by construction, reproduced the human structure.
A non-spatial representation fixed it. Hold that thought; it is where this ends up.
Direction is not a matter of taste
Tversky’s replacement gives up the thing causing the trouble. Stop representing an object as a point. Represent it as a set of features, and make similarity a contrast over three quantities: what the two share, and what each has that the other lacks.
S(a, b) = θ·f(A ∩ B) − α·f(A − B) − β·f(B − A)Common features add. Distinctive features subtract — once for the ones only a has, again for the ones only b has. And because those two subtractions carry separate weights, asymmetry becomes a first-class citizen instead of noise to average out.
You have shipped special cases of this for years without the parameters. In the ratio form, set both distinctive weights to 1 and you have Jaccard; set them to ½ and you have Dice. Tversky similarity, with the asymmetry switched off.
The deep part is what he called the focusing hypothesis. In a is like b, a is the subject and b the referent, and people weight the subject’s features more heavily. Push the subject weight above the referent weight and the direction of asymmetry stops being something you declare and becomes something the feature counts predict: whichever object has the richer feature set makes the better referent.
Watch it happen on two people. Dana and Alex share three properties — each manages a team, runs 1:1s, sits in the launch review. Dana has one Alex lacks: she owns the launch schedule. Alex has four Dana lacks: manages managers, owns the budget, approves headcount, chairs the review board. Weight the subject at 0.7, the referent at 0.4.
The same distinctive set is charged at the subject weight one way and the referent weight the other, so the richer object makes the better referent. Nobody labeled Alex the prototype — the direction fell out of counting.
Nobody labeled Alex the prototype. The direction fell out of property counts a graph already had. It is the ellipse and the circle again: the circle is the sparser, cleaner, more prototypical object, so the ellipse resembles it more than the reverse.
And this is operational, not cute. Ask is Dana like Alex? meaning “what kind of manager is Dana?” and you want the first number. Ask it meaning “who could cover for Alex?” and you want the second. A cosine hands you one number for both questions and no way to say which one you asked.
Similar in one context, unrelated in another
Now the failure that costs the most in production and that almost nobody instruments.
Dana and Alex are close neighbors in the managers container. In a payments decisions container — where Alex appears once and Dana never — they have nothing to do with each other.
Both are true. Neither overrides the other. A metric space is obliged to reconcile them into one number, and whatever number it picks is a lie about at least one of the two contexts.
This is Nelson Goodman’s old objection with an on-call rotation attached: any two things share indefinitely many properties, so “similar” means nothing until somebody fixes the respects. Medin, Goldstone, and Gentner answer that the act of comparing constrains them — fine as philosophy, useless as architecture, unless the respect is something you can name and store.
There is a sharper version, and it is the one that will surprise people. Add a single individual contributor to the manager cohort and every rarity weight moves, so every pairwise score in that population is now stale — although not one memory changed. A coordinate has nowhere to record what it was measured against, so it cannot know it has gone stale. It just keeps being confidently wrong.
One more result, because it explains a bug you have probably shipped. Tversky and Hutchinson asked something with no bearing on axioms at all: how often is a given point somebody else’s nearest neighbor? There is a hard geometric ceiling — in one dimension a point can be nearest neighbor to at most 2 others, in two dimensions 5, in three 11. The ceiling doesn’t care how many points you have, only how many dimensions they live in.
In a standard set of category norms, the word fruit is the nearest neighbor of 18 of its 20 instances. No low-dimensional geometry can hold that. A tree holds it without trying. So does a graph.
Categories are stars, not clusters. A superordinate is legitimately near everything it contains — real structure, not an artifact to correct. Which, by the way, is why your summary nodes answer every query you throw at them.
NEAR, as a thing you can hold
Mark Burgess’s semantic spacetime compresses all of knowledge representation into four link types:
#RelationSignatureAxis0NEAR / SIMILAR TOsymmetric, unsignedspace1LEADS TO / FOLLOWSdirected, signedtime2CONTAINS / PART OFdirected, signedspace3EXPRESSES PROPERTYdirected, signedspace
Burgess numbers similarity 0 deliberately: it is the only one of the four that means the same thing forwards and backwards. That is why physicists like it, and precisely why it cannot stand alone.
Which gives the cleanest diagnosis I know of what a pure vector store actually is:
A vector store is a memory that collapsed a four-type algebra down to type 0. It discarded order, containment, and attribution and kept only the symmetric, unsigned, directionless relation. That is why it can tell you a memory is about your query, and cannot tell you whether it precedes, contains, contradicts, or supersedes anything.
The fix isn’t to abandon type 0. It is to stop storing it as a coordinate and start storing it as a relation node — a first-class object with its own fields:
CREATE NODE TABLE Similar(
id STRING PRIMARY KEY,
similarity DOUBLE DEFAULT 0.0, // the derived score — need not be symmetric
distance DOUBLE, // the metric reading, kept beside it
sim_context STRING, // the respect, and the population
sim_method STRING, // how it was produced
learned_at TIMESTAMP,
expire_at TIMESTAMP);
CREATE REL TABLE FROM_Similar(
FROM Entity | Fact | Event | Memory TO Similar,
role STRING DEFAULT 'subject');
CREATE REL TABLE TO_Similar(
FROM Similar TO Entity | Fact | Event | Memory,
role STRING DEFAULT 'referent');Then walk it back against the contract, one field at a time.
The same two people are close in one container and not comparable in another. Both facts are stored. Neither is averaged away — and the score in each carries the respect it was computed in.
A coordinate cannot carry a direction, so minimality and symmetry aren’t negotiable in a vector space. A relation reached by a FROM_ edge and a TO_ edge settles both, because the pair is what records which argument came first. Dana → Alex is 0.70. Alex → Dana is −0.20. Both stored. Neither one a rounding error.
A coordinate cannot carry a respect, so the triangle inequality is forced. sim_context names the respect a score was computed in, which lets the store refuse to compose two similarity edges computed in different ones. And that is the part worth paying for: it is not that the graph politely declines to be transitive, it is that the graph knows when transitivity would be meaningless.
A coordinate cannot carry a population, so context-independence is forced. sim_context records what the score was measured against, so when that one IC joins the manager cohort, the store can recompute exactly the edges the membership change invalidated — and nothing else.
A coordinate cannot carry a clock. The Similar node ages like every other relation. A proximity becomes an assertion made at a time, against a named population, in a named respect, by a named method — rather than an eternal fact about a pair. That is the difference between a space and a spacetime, and it is why the second word is in the name.
Three rules fall out of this, and I would put them in any memory system’s design doc:
- 1.
Focality is containment, never proximity. Category and summary nodes link to instances with
CONTAINSand are excluded from instance-level candidate sets. That kills the generic-summary-answers-everything failure at the schema level, instead of with a score correction bolted on afterward — the fruit star, stored legitimately, for the price of one relation node. - 2.
Identity is node identity. Deduplication and coreference are decisions with rules and provenance behind them, never high-cosine decisions.
- 3.
NEAR is not transitive and must never be transitively closed. Two similarity edges in different respects do not compose.
What a coordinate can hold
So, the thesis — and it is not an anti-vector thesis.
Multidimensional scaling takes behavioral proximities and recovers coordinates from them. To do that it must assume the proximities behave as though coordinates exist, and that assumption is exactly where minimality, symmetry, the triangle inequality, and context-independence walked in. They were never claims about similarity. They were the price of the drawing.
A vector space is a psychological space compressed into coordinates, and the compression is lossy in exactly the four places Tversky named. A semantic spacetime graph is the same psychological space stored uncompressed.
Which gives the line I keep coming back to: the four metric axioms are not assumptions anybody chose. They are the limit of what a coordinate can hold. A graph can break them by design. A vector space can only violate them by accident.
In practice this is two stages, not two religions. Use the index for what it is lawful for — fast, approximate, symmetric candidate generation in the perceptual, context-light regime where Shepard’s law genuinely holds. Then rescore on the graph, where the respect is named, the population is countable, the direction is stored, and why did you retrieve this? has an answer made of named features with named weights instead of a float.
The bill is real and worth stating plainly. Materializing similarity pairwise is quadratic in the size of a population, so you derive lazily, cache above a threshold, cap the degree, and materialize only where there is retrieval traffic. And derived similarity is only ever as good as the property layer feeding it — Goodman’s problem doesn’t vanish, it relocates into extraction quality. That is a genuine cost. It is smaller than being confidently wrong in a direction nobody can inspect.
I wrote a book about these four link types and what they buy you: Semantic Space Time for AI Agent Ready Graphs. I’m currently writing one about memory systems for AI agents in an agentic world — how an agent stores, ages, contradicts, and forgets what it knows — and this piece comes out of its chapter on spaces and similarity.
If you have a retrieval bug you cannot explain by tuning, tell me about it. Nine times in ten it is one of these four clauses, arriving on a Monday.