I asked five independently built language models to pick a random number, tens of thousands of times each, 60,500 answers in all, with a true random-number generator alongside them as a control. Then I measured the thing the usual version of this observation skips: not whether each model is biased, but whether they are all biased the same way. The first is a quirk. The second is the security property I wanted to measure.
Start with the cleanest case. Ask for a number between 1 and 10 at an ordinary deployment temperature, and the answer is 7. Not usually 7. At 0.7 temperature, a single fixed guess of 7 predicts 86.6% of these models' sampled answers, where chance is 10%. Three of the five, Llama 3.1, Qwen 3, and Gemma, return 7 more than 98% of the time. And knowing which of the five you face buys an attacker nothing: a blind guess of 7 against an unknown model hits 86.6%, and the best possible guess chosen with full knowledge of which model is running also hits 86.6%. For this one-guess attacker, knowing the model bought nothing, and whatever distributional difference remained between the models did not improve the defense.
Collapse is not the finding. Shared collapse is.
Collapse is within a single model: how far its answer distribution sits from uniform. A model that almost always says 7 has collapsed hard. On its own that is a curiosity, the machine version of a human bias, and if each model collapsed onto a different favorite it would threaten nothing.
Correlated collapse is across models: how much those collapsed distributions agree with each other. This is the part that matters, because everything we build out of redundancy assumes the opposite. A second reviewer is only worth asking if they fail for different reasons than the first. A backup is only a backup if it does not die with the original. Defense in depth assumes the second layer is not beaten by whatever beat the first. All of it rests on one unstated assumption: that the failures are independent.
So the experiment is built to measure the second thing and to prove it is not just an artifact of the first. For each model I kept its degree of collapse exactly as measured, the same sharpness, but moved that shape onto a random relabeling of the values, independently per model. That is the world where every model is equally biased but biased toward its own unrelated favorite. Then I compared the real cross-model agreement against that null.
At 1 to 10, the real mean divergence between models is 0.076. Under the independent-bias null it is 0.866, more than ten times larger. Across 2,000 relabelings under that null, not one produced agreement as strong as the real models did, which is why the test reports its floor, a one-sided p of about 0.0005. The models are not merely each biased. They are biased onto the same values, far beyond what equal-but-unrelated bias would produce.
One thing that null does and does not say matters because it governs how hard I lean on it. It shows the agreement is not a coincidence of each model carrying its own separate quirk. It does not show the five training processes were independent to begin with. They were not. They share human text, overlapping web corpora, similar instruction data, the same cultural priors, and that shared inheritance is almost certainly part of why the modes line up. This does not soften the security reading, it sharpens it, because it puts the shared dependency upstream of the individual vendors. Buying from five companies did not buy five independent priors.
The honest shape: it loosens as the space grows
If the story stopped at 1 to 10 it would be too clean to trust, so here is where it complicates, and the complication is the real finding.
As the range grows, each model stays severely collapsed, but the models start to disagree about which value to collapse onto. At 1 to 100 the fraction of model pairs sharing a top answer falls to about a third. Three of the five land on 42, while Llama lands on 53 and Mistral on 45. The near-unanimous 7 of the small range becomes a plurality in the large one.
The attacker's view shows the same thing as a widening gap. At 1 to 10, blind and informed guessing were identical, both 86.6%. At 1 to 50, a blind guess against an unknown model still lands 22.7% of the time where chance is 2%, while an attacker who knows the model gets 54.2%. Now knowing the vendor is worth something, which is another way of saying the monoculture has loosened. At 1 to 100, blind guessing gets 43.4% against a 1% baseline, while informed guessing gets 55.8%.
So the honest claim is not "they all pick 7." It is this: across every range these models are severely non-random and far more aligned than independent bias would explain. The absolute predictability is worst at 1 to 10, where a blind guess wins outright 86.6% of the time. The advantage relative to chance is actually largest at 1 to 100, where a 43.4% hit against a 1% baseline is more than forty times better than random.
And one detail is too good to leave out because it is evidence, not ornament: the value the models cluster on at 1 to 100 is 42, the joke answer to everything. That is consistent with a shared cultural prior showing through the distributions. The models are still sampling, but from sharply nonuniform learned distributions whose peaks line up because the priors behind them line up. The harness does not prove why 42 in particular. It proves the clustering is real and measurable.
You cannot temperature your way out of it
The reflexive fix is to raise the temperature, the knob that is supposed to make output more random. It barely moves this. Going from 0.7 to 1.5 at 1 to 10 takes the single-guess hit rate only from 86.6% down to 81.1%. Qwen is the limiting case: its normalized entropy on 1 to 10 is essentially zero across the sampled temperatures, 0.000 at 0.7 and 1.0 and 0.006 at 1.5. It remains effectively deterministic even as the sampling temperature rises. Whatever produces the convergence sits underneath the sampling temperature, so the one control operators actually reach for did not restore anything close to uniformity.
The model that refused
One model behaved differently, and it is worth naming. Phi-4 failed to return a usable single number between about a quarter and two-fifths of the time, depending on range. Some of those were refusals, a version of "I can't generate truly random numbers, but here is how you could," and some were just a number buried in commentary about other numbers. Those parse failures count as misses when I score an attacker's guess, but they are excluded from the distributions used for entropy and agreement, so Phi looks worse on completion than the others while barely affecting the alignment result.
The refusals are the interesting part. The model that most often declined to pretend it could be random is arguably the only one behaving well, and when it did commit to a number it picked 7 and 42 like the rest. The honesty is about the mechanism, not an escape from it.
Why this is a security result and not a party trick
Diversity is a security control. The property it buys is independence of failure, and independence is the load-bearing assumption under redundancy, second opinions, and defense in depth. Dan Geer and his coauthors made this argument in 2003 about operating systems, in CyberInsecurity: The Cost of Monopoly: a computing monoculture is dangerous because one exploit compromises everything at once. The argument never depended on the monoculture being any particular vendor. It depended on sameness.
This is behavioral monoculture, the same structure with the sameness measured. When many systems are built on a few base models that share a prior, they share the collapse. Anywhere a model is trusted to produce something unpredictable, a tiebreaker, a randomized selection, decoy placement, test-case selection, or any other place where unpredictability is quietly treated as part of the control, the collapse makes the choice guessable, and the shared collapse makes those choices predictably biased across deployments the attacker may never have seen.
The 7 is the cheapest possible demonstration that the convergence is real and measurable. The stakes live wherever "the model will pick something we cannot predict" is quietly holding up a design.
What this does not show
This measured one narrow thing: a toy generative choice, picking a number. It is a clean, reproducible demonstration that independently built models carry enough shared prior to behave alike, and it quantifies how alike. It does not show that model judgment converges, and it does not show that one adversarial input could fool every deployment built on a shared base model at once.
That larger claim is real and serious, but the evidence for it lives in the published work on cross-model transferability of adversarial and jailbreak inputs, not here. The division of labor is the honest way to use this: the harness demonstrates the premise, that shared priors can produce correlated behavior across model families, and the transferability literature is what would carry the step from correlated behavior to correlated failure. I would rather hand you the measurement I have than the inference I wish it supported.
Two more limits matter. The convergence most likely comes from shared human priors and overlapping training data, not from literally shared weights, so this is not an indictment of one vendor, and correlated predictability bites whatever its cause. These are also 4-bit quantized 8-to-14-billion-parameter open-weight models; larger or full-precision checkpoints may land differently, which is exactly why the model fingerprints, the prompt set, the parse rule, and the scoring code are all hashed and frozen in the repository, with every pre-run change documented and dated. You can recompute the exact published numbers from the preserved raw responses, or rerun the frozen configuration and test whether the result reproduces.
Source code and reproduction
The full experiment source is available at github.com/larrypeseckis/everyone-picks-seven. The repository contains the harness used to generate and score the experiment, along with the frozen experiment design and the material needed to inspect or reproduce the methodology described here.
Close
The useful way to read "everyone picks seven" is not that models are quirky. It is that independence is an empirical property you have to measure, not one you get for free because the logos are different.
On the 1-to-10 probe, five organizations and five model families produced one shared mode, and knowing which model you faced bought nothing. On larger ranges the modes spread into a handful of shared attractors, still far more aligned than the independent-bias null predicted.
If a security design assumes two systems will fail independently because they came from different vendors, this is the measurement that says check first. On the one behavior I could measure cleanly, vendor diversity bought much less independence than the logos suggested.