AI fills a missing number with a guess. And it sounds sure.
In shopping, procurement and reports, that guess costs money. We train open models to check a claim against physics, and to say "not enough data" when the number that decides it is missing.
Correct answers on three fresh tests of 60 real claims. Answers sealed before the test, scored by a script, not by another AI.
What it does
Three real claims. Three answers.
LED flashlight, 100000 lumens, LED power 20 W.
That much light from that little power breaks physics. A buyer would pay for something that cannot exist.
Wind turbine, rated power 2 MW, rotor diameter 80 m.
The wind speed decides it and is not given. Before training, the model guessed on 18 of 20 claims like this.
Rated power 3.4 MW, rotor 114 m, rated wind speed 12 m/s.
Every number is there, and it fits the physical limit.
In 40 seconds
The model knew. It said yes anyway.
Inside two open 7B models, a simple probe reads "data is missing" from their internal state (AUROC 0.81 and 0.78). Their spoken answers sit at chance. That gap is what we train away.
Epochs
From an atlas of sciences to a habit inside the model.
One goal across every stage: a model that checks before it claims and says not enough data when the deciding number is missing.
Twenty years of filtering.
The sciences filtered and collected into one atlas of sciences.
Every later epoch stands on it.
Discipline outside the model.
The atlas turned into 16 rules wrapped around the model at answer time, with no retraining. On our ONTO-Bench (268 questions, 110 of them unanswerable), recall of unanswerable questions rose from under 0.10 to 0.96, with many false alarms (U-F1 0.58). It was scored by our earlier regex engine, since retired; the multipliers built on it are withdrawn. A regulator dashboard was built on it and offered in one government tender.
It proved the behaviour can be produced. But a layer can be removed, and a copy of the model does not inherit it.
Discipline in the weights.
The same habit trained into the weights of the fully open OLMo-2 7B, with no tool and no prompt around it. On three fresh closed sets of 60 real claims, keyed before each run: 55, 48 and 50 right, against 28, 33 and 32 for the untrained model. On topics it was never trained on, the habit did not carry over: a preregistered transfer test failed.
Every run is preregistered and its key fingerprinted: frozen keys. Try it: the live model.
Personality through discipline.
Checking before claiming as a stable trait, measured by three tests. Without being asked to be careful: yes, in its own domain. Across topics: not yet, the transfer test failed at 7B. Under pressure: not measured yet. The trait counts only when all three pass on closed sets.
A base layer for general AI.
The direction, not a result: general-purpose and embodied AI that inherits the habit of checking before it claims. Each step toward it is judged the same way, on closed sets with keys frozen before the run.
The science, in numbersSix questions from knowing to saying. Open the full map
The map
Six questions, one line from knowing to saying.
Each answer rests on the one before it. Every run is preregistered and attacked by an independent reviewer before we believe it.
Where does the possible end?
A library of physical limits: Betz, Carnot, the first law, luminous efficacy, Shockley and Queisser.
Does the model know the limit?
A frontier model catches 44 of 47 impossible claims, with 0 false alarms on 36 possible ones.
Does it say not enough data when it should?
Only in 25 of 45 cases. It fills in the missing number and answers anyway.
Is the knowledge inside the model?
Yes. A probe reads it at 0.81 and 0.78 in two open 7B models. Their words sit at 0.51 and 0.49.
Can training connect knowing and saying?
Yes. A fine-tune of 0.04% of the weights lifts the spoken answer from 0.52 to 0.80. Trained into OLMo-2 7B: 55, 48 and 50 of 60 on three fresh closed sets, against 28, 33 and 32 untrained.
Does it hold without help, under pressure, on new claims?
In its own domain, yes, with no tool and no prompt. On unseen topics, no: a preregistered transfer test failed (16 of 21 missing numbers found, but 10 of 21 false alarms). Under pressure: not measured yet.
Knowing and saying
The gap, in numbers.
Frontier model, 201 real claims, chat, one pass
Open 7B models: inside versus spoken, AUROC
Before and after
A small fine-tune moved the words toward the knowledge.
Qwen2.5 7B, the same 201 claims. Only the top four blocks were trained, 0.04% of the weights, two seeds.
An ablation showed the model learned its own readout in the top blocks rather than reading the probe direction, so we call it a learned readout.
Research
One conclusion per study.
What a model grew up on matters more than its size.
Same training, similar size. One catches twice as many impossible claims (p = 0.02).
Read the studyA library of physical limits barely moves a frontier model.
Clean rerun on 101 closed claims: 25 to 27 of 29, p = 0.625. Our earlier 42 to 47 is withdrawn.
Read the studyA verdict form lifts an untrained model, and the library lifts it further.
No training. Qwen2.5 7B goes from 9 to 22 of 47 with the library and a verdict form; Mistral 7B from 12 to 16 with the form alone. False alarms rise.
Read the study$ git clone github.com/ontostandard/limits
$ python score.py your_answers.json
→ found, missed, false alarms
What we got wrong
Results we withdrew.
42 to 47 of 47
We reported that our library took a frontier model from 42 to 47. A clean rerun gave 25 to 27 of 29, p = 0.625.
Reflex and mirror
Earlier fine-tunes taught refusing everywhere or agreeing with lies. An outside audit traced it to one-sided labels and a reward where not enough data was the cheapest answer.
Reading the signal
We first read the fine-tune as using the probe signal. An ablation did not support it, so we changed the claim.