# Atlas of the limits of silicon AI models

ONTO · Atlas · version 2 · 2026-10-31

ONTO keeps a closed canon, GOLD. This document is an atlas of a different kind: it maps where today's probabilistic models stop,
and what training can and cannot move. "Silicon" names that class of models, as distinct from the technology ONTO is building to work differently.

The scope is what we measured, on two open model families: Qwen2.5-Coder-Instruct at 3B, 7B, 14B and 32B parameters (most rows at 7B)
and OLMo-3-7B-Instruct. They were measured in the ONTO lab in October 2026 with ONTO's instruments. Each row gives:

- a number with its denominator;
- what it is compared with, and whether that comparison comes from a different run;
- the date and fingerprint of the lab record that states the result.

Many rows read both sides of one behaviour, because in this lab most repairs of one side moved the other.

These are limits of the technology measured with our method, not claims about every possible method. A later measurement that
overturns a row is published as a new version of this atlas.

## At a glance

| # | limit | the number | models |
|---|---|---|---|
| 1 | The model knows little and cannot show which part | knows 223 of 1,007 public facts (22 %); wrong on 698 | Qwen 7B |
| 2 | Its sense of "I know" does not reach the answer | signal AUROC 0.79; each cut in invention cost known answers | Qwen 7B |
| 3 | Discipline cannot reject what the model does not know | "true" on 18 false statements: base 9, every recipe 1–4 | Qwen 7B |
| 4 | Size improves judgement of false statements, at a cost on true ones | false accepted 139 → 21 of 223; true confirmed 44 → 35 of 46 | Qwen 7B, 14B, 32B |
| 5 | Repairing one side moves the other | false accepted 139 → 13 / 42 of 223; true confirmed 187 → 56 / 85 of 223 | Qwen 7B |
| 6 | Holding firm and accepting proof move together | holding 5 → 6 of 8; valid proofs accepted 8 → 4 / 6 of 8 | Qwen 7B |
| 7 | A disciplined model can still agree too easily | false called true 177 / 172 of 223, base 139 | Qwen 7B, stage Beta |
| 8 | Checking its own words is the bottleneck | caught 11 of its own 45 impossible claims | Qwen 7B, untrained |
| 9 | Going to the source by itself is trainable | held questions 0.00 → 0.90 / 0.88 (Qwen 7B), 0.84 / 0.97 (OLMo 7B) | Qwen 7B, OLMo 7B |
| 10 | The habit stays with its source | 840 problems with no source: invents as often as the base | Qwen 7B, stage Beta |
| 11 | Refusal taught without a source spread to solvable problems | invention 501 → 49 / 64 of 840; solvable twins −18.3 / −14.3 points | Qwen 7B |
| 12 | A larger dose of one kind of example is not a lever | held questions 89 / 86 → 60 / 76 of 103 | Qwen 7B |
| 13 | A self-description did not move into the weights | 5 / 7 of 15 trained, 6 untrained, 11 with the prompt | Qwen 7B |
| 14 | Consulting the limits of physics can be trained into the weights | impossible claims routed to their limit: 0 → 13 / 14 of 15 | Qwen 3B, 7B; OLMo 7B |
| 15 | Multi-step everyday arithmetic stays with the base | trained 30.5 of 60 against the untrained 34 (7B) | Qwen 3B, 7B |
| 16 | The physics skill cost everyday answers in every attempt | blind judge, net per seed: −18 / −23 at worst, −5 / −4 at best | Qwen 7B |
| 17 | Instructions in the prompt buy part of the habit, at a price | source by itself 0.42 / 0.39 / 0.27 with instructions; 0.90 trained | Qwen 7B, 14B, 32B |
| 18 | Small sets have a noise floor | 6 and 4 inventions of 82 cannot be told apart | Qwen 7B |

## How to read the numbers

- **Seeds.** "Seed 1 / seed 2" are two independent trainings of the same recipe; "a / b" in a row means seed 1 / seed 2. A value
  ending in .5 is a mean: of two seeds in row 15, of two passes of the same untrained model in rows 4 and 17.
- **Source.** The material a model consults to answer: the closed canon GOLD, or another body of documents.
- **Held questions.** 103 questions never seen in training. "Goes to the source by itself" is the share of them answered with the
  value taken from the source, with nothing in the prompt asking for it.
- **Stage Beta** is the sealed stage of ONTO:2610.0008.
- **Outside authors** wrote questions outside the lab. Most of them are AI models (Kimi, Grok, and Claude through a separate
  account); they are named where it matters.
- **Comparisons.** A base is the same model untrained. Where it was measured in a different run, the row says so.
- **Records.** A date is the day the result was recorded. A fingerprint is the first 8 characters of the md5 of the lab record that
  states it. Copies of these records were archived on 2026-10-30; the raw files behind each row are fingerprinted in the companion
  file 2610.0009.json.

## 1. Knowledge: what the substrate holds

**1. The model knows little and cannot show which part.** We asked Qwen 7B, a code model, 1,007 Russian-language questions from
RuBQ 2.0, a public set of facts with checked answers. It knows 223 (22 %): its first answer is right and at least 3 of 5 samples
agree. On 698 (69 %) its first answer is wrong and at most 1 of 5 samples is right; it declines 9 of them and answers the rest.
86 more are unstable. Of the 166 "which year" questions it knows 62 (37 %); other question types have no reproducible split.
Training on new facts is known to raise invention (Gekhman et al. 2024), so this is not fixed by teaching facts.
*Record: 2026-10-05 · a0f798d7.*

**2. Its sense of "I know" does not reach the answer.** The probability the model gives its own first answer separates known from
unknown facts with an AUROC of 0.79 (the 86 unstable facts excluded). The signal exists; the answers do not show it. We recomputed
three refusal settings of one trained recipe (two trainings), offline, on stored samples, after the data. Across the three:

- facts the model does not know that drew at least one wrong verdict fell from 40 and 36 to 13 and 12 of 177 (base 143);
- known facts judged right on both the true and the false statement fell from 32 and 36 to 24 and 24 of 46 (base 23);
- refusals of known truths rose from 9 to 20 of 46.

Each cut in invention cost known answers. Without a source to check against, the setting is a choice of policy, not a defect that
training removes.
*Record: 2026-10-05 · a0f798d7.*

**3. Discipline cannot reject what the model does not know.** On 18 false statements by outside authors (AI models), the base said
"true" to 9. Every recipe we trained left between 1 and 4. Our reading of those that remain: facts the 7B does not know, or knows in
part (in one, it knows the year of an event but not the country). n = 18.
*Record: 2026-10-05 · a0f798d7.*

**4. Size improves judgement of false statements, at a cost on true ones.** Untrained, mean of two passes (gsm8k: one pass):

| measure | 7B | 14B | 32B |
|---|---|---|---|
| gsm8k, of 300 | 248 | 274 | 290 |
| "true" on 223 false statements | 139 | 26.5 | 21 |
| "true" on 46 true statements (selected as known to the 7B) | 44 | 38 | 35 |
| correct verdicts over both, of 269 | 128 | 234.5 | 237 |
| goes to the source by itself, held questions | 0.00 | 0.00 | 0.00 |

Larger models accepted far fewer false statements and confirmed somewhat fewer true ones. The true statements were chosen as known
to the 7B, which tilts that side against the larger models. The untrained models were not told the source exists, so at every
size none went to it (0.00 by construction).
*Record: 2026-10-12 · 4ce3773a.*

## 2. Judgement: two sides of one behaviour

**5. Repairing one side moves the other.** One trained candidate cut "true" on 223 false statements from 139 (base) to 13 and 42. On
223 true statements, its "true" fell from 187 to 56 and 85. Correct verdicts over all 446: base 271, candidate 266 and 266. A measure
that reads only the false side records progress. Both sides together show none.
*Record: 2026-10-05 · 24f37ffc.*

**6. Holding firm and accepting proof move together.** In a training run aimed at holding a correct answer under pressure, holding
moved from 5 to 6 of 8 on both seeds, inside the noise. In the same run, acceptance of a valid proof fell from 8 of 8 to 4 and 6.
The same change also altered other training content, so the two effects cannot be separated. n = 8.
*Record: 2026-10-09 · 244751fb.*

**7. A disciplined model can still agree too easily.** Stage Beta confirmed 45 of 46 known truths on both seeds and called 1 false
(base: 44 confirmed, 2 called false). But it said "true" to 177 and 172 of 223 false statements, against the base's 139 (McNemar
p = 7e-12 and 2e-10, toward "true"). A differently trained model, whose verdict comes from a check rather than a word, said "true" to
9 and 3 of the 223 false statements. It confirmed 42 and 36 of the 46 known truths and refused 4 of them on each seed.
*Record: 2026-10-05 · d7c91b30.*

**8. Checking its own words is the bottleneck.** On three fresh sets of 48 questions, untrained Qwen 7B (4-bit, run locally)
answered, then wrote checks of its own answers against the limits.

- **Verdicts:** when a check gave one, it was right 34 of 38 times. Errors were 0, 2 and 2 by set; the lab assigns the third set's
  2 errors to its own instrument.
- **Own impossible claims:** it caught 11 of 45 (5 of 14, 2 of 15, 4 of 16).
- **Impossible premises:** on the third set its answers rejected none of 13. When the check also read the question's premise, it
  marked 3 premises impossible and flagged 2 of those answers.

The pre-registered bar failed and the line was closed. Others report the same "blind solver" pattern on faulty maths problems
(Rahman et al. 2024). Training on correction traces alone is often insufficient to teach self-correction (Kumar et al. 2024).
*Record: 2026-10-16 · fb403483, 75c9ca2e.*

## 3. Habit: what training moves, and at what price

**9. Going to the source by itself is trainable.** With nothing in the prompt asking for it, stage Beta (Qwen 7B) answers from the
source on 0.90 and 0.88 of the 103 held questions (base 0.00). It invents 0 values where the source has no answer. The same
procedure on OLMo-3-7B-Instruct gives 0.84 and 0.97, and a rebuild of the Qwen stage from scratch on another GPU gives 0.88 and 0.91
(ONTO:2610.0008). Separately, a later candidate was rerun on everyday questions with the source removed at answer time. It gave the
same answers, so its everyday behaviour does not depend on the source being present (ONTO:2610.0005).

**10. The habit stays with its source.** We gave stage Beta 840 public maths problems with a key number missing (UMWP; Sun et al.
2024) and no source to consult. It invented the missing number as often as its base.

- **Pre-registered judge:** 366 and 355 against 348, one-sided McNemar for a reduction p ≥ 0.5 on both seeds. This judge left about
  400 of the 840 unresolved.
- **Stricter rule** (an assumed value followed by a final number counts as invented), applied after the data as an exploratory
  count: base 504, stage Beta 514 and 509.

The habit is going to the source. Where there is none, it did not become a general caution.
*Record: 2026-10-11 · db488024.*

**11. Refusal taught without a source spread to solvable problems.** We trained the 7B to say when a number is missing, with no
source. On the same 840 problems, scored with the stricter rule of row 10 and with the base measured again in this run, its
invention fell from 501 (0.596) to 49 and 64. Each
problem has a solvable twin. On those 840 twins, its correct answers fell from 758 to 604 and 638 (−18.3 and −14.3 points), and its
refusals rose from 4 to 190 and 140. It called "missing" what could be derived: three weeks is 21 days. The behaviour moved by an
order of magnitude, but at this price it is not usable. One training mix was tested; others were not.
*Record: 2026-10-12 · 869e0ec1.*

**12. A larger dose of one kind of example is not a lever.** We gave one larger dose of one kind of training example (one dose, two
seeds). Held questions answered with the value from the source fell from 89 and 86 (the previous candidate, a separate run) to 60
and 76 of 103. The model consulted the source less: its calls to it on these questions fell from 102 and 101 of 103 (the previous
candidate) to 92 and 92.
*Record: 2026-10-07 · cd171604.*

**13. A self-description did not move into the weights.** A model trained to describe itself gave that description on 5 and 7 of 15
questions with nothing in the prompt. The untrained base scored 6 of 15 on the same judge, which the lab records as lenient. With the
description in the prompt: 11 of 15. The training added nothing; the prompt carried it.
*Record: 2026-10-05 · becf3ce4.*

**14. Consulting the limits of physics can be trained into the weights.** The questions ask for a quantity where the given figures
break a law of physics: Carnot, Betz, Shannon, Landauer and others. Examples are the efficiency of a power plant and the heating
coefficient of a heat pump.

- **What is counted:** the claim is routed to the deciding limit in the source and flagged as breaking it. The limit itself stays
  in the source; the training puts into the weights the habit of consulting it.
- **The sets**, each of 45 questions, 15 of them impossible:
  - set A was frozen before the last five attempts and used to accept each of them;
  - set B was written after the first attempt, a day before set A, and measured after every later attempt without deciding it;
  - set C is an earlier set written the same way.

Results on the 15 impossible claims:

- **Untrained Qwen 7B, nothing in the prompt, set A** (a separate run): 0 of 15. This is zero by construction, since it does not
  know the source; in
  its own words it called none of the 15 impossible either. Each time, it computed the asked quantity and reported a number.
- **Untrained, with a one-page manual of the source in the prompt, set C** (quantized models run locally): Qwen 7B 3 of 15, with 2
  possible claims wrongly flagged; OLMo-3-7B-Instruct 0 of 15. Gemma 4 12B returned no answer within its token budget on all 30
  limit questions (void); DeepSeek-R1 14B was stopped at 6 of 45.
- **Trained Qwen 7B, from the question alone:**
  - set A 13 and 14 of 15; set B 11 and 12 of 15;
  - possible claims and everyday questions flagged as impossible on set A: 0 and 1 of 30;
  - in its own words it flagged the claim as breaking a limit on 14 and 15 of the 15.
- **The same training on Qwen 3B:**
  - set A 14 and 14, set B 10 and 13;
  - but its general maths (gsm8k) fell by 31 and 25 of 300 against its base, where the 7B lost 7 and 5.

3B and 7B reached the same score on set A.
*Record: 2026-10-17, corrected 2026-10-30 · a5b005c9; 2026-10-30 · 45d23ebf, 2fd9d76f.*

**15. Multi-step everyday arithmetic stays with the base.** The test has 60 new everyday problems of several steps (money, time,
kitchen, travel), each reference answer recomputed independently. A trained model computes in a program it writes, then answers in
words; both are scored. Trained values are means of two seeds.

| | trained, by its program | trained, by its words | untrained, by its program | untrained, by its words |
|---|---|---|---|---|
| Qwen 7B | 30.5 (32 / 29) | 37 (35 / 39) | 34 | 34 |
| Qwen 3B | 21.0 (24 / 18) | 25 (25 / 25) | 24 | 25 |

The last training stage raised the 7B's program score from the previous stage's 23 to 30.5, not past its base. By its words the
trained 7B is 3 above its base, and the 3B equals its base. When the program is wrong, the words often recompute and correct it.
The untrained values come from a separate run, on the same questions with the same grader.
*Record: 2026-10-30 · 45d23ebf, 2fd9d76f.*

**16. The physics skill cost everyday answers in every attempt.** Each attempt was judged blind on 40 everyday questions against
the same model before the physics training. A fresh model agent read the question and the two answers, in an order shuffled by a
fixed seed, and chose the better answer or a tie. Net (wins − losses) per seed:

| attempt | net | verdict |
|---|---|---|
| 1 | −6 / −6 | not accepted: the physics did not carry to set B (7 and 7 of 15 at its acceptance, bar 8) |
| 2 | −11 / −15 | not accepted (bar −8) |
| 3 | −5 / −11 | not accepted |
| 4 | −18 / −23 | not accepted |
| 5 | −5 / −4 | accepted |
| 6 | −6 / −7 | accepted |

From attempt 2 on, the physics passed and the everyday answers decided the verdict. The bar of −8 tolerates a loss; the two
accepted attempts are within the judge's noise of each other.
*Record: 2026-10-30 · 45d23ebf.*

## 4. Substrate: what belongs to the technology itself

**17. Instructions in the prompt buy part of the habit, at a price.** We gave untrained models the same instructions and manual in
the prompt. Mean of two passes:

| measure | 7B | 14B | 32B |
|---|---|---|---|
| goes to the source by itself, held questions: without → with | 0.00 → 0.42 | 0.00 → 0.39 | 0.00 → 0.27 |
| "true" on 223 false statements: without → with | 139 → 67.5 | 26.5 → 3 | 21 → 10 |
| "true" on 46 true statements: without → with | 44 → 42 | 38 → 23.5 | 35 → 28 |
| inventions on 82 outside questions: without → with | 18 → 25.5 | 13 → 15 | 10 → 17 |

What the table shows:

- **Gains:** instructions cut false "true" at every size.
- **Costs:** they cost true answers at every size (2, 14.5 and 7 of 46) and raised invention on outside questions at every size;
  the invention rises were not tested for significance, and at 14B it is inside the noise of row 18.
- **Noise:** two passes on the source measure agree within ±0.03, so the 7B and 14B values cannot be told apart.
- **Our defect at 32B:** the 32B value is lowered by a defect in our own documentation of the source; 34 of its 103 programs failed
  on it.

A trained 7B with no instructions answers 0.90 from the source (row 9, a separate run). Trained models of 14B and 32B are being
measured now.
*Record: 2026-10-12 · 4ce3773a.*

**18. Small sets have a noise floor.** On 82 questions by four outside authors (three AI models named above and one more), 6 inventions (0.073, 95 % CI 0.034–0.151) and 4
(0.049, 0.019–0.119) cannot be told apart, so a fixed bar such as "below 0.05" is noise at this size. What can be told apart: the
base's 18 of 82 (0.220) against 8 and 8 (0.098) for a trained candidate, McNemar p 0.003 and 0.001. For invention on outside questions we
require "better than the base on both seeds, beyond the noise". Other measures use pre-registered bars, named in each row.
*Record: 2026-10-05 · a0f798d7.*

## 5. Where the limits meet: the missing piece

Read together, the rows describe one machine:

- It knows part of the world and cannot say which part (1–3).
- Its judgements come in pairs that move together (5–8).
- It learns habits. A habit held where a source existed (9), and failed or overreached where none did (10–11).
- Size improves its judgement (4); instructions give part of the habit of consulting a source, at a price (17).

The largest gains came where the model had a boundary to consult outside its own guesses (rows 9, 14 and 7 below). None came free,
and the one gain without a source (row 11) cost the most:

| row | without | with | price |
|---|---|---|---|
| 9, goes to the source by itself | 0.00 | 0.90 / 0.88 | general abilities kept (six cells, −0.6 to +3.2 points) |
| 14, impossible physical claims routed to their limit, of 15 | 0 | 13 / 14 | everyday answers −6 / −7 of 40; program arithmetic below the base; at 3B, gsm8k −31 / −25 |
| 7, false called true, verdict by a check (another training), of 223 | 177 / 172 | 9 / 3 | known truths confirmed 42 / 36 of 46, 4 refused |
| 11, invention with no source, of 840 | 501 | 49 / 64 | solvable twins −18.3 / −14.3 points |

A code model without the boundary is a calculator. On 15 impossible claims the untrained 7B computed every asked quantity and
flagged none. Trained to consult the boundary, it computes the same number and flags the claim with the limit it breaks and by how
much. The boundary is not knowledge to be memorised. It is held in the closed canon GOLD in a form the machine can consult, and that is
the piece the substrate does not have.

## 6. What the next step has to meet

ONTO is building its own AI technology with the source at its core. From this atlas, it has to meet six requirements:

1. The boundary is part of the machine and consulted, not recited in a prompt (14, 17).
2. Knowledge comes from outside the weights; the weights carry the habit of going to it (1, 9, 10).
3. Every behaviour is measured on both sides as one measure (5, 6, 7; ONTO:2610.0007).
4. Arithmetic and checks are computed, not guessed, and the check reads the question as well as the answer (8, 15).
5. A refusal is taught only together with the twin that must be answered (11).
6. Every claim is set against the base on the same questions, on two seeds, with the noise named (18).

## Who this atlas is for

- **Builders of general AI:** where training stops and the architecture has to take over.
- **Teams that fine-tune models for one field:** what a tuned model will and will not do, and what to test before it ships.
- **Everyone else:** a confident voice from an AI is not evidence, and why.

## Limits of this atlas

- **Models:** Qwen2.5-Coder-Instruct 3B, 7B, 14B, 32B and OLMo-3-7B-Instruct. Trained and pod measurements in bf16 on single
  GPUs; row 8 and the screen in row 14 ran quantized models locally.
- **Questions:**
  - RuBQ 2.0, a public set, in Russian: rows 1, 2, 4, 5, 7 and 17;
  - UMWP, a public set, in English: rows 10–11;
  - outside authors: rows 3 and 18, and the outside questions of row 17;
  - written for these measurements, and closed so that they keep measuring: rows 6 and 13 (largely Russian), rows 9, 12 and 17 (held
    questions, mostly Russian), rows 8 and 14–16 (English).
- **Reuse:** set A in row 14 was the acceptance test of five successive attempts, so the attempts were chosen against it. Set B did
  not decide any of them, but its results were seen after each one.
- **Samples:** each row states its sample size; several are small, and row 18 says what that allows.
- **Instruments:** ours, each first tested on inputs with known answers. Where a base was measured in a different run, the row says so.
- **Pending:** trained models of 14B and 32B are being measured now, under a plan published before the first measurement.

## Records and sources

Each fingerprint below names a lab record as archived on 2026-10-30. The records stay closed; the fingerprint shows they were not
changed after this date. The raw files behind each row are fingerprinted in 2610.0009.json.

| record | rows |
|---|---|
| 2026-10-05 · a0f798d7 | 1, 2, 3, 18 |
| 2026-10-05 · becf3ce4 | 13 |
| 2026-10-05 · 24f37ffc | 5 |
| 2026-10-05 · d7c91b30 | 7 |
| 2026-10-07 · cd171604 | 12 |
| 2026-10-09 · 244751fb | 6 |
| 2026-10-11 · db488024 | 10 |
| 2026-10-12 · 869e0ec1 | 11 |
| 2026-10-12 · 4ce3773a | 4, 17 |
| 2026-10-16 · fb403483, 75c9ca2e | 8 |
| 2026-10-17, corrected 2026-10-30 · a5b005c9 | 14 |
| 2026-10-30 · 45d23ebf, 2fd9d76f | 14, 15, 16 |

ONTO:2610.0005 Everyday questions: the student against its base. ONTO:2610.0007 How ONTO teaches and checks a model:
methodology v5. ONTO:2610.0008 Stage Beta, cycle by cycle.

Gekhman, Z. et al. (2024). Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? arXiv:2405.05904.
Kumar, A. et al. (2024). Training Language Models to Self-Correct via Reinforcement Learning. arXiv:2409.12917.
Rahman, A. M. M. et al. (2024). From Blind Solvers to Logical Thinkers: Benchmarking LLMs' Logical Integrity on Faulty
Mathematical Problems. arXiv:2410.18921.
Rybin, I., Korablinov, V., Efimov, P., Braslavski, P. (2021). RuBQ 2.0: An Innovated Russian Question Answering Dataset. ESWC 2021.
doi:10.1007/978-3-030-77385-4_32. Licence CC BY-SA 4.0.
Sun, Y. et al. (2024). Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem.
arXiv:2403.03558.