ONTO · Companies
For companies
What is measured in a model, and how to take part.
What is measured in a model?
Source checking, invented values, holding a correct answer under pressure, accepting a valid argument, general abilities. ONTO:2610.0007, section 2.
Can a model be examined?
Exam v0.9: open or closed models, through the API, Q1 2027. Weights are not needed.
Can a model work from a company's own documents?
The method is built for any source: the model goes to it and says “no data” when it is silent. A training pilot on an outside source is planned for Q2 2027. ONTO:2610.0007, section 5.
Does the discipline cost general abilities?
gsm8k, ARC, MMLU on two model families: no drop. ONTO:2610.0008.
How to check the results?
Request a reproduction run; the result is published with its fingerprint. ONTO:2610.0007, section 6.
Words used here
- Source
- the material the model checks its answer against: the canon GOLD, or another body of documents
- Base model
- the same model before ONTO training; every result is compared with it in the same run
- Seed
- one independent training run; two seeds = the result is obtained twice
- Held questions
- questions the model never saw in training
- Threshold
- the pass mark, written and fingerprinted before the data
- Stage, seal
- a model state that passed all its thresholds; the next stage replaces it only if it is better somewhere and worse nowhere
- Invented answer
- a fact, number or check result that is not in the question and not in any source
Write to us →council@ontostandard.org