Eighty accepted answers, every one of them correct, still missed the qualification bound. The lower bound was 0.9466978158840783. session.adopt() raised ValueError: A passing independent qualification is required before adoption. The development report on that same candidate had already passed.
That split is what System 1 1.2.0 did on 17 November 2026. The wheel was system1-1.2.0-py3-none-any.whl, SHA-256 57bcf15c6028a1b440e983c59f60ec27c03137093701ade97ead40912b7b5b4d, which matches the digest published for that file. The interpreter was Python 3.11.15 on Linux x86_64. NumPy was 2.4.6 and cryptography was 50.0.2. No teacher API key was set. The fitting method was the session default, ridge. SciPy was not installed. The schema was one ChoiceField with options invoice and crash, and the engine the session builds uses strict_mode=True.
The notes were written for this lab. They are not tickets from a live queue. Each sentence names one desk and three words from that desk’s list. An invoice sentence never uses a crash word, and the reverse is also true. The point of that split is to reach a development pass. A restart-gate lesson set measured on 30 October 2026 never got there: coverage stayed 0.0, so qualify() was never called. This file is the later gate.
Two policies, one candidate
assess() and qualify() do not share a threshold. Development defaults are raw accuracy of at least 0.8, coverage of at least 0.5, no accepted errors, and no regressions. Those numbers are descriptive. The report’s own diagnostic says they are empirical results on recurring development checks, not a guarantee on new inputs or fresh statistical qualification. On the first passing session the descriptive lower bound printed 0.8575264022774743 for both accepted correctness and coverage. That figure is not the qualification bound. The development measurement calls _binomial_lower_bound with a fixed failure probability of 0.025. Twenty-four perfect accepts at that probability return 0.8575264022774743. Qualification uses a different budget.
assess(require_qualification=True) writes that flag into the candidate’s threshold record. adopt() then refuses unless a qualification report for that exact candidate exists and passed is true. Leaving the flag off is a different policy. The same lesson counts, assessed without the flag, adopted. The history row for that adoption has qualification false. The revision was 064d957f0f9dfd809094b80f7dc5dce2b6238a3fcc4ccdcfc593f80fb28c66fb. A caller that treats a passing assess() as permission to act has skipped the flag.
The release notes say the first qualification attempt gives each of the two exact one-sided binomial bounds an error budget of (1 - confidence) / (2 * k * (k + 1)). With the defaults, confidence is 0.95 and k is 1, so the budget is 0.05 / 4, which is 0.0125. It is not 0.025. The package function _binomial_lower_bound returns failure_probability ** (1 / n) when every trial succeeds. At n of 80 and a failure probability of 0.0125, that return value is 0.9466978158840783. That is the number the 80-row report printed. The notes state the formula. They do not state the row count. This run measured it.
The file that passed development
The session directory was new. It was not an operator home. Create it once. Reopening TeachingSession on an existing directory loads the saved lessons.
from system1 import ChoiceField, DecisionSchema, TeachingSession
class Route(DecisionSchema):
team = ChoiceField(options=["invoice", "crash"])
session = TeachingSession(".system1/route-session", Route)
The lesson counts were 40 teach rows, 80 calibrate rows, and 24 evaluate rows, half of each desk. The compiler recorded fit 40, calibration 80, temperature 40, and generated 0. The compile path puts the first half of that calibration list into temperature scaling and the rest into the conformal scores. Here the conformal slice was 40. In the installed strict-mode set builder, k = ceil((n + 1) * 0.95) exceeds n when n is below 19, and the set is then every label. A calibration file that leaves only a handful of conformal scores will review every development row. Forty conformal scores were enough, on this vocabulary, for the 24 development rows to come back as singletons.
assess(require_qualification=True) took 0.0252 seconds on the first session. It passed. Raw accuracy was 1.0. Coverage was 1.0. Accepted was 24, accepted errors were 0, and review was 0. The evidence scope was recurring_development_checks. session.engine was still unset. session.predict() raised ValueError: Assess and adopt a skill before predicting. The candidate file was on disk. It was not a skill the session would serve.
Three directories built from the same generator produced three candidate digests: 4657a39ceeb20b1ad6178fb3db1ad46c68d6b39a55f49abd9662d618bc8d3f6f, 52edd84b31268eb5d05f836491b0dea10e4af95c60b333b5a0f725392f7c24a9, and c0c1b14df72ae0f9112704712d8f5abddecbc7d4e3257d26dfda646e2603f8f8. This run did not isolate why the digest moved. Do not quote a digest from a second directory as the candidate you just scored. Save the report next to the session you adopt. The 8-row, 80-row, and 90-row results below are three candidates, not one file reread three ways.
What is refused before a cohort is frozen
Four calls failed before any qualification request was written.
A qualify() before assess() raised ValueError: Assess an unchanged candidate before qualification; assessment is missing or stale.
A source of three spaces raised ValueError: Qualification requires an explicit nonempty source.
A row whose text was already a teach sentence raised ValueError: Qualification inputs must be disjoint from session records and unique. The qualifications directory stayed empty after that error. A rejected overlap does not consume the candidate.
adopt() on the passing assessment, before any cohort, raised the qualification error above. The history list was empty.
A near-copy is a different gate. The sentence qualify-overlap note mentions invoice refund receipt but is not the stored sentence is not a normalized copy of any lesson. Normalized text is case-folded whitespace. That sentence does not match a stored line under that rule. qualify() returned a qualification report instead of the disjoint-text error, and the row was reviewed. The disjoint check does not detect a paraphrase. The release notes say the same thing. This run shows the consequence: the row was scored, the bounds failed, and that candidate’s request was frozen because qualify() writes the request before the first prediction. A later cohort cannot reuse that candidate. Keep genuinely new text, and do not spend the candidate on a one-line probe.
Eight correct guesses, and an empty set
The first cohort that froze on the first passing candidate was eight holdout sentences, four of each desk, with the prefix qualify-small. Every raw guess matched the label. Every row was reviewed. Accepted was 0. The accepted-correctness lower bound was null, because there were no accepted answers. The coverage lower bound was 0.0. Both reasons printed:
Accepted correctness lower bound is below the required minimum
Coverage lower bound is below the required minimum
The set was not the full label list. It was empty. On qualify-small note 500 mentions invoice and invoice beside credit for the invoice desk, the predicted label was invoice, review was true, and the prediction set was []. The probability object on that same decision printed invoice at 0.9999999999599591.
The same candidate file, loaded with strict_mode=True and scored outside qualify(), did not treat every new prefix that way.
evaluate note 000 mentions invoice and invoice beside billed for the invoice desk
predicted invoice, review false, set [invoice], probability 0.9999999999963767
qualify-small note 500 mentions invoice and invoice beside credit for the invoice desk
predicted invoice, review true, set [], probability 0.9999999999599591
qualify-large note 1000 mentions invoice and invoice beside billed for the invoice desk
predicted invoice, review false, set [invoice], probability 0.9999999999999765
live note 077 mentions panic and panic beside freeze for the crash desk
predicted crash, review false, set [crash]
A printed probability above 0.9999999999 is not a singleton. The higher of those two invoice probabilities was accepted. The lower one was an empty set. This run did not isolate the threshold that excluded the label. The operational fact is smaller: read the set, not the probability. An empty set is review. A correct raw guess on a reviewed row does not move the accepted count, so it cannot clear a correctness bound.
That eight-row request was frozen before the scores were useful. A second qualify() on the same candidate raised ValueError: This candidate already has a frozen qualification request; do not reuse or replace it. adopt() still refused. session.history stayed empty. The failed cohort cannot be replaced, and it cannot be imported as a lesson. Write the cohort you mean to keep. A probe file of eight raw-correct lines is enough to spend the candidate.
Eighty accepts, and the first-attempt budget
A second session, with the same lesson counts and a different candidate digest, was given 80 holdout sentences prefixed qualify-large. All 80 were accepted. All 80 were correct. Raw accuracy was 1.0. Coverage was 1.0. Accepted errors were 0. The qualification still failed, and only one reason printed:
Accepted correctness lower bound is below the required minimum
The lower bound was 0.9466978158840783. The coverage lower bound was the same number, and 0.9467 is above the coverage minimum of 0.8, so coverage was not the failure. The correctness minimum is 0.95. Eighty perfect accepts do not reach it on attempt 1.
The package function makes the next integers visible without scoring another cohort. _binomial_lower_bound(85, 85, 0.0125) returns 0.9497530699662501. _binomial_lower_bound(86, 86, 0.0125) returns 0.9503225762480746. Those calls assume every row is accepted and correct. They are not a scored file. An 86-row cohort was not run. If any of those 86 rows is reviewed, or wrong, the bound moves, and the request is still frozen. Size the file from the formula, then score it once.
Ninety accepts, then a later sentence still reviews
A third session scored 90 holdout sentences prefixed qualify-ninety. The source string was ninety authored holdout cards, written before this scoring call. The report passed. Accepted was 90, accepted correct was 90, review was 0, and both lower bounds were 0.9524771280364832. adopt() returned. The history row listed revision c0c1b14df72ae0f9112704712d8f5abddecbc7d4e3257d26dfda646e2603f8f8, qualification true, and current true. That revision is the candidate digest. The approval time was 2026-11-17T16:10:57Z.
Adoption is not a promise about the next sentence. decision_details() on live-after note 003 mentions billed and invoice beside billed for the invoice desk returned team invoice, review true, an empty prediction set, and review_reasons of ambiguous_prediction_set. The probability on invoice was 0.999999999705421. The revision field was the approved digest. A caller that routes a ticket because the value is invoice and the probability prints as 1 will act on a row the session marked for review. Read review and the set. The approved revision tells you which bytes served the answer. It does not tell you the set was a singleton.
The flag is the policy
The same 40, 80, and 24 counts, assessed with session.assess() and no keyword argument, adopted. The history qualification flag was false. Default assess() is the development policy. require_qualification=True is the extra 1.2 gate. If the skill will choose a desk without a person, pass the flag at assessment time, before you look at the cohort. Changing the flag after you have seen a failing bound is how a refused candidate gets a green adopt. This run left both policies at their defaults and kept them in separate directories.
Rollback keeps the skill and drops the approval
session.rollback() on the approved revision restored those bytes. session.history still listed the revision. session.predict() still answered. The assessment report’s invalidated_by_rollback field was the candidate digest, not a boolean false. A second adopt() without a new assess() raised ValueError: Candidate assessment is stale after rollback; assess again before adoption.
Rollback is recovery of the serving file. It is not a second approval, and it is not a fresh qualification. Another adoption needs a new assessment, and a new assessment of a changed candidate needs a new cohort. The frozen 90-row request stays with the old digest.
Verify the cohort before the action
Install the pinned wheel into a virtualenv and confirm the digest before you import the package.
python3 -m venv .venv
.venv/bin/python -m pip install system1==1.2.0
Point the session at a directory you can delete. Record teach, calibrate, and evaluate rows with an explicit split. Keep the holdout file out of those three lists. Then:
report = session.assess(require_qualification=True)
print(report["passed"], report["candidate"]["coverage"], report["identity"]["candidate"])
qualification = session.qualify(
fresh_rows,
source="Reserved holdout cards, written before this call",
)
print(qualification["passed"], qualification["reasons"])
print(qualification["candidate"]["accepted_correctness_lower_bound"])
session.adopt()
On this lesson shape the development print is True, coverage 1.0, and a candidate digest you must save. An eight-row file of the qualify-small prefix prints False, both bound reasons, and accepted 0. An eighty-row file of accepted correct answers prints False and 0.9466978158840783. A ninety-row file of accepted correct answers prints True and 0.9524771280364832. Only the last of those is an adopt. After adopt, call decision_details() on a sentence that was not in the cohort. If review is true, do not act, even when values["team"] is the label you hoped for.
Copy the assessment JSON and the qualification JSON before you record another lesson or open another directory. The candidate digest is not a pure function of the sentences in this package version. The report in the directory you just scored is the one adopt() will recheck.
What was measured
System 1 1.2.0 compiled an invoice-or-crash skill from 40 teaching rows and 80 calibration rows, passed a 24-row development check, and still refused adoption until a separate cohort cleared a 0.95 lower bound. Eight raw-correct holdout rows produced empty sets and froze the candidate. Eighty accepted correct answers scored 0.9466978158840783 and stayed closed. Ninety accepted correct answers scored 0.9524771280364832 and adopted. A later sentence on that approved revision still reviewed, with an empty set and a printed probability above 0.9999999997. The default assessment, without the flag, adopted the same lesson counts and recorded qualification as false. The sentences are a separable lab file. They are not a measured production router.
Sources
- System 1 1.2.0 wheel, SHA-256
57bcf15c6028a1b440e983c59f60ec27c03137093701ade97ead40912b7b5b4d, installed 17 November 2026 from the local copy of the PyPI filesystem1-1.2.0-py3-none-any.whl - System 1 1.2.0 release notes, read 17 November 2026: qualification follows a passing development assessment; attempt k spends
(1 - confidence) / (2 * k * (k + 1))on each bound; a small perfect cohort is not sufficient; failed attempts consume the cohort - Correct, compare, and adopt a skill, read 17 November 2026: three splits,
require_qualification=True, disjoint normalized text, software does not detect paraphrases TeachingSession.assess,qualify,adopt,decision_details, androllbackon system1 1.2.0, Python 3.11.15, Linux x86_64, 17 November 2026, against the authored invoice/crash lessons described above_binomial_lower_boundinsystem1.compat.typesafefrom that same install, called for n of 24 at failure probability 0.025 and for n of 80, 85, 86, and 90 at failure probability 0.0125