restart came back with a printed probability of 1.0. The conformal set still contained restart, wait, and escalate. session.adopt() then raised ValueError: Candidate did not pass its adoption checks, and session.predict() raised ValueError: Assess and adopt a skill before predicting. The candidate file was on disk. It was not a skill the session would serve.
That is the check. A System 1 teaching session can print a near-certain label and still withhold every row. If you restart a process because the raw guess looked sure, you skipped the gate the 1.2.0 session actually enforces.
The decision this session was taught
The decision was a restart gate with three actions: restart, wait, and escalate. It is not a support-ticket router. The notes were written for this lab. They are not pages from a live on-call rotation.
The authored rule was small. A note that only describes one wedged, unresponsive, stuck, or not-listening process is restart. A note that only describes a deploy, migration, rollout, or closed change window is wait. A note that mentions a checksum mismatch, split brain, lost quorum, data corruption, or an unknown error class is escalate. A note that mixes two of those families is also escalate, even when one family is a simple restart or a simple wait. A note that matches none of them has no action.
System 1 1.2.0 was installed from the PyPI wheel system1-1.2.0-py3-none-any.whl. The SHA-256 was 57bcf15c6028a1b440e983c59f60ec27c03137093701ade97ead40912b7b5b4d, which matches the digest published for that file. The interpreter was Python 3.11.15 on Linux x86_64. No teacher API key was set. The fitting method was the session default, ridge. The schema was one ChoiceField, and ambiguity escalation was left at the schema default. The teaching helper rejects a field that disables that escalation, because the session is built to review uncertain answers rather than hide them.
from system1 import ChoiceField, DecisionSchema, TeachingSession
class RestartGate(DecisionSchema):
action = ChoiceField(options=["restart", "wait", "escalate"])
session = TeachingSession(".system1/restart-session", RestartGate)
Create the directory only once. Reopening TeachingSession on an existing directory loads the saved lessons. A second record_many of the same text replaces the label in that split. It does not give you a clean second experiment. Use a new directory when you want a new candidate.
Three splits, and a candidate that is not serving
The correcting-skills guide separates the rows on purpose. teach fits the head. calibrate is held out of that fit so review behavior is not tuned on the same sentences. evaluate is the comparison set. All three have to be present before assess() will run. This run used 24 teach rows, 9 calibrate rows, and 16 evaluate rows, eight lexical paraphrases of each action in teaching and three of each in calibration. The evaluation set kept four clear paraphrases of each action and added four mixed lines, all labeled escalate under the rule above.
assess() compiles a candidate, writes it beside the session, and compares it with the adopted skill if one exists. It does not install that candidate. Before the first adoption, session.engine is None. The default demonstration policy, which this run did not override, requires raw accuracy of at least 0.8, coverage of at least 0.5, no accepted mistakes, and no regressions. Passing those checks is still not a production qualification. Lowering a threshold after you have seen the report is how a failed candidate gets a green adopt. This run left the defaults alone.
The first assess() took 0.0192 seconds. It did not pass.
Raw accuracy is below the required minimum
Local acceptance coverage is below the required minimum
The candidate summary was 12 of 16 raw-correct, raw accuracy 0.75, accepted 0, coverage 0.0, review rate 1.0. adopt() refused with the value error above. There was no incumbent, so there was also nothing to roll back to. Qualification was never called. In 1.2.0, qualify() is available only after a passing development assessment. A four-row fresh file would not have been reached, and it would not have repaired a candidate that had already failed coverage.
The four lines that were raw-wrong
Every clear evaluation line was raw-correct and still reviewed. The conformal set on each of them was the full action list. The four raw errors were the mixed lines, and the raw guess followed one lexical pole rather than the authored escalate rule:
health check failed during an active deploy
label escalate, predicted restart, probability 0.860704
process is wedged and the volume checksum mismatched
label escalate, predicted restart, probability 0.999648
not listening while the migration is running
label escalate, predicted wait, probability 0.999976
split brain and the change window is closed
label escalate, predicted wait, probability 0.954202
Review saved those rows from becoming accepted errors. accepted_errors stayed 0 because nothing was accepted, not because the raw guess was safe. The diagnostic text on the report said the candidate was choosing wrong labels before review gating, and that more calibration alone does not repair the learned choices. The wider session below agreed with that sentence.
The probabilities are the values printed to six decimal places from the candidate file of a clean session built with the same lessons. A probability of 1.0 in that print is the rounded figure for the clear restart and wait lines, not a claim that the head stores an integer one. The point is the gap between that figure and the set. On the service process is wedged and not listening, the printed probabilities were restart 1.0, wait 0.0, escalate 0.0, and the set was still all three actions. is_ambiguous was true. The session treats a set that is not a single action as review. A caller that reads only values["action"] will restart the service. A caller that reads is_ambiguous, or that refuses to act unless adopt() has succeeded, will not.
The unrelated line the coffee machine is beeping in the kitchen is the same trap in the other direction. The raw guess was escalate at 0.980278. The set was still all three actions, so the row was reviewed. There is no escalate family in that sentence. A high probability on an out-of-vocabulary note is not evidence that the note belongs to the winning class. decision_details() can name unknown_vocabulary after a skill has been adopted. This session never adopted, so that helper was not available through session.predict(). Loading the candidate file directly is how the probabilities above were read. That load is an inspection. It is not the serving path.
A perfect raw score still failed coverage
The mixed lines made the first report fail two checks at once. A second session dropped those four lines from evaluation and kept the same 24 teach rows and 9 calibrate rows. Evaluation was then 12 clear paraphrases, four of each action.
That candidate was raw-correct on all 12. Raw accuracy was 1.0. Accepted was still 0. Coverage was still 0.0. The only refusal reason was coverage. adopt() raised the same value error. Fixing the raw-accuracy miss does not create an action the session will serve. The calibration count on that candidate was 9 rows, of which the compiler recorded 5 as temperature samples. Those counts are sample counts in sample_counts. They are not a temperature of 5. With a conformal slice that small, a set that contains every action is the result this helper produced, including on rows whose printed probability was 1.0.
A third session put 12 more authored paraphrases into calibration, 21 calibration rows in total, and put the mixed lines back into evaluation. The compiler recorded 11 temperature samples. Coverage was still 0.0. The same four mixed lines were still the raw errors, with the same predicted labels. Adding near-duplicate calibration text did not shrink a set to one action, and it did not teach the two-family escalate rule. The guide’s warning held on this data: calibration volume is not a repair for a head that has learned the wrong distinction.
What a keyword rule and a chat planner did with the same notes
Two other deciders saw the same seven notes: four mixed lines, two clear lines, and the coffee-machine line. Neither of them is System 1. The keyword rule was a local function with the same three families. One matching family returned that action. Two matching families returned escalate. No match returned review. The chat planner was Grok Build 1.0.67, model grok-4.6-build, one headless turn, --permission-mode plan, web search disabled, subagents disabled, and a JSON schema that required an action per note. --sandbox read-only did not start. The process exited 1 with bwrap exec failed: No such file or directory and refused to run while a deny list could not be enforced. The planner result below is the plan-mode turn, not a sandboxed one.
On the two clear lines, all three agreed: wedged and not listening was restart, and a deploy still applying was wait. On the coffee-machine line, the keyword rule and the structured planner answer were review. The System 1 raw guess was escalate.
The mixed lines are where the deciders split:
health check failed during an active deploy
authored escalate
system1 raw restart
keyword wait
grok structured answer review
process is wedged and the volume checksum mismatched
authored escalate
system1 raw restart
keyword escalate
grok structured answer escalate
not listening while the migration is running
authored escalate
system1 raw wait
keyword escalate
grok structured answer escalate
split brain and the change window is closed
authored escalate
system1 raw wait
keyword escalate
grok structured answer escalate
The keyword rule matched the authored escalate label on three of the four mixed lines because it treats two keyword families as escalate. It missed health check failed during an active deploy, because “health check failed” was not in the restart list and “deploy” was. It would have acted. System 1 reviewed that row instead, and its raw guess was the third label, restart.
The planner’s structured answer matched the authored escalate label on the three lines that name checksum, migration, or split brain beside a second family. On the health-check-during-deploy line it returned review, not escalate. The same turn’s reasoning trace had already called that note escalate, then settled on review before the schema object was emitted. A caller that stores only structuredOutput will see review. A caller that stores the trace will see both. Neither field is the System 1 raw guess. If you use a chat model as the teacher, record the field you will actually label from. This run did not feed the planner’s answers back into record(). The comparison is one pass over seven notes, not a teaching loop.
The planner turn reported 13,853 input tokens and 3,421 output tokens, 3,348 of them reasoning tokens. The System 1 assess() that compiled and scored 16 rows took 0.0192 seconds and did not call that model. Those are different objects. The fast local score is not an accepted action, and the planner’s structured label is not a conformal set.
A second assess replaces the candidate file
After the first report was saved, the teach line api process is wedged and health check failed was recorded again with the label wait. That is a wrong correction under the authored rule. assess() ran again. It still failed, this time on coverage only, and adopt() still raised ValueError: Candidate did not pass its adoption checks.
The candidate file in that session directory was no longer the first candidate. On the first report, health check failed during an active deploy was raw restart. On the second report, the same line was raw wait. Both were reviewed. Both were wrong relative to escalate. If you inspect candidate.s1m after the second assess and quote it as the first result, you will publish the wrong probability. Copy the assessment JSON before you record another lesson. The file on disk is the latest candidate, and a failed candidate is still a candidate. It is not an adopted skill, and it is not a stable name for the report you just read.
Two sessions built from the same lesson text also produced different candidate digests, 4d8596e40a1fb099e3fe928e7e653630a7f872112d194daadc3be20edd33c70c and 31c51d965009d3b728b9d6acda619672528f407b8f7ae8484411c9b958df89bf, while the session digest stayed 5a12f42773a4ade1c9c6057e88c4bc495811ac1a0329535afd65b0de76e0f57d. This run did not isolate why the candidate digest moved. Do not treat that digest as a pure function of the sentences. Judge the saved report.
session.history stayed empty. Nothing was approved, so rollback() had nothing to restore. That is the expected state when every assess fails. It is not evidence that a later passing candidate will keep an earlier file. The 1.2.0 guide says adoption rechecks the exact passing candidate, and a later edit makes that approval stale. This lab never reached that recheck.
Test the refusal before you wire the action
Install the pinned wheel into a virtualenv, confirm the digest, and point the session at a directory you can delete.
python3 -m venv .venv
.venv/bin/python -m pip install system1==1.2.0
Record teach, calibrate, and evaluate rows with an explicit split. Keep mixed lines out of teaching if you want the evaluation to be the first time the head sees two families at once. That is what this run did. Then assess, and attempt adopt only so you can see the refusal. Do not catch the value error and continue into a restart.
report = session.assess()
print(report["passed"], report["reasons"])
print(report["candidate"]["raw_accuracy"], report["candidate"]["coverage"])
session.adopt()
On this lesson set the print is False, the two reasons above, raw accuracy 0.75, and coverage 0.0. The clear-only session prints raw accuracy 1.0 and coverage 0.0, with coverage as the only reason. Either result means the same operational thing: do not call the action. session.predict() will refuse until an assess has passed and adopt() has returned. If you need the probability for a write-up, load the candidate file in a separate inspection and label that output as inspection. Do not point the process supervisor at that load.
When a later lesson set does pass, the 1.2.0 path is still not “adopt because coverage cleared 0.5.” The release notes require a fresh qualification cohort before a qualified adoption, and a small perfect cohort does not establish the 0.95 accepted-correctness bound. This lab cannot show that refusal, because development assessment never passed. Keep that gate in the procedure anyway. A coverage pass on paraphrases of your own sentences is the development check. It is not the independent cohort.
Save assessment.json, or the JSON you printed, next to the candidate digest you inspected. After any record() and second assess(), read the new report. The previous probabilities are in the file you saved, not in candidate.s1m.
What was refused
System 1 1.2.0 compiled a restart gate from 24 authored lessons, reviewed every evaluation row, and refused adoption. A printed probability of 1.0 did not shrink the conformal set. Dropping the four raw errors raised raw accuracy to 1.0 and left coverage at 0. Twelve extra calibration paraphrases did not create an accepted row. A keyword rule would have acted on the mixed lines. A Grok Build 1.0.67 plan-mode turn named a different action from the local raw guess on the line both of them found ambiguous. The session served nothing, which is the result the adopt error is for.
Sources
- System 1 1.2.0 wheel, SHA-256
57bcf15c6028a1b440e983c59f60ec27c03137093701ade97ead40912b7b5b4d, downloaded 30 October 2026 from the PyPI file URL forsystem1-1.2.0-py3-none-any.whl - Correct, compare, and adopt a skill, read 30 October 2026: three splits, default thresholds,
assess()does not install,predict()requires adoption, calibration does not repair wrong labels - System 1 1.2.0 release notes, read 30 October 2026: qualification follows a passing development assessment; a small perfect cohort is not sufficient evidence; failed candidates do not enter approved history
TeachingSession.assessandTeachingSession.adopton system1 1.2.0, Python 3.11.15, Linux x86_64, 30 October 2026, against the authored restart-gate lessons described above- Grok Build 1.0.67
grok --version, one--permission-mode planturn on modelgrok-4.6-build, 30 October 2026.--sandbox read-onlyexited 1 before a turn because bubblewrap was not available