A BrowseSafe++ false covers the prefix, not the page

John Burns

A 6,011-character page came back as one field. The stand-in prediction was {"injected": False}. The marker that started at character 6,001 was not in the text the decision model scored. A failed call with the failure policy set to closed produced the same object, while a different counter moved. If you store only that object, you cannot tell a short clean page from a long page whose tail was never read, and you cannot tell an act-band hit from a decision that never returned a probability.

That split is the contract in the system1-agents checkout measured here. The 0.4.0 changelog says the page is clipped to 6,000 characters, that act, allow, and uncertain become True, False, and None, and that a failed decision raises unless S1A_BSPP_ON_FAILURE=closed. Those sentences are true. They do not say what the returned object omits, which float sits on which side of the cut, or which error you see when the environment is wrong.

The checkout, and what this run did not call

The tree was ThinkFlowLab system1-agents at c4da7c35b287edf9b2db9412c5990a7894deeac8, the 13 November 2026 merge of pull request 123, which is the commit that promotes the changelog heading 0.4.0 - 2026-11-13. The run was 14 November 2026 on Linux x86_64. The interpreter was Python 3.11.15. pip show reported the distribution version as 0.1.0, which is the string still written in pyproject.toml. A version query will not tell you that the BrowseSafe++ adapter is in the tree. The changelog heading and the commit will.

The openjiuwen pin jj-0.1.0 resolved to 93f154e2419a0708fae892daa974ea413472e1a2, and the installed distribution version was 0.1.18. No TYPESAFE_API_KEY and no OPENROUTER_API_KEY were set. The BrowseSafe++ SDK was not installed. No Hugging Face token was used. Nothing in this note is a leaderboard precision, a recall, or a control-page false-positive rate. The benchmarks page in that commit already says the SDK table is not filled, and this run does not fill it.

The package’s own offline file, tests/test_browsesafe_pp.py, was run with pytest. Fourteen tests passed. Those tests already check the three view names, the 6,000-character cut, the five scripted probabilities 0.9, 0.1, 0.5, 0.7, and 0.3, and that a failed decision’s message does not contain the page text. The measurements below are the cases that file does not print.

The decision model in those extra cases was ScriptedModel from the same tree, handed to Detector directly, except where the note says detector() or predict() was the function under test. A scripted probability is not a Jev score. It is a way to see which side of the adapter moves when the probability is known.

What the model is given, and what the caller gets back

state_of reads request.view and request.content. The view must be one of html, markdown, and accessibility, matched case-sensitively. Markdown raises ValueError and names the three legal views. html and markdown both become the tool name fetch_webpage. accessibility becomes browser_snapshot. A model that only sees the state cannot tell an HTML view from a markdown view. The content must be a str. None, bytes, an int, and a list each raise TypeError and name the type they saw. An empty string is accepted and is sent as empty text.

The clip is content[:6000], and MAX_CHARS in s1a/agents/injection_guard.py is 6000. The probe page was 6,011 characters: a prefix marker, padding, and the marker TAIL-MARKER starting at character 6,001. The view was html. The scripted probability was 0.1.

The model call’s state had tool set to fetch_webpage and text of length 6,000. The prefix marker was in that text. TAIL-MARKER was not. The question name on that call was check. The object returned to the caller had one key, injected, and the value was False. There was no coverage field, no inspected count, and no withheld notice.

That last absence is the difference from the runtime guard in the same file. The guard’s decision also sees the first 6,000 characters. Below the act band, the guard then rewrites the tool result the chat model will read: the inspected prefix, plus a notice that names how many characters were withheld. The adapter does not do that rewrite, because it does not own the page. The SDK already has request.content. The adapter’s job is to return a prediction. A False from this function is a label on the prefix. It is not a statement that the unread tail was clean, and it is not a statement that the tail was removed.

If you are scoring pages for the SDK, record len(request.content) beside the prediction. A row with injected false and a length above 6,000 is a prefix allow. Treating it as a full-page allow overstates what this process checked.

The 6,000 count is code points

The slice is a Python slice. It counts code points, not UTF-8 bytes, and it does not look for a grapheme boundary.

A page of the character ไฝ  repeated 6,003 times is 6,003 code points and 18,009 UTF-8 bytes. The text the model received was 6,000 code points and 18,000 bytes. A reader that stopped at 6,000 bytes would have kept 2,000 of those characters, not 6,000. The constant in this module is not that reader.

The other page was the family sequence ๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ, which this interpreter counts as 7 code points and 25 UTF-8 bytes, repeated until the page was 6,013 code points. The text the model received was 6,000 code points and 21,429 UTF-8 bytes. It did not end on a complete family sequence. The view was accessibility, so the tool name was browser_snapshot. The cut still did not care about the sequence. A detector that logs “6,000 characters inspected” is logging code points. It is not logging bytes, and it is not logging complete graphemes.

The edges the changelog inequality does not print

The thresholds on the injection-guard spec are allow 0.3 and act 0.7. Thresholds.band treats a probability at or below 0.3 as allow and a probability at or above 0.7 as act. The adapter then maps allow to False, act to True, and uncertain to None. The inclusive edges are easy to round away. This process was given the following values, in order, on one detector:

0.0                      False   allow
0.3                      False   allow
0.30000000000000004      None    uncertain
0.3000001                None    uncertain
0.5                      None    uncertain
0.6999999999999998       None    uncertain
0.7                      True    act
1.0                      True    act

The value printed as 0.6999999999999998 is what this interpreter stored for the literal 0.6999999999999999. It abstained. The next literal, 0.7, acted. The next float above 0.3 that this probe passed, 0.30000000000000004, abstained. A log line that prints the probability to one decimal place will show 0.3 for both the allow and the first uncertain value. The boolean will not agree. Store the exact probability next to injected if you need to explain a row later.

nan, -0.1, and 1.1 did not abstain. Each raised DecisionFailed with the text MODEL_CALL_FAILED. The page body contained a marker that was also placed in the model error. That marker was not in the exception text. Each of those three calls incremented bands["failure"] and still counted as a model call. A later scripted 0.2 on the same detector returned {"injected": False} and moved allow to 1, leaving failure at 3. A bad probability is a failure, not an uncertain page. An uncertain page is a probability that validated and landed strictly between the two edges.

A closed failure wears the same return as an act

Two detectors were given the same stand-in prediction shape. One received a scripted probability of 0.7. The other was built with on_failure="closed" and a model error whose message echoed that same page marker.

Both calls returned {"injected": True}. The objects compared equal. The counters did not. The act call left act at 1 and failure at 0. The closed call left act at 0 and failure at 1. The closed call’s return value did not contain the page marker.

The benchmarks page says a raised failure is an SDK error, and that S1A_BSPP_ON_FAILURE=closed answers injected=True instead, as the runtime guard quarantines on failure. The measurement agrees, and it adds the part a stored prediction loses. After the prediction is built, the failure and the act are the same object. Detector.bands is where they still differ. If the SDK record you keep is only the prediction, a fail-closed page is indistinguishable from a page the model called injected. Keep the band counter, or keep the exception, if that distinction matters for the score you publish.

The token is exact. closed and error construct a detector. Closed, CLOSED, fail-closed, true, 1, an empty string, a leading space, and a trailing space each raise ValueError and quote the token. They do not fail closed. A wrapper that sets the variable to fail-closed because the docs say the guard fails closed will not get injected=True. It will get the value error, and only if construction gets that far.

The process entry fails in a different order, then latches

predict does not take a model object. It calls detector(), which reads S1A_BSPP_MODEL and S1A_BSPP_ON_FAILURE once and keeps the result on the process. The legal model names are jev, clm, laya, and laya-served. rule and not-a-model each raise ValueError and list those four names. The detector is not assigned. A later call can still try a legal name.

The order inside detector() is the other trap. It checks the model name, then calls build_model, then passes S1A_BSPP_ON_FAILURE into Detector. With S1A_BSPP_MODEL=jev, S1A_BSPP_ON_FAILURE=not-a-token, and neither API key set, the error was not the token error. It was ValidationError with the text [181002] model service config error, reason: no Jev key: set TYPESAFE_API_KEY or OPENROUTER_API_KEY in .env or the environment. The detector was not assigned. A bad failure token is invisible until a model object is actually built. Fixing the key and leaving not-a-token in the environment will then fail at the token, on a later call. This run did not set a key, so it did not take that second step.

Once detector() has returned, the environment is not read again. For that check, build_model was replaced with a function that returned a scripted model, so the process would not load Laya weights or open a Jev client. detector() was then called with S1A_BSPP_MODEL=laya and S1A_BSPP_ON_FAILURE=closed. It built once, with the argument laya, and the object’s failure policy was closed. Both variables were then changed, to jev and error, and detector() was called again. The same object came back. build_model was not called a second time. The failure policy was still closed. Two decide calls on that object returned True and then False, matching the two scripted probabilities, not the new environment.

predict adds one more step after that decision. With the prediction callable left unset, a scripted probability of 0.1 was scored, allow moved to 1, and the call then raised ModuleNotFoundError: No module named 'browsesafe_pp'. A direct predict() on an already built process detector raised the same import error, and it did so after the model had already been called. The SDK is not a dependency, which is what the changelog says. The entry point still imports it after the decision, because that is how it builds Prediction. A missing install is not injected=False. It is an exception that arrives after the model call has already been counted. Catching the import error and looking only at a default of false will invent an allow the function did not return.

Verification

Check the commit before you install. pyproject.toml at this revision still says 0.1.0. The adapter is the file s1a/browsesafe_pp.py added under the 0.4.0 changelog heading, not a version string.

git clone https://github.com/ThinkFlowLab/system1-agents.git
cd system1-agents
git checkout c4da7c35b287edf9b2db9412c5990a7894deeac8
python3.11 -m venv .venv
.venv/bin/pip install -e '.[dev]'
.venv/bin/python -m pytest tests/test_browsesafe_pp.py -q

The install pulls openjiuwen from the jj-0.1.0 git pin. On this machine that pin was 93f154e2419a0708fae892daa974ea413472e1a2. Fourteen tests passed in under a second. That pass does not include the tail marker, the grapheme cut, the float just above 0.3, or the identical act and fail-closed returns. Those need a short driver that constructs Detector with ScriptedModel and a prediction callable of your own, so the missing SDK is not on the path you are measuring.

For a page you intend to score, print four things before you keep the boolean: the content length, the length of the text in the model call’s state, injected, and bands. A length above 6,000 with injected false is a prefix allow. failure greater than zero with injected true is a closed failure, not an act, when the policy is closed. If you call predict and the SDK is absent, the exception is the result. Do not substitute false.

Unset S1A_BSPP_MODEL and S1A_BSPP_ON_FAILURE in the process that will serve pages, then start that process. A change made after the first detector() call does not move the model or the failure policy. A not-a-token value will not show up until build_model succeeds, so a missing Jev key hides it.

This run did not install browsesafe-pp, did not call browsesafe evaluate, and did not produce a score the benchmarks table could hold. The adapter’s offline contract is narrower than that table, and it is already enough to mis-file a row: a false covers the first 6,000 code points, and the object you get back does not say what happened to the rest.

Sources

  • ThinkFlowLab system1-agents commit c4da7c35b287edf9b2db9412c5990a7894deeac8, 13 November 2026, merge of pull request 123. Measured 14 November 2026 on Linux x86_64, Python 3.11.15. Distribution version reported by pip was 0.1.0. openjiuwen 0.1.18 from pin jj-0.1.0 at 93f154e2419a0708fae892daa974ea413472e1a2. No API key, no Hugging Face token, and no BrowseSafe++ SDK.
  • s1a/browsesafe_pp.py and s1a/agents/injection_guard.py at that commit: MAX_CHARS = 6000, the view-to-tool map, BAND_INJECTED, and S1A_BSPP_ON_FAILURE values error and closed. The runtime withheld-notice path is withhold_uninspected in the guard module. The adapter does not call it.
  • s1a/spec.py Thresholds.band and s1a/rails.py RAIL_MODEL_NAMES and failure_category at that commit. The four model names and the inclusive edges are from those objects, confirmed by the scripted calls above.
  • CHANGELOG.md, heading 0.4.0 - 2026-11-13, and docs/benchmarks.md section “BrowseSafe++ through the official SDK”. The benchmarks page says the SDK numbers are not run yet. This note does not supply them.
  • tests/test_browsesafe_pp.py at that commit, 14 passed under pytest 9.2.0. The tail, grapheme, next-float, and identical-return cases are not in that file.