A checkout that grows a second toolchain still has one smoke test. On this Hermes binary, hermes verify picks that test from the files at the project root, then runs the strings it picked. The pick is not the language you think the project is. It is a kind, taken from the first matching manifest, plus a test list that can also name commands from files the kind ignored.
This run measured that split on disposable directories, then pinned one test command in .hermes/environment.json and ran it. The pinned command is what the phase line named. The extra commands from the other files were not on that list.
What this run used
The binary was Hermes Agent v0.22.5+471.g5894e75, local commit 5894e750c3db, on Linux x86_64, kernel 7.0.14-6-pve, on 11 November 2026. hermes --version also printed Python 3.14.9 for the agent runtime. The unittest process was Python 3.13.5. HERMES_HOME pointed at a new directory. No login was run. No prompt was sent. The operator Hermes home’s verification database timestamp was the same after the run as before it.
The published command table, read the same day, does not list hermes verify. The checkout’s Markdown, at commit 5894e750c3dba980d43b25aa29e3958149ba2c61, does not either. The command is in hermes --help and in hermes verify --help. The help text says it detects how the project is built, tested, and started, or loads .hermes/environment.json, then runs bootstrap, build, test, and a start-and-readiness pass. It does not say which file wins, or that a detected test list can name a command from a file the kind did not choose. Those two facts are what the fixtures measured.
Ask for the recipe before you run it
--detect-only prints the recipe and runs nothing. Add --json when a script will read it. The object has source and recipe. source is detected or manifest. recipe.kind is the toolchain label. recipe.test is the list of shell strings the test phase will run, in order. recipe.bootstrap and recipe.build are the other command lists. start is one string or null.
Point the command at the project root:
hermes verify --detect-only --json .
On a directory whose only project file was a pyproject.toml with a [project] table and a tests/ directory, that object said source was detected, kind was python, and test was ["pytest"]. There was no [tool.pytest] table and no pytest.ini. The tests/ directory was enough for this detector to name pytest. The same pyproject.toml with no tests/ directory named python -m unittest discover instead. --save on that second tree wrote the recipe to .hermes/environment.json with "version": 1 and an updatedAt timestamp. The printed object still said source was detected, because --detect-only reports the recipe it just detected, including when it also saved a copy.
An empty package.json, the two-character file {}, changed the first tree. kind became node. bootstrap became ["npm install"]. test became []. The tests/ directory was still there. It did not add pytest. The Node recipe does not look at tests/, and the extra-command pass adds pytest only when pytest.ini exists or the text [tool.pytest appears in pyproject.toml. A job that sees package.json and assumes the Python tests still run will skip them.
The object has to parse as a JSON object. A file containing [], and a file containing a single {, both fell through to the Python recipe on trees that also had pyproject.toml. A corrupt .hermes/environment.json did the same in the other direction. The manifest was ignored, and {} in package.json selected node.
The kind and the test list are different decisions
The kind follows one file. On this binary the order is package.json when it parses as an object, then Python (pyproject.toml, requirements.txt, manage.py, or setup.py), then go.mod, then Cargo.toml, then Maven or Gradle, then a Makefile, then a Compose file. The first match sets kind, name, and the commands that detector knows about.
A second pass, used only when source is detected, appends verify commands the first pass did not already list. It looks for scripts/run_tests.sh, for the script names test, tests, lint, typecheck, check, build, fmt, and format in package.json, for pytest when a pytest config is present, and for those same names as Makefile targets. The appended strings go on recipe.test, even when the target was build.
That is why a tree with all four of those files did not have a test list of one command. package.json had a test script of node -e "process.exit(0)". pyproject.toml contained [tool.pytest.ini_options]. The Makefile had a build target. scripts/run_tests.sh was executable and exited 0. --detect-only --json said source was detected, kind was node, bootstrap was ["npm install"], and test was:
["npm run test", "scripts/run_tests.sh", "pytest", "make build"]
kind was node. The test list also named a shell script, pytest, and make build. The evidence line said Detected package.json and Scripts: test. It did not mention the Makefile or the pytest table. The evidence list is not the test list.
--phase test --skip-start then ran that list and stopped at the first failure. npm run test passed in 0.4 seconds. scripts/run_tests.sh passed. pytest failed in 0.0 seconds with /bin/sh: 1: pytest: not found. make build was on the detected list and was not reached. The process exited 1. The human report said Result: FAILED and printed the pytest tail. A script that only checks whether npm is installed will miss the failure. A script that only reads kind will not know pytest was going to be executed. The /bin/sh prefix is the evidence that each string is a shell command, not an argument vector. A space inside one array element is part of that command.
Two lockfiles change the spelling, not the split. With both pnpm-lock.yaml and package-lock.json present, and a test script in package.json, the detect-only test list was ["pnpm test", "pnpm run test"]. The lockfile order prefers pnpm, so the recipe’s own runner is pnpm test. The extra-command pass adds pnpm run test because that exact string is not already in the list. This run did not execute that pair. On the tree that did run, the test list was executed in order until the first non-zero status. Read the list before you assume one spelling.
A go.mod beside a Makefile with a test target kept kind as go and still appended make test. The detect-only test list was ["go test ./...", "make test"]. start was null because main.go was absent. The Go detector does not invent a start command for a module with no main.go. The Makefile target still joins the test list. That pair was not executed. go was not required for the detection result.
A comment can select the framework
Python detection reads pyproject.toml and requirements.txt as lowercase text. It does not parse the dependency table before it searches for framework names. Django is checked first, then FastAPI or uvicorn, then Flask.
A pyproject.toml whose only use of the word django was a comment, # note: this is not a django project, and which had no manage.py, was detected as django. The test command was python manage.py test. The start command was python manage.py runserver 0.0.0.0:8000. --phase test exited 1. The tail named the missing manage.py under the project directory. The comment was enough. The missing file is what the phase reported.
The same shape with dependencies = ["flask"] and no uvicorn word was detected as flask, with start flask --app app.py run --host 0.0.0.0 --port 5000, even though app.py was not in the directory. Add the comment # do not add uvicorn and the kind flipped to fastapi, with start uvicorn main:app --host 0.0.0.0 --port 8000 and an empty test list, because that tree had no tests/ directory. The dependency line still said flask. The comment won, because uvicorn is checked before flask, and the search is a substring of the file text. Those two trees were detect-only. Nothing was started.
If you are about to let verify start the process, read start in the detect-only object before you omit --skip-start. This run set --skip-start on every execution. A full run of the django recipe would also have tried pip install -e . in the bootstrap phase, which this run did not execute. The detected bootstrap string is a command, not a suggestion. Leave it in the recipe and omit --phase, and the next run will try to install.
The Makefile colon is narrower than the merge
A Makefile whose targets were test:deps and check: was detected as make. The evidence line said Targets: check. The recipe’s own test command was make check. The extra-command pass still appended make test, because its target pattern allows zero spaces after the colon, and test:deps matches that pattern. The detect-only test list was ["make check", "make test"]. That Makefile was not executed.
A target written test: deps, with a space, is visible to both passes. The recipe evidence then names test, and the test list is ["make test"] with no second copy, because the string is already present.
Write the target with a space if you want the make recipe itself to see it. Do not assume a tight colon hides the target from the test phase. The merge still sees it, and it appends make test onto whatever kind won.
Pin the command list
The file that stops the merge is .hermes/environment.json. --save writes the detected recipe into it. After that, edit recipe.test down to the commands you intend to run. A saved recipe is used as written. The extra-command pass does not run when source is manifest.
The wrapper this run loaded had version 1, name pinned, kind python, one bootstrap string, an empty build list, and one test string:
{
"version": 1,
"recipe": {
"name": "pinned",
"kind": "python",
"bootstrap": ["python -c \"print(123)\""],
"build": [],
"test": ["python -m unittest discover -s tests -v"],
"start": null,
"port": null,
"readinessPath": "/"
}
}
The same directory still held package.json, the pytest table, the Makefile build target, and scripts/run_tests.sh. --detect-only --json said source was manifest and test was exactly that one unittest command. None of the other four strings were appended.
Two shapes are accepted. The wrapped object above is what --save writes. A bare recipe object, with name at the top, also loads. version is stored and not checked. A file with "version": 99 and a recipe name still reported source manifest. A file at the manifest path is not itself the win. The recipe needs a non-empty name. A versioned object whose recipe had kind and no name was ignored, and detection selected the Python project underneath it.
A corrupt manifest is also ignored. One { in .hermes/environment.json, beside a package.json of {}, selected node. If you pin a recipe, check that detect-only says manifest before you trust the file you wrote. --save does not switch the printed source on that same invocation. Run detect-only again after the file is on disk if you need the loaded source, not the recipe that was just written.
Run the phase you mean
--phase may be repeated. --skip-start skips the readiness poll. This is the check for a test command that should not boot a server:
hermes verify --skip-start --phase test .
On the pinned tree, that command exited 0. The JSON result said ok true, source manifest, and one phase. The phase was test, the command was python -m unittest discover -s tests -v, the exit code was 0, and the duration was 0.042 seconds. The tail was one test, test_ok, and OK. The human report names the same command on the phase line. That line is the check. Result: OK means every phase that ran had exit code 0, and that a readiness poll, if one ran, was ready.
The same manifest against a test that asserted 1 == 2 exited 1. The phase line said FAIL and named the unittest command. The tail included AssertionError: 1 != 2 and FAILED (failures=1). The manifest did not hide a failing test. It limited the run to the command it named.
An empty test array is a different result, and it is easy to misread if you only store the exit code. A manifest named empty, kind python, test set to [], with --phase test --skip-start, printed:
Recipe: empty (python) — source: manifest
(no phases executed)
Result: OK
The process exited 0. No command ran. Result: OK here means the selected phase had nothing to execute, and the start phase was skipped. Require a phase line that names the command you wrote. If the report says no phases executed, the manifest’s list for that phase is empty, or you asked for a phase the recipe does not have.
--timeout defaults to 600 seconds per phase. A hung install will sit there. This run did not execute bootstrap. If you leave the detected pip install -e . or npm install in the manifest and omit --phase, the next full run will try to install into the environment that invokes Hermes. Edit those lists before you drop --phase. --port overrides the port used for the readiness poll. It does not rewrite a start command that already contains a different port. This run did not poll readiness, so that override was not exercised.
What to require before a job treats the tree as checked
Read the detect-only object first. Require source to be manifest once you have pinned a recipe. Require recipe.test to be the commands you intend, with no extras you did not write. Then run --skip-start --phase test unless you have also reviewed start and you mean to boot it.
Accept the run only when the exit code is 0 and a phase line names each required command with PASS. Reject Result: OK when the body says no phases executed. Reject a detected recipe whose kind is node or django if the project you meant to test is the Python package the comment or the empty package.json hid. The phase tail separates those cases. This run produced a missing manage.py, a missing pytest, and an assertion failure, and each one named the command that failed.
The published command table still does not list the command. hermes verify --help does. The help text is the contract for the flags. The detect-only object is the contract for which strings will run.
References
- Hermes Agent v0.22.5+471.g5894e75, local commit
5894e750c3db, Linux x86_64, kernel 7.0.14-6-pve, 11 November 2026.hermes verify --helpdescribes--detect-only,--save,--skip-start,--phase, and.hermes/environment.json.HERMES_HOMEpointed at a new directory. No login was run. The operator verification database timestamp was unchanged. The unittest interpreter was Python 3.13.5. - Hermes Agent CLI commands reference, https://hermes-agent.nousresearch.com/docs/reference/cli-commands, read 11 November 2026. The command table does not list
hermes verify. - Checkout files
agent/verify/recipes.py,agent/verify/environment.py,agent/verify/runner.py, andhermes_cli/verify_cmd.pyat5894e750c3dba980d43b25aa29e3958149ba2c61. Detection order, the substring framework check, the Makefile colon rule, the detected-only extra-command merge, and the shell execution of recipe strings are in those files. The fixtures above are what this binary did with them.