Checks

What a check is, its fields (id, executable, args, cwd, timeout, shot), how it runs and what environment it gets, and which test output smith parses.

A check is a command smith runs in a run's workspace; a non-zero exit fails it. A project's tracker record and harness.json/template nodes describe checks in two shapes. Checks run directly, not through a shell, in a sterile six-key environment.

01record vs harness#

• A project's record check (worksource.ProjectCheck) takes exactly id, executable, args, timeout.
• A record check with cwd or shot is refused, because proposeRecord decodes with DisallowUnknownFields.
• commandChecks copies only ID, Executable, Args and Timeout from the record. So a record check always runs in the run's workspace root.
• harness.json checks and a template's check nodes use harness.CommandCheck: id, executable, args, cwd, timeout, shot. It also has area and paths, used by area checks.

02id#

Required, unique within the set (validateRecord; decodeCommandChecks trims it). It names the check in gates, gate summaries and send-backs.

03executable#

Required: a program name or path. It is run directly, not through a shell: no pipes, no &&, no globbing, no $VAR or %VAR% expansion. Put each argument in args as its own string.

Resolved with Go's exec.LookPath against the PATH of the smith core process (the check environment passes that same PATH on).

• Linux: the first executable file named so on PATH; a name with a slash is used as a path.
• Windows: exec.LookPath tries each PATHEXT extension (.exe, .cmd, ...) and returns the first match on PATH. The WindowsApps python.exe store alias is found if it is on PATH and may not be a real interpreter. See windows#executables.

04args#

List of strings, literal, passed as argv.

05cwd#

Harness shape only. Relative to the handed-off workspace root (the run's own copy of the clone); default .. An absolute path is refused ("must be relative to the handed-off workspace"), a path that escapes the root (also via symlink) is refused, and it must be an existing directory (checkWorkspaceCWD).

06timeout#

A Go duration string (time.ParseDuration) such as 90s, 10m, 1h30m. An invalid one is refused at save. Empty or absent means the node's timeout config, else the 30m default (runtime.DefaultExternalTimeout). The node timeout bounds the whole checks node, not each check.

07shot#

Harness shape only. A check with shot is optional and never fails the run. After the command smith looks for <cwd>/.smith/shots/<shot>.png; if present, the result names it; otherwise the result carries a note ("no shot taken: ..." from the script's stdout, else the termination reason) (shotOutcome).

A template gets a shot check only when its check node declares page_shot ({executable, args, timeout}) and the ticket body has a line page: <name> (ShotConfigField, TicketPage).

08environment#

Checks get a sterile environment of six keys, plus one when the host has an npm cache:

• PATH: smith's own.
• HOME: a fresh temp dir per run.
• GOCACHE, GOLANGCI_LINT_CACHE, GOMODCACHE.
• SMITH_ANTHROPIC_API_KEY: a dummy.
• NPM_CONFIG_CACHE: only when smith's environment has NPM_CONFIG_CACHE or npm_config_cache, or ~/.npm exists. The path is bound read-only. --offline is left to the check.

A check whose executable sits under the real home (for example ~/.nvm/versions/node/vX/bin/npm) also gets that toolchain directory (~/.nvm/versions/node/vX) bound read-only (checkToolchainPaths). See containment.

Nothing else of the user's environment is passed, on Windows too (windowsjob AuthEnvironment returns the env unchanged), so a tool that needs SystemRoot, USERPROFILE or TEMP may behave differently from a terminal. Checks run with no network, home hidden and a read-only system only on a Linux host that can isolate them. On Windows the job applies memory, process and cpu limits only, so checks run uncontained with network and home visible; the gate shows a red "checks ran uncontained" line. Smith has no Windows network or home isolation yet. checks_require_containment refuses checks on hosts that can't isolate them.

09settings#

Two top-level harness.json knobs apply to every command_check node.

• checks_cpu_percent: the cpu quota percent of each check. Unset or 0 means the default: all cores but two, as (cores-2)*100, never below 200. A negative value is refused with "checks cpu percent must not be negative".
• checks_require_containment: sets require_containment on every command_check node. A check is then refused before launch when the host can't contain it, with command check "<id>" containment: the host can't contain checks (<note>) and checks_require_containment is set. On Windows (and the macOS process-group host) this refuses every check, since only resource limits apply there. Without it, checks run uncontained and the core logs "smith: checks will run uncontained".

10asset checks#

An asset's check (assets.Check) has the harness shape without shot: id, executable, args, cwd and timeout.

11builtins#

• command_check is the checks node builtin.
• check_router routes authoritative command-check results to repair_outlet up to repair_bound times, then to failed_outlet.

12sharding#

A go test check can split its heaviest packages into parallel shards. Three top-level harness.json knobs (next to checks_cpu_percent) switch it on; with checks_shard_packages empty (the default) nothing is sharded.

• checks_shard_packages: checkout-relative package dirs to shard, such as internal/desk, internal/service, internal/tickets/svc.
• checks_shards: the most shard processes running at once across all those packages; 0 means 6.
• checks_shard_min_tests: a package with fewer top-level tests than this runs as one part, with no -test.run; 0 means 100.

What is sharded. Only a check whose args are test, package patterns, -race and -p N (go test -race -p 8 ./...); any other flag runs the check plain and the result's reason says so. The listed packages are taken out of the go test command (out of ./..., or out of the packages check scoping chose) and the rest still run as one plain go test. Without a package list (a workspace that isn't a git checkout) nothing is sharded.

How. Each sharded package's test binary is built once (go test -c, with -race), its top-level tests are listed (-test.list .), and the names are dealt into shards run as <binary> -test.paniconexit0 -test.run '^(TestA|TestB)$' -test.v from the package dir. The verbose output is turned back into plain go test layout while it streams: === lines and passing tests' lines are dropped, and a failing test keeps its --- FAIL line followed by its own log lines (its file:line). The pattern is anchored and regexp.QuoteMeta-escaped, so TestFoo does not run TestFooBar; names hold no /, so subtests run under their parent.

Balance. With recorded times the names go longest first into the lightest shard; with none they are dealt evenly. The time of a test is its own --- PASS/FAIL/SKIP: Name (Ns) time from the shard's verbose output, journalled after each sharded run as a node-observed event check shards <id> and read back from this run and the newest runs of every patch in the same .smith/harness directory (the project's launches), the newest time for each test winning. It is kept out of check_run.

Containment and memory. Every build, list and shard is its own check unit, run with the check's resolved limits and its own attempt id (race-suite#internal/desk-3); one pool runs at most checks_shards of them at once. max_memory_bytes stays 4294967296 and the default is 6 shards: the measured peak was about 4.1 GB over 8 shards, so 6 come to about 3 GB, the budget one check had. The units do not share a cgroup, so the worst case is still 6 x 4 GB. Raise the knob only with that in mind.

Result. The parts fold into one result under the check's own id: it passes only if every part passed, the output is the parts passing first and failing last with an added ok/FAIL line per binary part, and the exit, termination and reason come from the first failing part, so a failing shard reports its --- FAIL tests and file:line as an unsharded run does.

13parsed-output#

Only these are parsed (failingTests, failingWithLocations):

• go test: --- FAIL: <Name>; the location is the first file.go:N on a following line before the next --- line.
• TAP / node --test: not ok <n> - <name>.
• node spec reporter: ✖ <name>.
• node test file that fails to load (✖ <file>.mjs with only 'test failed'): the row names the thrown error (ReferenceError: …) and its file:line, taken from the path:line / source / error block printed above it.
• golangci-lint text lines path/file.go:LINE:COL: message (linter) become failing rows with file:line.

At most 10 names are listed, then "… and N more". Locations exist for .go files and node load failures only. Any other format still fails the check on a non-zero exit and shows the stderr/stdout tail instead of named tests.

Exit 126/127, "command not found", or npm's ENOTCACHED (" is not in the npm cache; add it on the host and relaunch") for every failing check ends the node as an environment failure, not a repair loop (checkEnvironmentBroken).

14when you're the conductor#

• Put every argument in args; executable is never a shell line.
• A record can't set cwd or shot.
• Read the reference from the projects tool called with a name; a refused record draft repeats the field meanings.