AI that can be checked

In my research, AI agents help build and check the models, but they are not the scientific model; the models themselves use modern machine learning where it serves the question, and their parts stay measurable. AI agents write and validate analyses, programmed rules and models; results that matter are re-checked independently, and every decision and result is documented with its evidence. I develop the open software that runs these agents on a shared computing cluster: agent_sandbox, which contains them, and Nexus, which coordinates them.

Why: models that can be checked

Three principles carry through my work. Every prediction comes with its uncertainty, so a supported result can be told apart from an extrapolation. What my models learn on the way to a prediction, such as a density of cell states or a change in expression at every state, is a biological quantity that can be measured and tested directly. And AI agents help build and check the models without being the scientific model. The first two are described on the research page; this page is about the third.

When agents annotate cell types, for example, they do not label cells themselves. They write programmed rules over marker expression and validate them against the data; the rules, not the agents, make every call. Each call is therefore reproducible, and each rule is documented with the evidence for it. The same holds for analyses and models: what an agent produces is code and a record, which can be rerun and audited, not an answer that has to be trusted.

The same habits shape how I mentor people: specify the task, have the result checked independently, and record each decision with who made it.

How a task runs

Nexus makes GitHub issues the control surface for a team of AI coding agents. I file a task as an issue. An orchestrator agent triages it and spawns a worker agent, which owns the task from start to finish: it writes code, runs analyses on the cluster, and ends with a structured report that is checked against a schema and linked on the issue.

For results that matter, the orchestrator also spawns a skeptic: a separate agent, starting from a fresh context, that tries to break the result by re-deriving it, reproducing it on cases with known answers and tracing its assumptions to their source. Its verdict is credible, check, suspect or refuted, and a refuted verdict overturns the worker's result. Where a skeptic is required, the worker cannot be retired until the verdict exists, and a worker is not told in advance that a skeptic may follow, so it cannot tailor its account to the reviewer.

Schematic. The researcher sets tasks through GitHub issues. Inside agent_sandbox, an orchestrator agent spawns worker agents and skeptic agents; a worker's result goes to a skeptic, whose verdict is credible, check, suspect or refuted. Slurm jobs from the workers pass through a chaperon and run sandboxed on the compute nodes. Reports and verdicts form the record, which is linked on the issue.
Schematic of one task. The researcher works through GitHub issues; the agents run inside agent_sandbox; Slurm jobs they submit pass through the chaperon and run sandboxed on the compute nodes; every agent files a report, and skeptics add their verdicts to the record.

Containment

Nexus starts its agents without a confirmation for each command, so it runs only inside a containment layer. agent_sandbox is that layer for shared computing clusters. It isolates agents with kernel namespaces rather than containers, so cluster modules, conda environments and GPU libraries work as they do on the host. Writes are confined to the project directory, paths that are not mounted do not exist inside the sandbox, and the operator decides which credentials and data reach the agents; in my own setup, the credentials and data agents can read are ones I chose to share.

Slurm commands go through a chaperon that runs outside the sandbox. It checks each submission, shows agents only their own project's jobs, and runs every job inside the sandbox on the compute node, so submitting a job is not a way out. agent_sandbox is best-effort isolation in user space, not a security product, and its documentation lists its limits.

The record so far

Since April 2026 these tools have run research analyses. Counts measured on 17 October 2026:

Agent tasks
4,079
Independent re-checks
1,593 skeptic verdicts, of which 69 found the result suspect or refuted it
Cluster jobs
5,256 Slurm jobs, about 52,000 CPU-hours
Concurrent agents
at least 21 at the peak
Reports
4,480

About one re-check in twenty-three found a result suspect or wrong. Agents can be confidently wrong, which is why a second agent that does not trust the first checks the results that matter before they reach me. Every result and decision is documented in reports, websites and writeups.

The software

Developer · MIT license

agent_sandbox

Kernel-enforced sandboxing for AI coding agents on shared HPC clusters. The operator controls which files and credentials agents can reach, and Slurm jobs that agents submit stay sandboxed. Released through Homebrew, with continuous integration and documentation.

Developer · MIT license

Nexus

Orchestration of teams of coding agents from GitHub issues, with independent re-checks of results and every result and decision documented. Public and documented; shipped disabled by design, to be enabled deliberately by each group that adopts it. Its skeptic protocol adapts the verification design of Erick Matsen's bipartite project.

Smaller tools

  • labsh: project-local JupyterLab management from the command line, for researchers and AI agents on shared clusters. MIT license.
  • cc-timestamps: a Claude Code plugin that marks each message and tool call in a transcript with its local time and the time since the previous one.