Dan Walsh

Senior Software Engineer | Evaluation infrastructure, sandboxing, verification

Talent

LondonMember since October 10, 2026

Inspect AIControlArenaLLM-as-judge validationadversarial reviewDockerKubernetesAWSGCPCloudflareFly.ioPythonTypeScriptGoNode.jsReactNext.jsGraphQLWebdriverIO

Hire this person

Send a job offer directly to this candidate

Inspect AIControlArenaLLM-as-judge validationadversarial reviewDocker

About

Senior Software Engineer (9+ years) building production systems, full-stack, in React/TypeScript and Python.

I build evaluation and control tooling: evals harnesses, LLM-as-judge validation, sandboxes and run provenance. The finding that recurs across my work is a check that is present, documented and green, yet catches nothing. So I measure what a control actually catches, not whether it exists, and I apply that to my own tools first.

Safety focus: runtime monitoring and control of agent behaviour. Co-author of the v1 draft of the AI-assisted development standard at PensionBee, a UK pensions firm.

Experience

Senior Software Engineer at PensionBee (2024-01 – Present)

React, TypeScript, Next.js, React Native, Node.js, Python, Heroku, Postgres, Docker, Cloudflare (Workers, Workflows, D1, KV, cron), GitHub Actions, Salesforce

AI-assisted development standard: Co-author of the firm's internal standard for AI-assisted software development (v1 draft, currently under adversarial review): every requirement names an acceptance bar and an artefact an auditor can open, and fully agentic work always requires a human to attest to it.
Attestation tooling: Fixed a transcript-parsing defect in my own evidence tooling that was silently dropping failed checks from the evidence record: an evidence tool reporting green by omission. The tooling is a Claude Code skill (a reusable agent workflow) that writes the pull-request description from the agent's session transcript, so the description records what was actually done and checked rather than the agent's own summary of it. I then extended it and packaged it as a plugin for rollout across the organisation.
Shadow-mode evaluation harness: Built a shadow-matching harness and an extract mode for a bulk comparison tool, so a document-automation pipeline could be compared against the live process on the same real inputs in a read-only trial before it was allowed to act.
Verification catches: Caught controls and claims that did not survive checking. A committed CI bundle guard pointed at a gitignored build directory, so it passed only where stale local output existed and would have failed on any clean checkout, and was never exercised in CI: present, locally green, and never a check. A base64 decode path with a ~32x memory blowup, where the first measurement of the fix was itself wrong. An integration step that reported success it had not achieved.
Guardrails tested before install: Set a prove-before-install rule for agent configuration: every proposed global rule had to be shown to work before it was installed, not adopted because it sounded right. Derived team guardrails from real incidents: a secrets rule after a sourced environment file leaked a sandbox RSA private key and forced a key rotation, a clean-checkout check for CI guards, and a mutation check on new tests.
Evidence-backed review: Ran pull-request reviews as parallel review lenses whose contested findings had to be settled by evidence before reporting: reproductions in isolated worktrees, mutation checks on the tests, and throwaway real-database tests, including one proving that a transaction rollback actually rolled back.
Reliability under concurrency: Built an idempotency-aware Cloudflare D1 retry layer for concurrent writes - transient-only retries with backoff, fail-fast on constraint errors - resolving a live data-loss issue.
Login security hardening: Strengthened login security to prevent brute-force attacks and DDoS vulnerabilities.
Validated classifier: Built and validated an email-classification matcher against 614 real subjects, rather than against assumptions about what the data looked like.
Delivery breadth: Led the customer withdrawals rewrite (legacy → React/Next.js) across Product, UX, Banking and Compliance, and co-created a shared React component library replacing fragmented UK/US libraries.

Software Engineering Manager at PensionBee (2023-08 – 2024-01)