First Run the Tests: The New Imperative in AI-Assisted Software Development

In the rapidly evolving landscape of AI-assisted programming, a deceptively simple directive is gaining traction among developers: "First run the tests". Coined by software engineer and investigative technologist Simon Willison, this four-word command has become a cornerstone of agentic engineering — a new discipline where human developers collaborate with AI agents to build, test, and maintain code. Far from being a mere suggestion, this phrase encapsulates a paradigm shift in how software quality is ensured in the age of generative AI.

Traditionally, automated testing was often sidelined in fast-moving projects due to perceived overhead: tests were seen as brittle, time-consuming to maintain, and easily outpaced by iterative development. But with the advent of sophisticated coding agents capable of reading, writing, and updating test suites in minutes, those excuses have evaporated. As Willison notes in his guide on Agentic Engineering Patterns, AI agents don’t just write code — they analyze context, infer intent, and, crucially, rely on test suites to validate their outputs. Without tests, AI-generated code is essentially a gamble: if it’s never been executed, its functionality is a matter of luck, not engineering.

The power of the prompt lies not just in its instruction but in its psychological and procedural framing. When a developer begins a session with an AI agent by typing First run the tests — or, in Python environments, uv run pytest — they immediately signal that testing is non-negotiable. This forces the agent to engage with the codebase’s existing test infrastructure, uncovering its structure, scope, and edge cases. In doing so, the agent gains critical context about the project’s architecture, dependencies, and expected behavior. This contextual grounding significantly reduces hallucinations and ensures that new code aligns with established patterns.

Moreover, the prompt triggers a cognitive shift in the agent’s behavior. Research in AI-assisted programming suggests that agents exhibit a strong bias toward testing when exposed to existing test suites. Once an agent observes that tests exist, it begins to treat them as a contract — a set of expectations that any new code must satisfy. This mirrors the principles of Test-Driven Development (TDD), where tests precede implementation. As Willison observes, this practice is now being internalized by AI models, making "red-green TDD" a de facto workflow even without explicit human prompting.

For teams adopting AI agents, the implications are profound. Starting every session with a test run reduces integration failures, accelerates onboarding, and creates a feedback loop that reinforces quality. It also serves as a diagnostic tool: if tests fail after an agent’s change, developers immediately know the nature and location of the regression. This transparency is invaluable in complex systems where human reviewers may not be familiar with every module.

While some may dismiss this as a mere workflow hack, its adoption by leading engineers signals a deeper truth: in AI-assisted development, human oversight is not about writing code — it’s about setting boundaries, enforcing discipline, and embedding best practices into the agent’s decision-making loop. The phrase "First run the tests" is more than a prompt; it’s a ritual of accountability.

As AI agents become ubiquitous in development workflows, organizations that institutionalize such minimal, high-impact prompts will outperform those that treat AI as a black-box code generator. The future of software engineering isn’t about replacing developers — it’s about augmenting them with disciplined, test-aware agents. And it all begins with four words.