There is a version of the "AI writes code now" argument that concludes tests matter less — if the assistant is good enough to write correct code, why spend time verifying it. I have found the opposite to be true in practice, for a reason that is more about how AI-generated code fails than about AI capability: it fails in a way that is specifically harder for a human reviewer to catch by reading alone, which makes automated verification more valuable, not less.
Why AI-generated bugs are harder to spot by eye#
A human writing code slowly tends to make mistakes that look like mistakes — a typo, an obviously incomplete branch, a TODO left in place. An AI assistant generating a full, syntactically clean implementation in one pass tends to produce code that is internally consistent and confident-looking even when it is wrong, because the model is not distinguishing "I am sure this is correct" from "this is the most statistically likely continuation" — both produce the same fluent output. There is no visual tell for "this branch was generated with less certainty than that one."
This is the same trust gap I described in the vibe coding post, specifically applied to why reading a diff is not sufficient review on its own: a diff that reads cleanly and a diff that is correct are not the same claim, and AI-generated diffs tend to read cleanly regardless of which one they actually are.
ℹThe core argument, stated plainly
A test does not care how confident the code looked while it was being written. It checks actual behavior against an expected value, which is exactly the kind of check that a fluent, plausible- looking implementation cannot fake its way past.
What I actually test, and how#
- Boundary conditions the prompt never mentioned. If I asked for "a function that formats a date," I write a test for the empty string, an invalid date, a date at a timezone boundary — the inputs I did not explicitly describe when I asked for the implementation, because those are exactly the inputs the assistant had no signal to specifically handle.
- The specific bug class from this project's own history. DevStash has real prior incidents — a canonical URL double-prefix bug, a duplicate JSON-LD schema being emitted, a metadata field named
pathwhen the actual type usedcanonical. When an assistant touches code in those areas again, I specifically re-check for that exact class of regression, because "the model made this mistake once" is a real prior, not paranoia. - End-to-end checks for anything user-facing, via the Playwright suite already wired into
pnpm qa— responsive layout across breakpoints, and the static security audit. These catch a category of bug that unit tests structurally cannot: the assistant's code being individually correct but composing badly with an existing component in a way that only shows up when rendered together.
💡A habit that catches more than it should
I ask the assistant itself to write the test cases for edge conditions before accepting its implementation, in a separate step. A model reasoning about "what could break this" as an explicit task surfaces cases it did not spontaneously handle while writing the original implementation — the two tasks draw on different parts of its reasoning even though it is the same underlying model.
This is not about distrust of the tool specifically#
I would apply the same discipline to code written by any contributor I had not personally watched write it line by line, AI or human. What changed is the volume: AI-assisted development means more code moves through review per hour than before, so the fixed cost of a solid test suite gets amortized over more changes, and the fixed cost of not having one compounds faster too. Faster code generation without a proportional increase in verification is the actual risk — not the code generation itself.
The honest summary: tests were never really about catching mistakes a careful human reviewer would obviously spot. They were always about catching the ones that look fine on a read-through. AI-generated code raised the rate of exactly that failure mode, which is precisely why the tests matter more now, not less.
Quick answers
Do AI-written functions need tests?
Yes. AI code is syntactically clean and plausible, so its bugs do not look like mistakes the way a typo or a TODO does, and a diff that reads well is not the same as a diff that is correct.
What do you test in AI-generated code?
Boundary conditions the prompt never mentioned, the bug classes from the project's own history, and end-to-end checks for anything user-facing through the Playwright suite in pnpm qa.
Is this distrust of AI specifically?
No. The same discipline applies to any contributor you did not watch write the code. What changed is the volume of code moving through review per hour.
Need something like this built or fixed?
Custom websites, landing pages and speed fixes. I reply within 24 hours.
Adesh Shukla
Front-end developer who came to the web from accounts, by way of an interest in UX. Building DevStash — a developer ecosystem covering automation, AI workflows, and modern frontend systems.