---
title: What is the risk when AI writes code nobody on the team understands?
author: Giacomo Balli
published: 2026-10-04
url: https://BigBalli.com/blog/ai-written-code-that-nobody-understands
tags: AI Coding, Software Development, Code Review, Hiring Developers, Vendor Evaluation
---

# What is the risk when AI writes code nobody on the team understands?

*By Giacomo Balli* | *Published: October 4, 2026*

**Short answer:** The risk is that nobody can fix, change, or explain the software when it breaks, because no person understands how it works. The code may run fine today. Protect yourself by naming who understands the system, requiring tests and architecture notes as deliverables, setting a human review policy, and paying for a real handover.

The main risk when AI writes code nobody understands is software that nobody at your company or your agency can fix, change, or explain when it matters. The code can work perfectly on day one. AI coding tools write code faster than people can read it, and most developers now use them. This week, three Hacker News threads with more than 1,100 comments between them described what that looks like inside real teams.

1. **1. AI writes it**Thousands of lines in minutes
2. **2. Review shrinks**Approvals with no comments
3. **3. Nobody learns it**The team asks the AI instead
4. **4. It breaks**No person can say why
The failure shows up months after the code shipped, usually during an outage or a vendor change.

## What are engineers reporting about AI-written code this week?
Commenters on Hacker News this week reported three patterns in teams that rely on AI coding tools: code review shrinking to approvals nobody reads, developers who ship for months without learning the system, and architecture decisions that nobody can trace. The reports came from threads with 419, 222, and 506 comments.

In the ["Plan mode is dead" thread](https://news.ycombinator.com/item?id=49840054), one commenter wrote: "I'm actively watching understanding slip away from developers, code review getting paired down to no comment checkmarks." In ["Coding is not solved"](https://news.ycombinator.com/item?id=49877988), another said code review is "effectively dead in the water since no human can actually review such amounts of code." The same commenter noted that some companies now have AI review the AI's code.

The most specific account came from an engineer who switched jobs in July. For four weeks, asking Claude how the legacy codebase worked felt like freedom. After two months: "I still know nothing." They had cut back on AI and forced themselves to read and write code again. That was in the [thread on Simon Späti's post](https://news.ycombinator.com/item?id=49880312).

## Why is code nobody understands a business risk?
Code nobody understands is a business risk because software costs most after it ships. Bugs, outages, security fixes, and new features all need someone who knows why the system works the way it does. When that knowledge exists only in chat logs with an AI, every change becomes slower, riskier, and harder to price.

Simon Späti's post, [The Problem is not the AI Code, but Nobody Knows Anything Anymore](https://www.ssp.sh/brain/the-problem-is-not-the-ai-code-but-nobody-knows-anything-anymore/), quotes a comment from one of his discussions: "nobody knows anything, and everyone just asks Claude." His conclusion: "the final boss is, and always will be, maintainability." The easier software is to generate, the more of it someone has to maintain.

Ayman Nadeem, previously a senior engineer at GitHub, framed the same problem in ["Plan mode is dead"](https://www.aymannadeem.com/artificial/intelligence,/developer/tools/2026/09/24/plan-mode-is-dead.html): how do people keep "a coherent mental model of a software system while machines are changing it faster than humans can inspect the changes?" Nadeem writes that the problem is still unsolved. In the thread on the post, a commenter who said they work on Claude Code agreed that plan mode, a step that made the AI lay out its approach before coding, is no longer useful. For an owner, that means your agency may not have solved it either.

## Is AI-written code good enough for most business software?
AI-written code is often good enough for simple business software such as internal tools, admin screens, and standard web forms. One Hacker News commenter argued that most enterprise software is routine. The risk grows where mistakes cost money, customer data, or legal exposure, and there a person must understand and answer for the code.

The counterview is real. One commenter in the "Coding is not solved" thread said the author "underestimates how boring and simple 90% of enterprise software is," and that most of it "runs 24/7 without a glitch." Another said the article would have been fully correct a year ago and far less so now, given current models.

The survey data is less confident. In the [Stack Overflow 2025 Developer Survey](https://survey.stackoverflow.co/2025/ai), 84% of respondents use or plan to use AI tools, yet 46% distrust their accuracy against 33% who trust it. The top frustration, cited by 66%, is "AI solutions that are almost right, but not quite." Almost right is the kind of bug that needs a person who understands the code to find.

## Does AI make experienced developers faster?
AI does not reliably make experienced developers faster on complex, familiar code. A July 2025 study by METR found 16 experienced open-source developers took 19% longer on 246 real tasks when allowed AI tools, while believing afterward that AI had sped them up by 20%. Perceived speed and measured speed can point in opposite directions.

The [METR study](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/) used models from early 2025, and tools have improved since. The gap between feeling and measurement is the lasting lesson. Alex Ewerlöf, author of ["Coding is not solved"](https://blog.alexewerlof.com/p/coding-is-not-solved), gave a smaller example: his agent took 12 minutes and 72 steps to update five npm dependencies he could have updated in under a minute.

Ewerlöf also names the part that matters to an owner: "wherever a mistake can cost money, lives or legal consequences you need accountability." An AI tool cannot be held accountable. Your developer or agency can, but only if someone there understands what shipped.

## What should owners require from developers using AI?
Owners hiring developers or agencies that use AI should require four things in the contract: a named person who understands each part of the system, automated tests and short architecture notes as paid deliverables, a written human review policy, and a handover that proves a new developer can run and change the software without the original team.

1. **Named owner**A person, not a tool, for each part of the system
2. **Tests and notes**Automated tests plus why each major choice was made
3. **Review policy**Which changes a human must read before release
4. **Tested handover**An outside developer sets it up from the notes alone
Ask for all four in writing before signing, and tie a payment to the handover.

A commenter in the Späti thread who works in safety-critical software described their rule: "We just don't ship code that isn't 100% human reviewed." Most businesses do not need that standard everywhere. They do need it on payments, logins, and customer data. In the interview, ask a developer to walk you through a recent change and explain why it works. A pause is fine. Scrolling a chat log for the answer is a warning.

If you already have AI-built software and nobody can explain it, an [independent review](https://BigBalli.com/second-opinion) can tell you how much of it someone actually understands before you fund the next phase.

## Related guides
- [What are the red flags in a software development proposal?](https://BigBalli.com/blog/software-proposal-red-flags)
- [Agency or freelancer for a mobile app?](https://BigBalli.com/agency-vs-freelancer-mobile-app)
- [Do I need a fractional CTO or an advisor?](https://BigBalli.com/do-i-need-a-fractional-cto-advisor)
- [The Second Opinion: an independent review of a software project](https://BigBalli.com/second-opinion)
- [METR: impact of early-2025 AI on experienced open-source developers](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/)
- [Stack Overflow 2025 Developer Survey: AI](https://survey.stackoverflow.co/2025/ai)

## Key takeaways
- The cost of code nobody understands arrives after launch, during outages, security fixes, and vendor changes.
- Engineers report AI-era code review shrinking to approvals with no comments, which removes the main check on quality.
- AI-written code is often fine for routine internal tools, and riskier wherever money, data, or legal exposure is involved.
- METR measured experienced developers 19% slower with AI tools while they believed they were 20% faster.
- Contract for a named system owner, tests, architecture notes, a review policy, and a handover a stranger can complete.

## Frequently asked questions
### Should I stop my agency from using AI coding tools?
No. Banning the tools is hard to enforce and gives up real speed on routine work. Require results instead: a named person who understands each part of the system, automated tests, short architecture notes, and a human review policy for sensitive code such as payments, logins, and anything that touches customer data.

### How can a non-technical owner tell if developers understand their code?
Ask a developer to explain a recent change: what it does, why it was built that way, and what would break if it failed. Then ask what tests cover it. Clear, specific answers are a good sign. Vague answers, or reading from an AI chat to answer, suggest nobody owns that part.

### What should a software handover include when AI wrote much of the code?
A software handover should include the source code in your own account, setup instructions, automated tests that run, architecture notes explaining the main decisions, and the prompts or specs used for major features. Test it by having a developer outside the original team set up and change the system using only those materials.

### Is AI-written code less secure than human-written code?
Not by definition, but code nobody reviewed is harder to trust. Stack Overflow's 2025 survey found 46% of developers distrust the accuracy of AI tools. For payments, logins, and personal data, require human review and a security check before release, whoever or whatever wrote the code.

## About the author
Giacomo Balli is an independent technology advisor in San Francisco. He has built software and mobile apps since 2010, runs a portfolio of more than forty live apps of his own, and reviews software, AI, and vendor decisions for owners before they commit the money.

## Disclosure
Giacomo Balli sells fixed-fee independent reviews of technology decisions, including software projects and agency proposals. He does not build software for clients, resell platforms, or take referral fees. No company named on this page paid to be mentioned.

## Questions this answers

**Should I stop my agency from using AI coding tools?**

No. Banning the tools is hard to enforce and gives up real speed on routine work. Require results instead: a named person who understands each part of the system, automated tests, short architecture notes, and a human review policy for sensitive code such as payments, logins, and anything that touches customer data.

**How can a non-technical owner tell if developers understand their code?**

Ask a developer to explain a recent change: what it does, why it was built that way, and what would break if it failed. Then ask what tests cover it. Clear, specific answers are a good sign. Vague answers, or reading from an AI chat to answer, suggest nobody owns that part.

**What should a software handover include when AI wrote much of the code?**

A software handover should include the source code in your own account, setup instructions, automated tests that run, architecture notes explaining the main decisions, and the prompts or specs used for major features. Test it by having a developer outside the original team set up and change the system using only those materials.

**Is AI-written code less secure than human-written code?**

Not by definition, but code nobody reviewed is harder to trust. Stack Overflow's 2025 survey found 46% of developers distrust the accuracy of AI tools. For payments, logins, and personal data, require human review and a security check before release, whoever or whatever wrote the code.

---

**Tags:** AI Coding, Software Development, Code Review, Hiring Developers, Vendor Evaluation

---

*View the [HTML version](https://BigBalli.com/blog/ai-written-code-that-nobody-understands) of this post.*