AI-generated code can look unusually convincing before it is actually ready to carry a business.
The page renders. The form submits. The dashboard loads. The tests the agent wrote are green.
That is useful evidence, but it answers only one question:
Can the implementation complete the path we just tried?
Production review asks harder questions. Can the wrong user read the record? What happens when a webhook arrives twice? Can a client change a price in the browser? Is a secret in the bundle? Does a failed write leave half a transaction behind? Can another developer understand and safely change this six months from now?
If you use Lovable, Codex, Claude Code, Copilot or another AI coding tool, the goal is not to distrust generated code because AI touched it. The goal is to remove the dangerous shortcut between “it works” and “ship it.”
The short answer
Before production, review AI-generated changes in this order:
- Understand what changed and what the feature is supposed to guarantee.
- Identify the high-risk boundaries: auth, permissions, user data, payments, secrets, database writes, files and external APIs.
- Test both allowed and denied behavior.
- Review dependencies, configuration and generated database/infrastructure changes.
- Run appropriate secret, dependency and security scans.
- Test retries, duplicates, invalid input and partial failure.
- Confirm logs and monitoring can tell you when the feature fails after launch.
- Make sure someone human can explain the architecture and owns the decision to ship.
OpenAI’s current Codex guidance still says users should manually review and validate agent-generated code before integration and execution. Its code-review guidance describes Codex as an additional reviewer, not a replacement for human review: https://openai.com/index/introducing-upgrades-to-codex/
OWASP’s current Secure Coding with AI guidance makes the same accountability point: AI-generated changes should have a human owner who reviews and approves them before merge: https://cheatsheetseries.owasp.org/cheatsheets/Secure_Coding_with_AI_Cheat_Sheet.html
That is the right starting model.
Start with the business invariant, not the diff
A line-by-line review is much weaker when you do not know what must remain true.
Suppose AI builds a tenant payment feature. The code can be tidy and still be wrong if the real rules are:
- one tenant must never see another tenant’s payment;
- a payment provider callback cannot be trusted merely because it contains a reference;
- the same verified payment cannot credit the ledger twice;
- a failed receipt email must not reverse a successful payment;
- a refund or reversal must preserve an audit trail.
Those are business invariants. They tell you what the code must protect.
Before reviewing implementation details, write down the few things that would make the feature seriously wrong if violated.
For a booking system, that may be double-booking.
For an ecommerce checkout, it may be accepting an unverified payment or charging twice.
For a multi-user SaaS app, it is often cross-account data access.
This is where product thinking improves code review. A scanner can identify many technical patterns. It cannot automatically know every promise your product makes to a user.
Review risk, not every file with equal intensity
AI can generate a large diff quickly. Treating every changed line as equally risky wastes review attention.
I would slow down most around these areas:
Authentication and authorization
Authentication asks who are you? Authorization asks are you allowed to do this specific thing?
A page being hidden from the navigation is not authorization. A client-side role check is not enough to protect a server operation. A database query that accepts a record ID still needs to establish whether the caller owns or may access that record.
For important endpoints, test at least three states where relevant:
- the correct authenticated user;
- an authenticated user who should not have access;
- an unauthenticated request.
The denied cases are often more informative than the happy path.
User data and multi-tenant boundaries
Search for every place the feature reads, updates, deletes or exports user-controlled data.
Ask whether the boundary is enforced on the server or database side, not merely implied by the UI.
If the product has organizations, properties, teams, projects, patients, clients or tenants, deliberately try to cross that boundary in a safe test environment.
A production-ready system should fail closed.
Payments and state transitions
Payment code deserves a different review standard from a marketing page.
Do not review only whether the success screen appears. Verify what makes the system decide that money was actually received, how duplicate provider events are handled, what happens when verification times out, and whether the ledger/order/subscription can enter contradictory states.
I treat payment state as a state machine, not a button click.
If you are building this kind of flow, my guide on payment callbacks vs webhooks and verification explains why the browser redirect should not become the source of truth for money.
Database migrations and destructive operations
AI agents are very good at producing SQL that looks reasonable. That does not mean you should run an unfamiliar migration against live data.
Review drops, renames, type changes, constraints, backfills and permission changes separately. Ask what happens to existing rows and whether the operation is reversible.
If preview and production disagree, diagnose database state before generating more SQL. I covered that failure mode in Lovable database migration drift.
Test the negative path on purpose
One of the easiest ways to overestimate an AI-built feature is to test it exactly as the builder expected.
Try the things a real system has to survive:
- missing required fields;
- malformed input;
- a stale session;
- an expired token;
- a user requesting someone else’s record;
- a request submitted twice;
- a webhook delivered twice;
- a network timeout after the server already completed the write;
- an upload that is too large or the wrong type;
- an external provider returning an error;
- two users changing the same resource close together.
A polished happy path can hide weak failure semantics.
The question is not simply whether an error appears. Check whether the system leaves the data in a truthful state afterward.
Check secrets and configuration separately from application logic
A code review can miss the fact that the deployment itself is unsafe.
Search the changed files and repository history for credentials, API keys, private tokens and copied .env values. Then inspect which environment variables are intentionally exposed to browser code.
GitHub’s current documentation supports secret scanning from compatible AI coding agents and can identify keys, tokens and credentials before they reach the repository: https://docs.github.com/en/code-security/how-tos/use-ghas-with-ai-coding-agents/scan-for-secrets-with-github-mcp-server
But “the scanner found nothing” is not proof that your configuration model is correct. You still need to understand which secrets belong only on the server and which public keys are designed for client use.
For Supabase specifically, see whether the anon key is safe to expose rather than treating every visible key as equivalent.
Review every new dependency the agent introduced
AI agents can solve a small task by adding a surprisingly large dependency.
Before accepting a new package, ask:
- Did we need a package for this at all?
- Is this the package we intended to install?
- Is it actively maintained?
- What permissions or runtime access does it need?
- Does it substantially increase the client bundle or server attack surface?
- Did the lockfile change in a way we understand?
Run the ecosystem’s dependency audit and the security tooling appropriate to the repository. Do not copy an install command simply because the generated code imports the package successfully.
Use AI to challenge the code, not merely praise it
There is nothing wrong with using a second AI pass for review. The mistake is asking only:
“Is this code good?”
Give the reviewer a job that can fail.
For example:
Review this diff against these invariants. Look specifically for authorization bypass, duplicate side effects, unvalidated client input, leaked secrets, unsafe migrations and failure states that could leave data inconsistent. Do not rewrite yet. Show the evidence for each finding and tell me which assumptions you could not verify.
That prompt is useful because it defines the product’s risk model instead of asking for generic approval.
OpenAI’s Codex Security goes further by building a codebase-specific threat model, attempting to validate suspected vulnerabilities in an isolated environment and proposing patches for human review: https://help.openai.com/en/articles/20001107-codex-security
Even there, the proposed patch is surfaced for review rather than silently becoming production code.
A green test suite can still test the wrong thing
AI is useful for generating tests, but generated implementation and generated tests can share the same mistaken assumption.
Read the important assertions.
For high-risk behavior, make sure tests prove the invariant rather than merely prove the function returned something.
Instead of only:
returns 200 for a valid request
also prove things such as:
- another user receives 403 or no protected data;
- the same payment event cannot create two credits;
- invalid state transitions are rejected;
- a transaction rolls back when a dependent write fails;
- deleted or archived records cannot be silently resurrected;
- retrying an idempotent request does not duplicate the side effect.
This is the difference between testing the implementation and testing the product rule.
Do not let one scanner become your definition of security
Static analysis, dependency scanning, secret scanning, AI review and runtime testing find different classes of problems.
OWASP’s AGHAST project makes the distinction clearly: traditional static checks are useful for known patterns, while codebase-specific authorization and business validation can require contextual analysis: https://aghast.owasp.org/
Use layers.
A practical small-team gate might include:
- lint/typecheck/build;
- unit and integration tests;
- secret scanning;
- dependency review;
- static/security analysis where appropriate;
- focused manual review of auth, data and payment boundaries;
- a staging smoke test of the actual user journey.
You do not need enterprise ceremony for every MVP. You do need evidence proportionate to what can go wrong.
Check observability before launch
Production introduces failures your local test cannot predict.
Before shipping, ask:
- Will a failed server operation produce a useful log?
- Can you correlate an error with the request, job or payment reference without logging sensitive data?
- Can you tell the difference between a notification failure and a failed business transaction?
- Do you know when background jobs or webhooks stop succeeding?
- Can you see a sudden increase in 401, 403, 500 or timeout responses?
A feature that fails invisibly is expensive to operate even when the code itself is clean.
That same principle is why a contact form should not rely on one notification email as the only copy of a lead.
Make rollback part of the review
Before a risky deployment, know how you would undo it.
Application code can often be rolled back quickly. Database migrations, user-generated data and external side effects are harder.
For a meaningful production change, ask:
- What will we monitor immediately after release?
- What result means we stop the rollout?
- Can the application be reverted independently of the database change?
- If the schema changed, is the previous application version still compatible?
- Did the deployment send irreversible emails, payments or external API actions?
A rollback plan exposes architecture assumptions before an incident does.
The production-readiness test I care about
I do not need a developer to remember every line an AI agent generated.
I do need someone to be able to explain:
- what the feature is supposed to guarantee;
- where authorization is enforced;
- which system is the source of truth;
- what happens on duplicate or partial execution;
- what data can be lost or exposed;
- how secrets are kept out of the client;
- how the change was tested;
- what we will observe after launch;
- how we recover if the assumption is wrong.
If nobody can answer those questions because “the AI handled it,” the feature is not ready simply because it works in preview.
AI changes how quickly we can build. It does not remove the need to understand the system we are putting in front of users.
If you already have an AI-built application and need to decide whether to fix, harden or rebuild it, start with the AI-built app production checklist. If the product is beyond a prototype and you want help reviewing the architecture, payment/data boundaries or production path, see MVP development or contact me.
