Claude Code + Codex Review: What 5 PRs Showed (Sep 2026)

Key takeaway — In September a code review I’d automated inside Claude Code fired about 400 times in three days, on Opus, and helped burn my weekly limit. I split review into two tiers: a light one on Sonnet inside Claude Code, and one deeper review per pull request by OpenAI’s Codex, on a separate subscription. On a 5-PR test Codex had fewer false alarms than Opus and found one bug no other model did. The bigger lesson was that different models miss different bugs.

This is the follow-up to how I stopped hitting the Claude Max weekly limit. That article got a September update today: I hit the limit again, and a big part of the cause was a code review.

The review that ate the quota

I had a plugin that asked Opus to review the diff every time a Claude Code session or subagent finished. It sounded like cheap insurance. In four weeks it ran 854 reviews. 405 of them happened in three days, September 13 to 15. Sessions were fanning out dozens of subagents, one of them 120, and every subagent that finished triggered its own review. Each review was a full Opus call on the same weekly quota I needed for the actual work.

That’s the thing about automated review inside the tool that writes the code: it scales with the fan-out, not with the number of changes worth reviewing. A hundred subagents finishing isn’t a hundred changes. Most of them were searches and reads.

Two tiers, two vendors

I split it:

• Inside Claude Code, a light review on Sonnet. It no longer fires on every subagent finish. It catches the obvious stuff while the session is still open.
• Once per pull request, a deeper review by a different vendor. OpenAI’s Codex, run from the command line on a ChatGPT plan. It reads a read-only snapshot of the change, never edits anything, and writes a report. Its usage meter is separate from Claude’s.

Why a different vendor and not just a second Claude call? Two reasons. The quota one is obvious: the deep review stops competing with the work. The other is blind spots. A model reviewing code written by the same model tends to share its assumptions. I wanted a reviewer that was trained differently.

I didn’t take that on faith. The published evidence for cross-model review is thin: vendor claims, and one controlled study where a weaker reviewer helped a weaker writer but hurt a stronger one. So I tested it on my own code.

The test: 5 merged pull requests

Setup, kept as fair as I could make it:

• Five merged pull requests from my own product, not client code. Real changes: background jobs, a credential flow, a status screen, a data migration.
• The same prompt and the same context for every reviewer: the diff, a short pack of related code, and instructions to verify the PR’s claims against the code.
• Three Codex models (gpt-5.6-terra, gpt-5.6-sol, gpt-6-astra) at high effort, plus a control: Claude Opus with the byte-identical prompt.
• Grading against the code, not against the reviewer’s argument. Every finding was checked at the merge commit and marked real, minor, or false. Five distinct real defects turned up across the five PRs; two of the PRs had none.

	Terra	Sol	Astra	Opus (no tools)
Real defects found (of 5)	3	4	4	3
Findings	12	11	15	23
False alarms	6	1	3	3
Found by this model only	0	1	0	0

What that means in practice:

• Terra was noisy. Half of its findings were wrong. I don’t use it as the default.
• Sol was the most precise. One false alarm in eleven findings. It was also the only model that caught a screen showing an account as “connected” before the account had been verified. That one shipped, and it was fixed the same day by the next PR.
• Astra was the fastest and the tersest, but padded its reports with “cannot be verified” items.
• Opus was more verbose. Twenty-three findings, fifteen of them true but minor. On one PR it quoted code that doesn’t exist in the repository.

The finding I trust most isn’t the ranking. It’s this: no single model found all five, but two different models together did. Sol plus Opus covered all five. Sol plus Astra did too.

What the test can’t tell you

It’s five pull requests. One defect of difference is twenty points of recall, so Sol’s 4 against Opus’s 3 is inside the noise. The grading was done by a Claude session checking each finding against the code, which is the right method but wasn’t blind to which model wrote what. The prompt was tuned after one of the five PRs, so that one is in-sample. And there’s a confound that favors Codex: it ran on a snapshot of the repository and could read files beyond the prompt, while Opus saw only the prompt.

So I don’t claim Codex is a better reviewer than Claude. I claim a second model from a different vendor, run once per PR, catches things the first one misses, at no cost to the quota I need for the work.

Reviewing the plan, before the code

The same wrapper can review a spec instead of a diff. I ran it on four plans from the same product, each against the code as it was before the work started, and checked the findings against what actually happened later.

About one finding in ten matched a flaw that a later commit fixed. About one in six was wrong. That’s useful, not a gate. But one result stood out: on a backlog item, it flagged that a newly saved credential would show as connected before validation. That’s the same bug that later shipped, and that only Sol caught in the diff test. At plan stage it was visible before any of that code was written.

It reviewed this update too

Today I refreshed the original article with the September numbers. Before publishing, I ran two checks: a Claude Opus fact-check against my telemetry ledger, and a Codex review of the change. Both came back with notes. Opus caught wording that overstated what the data showed. Codex caught a claim that still gave the whole improvement to two causes when four things changed that week, a precise percentage I can no longer re-verify, and a date bug on the insights index. It also flagged a naming “violation” that wasn’t one. I fixed the valid findings and skipped that one.

That’s the workflow in miniature: the writer’s own model checks the facts, a second vendor checks the reasoning, and a human decides.

How I run it now

• Once per pull request, never per commit. The per-event review is what burned the quota.
• Sol by default. Sol and Astra together only for the rare high-risk change.
• Report only. It reads a snapshot and writes a report. It can’t touch the repository.
• Read the findings, not the verdict. More than once a model printed “PASS” while listing real findings underneath.
• Verify before acting. Every finding gets checked against the code. About one in ten is wrong, and a confident wrong finding costs more than no finding.
• Own code only. Training is off in the vendor’s data settings, client repositories never go through it, and client names are filtered out of everything sent.

What didn’t matter

Local LLMs. I run two of them on my own GPUs, and in May I counted local delegation as a quota lever. In September it handled about 36K input tokens in a week against roughly 2 billion on Claude. The details are in the update to the original article. The local models still earn their keep on other work. They just don’t stretch the Claude quota, and they aren’t good enough to be the second reviewer.

Closing

The week after the lockout, usage per day was down about 70%. Sonnet went back to being the default, subagents got fewer and default to Sonnet, context gets compacted earlier, and the heavy review moved to a different vendor, once per PR. One partial week can’t split the 70% among those changes, so I’m re-measuring on a full week and tracking it weekly.

The pattern is the same one I use on client data work: measure the real cost of something opaque, find the few changes that matter, and put an independent check on the result before anyone relies on it. If your team is working through the same questions with AI-assisted engineering, let’s talk.