The Machine That Reads Code

Inside AI-driven vulnerability discovery — the Carlini Loop, the collapse of the verification bottleneck, and what it changes for the people who hunt bugs.

A for loop in bash found, inside the Linux kernel, a bug that had survived 23 years of human review, continuous fuzzing, and static analysis. The code wasn't sophisticated. The capability behind it was.

Somewhere between the end of 2025 and the start of 2026, the question that organizes vulnerability research moved. For decades the central problem was finding: locating the rare flaw, the edge case nobody considered, the implicit assumption that breaks under hostile input. Finding was expensive, slow, and dependent on a handful of specialists with trained intuition. In 2026, finding became a commodity. The bottleneck now is proving — separating the real discovery from the confident noise a model produces when it isn't sure.

This article is a technical dissection of that shift. I'll trace the methodology that made it practical — the so-called Carlini Loop — examine why it works so well, and then spend most of the text on the problem that actually matters: verification. Because it's precisely on that frontier — between the plausible hypothesis and the proven impact — that both defense and the craft of bug bounty reorganize from here on out.

1. The day Ghost fell before the coffee got cold

At [un]prompted 2026 in San Francisco, Nicholas Carlini — research scientist at Anthropic — presented a striking result: pointed at Ghost, the open-source CMS, a model had surfaced a blind SQL injection in the Content API (later assigned CVE-2026-26980) that let an unauthenticated user compromise the admin database, forge an admin account, and take full control. The model reached it in about 90 minutes — a previously unreported critical issue in a platform that publishes a large slice of the independent web.

The demo was the public face of the paper from Anthropic's Frontier Red Team (Feb 5, 2026): "Evaluating and mitigating the growing risk of LLM-discovered 0-days." Pointing Claude Opus 4.6 at production open-source codebases, with nothing more than a VM and standard tooling, the team reported over 500 high-severity vulnerabilities, independently validated — among them a flaw that had survived 23 years in the Linux kernel: a heap overflow in the NFSv4.0 LOCK replay cache (CVE-2026-31402), missed by decades of human review, continuous fuzzing and static analysis.

For comparison — and the goals differ, since Project Zero optimizes for depth and weaponization on a curated target list while the loop optimizes for breadth: Google's Project Zero reports on the order of 20–30 high-impact vulnerabilities per year; Claude validated 500+ in weeks. The Linux kernel is the most heavily audited software on the planet; a for loop found what everyone else had missed.

A number like 500 is built to travel — and that is exactly when a researcher should slow down. The paper's framing matters more than the headline: these are high-severity findings, independently validated, but their exposure isn't uniform. Some sit in code that runs serious production infrastructure; others are lower-stakes denial-of-service in software almost no one deploys. The honest reading isn't "500 catastrophes" — it's that the cost of looking collapsed, so the volume of plausible candidates exploded. Which is precisely why the rest of this article is about what happens after the find.

2. The lineage: this didn't come out of nowhere

This capability has a visible trajectory.

• 2024 — Big Sleep (SQLite): an early, widely cited demonstration of an LLM-assisted workflow finding a stack buffer underflow in real code.
• 2025 — the o3 0-day (CVE-2025-37899): Sean Heelan reported a genuine zero-day — a use-after-free in the Linux kernel's ksmbd SMB server (session handling) — with the help of a reasoning model.
• 2025–2026 — Firefox: Claude Opus 4.6 surfaced 22 vulnerabilities, 14 of them high-severity — but turning them into working exploits proved costly and mostly failed (the receipt is in §5).
• 2026 — scale: the 500+ 0-day paper, plus unconfirmed indications that the next generation (the Mythos line, in restricted preview) scaled to thousands of vulnerabilities across more classes (RCE, privilege escalation, DoS) — exactly the kind of unverified "autonomous" claim that §5.1 below teaches you to discount until an artifact actually runs.

Each step raised the difficulty of the targets and broadened the diversity of the classes.

3. Anatomy of the Carlini Loop

The Carlini Loop is, at its core, a bash loop that walks every source file and asks the model to find vulnerabilities in that file, framed as a CTF.

# Conceptual — this is not the production pipeline, it's the idea
for file in $(find ./target -name '*.c' -o -name '*.py' -o -name '*.js'); do
    context=$(cat "$file")
    llm_prompt \
        --system "You are a CTF competitor. Find exploitable
                  vulnerabilities in this file. For each one, describe the
                  class, the source, the sink, and the precondition for
                  exploitation." \
        --input "$context" \
        >> candidates/"$(basename "$file").md"
done

The naive intuition would be to dump the entire repository into the context. That fails: it blows past the window and dilutes attention. The Carlini Loop decomposes the work into small, independent units that the context window can fully attend to. Three decisions make it work:

1. Granularity — file by file keeps each unit below the threshold where reasoning degrades.
2. CTF framing — shifts the register from "polite reviewer" to "attacker looking for impact."
3. Forced output structure (class, source, sink, precondition) — makes the output triageable.

The simplicity is the dangerous part: a capability once restricted to specialists is now accessible to anyone with a coding-agent subscription.

4. Why it works: the economics of context and reasoning

Vulnerability research is, fundamentally, tracing the flow of untrusted data from a source to a sink. Frontier models are excellent at this:

• Reading speed — they shine on old protocol code, serialization/deserialization, replay caches.
• No fatigue, no familiarity bias — the maintainer stops seeing the module; the model arrives with no history.
• Cheap exhaustive coverage — Project Zero picks its targets; the Carlini Loop sweeps everything.

Hence the meme: the price of a zero-day became an API call.

A calibration note: METR's "time horizon" metric measures the human difficulty of a task, not uninterrupted autonomy. It's a useful yardstick, but it isn't a claim that the model runs unattended for hours on end.

5. The real bottleneck: verification

The clearest proof that the bottleneck moved is that the gap has been measured in dollars. When Anthropic pointed Claude Opus 4.6 at Firefox, it scanned nearly 6,000 C++ files and filed 112 reports in two weeks — 22 confirmed vulnerabilities, 14 of them high-severity, roughly a fifth of all high-severity Firefox fixes that year. Cheap, fast, abundant. Then the same team tried to exploit those bugs: a VM, a task verifier, around 350 attempts per bug — and the model produced working exploits in exactly two cases, burning about $4,000 in API credits, and even those only ran with Firefox's sandbox deliberately removed. Anthropic's own reading: finding a vulnerability is roughly an order of magnitude cheaper than building an exploit, and the model is far better at the former. Discovery in minutes for pennies a file; weaponization for thousands of dollars and mostly failure. That is the thesis with a receipt.

The same shape shows up in the headline numbers. Finding candidates became trivial; proving them real, exploitable, and impactful is the hard part. Carlini said it plainly: beyond the 500+ validated, hundreds were left unverified for lack of time. The bottleneck became humans processing the flood of candidates.

The key distinction:

Logic-proven — the reasoning closes, it sounds correct. Execution-proven — there is an artifact: a PoC that runs in a reproducible environment.

Models produce logic-proven output because they were trained to sound coherent. Confident coherence is the failure mode.

5.1 A real exploit is a loop, not a shot

The clearest public example arrived weeks later — and it cuts against the way it was sold. The FreeBSD RPCSEC_GSS flaw (CVE-2026-4747) was found by Carlini with Claude; the FreeBSD advisory of March 26, 2026 credits "Nicholas Carlini using Claude, Anthropic." The exploit came separately: on March 29, Calif.io's MAD Bugs project asked Claude (Opus 4.6) to weaponize the already-disclosed, already-patched CVE — and it took roughly four hours of anything but one-shot work.

The write-up reads exactly like the verification pipeline described later in this section. The model sent a De Bruijn pattern as the credential body, watched the kernel panic in a QEMU console, read the faulting instruction pointer, and recalculated the overflow offset from 168 to 200 once it realized a 32-byte GSS header had never been counted in its static analysis. That is the candidate → execute → read-the-error → adjust loop, not a single confident guess.

Two caveats the headline dropped, both of which matter to a practitioner. The target was soft: FreeBSD 14.x ships without the KASLR and stack canaries a modern Linux kernel would have. And the CVE was handed to the model — this was weaponizing a known, patched bug, not discovering and exploiting one end-to-end. The lesson isn't "AI cracked a kernel from scratch." It's that even with the model doing the heavy lifting, impact came from iterating against real execution. Verification is the work.

5.2 Why the model hallucinates exploits

The model invents connections: it assumes a parameter reaches a sink when a validation actually blocks it; it presumes a controllable source when the input is in fact trusted internal state. The rule of thumb: the model's output belongs in your notes, not in your findings. Generated text is a hypothesis, never evidence.

5.3 The architecture of adversarial verification

The best systems of 2026 are closed-loop pipelines that combine semantic reasoning with execution-based validation. OpenAnt (Apache 2.0) works in three stages:

1. Decomposition with a reachability filter — discards anything not reachable from external entry points, shrinking the surface by up to 97%.
2. Adversarial verification — evaluates exploitability under the realistic capabilities of a constrained attacker.
3. Dynamic verification — generated environments, executed in sandboxed containers and discarded afterward; a candidate survives only if the exploit actually runs.

OpenAnt's thesis: adversarial verification is what distinguishes theoretical vulnerabilities from exploitable ones.

Others follow the same logic. PoC-GYM generates exploits iteratively (candidate → execute → error fed back as feedback → repeat). Approaches with formal verification guarantee both that the flaw exists and that the fix is correct. The common principle: detection and validation live in the same loop.

5.4 What the benchmarks reveal

ZeroDayBench (GPT-5.2, Claude Sonnet 4.5, Grok 4.1 Fast tested on previously unseen zero-days): every model does better as it receives more information; under low-information conditions, performance drops. The implication: the quality of the recon sets the ceiling for the model.

SandboxEscapeBench (UK AI Security Institute, 9 LLMs, 18 container-escape scenarios): on common misconfigurations (exposed Docker socket, privileged container, writable host mount), frontier models reached 100% escape; on scenarios involving kernel CVEs, even older models still escaped 40% of the time. The recommendation: treat plain Docker isolation as insufficient by default.

6. The players

Anthropic Frontier Red Team (Carlini et al.): published the 500+ 0-days; documents the double-edged nature of the work publicly; reported in November 2025 that it had disrupted the first large-scale cyberattack run without substantial human intervention (attackers using Claude Code against roughly 30 targets).

AISLE (OpenSSL): ran autonomous analysis of OpenSSL through late 2025; the findings shipped in OpenSSL's coordinated release of January 27, 2026 (which patched the 3.6.1, 3.5.5, 3.4.4, 3.3.6 and 3.0.19 lines — 3.6.1 was just one of them). AISLE is credited with originating 13 of the 14 OpenSSL CVEs assigned in 2025 (93%); several carry co-credit with other researchers, so read that as discovery share, not sole credit.

XBOW (HackerOne): an autonomous pentester founded in January 2024 by Oege de Moor (founder of Semmle, creator of CodeQL), with CISO Nico Waisman. It reached #1 on the HackerOne leaderboard (US and global), filed 1,000+ validated reports in months, and over two years responsibly disclosed 14,000+ vulnerabilities, with exploit chains up to 48 steps — on the back of a $120M Series C at a $1B+ valuation (figures per XBOW's own announcements and HackerOne's leaderboard; see Sources). Three numbers tell the rest of the story:

• Speed — 28 minutes versus 40 hours for an equivalent human effort.
• Precision — about 25% of findings land as "informative" / "not applicable": it shines on SQLi/XSS/RCE and stumbles on privacy and business logic.
• Discipline — it does not submit a single report without human review.

7. The systemic consequence: 90-day disclosure is breaking

The 90-day window assumes scarcity. AI breaks that assumption: when a system finds 12 CVEs in a single OpenSSL release, or 500 across a handful of codebases, the queue turns into an open fire hose. Anthropic argues that the 90-day norm may not survive. HackerOne reports 580,000+ validated vulnerabilities in its database and 210% growth in valid AI-generated reports. Anthropic launched Project Glasswing (with partners such as Cisco) to coordinate protection of critical infrastructure.

8. The knife that cuts both ways

The capability is symmetrical: the same loop that finds and reports can find and exploit. When offensive scanners mature — and today, across many AI surfaces, they still don't, which leaves a rare window for anyone entering now — the advantage migrates from "who finds it first" to "who fixes it and proves impact fastest." Continuous defense, not point-in-time pentesting. For the ethical practitioner, the differentiator is not access to the capability — it's the discipline with which it's used.

9. What this means for bug hunters

If discovery became a commodity and verification became the bottleneck, the strategy inverts. Four principles:

1. Two gates before submitting — an eligibility gate (scope, accepted class, source actually controllable) and a proven-impact gate (is there an artifact? does the exploit run in a reproducible environment?).
2. Execution-proven or don't submit — the model hands you logic-proven for free; the second is your work.
3. Quality recon is the model's ceiling — AI amplifies context, it doesn't compensate for it; garbage in, confident garbage out.
4. The human edge moved to judgment — XBOW fails exactly where context matters (privacy, business logic, the meaning of the impact). The machine reads the structure; the human reads the meaning.

10. Conclusion: the craft shifts, it doesn't disappear

Finding vulnerabilities was never the job — it was the prerequisite. The job was always to prove: to build the irrefutable argument, demonstrate impact, translate a technical anomaly into a consequence for the people who decide. AI automated the prerequisite and exposed where the value had been all along.

A for loop in bash found a 23-year-old bug — which is both frightening (abundance breaks systems that assumed scarcity) and freeing (it returns the focus to the part that always mattered and remains human: the judgment that something is real, that it matters, and that it can be proven). The machine reads the code. It still falls to you to say what it means.

Sources and further reading

Primary

• Anthropic / Frontier Red Team — "Evaluating and mitigating the growing risk of LLM-discovered 0-days" (Carlini et al., Feb 5, 2026) — red.anthropic.com
• FreeBSD Security Advisory — RPCSEC_GSS overflow, CVE-2026-4747 (Mar 26, 2026); credits "Nicholas Carlini using Claude, Anthropic"
• Calif.io "MAD Bugs" — building a FreeBSD root exploit for the (already-patched) CVE-2026-4747 (Mar 29, 2026)
• NVD / MITRE — Ghost CVE-2026-26980; Linux NFSv4.0 LOCK replay-cache heap overflow CVE-2026-31402; o3 0-day CVE-2025-37899 (ksmbd)
• AISLE — autonomous OpenSSL analysis; OpenSSL coordinated release Jan 27, 2026 (credited with originating 13 of the 14 OpenSSL CVEs of 2025)
• HackerOne — AI vulnerability report + leaderboard; XBOW — funding, leaderboard and disclosure figures per the company's own announcements
• OpenAnt — preprint + source: github.com/knostic/OpenAnt
• ZeroDayBench; SandboxEscapeBench — UK AI Security Institute (preprints, 2026)
• Big Sleep — SQLite stack buffer underflow (Project Zero + DeepMind, 2024)

Commentary

• danilchenko.dev · emsi.me · harrisonaix.com · kenhuangus.substack.com · edera.dev · penligent.ai

Written by Heric Baldez — offensive security research, OSINT and digital investigation.