The outside check for AI code · deterministic · bring your own model

Know what the AI did to your codebase, before you merge it.

You can't read every diff the AI ships into code you inherited. So read what it did. Point doloop at a repo you've never opened and learn what the change touched, which of the codebase's own conventions it broke, and what now leans on it. Then a separate deterministic check judges whether the change fits the codebase it landed in, and returns the same verdict every run. It runs outside the model. The thing grading your AI isn't another AI you'd have to trust to grade the first one.

Same code in, same verdict out
Fig. 1 A tree: what the model writes A lattice: what your reviewer reads
Read code you didn't write

Open a codebase you have never seen, and let it introduce itself.

Before the gate ever blocks a thing, doloop reads. Point it at a codebase you have never opened and in about the time it takes to find the light switch, the place introduces itself, in the order an expert would walk you through it: the doors you come in by, the engine room where the decisions get made, the words this codebase uses for its own things. Not a summary it wrote. The same map of the actual code, drawn the same way on every run.

doloop shows you where to look, and the same places every time. Deterministic, reproducible, entirely on your own machine, not a token sent anywhere. Not a smarter reader than a strong model, a reproducible one: the model tells you a story about the code; the read hands you the floor plan, the same one on every open.

doloop read on the flask codebase: an architecture spine of the load-bearing files, the conventions the codebase invented and where they drift, what a file does, the call graph, and the files no lens flags - computed locally in one pass
doloop read on flask, a repo it has never seen: the load-bearing files to start from, the conventions the codebase invented, what each file does, and the call graph, drawn the same way on every run.
The doors

Where you come in

The entry points, surfaced first, the way a guide opens with the front door and not the wiring. The few places the rest of the codebase actually starts from.

The engine room

Where the decisions live

The handful of files the whole codebase turns on, picked out from the ordinary rooms, so you spend your first minutes where the load is, not wandering the corridors.

The vocabulary

The words this codebase uses

The shared types and names the codebase speaks in, gathered up so you learn the local language before you try to say anything in it.

Walk in three ways

Paste any public repository into the hosted read view and read the map with nothing to install, or run the read entirely on your own machine where nothing leaves it, or paint it onto the file open in your editor.

doloop read .                 # the read, entirely local, nothing leaves your machine

The read view takes any public GitHub repo and draws the map, no install. The extension, doloop-decorations, paints the read onto the open file on VS Code and Cursor. All three read the same way and never send your source anywhere.

The whole map at a glance

Zoom out: the whole codebase as one map, and a lens for every question.

Every module is a tile, sized by how much code it holds. The map stays put. You switch the lens and the same tiles recolor to answer a different question, so you read the system the way the question needs, all of it computed locally in one pass.

The scikit-learn codebase as a squarified treemap under the Foundation lens: tiles darker where more of the system imports them, with utils and the shared core the darkest
Foundation - darker tiles are imported by more of the system. Read those first; the pale tiles are features on top.
The same treemap under the Your-changes lens: the subsystems this branch touched lit in orange, everything else greyed
Your changes - the subsystems this branch touched, lit; the rest greyed, so you see the blast radius at a glance.
The same treemap under the Security lens: tiles redder where they carry more injection sinks, utils the reddest
Security - redder tiles carry more injection sinks to check. Same map, a different question.

Shown on scikit-learn, a repo doloop had never opened: one map, three of its lenses, the same read on every run.

Install in one line, gate on every commit

pip install doloopio
doloop gate --staged          # gate your staged change against the codebase's own conventions

It runs locally: it reads your codebase, infers the conventions it already keeps, and flags a change that breaks them, with the rule, the line, and the rate. Your source never leaves your machine and we never train on it. Same input, same verdict, every run. Prefer a hosted endpoint? Send a single function to the API instead, nothing else leaves:

curl -s https://api.doloop.io/v1/check-code \
  -H "content-type: application/json" \
  -d '{"code": "<your function>"}'

The full command reference, with real output →

The gap tests cannot see

Tests answer one question: does the code do what it should. They have always been able to answer it. The question they cannot answer is whether the code belongs in this codebase, and that is the question your reviewer answers, one change at a time.

45% flawedIn a controlled study, 45% of AI-generated code introduced a known security weakness, code that compiles and passes its tests. Veracode, 2025.evidence →

Cognition's FrontierCode benchmark (June 2026) measured the gap directly. Graded on whether a real maintainer would merge the change, not just on whether it passes, the strongest model cleared only 13.4% of the hardest set. The benchmark's canonical failure shows the part no test can reach: a patch that built cleanly, passed the format checker and every test, and was still hard-blocked, because it routed the first line of a warning through the codebase's logging helper and the rest of the warning around it. No test sees that. A maintainer sees it instantly.

A randomized controlled trial measured the same gap from the other side. METR (2025) had sixteen experienced developers work 246 tasks in repositories they had maintained for years, randomly allowing or forbidding AI. Allowing it made them 19% slower, against their own forecast of a 24% speedup. One factor the study weighs is the projects' very high quality standards: the implicit requirements around testing, documentation, and formatting that a human takes substantial time to learn and an AI rewrite does not meet.

doloop is built for that class of rejection: the no that no test can explain.

13.4%
Strongest model · the hardest set, maintainer-merge bar · FrontierCode · June 2026
19%
Slower with AI · experienced devs in mature repos they know · randomized trial · METR · 2025
WARNING log_warn( ) line 1 the rest OUTPUT
Fig. 2 One warning, two streams: line 1 through the helper, the rest around it

Your AI reviewer has an opinion. Run it twice on the same code and you can get two different scores. The check has a verdict, the same code in and the same verdict out, with the rule and the line. A citation you can replay.

Why a bigger model will not close it

Scale learns every convention that recurs across the world's code, however subtle. It cannot learn the convention that exists only in your repository, because that convention appears roughly once in any training set, and once rounds to never (Kandpal et al., 2023, on long-tail knowledge). Reading your repository at generation time does not close the gap either: the model still has to guess which of your repetitions are load-bearing and which are accidents, and FrontierCode's agents had the full repository in front of them.

The gap is not a shortfall of capability. It is a fact about where your conventions live, which is in your code and nowhere else. The convention that matters most on review is the one your codebase never wrote down.

Three kinds of rule
Fig. 3aCity block
City code

What every codebase keeps

Written and checkable, wherever the line is drawn. A linter covers this, and a capable model already writes it correctly too.

Fig. 3bStreet of facades
The neighborhood

What your codebase keeps

The conventions no one ever wrote down: how your team acts on errors, pairs an open with a close, routes a warning. No linter ships this rule, because nobody ever wrote it down. doloop infers it from your code, zero-config.

Fig. 3cCodebase plan
The codebase

What this project keeps

The corrections you make as you go. doloop stores the rules you confirm and applies them on every later call. Store-and-apply ships today; automatic rule-learning is in build.

Solid: measured, ships Dashed: in build Hairline: structure only

Your code is perfectly valid, but it's not welcome in this codebase.

Every codebase settles into its own way of doing things: the conventions nobody wrote down but everybody on the team follows. Your AI does not know them, so it writes code that is correct everywhere and at home nowhere. Valid travels anywhere; welcome is local. Here is exactly that, set up on purpose.

A doorman raises a hand to turn away a man in a suit: I'm sorry, that's async, this is a synchronous establishment The same doorman waves in a man carrying a flask: right this way, sir
Same guest, two codebases, opposite answers. Welcome is local.

flask and quart are sibling projects: quart is the async version of flask, so they share a design and even the same function names. Take one shared function, dispatch_request. flask writes it the plain, synchronous way; quart writes it the async way; and inside each project, every similar function matches that style.

Now drop flask's version, unchanged, into quart. It still compiles and runs, nothing is broken. doloop flags it anyway, because it is written the way flask does things, not the way quart does. Drop quart's version into flask and the flag jumps to the other side. The code never changed. Only the codebase around it did.

host: flask sync 100% host: quart async 82%
flask dispatch_request PASS FLAG sync, here it is async
quart dispatch_request FLAG async, here it is sync PASS

Read each column as one codebase judging both versions: the green PASS is the version written that codebase's own way, the red FLAG is the visitor from the other codebase. A linter would pass all four, because a linter checks against one fixed rulebook for the whole world. doloop checks against the codebase the change actually landed in, which is what a human reviewer does, without the per-repo rule-writing a linter would need.

And this is not one lucky convention. Point doloop at a different pair with a different habit, and it flips again. doit writes its status messages with print; nox writes the same kind of message through a logger. Paste doit's code into nox and it is flagged for printing where this project logs, and the reverse flags too. A completely unrelated convention, and the verdict still followed the codebase, because doloop is not applying one rule, it is reading the codebase.

doit · prints 88% nox · logs 82% execute() execute() same bytes PASS FLAG · prints, this codebase logs
Fig. and the reverse flips the other way doit/nox · leave-one-out · deterministic

Same code. The verdict follows the codebase.
Two convention classes, three sibling pairs (pilot, n=3 pairs): request-dispatch shape (flask/quart, requests/httpx) and diagnostic style, print vs log (doit/nox) · leave-one-out calibration · deterministic across re-runs

The same method holds across languages

This is not a Python trick. It is a property of reading a codebase rather than applying a fixed rulebook, so it reproduces in languages that share nothing with each other. Each result is shown at its honest strength, no further.

language the convention that flips shown at
Python sync vs async dispatch; print vs log demonstrated · 3 pairs, leave-one-out
TypeScript arrow vs function-declaration helper (hono / kysely) first-cell (pilot, n=1 pair) · survives ESLint-subtraction
COBOL bare-period vs END-IF (legacy vs modern dialect) first-cell · 525/525 leave-one-out · verified 3 ways: tokenizer, GnuCOBOL, ANTLR AST

The same engine handles all three, from the language your team ships in to one written before it existed, because it reads each codebase's own conventions rather than a language-specific rulebook.

And a handful of universal safety checks: the real Heartbleed function

Below is the load-bearing part of tls1_process_heartbeat from OpenSSL 1.0.1f (CVE-2014-0160). Run the check. Then load the patched version and run it again. Same code in, the same verdict out. This demo runs the universal rules on one function; pointed at a whole repository, the gate also infers that codebase's own conventions and checks against those.

no key, deterministic, nothing stored.
runs the deterministic rules, no model.

It catches what you would have gone back to fix

470
flag-then-fix observations · 53 repositories · release over release
0.7%
false blocks · floored and hand-verified · 11 fires in 578 accepted commits, nine repos · an earlier four-repo holdout corroborated at 0.4%

Across the release history of mature open-source libraries (pilot, 53 repositories), in the cases we checked the gate flagged a deviation in one release that the maintainers themselves deleted by the next. We read this backward, against history that already happened; it is not a forecast of which lines will break later. It does not invent a rule. It found the same spot the project went on to fix on its own, at the commit rather than weeks into production.

doloop and your linter, where each one wins

Linters and Semgrep enforce the rules someone wrote down: universal patterns, a config, the same verdict in every codebase. doloop enforces the rules nobody wrote down, the conventions your codebase already keeps, inferred from its own code, gated in the same CI seat. Same statement, opposite verdicts in two systems, because the rule comes from yours. They run together, and doloop is the layer after the linter.

Linters / Semgrep doloop
Where the rule comes from a human wrote it (a config, a ruleset) inferred from your codebase's own consistency
Same code, two repos one verdict (the rule is global) opposite verdicts (read from each host)
Universal anti-patterns, CWE, taint flows yes, their strength a few safety lenses, not a full scanner
Your system's unwritten conventions no (not without a custom rule written per repo) yes, the layer after the linter
Config to maintain you write and own the ruleset zero, per system
CI gate that blocks yes yes, the same seat

Keep your linter and your taint scanner for the universal patterns they catch. doloop is the check for the one thing they will not catch out of the box: the convention your codebase holds that nobody ever wrote down, inferred zero-config instead of written as a custom rule per repo.

How it works

Infer

Reads your conventions from your code

doloop ships with no rules. It reads every file and works out the conventions your codebase already holds itself to, the ones it follows in at least 70% of the places they apply. That inferred standard is the only thing it checks against.

Check

Flags only where a change breaks them

On a commit it looks at the changed functions and flags the ones that break a convention the rest of your code keeps, and shows you the rate. If your whole project does something one way, one more the same way is fine. It speaks up only where you are out of step with yourself.

Block

Cites the rule and the line

It blocks the commit on the safety breaks, the conventions that hold across codebases, and warns on the rest. Every finding names the convention, the line, and the rate. The verdict re-runs to the same answer, every time.

Two runs on the same code give the same verdict and the same citations. A teammate or a regulator re-runs it and gets your exact result. A record you can audit.

Ship velocity with confidence

The point is to keep your AI's pace without inheriting its blind spots. The gate reads a whole codebase and calibrates it in a fraction of a second, so it runs on every commit without slowing you down. Same code in, same verdict out, in under a second, where an AI reviewer takes minutes and hands you a different answer each run.

The loop

Your AI generates. doloop checks the change against your codebase's own standard and feeds the specific problem back, located and cited. Your AI revises. Generate, check against the whole, adapt. The loop is the method, and the method is the name.

GENERATE { } CHECK ADAPT
Fig. 4The do-loop · generate → check → adapt

From the API, in your terminal or your CI

Send only the function you want checked. It is processed and nothing is stored, and the verdict is the same on every run, so it drops into a CI step or a pre-commit hook by blocking when the verdict is not a pass. For a sealed deployment where your source never leaves your machine, become a design partner.

  curl https://api.doloop.io/v1/check-code \
    -H 'content-type: application/json' \
    -d '{"code": "...your function..."}'

The theory behind it, for the regulated and academic reader: the method behind it →

The questions a skeptic asks first

Isn't this just a linter?

A linter runs one rulebook that a human configures by hand. doloop writes no rules. It reads your codebase, infers the conventions you already keep, and flags a change that breaks them. The proof: the same function passes in one codebase and is flagged in a sibling, because the verdict follows the codebase. A linter gives both the same verdict, because it applies one fixed rule everywhere.

Why not just use an LLM to review the code?

A strong model is a capable reviewer. The catch is reproducibility: ask it twice and you can get two different verdicts, so you cannot gate a pipeline on it and you cannot hand an auditor a result that replays. doloop returns the same verdict, with the rule, the line, and the rate behind it, on every run.

Won't my tests catch this?

Tests check that the code does what it should. They pass on code that works but does not fit: an async handler in a synchronous codebase, a print where the codebase logs, an END-IF in a system that ends every IF with a period. That gap is invisible to a test and is exactly what a reviewer rejects a passing change for.

How noisy is it, and does it impose someone else's idea of good code?

It never enforces a convention your codebase does not already keep. If your codebase swallows errors, it will not tell you to stop. It only flags a change that breaks a rule your own code holds in at least 70% of the places it applies. Below that floor it stays silent. On public release history, hand-verified, the false-block rate floors at about 0.7%. That is the false-block floor; whether it catches the breaks your own reviewers would reject, the number that actually decides this, is what the design-partner program measures on your own code.

Doesn't a better model make this obsolete? Won't the company that makes the model just build it in?

Two questions hide in that one. Does a better model close the gap? No: sampling is non-deterministic however good the model gets, so the gap is reproducibility, not capability, and scale does not touch it. We measured it. A strong model is an accurate reader, but across identical runs it disagreed with itself on about a fifth of programs, verdict-changingly. A 2027 model will be more accurate and exactly as non-reproducible. Could a model's maker build a separate deterministic gate? Yes. What holds the position is not a technical fortress, it is independence (a lab gating its own model's output is the maker auditing itself, and regulated review requires independent validation as a control, so a built-in feature cannot satisfy that need), a per-tenant convention memory that compounds with use, a wall that deepens over time rather than one that stands on day one, and the fact that doloop runs on any model or none where a built-in feature locks to one lab.

Code review has real human value. Aren't you trying to remove it?

No. doloop does not replace review. It removes the convention-fit checks that burn a senior reviewer's limited time, so those hours go to the part only a human does: intent, architecture, and whether the change is the right thing to build at all.

Where does my code go, and is it ready?

The gate runs locally. Your source never leaves your machine, and we never train on it. Bring your own model. The Python gate is live and the method is demonstrated in Python, first-celled in TypeScript, and verified three ways in COBOL. The wider language and prose surfaces are on the roadmap, stated at their honest stage on the paper.

Become a design partner

We are taking on a small number of teams as design partners, in any language. The convention inference runs on any codebase; the lenses are deepest in Python today, and for other languages we build them with you on your own code. You get a free instrumented run on your repo and your codebase's own consistency report, the conventions it already holds. In return we measure the one number that decides whether this is real: does the gate catch the convention breaks your reviewers actually reject. Your source never leaves your machine.