The outside check for AI code · deterministic · bring your own model
Know what the AI did to your codebase, before you merge it.
You can't read every diff the AI ships into code you inherited. So read what it did. Point doloop at a
repo you've never opened and learn what the change touched, which of the codebase's own conventions it
broke, and what now leans on it. Then a separate deterministic check judges whether the change fits the
codebase it landed in, and returns the same verdict every run. It runs outside the model. The thing
grading your AI isn't another AI you'd have to trust to grade the first one.
Fig. 1A tree: what the model writesA lattice: what your reviewer readsRead code you didn't write
Open a codebase you have never seen, and let it introduce itself.
Before the gate ever blocks a thing, doloop reads. Point it at a codebase you have never opened and in
about the time it takes to find the light switch, the place introduces itself, in the order an expert would
walk you through it: the doors you come in by, the engine room where the decisions get made, the words this
codebase uses for its own things. Not a summary it wrote. The same map of the actual code, drawn the same way
on every run.
doloop shows you where to look, and the same places every time. Deterministic, reproducible,
entirely on your own machine, not a token sent anywhere. Not a smarter reader than a strong model, a
reproducible one: the model tells you a story about the code; the read hands you the floor plan, the same
one on every open.
doloop read on flask, a repo it has never seen: the load-bearing files to start from, the conventions the codebase invented, what each file does, and the call graph, drawn the same way on every run.
The doors
Where you come in
The entry points, surfaced first, the way a guide opens with the front door and not the wiring. The few
places the rest of the codebase actually starts from.
The engine room
Where the decisions live
The handful of files the whole codebase turns on, picked out from the ordinary rooms, so you spend your
first minutes where the load is, not wandering the corridors.
The vocabulary
The words this codebase uses
The shared types and names the codebase speaks in, gathered up so you learn the local language before
you try to say anything in it.
Walk in three ways
Paste any public repository into the hosted read view and read the map with nothing to install, or run the
read entirely on your own machine where nothing leaves it, or paint it onto the file open in your editor.
doloop read . # the read, entirely local, nothing leaves your machine
The read view takes any public GitHub repo and draws the map, no install. The extension, doloop-decorations,
paints the read onto the open file on VS Code and Cursor. All three read the same way and never send your
source anywhere.
The whole map at a glance
Zoom out: the whole codebase as one map, and a lens for every question.
Every module is a tile, sized by how much code it holds. The map stays put. You switch the lens and the
same tiles recolor to answer a different question, so you read the system the way the question needs, all
of it computed locally in one pass.
Foundation - darker tiles are imported by more of the system. Read those first; the pale tiles are features on top.Your changes - the subsystems this branch touched, lit; the rest greyed, so you see the blast radius at a glance.Security - redder tiles carry more injection sinks to check. Same map, a different question.
Shown on scikit-learn, a repo doloop had never opened: one map, three of its lenses, the same read on every run.
Install in one line, gate on every commit
pip install doloopio
doloop gate --staged # gate your staged change against the codebase's own conventions
It runs locally: it reads your codebase, infers the conventions it already keeps, and flags a change that breaks them, with the rule, the line, and the rate. Your source never leaves your machine and we never train on it. Same input, same verdict, every run. Prefer a hosted endpoint? Send a single function to the API instead, nothing else leaves:
Tests answer one question: does the code do what it should. They have always been able to answer it. The
question they cannot answer is whether the code belongs in this codebase, and that is the question your
reviewer answers, one change at a time.
Cognition's FrontierCode benchmark (June 2026) measured the gap directly. Graded on whether a real maintainer
would merge the change, not just on whether it passes, the strongest model cleared only 13.4% of the hardest
set. The benchmark's canonical failure shows the part no test can reach: a patch that built cleanly, passed the
format checker and every test, and was still hard-blocked, because it routed the first line of a warning
through the codebase's logging helper and the rest of the warning around it. No test sees that. A maintainer
sees it instantly.
A randomized controlled trial measured the same gap from the other side. METR (2025) had sixteen
experienced developers work 246 tasks in repositories they had maintained for years, randomly
allowing or forbidding AI. Allowing it made them 19% slower, against their own forecast of a 24%
speedup. One factor the study weighs is the projects' very high quality standards: the implicit
requirements around testing, documentation, and formatting that a human takes substantial time to
learn and an AI rewrite does not meet.
doloop is built for that class of rejection: the no that no test can explain.
13.4%
Strongest model · the hardest set, maintainer-merge bar · FrontierCode · June 2026
19%
Slower with AI · experienced devs in mature repos they know · randomized trial · METR · 2025
Fig. 2One warning, two streams: line 1 through the helper, the rest around it
Your AI reviewer has an opinion. Run it twice on the same code and you can get two different scores. The check
has a verdict, the same code in and the same verdict out, with the rule and the line. A citation you can replay.
Why a bigger model will not close it
Scale learns every convention that recurs across the world's code, however subtle. It cannot learn the convention
that exists only in your repository, because that convention appears roughly once in any training set, and once
rounds to never (Kandpal et al., 2023, on long-tail knowledge). Reading your repository at generation time does not
close the gap either: the model still has to guess which of your repetitions are load-bearing and which are
accidents, and FrontierCode's agents had the full repository in front of them.
The gap is not a shortfall of capability. It is a fact about where your conventions live, which is in your code and
nowhere else. The convention that matters most on review is the one your codebase never wrote down.
Three kinds of rule
Fig. 3aCity blockCity code
What every codebase keeps
Written and checkable, wherever the line is drawn. A linter covers this, and a capable model already writes it correctly too.
Fig. 3bStreet of facadesThe neighborhood
What your codebase keeps
The conventions no one ever wrote down: how your team acts on errors, pairs an open with a close, routes a warning. No linter ships this rule, because nobody ever wrote it down. doloop infers it from your code, zero-config.
Fig. 3cCodebase planThe codebase
What this project keeps
The corrections you make as you go. doloop stores the rules you confirm and applies them on every later call. Store-and-apply ships today; automatic rule-learning is in build.
Solid: measured, shipsDashed: in buildHairline: structure only
Your code is perfectly valid, but it's not welcome in this codebase.
Every codebase settles into its own way of doing things: the conventions nobody wrote down but everybody on the
team follows. Your AI does not know them, so it writes code that is correct everywhere and at home nowhere. Valid
travels anywhere; welcome is local. Here is exactly that, set up on purpose.
Same guest, two codebases, opposite answers. Welcome is local.
flask and quart are sibling projects: quart is the async version of flask, so they share a design and even the
same function names. Take one shared function, dispatch_request. flask writes it the plain,
synchronous way; quart writes it the async way; and inside each project, every similar function matches that style.
Now drop flask's version, unchanged, into quart. It still compiles and runs, nothing is broken. doloop flags it
anyway, because it is written the way flask does things, not the way quart does. Drop quart's version into flask
and the flag jumps to the other side. The code never changed. Only the codebase around it did.
host: flask sync 100%
host: quart async 82%
flask dispatch_request
PASS
FLAGsync, here it is async
quart dispatch_request
FLAGasync, here it is sync
PASS
Read each column as one codebase judging both versions: the green PASS is the version written that codebase's own
way, the red FLAG is the visitor from the other codebase. A linter would pass all four, because a linter checks against
one fixed rulebook for the whole world. doloop checks against the codebase the change actually landed in, which is
what a human reviewer does, without the per-repo rule-writing a linter would need.
And this is not one lucky convention. Point doloop at a different pair with a different habit, and it flips again.
doit writes its status messages with print; nox writes the same kind of message through a logger. Paste
doit's code into nox and it is flagged for printing where this project logs, and the reverse flags too. A completely
unrelated convention, and the verdict still followed the codebase, because doloop is not applying one rule, it is reading
the codebase.
Fig.and the reverse flips the other waydoit/nox · leave-one-out · deterministic
Same code. The verdict follows the codebase.
Two convention classes, three sibling pairs (pilot, n=3 pairs): request-dispatch shape (flask/quart, requests/httpx) and diagnostic style, print vs log (doit/nox) · leave-one-out calibration · deterministic across re-runs
The same method holds across languages
This is not a Python trick. It is a property of reading a codebase rather than applying a fixed
rulebook, so it reproduces in languages that share nothing with each other. Each result is shown at its honest
strength, no further.
language
the convention that flips
shown at
Python
sync vs async dispatch; print vs log
demonstrated · 3 pairs, leave-one-out
TypeScript
arrow vs function-declaration helper (hono / kysely)
The same engine handles all three, from the language your team ships in to one written before it
existed, because it reads each codebase's own conventions rather than a language-specific rulebook.
And a handful of universal safety checks: the real Heartbleed function
Below is the load-bearing part of tls1_process_heartbeat from OpenSSL 1.0.1f (CVE-2014-0160). Run the
check. Then load the patched version and run it again. Same code in, the same verdict out. This demo runs the universal rules on one function; pointed at a whole repository, the gate also infers that codebase's own conventions and checks against those.
no key, deterministic, nothing stored.
runs the deterministic rules, no model.
It catches what you would have gone back to fix
470
flag-then-fix observations · 53 repositories · release over release
0.7%
false blocks · floored and hand-verified · 11 fires in 578 accepted commits, nine repos · an earlier four-repo holdout corroborated at 0.4%
Across the release history of mature open-source libraries (pilot, 53 repositories), in the cases we
checked the gate flagged a deviation in one release that the maintainers themselves deleted by the next.
We read this backward, against history that already happened; it is not a forecast of which lines will
break later. It does not invent a rule. It found the same spot the project went on to fix on its own, at
the commit rather than weeks into production.
doloop and your linter, where each one wins
Linters and Semgrep enforce the rules someone wrote down: universal patterns, a config, the same verdict
in every codebase. doloop enforces the rules nobody wrote down, the conventions your codebase already
keeps, inferred from its own code, gated in the same CI seat. Same statement, opposite verdicts in two
systems, because the rule comes from yours. They run together, and doloop is the layer after the linter.
Linters / Semgrep
doloop
Where the rule comes from
a human wrote it (a config, a ruleset)
inferred from your codebase's own consistency
Same code, two repos
one verdict (the rule is global)
opposite verdicts (read from each host)
Universal anti-patterns, CWE, taint flows
yes, their strength
a few safety lenses, not a full scanner
Your system's unwritten conventions
no (not without a custom rule written per repo)
yes, the layer after the linter
Config to maintain
you write and own the ruleset
zero, per system
CI gate that blocks
yes
yes, the same seat
Keep your linter and your taint scanner for the universal patterns they catch. doloop is the check for the
one thing they will not catch out of the box: the convention your codebase holds that nobody ever wrote down,
inferred zero-config instead of written as a custom rule per repo.
How it works
Infer
Reads your conventions from your code
doloop ships with no rules. It reads every file and works out the conventions your codebase already
holds itself to, the ones it follows in at least 70% of the places they apply. That inferred standard
is the only thing it checks against.
Check
Flags only where a change breaks them
On a commit it looks at the changed functions and flags the ones that break a convention the rest of
your code keeps, and shows you the rate. If your whole project does something one way, one more the
same way is fine. It speaks up only where you are out of step with yourself.
Block
Cites the rule and the line
It blocks the commit on the safety breaks, the conventions that hold across codebases, and warns on
the rest. Every finding names the convention, the line, and the rate. The verdict re-runs to the same
answer, every time.
Two runs on the same code give the same verdict and the same citations. A teammate or a
regulator re-runs it and gets your exact result. A record you can audit.
Ship velocity with confidence
The point is to keep your AI's pace without inheriting its blind spots. The gate reads a whole codebase and
calibrates it in a fraction of a second, so it runs on every commit without slowing you down. Same code in,
same verdict out, in under a second, where an AI reviewer takes minutes and hands you a different answer
each run.
The loop
Your AI generates. doloop checks the change against your codebase's own standard and
feeds the specific problem back, located and cited. Your AI revises. Generate, check against the whole, adapt.
The loop is the method, and the method is the name.
Fig. 4The do-loop · generate → check → adapt
From the API, in your terminal or your CI
Send only the function you want checked. It is processed and nothing is stored, and the verdict is the same on every run, so it drops into a CI step or a pre-commit hook by blocking when the verdict is not a pass. For a sealed deployment where your source never leaves your machine, become a design partner.
A linter runs one rulebook that a human configures by hand. doloop writes no rules. It reads your codebase, infers the conventions you already keep, and flags a change that breaks them. The proof: the same function passes in one codebase and is flagged in a sibling, because the verdict follows the codebase. A linter gives both the same verdict, because it applies one fixed rule everywhere.
Why not just use an LLM to review the code?
A strong model is a capable reviewer. The catch is reproducibility: ask it twice and you can get two different verdicts, so you cannot gate a pipeline on it and you cannot hand an auditor a result that replays. doloop returns the same verdict, with the rule, the line, and the rate behind it, on every run.
Won't my tests catch this?
Tests check that the code does what it should. They pass on code that works but does not fit: an async handler in a synchronous codebase, a print where the codebase logs, an END-IF in a system that ends every IF with a period. That gap is invisible to a test and is exactly what a reviewer rejects a passing change for.
How noisy is it, and does it impose someone else's idea of good code?
It never enforces a convention your codebase does not already keep. If your codebase swallows errors, it will not tell you to stop. It only flags a change that breaks a rule your own code holds in at least 70% of the places it applies. Below that floor it stays silent. On public release history, hand-verified, the false-block rate floors at about 0.7%. That is the false-block floor; whether it catches the breaks your own reviewers would reject, the number that actually decides this, is what the design-partner program measures on your own code.
Doesn't a better model make this obsolete? Won't the company that makes the model just build it in?
Two questions hide in that one. Does a better model close the gap? No: sampling is non-deterministic however good the model gets, so the gap is reproducibility, not capability, and scale does not touch it. We measured it. A strong model is an accurate reader, but across identical runs it disagreed with itself on about a fifth of programs, verdict-changingly. A 2027 model will be more accurate and exactly as non-reproducible. Could a model's maker build a separate deterministic gate? Yes. What holds the position is not a technical fortress, it is independence (a lab gating its own model's output is the maker auditing itself, and regulated review requires independent validation as a control, so a built-in feature cannot satisfy that need), a per-tenant convention memory that compounds with use, a wall that deepens over time rather than one that stands on day one, and the fact that doloop runs on any model or none where a built-in feature locks to one lab.
Code review has real human value. Aren't you trying to remove it?
No. doloop does not replace review. It removes the convention-fit checks that burn a senior reviewer's limited time, so those hours go to the part only a human does: intent, architecture, and whether the change is the right thing to build at all.
Where does my code go, and is it ready?
The gate runs locally. Your source never leaves your machine, and we never train on it. Bring your own model. The Python gate is live and the method is demonstrated in Python, first-celled in TypeScript, and verified three ways in COBOL. The wider language and prose surfaces are on the roadmap, stated at their honest stage on the paper.
Become a design partner
We are taking on a small number of teams as design partners, in any language. The convention inference
runs on any codebase; the lenses are deepest in Python today, and for other languages we build them with
you on your own code. You get a free instrumented run on your repo and your codebase's own consistency
report, the conventions it already holds. In return we
measure the one number that decides whether this is real: does the gate catch the convention breaks your
reviewers actually reject. Your source never leaves your machine.
Routes you to a real doloop page, asks when your question is ambiguous, or tells you when there is no answer. No model runs on the answer path, so it cannot invent one.