In three weeks, the coding agents in our AI software factory took 57 issues from our own backlog, and 54 ended in a merged change. Every agent works in a full local setup of Numonic, records proof of its work for the merge request, and gets review findings at medium or above sent back to it before a person sees the work. This post shows how the factory works, stage by stage, and the full count, failures included.
How the Factory Works
A person writes an issue and marks it as safe for an agent; the agent works in a full local setup of Numonic and opens a merge request that goes through automated checks, an AI review, and finally a person, who decides whether it merges.
Stage by stage
Explore the AI Software Factory
Every stage, with how it works and our numbers.
Inside the AI software factory, in order: pick-up, agent run, proof, automated checks, AI review, a person, merged. Failed checks and review findings go back to the agent to fix. Anything still unresolved goes to a person. Learning is an optional path after a review.
Pick-upwritten for an agent
A person chooses the work: agents only pick up issues written and marked for them.
How it works
A person writes an issue saying what done looks like (acceptance criteria) and labels it as safe for an agent. A small controller takes those issues oldest first. It holds back anything that overlaps work already under way, so two agents don’t collide.
In our numbers
57 issues.
Agent runfull local setup of Numonic
Every coding agent gets a full local setup of Numonic and works the way a developer does.
How it works
The agent gets a fresh copy of the code and runs the real app against its own database and backend services, using it in a browser the way a person would. It runs the same automated checks we run on every change, then opens a merge request: a proposed change for review. It never merges anything itself.
Proofscreenshots and video
The agent records screenshots and video showing that what it did solves the issue.
How it works
They go on the merge request, each tied to something the issue asked for. A person can see the change working without having to run it themselves.
Automated checks (CI)pass or fix, automatically
When the automated checks pass, the AI review starts automatically; when they fail, the agent gets a chance to fix it before anyone is interrupted.
How it works
When the checks pass, the AI review starts on its own, with nobody needing to kick it off. When they fail, the agent gets the failure log and tries again, so a failed run doesn’t immediately need a person. If the same checks are also failing on our main code, the agent’s change isn’t the cause, so it goes straight to a person with a note saying so.
AI reviewfindings fixed first
A second agent reviews the work, and any finding at medium or above is sent back to be fixed before a person sees it.
How it works
It checks the change against what the issue asked for and against our security rules as they stood before the change, so a change can’t weaken its own review. A finding of medium or above goes back to the agent to fix, and the change is reviewed again. The review also rates the blast radius: how far a mistake would reach.
In our numbers
13 of the 54 merged changes went back for a fix after AI review.
A personblast radius sets focus
The blast radius tells a person how closely to look: a contained change they can practically just merge, a critical one gets a close look.
How it works
Only a person merges. A contained change is one a revert would undo; a critical one touches data or trust, where a revert may not be enough. One label on each merge request shows whose turn it is, and says when a check must run first.
In our numbers
54 merged, with a median of 33 hours from opening to merge.
When things go wronga person takes over
When a run fails, a person gets a plain note: what happened and what to do next.
How it works
Failures go to a person, not back into the queue, so an agent doesn’t keep spending effort on the same failure. A person decides when the issue goes round again.
In our numbers
3 closed without a merge.
Learningproposes, a person accepts
The agent proposes improvements to its own instructions, and what a person accepts applies to every run that follows.
How it works
After a review, the agent can suggest an edit to its standing instructions, as a merge request a person reviews like any other change. A review comment is read once; an accepted instruction stays.
In our numbers
11 proposed, 10 accepted.
Rounded ends start and finish, boxes are agent work, hexagons are gates, and the dashed path is optional. Figures as of November 5, 2026.
We have written before about parts of this setup: an agent that judges its own cost and how we test what a model says about itself. The count below is as of November 5, 2026.
Prefer to watch? I asked our AI agents to make a video explaining how the factory works, and they built this entirely on their own. You can probably tell. That doesn’t mean it isn’t fun to watch. Here you go.
The Count: 57 Issues, 54 Merged
Of the 57 issues, 54 ended with a merged change (94.7%) and three were closed without one (5.3%). Every issue has a final outcome.
With two agents running side by side, 28 of those 57 issues were picked up and worked in seven days, and 27 of the 28 merged.
What Didn’t Merge
Three of the 57 issues closed without an agent merge.
- Two were completed by a person.
- One was taken back, because deletion work needs a person to design it first. The agent’s own review recorded the lesson: the change fixed one code path and missed a sibling path doing the same job.
Where the Time Goes
The agent’s own run time per merged issue, adding up every run on it, had a median of 37 minutes, and one in ten took more than 72 minutes. From opening a change for review to merging it, the median was about 33 hours, or about a day and a half. The middle half took between 19.4 and 65.5 hours, and one in ten took longer than 74.6 hours.
Most of that elapsed time is a change waiting for review, which is what the blast radius is there to focus. A contained change needs little attention, and a critical one gets a close look.
What Review Catches
- AI review. Thirteen of the 54 merged changes went back to the agent for a fix after AI review.
- After merge. One merged change tightened a default in one publishing path. Six days later a person applied the same rule to three related paths. The agent’s change stood, and the follow-up extended it.
That follow-up and the change we took back have the same shape: a rule applied where the issue pointed, and not in a sibling path doing the same job. That gives reviewers a specific question to ask of any agent change.
As of November 5, none of the 54 merged changes had been reverted and none of their issues reopened; nine of those merges were under a week old.
Learning
After a review, the agent can propose an edit to its own standing instructions, as a merge request a person reviews like code. It proposed 11, and 10 were accepted. A review comment is read once. An accepted instruction is read by every run that follows.
What’s Next: Making Review the Fast Part
The agents now finish work faster than one person can review it. Most of the day and a half from opening a change to merging it is the change waiting its turn. So the next round of work is on review, not on the agents. Here is what we are building now:
- Evidence first. Every change opens with what was not verified, the risk, and how to undo it, so a reviewer reads the evidence before the code.
- Review effort where it matters. Review depth scales with the blast radius, riskier changes get a separate security pass, and a second agent checks each finding before the work is sent back.
- Changes small enough to review well. An agent hands work back instead of opening a change that has grown too large.
- Findings that block the merge. Each review finding becomes a thread that has to be resolved, and changes to sensitive areas always go to the right reviewer.
- A full queue pauses the factory. When enough changes are waiting for review, agents stop starting new work until the queue drains.
- Measuring what holds up. Every merged agent change gets a 30-day check for reverts and follow-up fixes.
We will share how far it moves the review time once we have the numbers.
How We Counted
The figures cover the issues our agents worked in our own repository from October 16, 2026, when the agent controller went live, to November 5, 2026. Rates count issues: an issue counts as merged when an agent merge request for it merged. Earlier trial runs since July are excluded; with them the totals are 62 issues and 59 merged (95.2%). The figures come from the merge request export, the agents’ run records, and our git history. Reasons for closing are read from titles, timing, and history. Production checks are tracked on each issue.
What We’d Tell a Team Starting Out
- Write the issue for the agent, with acceptance criteria a reviewer can check.
- Never let the agent merge. Every one of our 54 merges went through a person.
- Rate how far a mistake would reach, so review time goes where it matters.
- Count per issue, and count closes as well as merges. A merge rate means little without the issues that did not merge.
- Check that nobody is already doing the work before an agent picks it up.
- Plan for review to be the slow part. In our numbers it was the queue, not the agent.
- Ask about sibling paths in review.
- Keep run records from day one. Together with merge request and git history, they are the source of every number here.
Key Takeaways
- In three weeks on our own backlog, 54 of 57 issues ended with a merged change (94.7%).
- Every coding agent works in a full local setup of Numonic, and only a person merges.
- With two agents side by side, 28 issues were picked up and worked in seven days, and 27 merged.
- Agent run time per merged issue had a median of 37 minutes; from opening for review to merging took a median of about 33 hours, most of it waiting for review.
- As of November 5, none of the 54 merged changes had been reverted.
- Accepted edits to the agent’s instructions apply to every run that follows.
- Next up is review itself: evidence first, effort scaled to the blast radius, and a queue that pauses the factory when it is full.