Three of us watched my AI agent’s first short film, and we all saw the same thing within a minute: the paper crane kept changing. A head became two heads. The red wax seal moved from one wing to the other, then to the tail.
The film was good otherwise, and almost all of it was the agent’s own work. From a one-line brief, Claude Code wrote the story, bred the film’s look over three generations, made the clips and the score, and filed every generation in Numonic. But continuity is the first thing a viewer notices, and generative video still gets it wrong by default. Patching the broken frames afterwards made them look worse.
So I asked it to start over, learn from the first attempt, and check every starting frame before animating it. This post is what it changed. It kept everything it made in Numonic, including every frame it rejected, and you can browse the public walk-through.
How the First Film Was Made
The first film started with one line. I connected Claude Code to ComfyUI and to Numonic through the Model Context Protocol, gave it a credit budget, and wrote: “Evolve a Film: generate competing visual directions, score them, breed successive generations, and preserve the whole creative family tree.” There was no story, no style, and no shot list.
Everything else was the agent’s call. It wrote a twelve-beat story: an old letter folds itself into a paper crane on a seaside windowsill, flies over rooftops, a market, a viaduct, rain under a bridge, and the open sea, crosses a storm to a lighthouse, and unfolds back into the letter beside a photograph of the town. It picked a paper crane on purpose. A prop has no face or hands, so keeping it consistent should come down to one marker, a red wax seal.
Then it designed the evolution, on the pattern of a genetic algorithm. It wrote each visual direction as six genes (medium, palette, light, camera, texture, and mood), drew every direction on the same story moments, and scored each frame for aesthetics, story, identity, range, how well it would animate, and distinctiveness. A frame that broke a hard rule, such as a deformed crane, could not breed.
How the Agent Evolved the Look
It built the family tree in Numonic as it went, and the structure was its own design: a collection for each generation with a scorecard, a frozen copy of each generation as it was judged, then the storyboard, the shots, and the final cut. Every image was tagged with its generation, its genes, its score, and its parents, and Numonic read the full ComfyUI workflow out of each file.
The first film’s family tree in Numonic
- evolved_paper_crane/overview card: the brief, the method, the winner
- gen_0/8 directions × 2 story beats: 16 frames + scorecard
- gen_1/5 children (4 crossovers, 1 mutation): 10 frames + scorecard
- gen_2/4 children × 3 beats: 12 frames + scorecard
- K4 Letter Worldwinner, 39/40: Paper Noir × Paper Diorama, story-led light, plus a mutation
- storyboard/12 keyframes styled on Letter World
- shots/13 video clips, including a rejected take
- final/the film and its score
- …_gen{0,1,2}_judged/three frozen copies, one per generation, as it was scored
In its notes afterwards, the agent picked out two lessons. The directions with a fixed sunny light fell apart in the storm, and the mutation that fixed them was a rule, not a look: light that follows the story. Every later winner inherited it. And the final winner, Letter World, made the theme the material, with every surface in the world written on like a letter. Crossing the existing genes could not have produced that; a mutation did.
It then storyboarded twelve shots, animated them, composed its own score, and cut a two-minute film for about 270 cloud credits, under £3. Its notes were also honest about the limits. It had scored its own work, with a vision model helping, and its top four scores were 38 and 39 out of 40, too close together to claim the loop had found the best film rather than a good one.
What Its Own Checks Missed
The agent checked the film before handing it over: no black frames, sensible brightness, a loudness curve that rose and fell. Three of us then watched it and found three problems: the crane changing between shots, music and noise from the video clips playing under the score, and a few sour notes. When we told it, it tested each complaint with its own tools and concluded we were right on every point. Its verdict on its own quality check was blunt: it had tested the wrong things.
What Caused the Continuity Errors
The cause was simple once the agent looked for it. Nothing in the first pipeline fixed the crane before the scenes were made. Each storyboard frame was generated on its own, so each one invented its own folds. The only instruction about identity was “a red wax seal on its left wing,” which a model can read as the crane’s left or the viewer’s left, and which stops meaning anything once the crane turns around.
The video model made it worse. Asked to make a paper bird fly, it flapped the wings and turned the body, and whenever the crane’s back faced the camera the seal vanished or came back on the other side. Because every check ran after the clips existed, every fix meant editing a frame that already looked right in every other way.
The Rules It Wrote Before Generating Anything
The agent’s first move in the remake was to turn each lesson into a rule, so that a mistake could not happen again instead of being caught later.
First film vs. remake
| Lesson | First film | Remake |
|---|---|---|
| Identity | Each frame invented its own crane | One canonical crane passed into every frame |
| Marker | "A seal on its left wing" (whose left?) | Seal on the body; head always faces right |
| Light | A fixed sunny look broke the storm | Light follows the story in every look |
| Testing a look | Two or three moments | The whole story, then head to head |
| Checking frames | After the clips, by editing | Before any clip, by rerolling |
| Motion | Crane asked to fly | Crane held still; the world moves |
| Music | Climax before the storm | Chosen by where the climax lands |
Cast the Crane Before Building the World
Before any scene existed, it designed one canonical crane: one head, one tail, two wings, and the seal on the body, where it does not move when the wings do. That single image went into every later frame as a reference. The crane also always faces right, which is a film rule as much as a continuity one: it travels from home toward the lighthouse, left to right, until it arrives.
Then it evolved the look around the crane: eight competing visual directions, scored and bred over three generations, with the first film’s winner, Letter World, entered again to compete fairly. It did not win again, though its genes survive in two of the four finalists. The winner was new: a shadow-theatre world of paper silhouettes in lamplight, where the crane, folded from a handwritten letter, is the only lit thing on the stage.
The Failure Worth Showing: Stickers
The first generation went wrong in an instructive way. Holding identity with a reference image worked too well. In all 24 frames the crane was pasted in at the same pose, size, and studio light, like a sticker on a postcard. Identity was perfect; the scenes were not.
The fix was to stop leaving placement to the image model. Each shot got a layout box: this crane, here, this size, lit by this scene. That exposed the next defect. The red seal started showing up on crates, walls, and letters lying on the ground, so from then on a second seal anywhere in the frame disqualified it.
Check Every Starting Frame Before It Moves
This was the change I asked for, and it turned out to be the cheapest one. Local image generation took about five seconds a frame, so instead of fixing frames the agent made six candidates for each shot and rejected any that failed. The gate checked for one crane, the right anatomy, one seal on the body, and a head facing right. More usefully, it checked for everything the script said must not be there.
Most rejections came from that second list:
- a lighthouse in the market and over the viaduct, where the script has none;
- a second lighthouse, standing behind the one the crane had just landed on;
- people and birds in silhouette; and
- a loose second letter on the desk.
When every candidate for a shot fails the same way, the cause is rarely luck. The stray lighthouses came from the look’s own description of its world, which named a lighthouse. Overriding that one line for those shots fixed them. The twelve frames that passed show the same crane in every scene.
Hold the Crane, Move the World
Video was where the first film broke, so this time the agent tested on one shot before rendering twelve. Asking the crane to glide still let the model turn it. Giving the video model the crane image as an extra reference held the crane but leaked paper into the set. What worked was an old trick from puppetry: the crane holds its pose like a shadow puppet on a rod, and the world slides past it. Rooftops, rain, a storm, the beam of the lighthouse. After that, every shot had a take in which the crane, its seal, and its heading held from the first frame to the last.
It is a trade. The crane moves less than a bird would. In a shadow-theatre film I think it reads as intended, but that is a matter of taste, and you can judge it yourself above.
Measure the Music, Too
The first film’s score peaked before its storm. This time the prompt asked for the climax on the storm, and the agent did not take the prompt’s word for it. It generated four scores and measured where each was loudest. One of the four peaked on the storm. That is the one in the film.
Who Made What
Credit matters on a project like this, so plainly: both films were made by Claude Code, an AI agent. For the first, from that one-line brief, it chose the story, designed the evolution, generated and scored every candidate, picked the survivors, made the clips and the score, assembled the film, and built its family tree in Numonic. For the remake, from a one-paragraph brief, it chose the rules, the looks, the score, and every keep-or-reject decision. In both, the scoring was the agent’s own, helped by a vision model, not an independent panel.
I commissioned both films, set the budgets, and approved a few steps, and three of us reviewed the first one. So the work was mostly agentic, not fully autonomous: the briefs, the budgets, and the tools were human choices. The first film used gpt-image-2.5 and Nano Banana 2 for images, LTX-2.3 for video, and ACE-Step 1.5 for music, through ComfyUI in the cloud. The remake used FLUX 3 Klein, LTX-2.5, and ACE-Step 1.5 on one local GPU.
See the Whole Family Tree
The film is the hook. The part I care about is that nothing was thrown away, in either film. The first film’s tree, every generation with its scores and parents, sits in the same Numonic workspace as the remake’s 298 pieces, as its ancestor. The public walk-through opens with both films and six frames from the first film’s evolution, then shows the remake’s pieces in the order they were made, with a note at each stage on what was kept or cut and why. Every image carries the prompt, model, seed, and workflow that produced it. The finished film carries a record of its twelve shots, so each one traces back to the frame and the settings it came from.
Key Takeaways
- Give an agent a method, not a look. Genes, a score, and hard rules let it compare directions fairly and breed better ones.
- Have someone other than the maker watch the result. The agent’s own checks passed a film that three people faulted in a minute.
- Continuity in AI film fails when identity is left to each frame. Fix the subject first and pass it into every frame.
- Put a marker where the motion is not, and fix the screen direction, so “which side” never has to be asked.
- When frames are cheap, generate several and reject the bad ones instead of editing them.
- Check what the script says must not be there, not only what must.
- For a rigid prop, hold its pose and move the world around it.
- Keep every candidate and how it was made. The rejects explain the film.