Why AI-Generated Animations Need a Verifier

A model can produce an animation in seconds. Whether it's correct is a coin flip. The gap between those two facts is where all AI-generated work now lives.

Ask a good model to animate a signup confirmation — the badge pops as the checkmark draws on, the label slides up underneath — and you get something in seconds. It will look plausible. It will also, roughly half the time, be subtly wrong: the label overlaps the ring for two frames, the checkmark finishes after the badge has already settled, the whole thing eases in a way you didn’t ask for. The research I leaned on puts unaided LLM animation synthesis around 59% correct (MoVer, SIGGRAPH 2025). The interesting number is the second one: wrap that same generation in a formal verify-and-correct loop and it climbs to about 94%.

That gap — 59 to 94, same model — is not about intelligence. It’s about whether anything checks the work.

I keep meeting this shape. It is the same one behind everything I’ve written this year. A model is now cheap at producing an artifact and unreliable at being correct, and the distance between the two is only closeable by something that inspects the actual output and says yes or no. For code, that something is tests. For a financial report, it’s reconciliation against the source of truth. For an animation, it was — until recently — nothing. You looked at it. If it looked right, it shipped.

Looking is exactly the wrong instrument, because animation is a claim about time, and your eye is bad at time. “The toast slides in before it fades and has settled by 1.2 seconds” is a precise, checkable statement, and no screenshot can evaluate it. This is why the visual-regression tools — Chromatic, Percy, Applitools — don’t solve it. They diff frames. They can tell you a frame looks different from last week; they cannot tell you the motion is wrong, because a frame has no memory of the frames before it.

So I built the missing instrument and put it in the open. It’s called Choreo, it’s MIT, and it came out of the operations company I’m building (JieGou) as a side-output, not a product. The part that matters is small: you write down the motion contract — during 0..400: fadesIn(toast), at 1200: inside(msg, toast), at end: settled(all) — and a verifier proves the rendered animation satisfies it. It samples the real trace with Playwright: element positions, opacity, size, over time. Then it asserts first-order predicates over that trace and hands back a machine-readable verdict. 19/19 checks passed, or a failure report precise enough that a model can read it and correct itself. It verifies rendered traces, not source, so it runs against any web animation — hand-written CSS, GSAP, Motion, a Remotion export — not just its own.

There is a DSL layered on top that generates those checks from a description, and I’ll be honest about it the way the work forced me to be honest in the last essay: I don’t yet know that the DSL beats plain CSS plus the standalone verifier. There’s a benchmark in the repo built to answer exactly that, and the current read is “the durable win is authoring cost, not correctness.” The reason-to-believe was never the language. It’s the verifier.

And the verifier taught me the same lesson every honest checker teaches: it is only as good as the contract you give it. A predicate that asserts the wrong thing will pass the wrong animation, confidently, forever — a green checkmark on a broken motion is worse than no check, because it launders the error. Verification doesn’t remove the need for judgment. It relocates judgment to the one place it compounds: the written contract, which you can review, version, and reuse, instead of re-deciding by eye on every change.

Step back from animation and the argument is general. As agents produce more of everything — code, copy, reports, motion, decisions — the scarce input stops being generation and becomes the checkers: the artifacts that inspect a produced thing and assert whether it’s actually right, in a form a machine can act on. The model is the commodity. The floor under it — the thing that catches it when it’s wrong and improves every time it does — is what’s durable, and it’s almost always cheaper to build than the intelligence it governs. I measured that directly for enterprise agents last month: a governed floor turned a frontier-model workload into a cheapest-model workload without losing a required criterion. Choreo is the same claim in a modality where “looks right” is seductive and “is right” is invisible.

If your agents are generating anything you can’t fully check by looking, the leverage isn’t a better model. It’s a verifier for the thing they make. Animation happens to be a clean, visible place to prove it — you can literally watch the checker catch the regression. But the pattern is the whole game now. Build the floor.

Choreo is at github.com/JieGouAI/choreo — @choreo-oss/lang and @choreo-oss/verify, MIT, released as-is. It’s a research artifact, not something we sell. I’d genuinely like to know if the trace-verification idea holds up outside my own examples.