The reviewer in the room

The reviewer at an education charity opened the draft, looked at the seed images (the reference pictures the model uses to set the look and the cast of a generated video, before it renders the full thirty minutes), and sent them back.

The images were fine, in the sense that the model had done what it had been asked to do. But they weren't right in the sense that every face in them was white. The charity's work is widening participation: reaching learners across South Asia, Africa, the Gulf, parts of the world where the last thing a training video should do is imply that the people in it don't look like you. The prompt hadn't said anything about who the learners were. The model had filled the gap with a default, and the default was exclusionary.

This is bias in the ordinary sense: the model tends toward certain kinds of output because of the material it was trained on, and that material skews. A generative model asked for a person, with no other steer, will produce what its training data over-represents. For most prompts that's invisible. For a charity whose whole mission is reaching learners who have been under-served, it's the mission failing quietly, one seed image at a time.

Nobody had told the model that part mattered. That was the point.

We were building an agent platform for the charity that turned course material into training videos. Half-hour videos. That length matters, because generating one is not cheap. Every run spends real money, and a run that comes out wrong spends the same money as a run that comes out right.

The human-in-the-loop step (the moment where a subject-matter expert reviews the script and the seed images before the agent is allowed to render the full video) was in the design from the first week. Two reasons, we told the client. The expert confirms the output is accurate for their learners, and owns that it's accurate. And the manager gets a gate on spend, so the agent isn't burning budget on a run that was going to be sent back anyway.

Both of those were true. Neither of them was the most interesting reason.

The interesting reason emerged from the review itself. When the reviewer flagged the diversity problem, the fix wasn't just to regenerate that video with a better prompt. It was to go back to the base prompts for the skill (the standing instructions the agent uses every time it builds a video on this topic, the template sitting behind the run) and update them so the next video, and the one after that, started from a better place.

The review caught a wrong output. The correction caught a wrong default. The next hundred videos are better because one reviewer, on one afternoon, sent one set of images back.

That rhythm is the piece. The agent produces, a reviewer pushes back, the prompts get fixed, the issue resolves. Then round again. The loop isn't the scaffolding around the real work. It is the real work, running every time the agent runs.

This is what human-in-the-loop actually does in an agentic system. It isn't a safety net you remove once the agent has proved itself. It's the mechanism by which the agent learns what accurate means for this organisation and its audience. The model is competent. It is not, on day one, correctly calibrated. You calibrate it by letting a person who knows the audience look at what it produced and say not like that.

It's also one of the few practical answers to model bias that doesn't require retraining the model. Retraining is somebody else's job on somebody else's timeline. The loop is how this organisation's standards override the model's defaults, on every run, today. The bias doesn't go away. It gets caught, named, and written into the prompts that sit in front of the model, so the next run starts from the charity's view of the world instead of the internet's.

Humans don't stay in every loop forever. Parts of the pipeline that the experts stopped catching anything on, we automated further. The goal is to earn autonomy, piece by piece, as the prompts improve and the patterns that used to need a human eye stop showing up.

The loop pays for itself on the way. Every issue caught at the seed-image stage is a thirty-minute render that never happens, so the money isn't spent. Every base-prompt update means the next run starts closer to correct, so fewer rejections down the line, so fewer wasted renders after that. The agent gets more accurate and cheaper to run in the same motion.

The shortcut is to skip the review, trust the model's defaults, and ship. The shortcut is how a skills charity ends up with training videos that quietly exclude the learners they're meant to serve, at full price.

If you're running an agent in production, the reviewer isn't slowing the system down. They're what makes the system yours.